Generative AI improvements are increasingly being made through data curation and collection — not architectural — improvements. Big Tech has an advantage.
Based on the post title alone, I call bull because I could buy enough storage and pirate enough books in order to create an AI, using copyrighted material as the training data. Yes it would be an absolutely horrible AI since I don’t have a clue what I’d be doing, but it’s possible.
The training data is important, but currently the bottleneck is computing power. Buying so many chips and having them run full blast 24/7 costs a lot of money.
You can get your hands on books3 or any other dataset that was exposed to the public at some point, but large companies have private human-filtered high-quality datasets that perform better. You’re unlikely to have the resources to do the same.
Based on the post title alone, I call bull because I could buy enough storage and pirate enough books in order to create an AI, using copyrighted material as the training data. Yes it would be an absolutely horrible AI since I don’t have a clue what I’d be doing, but it’s possible.
Then go ahead and buy 2000 Nvidia cards.
The training data is important, but currently the bottleneck is computing power. Buying so many chips and having them run full blast 24/7 costs a lot of money.
You can get your hands on books3 or any other dataset that was exposed to the public at some point, but large companies have private human-filtered high-quality datasets that perform better. You’re unlikely to have the resources to do the same.