10 Terabytes. 1 letter is 2 bytes in unicode. 10e12 bytes. 5e12 letters. 1 word is about 5 letters. 1e12 words. Assuming 500 words per page, that would be 2e9 pages. Page thickness assumed at 1e-4. 2e5 meters, if you stack all the pages. 200 kilometers high tower of paper.
<pedantry>For English language text letters are 1 byte when encoded in UTF-8 (which is how essentially all text is stored and transmitted these days).</pedantry>
Though it probably balances out in your final answer since you’re not counting spaces, punctuation, formatting, or other markup.
10 Terabytes. 1 letter is 2 bytes in unicode. 10e12 bytes. 5e12 letters. 1 word is about 5 letters. 1e12 words. Assuming 500 words per page, that would be 2e9 pages. Page thickness assumed at 1e-4. 2e5 meters, if you stack all the pages. 200 kilometers high tower of paper.
<pedantry>For English language text letters are 1 byte when encoded in UTF-8 (which is how essentially all text is stored and transmitted these days).</pedantry>
Though it probably balances out in your final answer since you’re not counting spaces, punctuation, formatting, or other markup.
Meaning that to train even the tiniest LLM you can get you need a stack of paper so tall it could get knocked by satellites.
This is the scale that people miss when they talk about “writing some stuff to train an AI model”.