Minor nit: In familiarity, you gloss over the fact that it's character rather than token based which might be worth a shout out:
"Microgpt's larger cousins using building blocks called tokens representing one or more letters. That's hard to reason about, but essential for building sentences and conversations.
"So we'll just deal with spelling names using the English alphabet. That gives us 26 tokens, one for each letter."
hm. the way i see things, characters are the natural/obvious building blocks and tokenization is just an improvement on that. i do mention chatgpt et al. use tokens in the last q&a dropdown, though
About how many training steps are required to get good output?
not many. diminishing returns start near 1000. past that you should just add a second/third layer
Minor nit: In familiarity, you gloss over the fact that it's character rather than token based which might be worth a shout out:
"Microgpt's larger cousins using building blocks called tokens representing one or more letters. That's hard to reason about, but essential for building sentences and conversations.
"So we'll just deal with spelling names using the English alphabet. That gives us 26 tokens, one for each letter."
hm. the way i see things, characters are the natural/obvious building blocks and tokenization is just an improvement on that. i do mention chatgpt et al. use tokens in the last q&a dropdown, though