Okay so the jump from ‘draw a pelican on a bicycle’ to ‘generate a full fantasy world with internal economy’ sounds like pure progress until you remember the model has zero way to audit what it just made. Like building a piece of live audio equipment and having no way to actually hear it. You’d ship broken stuff constantly.
The tricky part is I’m not sure if this is a training problem or architectural.
source: Andrej Karpathy (@karpathy) / X
