The humming thread is no longer humming; it’s demoing. OpenAI’s GPT-3 API went into beta last month and my Twitter feed has spent the fortnight full of demos that would have been dismissed as staged a year ago: plain-English descriptions turned into working UI code; legal prose translated to plain language; essays continued in the style of their opening paragraph; the beta invite queue is currently the hottest ticket in tech. It’s “just” next-token prediction at 175 billion parameters, and the honest staff-level assessment after two weeks of playing with it: it is simultaneously the most impressive parlor trick I’ve ever used and obviously, obviously, a new kind of infrastructure. It confabulates with total confidence (ask it for citations and it invents plausible ones, the failure mode is fluency, which is worse than being wrong badly). It has no idea what it doesn’t know. And none of that changes the trajectory: the Transformer paper, the BERT benchmark, the staged release, and this entry logs the first fortnight I watched non-ML engineers build products on the thread. The extrapolation exercise is now everyone’s homework.
Geopolitics did its own platform work: India banned TikTok outright (59 Chinese apps, 200M Indian users, gone by executive order, platform-risk-at-nation-scale, now the largest single market removal in history), and the US is making similar noises. The splinternet is arriving app by app.
TIL: few-shot prompting, the model does tasks it was never trained for, given examples in the prompt. The interface is the training. I don’t fully understand the implications and I’ve stopped pretending anyone does; the archive will keep minutes (its actual job).