Tweet by xjdr:

if you will forgive a bit of anthropomorphizing, to the extent models think, they do not think in tokens. tokens are merely our samplers' best interpretation of the probability distributions that emerge from the actual 'thinking' process. they're an incredibly low bandwidth…