Explainer of Diffusion LLMs from Andrej Karpathy: "Most of the LLMs you've been seeing are ~clones as far as the core modeling approach goes. They're all trained "autoregressively", i.e. predicting tokens from left to right. Diffusion is different - it doesn't go left to right, but all at once. You start with noise and gradually denoise into a token stream."
LLaDA - new MIT-licensed Diffusion LLM
https://ml-gsai.github.io/LLaDA-demo/
0 Comments
No comments yet.