The Curse of Recursion: Training on Generated Data Makes Models Forget

Synthetic data is often used to trian model because data is hard to collect. But, now that we have a lot data generated by AI, we are realizing it may not be as valuable