EXPERIMENT IV · DATA

第几代开始退化Which Generation Degrades

用模型生成的数据训练模型，第几代开始退化？When a model is trained on model-generated data, at which generation does it start to degrade?

待开始Not started合成数据Synthetic data模型坍缩Model collapse蒸馏Distillation
  1. Q提出可证伪的问题State a falsifiable question当前阶段Current stage
  2. R做最小运行路径Build a minimal runnable path未开始Not started
  3. E连接真实验证Validate against reality未开始Not started
  4. P公开证据与限制Publish evidence and limits未开始Not started

为什么值得做Why it is worth doing

好数据越来越难拿，合成数据是绕不开的路。但模型学模型，分布会一代代变窄。什么时候开始变窄、混入多少真实数据能止住，决定了合成数据能走多远。Good data keeps getting harder to get, so synthetic data is unavoidable. But when models learn from models, the distribution narrows generation by generation. When it starts narrowing, and how much real data holds it off, decides how far synthetic data can go.

Q

假设Hypothesis

HYPOTHESIS

纯合成数据训练在两到三代后出现可测的多样性下降；每代混入一定比例的真实数据能显著延缓退化。Pure synthetic training shows measurable diversity loss after two or three generations; mixing in a fixed share of real data per generation noticeably delays the degradation.

R

最小装置Minimal setup

MINIMAL SETUP
  1. 01用小模型在一个窄领域上自我蒸馏，迭代五代Self-distill a small model in a narrow domain for five generations
  2. 02对照：纯合成 / 每代混入 10% 与 30% 真实数据Compare pure synthetic / 10% and 30% real data per generation
  3. 03每代测量准确率、输出多样性（n-gram 熵、语义去重率）与长尾题表现Measure accuracy, output diversity (n-gram entropy, semantic dedup rate), and tail-question performance each generation
E

什么会证明我错了What would prove me wrong

FALSIFIED IF

五代内各组多样性与准确率都没有可测差异。Within five generations, neither diversity nor accuracy differs measurably across groups.

P

将会公开What will be published

WILL PUBLISH

每代的数据样本、多样性曲线和各组训练配置。Samples from each generation, diversity curves, and every training configuration.

前期线索Early leads相关的已有工作，不是这个实验的结论Related earlier work, not a result of this experiment

  • 合成数据质检Synthetic data QA做过化学合成数据的质量检查与错误归因。Ran quality checks and error attribution on chemical synthesis data.

实验日志Run logRUN LOG

还没有记录。实验开始后，每一次运行、失败和修正都会记在实验室里。Nothing logged yet. Once the experiment starts, every run, failure and fix will be recorded in the lab.

去实验室Go to the lab ↗