EXPERIMENT I · DATA

洗掉一半Prune Half

同样的训练预算，洗掉一半低质量数据，模型会更好还是更差？With the same training budget, does discarding half the low-quality data make the model better or worse?

待开始Not started数据质量Data quality数据过滤Data filteringSFTSFT
  1. Q提出可证伪的问题State a falsifiable question当前阶段Current stage
  2. R做最小运行路径Build a minimal runnable path未开始Not started
  3. E连接真实验证Validate against reality未开始Not started
  4. P公开证据与限制Publish evidence and limits未开始Not started

为什么值得做Why it is worth doing

我相信数据决定模型的上限，但“质量”常常只是一句口号。要让它成立，就得说清楚按什么标准洗、洗掉多少，以及模型到底在哪些能力上变好。I believe data sets the ceiling on model quality, but "quality" is often just a slogan. To make it real, I have to specify the criteria, how much is pruned, and exactly which capabilities improve.

Q

假设Hypothesis

HYPOTHESIS

按事实、推理和格式三类错误打分过滤后的一半数据，微调效果不低于全量数据，且在事实类题目上更好。Half the data, filtered by a score over factual, reasoning, and formatting errors, fine-tunes no worse than the full set and better on factual questions.

R

最小装置Minimal setup

MINIMAL SETUP
  1. 01取一份公开指令数据集，用规则 + 模型评审给每条打质量分Take a public instruction dataset and score every sample with rules plus a model judge
  2. 02对照三组：全量 / 随机一半 / 高分一半，训练步数与 token 数对齐Compare three groups — full set / random half / high-scoring half — with training steps and token counts matched
  3. 03在固定评估集上分维度比较，并人工抽检被过滤掉的样本Compare per dimension on a fixed evaluation set and manually spot-check the filtered samples
E

什么会证明我错了What would prove me wrong

FALSIFIED IF

高分一半在所有维度上都不优于随机一半，说明打分并没有抓住“质量”。If the high-scoring half beats the random half on no dimension, the scoring failed to capture "quality."

P

将会公开What will be published

WILL PUBLISH

打分规则、过滤前后的数据分布、三组训练配置和分维度结果。The scoring rules, data distributions before and after filtering, all three training configurations, and per-dimension results.

前期线索Early leads相关的已有工作，不是这个实验的结论Related earlier work, not a result of this experiment

  • 标注与质检实践Annotation and QA practice做过金融长文本、金法医等标注，在供应商侧参与混元 HY4 评测与质量管理。Annotated long financial text and forensic accounting data; took part in Hunyuan HY4 evaluation and quality management on the vendor side.

实验日志Run logRUN LOG

还没有记录。实验开始后，每一次运行、失败和修正都会记在实验室里。Nothing logged yet. Once the experiment starts, every run, failure and fix will be recorded in the lab.

去实验室Go to the lab ↗