RESEARCH / QUESTIONS BEFORE ANSWERS

问题,Questions,先于答案。before answers.

我相信数据决定模型的上限。我的研究沿着一个模型的一生展开:数据从哪来,怎样后训练,结构怎样改变成本,怎样被高效地推理,最后落在哪些基础设施和科学问题上。六个方向,十二个实验,每一个都从一个可以被证伪的问题开始。I believe data sets the ceiling on what a model can be. My research follows the life of a model — where its data comes from, how it is post-trained, how architecture changes cost, how it is served efficiently — down to the infrastructure and scientific questions it finally lands on. Six directions, twelve experiments, each starting from a question that can be proven wrong.

↓   六个研究方向↓   Six research directions
I / DIRECTIONS沿着一个模型的一生:数据 → 训练 → 结构 → 推理 → 地基与科学。The life of a model: data → training → architecture → inference → infrastructure and science.

研究方向Research DirectionsWHAT I STUDY

数据决定模型的上限。质量差的数据一定训练不出好模型,但“质量”必须能被拆开、被测量,才不只是一句口号。Data sets the ceiling on model quality. Bad data cannot train a good model, but "quality" has to be decomposed and measured before it means anything.

做过金融长文本、金法医等标注与质检,在供应商侧参与腾讯混元 HY4 的评测与质量管理:制定 Rubric、复核事实与计算、构造高难样本。这些经验是这个方向的起点,但还没有形成公开的数据实验。I have done annotation and QA on long financial text and forensic accounting data, and worked on evaluation and quality management for Tencent Hunyuan HY4 on the vendor side: writing rubrics, double-checking facts and calculations, and constructing hard samples. That work is the starting point for this direction, but it has not yet produced a public data experiment.

正在追的问题Open Questions

  1. 同样的训练预算,按什么标准洗掉哪一半数据,模型会变得更好?With the same training budget, filtered by what criteria, does dropping half the data make the model better?
  2. 合成数据能补多少数量,又从第几代开始让模型的分布变窄?How much volume can synthetic data add, and from which generation does it start narrowing the model's distribution?
  3. 事实、推理和格式三类错误,哪一类对最终能力的伤害最大?Of factual, reasoning, and formatting errors, which hurts final capability the most?

造物之书里的实验Experiments in the Book

方法、下一步与局限 ↗Methods, next steps, and limits ↗

奖励信号和评测本身就是数据问题。训练在优化什么、评测在奖励什么,两者一旦对不上,提升就可能只是表面的。Reward signals and evaluation are themselves data problems. Once training optimizes one thing while evaluation rewards another, improvements can be surface-level only.

2025 年起在 Qwen、LLaMA 上做过 SFT 与 DPO 的练习与对比;在混元 HY4 评测中写 Rubric、处理资料冲突和异常结果,对“评测标准怎样变成训练信号”有第一手感受。Since 2025 I have run SFT and DPO exercises and comparisons on Qwen and LLaMA. During Hunyuan HY4 evaluation I wrote rubrics and handled conflicting sources and anomalous results, which gave me a first-hand feel for how evaluation criteria become training signals.

正在追的问题Open Questions

  1. 按 Rubric 逐维构造的偏好对,能否让模型学到评测真正在乎的维度?Can preference pairs built dimension by dimension from a rubric teach a model the dimensions evaluation actually cares about?
  2. DPO 的提升里,有多少其实来自“回答更长”?How much of DPO's gain actually comes from longer answers?
  3. 可验证奖励(RLVR)能否教会小模型多步调用工具,而不只是做对数学题?Can verifiable rewards (RLVR) teach small models to call tools over multiple steps, not just solve math problems?

造物之书里的实验Experiments in the Book

方法、下一步与局限 ↗Methods, next steps, and limits ↗

推理成本决定一个模型能被多少人真正用上。每一次重复计算、每一次缓存失效,最后都会变成用户等待的时间。Inference cost decides how many people can actually use a model. Every redundant computation and every cache miss ends up as time a user spends waiting.

在 Eazo 参与 Prompt Caching 与模型调用链路优化:连接复用、配置并行初始化、扩展增量同步。也做过 cctop,把本地 token 记录与服务账单对账。At Eazo I worked on prompt caching and the model-call path: connection reuse, parallel config initialization, and incremental sync for extensions. I also built cctop, which reconciles local token records against service bills.

正在追的问题Open Questions

  1. 一个 Agent 的多轮对话里,有多少 token 在被重复计算?上下文怎样排布才能让前缀缓存命中?In a multi-turn agent conversation, how many tokens are recomputed? How should context be arranged so prefix caching hits?
  2. 模型能否按题目难度提前退出,把简单题的算力省下来?Can a model exit early based on question difficulty and save compute on easy questions?
  3. 投机解码在代码、中文长文本上的接受率有多高?What acceptance rate does speculative decoding reach on code and long Chinese text?

造物之书里的实验Experiments in the Book

方法、下一步与局限 ↗Methods, next steps, and limits ↗

同样的效果,换一个结构可能只要一半的算力。最值得研究的,是那些改变成本曲线、而不只是堆更多参数的设计。The same result may take half the compute under a different structure. What is worth studying are designs that change the cost curve, not just ones that add more parameters.

此前做过循环 Transformer 的小规模复现;正在调研 Jev 这类类型化决策模型。MLA、稀疏注意力与 MoE 目前停留在读论文和读实现阶段。I have previously done a small-scale reproduction of a looped Transformer and am now surveying typed decision models such as Jev. MLA, sparse attention, and MoE are still at the stage of reading papers and implementations.

正在追的问题Open Questions

  1. 同一组参数迭代 k 次,能不能替代堆叠 k 层?Can iterating the same set of parameters k times replace stacking k layers?
  2. 长上下文里,模型每一步真正在看的 token 有多少?稀疏注意力的前提在哪些任务上成立?In long context, how many tokens is the model really looking at each step, and on which tasks does the sparse-attention premise hold?
  3. MoE 的路由在训练中怎样塌缩,负载均衡损失又牺牲了什么?How does MoE routing collapse during training, and what does the load-balancing loss give up?

造物之书里的实验Experiments in the Book

方法、下一步与局限 ↗Methods, next steps, and limits ↗

在科学里,一个配平错误或一个不存在的反应条件,都会让模型带着同样的错。科学 AI 首先是数据质检问题。In science, one unbalanced equation or one impossible reaction condition teaches the model the same mistake. AI for science is, first of all, a data-QA problem.

做过化学合成数据的质检与错误归因,以及化学与学科资料的证据寻源。这是从数据角度进入科学 AI,还没有做过模型层面的科学实验。I have done QA and error attribution on chemical synthesis data, as well as evidence sourcing for chemistry and discipline-specific material. That is entering science AI from the data side; I have not yet run science experiments at the model level.

正在追的问题Open Questions

  1. 化学合成数据里,模型最常犯的是哪一类错?规则与工具能拦下多少?What errors does a model make most often in chemical synthesis data, and how many can rules and tools catch?
  2. 只有几百条实验数据时,预训练模型 + 少量微调能走多远?With only a few hundred experimental records, how far can a pretrained model plus light fine-tuning go?
  3. 科学问答的“正确”怎样被验证:文献、计算,还是实验?How is "correct" verified in scientific Q&A: literature, computation, or experiment?

造物之书里的实验Experiments in the Book

方法、下一步与局限 ↗Methods, next steps, and limits ↗

模型能力最终都落在基础设施上。执行离数据和环境越近,Agent 越快、越稳,也越容易被观察。Model capability ultimately rests on infrastructure. The closer execution is to the data and the environment, the faster, steadier, and more observable an agent becomes.

在 Eazo 参与 V4 到 V5 升级:把 Agent 执行从后端 Worker 迁到 E2B 沙箱,拆分控制面与执行面,前端经 WebSocket 直连沙箱;也参与 Skills 驱动的多 Agent 体系。At Eazo I worked on the V4 to V5 upgrade: moving agent execution from a backend worker into E2B sandboxes, splitting the control and execution planes, and connecting the frontend to the sandbox directly over WebSocket. I also worked on a Skills-driven multi-agent system.

正在追的问题Open Questions

  1. 让前端直连沙箱执行,多轮任务能省下多少等待?When the frontend connects directly to the sandbox, how much waiting does a multi-turn task save?
  2. 控制面与执行面拆开后,失败恢复和权限边界怎样设计?Once the control plane and execution plane are split, how should failure recovery and permission boundaries be designed?
  3. 一个小团队的评测平台,最少需要哪些组件才能支撑每天的回归?What is the minimum set of components an evaluation platform needs to support daily regression for a small team?

造物之书里的实验Experiments in the Book

方法、下一步与局限 ↗Methods, next steps, and limits ↗
II / THE BOOK OF EXPERIMENTS十二张牌,十二个从零开始的实验。Twelve cards, twelve experiments built from scratch.

造物之书The Book of Experiments✦THE BOOK OF EXPERIMENTS

每张牌背后是一个实验:Behind every card is an experiment:
从哪个问题出发,怎样验证,which question it starts from, how it is tested,
什么结果会证明我错了。and what result would prove me wrong.

实验室里的记录 ↗Records from the lab ↗
BEYOND THE KNOWN越过已知,再多走一步。One step beyond the known.进入实验室 ↗Enter the lab ↗
III / EXPEDITION PROTOCOL每一个实验都走同一个闭环。Every experiment follows the same loop.

探索协议Expedition ProtocolFROM QUESTION TO PUBLIC EVIDENCE

  1. Q01提出可证伪的问题State a falsifiable question

    先写清楚什么现象会证明当前判断是错的。Write down, before anything else, what observation would prove the current judgment wrong.

  2. R02做最小运行路径Build a minimal runnable path

    用代码、实验或日志代替只靠概念推演。Replace conceptual reasoning alone with code, experiments, or logs.

  3. E03连接真实验证Validate against reality

    在固定评估集、真实数据和真实负载上检验,不只看训练损失。Test on a fixed evaluation set, real data, and real load, not just training loss.

  4. P04公开证据与限制Publish evidence and limits

    代码、数据、记录和未完成项一起公开。Publish code, data, records, and what remains unfinished, together.