EXPERIMENT VII · ALGORITHMS

循环的深度Depth Through Loops

同一组参数迭代 k 次，能不能替代堆叠 k 层？Can iterating the same parameters k times replace stacking k layers?

待开始Not started循环 TransformerLooped Transformer权重共享Weight sharingPyTorchPyTorch
  1. Q提出可证伪的问题State a falsifiable question当前阶段Current stage
  2. R做最小运行路径Build a minimal runnable path未开始Not started
  3. E连接真实验证Validate against reality未开始Not started
  4. P公开证据与限制Publish evidence and limits未开始Not started

为什么值得做Why it is worth doing

参数共享的循环结构让“想得更久”变成调节迭代次数，而不是换更大的模型。它在什么任务上成立，需要亲手复现。A weight-shared looped structure turns "thinking longer" into adjusting iteration count rather than switching to a bigger model. On which tasks that holds is something I need to reproduce myself.

Q

假设Hypothesis

HYPOTHESIS

在需要多步推理的小规模算法任务上，循环 Block 以更少参数接近同深度堆叠模型的表现。On small-scale algorithmic tasks that need multi-step reasoning, a looped block approaches a stacked model of equal depth with fewer parameters.

R

最小装置Minimal setup

MINIMAL SETUP
  1. 01实现权重共享的循环 BlockImplement a weight-shared looped block
  2. 02在加法、排序等算法任务上训练循环 / 堆叠 / 显式思维链三种模型Train looped / stacked / explicit-chain-of-thought models on tasks such as addition and sorting
  3. 03比较参数量、准确率与推理时延Compare parameter count, accuracy, and inference latency
E

什么会证明我错了What would prove me wrong

FALSIFIED IF

同等迭代深度下，循环模型的准确率显著低于堆叠模型。At equal iteration depth, the looped model is clearly less accurate than the stacked one.

P

将会公开What will be published

WILL PUBLISH

训练代码、配置、曲线和失败的设置。Training code, configurations, curves, and the settings that failed.

前期线索Early leads相关的已有工作，不是这个实验的结论Related earlier work, not a result of this experiment

  • 循环 Transformer 复现Looped Transformer reproduction此前做过小规模复现，这次从零重做并完整公开。A small-scale reproduction was done earlier; this rebuilds it from scratch and publishes it in full.

实验日志Run logRUN LOG

还没有记录。实验开始后，每一次运行、失败和修正都会记在实验室里。Nothing logged yet. Once the experiment starts, every run, failure and fix will be recorded in the lab.

去实验室Go to the lab ↗