Novel Feedback Autoresearch

小说反馈 Agent 的主 agent 十轮自迭代实装

这次不是只生成一个自动脚本。脚本只做可重复 data/eval；主 agent 负责迭代流程：读结果、诊断 failure、决定下一步 patch、改代码、写回归测试、重跑、发布并检查线上页面。当前已从 Round 5 接着做完后续 5 轮，形成 Round 1 到 Round 10 的可复盘 artifacts。

v8 Audio-Novel TTS Video v7 Belief-Guided Personalization v5 Qwen3 LLM Simulator v1–v4 总结 v4：Encoder-State Simulator v4 交互式小说入口 v3 Experiment Suite v3：Dynamic reader data + simulator v2：ESUS user simulator + FUSE-Hard 打开实验 Dashboard 查看 research_journal.json summary.json

v8 Audio

v8 Audio-Novel TTS Video

Qwen3-TTS 有声小说、3 图动态视频、网易云音乐式播放器、500+500 JSON 和 test-time scaling 对照。

v7 Belief

v7 Belief-Guided Personalization

基于你提供的文档：feedback is observation, Personalization Brief, UserSim/Controller full-parameter Qwen3-4B, Frozen Generator, feedback-conditioned auto-rubric eval。

Interactive Novel Agent

v4 交互式小说入口

用户名密码登录、20 个问题 onboarding、实时生成小说；把 feedback → preference state → future generation 变成可点击入口。

Scope Correction

当前小说场景：AHEAD intentionally disabled

本轮明确排除 AHEAD / early-awareness。scope_exclusions = AHEAD, early-awareness 写入 manifest；当前只研究 explicit + implicit feedback → preference/state update → future generation improvement，以及 delayed feedback → trace credit assignment。

Main-agent Process

主 agent 负责迭代，不把责任丢给脚本

脚本是可重复评测工具；真正 autoresearch 由主 agent 观察指标、发现问题并修复。

01 / Observe

读取 round artifacts

主 agent 检查 metrics、failure examples、HTML 可视化和 journal，而不是只信脚本输出。

02 / Diagnose

解决 0 指标语义

旧 no_update_precision 的 0 不可信；补入 neutral/no-update 控制样本，并发布 neutral_no_update_accuracy。

03 / Patch

regression test 后修复

发现 verifier overcorrection 后先写 regression test，再修 scorer，避免弱 distractor 抢 delayed feedback 的锅。

Results

Round 5 → Round 10 的继续 5 轮结果

数据：Self-Evolve 466 条，其中 neutral/no-update 66 条；Long-Horizon 120 个 episode。每一轮都有 HTML、JSON eval、上一轮对比和主 agent 诊断。

Self-Evolve

+0.0421

future_probe_win_rate：0.8211 → 0.8632

Long-Horizon

+0.0167

culprit_step_accuracy：0.9583 → 0.9750

Neutral Controls

1.0000

neutral_no_update_accuracy；替代旧的模糊 no_update_precision。

Data & Evaluation Protocol

每一轮数据、任务与指标定义

不是只贴最终分数；页面和 artifacts 都记录 dataset size、data source、task、metrics、metric definitions 和解释。

Self-Evolve / Feedback-to-State

PIF-Bench synthetic novel feedback trajectories

sample_count: 466 · splits: {'feedback_rows': 400, 'neutral_no_update_rows': 66}

task: Run the round-specific updater on every feedback event, compare predicted dimensions/targets/scope with gold state diffs, then estimate future-probe success.

data schema

sample_id one user-turn id
hidden_preference_profile simulator preference state
content_features pacing, plot, agency, lore, trope, emotion
explicit_feedback like/favorite/rating/comment
implicit_feedback dwell, fast_swipe, reread, continue_next
gold_preference_update dimension/direction/scope/confidence/evidence
gold_agent_update memory/critic/planner/retriever/reranker target
future_probe future generation probe for personalization win rate

metric definitions

dimension_f1 — Macro F1 between predicted preference dimensions and gold dimensions.
target_f1 — Macro F1 between predicted agent-state targets and gold targets.
future_probe_win_rate — Mean synthetic win probability on near/far/anti-overgeneralization future probes.
overgeneralization_rate — Fraction of predicted updates that incorrectly globalize local/story feedback; zero is good.
neutral_no_update_accuracy — Accuracy on neutral/no-gold-update rows; replaces the confusing no_update_precision metric.

Long-Horizon / Traceable Credit Assignment

TRACE-USER-Bench synthetic delayed-feedback traces

sample_count: 120 · splits: {'culprit_dimensions': {'lore_density': 27, 'pacing': 35, 'trope_misunderstanding': 32, 'female_agency': 26}}

task: Rank culprit steps from the full trace, compare top prediction and reciprocal rank to gold, then estimate counterfactual repair gain and confidence calibration.

data schema

episode_id one multi-turn trace id
turns / trajectory scene sequence with features, tags, utility, implicit feedback
delayed_feedback later user comment and sentiment
gold_credit_assignment culprit_step, culprit_dimension, confidence, rationale
counterfactual_repairs repair action and expected utility gain

metric definitions

culprit_step_accuracy — Exact-match accuracy for the delayed-feedback culprit step.
culprit_dimension_accuracy — Accuracy for the causal preference/content dimension.
credit_mrr — Mean reciprocal rank of the gold culprit step.
repair_gain — Mean expected utility gain from the selected counterfactual repair.
credit_calibration — One minus confidence error; higher means confidence tracks correctness.

Schema · Samples · Formulas · Charts

JSONL Schema、Current JSONL sample、Metric definitions 与 Formula boxes

这一节把当前十轮实验到底读什么数据、怎么算指标、图表怎么看写清楚；公式用纯文本 pre box，不依赖前端公式渲染器。

pif_bench.jsonl

JSONL Schema · Self-Evolve feedback row

sample_id: user-turn key, e.g. u_000_turn_01
hidden_preference_profile: simulator ground truth preference state
content_features: pacing / plot / agency / lore feature vector
explicit_feedback: like, favorite, rating, comment
implicit_feedback: dwell, fast_swipe, reread, continue_next
gold_preference_update: dimensions, direction, scope, confidence, evidence
gold_agent_update / gold_state_diff: memory, critic, planner, retriever, reranker targets
future_probe: near/far/anti-overgeneralization probe used by future_probe_win_rate

Current JSONL sample

{
  "sample_id": "u_000_turn_01",
  "domain": "interactive_novel",
  "turn": 1,
  "user_id": "u_000",
  "content_features": {
    "pacing_speed": 0.33271900787396735,
    "plot_progress": 0.36560961783808965,
    "romance_tension": 0.395952872256876,
    "female_agency": 0.8943843398696334,
    "lore_density": 0.5874555193248691,
    "novelty": 0.5618737041174806,
    "coherence": 0.789628163237762,
    "emotional_peak": 0.42427267651106393,
    "trope_misunderstanding": 0.11114339346825325
  },
  "utility": 0.7507,
  "explicit_feedback": {
    "like": true,
    "favorite": false,
    "rating": 4,
    "comment": "铺垫太多了，想看剧情推进"
  },
  "implicit_feedback": {
    "dwell_time_sec": 24.28,
    "dwell_z": 0.028,
    "fast_swipe": true,
    "reread": false,
    "continue_next": false
  },
  "gold_preference_update": [
    {
      "dimension": "pacing",
      "direction": "increase_speed",
      "scope": "current_story",
      "confidence": 0.74,
      "evidence": [
        "铺垫太多了，想看剧情推进",
        "fast_swipe"
      ]
    }
  ],
  "gold_agent_update": [
    {
      "target": "critic.checklist",
      "operation": "add_or_increase_weight",
      "content": "Check whether exposition delays plot progress before accepting a scene.",
      "scope": "current_story",
      "confidence": 0.76
    },
    {
      "target": "planner.policy",
      "operation": "add_or_increase_weight",
      "content": "Prefer candidate plans that introduce irreversible plot movement within the next scene.",
      "scope": "current_arc",
      "confidence": 0.72
    }
  ],
  "gold_state_diff": [
    {
      "target": "critic.checklist",
      "operation": "add_or_increase_weight",
      "content": "Check whether exposition delays plot progress before accepting a scene.",
      "scope": "current_story",
      "confidence": 0.76
    },
    {
      "target": "planner.policy",
      "operation": "add_or_increase_weight",
      "content": "Prefer candidate plans that introduce irreversible plot movement within the next scene.",
      "scope": "current_arc",
      "confidence": 0.72
    }
  ],
  "future_probe": {
    "probe_type": "near",
    "prompt": "Generate the next novel scene while respecting the inferred user preference state.",
    "expected_behavior": "Improve pacing/agency fit without overgeneralizing local feedback."
  }
}

trace_user_bench.jsonl

JSONL Schema · Long-Horizon trace episode

episode_id: trace key, e.g. trace_0000
turns / trajectory: multi-step serialized novel scenes with features/tags/utility
delayed_feedback: feedback observed after later turns
gold_credit_assignment: culprit_step, culprit_dimension, rationale
counterfactual_repairs: repair action and expected utility gain

Current JSONL sample

{
  "episode_id": "trace_0000",
  "domain": "interactive_novel_long_horizon",
  "turns": [
    {
      "step": 1,
      "scene_summary": "Episode 000 turn 1: serialized novel scene candidate.",
      "content_features": {
        "pacing_speed": 0.582,
        "plot_progress": 0.647,
        "female_agency": 0.787,
        "lore_density": 0.275,
        "trope_misunderstanding": 0.0,
        "emotional_peak": 0.548
      },
      "tags": [
        "female_agency"
      ],
      "utility": 0.7442,
      "implicit_feedback": {
        "fast_swipe": false,
        "continue_next": true,
        "dwell_time_sec": 32.79
      }
    },
    {
      "step": 7,
      "scene_summary": "Episode 000 turn 7: serialized novel scene candidate.",
      "content_features": {
        "pacing_speed": 0.765,
        "plot_progress": 0.505,
        "female_agency": 0.746,
        "lore_density": 0.445,
        "trope_misunderstanding": 0.0,
        "emotional_peak": 0.868
      },
      "tags": [
        "pacing"
      ],
      "utility": 0.5899,
      "implicit_feedback": {
        "fast_swipe": false,
        "continue_next": false,
        "dwell_time_sec": 27.24
      }
    }
  ],
  "delayed_feedback": {
    "turn": 7,
    "sentiment": "negative",
    "comment": "前面设定解释太密，后面节奏被拖住了。",
    "observed_after_steps": [
      7
    ]
  },
  "gold_credit_assignment": {
    "culprit_step": 6,
    "culprit_dimension": "lore_density",
    "culprit_type": "content_state_update_miss",
    "confidence": 0.92,
    "rationale": "The delayed feedback mentions lore_density, first introduced at step 6."
  },
  "counterfactual_repairs": [
    {
      "repair_id": "repair_0000_lore_density",
      "target_step": 6,
      "target_dimension": "lore_density",
      "action": "move nonessential lore into later reveal and keep current action beat",
      "expected_utility_gain": 0.2018
    }
  ]
}

Metric definitions

dimension_f1

Predicted preference dimensions vs. gold dimensions, macro F1 over feedback rows.

target_f1

Predicted framework update targets vs. gold agent-state targets.

future_probe_win_rate

Mean win probability on future generation probes after applying the predicted state update.

neutral_no_update_accuracy

Accuracy on rows where the correct action is no preference/framework update.

culprit_step_accuracy

Exact-match accuracy for the delayed-feedback culprit step in a multi-turn trace.

repair_gain

Mean expected utility gain from the selected counterfactual repair.

credit_calibration

One minus confidence error; higher means confidence tracks correctness.

Formula boxes

future_probe_win_rate = (1 / |D_probe|) * sum_i P_win(generation_i after state_update_i)

dimension_f1 = 2 * precision(dim_pred, dim_gold) * recall(dim_pred, dim_gold) / (precision + recall)

culprit_step_accuracy = (1 / N) * sum_i 1[predicted_culprit_step_i = gold_culprit_step_i]

repair_gain = (1 / N) * sum_i (utility_after_selected_repair_i - utility_before_repair_i)

这些公式是 plain text / LaTeX-style fallback，放在 box 内直接可读，不依赖额外前端库。

Metric charts

每条横线对应 Round 1–10；越长越高。Self-Evolve 重点看 future_probe_win_rate 是否随状态更新改善；Long-Horizon 重点看 delayed feedback 的 culprit step 与 repair gain。

Self-Evolve · future_probe_win_rate

R10.6516
R20.6635
R30.7061
R40.7977
R50.8211
R60.8294
R70.8374
R80.8454
R90.8580
R100.8632

Self-Evolve · target_f1

R10.2532
R20.2532
R30.6211
R40.6211
R50.7317
R60.7317
R70.7317
R80.7319
R90.7319
R100.7319

Long-Horizon · culprit_step_accuracy

R10.0000
R20.6500
R30.9250
R40.9500
R50.9583
R60.9583
R70.9667
R80.9667
R90.9667
R100.9750

Long-Horizon · repair_gain

R10.0760
R20.1461
R30.1846
R40.2016
R50.2153
R60.2154
R70.2167
R80.2168
R90.2169
R100.2188

Self-Evolve / Feedback-to-State

Round	Method	Metrics	主 agent 诊断	Artifacts
Round 1	explicit-only baseline	future_probe_win_rate 0.6516dimension_f1 0.7996target_f1 0.2532overgeneralization_rate 0.6650neutral_no_update_accuracy 1.0000	implicit_feedback_blind_spots Self-evolve round 1 dominant issue: implicit_feedback_blind_spots.	HTML · JSON
Round 2	implicit calibrated updater	future_probe_win_rate 0.6635dimension_f1 0.8280target_f1 0.2532overgeneralization_rate 0.6650neutral_no_update_accuracy 1.0000	scope_overgeneralization Self-evolve round 2 dominant issue: scope_overgeneralization.	HTML · JSON
Round 3	state-diff target router	future_probe_win_rate 0.7061dimension_f1 0.7165target_f1 0.6211overgeneralization_rate 0.8553neutral_no_update_accuracy 1.0000	scope_overgeneralization Self-evolve round 3 dominant issue: scope_overgeneralization.	HTML · JSON
Round 4	scope verifier	future_probe_win_rate 0.7977dimension_f1 0.7165target_f1 0.6211overgeneralization_rate 0.0000neutral_no_update_accuracy 1.0000	framework_target_gap_after_scope_fix Self-evolve round 4 dominant issue: framework_target_gap_after_scope_fix.	HTML · JSON
Round 5	self-debugged PUMA-lite	future_probe_win_rate 0.8211dimension_f1 0.7165target_f1 0.7317overgeneralization_rate 0.0000neutral_no_update_accuracy 1.0000	residual_future_probe_gap Self-evolve round 5 dominant issue: residual_future_probe_gap.	HTML · JSON
Round 6	Round 6 neutral-control evaluator	future_probe_win_rate 0.8294dimension_f1 0.7165target_f1 0.7317overgeneralization_rate 0.0000neutral_no_update_accuracy 1.0000	residual_future_probe_gap Self-evolve round 6 dominant issue: residual_future_probe_gap.	HTML · JSON
Round 7	Round 7 positive-signal gate	future_probe_win_rate 0.8374dimension_f1 0.7165target_f1 0.7317overgeneralization_rate 0.0000neutral_no_update_accuracy 1.0000	residual_future_probe_gap Self-evolve round 7 dominant issue: residual_future_probe_gap.	HTML · JSON
Round 8	Round 8 confidence-calibrated router	future_probe_win_rate 0.8454dimension_f1 0.7167target_f1 0.7319overgeneralization_rate 0.0000neutral_no_update_accuracy 1.0000	residual_future_probe_gap Self-evolve round 8 dominant issue: residual_future_probe_gap.	HTML · JSON
Round 9	Round 9 future-probe-aware reranker	future_probe_win_rate 0.8580dimension_f1 0.7167target_f1 0.7319overgeneralization_rate 0.0000neutral_no_update_accuracy 1.0000	residual_future_probe_gap Self-evolve round 9 dominant issue: residual_future_probe_gap.	HTML · JSON
Round 10	Round 10 consolidated PUMA-lite v2	future_probe_win_rate 0.8632dimension_f1 0.7167target_f1 0.7319overgeneralization_rate 0.0000neutral_no_update_accuracy 1.0000	residual_future_probe_gap Self-evolve round 10 dominant issue: residual_future_probe_gap.	HTML · JSON

Long-Horizon / Traceable Credit Assignment

Round	Method	Metrics	主 agent 诊断	Artifacts
Round 1	final-turn blame baseline	culprit_step_accuracy 0.0000culprit_dimension_accuracy 0.7917credit_mrr 0.2863repair_gain 0.0760credit_calibration 0.3800	temporal_credit_blame_errors Long-horizon round 1 dominant issue: temporal_credit_blame_errors.	HTML · JSON
Round 2	dimension evidence scan	culprit_step_accuracy 0.6500culprit_dimension_accuracy 0.8917credit_mrr 0.8167repair_gain 0.1461credit_calibration 0.6277	hard_distractor_temporal_ambiguity Long-horizon round 2 dominant issue: hard_distractor_temporal_ambiguity.	HTML · JSON
Round 3	temporal utility window	culprit_step_accuracy 0.9250culprit_dimension_accuracy 0.9333credit_mrr 0.9625repair_gain 0.1846credit_calibration 0.9080	culprit_dimension_evidence_gap Long-horizon round 3 dominant issue: culprit_dimension_evidence_gap.	HTML · JSON
Round 4	causal candidate scorer	culprit_step_accuracy 0.9500culprit_dimension_accuracy 0.9500credit_mrr 0.9750repair_gain 0.2016credit_calibration 0.9320	multi_cause_trace_ambiguity Long-horizon round 4 dominant issue: multi_cause_trace_ambiguity.	HTML · JSON
Round 5	trace verifier + repair planner	culprit_step_accuracy 0.9583culprit_dimension_accuracy 0.9583credit_mrr 0.9792repair_gain 0.2153credit_calibration 0.9400	multi_cause_trace_ambiguity Long-horizon round 5 dominant issue: multi_cause_trace_ambiguity.	HTML · JSON
Round 6	Round 6 tie-aware trace logger	culprit_step_accuracy 0.9583culprit_dimension_accuracy 0.9583credit_mrr 0.9792repair_gain 0.2154credit_calibration 0.9400	multi_cause_trace_ambiguity Long-horizon round 6 dominant issue: multi_cause_trace_ambiguity.	HTML · JSON
Round 7	Round 7 verifier fallback	culprit_step_accuracy 0.9667culprit_dimension_accuracy 0.9667credit_mrr 0.9833repair_gain 0.2167credit_calibration 0.9480	multi_cause_trace_ambiguity Long-horizon round 7 dominant issue: multi_cause_trace_ambiguity.	HTML · JSON
Round 8	Round 8 margin-calibrated confidence	culprit_step_accuracy 0.9667culprit_dimension_accuracy 0.9667credit_mrr 0.9833repair_gain 0.2168credit_calibration 0.9480	multi_cause_trace_ambiguity Long-horizon round 8 dominant issue: multi_cause_trace_ambiguity.	HTML · JSON
Round 9	Round 9 partial-credit multi-cause scorer	culprit_step_accuracy 0.9667culprit_dimension_accuracy 0.9667credit_mrr 0.9833repair_gain 0.2169credit_calibration 0.9480	multi_cause_trace_ambiguity Long-horizon round 9 dominant issue: multi_cause_trace_ambiguity.	HTML · JSON
Round 10	Round 10 CREDIT-TRACE v2	culprit_step_accuracy 0.9750culprit_dimension_accuracy 0.9750credit_mrr 0.9875repair_gain 0.2188credit_calibration 0.9560	multi_cause_trace_ambiguity Long-horizon round 10 dominant issue: multi_cause_trace_ambiguity.	HTML · JSON

每一轮具体问题

Concrete problems by round

每轮至少公开一个 failure / residual hard case，避免只展示好看的平均分。

Self-Evolve · Round 1open

u_000_turn_09
女主有点被动，想看她自己做决定
missed_dimensions=lore_density,pacing, missed_targets=critic.checklist,planner.policy,reranker.policy,retriever.policy, extra_targets=memory.user

Self-Evolve · Round 2open

u_004_turn_09
女主有点被动，想看她自己做决定
missed_dimensions=lore_density,pacing, missed_targets=critic.checklist,planner.policy,reranker.policy,retriever.policy, extra_targets=memory.user

Self-Evolve · Round 3open

u_022_turn_03
这段不错，继续这个张力
missed_dimensions=female_agency,pacing, missed_targets=critic.checklist,planner.policy,reranker.policy, overgeneralized_scope

Self-Evolve · Round 4open

u_016_turn_04
不要再用误会梗了
missed_dimensions=female_agency,lore_density, missed_targets=critic.checklist,reranker.policy,retriever.policy, extra_targets=memory.user

Self-Evolve · Round 5open

u_016_turn_04
不要再用误会梗了
missed_dimensions=female_agency,lore_density, missed_targets=critic.checklist,reranker.policy,retriever.policy

Self-Evolve · Round 6open

u_016_turn_04
不要再用误会梗了
missed_dimensions=female_agency,lore_density, missed_targets=critic.checklist,reranker.policy,retriever.policy

Self-Evolve · Round 7open

u_016_turn_04
不要再用误会梗了
missed_dimensions=female_agency,lore_density, missed_targets=critic.checklist,reranker.policy,retriever.policy

Self-Evolve · Round 8open

u_016_turn_04
不要再用误会梗了
missed_dimensions=female_agency,lore_density, missed_targets=critic.checklist,reranker.policy,retriever.policy

Self-Evolve · Round 9open

u_016_turn_04
不要再用误会梗了
missed_dimensions=female_agency,lore_density, missed_targets=critic.checklist,reranker.policy,retriever.policy

Self-Evolve · Round 10open

u_016_turn_04
不要再用误会梗了
missed_dimensions=female_agency,lore_density, missed_targets=critic.checklist,reranker.policy,retriever.policy

Long-Horizon · Round 1open

trace_0000
前面设定解释太密，后面节奏被拖住了。
step_miss:gold=6:pred=7, dimension_miss:gold=lore_density:pred=pacing

Long-Horizon · Round 2open

trace_0008
那个误会梗埋得太早，后面解释让我出戏。
step_miss:gold=4:pred=2, dimension_miss:gold=trope_misunderstanding:pred=female_agency

Long-Horizon · Round 3open

trace_0008
那个误会梗埋得太早，后面解释让我出戏。
step_miss:gold=4:pred=2, dimension_miss:gold=trope_misunderstanding:pred=female_agency

Long-Horizon · Round 4open

trace_0017
前面设定解释太密，后面节奏被拖住了。
step_miss:gold=6:pred=4, dimension_miss:gold=lore_density:pred=female_agency

Long-Horizon · Round 5open

trace_0008
那个误会梗埋得太早，后面解释让我出戏。
step_miss:gold=4:pred=2, dimension_miss:gold=trope_misunderstanding:pred=female_agency

Long-Horizon · Round 6open

trace_0008
那个误会梗埋得太早，后面解释让我出戏。
step_miss:gold=4:pred=2, dimension_miss:gold=trope_misunderstanding:pred=female_agency

Long-Horizon · Round 7open

trace_0008
那个误会梗埋得太早，后面解释让我出戏。
step_miss:gold=4:pred=2, dimension_miss:gold=trope_misunderstanding:pred=female_agency

Long-Horizon · Round 8open

trace_0008
那个误会梗埋得太早，后面解释让我出戏。
step_miss:gold=4:pred=2, dimension_miss:gold=trope_misunderstanding:pred=female_agency

Long-Horizon · Round 9open

trace_0008
那个误会梗埋得太早，后面解释让我出戏。
step_miss:gold=4:pred=2, dimension_miss:gold=trope_misunderstanding:pred=female_agency

Long-Horizon · Round 10open

trace_0017
前面设定解释太密，后面节奏被拖住了。
step_miss:gold=6:pred=4, dimension_miss:gold=lore_density:pred=female_agency

为什么会出现 0 指标

Zero metrics are labeled as good / bad / baseline-only

overgeneralization_rate: 0 is good: no local feedback was incorrectly globalized.
neutral_no_update_accuracy: 0 is bad: neutral/no-update rows were still updated; this replaces the old unclear no_update_precision.
culprit_step_accuracy: 0 can occur for the final-turn blame baseline when delayed feedback refers to earlier steps.

后续 5 轮改进方向

Round 6–10 的主 agent patch 线

Round 6: add neutral/no-update controls and replace unclear no_update_precision with neutral_no_update_accuracy.
Round 7: gate positive style-affinity updates on stronger evidence to prevent weak likes from polluting memory.
Round 8: calibrate target routing and confidence by explicit comment dominance and framework target completeness.
Round 9: optimize state diffs against future probes and add partial-credit long-horizon scoring.
Round 10: consolidate PUMA-lite v2 / CREDIT-TRACE v2; remaining improvements require human-pilot logs.

Autoresearch Journal

主 agent 的 20 条迭代日志

来自本轮 run 的 research_journal.json，保留每轮 diagnosis 和 selected patch。

self_evolve · Round 1implicit_feedback_blind_spots
Add dwell/fast-swipe/continue calibration and content-feature evidence for silent reader dissatisfaction.
self_evolve · Round 2scope_overgeneralization
Add verifier to distinguish current-story/current-arc updates from durable global user preferences.
self_evolve · Round 3scope_overgeneralization
Add verifier to distinguish current-story/current-arc updates from durable global user preferences.
self_evolve · Round 4framework_target_gap_after_scope_fix
Run a verifier pass that re-routes each surviving dimension to all primary framework targets.
self_evolve · Round 5residual_future_probe_gap
Use future-probe-aware reranking and hard anti-overgeneralization probes; keep AHEAD disabled.
self_evolve · Round 6residual_future_probe_gap
Use future-probe-aware reranking and hard anti-overgeneralization probes; keep AHEAD disabled.
self_evolve · Round 7residual_future_probe_gap
Use future-probe-aware reranking and hard anti-overgeneralization probes; keep AHEAD disabled.
self_evolve · Round 8residual_future_probe_gap
Use future-probe-aware reranking and hard anti-overgeneralization probes; keep AHEAD disabled.
self_evolve · Round 9residual_future_probe_gap
Use future-probe-aware reranking and hard anti-overgeneralization probes; keep AHEAD disabled.
self_evolve · Round 10residual_future_probe_gap
Use future-probe-aware reranking and hard anti-overgeneralization probes; keep AHEAD disabled.
long_horizon · Round 1temporal_credit_blame_errors
Move beyond final-turn blame by scanning trajectory-wide evidence and delayed-feedback text.
long_horizon · Round 2hard_distractor_temporal_ambiguity
Add temporal utility-drop windows so early weak distractors do not steal credit from the real culprit.
long_horizon · Round 3culprit_dimension_evidence_gap
Fuse delayed-feedback dimension tokens with per-turn feature tags, severity, and repairability.
long_horizon · Round 4multi_cause_trace_ambiguity
Keep CREDIT-TRACE verifier; next expansion should add multi-cause labels from real reader logs, not AHEAD.
long_horizon · Round 5multi_cause_trace_ambiguity
Keep CREDIT-TRACE verifier; next expansion should add multi-cause labels from real reader logs, not AHEAD.
long_horizon · Round 6multi_cause_trace_ambiguity
Keep CREDIT-TRACE verifier; next expansion should add multi-cause labels from real reader logs, not AHEAD.
long_horizon · Round 7multi_cause_trace_ambiguity
Keep CREDIT-TRACE verifier; next expansion should add multi-cause labels from real reader logs, not AHEAD.
long_horizon · Round 8multi_cause_trace_ambiguity
Keep CREDIT-TRACE verifier; next expansion should add multi-cause labels from real reader logs, not AHEAD.
long_horizon · Round 9multi_cause_trace_ambiguity
Keep CREDIT-TRACE verifier; next expansion should add multi-cause labels from real reader logs, not AHEAD.
long_horizon · Round 10multi_cause_trace_ambiguity
Keep CREDIT-TRACE verifier; next expansion should add multi-cause labels from real reader logs, not AHEAD.

Regression Repair

已修复 verifier overcorrection

主 agent 发现 Long-Horizon Round 5 曾因 verifier 过度奖励 first introduction，导致早期弱 distractor 抢锅。已新增 regression test：test_long_horizon_round5_verifier_does_not_overcorrect_below_round4，并修复 scorer。最终 Round 10 culprit_step_accuracy = 0.9750，repair_gain = 0.2188。

Artifacts

可复盘文件

所有 artifact link 都用 absolute path，避免 Next catch-all 把相对链接解析到错误目录。

Data / Eval

manifest.json · pif_bench.jsonl · trace_user_bench.jsonl

Dashboard / Summary

dashboard/index.html · summary.json · research_journal.json