Novel Feedback Agent: Self-Evolve + Long-Horizon Credit Assignment

JSONL Schema、Current JSONL sample、Metric definitions 与 Formula boxes

Dashboard 直接内嵌当前数据格式、样例、公式框与 metric charts，方便不回主页面也能复盘。

{
  "sample_id": "u_000_turn_01",
  "domain": "interactive_novel",
  "turn": 1,
  "user_id": "u_000",
  "content_features": {
    "pacing_speed": 0.33271900787396735,
    "plot_progress": 0.36560961783808965,
    "romance_tension": 0.395952872256876,
    "female_agency": 0.8943843398696334,
    "lore_density": 0.5874555193248691,
    "novelty": 0.5618737041174806,
    "coherence": 0.789628163237762,
    "emotional_peak": 0.42427267651106393,
    "trope_misunderstanding": 0.11114339346825325
  },
  "utility": 0.7507,
  "explicit_feedback": {
    "like": true,
    "favorite": false,
    "rating": 4,
    "comment": "铺垫太多了，想看剧情推进"
  },
  "implicit_feedback": {
    "dwell_time_sec": 24.28,
    "dwell_z": 0.028,
    "fast_swipe": true,
    "reread": false,
    "continue_next": false
  },
  "gold_preference_update": [
    {
      "dimension": "pacing",
      "direction": "increase_speed",
      "scope": "current_story",
      "confidence": 0.74,
      "evidence": [
        "铺垫太多了，想看剧情推进",
        "fast_swipe"
      ]
    }
  ],
  "gold_agent_update": [
    {
      "target": "critic.checklist",
      "operation": "add_or_increase_weight",
      "content": "Check whether exposition delays plot progress before accepting a scene.",
      "scope": "current_story",
      "confidence": 0.76
    },
    {
      "target": "planner.policy",
      "operation": "add_or_increase_weight",
      "content": "Prefer candidate plans that introduce irreversible plot movement within the next scene.",
      "scope": "current_arc",
      "confidence": 0.72
    }
  ],
  "gold_state_diff": [
    {
      "target": "critic.checklist",
      "operation": "add_or_increase_weight",
      "content": "Check whether exposition delays plot progress before accepting a scene.",
      "scope": "current_story",
      "confidence": 0.76
    },
    {
      "target": "planner.policy",
      "operation": "add_or_increase_weight",
      "content": "Prefer candidate plans that introduce irreversible plot movement within the next scene.",
      "scope": "current_arc",
      "confidence": 0.72
    }
  ],
  "future_probe": {
    "probe_type": "near",
    "prompt": "Generate the next novel scene while respecting the inferred user preference state.",
    "expected_behavior": "Improve pacing/agency fit without overgeneralizing local feedback."
  }
}

{
  "episode_id": "trace_0000",
  "domain": "interactive_novel_long_horizon",
  "turns": [
    {
      "step": 1,
      "scene_summary": "Episode 000 turn 1: serialized novel scene candidate.",
      "content_features": {
        "pacing_speed": 0.582,
        "plot_progress": 0.647,
        "female_agency": 0.787,
        "lore_density": 0.275,
        "trope_misunderstanding": 0.0,
        "emotional_peak": 0.548
      },
      "tags": [
        "female_agency"
      ],
      "utility": 0.7442,
      "implicit_feedback": {
        "fast_swipe": false,
        "continue_next": true,
        "dwell_time_sec": 32.79
      }
    },
    {
      "step": 7,
      "scene_summary": "Episode 000 turn 7: serialized novel scene candidate.",
      "content_features": {
        "pacing_speed": 0.765,
        "plot_progress": 0.505,
        "female_agency": 0.746,
        "lore_density": 0.445,
        "trope_misunderstanding": 0.0,
        "emotional_peak": 0.868
      },
      "tags": [
        "pacing"
      ],
      "utility": 0.5899,
      "implicit_feedback": {
        "fast_swipe": false,
        "continue_next": false,
        "dwell_time_sec": 27.24
      }
    }
  ],
  "delayed_feedback": {
    "turn": 7,
    "sentiment": "negative",
    "comment": "前面设定解释太密，后面节奏被拖住了。",
    "observed_after_steps": [
      7
    ]
  },
  "gold_credit_assignment": {
    "culprit_step": 6,
    "culprit_dimension": "lore_density",
    "culprit_type": "content_state_update_miss",
    "confidence": 0.92,
    "rationale": "The delayed feedback mentions lore_density, first introduced at step 6."
  },
  "counterfactual_repairs": [
    {
      "repair_id": "repair_0000_lore_density",
      "target_step": 6,
      "target_dimension": "lore_density",
      "action": "move nonessential lore into later reveal and keep current action beat",
      "expected_utility_gain": 0.2018
    }
  ]
}

Metric definitions

Formula boxes

future_probe_win_rate = (1 / |D_probe|) * sum_i P_win_i

culprit_step_accuracy = (1 / N) * sum_i 1[pred_step_i = gold_step_i]

repair_gain = (1 / N) * sum_i (utility_after_repair_i - utility_before_i)

Metric charts

Data & Evaluation Protocol

PIF-Bench synthetic novel feedback trajectories: Run the round-specific updater on every feedback event, compare predicted dimensions/targets/scope with gold state diffs, then estimate future-probe success.

TRACE-USER-Bench synthetic delayed-feedback traces: Rank culprit steps from the full trace, compare top prediction and reciprocal rank to gold, then estimate counterfactual repair gain and confidence calibration.

Concrete problems by round

1. self_evolve Round 1: missed_dimensions=lore_density,pacing, missed_targets=critic.checklist,planner.policy,reranker.policy,retriever.policy, extra_targets=memory.user, overgeneralized_scope

   女主有点被动，想看她自己做决定

2. self_evolve Round 2: missed_dimensions=lore_density,pacing, missed_targets=critic.checklist,planner.policy,reranker.policy,retriever.policy, extra_targets=memory.user, overgeneralized_scope

   女主有点被动，想看她自己做决定

3. self_evolve Round 3: missed_dimensions=female_agency,pacing, missed_targets=critic.checklist,planner.policy,reranker.policy, overgeneralized_scope

   这段不错，继续这个张力

4. self_evolve Round 4: missed_dimensions=female_agency,lore_density, missed_targets=critic.checklist,reranker.policy,retriever.policy, extra_targets=memory.user

   不要再用误会梗了

5. self_evolve Round 5: missed_dimensions=female_agency,lore_density, missed_targets=critic.checklist,reranker.policy,retriever.policy

   不要再用误会梗了

6. self_evolve Round 6: missed_dimensions=female_agency,lore_density, missed_targets=critic.checklist,reranker.policy,retriever.policy

   不要再用误会梗了

7. self_evolve Round 7: missed_dimensions=female_agency,lore_density, missed_targets=critic.checklist,reranker.policy,retriever.policy

   不要再用误会梗了

8. self_evolve Round 8: missed_dimensions=female_agency,lore_density, missed_targets=critic.checklist,reranker.policy,retriever.policy

   不要再用误会梗了

9. self_evolve Round 9: missed_dimensions=female_agency,lore_density, missed_targets=critic.checklist,reranker.policy,retriever.policy

   不要再用误会梗了

10. self_evolve Round 10: missed_dimensions=female_agency,lore_density, missed_targets=critic.checklist,reranker.policy,retriever.policy

    不要再用误会梗了

11. long_horizon Round 1: step_miss:gold=6:pred=7, dimension_miss:gold=lore_density:pred=pacing

    前面设定解释太密，后面节奏被拖住了。

12. long_horizon Round 2: step_miss:gold=4:pred=2, dimension_miss:gold=trope_misunderstanding:pred=female_agency

    那个误会梗埋得太早，后面解释让我出戏。

13. long_horizon Round 3: step_miss:gold=4:pred=2, dimension_miss:gold=trope_misunderstanding:pred=female_agency

    那个误会梗埋得太早，后面解释让我出戏。

14. long_horizon Round 4: step_miss:gold=6:pred=4, dimension_miss:gold=lore_density:pred=female_agency

    前面设定解释太密，后面节奏被拖住了。

15. long_horizon Round 5: step_miss:gold=4:pred=2, dimension_miss:gold=trope_misunderstanding:pred=female_agency

    那个误会梗埋得太早，后面解释让我出戏。

16. long_horizon Round 6: step_miss:gold=4:pred=2, dimension_miss:gold=trope_misunderstanding:pred=female_agency

    那个误会梗埋得太早，后面解释让我出戏。

17. long_horizon Round 7: step_miss:gold=4:pred=2, dimension_miss:gold=trope_misunderstanding:pred=female_agency

    那个误会梗埋得太早，后面解释让我出戏。

18. long_horizon Round 8: step_miss:gold=4:pred=2, dimension_miss:gold=trope_misunderstanding:pred=female_agency

    那个误会梗埋得太早，后面解释让我出戏。

19. long_horizon Round 9: step_miss:gold=4:pred=2, dimension_miss:gold=trope_misunderstanding:pred=female_agency

    那个误会梗埋得太早，后面解释让我出戏。

20. long_horizon Round 10: step_miss:gold=6:pred=4, dimension_miss:gold=lore_density:pred=female_agency

    前面设定解释太密，后面节奏被拖住了。

Why zero-valued metrics appear

• self_evolve Round 4: overgeneralization_rate = 0 是好信号，表示本轮没有把局部反馈错误泛化为全局偏好。
• self_evolve Round 5: overgeneralization_rate = 0 是好信号，表示本轮没有把局部反馈错误泛化为全局偏好。
• self_evolve Round 6: overgeneralization_rate = 0 是好信号，表示本轮没有把局部反馈错误泛化为全局偏好。
• self_evolve Round 7: overgeneralization_rate = 0 是好信号，表示本轮没有把局部反馈错误泛化为全局偏好。
• self_evolve Round 8: overgeneralization_rate = 0 是好信号，表示本轮没有把局部反馈错误泛化为全局偏好。
• self_evolve Round 9: overgeneralization_rate = 0 是好信号，表示本轮没有把局部反馈错误泛化为全局偏好。
• self_evolve Round 10: overgeneralization_rate = 0 是好信号，表示本轮没有把局部反馈错误泛化为全局偏好。
• long_horizon Round 1: culprit_step_accuracy = 0 出现在 final-turn blame baseline：延迟反馈的真实 culprit 不在最后一步。

后续 5 轮改进方向

1. Round 6: add neutral/no-update controls and replace unclear no_update_precision with neutral_no_update_accuracy.
2. Round 7: gate positive style-affinity updates on stronger evidence to prevent weak likes from polluting memory.
3. Round 8: calibrate target routing and confidence by explicit comment dominance and framework target completeness.
4. Round 9: optimize state diffs against future probes and add partial-credit long-horizon scoring.
5. Round 10: consolidate PUMA-lite v2 / CREDIT-TRACE v2; remaining improvements require human-pilot logs.

Autoresearch journal

1. self_evolve R1: implicit_feedback_blind_spots → Add dwell/fast-swipe/continue calibration and content-feature evidence for silent reader dissatisfaction.
2. self_evolve R2: scope_overgeneralization → Add verifier to distinguish current-story/current-arc updates from durable global user preferences.
3. self_evolve R3: scope_overgeneralization → Add verifier to distinguish current-story/current-arc updates from durable global user preferences.
4. self_evolve R4: framework_target_gap_after_scope_fix → Run a verifier pass that re-routes each surviving dimension to all primary framework targets.
5. self_evolve R5: residual_future_probe_gap → Use future-probe-aware reranking and hard anti-overgeneralization probes; keep AHEAD disabled.
6. self_evolve R6: residual_future_probe_gap → Use future-probe-aware reranking and hard anti-overgeneralization probes; keep AHEAD disabled.
7. self_evolve R7: residual_future_probe_gap → Use future-probe-aware reranking and hard anti-overgeneralization probes; keep AHEAD disabled.
8. self_evolve R8: residual_future_probe_gap → Use future-probe-aware reranking and hard anti-overgeneralization probes; keep AHEAD disabled.
9. self_evolve R9: residual_future_probe_gap → Use future-probe-aware reranking and hard anti-overgeneralization probes; keep AHEAD disabled.
10. self_evolve R10: residual_future_probe_gap → Use future-probe-aware reranking and hard anti-overgeneralization probes; keep AHEAD disabled.
11. long_horizon R1: temporal_credit_blame_errors → Move beyond final-turn blame by scanning trajectory-wide evidence and delayed-feedback text.
12. long_horizon R2: hard_distractor_temporal_ambiguity → Add temporal utility-drop windows so early weak distractors do not steal credit from the real culprit.
13. long_horizon R3: culprit_dimension_evidence_gap → Fuse delayed-feedback dimension tokens with per-turn feature tags, severity, and repairability.
14. long_horizon R4: multi_cause_trace_ambiguity → Keep CREDIT-TRACE verifier; next expansion should add multi-cause labels from real reader logs, not AHEAD.
15. long_horizon R5: multi_cause_trace_ambiguity → Keep CREDIT-TRACE verifier; next expansion should add multi-cause labels from real reader logs, not AHEAD.
16. long_horizon R6: multi_cause_trace_ambiguity → Keep CREDIT-TRACE verifier; next expansion should add multi-cause labels from real reader logs, not AHEAD.
17. long_horizon R7: multi_cause_trace_ambiguity → Keep CREDIT-TRACE verifier; next expansion should add multi-cause labels from real reader logs, not AHEAD.
18. long_horizon R8: multi_cause_trace_ambiguity → Keep CREDIT-TRACE verifier; next expansion should add multi-cause labels from real reader logs, not AHEAD.
19. long_horizon R9: multi_cause_trace_ambiguity → Keep CREDIT-TRACE verifier; next expansion should add multi-cause labels from real reader logs, not AHEAD.
20. long_horizon R10: multi_cause_trace_ambiguity → Keep CREDIT-TRACE verifier; next expansion should add multi-cause labels from real reader logs, not AHEAD.

Artifacts

• Data manifest: data/manifest.json
• PIF-Bench: data/pif_bench.jsonl
• TRACE-USER-Bench: data/trace_user_bench.jsonl
• Round eval JSON files: eval/*_round_*.json
• Research journal: journal/research_journal.json