Adversarial contradictory request
adversarial_contradiction severity 0.891. Evidence: oracle gap 0.169, raw overgen 0.538, stale memory 0.400.
v4 fix: add ordinal/confidence labels, ECE, ask/hold gate and contradiction stress tests
Base & Replicate · Bridge Synthetic-to-Real · no personalized reranking
Novel Feedback v4 Encoder-State Simulator
这一版把 v3+ 的 96-run suite 往“可训练 encoder + 可解释指标 + GPU 实验”推进：复现 v3、诊断 worst 3 scenarios、设计 UserTrajectory schema、训练 encoder-state simulator，并把 NCFU、state-delta cosine、story consistency、feedback credit assignment 放到同一个可读 dashboard。
v3 reproduction
96 runs
Rechecked v3 suite: 12 scenarios × 8 methods. State vs memory uplift 0.2748.
Research audit
19 sources
Personalized reward/user simulator/novel consistency/long-horizon evaluation sources mapped to encoder and metric requirements.
Encoder
AUC 0.709
Transformer-state simulator improves behavior fidelity and state-delta prediction over non-history/zero-delta baselines.
GPU experiment
volc_submitted
Local device: cuda. VolcEngine task metadata is preserved in gpu_task_submission.json.
Base & Replicate
v3 reproduction + worst 3 scenarios
不是重新包装平均分，而是把 v3 的 uncertainty_aware_planning / state_conditioned_planning 复现出来，再按 oracle gap、robustness、calibration、overgeneralization、stale memory 排序。
Long-horizon fatigue accumulation
long_horizon_fatigue severity 0.814. Evidence: oracle gap 0.162, raw overgen 0.384, stale memory 0.440.
v4 fix: evaluate feedback-to-future trace, delayed credit MRR and rollback scar rate
Rollback after correction
rollback_recovery severity 0.773. Evidence: oracle gap 0.150, raw overgen 0.383, stale memory 0.413.
v4 fix: evaluate feedback-to-future trace, delayed credit MRR and rollback scar rate
Deep code walk
How state becomes generation
state_conditioned_planning consumes explicit + implicit feedback, story axes, trust/fatigue/created_expectations and scoped diffs; it updates planner/critic constraints with TTL/provenance and applies them to a direct_generation_prompt. No candidate_set, no reranked_candidates.
uncertainty_aware_planning adds confidence and an ask/hold gate; low-confidence implicit-only or contradictory feedback can be held or clarified instead of becoming global memory.
State update scope
Scope / source / TTL are metrics
source: explicit+implicit feedbackscope: next_1_to_2_chaptersanti_scope: global genre preferencettl: until_arc_resolutionprovenance: feedback_turn
V4 treats these fields as supervised/validated state-diff atoms, not as prose hidden in a prompt.
Bridge Synthetic-to-Real
UserTrajectory schema and real-data path
真实 reader-response 数据优先走 opt-in/pilot 或 DTA。Qidian comments/replies 需要 Data Transfer Agreement；v4 不绕过、不抓 protected 数据。
UserTrajectory schema
{
"title": "UserTrajectory",
"required": [
"trajectory_id",
"user_id",
"source",
"license_class",
"uses_reranking",
"events"
]
}Sample records include explicit rating/comment, implicit dwell_time_sec, dynamic state_before/state_after, gold_agent_state_update, and future_probes.
Crowdsourced browser-plugin reading experiment
Opt-in readers consume public-domain or author-consented serialized fiction. The plugin logs dwell/scroll/continue/reread and explicit A/B choices locally, exports anonymized UserTrajectory rows, and never scrapes protected platform comments. This gives a human-pilot calibration split for the simulator.
Encoder training
Neural encoder-state simulator smoke run
Inputs: profile + state_before + history window + candidate narrative axes. Heads: continue, comment, dwell, state_delta. The dashboard exposes behavior fidelity and state fidelity separately.
| Model | continue AUC | dwell log MAE | state-delta MSE | state-delta cosine | counterfactual consistency |
|---|---|---|---|---|---|
zero_delta_baseline | 0.5000 | 0.1886 | 0.00080 | 0.0000 | 0.5000 |
mlp_no_history | 0.7038 | 0.1385 | 0.00040 | 0.6630 | 0.9961 |
gru_history | 0.7037 | 0.1330 | 0.00040 | 0.6612 | 1.0000 |
transformer_state | 0.7087 | 0.1575 | 0.00060 | 0.5799 | 1.0000 |
transformer_prototype | 0.7236 | 0.1768 | 0.00050 | 0.6561 | 1.0000 |
Key readout: transformer_state beats mlp_no_history on behavior prediction and zero_delta_baseline on state_delta_mse; transformer_prototype preserves/improves counterfactual consistency for user/content/fatigue swaps.
metric definitions
NCFU, behavior, state, story consistency
所有指标都放在 robust formula boxes 中，避免 broken inline formulas；每个指标都说明“高/低代表什么”。
ncfu — Normalized Causal Future Uplift: future reward caused by model state update, normalized by the oracle update improvement over no-update.
NCFU = E[J(G(S_t ⊕ ΔS_model)) - J(G(S_t))] / (E[J(G(S_t ⊕ ΔS_gold)) - J(G(S_t))] + ε)
Higher is better. Unlike raw return, NCFU asks whether the state update itself caused future improvement.
behavior_fidelity — How well the simulator predicts observed user behavior such as continue/drop, dwell, comment and fast-swipe.
BehaviorFidelity = mean( AUC_continue, AUPRC_comment, 1 - normMAE_log_dwell, 1 - ECE_behavior )
Higher means the simulator can reproduce behavior distributions; report by user segment and scenario.
state_delta_cosine — Cosine similarity between predicted and gold state-delta vectors for trust/fatigue/satisfaction/preferences.
StateDeltaCosine = cos( Δs_pred, Δs_gold ) = (Δs_pred · Δs_gold) / (||Δs_pred|| ||Δs_gold|| + ε)
Higher means the encoder learns direction of state change, not only whether the current chapter was liked.
story_consistency — Canon/entity/plot continuity score for generated later content under personalized state updates.
StoryConsistency = 1 - weighted_mean(contradiction_rate, unsupported_major_fact_rate, invalid_retcon_rate, entity_status_error_rate)
Personalization cannot win by breaking the novel. Track story consistency jointly with engagement.
feedback_credit_assignment — Ability to trace later content changes back to the right feedback event and state field.
CreditAssignment = mean(culprit_event_F1, state_field_F1, credit_nDCG, repair_gain_correlation)
Higher is better for delayed feedback and long-horizon fatigue scenarios.
encoder_state_score — Aggregate score for the v4 encoder-state simulator across behavior, state and intervention validity.
EncoderStateScore = 0.35*BehaviorFidelity + 0.35*StateDeltaCosine + 0.20*CounterfactualConsistency + 0.10*(1-ECE)
Use as a compact dashboard score, but always show component metrics to avoid hiding failures.
calibration_ece — Expected calibration error for behavior and state-update confidence.
ECE = Σ_b |B_b|/n · | observed_accuracy(B_b) - mean_confidence(B_b) |
Lower is better. Essential for implicit-only, noisy and sarcastic feedback.
Research audit
Why these metrics and this encoder
个性化 reward/user simulator 文献给 user-swap 与 state-transition；story generation/consistency 文献给 canon、entity、outline、long-horizon probes。
Personalized reward / preference learningLaMP: When Large Language Models Meet Personalization
Use static-profile, full-history and retrieval personalization baselines; report lift over generic generation.
Personalized reward / preference learningPRISM Alignment DatasetModel plural user-specific preference, not a single universal reward.
Personalized reward / preference learningP-GenRM / Personalized Generative Reward ModelsCondition reward/critic on user history and dynamic state; evaluate user-swap separation.
User simulator and dynamic stateGenerative Agents: Interactive Simulacra of Human BehaviorSeparate memory, reflection/planning, and behavior emission; simulator should update latent state.
User simulator and dynamic stateProPerSim / proactive personalized user-assistant simulationUse stateful user-assistant rollouts rather than static persona prompting.
User simulator and dynamic statePrefDisco / preference discoveryAdd active preference discovery and value-of-information metrics.
User simulator and dynamic stateSwap-guided preference learningUse same-text/different-user and same-user/different-content hard negatives.
Novel consistency and long-horizon coherenceROCStories / Story Cloze TestShort causal story-ending checks are a minimal local coherence control.
Novel consistency and long-horizon coherenceWritingPrompts / Hierarchical Neural Story GenerationOpen-ended reference metrics are weak; use human/LLM pairwise and axis-specific rubrics.
Novel consistency and long-horizon coherencePlotMachines: Outline-Conditioned Generation with Dynamic Plot State TrackingMaintain explicit plot/story state and test plan adherence.
Novel consistency and long-horizon coherenceSTORIUM: Dataset and Evaluation Platform for Machine-in-the-Loop Story GenerationEvaluate continuations against structured character/location/plot constraints.
Novel consistency and long-horizon coherenceRe3: Generating Longer Stories with Recursive Reprompting and RevisionLong-form generation needs recursive revision and long-horizon coherence checks.
Novel consistency and long-horizon coherenceDOC: Improving Long Story Coherence With Detailed Outline ControlScore detailed-outline adherence separately from prose quality.
Novel consistency and long-horizon coherenceOpenMEVA story-generation metric benchmarkAutomatic metrics need validation against human preference; avoid BLEU-only scoring.
Novel consistency and long-horizon coherencePG-19 long-form language modeling corpusPublic-domain long novels support long-context smoke tests, though not preference labels.
Novel consistency and long-horizon coherenceNarrativeQAQA over prior story state can probe whether generated chapters preserve canon.
Novel consistency and long-horizon coherenceQuALITY long-document QALong-context multiple-choice probes can be converted into chapter-continuity checks.
Novel consistency and long-horizon coherenceLitBank and BookNLPCharacter/entity extraction enables automated character consistency metrics.
Novel consistency and long-horizon coherenceFactCC / QAGS / FEQA / SummaC factual consistency familyAdapt claim extraction + retrieval + NLI/QA to plot/canon contradictions.
GPU experiment
volc_submitted
Local CUDA/PyTorch smoke experiment completed with cuda; VolcEngine metadata/status is recorded separately. This gives a reproducible CPU/CUDA fallback even if remote queue is delayed.
{
"status": "volc_submitted",
"volc_task_id": "t-20260512105935-c47bs",
"local_device": "cuda",
"torch_version": "2.8.0+cu128",
"workspace": "/root/code/vepfs/hermes/gpu",
"queue": "q-20251224104457-jzdpj",
"resource_queue_name": "c20250509",
"flavor": "ml.pni2.28xlarge",
"entrypoint": "python3 /root/code/vepfs/hermes/novel-demo/personalized-user-feedback-agent-evolution/v4-encoder-simulator/scripts/run_v4_iteration.py --artifacts-dir /root/code/vepfs/hermes/novel-demo/personalized-user-feedback-agent-evolution/v4-encoder-simulator/artifacts/v4_encoder_state_simulator_volc --gpu-status volc_completed",
"safety": "Volc credentials/logs/YAML stay under /root/code/vepfs/hermes/gpu; no AK/SK is persisted in artifacts."
}No personalized reranking
Direct generation remains the research target
V4 uses reward/simulator models for diagnosis and state-update learning, but generated content is one state-conditioned plan/excerpt, not K candidates plus personalized reranking. Every public JSON/JSONL artifact includes uses_reranking=false.
Artifacts
Raw v4 artifacts
Public artifacts include the research audit, v3 baseline diagnosis, UserTrajectory schema/samples, encoder train metrics, prediction samples, eval design and GPU task metadata.