Dynamic User Simulator · state-conditioned planning · no personalized reranking
Novel Feedback v3 Experiment Visualization
CPU smoke experiment for Qidian DTA adapter + augmented real-data-calibrated trajectories + Dynamic User Simulator + closed-loop evaluation. The generation policy uses state-conditioned planning and direct generation; it does no personalized reranking.
Qidian DTA adapter
Real ingest status
The Qidian-Webnovel metadata/code repository is cloned under raw/qidian_code_repo. README confirms reader-response comments require a Data Transfer Agreement, so the current experiment does not bypass access restrictions; it provides an adapter path and uses public WebNovelBench/NovelUpdates snapshots plus simulator-calibrated augmentation.
Simulator metrics
Dynamic User Simulator
{
"n_events": 360,
"train_events": 234,
"test_events": 126,
"rating_mae": 0.172,
"continue_auc": 0.9258,
"fast_swipe_auc": 0.9879,
"transition_trust_mae": 0.0134,
"transition_fatigue_mae": 0.0,
"calibration_ece": 0.0849,
"counterfactual_separation": 0.1333
}closed-loop evaluation
trajectory-level personalization returns
| method | dynamic_return | expectation_alignment | personalization_diversity | state_growth |
|---|---|---|---|---|
| no_update | 2.3827 | 0.0 | 0.225 | 0.0 |
| static_profile | 2.4816 | 0.3333 | 0.2948 | 0.667 |
| memory_only | 2.4216 | 0.2667 | 0.2202 | 0.5 |
| state_conditioned_planning | 2.5917 | 0.7667 | 0.277 | 1.833 |
| oracle_update | 2.7894 | 0.8 | 0.485 | 3.833 |
Interpretation
What this tests
The key test is whether user-feedback state updates and transitions alter later generated content at the trajectory level. The state-conditioned planning method updates planner/critic/retriever/generator state and then directly generates the next chapter; it does not sample multiple candidates for personalized reranking.