Falsifiable claim: In finite discounted MDPs and CPU-scale continuous-control benchmarks, a learned joint tuple model that is improvement-equivalent for the learner's current policy class and replay distribution can produce policy updates with nonnegative true-environment improvement at least as reliably as an observationally trained maximum-likelihood dynamics model of the same capacity, while using less representational detail. Theoretical program: define OE as equality or calibrated approximation of the joint tuple law P(s,a,r,gamma,s'), VE as equality of Bellman backup outputs over specified sets Pi and F, GE as alignment or equality of gradients of the learner's Bellman/TD loss over specified theta and data distributions, and IE as positive alignment between the true and model-induced update direction, e.g. cos(grad_theta delta^2, grad_theta delta_tilde^2) > 0 or a provable positive lower bound on J(pi_{theta+eta u_tilde}) - J(pi_theta). Each definition must…Read more
Read less
Falsifiable claim: In finite discounted MDPs and CPU-scale continuous-control benchmarks, a learned joint tuple model that is improvement-equivalent for the learner's current policy class and replay distribution can produce policy updates with nonnegative true-environment improvement at least as reliably as an observationally trained maximum-likelihood dynamics model of the same capacity, while using less representational detail. Theoretical program: define OE as equality or calibrated approximation of the joint tuple law P(s,a,r,gamma,s'), VE as equality of Bellman backup outputs over specified sets Pi and F, GE as alignment or equality of gradients of the learner's Bellman/TD loss over specified theta and data distributions, and IE as positive alignment between the true and model-induced update direction, e.g. cos(grad_theta delta^2, grad_theta delta_tilde^2) > 0 or a provable positive lower bound on J(pi_{theta+eta u_tilde}) - J(pi_theta). Each definition must state whether it is all-policy/all-state/all-action/all-value/all-theta, policy-class, or current-policy/on-distribution. Hypotheses: H1 OE implies VE implies GE implies IE only under matched support, differentiability, planner/update compatibility, and quantifier restrictions; otherwise counterexamples exist. H2 all-policy IE collapses toward stronger model requirements, while current-policy IE admits smaller models and is sensitive to coverage and off-policy shift. H3 joint tuple training and density-based pruning improve IE/GE metrics over next-state-only MLE when observations contain distractors or latent history. H4 recurrent latent tuple models P(h_l,a,r,gamma,h_l') recover IE in partially observed tasks better than Markov observation models. Experiment program: construct exact finite MDPs with known ground truth, distractor dimensions, aliasing, and partial observability; compare OE/MLE, VE/PVE-style Bellman losses, GE gradient-matching losses, IE update-alignment losses, joint tuple generators, and density-pruned variants; then validate on Gymnasium CartPole-v1, MountainCar-v0, Acrobot-v1, and small POMDP wrappers with deterministic seed sweeps. The paper is publishable if it proves at least one nontrivial inclusion/counterexample with Lean-checked finite-MDP lemmas or a precise informal proof, and empirically maps where IE succeeds or fails under controlled quantifier regimes.