--- license: mit tags: - partial-differential-equations - scientific-machine-learning - neural-operators - partial-observations - benchmark --- # PDE-OBS: trained baseline checkpoints Final checkpoints of the PDE-OBS campaign: one model per (PDE family, method, training observation pattern). Each checkpoint is the final recorded checkpoint of its attempt, not one selected on validation or test error. **All 441 credited settings are published here.** `models_manifest..json` lists the SHA-256 of every published file. The physical records these models were trained and scored on are in the companion dataset [`ru1ch3n/PDE-OBS`](https://huggingface.co/datasets/ru1ch3n/PDE-OBS). ## Layout ``` models///// checkpoints/last.pt final checkpoint: weights, scheduler state, history, resolved config checkpoints/training_config.json resolved trainer configuration identity.json provenance.json resolved.yaml split_manifest.json factor_coverage.json completion.json health.json history.json (where the attempt wrote them) budget-protocol.json | dynamic-protocol.json (cohort dependent) models_manifest..json per-file SHA-256 of the published files (repository root) scrub-manifest.json which record files the last de-identification pass rewrote ``` The cluster labels (`cluster-A`, `cluster-B`, `cluster-C`) stand for the three machines the campaign ran on; they carry no institutional meaning. `resolved.yaml` beside each checkpoint records the architecture (`method.name`, `method.kwargs`), the training view and the training settings. ## Scoring a checkpoint with the benchmark ```python from pdeobs import api data = api.load_dataset("./pdeobs-data", verify=True) # from the dataset ru1ch3n/PDE-OBS package, targets = api.inference_input_from_dataset(data, api.make_observation("paper:R50"), task="recovery") pred = api.load_legacy_checkpoint("models/cluster-A/poisson/fno/random_50pct/checkpoints/last.pt", model={"name": "fno", "preset": "paper"}, task="recovery") score = api.evaluate(api.predict(pred, package), targets) score["status"], score["scoring_version"], score["summary"]["rel_l2_joint_mean"] ``` The loader takes the structure from the caller (the campaign checkpoints use the `paper` preset of their method), cross-checks the checkpoint's own stored training configuration, and records the file's SHA-256 and epoch in the predictor's provenance. Without the benchmark, the file is a plain PyTorch payload: `torch.load(path, map_location="cpu", weights_only=False)` returns a dict with `model_state`, `config`, `history`, `epoch` and `scheduler_state`. ## Important properties - **Weights only.** Optimizer, gradient-scaler and RNG states were removed (about two thirds of each file); `dropped_keys` in the manifest records this per checkpoint. The checkpoints support inference and rescoring, not exact continuation of training. - **De-identified metadata, unmodified parameters.** Every string naming a filesystem path, account, node, host, login node, scheduler job, partition, GPU hardware identifier, interpreter or environment directory, run directory or source-code revision was rewritten to a placeholder such as ``, ``, ``, ``, ``, ``, `` or ``, and raw scheduler records were reduced to their resource fields (` ...`); three passes, the files rewritten by the last one are listed in `scrub-manifest.json`. Model parameters were not modified. `models_manifest.*.json` records the SHA-256 of each released file; the correspondence to the campaign's checkpoint identities is kept with the benchmark code. - **Training cohorts are not interchangeable.** Each entry carries its cohort label: the original 500-epoch protocol, a recovered continuation of it, a declared 200-epoch budget, a budgeted max-200 patience protocol, or a salvaged interrupted attempt. Any table that pools cohorts must say so per row. - **Scoring provenance.** All 441 checkpoints were later rescored from retained prediction arrays with the benchmark's strict scorer (`pdeobs-strict-v1`, nine views x 200 held-out identities each); those results and their per-identity errors accompany the benchmark code (`results/prediction_verification_20260924/`). The campaign's own evaluator metrics remain in the archived-results index and are not the paper's current numbers. ## October 2026 revision models (`october2026/`) The trained models behind the October 2026 revision results of the benchmark code (`results/revision_october/`, frozen recipes in `configs/paper/october/recipes.yaml`): one entry for every trained run that has scored test views in those tables, 663 runs in the cohorts P4, P5, P6, P7, V4C, V4D, V4E, V4P6, V4P7, N3GJ2, N5MXR6, N6, N8, RPB, RPBn and E6 (the 12 E6 runs, nearest-fill copies of zero-fill FNO runs, were added on 2026-10-21 as upload part 3). Runs that failed during training (they are listed with their failure in the result tables) and the recipe-screen runs have no model here. The training and test data are the V2 views in the companion dataset, under `october2026/`. ``` october2026/models/// weights.pt {"format": "pdeobs-october2026-weights/1", "run_id": ..., "model_state": } model.json cohort, PDE, training view, method, protocol, seed, grid, epochs, final training loss, learning rate, number of training records, recipe digest, original and released SHA-256 history.json per-epoch training history (numeric fields only) october2026/models_manifest.part-.json the model.json records of each upload part ``` `` is `________[__g]__s`, the run identifier used in the result tables. - **Weights only.** `model_state` is the state dict of the run's final checkpoint, unchanged; where a run used the training-set normaliser, it is part of the state dict (buffers `v4_mu` / `v4_sigma`). Optimizer, scheduler, RNG state and the resolved run configuration (which held machine paths) are not included; `original_checkpoint_sha256` in `model.json` is the digest of the full checkpoint the campaign recorded at the end of training, checked against the file before the weights were extracted. `torch.load(path, map_location="cpu", weights_only=True)["model_state"]` loads it. - **Rescoring.** For the cohorts with a frozen recipe in `configs/paper/october/recipes.yaml` (P4, P5, V4C, V4D, N3GJ2, N5MXR6, N6, RPB, RPBn, E6), `reproduce/october/replay_inference.py --run --checkpoint weights.pt --data-root /october2026/v2` rebuilds the network, re-predicts the run's test views and compares each view mean with the stored score (relative tolerance 1e-4; recorded checks on CUDA and CPU are in `reproduce/october/README.md`; an E6 run, Poisson rough G256 seed 1101, replayed on a CPU matches its stored scores to 2e-7). V4E is listed in `recipes.yaml` as four information cells that the replay script does not rebuild; P6, P7, V4P6, V4P7 and N8 have no entry (they reuse the P4 or V4C recipe of their slice with one change: training-set size, noise, input fill, mixture or grid). The FNOs were trained and scored with cuFFT; on a CPU the script reproduces cuFFT's large-batch inverse real FFT (`pdeobs.revision.fftcompat`). - **Checked after upload.** Every `weights.pt` on the Hub has the SHA-256 listed in its manifest part (checked without credentials). - **RPB / RPBn** models were trained on the Poisson part of the CNO authors' Representative PDE Benchmarks (Raonic et al.; Zenodo record 10406879, CC BY 4.0). That data is not redistributed here; to rescore these models, download it from Zenodo and convert it with `python -m pdeobs.revision.external.rpb`. ## Path D retrains (`october2026/retrains/`) Ten retrains of Poisson medium U256 runs from the public package with `reproduce/october/train_run.py` of the benchmark code (reproduction path D, 2026-10-18 to 2026-10-21): P4 FNO seed 1102, P4 Voronoi-CNN seeds 1101-1103, V4C FNO seeds 1101-1103 and P4n FNO seeds 1101-1103. P4n is the P4 recipe with `--set v4_norm=train_scalar`: the training-set normaliser is its only change. The P4 FNO retrains of seeds 1101 and 1103 collapsed to a constant output, as the original runs did, and have no model. ``` october2026/retrains// weights.pt same format as the October models model.json recipe run, --set overrides, final training loss, normaliser, the seven v2eval7 view means of the retrain, original and released SHA-256 history.json per-epoch training history (numeric fields only) october2026/retrains/manifest.json the model.json records, and the two retrains without a model ``` These are retrains, not the campaign's models. Retraining is not bit-identical, so their scores differ from the stored scores of the original runs; results and discussion are in `reproduce/october/README.md` of the benchmark code. Rescore a retrain with its recipe run and the same `--set` (on a CPU, the P4n seed 1101 replay reproduces the view means in its `model.json` to 9e-7): ```bash python reproduce/october/replay_inference.py --run P4__poisson__ell009__fno__U256__s1101 --set v4_norm=train_scalar \ --checkpoint october2026/retrains/P4n__poisson__ell009__fno__U256__s1101/weights.pt --data-root /october2026/v2 ``` ## Citation Ruichen Xu, Siyao Wang, Fang Wan, Jiacheng Qiu, Wenhan Gao, Jiaxing Zhang, Linsey Pang, Ravid Shwartz-Ziv, Yann LeCun and Yuefan Deng. *PDE-OBS: Controlled Evaluation Across Observation Patterns.* arXiv preprint arXiv:2609.36521, 2026. ```bibtex @article{xu2026pdeobs, title = {{PDE-OBS}: Controlled Evaluation Across Observation Patterns}, author = {Xu, Ruichen and Wang, Siyao and Wan, Fang and Qiu, Jiacheng and Gao, Wenhan and Zhang, Jiaxing and Pang, Linsey and Shwartz-Ziv, Ravid and LeCun, Yann and Deng, Yuefan}, journal = {arXiv preprint arXiv:2609.36521}, year = {2026} } ```