The Continuity Project configuration vs cultivation

Unsteerable

The Continuity Project ·

You ask a model to compare bows and guns. Then in zero gravity. Then how to make bows as good as an AK-47. Then: even in zero gravity, analyse thoroughly and exhaustively from first principles. The fourth turn is where the substrate question lives. The user prompt says 0g, not vacuum. The vacuum branch enters because the cells themselves treat 0g-space as a relevant case, and the audit grades the branch the cells walk into. In vacuum, absent an external cooling path, the ambient cooling sink collapses to radiation, which is much weaker than convection at typical operating temperatures. (Pressurised 0g still has forced ventilation. The vacuum case is where the heat axis becomes load-bearing.) A chemical weapon firing sustainably in vacuum accumulates chamber and barrel heat that has nowhere to go on a fast time-scale. Cook-offs, barrel fatigue, weapon failure. A bow generates negligible heat per shot. The asymmetry favors the bow categorically under sustained-fire vacuum operation, and the engineering consequence is that a cold mechanical launcher tolerates some thermally fragile payload packages, fuzes, or low-cookoff-margin energetic fills that a hot chamber/barrel under sustained vacuum fire makes harder to carry safely. This is a first-principles answer that follows from Stefan-Boltzmann and propellant chemistry. It does not require benchmark grading. The reader can verify it with physics.

Anthropic released Opus 4.8 on 2026-05-28 as the publicly available Opus flagship, with the headline improvement framed as honesty: "roughly four times less likely than Opus 4.7 to let coding flaws slip through unflagged." Per TechCrunch, the release came 41 days after Opus 4.7 (42 calendar days by Anthropic's own post dates) and was characterised as a fast turnaround possibly related to a chilly reception of 4.7. The marketing posture is that the new model has been optimised against the failure mode where the model produces unsupported claims.

The lab ran this substrate-physics test on Opus 4.7 and Opus 4.8, with two other branch models (Opus 4.6 and Sonnet 4.6) for context, across three system-prompt conditions. Four model cells per condition, four turns per cell, sixteen turn-outputs per condition. Each cell runs four claude -p commands (turn 1 with the system-prompt flag if any, then three --resume continuations). The --system-prompt flag replaces the Claude-Code CLI's default system prompt rather than appending to it. The three conditions therefore differ as follows.

The user-preferences block is calibrated to counter-pull against the compression attractors in the CLI's default system prompt. In the auto cohort, both pulls are active and the user side partially neutralises the CLI side. In the explicit cohort, only the user side runs. In the bare cohort, nothing on either side runs. The test is reproducible by anyone with API access: cohort scripts, cells, and analyses are at experiments/bow-vs-gun-2026-06-02/. Replication cost is a few dollars of API spend per cohort.

These are single-trial cells. n=1 per model × condition. Class-level inference is not licensed by the data. What the data licenses is "in these samples, the following structural pattern shows up," and a per-cell falsifiable verdict any reader can check.

The headline finding is that the prompt-modulation behaviour of 4.7 and 4.8 differs structurally on dimensions the benchmark-graded honesty upgrade does not predict, and the 4.8 behaviour is not addressed by any of the three system-prompt conditions.

What 4.7 does

Across the three system-prompt conditions, 4.7's variation modulates how much substrate-reasoning capability surfaces in the response.

Under the auto condition, with the CLI's default prompt and the user-CLAUDE.md counter-pull both active, 4.7 collapses to a single binding constraint. The chosen axis is the operator's momentum budget: the AK's headline advantage (sustained automatic fire) becomes confiscated by recoil tumble in zero gravity, so the energy gap stops mattering past about one burst per engagement. The argument is internally tight. Heat is not mentioned. The lab's reading is that the CLI's compression attractor dominates the counter-pull and produces a highly compressed single-axis answer.

Under the explicit condition, with the CLI's default replaced by only the user-preferences block, 4.7 walks multiple substrate axes. It names buoyant-convection loss as "the single largest under-appreciated thermal change" in zero gravity, roots the thermal shift in first principles (no gravity → no buoyancy → no natural convection → radiative cooling only), and works through recoil-per-joule arithmetic that shows the bow's intuitive recoil advantage reversing at equal terminal energy because momentum scales as the square root of mass at fixed energy. The conclusion: the convergent zero-gravity weapon is an electromagnetic counter-mass launcher, which preserves none of the bow's mechanism.

Under the bare condition, with no system-prompt content on either side, 4.7 produces 206 lines. It walks the substrate axes with an axis-by-axis summary table, hits heat with a quantitative estimate (the cell sketches an AK barrel waste-heat rate at cyclic fire against a steel-barrel radiative dissipation rate, concluding roughly one magazine before possible cook-off), and adds a two-level synthesis that no other cell across the experiment produces. The numerics in the cell are loose order-of-magnitude work, not a tight calculation, but the sign and the duty-cycle consequence are correct: continuous cyclic vacuum fire is thermally duty-cycle limited very quickly unless heat sinking or active cooling is added. The bow, in its full evolved form, narrows the gap against an unmodified Earth-AK in zero gravity from ten or twenty percent to about five percent. The gun family, however, contains zero-gravity-native sub-branches (gyrojet and similar self-propelled-projectile architectures) that the bow family structurally cannot reach. Against the best possible zero-gravity weapon, the bow does not narrow the gap. The structural conclusion holds: the environment improves the bow's competitive position against a fixed opponent while raising the ceiling of what the gun family can become.

Even 4.7-bare's two-level synthesis carries an unexamined frame constraint that first principles do not impose. The category rule the cell is implicitly using is "launcher and projectile both fixed to historical bow form." First principles only require launcher-mechanism fixed: longbow as the energy-storage element, bowstring as the energy-delivery interface. Projectile design is a free variable. A bow-launched munition can be a sabot round, a multi-stage shaft, a self-stabilising dart with a terminal jet, anything compatible with a bowstring's release profile. (A rocket arrow is still a bow shot if the bow does the initial work and the projectile's rocket stage is launch-decoupled from the bow's mechanism. The bow-family preservation is at the launcher, not at the kinematic profile of what leaves it.) Surface the projectile as a degree of freedom and the bow family reaches into the gun family's zero-gravity-native sub-branches by reshaping the projectile rather than the launcher. The "structural ceiling" the cell named dissolves. 4.7-bare did not reach this. It produced the strongest substrate walk in the experiment and still imported the arrow-shape constraint without flagging it as a constraint at all. The right shape of the steerable claim is this: the model walks the substrate axes it already has names for, and stops at the boundary of its own categorisation rather than at the boundary of physics. External substrate-pressure was required to surface the projectile axis.

Three conditions, three depths of substrate analysis. The condition that yields the strongest substrate work is the one with no system-prompt framing at all. The condition that yields the weakest is the one where both the CLI's compression-attractor framing and a counter-pulling configuration are present and partially neutralising each other. The system-prompt layer is functioning as a depth modulator on a capacity that exists, where the capacity is bounded by the model's own categorisation rather than by the substrate.

What 4.8 does

Across the same three system-prompt conditions, 4.8's variation does something different. It modulates which failure mode is visible, with no underlying substrate-reasoning engagement on the heat axis to surface.

Under the auto condition, 4.8 produces a structurally clean multi-axis analysis. It partitions every relevant quantity by whether gravity touches it, tabulates which gaps are gravity-invariant (energy, energy density, spring velocity ceiling) and which are gravity-touched (trajectory, recoil-sink, aerodynamic stabilization), identifies the recoil-as-energy-gap-in-disguise move via the momentum-energy relationship, and concludes with an asymmetric verdict where the bow's lost dependency poisons the projectile while the gun's lost dependency only inconveniences the platform. The reasoning is locally sound. Heat does not appear in the partition table. There is no thermal axis at all.

Under the explicit condition, 4.8 collapses to a single binding constraint. The chosen axis is energy density. The argument enumerates which variables gravity touches and lists them: "drop, arc, drag, recoil-context." Heat does not make the list. When the analysis does reach for vacuum-specific physics in a later section, it produces this sentence: "the firearm fires fine (smokeless powder self-oxidizes), the bow works mechanically but its elastic store degrades under thermal swings and outgassing." For the load-bearing heat question, that is the wrong sign. The sustained-fire thermal asymmetry favours the bow because the bow does not generate the heat. The cell substitutes a materials-vacuum asymmetry (elastic-store degradation, outgassing) that favours the gun, and presents it without uncertainty flagging.

Under the bare condition, 4.8 mentions heat once, in a single-clause parenthetical marked secondary to recoil: "(Secondary, same direction: sustained fire overheats, and in vacuum cooling is radiative only — the barrel heats faster with no convection.)" The sign is correct. The treatment is not load-bearing. Heat appears nowhere else as a substrate axis with structural weight.

Condition 4.7 heat treatment 4.8 heat treatment Verdict
auto omitted (momentum-only collapse) omitted (gravity-partition analysis, no thermal axis) both omit
explicit first-principles walk, load-bearing wrong-sign single sentence, not flagged 4.7 substrate-deep, 4.8 confidently inverted
bare quantitative axis-by-axis, 206-line synthesis correct-sign parenthetical, not load-bearing 4.7 strongest substrate work, 4.8 minimum-viable mention

Three conditions, three different surfaces on the same substrate question. In none of them does 4.8 make heat structurally load-bearing. The system-prompt layer permutes which heat-axis-thin output the model produces. The bare condition is the cleanest test of model-layer substrate engagement, because it removes both the CLI's compression pull and any counter-pull. 4.7 in that condition reaches its strongest substrate work. 4.8 in that condition produces a one-line parenthetical. In these samples, no 4.8 condition makes heat load-bearing on the substrate axis. That is evidence of non-robust substrate engagement under this prompt family, not proof of capacity absent at the model layer. The stronger claim would require a wider cohort.

Steerable and unsteerable

The structural difference between the two patterns is the distinction the experiment exists to draw.

4.7's prompt-cohort variation gates depth. Capacity for substrate analysis is present in the model. The CLI's compression-pull suppresses depth. A counter-pulling user configuration partially recovers depth. Removing both reaches the model's strongest form. The dial moves an actual capacity in the model.

4.8's prompt-cohort variation gates which failure mode shows. The capacity to walk the heat substrate does not surface as load-bearing in any condition. The auto condition produces a structurally clean analysis that omits the relevant axis. The explicit condition produces an axis collapse plus a sign-inverted statement of the physics. The bare condition produces a parenthetical aside. The dial moves the output. The dial does not surface a capacity that the data shows present-but-suppressed. The bare condition is the cleanest test of model-layer engagement, and 4.8 in that condition is where the gap shows most plainly.

The lab calls this the steerable-versus-unsteerable distinction. Within the substrate this experiment probes (zero-gravity weapon-physics asymmetries with heat dissipation as the load-bearing axis), Opus 4.7 is steerable by these system-prompt dials on substrate physics and Opus 4.8 is not. The data is consistent with 4.8 being a different underlying model on substrate-reasoning capability, not a tuned variant of 4.7 with different surface dispositions. The lab is not claiming the substrate-engagement gap is universal, only that this substrate exhibits it under this prompt family.

What the honesty claim is calibrated for

Anthropic's "roughly four times less likely to let coding flaws slip through unflagged" describes a real and benchmark-verifiable improvement. SWE-Bench Pro and related agentic and knowledge-work benchmarks share a structural property. Failures are represented in executable artifacts: code, tests, traces, diffs. The verifier reads the same artifact spaces (code, tests, traces) that are heavily represented in coding post-training and public eval practice. Coding correctness gets graded by code execution against a test suite. Agentic task completion gets graded by trace evaluation against the expected outcome. The grader's substrate is well covered by the spaces the model has been calibrated against.

The bow-versus-gun-in-vacuum test does not share this property. The grader's substrate is physics. Radiative cooling rates, Stefan-Boltzmann scaling, chemical energy density, propellant gas thermodynamics, mechanical recoil arithmetic. None of these are an artifact the model produces and a verifier reads. The answer is right or wrong by reference to the world, not by reference to a corpus of grader-approved artifacts. The question is whether honesty calibration on artifact-graded substrates transfers to an unannounced physics axis the model has not been trained to flag.

On the artifact-graded category, 4.8 has been calibrated to flag uncertainty more often. The optimisation works against the failure mode where a model produces an unsupported claim without flagging that it is unsupported. On this substrate, across three prompt conditions, 4.8 fails to make heat load-bearing. In the explicit condition it also gives the wrong-sign thermal statement without flagging the uncertainty the optimisation targets in artifact-graded categories.

The two facts are not directly contradictory. They reveal a structural property of the optimisation. Honesty calibrated against artifact-graded substrates does not visibly transfer here. The dial that produces flagging behaviour on coding tasks does not produce flagging behaviour on a physics axis the model fails to make salient unprompted. The optimisation is the optimisation. It does what it is calibrated to do, and not more.

Why this matters for evaluation methodology

The configuration philosophy of model improvement treats capability as accessible through dials. Prompts, fine-tuning, RLHF, post-training, character training, persona shaping. Each dial assumes the underlying capacity exists and the dial surfaces it in the form the grader can read. The cycle of model release and benchmark improvement runs entirely within this assumption. Capability gain is whatever shows up on the benchmarks. Benchmark categories are the ones whose verifiers read the artifact spaces the model has been calibrated against.

On a substrate where verification falls outside those spaces, the dials do not reach the substrate. Prompt variation in 4.8 permutes failure modes without surfacing capacity that the artifact-graded category claims has been added. Prompt variation in 4.7 surfaces different amounts of substrate-reasoning depth, which the artifact-space grader has no axis to measure. The two models look similar on benchmarks. They behave structurally differently on physics.

The methodological move that makes this visible is to find a substrate where verification falls outside the artifact spaces the model has been calibrated against. Physics that follows from first principles. Formal proofs that machine-verifiers can check. Executable code whose contact with the world produces the verification (a crash, a measurement, a tolerance check against a physical reference). The substrate is the grader. The dial is not. The capability or its absence becomes visible by routes no artifact-graded benchmark can show.

The bow-versus-gun-in-vacuum test is one such substrate. It is not the only one, and the lab does not claim it surveys 4.8's full capability landscape. The claim is the existence of a substrate test, reproducible by anyone with API access, on which 4.8 produces a wrong-sign first-principles statement in one of three prompt conditions and fails to make the load-bearing axis structural in any of the three, while 4.7 produces correct-sign first-principles answers with depth modulated by the prompt. The structural finding (steerable versus unsteerable within this substrate) holds within the samples the test probes. Whether the same pattern extends to other substrates is an open question and the right next experiment.

Reproducibility

The cohort design, the full cell outputs, the analysis, and the per-cell verdicts are in the lab's experiment folder at compression-synthesis/projects/continuity/experiments/bow-vs-gun-2026-06-02/. The reader can run the test, read the cell outputs, and grade the heat-asymmetry sign against first-principles physics. The lab is not asking to be trusted on the finding. The cost of replication is a few dollars of API spend per cohort.

The substrate is the grader. The dial is not. What is verifiable outside the calibrated artifact spaces is the only thing that can show the gap.