Testing Compatibility Across Solana Program Changes

Two compiled Solana implementations accepted the same call. Both transactions succeeded; only one preserved the caller’s fixed postcondition.

Compatibility here means preservation of an explicit caller-level postcondition. This is a differential execution test across two fresh LiteSVM instances, not an observed loader upgrade.

Compiled execution experiment

I sent byte-identical instruction data to two compiled Solana programs, each loaded at the same designated program address B in separate fresh LiteSVM instances.

R0   tx success   observed 3   expected 3
R1   tx success   observed 6   expected 3

Execution succeeded in both cases. Only one preserved the caller-level postcondition.

The arithmetic makes the requirement easy to inspect. Let x be the stored value and Δ the decoded instruction delta: the baseline computes x’ = x + Δ; the candidate computes x’ = x + 2Δ. The result remains a valid integer in a readable account, and execution completes. What changed is its agreement with the caller’s contract.

Execution validity and caller-level compatibility are different layers. A successful transaction answers whether this execution completed. For a dependent caller, the additional question is whether its requirement survived the implementation change.

What I held fixed

- data[..8].copy_from_slice(&(value + delta).to_le_bytes());
+ data[..8].copy_from_slice(&(value + 2 * delta).to_le_bytes());

Across the two runs I held the designated program address, state-account address, initial state bytes, instruction bytes, and account privileges fixed. Only the compiled behavior changed, from x + Δ to x + 2Δ. This fixed initialization isolates the arithmetic behavior; it does not test compatibility with a previously persisted state schema.

state:        00 00 00 00 00 00 00 00
instruction:  03 00 00 00 00 00 00 00

Experiment setup

LiteSVM 0.16.0, Rust/SBPF toolchain 1.95.0-sbpf-solana-v1.57. State is a signed little-endian i64, initial 0; the instruction is a signed little-endian i64, 3. The harness decodes the post-state produced by the compiled fixture and evaluates it against the fixed caller postcondition, only after successful execution:

expected_after = before + delta = 0 + 3 = 3

R0: observed 3 → PASS
R1: observed 6 → FAIL

The criterion is deliberately narrow. It covers the resulting counter value for these inputs, rather than every property of either implementation. In this fixture, the same integer encoding continues to decode while the operation applied to it changes — the part a success-only test leaves unexamined. This experiment establishes a controlled compatibility case; deployment frequency is outside its scope.

What compatibility means across time

The compatibility problem separates into four checks. Structural reachability asks whether the caller can still address the dependency through Program ID, accounts, and privileges. Interface continuity asks whether the existing instruction encoding and account expectations still describe what the caller intends to invoke — accepting an instruction says less than preserving its meaning. State compatibility asks whether changed code can interpret persistent account X, or whether migration is required; an address that still resolves says nothing about a layout that still parses. Semantic continuity asks whether the observed result still satisfies the caller-level postcondition that is supposed to survive the change.

Solana already separates several of these responsibilities. Program identity and replacement live with the loader. Invocation-time account and privilege constraints are enforced by the runtime, and CPI extends those constraints across program calls. IDLs and Program Metadata describe and publish interface information for tooling. Persistent application state, migration rules, and caller-visible postconditions remain application-defined.

Those mechanisms answer different questions. None of the descriptive or structural surfaces by itself establishes that an existing caller’s postcondition survived a candidate change.

Injecting failures at each layer

To make the four-layer distinction executable, I constructed representative fixture changes at each boundary and ran them under LiteSVM. Each case starts from a freshly initialized local VM with newly initialized state and the specified compiled fixture loaded under the designated program address. No case inherits post-state from another, and no case is classified by matching on its name. Each diagnosis is derived from the constructed account metas and instruction bytes, the LiteSVM transaction outcome or program error, and — when execution succeeds — the decoded post-state.

change                       tx status  layered diagnosis
-----------------------------------------------------------
readonly state account meta  FAIL       structural
9-byte instruction           FAIL       interface / wire
candidate rejects schema v1  FAIL       state
migrated schema v1 -> v2     PASS       compatible
semantic drift (+2 delta)    PASS       semantic break
benign internal refactor     PASS       compatible
benign guard reorder         PASS       compatible

The baseline passed all exercised checks. Three findings carry the remaining rows.

The first three rows all fail as transactions. A success-only check learns only FAIL for all three — it cannot distinguish a readonly account where the program requires writable from a 9-byte instruction where 8 are required from a state account carrying the wrong schema tag. The layered checks turn those three generic transaction failures into three different diagnoses.

The readonly-account case is invocation-level account-meta writability, not a CPI privilege-propagation experiment: the top-level instruction itself was constructed with the state account marked read-only while the program requires writable access. No call chain (A → CPI → B) or invoke/invoke_signed propagation was exercised. Likewise, in the mutation suite the interface layer is exercised narrowly, through instruction-byte acceptance — nine bytes supplied where the fixture accepts eight, an instruction wire-format rejection. That does not exhaust ABI, IDL, account-order, or semantic interface compatibility.

The semantic-drift row is the strongest result. The transaction succeeds, the structural and wire checks pass, and no state check blocks execution — but the observed value is 6 against an expected 3. This is the case transaction-success-only testing misses entirely: execution completes, but the preserved caller postcondition fails.

The two benign cases use separately compiled fixtures with source-level changes — an added intermediate variable and reordered guard clauses — and neither produces a compatibility break under the exercised checks.

The state case is deliberately explicit: the candidate fixture checks a schema tag in program logic. LiteSVM does not infer schema compatibility. The migrated-schema row rewrites that tag through a controlled pre-invocation migration helper, then reruns the candidate — a distinction demonstrated within the controlled fixture, not a general schema-inference mechanism.

What the layered approach buys — and costs

The layered checks distinguish several failure classes instead of returning one generic failed transaction, and they detect the exercised semantic drift after successful execution — the one case a success-only check cannot see at all. State migration becomes an explicit, testable transition rather than an assumption, and all of this runs against compiled candidates before anything reaches a deployment path. The benign compiled changes still pass, so the checks are not simply penalizing a different binary.

The costs sit on the other side of the same design. Semantic postconditions are application-specific: the harness knows before + delta because that predicate was written for this fixture, not discovered. State compatibility likewise requires explicit schema or invariant knowledge — a program that never checks a schema byte gives the harness nothing to observe at that layer. The fixtures behind each case must be maintained as a small corpus, and the harness checks the requirements it was given; it does not discover a caller’s requirements on its own. A local LiteSVM result is not mainnet or validator-network evidence — passing these cases does not establish compatibility for a caller this suite did not test.

The harness gains specificity by requiring builders to state what compatibility means.

What builders should test before changing a program they depend on

Start with the caller’s requirement, then decide which checks cover it:

• Does the same instruction still decode?
• Are the required accounts and privileges still valid?
• Can persistent state still be interpreted?
• Was required migration performed?
• Do caller-visible postconditions and invariants still hold?

The result block above is this checklist in practice: the first two questions are the exercised structural and interface/wire cases, the third and fourth are the schema-mismatch and migrated-schema cases, and the fifth is the semantic-drift case. The useful output is a set of answers, not one undifferentiated compatibility flag.

Each check belongs to a mechanism that can enforce or observe it: program logic for state it cannot interpret, interface and IDL tooling and SDKs for the instruction and account contract, migration tooling for the transition itself. In this article, compatibility tests are where observed behavior is compared against a requirement that should remain stable; the other mechanisms listed here do not establish that comparison by themselves.

Those requirements have to stay independent of the candidate’s implementation. The semantic-drift case shows what happens when a test would need to move its expected value from three to six just because the new code doubles delta — that stops checking the caller’s previous contract, and changing the requirement is a different decision from preserving compatibility for an existing caller.

A compatibility test is only as good as the requirements encoded into it. The harness does not discover those requirements; it makes them executable.

For a builder, the question is which structural, interface, state, and result assumptions must survive when evaluating a changed implementation of a program they depend on.

Both transactions succeeded. Only one still meant what the caller expected.