The experiment kept the useful canonicalization mechanic and rejected the stronger claim that canonical order made the Agent understand the evidence better.
The first result looked like a context-engineering law
Earlier work had seen evidence-order sensitivity.
That is plausible. Language models consume sequences. Put the most important facts early and perhaps the Agent reasons better. Shuffle them and perhaps it misses dependencies.
Once you believe that, a product idea appears almost automatically:
Canonicalize the evidence before every model call.
Sort it. Normalize it. Build an identity map. Compile the Context into a stable order.
P11 tried to make that idea lose.
Mechanically, the case for canonicalization was excellent
The first test did not involve a model at all.
Six Finance datasets were explicitly owner-declared as an unordered bundle.
Six objects have:
6! = 720 permutationsAll 720 raw presentations had distinct byte digests.
Stable dataset-identity encoding collapsed them to one digest.
For hashing, cache identity, reproducibility, and replay, that is an unambiguous improvement.
But deterministic identity and semantic tractability are different objectives.
Global sorting immediately failed on ordered events
The same normalizer cannot be applied blindly to everything.
P11 also took five Effect events with explicit orderingDomain + sequence.
Reconstructing them by sequence identity preserved the semantics.
Naively sorting the same events lexically was semantically valid in:
0 / 120 permutationsSo there was already no such thing as a global “sort evidence” law.
Canonicalization needed owner-declared semantics before it could even claim mechanical correctness.
Then the first model campaign turned out not to be a model result
P11-v1 appeared to contain a large experiment: 144 cells across representations and orderings.
The Tool wire broke first.
The original 13-field function schema produced truncated DeepSeek arguments in 141 of 144 cells.
That campaign was invalidated wholesale.
It was not rescored as evidence about context order.
If the observation channel is broken, the Agent's apparent mistake is not yet evidence about the Agent.
The second campaign found an even more interesting confound
P11-v2 repaired the Tool representation.
A large order effect still appeared.
Then the evaluator was inspected.
The prompt asked for the “tracking-error leader.” The oracle meant “the numerically largest latest tracking_error_annualized_pct.”
Those are not necessarily the same semantic instruction to a fresh Agent.
The campaign was retained as a diagnostic, not promoted as proof of order sensitivity.
The next experiment rewrote the target explicitly before touching the representation again.
Once the question was clear, the ugly baseline won
P11-v3 froze an unambiguous semantic target and compared three representations.
The canonical representations were mechanically cleaner.
They were not semantically more accurate.
The raw baseline scored highest.
Why the beautiful abstraction was so tempting
Canonicalization has almost every property engineers like:
- deterministic bytes;
- stable hashes;
- fewer cache identities;
- clean diffs;
- reproducible serialization;
- easier equality checks.
Those are real benefits.
The mistake is letting one kind of order leak into another claim:
mechanically canonical
≠ semantically clearer
≠ easier for a model
≠ more correct decisionP11 retained the first property and rejected the automatic promotion to the others.
Ambiguity can masquerade as context sensitivity
The most useful result may not be the 44/48 score.
It is the sequence of failed explanations.
Agent appears order-sensitive
→ wire representation was broken
→ fix wire
→ order effect still appears
→ semantic target was ambiguous
→ fix evaluator
→ canonicalization no longer winsIf the team had stopped after either earlier campaign, it could have built a ContextCompiler to solve a measurement artifact.
The apparatus was deleted after the result
The round did not turn the experiment runner into permanent infrastructure.
The full executable/raw campaign remains Git-recoverable from an exact snapshot. The current Computing tree removed the apparatus and retained only the compact report, evidence, and contraction receipt.
The verification found zero current experiment executable-like source under the retained research tree.
This is the same burden-of-proof applied one level higher: a useful experiment does not automatically earn a permanent research service.
What survived
The final law is smaller than “order matters” and smaller than “normalize everything.”
Before attributing model variance to presentation order, first freeze an unambiguous semantic question and independent oracle. Use deterministic canonical representation where the owner declares order irrelevant, but do not treat canonical bytes as proof of better semantic representation.
No EvidenceNormalizer, OrderService, or ContextCompiler was promoted from P11.
What would reopen the question
A future workload could still earn order-aware machinery if an unambiguous prospective experiment repeatedly shows a material semantic benefit that survives Provider changes and cannot be recovered by simpler prompt or evaluator clarification.
A system becomes easier to trust when it can keep the canonical hash and delete the story it wanted that hash to prove.
Primary records