This phase did two things: real domain experts tested the built tool, and several important corrections were made during design and build. Both are headlined here; the subpages carry the detail.

What the usability test found

23 peers and operators from the institutional buy-side ran ReconLayer unmoderated on the real add-on between July 6–30; 19 completed the core flow (reconcile → draft, 83%) and 6 submitted the full survey. Headlines (full detail in Results):

• The human-in-the-loop guardrail was validated by users, unprompted — 5 of 6 survey completers wanted a human review step before sending the AI-drafted email, arriving at the tool's core principle on their own. (One would send as-is; the rest would edit, check, or not trust it blind.)
• The break labels are not self-evident — about a third misread "population" vs. "value" on first contact. The clearest, most actionable finding, and it leads the roadmap.
• Trust is high but conditional on verifiability — one participant moved from a 3 to a 5 after re-running the tool with a changed NPV to test the logic themselves, validating the deterministic, auditable engine.
• Fast and easy — median time-to-reconcile 5 seconds; ease averaged 4.83/5.

Honest scope: n = 6 survey completers (23 ran the tool) — directional and qualitative, not statistically significant.

Functional validation (the engine)

Separately from the user test, the engine's correctness was validated against engineered synthetic data. Each break type was planted deliberately — noise below tolerance, genuine value gaps, one-sided trades on each side — so the expected result (+$327,500 net, 4 value breaks, 2 population breaks) was known before the code ran. Reproducing it exactly confirmed the matching, tolerance filtering, variance ranking, and one-sided detection all behaved correctly. This is unit-level validation of the logic, distinct from the user study above.

The iteration log — corrections made during design and build

The most valuable iterations were domain corrections, caught by treating "good enough" as not good enough:

• Population breaks were missing. The first concept ranked matched-trade variances and silently dropped one-sided trades — omitting the break type that is often the largest real driver of a dispute. Fix: one-sided trades became a first-class, pinned output.
• The instrument lane was inconsistent. An early data model mixed OTC-derivative fields (trade ID, notional, NPV) with securities identifiers (ISIN/CUSIP) that don't apply to bilateral swaps. Fix: commit to the OTC variation-margin lane and drop the securities identifiers.
• The dispute email was operationally incomplete. An early draft just listed disputed lines. Real practice is to pay the undisputed amount and hold the disputed delta. Fix: the generated email states the agreed-vs-disputed split explicitly.
• The terminology collided with real contract terms. The noise filter was first called a "threshold" — a distinct, defined term in a CSA. Fix: renamed to the Minimum Transfer Amount.
• The platform trade-off was made explicit. Google Sheets was fast to build on but forfeits the local-data guarantee a native Excel build could offer. Stated honestly: the prototype runs on a personal Google account and the free Gemini API (synthetic data only); a production version would move to an enterprise tenant with a zero-retention endpoint — not yet built.

Next Steps — what the test drove

The results turned deferred work into a prioritized roadmap:

1. Relabel / clarify the break types — inline tooltips or clearer names (from the comprehension finding).