gc-ir-reference

GC-IR Reference Implementation

Reproducibility artifact for “From Risk Register to Runtime Predicate: A Deterministic Method for Compiling AI Governance Assessments into Enforceable Controls”

Abhinandan Gill-Lakhowal, Member, IEEE — Gillian Holdings Incorporated, Calgary, AB, Canada

tests cross-platform determinism reproducibility

📊 View the results dashboard (live)

Rendered via GitHub Pages, deployed from this repository’s main branch root — the live page is served directly from the committed results.html with no separate copy. A single-page, chart-illustrated summary of every measured result below (case hashes, Q1-Q10, injections, determinism, Monte Carlo, L-DREA/ULB reproduction), plus a full campaign history table (every frozen release from preregister-tier0-v1 through the current v1.1.0) and a Paper 2 v1.2 development-phase section (Surface A, Case D, the 16-row validator matrix, 93-run local + cross-environment determinism, GD/GDmin Monte Carlo regression, compile timing) — explicitly marked as unfrozen, not-yet-reportable evidence, since no preregister-tier0-v5 tag exists yet. Regenerate it any time with make results / python tools/generate_results_page.py — it is built entirely from the same machine-readable evidence as this README, with no hand-edited numbers.

Current release: v1.1.0, reportable results in results/final_v4_1/. Executed after the public prospective scientific freeze preregister-tier0-v4.1, pushed to this repository before the campaign ran and verified from the remote (preregister-tier0-v4.1 corrects a manifest-generation provenance defect in the prior preregister-tier0-v4 tag; the two are scientifically byte-identical — see docs/V4_MANIFEST_DEFECT_REPORT.json). results/final_v3_1/, results/final_v3/, results/final_v2/, and results/final/ hold superseded campaigns, retained unedited for historical comparison, governed by the earlier preregister-tier0-v3.1/v3/v2.2/v1 freezes.

Current release: v1.1.0

Reportable campaign: results/final_v4_1/
Scientific freeze: preregister-tier0-v4.1
Freeze commit: efe4e165619ecb5d781fb795113f33720794a2e5
Manifest: 461 files, 32 roles, root 0640ad1e7a703879d4b9083a088f5b562b638173590292a2c1aed1625102e0e6

  Case A Case B v1.1
Bundle SHA-256 f5cbc3a8016159d2074005fa021bf76ca04fb7aed1408f5ec845418460a43536 0d8b602a4c8af888beb27058b7217893eff92d2f9df7f2944d35347c1031cfc1
Q1-Q10 10/10 10/10
Determinism (paired, 62 runs total) 1.000 1.000
Monte Carlo max FP_heat 0.321268 (risk R-06) 0.31798 (risk B-01)

Case B v1.1 structure: 6 risks, 9 Approved Control Specifications, 9 risk-derived predicates, 3 compiler invariants, 12 predicates total.

Negative fixtures: 19/19 detected (covering the ten canonical queries above; not ten additional queries).

Case B v1.1 injections (13 scenarios): authorization_decision {‘DENY’: 8, ‘HOLD’: 4, ‘PERMIT’: 1}; release_decision {‘ALLOW_RELEASE’: 12, ‘BLOCK_RELEASE’: 1} – release_decision is not a fourth peer authorization outcome.

Tests: 446 passed, 0 failed, 0 skipped.

ieee-check-v4: PASSED (3 stages). Freeze verification: PASS (45 checks, 0 failures).

L-DREA/ULB: original 284,807-row experiment (https://github.com/AGLakhowal/Gamma-Permit-Package.git @ 40fa8f046ad3) reproduced fresh, unmodified: 492 fraud-labelled, 0 false permits, 0 false denials. GC-IR predicate-family correspondence: 4 exact, 3 family, 5 not_established. No new v1.1 corpus replay was executed; no fraud-detection claim is made.

python tools/show_results_v1_1_0.py          # current release (this section)
python tools/show_results_v1_1_0.py --json
python tools/show_results.py                 # historical v1.0.5 / final_v2 campaign

This section is generated by python tools/update_readme_current_release.py; check it without modifying README with python tools/update_readme_current_release.py --check.

Measured results

Reportable campaign: results/final_v2/
Scientific freeze: preregister-tier0-v2.2
Freeze commit: c44f25d6fdb67e0bc4ac73a6217125dec8da1c0e
Artifact packaging: v1.1.0

Metric Case A Case B
Bundle SHA-256 f5cbc3a8016159d2074005fa021bf76ca04fb7aed1408f5ec845418460a43536 2850155a2ee7d4c01db4748a891891bae27c48f4c9c2c7eccd30beb7d1aa33ce
DC 1.000 1.000
RCY 0.8125 1.000
NDR 0.1875 0.000
OPR 0.000 0.000
ODC 1.000 1.000
PTC 1.000 1.000
CV 1.000 1.000
GD(15) 3 4
GD_min 3 0

Determinism: TD = 1.000; 62/62 executions passed (31 per case).

Monte Carlo: K = 250,000 per evaluated risk in Case A; K = 250,000 in Case B. Case A max FP_heat = 0.321268 (MCSE 0.000934); FP_C* = 0 in Case A and 0 in Case B. Observed Case A C* membership changes: 0.

Frozen campaign verification: 280/280 tests; adversarial corpus 62/62; structural checks 26/26; 16 property tests / 1,427 generated Hypothesis examples; 6 clean traceability queries per case, all empty; 18/18 negative controls detected.

Measured cross-platform evidence:

OS / environment Architecture Python TD Hashes
macOS / local host arm64 3.11.15 1.000 match
macOS / determinism-macos-latest-py3.11 arm64 3.11.9 1.000 match
macOS / determinism-macos-latest-py3.12 arm64 3.12.10 1.000 match
Linux / determinism-ubuntu-latest-py3.11 x86_64 3.11.16 1.000 match
Linux / determinism-ubuntu-latest-py3.12 x86_64 3.12.14 1.000 match
Windows / determinism-windows-latest-py3.11 AMD64 3.11.9 1.000 match
Windows / determinism-windows-latest-py3.12 AMD64 3.12.10 1.000 match
Linux / pinned container aarch64 3.11.11 1.000 match

These results establish properties of the reference governance-to-control compiler and its two pinned case artifacts. They do not establish production detection performance, legal compliance, or superiority over unaided practitioners. RQ5 remains deferred.

make results

Print the complete reportable result set directly from the machine-generated final_v2 evidence. After make install, use make results; alternatively run python tools/show_results.py or python tools/show_results.py --json.

This section is generated by make readme-results. Check it without modifying README with make verify-readme-results or python tools/update_readme_results.py --check.

Quick Start Commands

Get results in terminal in < 5 minutes:

# 1. Clone and setup (30 seconds)
git clone https://github.com/Sukhmangill977/gc-ir-reference.git
cd gc-ir-reference
make install

# 2. View results (1 second)
make results
python tools/show_results.py

# 3. Run verification (10 seconds)
make test                              # 320/320 tests
make verify-hashes                     # Both hashes match
python tools/freeze_check.py --final-v3

# 4. Full campaign (60 seconds)
python run_all.py --final-v3
cat results/final_v3/SUMMARY.md

Docker (if you have it):

docker build -t gc-ir .
docker run gc-ir make results
docker run gc-ir make verify-hashes

Specific operations:

# Compile Case A
python -c "from gcir.caseio import load_case; from gcir.compiler import compile_bundle; \
case = load_case('case_a'); result = compile_bundle(case.compiler_inputs()); \
print(f'Hash: {result.bundle.payload_hash}')"

# Compile Case B
python -c "from gcir.caseio import load_case; from gcir.compiler import compile_bundle; \
case = load_case('case_b'); result = compile_bundle(case.compiler_inputs()); \
print(f'Hash: {result.bundle.payload_hash}')"

# Run Q1-Q10 audit
python -c "from gcir.caseio import load_case; from gcir.compiler import compile_bundle; \
from gcir.audit_queries import run_all_audit_queries; \
case = load_case('case_b'); result = compile_bundle(case.compiler_inputs()); \
queries = run_all_audit_queries(result.bundle); \
for q, (passed, _) in queries.items(): print(f'{q}: PASS' if passed else f'{q}: FAIL')"

# Run 13-injection scenarios
python experiments/case_b_injection_scenarios.py

# Determinism check (62 runs)
python -m experiments.run_determinism --final-v3

5-minute verification

The shortest path from a fresh clone to the artifact’s central claims.

git clone https://github.com/Sukhmangill977/gc-ir-reference.git
cd gc-ir-reference

make install                  # .venv + the pinned dependency set (~30 s)
make results                  # machine-derived reportable results
make test                     # current scientific and packaging tests
make verify-hashes            # recompile both cases, check the hashes (~2 s)
python tools/freeze_check.py --final-v3        # the frozen metrics, recomputed
python tools/verify_reported_results.py        # every paper-facing number (~2 s)

make ieee-check runs that whole sequence plus the traceability and manifest checks, and returns non-zero if anything disagrees. See artifact_review/README.md for the reviewer-oriented artifact guide.

Campaign versus packaging tests. The measured-results section reports the frozen campaign test count. make test also runs subsequent reviewer-tooling tests; those additional tests do not change the scientific campaign.

make verify-hashes recompiles Case A and Case B from the committed governance artifacts and compares the SHA-256 of the RFC 8785 canonical bundle payload against the reference values committed in cases/*/expected/reference_hashes.json. If those match on your machine, the compiler is deterministic across your environment and the one that produced the reference.

Want the whole thing?

python run_all.py --final-v2   # all 13 steps end to end (~60 s)
cat results/final_v2/SUMMARY.md

Everything in that summary is read from a result file a script produced. No number in it is typed by hand.

What this is

Enterprise AI governance frameworks end with an approved risk assessment — a document. Deterministic runtime-enforcement architectures begin with an approved predicate set — code. This artifact implements the method the paper proposes for the step between them, and measures the properties the paper claims for it.

The research artifact is the method made executable:

Component What it is Where
Governance input model 𝒢 = (M, S, O, R, A) — the five-step assessment, machine-readable schemas/assessment.schema.json, src/gcir/models.py
Ψ_K support tooling Validates and assembles signed human selections. Infers nothing from prose. src/gcir/refinement.py, src/gcir/catalog.py
GC-IR The record family: compiled predicates, non-runtime / accepted / unresolved dispositions schemas/gcir.schema.json, src/gcir/models.py
Compiler Φ Deterministic compilation to a canonical, hashed, signed, immutable bundle src/gcir/compiler.py
RFC 8785 JSON Canonicalization Scheme, with ECMAScript number formatting and UTF-16 key ordering src/gcir/canonicalization.py
Authority closure Every action tuple resolves to S.authority_matrix src/gcir/authority.py
C* coverage The consequence-class gate coverage rule CV src/gcir/coverage.py
Lifecycle registry Retirement, supersession, revocation — signed, outside the payload src/gcir/lifecycle.py
Temporal traceability The origin chain and all six audit queries src/gcir/traceability.py, queries/traceability.sql
Metrics DC, RCY, NDR, OPR, ODC, PTC, CV, TD, GD, GD_min src/gcir/metrics.py
Case A Investment research agent — synthetic, 16 register rows cases/case_a/
Case B Transaction authorization agent — forensic reconstruction, 6 rows cases/case_b/

What this is not

Not a production system, not an AI agent, not a runtime enforcement engine, and not a user interface. The enforcement layer belongs to the cited prior work and is not restated here. This artifact stops at the compiled bundle.

Claim boundary

The paper’s own boundary governs. Applied to this artifact:

Claimed and measured here. Total disposition; deterministic compilation after approval; authority closure; C* coverage with the Proposition 1 and 2 premises instantiated on the pinned register; measured divergence from the declared heat-map comparator; origin-closed, temporally valid traceability; fail-closed unknown handling on mandatory gates; orphan-control impossibility within Φ-produced bundles.

Not claimed, and not measurable from anything in this repository.

docs/RESULT_INTERPRETATION.md states, metric by metric, what each number means and what it does not.

Reading the frozen documents. Several files under docs/ are part of the preregistration record and are covered by the freeze manifest. They still point at results/final/ (the v1 path) in places, and they have deliberately not been updated — rewriting a preregistered document after seeing the results would destroy the evidence the freeze exists to provide. docs/FROZEN_DOCS_READING_NOTE.md says how to read them; it is not frozen.

Architecture

   M, S, O, R, A                    approved governance assessment (Section III)
        │
        │   K  ── approved, versioned Control Derivation Catalog
        ▼
   ┌──────────────────────────────────────────────────────────┐
   │  Ψ_K   governed semantic refinement -- A RELATION        │
   │        closed into a mapping only by the signed          │
   │        judgment record J.  Human judgment happens HERE,  │
   │        once, and is signed.                              │
   └──────────────────────────────────────────────────────────┘
        │        (D, Δ)  ∈ Ψ_K(𝒢);  Ψ_K(𝒢, J) → (D, Δ)
        ▼
   ┌──────────────────────────────────────────────────────────┐
   │  Φ     deterministic compiler                            │
   │        exact catalog lookup · authority closure ·        │
   │        C* coverage · mandatory unknown-failure ·         │
   │        RFC 8785 · SHA-256 · no clock, no RNG, no LLM     │
   └──────────────────────────────────────────────────────────┘
        │
        ▼
   (P, G, E, B)   predicates · gate map · escalation map · signed immutable bundle
        │
        ├── lifecycle registry REG   (separately signed; NEVER in the payload)
        └── receipts                 ValidAt(t, b, REG) at decision time

The refinement boundary is the point of the design: human semantic judgment is exercised once, under an approved catalog, and recorded in a signed judgment record. Everything to the right of it is a function of signed inputs. Determinism is claimed for Φ, never for the interpretation of unconstrained prose.

tests/adversarial/test_no_inference.py asserts this statically over the shipped source: the compiler path imports no RNG, no similarity library, no model library, and reads no clock.

Project structure

gc-ir-reference/
├── src/gcir/              the implementation (16 modules, no framework)
├── schemas/               13 JSON Schema (draft 2020-12) documents
├── cases/case_a/          synthetic investment-research case
│   ├── inputs/            M, S, O, R, A + catalog + C* profile + INV + thresholds
│   ├── judgment/          the signed judgment record J
│   ├── dispositions/      Δ
│   ├── acs/               D -- the Approved Control Specifications
│   ├── lifecycle/         registry, receipts, actuations + negative/ fixtures
│   └── expected/          committed reference canonical payload hash
├── cases/case_b/          forensic-reconstruction transaction-authorization case
├── catalog/               the approved catalogs, published at the paths Appendix C names
├── invariants/            the approved invariant registers
├── queries/               traceability.sql -- the six audit queries
├── tests/                 unit · properties · adversarial · integration
├── experiments/           every measurement, plus reproduce_all
├── results/development/   exploratory results -- NOT reportable
├── results/final_v2/      ** THE REPORTABLE CAMPAIGN ** (freeze preregister-tier0-v2.2)
├── results/final/         the SUPERSEDED v1 campaign -- retained for provenance only
├── results/V1_V2_COMPARISON.md   every difference between the two, with its cause
├── artifact_review/       IEEE artifact-review package: reviewer README, the
│                          paper-to-result map, data inventory, self-review
├── preregistration/       TIER0_FREEZE_V2.md + FREEZE_MANIFEST_V2.sha256 (governing);
│                          TIER0_FREEZE.md + FREEZE_MANIFEST.sha256 (v1, historical);
│                          RQ5_DEFERRED_PROTOCOL.md, HELD_OUT_REGISTER.md
├── paper_update/          measured results, the placeholder replacement tables, and
│                          the manuscript-finalization audits
├── docs/                  requirements extraction, claim matrix, provenance, interpretation
├── tools/                 case-fixture generators, freeze_check, the result map and
│                          the reported-result verifier
└── keys/                  research signing keys -- TEST ONLY / NOT FOR PRODUCTION

Prerequisites

Python 3.11 or 3.12. No other runtime dependency. Four packages (jsonschema, referencing, cryptography, numpy) plus pytest and hypothesis for the test suites, all pinned in requirements.lock.

Local setup

make install                    # or, with an explicit interpreter:
make install PYTHON=python3.11

make install checks the interpreter first and stops with a clear message if it is older than 3.11 — without that guard, the first symptom is pip failing to resolve a transitive dependency, and that error never mentions the Python version.

Docker

make docker-build               # base image pinned by digest
make docker-reproduce           # docker run --rm gcir make reproduce

The container fixes LANG, LC_ALL, TZ and PYTHONHASHSEED so the baseline environment is declared rather than inherited — and the determinism experiment then deliberately varies all four, so the pinning is a stated starting point, not a way of avoiding the question.

Running things

make test                                    # all four suites
make ieee-check                              # the full reviewer check (~2 min)
.venv/bin/python -m pytest tests/properties -q --hypothesis-show-statistics

make reproduce                               # 11 steps -> results/development/
make reproduce-final-v2                      # the REPORTABLE campaign -> results/final_v2/
make reproduce-final                         # the SUPERSEDED v1 campaign -> results/final/

# individual experiments (add --final-v2 to write into results/final_v2/)
.venv/bin/python -m experiments.run_case --case case_a
.venv/bin/python -m experiments.run_determinism --runs-per-case 31
.venv/bin/python -m experiments.run_gate_divergence
.venv/bin/python -m experiments.run_monte_carlo --draws 250000
.venv/bin/python -m experiments.run_traceability
.venv/bin/python -m experiments.run_adversarial
.venv/bin/python -m experiments.run_metrics

make reproduce-final-v2 overwrites the reportable results. It exists so the frozen campaign can be re-executed and compared, but a reviewer verifying the published numbers should use python tools/verify_reported_results.py or make ieee-check, which read the committed evidence and write nothing into results/final_v2/.

Verifying hashes

make verify-hashes                           # against the committed references
make verify-manifest                         # the manifest is current
shasum -a 256 -c <(awk '!/^#/ && NF {print $1 "  " $3}' MANIFEST.sha256)

Every hash in MANIFEST.sha256 is computed from the file on disk by experiments/make_manifest.py. Each line carries the Appendix C role alongside the path.

make verify-manifest regenerates the manifest and diffs it against the committed one, excluding two rows that necessarily change: results/development/ (which make reproduce rewrites by design) and MANIFEST.json itself. MANIFEST.json embeds every file’s hash including its own from the previous generation, so it never reaches a fixpoint — a plain git diff --exit-code MANIFEST.sha256 after regenerating would therefore always report a difference and tell you nothing. The filtered comparison is what CI runs, and it is the meaningful check: every released path must match exactly.

Two manifests, two purposes — do not conflate them

File Purpose Changes when
MANIFEST.sha256 / .json Inventories the current repository / release state. 304 files, 30 roles. Any file is added or edited. Regenerate with make manifest.
preregistration/FREEZE_MANIFEST_V2.sha256 Proves the frozen scientific experiment: the 133 files covered by preregister-tier0-v2.2. Never. Regenerating it would destroy the evidence it exists to provide.

Inspecting Case A and Case B

# the register, its dispositions and its gates
python -c "import json;d=json.load(open('cases/case_a/inputs/assessment.json'));
[print(r['risk_id'], '|', r['event'][:70]) for r in d['risk_register']]"

jq '.records[] | {risk_id, status, reason_code}' cases/case_a/dispositions/dispositions.json
jq '.predicates[] | {gcir_id, gate_type, gate_source, mandatory_role}' results/development/case_a/gcir_records.json
jq '.c_star_coverage' results/development/case_a/coverage_matrix.json

The canonical bytes that were hashed are written verbatim to results/*/case_*/canonical_payload.json, so you can hash them yourself:

shasum -a 256 results/development/case_a/canonical_payload.json
cat results/development/case_a/payload_hash.txt

Result provenance

Every result file carries an environment block: git commit, git tag, whether the working tree was clean, Python version, platform, and every dependency version. phase says development or final. A number can always be traced back to the state that produced it.

Reproducibility and the two-phase workflow

results/development/ holds exploratory numbers from while the implementation was being built. They are not reportable, and they are committed rather than deleted so the development history the freeze separates from is visible.

results/final_v2/ holds the reportable campaign. Its freeze (preregistration/TIER0_FREEZE_V2.md, tag preregister-tier0-v2.2, commit c44f25d6fdb67e0bc4ac73a6217125dec8da1c0e) was pushed to this repository and verified from the remote before the campaign executed. Per Section XI-I: “The internal selection of frozen elements is not a freeze — the public timestamped commitment is.”

results/final/ holds the superseded v1 campaign. It is retained unedited, for provenance, and is not reportable: its freeze tag preregister-tier0-v1 existed only locally when that campaign ran, and was pushed afterwards. A later push cannot turn a completed experiment into a prospectively public preregistered one, so v1 was superseded rather than relabelled — its metadata was not edited and public_commitment_discharged was not flipped retroactively.

results/V1_V2_COMPARISON.md compares the two value by value. Every substantive measured value is identical across the two independently executed campaigns; what changed is the determinism run count (30 → 31 per case), the adversarial corpus size (59 → 62) and the addition of the Case B injection scenarios — all deliberate scope increases made before the v2 freeze.

Full detail: docs/REPRODUCIBILITY.md and docs/EXPERIMENT_PROTOCOL.md.

RQ5 remains deferred

The comparative expert study is preregistered and not executed. Its protocol is frozen in preregistration/RQ5_DEFERRED_PROTOCOL.md and hash-committed, so it is bound to be run as written rather than merely promised.

No adjudication panel has been convened. No practitioner has been recruited. No participant data exists. SNR and DF are reported as DEFERRED. No script in this repository generates, simulates or approximates human study data.

The held-out expert-study register is withheld until study close, and preregistration/HELD_OUT_REGISTER.md records that its hash commitment is openly outstanding rather than committing a hash of a placeholder.

Limitations

Beyond the claim boundary above, and beyond the manuscript’s Section XII:

Citing

See CITATION.cff. The artifact is released under MIT (LICENSE), matching the precedent named in the manuscript’s Appendix C.

Security

SECURITY.md. In short: this is a research artifact, the committed keys are test fixtures, and nothing here should be deployed.

This site is open source. Improve this page.