docs(sp40): RUNBOOK entry for Edifabric validation gate

This commit is contained in:
Nora
2026-07-07 18:19:58 -06:00
parent cfca1d8975
commit c3ef4f3847
+119
View File
@@ -323,6 +323,125 @@ curl -X PATCH http://127.0.0.1:8000/api/clearhouse \
To revert, set `stub: true` and `auth: {"method": "keychain", "secret_ref": "sftp.gainwell.password"}`.
### After SP40 lands — Edifabric /v2/x12/validate integration
SP40 wires Edifabric's reference X12 validator into the push flow
as a fail-closed pre-upload gate, plus a CLI / HTTP endpoint for
ad-hoc validation of single files. The integration catches the
same byte-level defects (`PER-04 required`, `SBR-09 required`,
etc.) Edifabric flagged on the SP39 regen `ingest/corrected-v2/`.
**One-time secret setup** (operator-only, never commit):
```bash
# Option A — macOS Keychain (preferred, paid tier key):
cyclone secrets set edifabric.api_key <paid-tier-key>
# → stores in Keychain under service='cyclone', account='edifabric.api_key'
# Option B — env var (CI / headless / per-process):
export CYCLONE_EDIFABRIC_API_KEY="<paid-tier-key>"
# Option C — file companion (matches the SP25 SFTP-password pattern):
export CYCLONE_EDIFABRIC_API_KEY_FILE=/path/to/edi-key.txt
# file should be chmod 600, single-line, no trailing newline needed
```
The dev-only `tests/fixtures/edifabric_api_key.txt` holds the
free `EdiNation Developer API` provisional key from the public docs
(`3ecf6b1c5cf34bd797a5f4c57951a1cf`); CI uses the mocked transport
so the key is never actually transmitted.
**Single-file validation — CLI**
```bash
# Status printout + OperationResult summary; exits 0 / 1 / 2.
.venv/bin/python -m cyclone.cli validate-837 ingest/corrected-v2/v2-batch-<id>-<N>-claims/tp-<id>-1of1.x12
```
Exit codes (matches the rest of the validator family):
- `0` — Edifabric returned `Status="success"` (warnings are non-blocking).
- `2` — `Status="error"`: at least one `Details` item has
`Status="error"`. The CLI prints the first 5 offending segment
IDs + messages, then exits.
- `1` — Edifabric client config error (no API key, or transport
exception). The operator must fix `cyc secrets set` before retrying.
**Single-file validation — HTTP (auth-gated)**
```bash
curl -b admin-cookies.txt -X POST \
-F "file=@ingest/corrected-v2/v2-batch-.../tp-...-1of1.x12" \
http://127.0.0.1:8000/api/admin/validate-837
# → 200 with the raw OperationResult JSON (Status + Details + LastIndex)
# Caller inspects Status to decide whether the file is acceptable.
```
`POST /api/admin/validate-837` is matrix-gated (ADMIN_ONLY); 401
without a session cookie, 403 with a viewer cookie.
**Pre-upload gate on resubmit-rejected-claims**
`cyclone resubmit-rejected-claims` runs Edifabric validation
before every SFTP `put`. Any file that returns `Status="error"`
is skipped (not uploaded, no `Resubmission` row inserted), and the
batch continues with the remaining files. After the run:
```bash
.venv/bin/python -m cyclone.cli resubmit-rejected-claims \
--ingest-dir /home/tyler/dev/cyclone/ingest/corrected-v2
# ...
# EDIFABRIC VALIDATION FAIL tp-cyc-deadbeef-1of1.x12: 2 error(s)
# [error] PER: PER-04 is required
# [error] SBR: SBR-09 is required
# ...
# DONE uploaded=361 skipped=0 failed=0 validated=363 validate_failed=2 ...
#
# 2 file(s) blocked by Edifabric gate:
# tp-cyc-aaaaaaaa-1of1.x12
# [error] PER: PER-04 is required
# tp-cyc-bbbbbbbb-1of1.x12
# [error] SBR: SBR-09 is required
```
To bypass the gate (dev only — for hot-fix scenarios where
Edifabric is the bottleneck):
```bash
.venv/bin/python -m cyclone.cli resubmit-rejected-claims \
--ingest-dir ... \
--skip-edifabric-validate
```
**Expected OperationResult shape** (verbatim from EdiNation docs):
```json
{
"Status": "success" | "warning" | "error",
"Details": [
{"Index": 5, "SegmentId": "PER", "Value": "...",
"Message": "PER-04 is required", "Status": "error"}
],
"LastIndex": 46
}
```
Warnings (`Status="warning"`) never block. Only `Details` items
with `Status="error"` fail the gate.
**Troubleshooting**
- `API key not configured` → run
`cyclone secrets set edifabric.api_key <key>` (or set
`CYCLONE_EDIFABRIC_API_KEY_FILE`).
- `502 Edifabric upstream failure` → check
https://status.edifabric.com/ (or the API status page); the
gate treats any 4xx/5xx from Edifabric as a hard fail for that
file but continues the batch.
- After fixing the byte defect, re-run
`cyclone resubmit-rejected-claims` for the blocked file —
idempotent on `(claim_id, interchange_control_number)`.
## Known historical drift — the 804 orphan 999s
The `acks` table may hold several hundred 999 acks whose source 837