docs(sp40): RUNBOOK entry for Edifabric validation gate
This commit is contained in:
+119
@@ -323,6 +323,125 @@ curl -X PATCH http://127.0.0.1:8000/api/clearhouse \
|
||||
|
||||
To revert, set `stub: true` and `auth: {"method": "keychain", "secret_ref": "sftp.gainwell.password"}`.
|
||||
|
||||
### After SP40 lands — Edifabric /v2/x12/validate integration
|
||||
|
||||
SP40 wires Edifabric's reference X12 validator into the push flow
|
||||
as a fail-closed pre-upload gate, plus a CLI / HTTP endpoint for
|
||||
ad-hoc validation of single files. The integration catches the
|
||||
same byte-level defects (`PER-04 required`, `SBR-09 required`,
|
||||
etc.) Edifabric flagged on the SP39 regen `ingest/corrected-v2/`.
|
||||
|
||||
**One-time secret setup** (operator-only, never commit):
|
||||
|
||||
```bash
|
||||
# Option A — macOS Keychain (preferred, paid tier key):
|
||||
cyclone secrets set edifabric.api_key <paid-tier-key>
|
||||
# → stores in Keychain under service='cyclone', account='edifabric.api_key'
|
||||
|
||||
# Option B — env var (CI / headless / per-process):
|
||||
export CYCLONE_EDIFABRIC_API_KEY="<paid-tier-key>"
|
||||
|
||||
# Option C — file companion (matches the SP25 SFTP-password pattern):
|
||||
export CYCLONE_EDIFABRIC_API_KEY_FILE=/path/to/edi-key.txt
|
||||
# file should be chmod 600, single-line, no trailing newline needed
|
||||
```
|
||||
|
||||
The dev-only `tests/fixtures/edifabric_api_key.txt` holds the
|
||||
free `EdiNation Developer API` provisional key from the public docs
|
||||
(`3ecf6b1c5cf34bd797a5f4c57951a1cf`); CI uses the mocked transport
|
||||
so the key is never actually transmitted.
|
||||
|
||||
**Single-file validation — CLI**
|
||||
|
||||
```bash
|
||||
# Status printout + OperationResult summary; exits 0 / 1 / 2.
|
||||
.venv/bin/python -m cyclone.cli validate-837 ingest/corrected-v2/v2-batch-<id>-<N>-claims/tp-<id>-1of1.x12
|
||||
```
|
||||
|
||||
Exit codes (matches the rest of the validator family):
|
||||
|
||||
- `0` — Edifabric returned `Status="success"` (warnings are non-blocking).
|
||||
- `2` — `Status="error"`: at least one `Details` item has
|
||||
`Status="error"`. The CLI prints the first 5 offending segment
|
||||
IDs + messages, then exits.
|
||||
- `1` — Edifabric client config error (no API key, or transport
|
||||
exception). The operator must fix `cyc secrets set` before retrying.
|
||||
|
||||
**Single-file validation — HTTP (auth-gated)**
|
||||
|
||||
```bash
|
||||
curl -b admin-cookies.txt -X POST \
|
||||
-F "file=@ingest/corrected-v2/v2-batch-.../tp-...-1of1.x12" \
|
||||
http://127.0.0.1:8000/api/admin/validate-837
|
||||
# → 200 with the raw OperationResult JSON (Status + Details + LastIndex)
|
||||
# Caller inspects Status to decide whether the file is acceptable.
|
||||
```
|
||||
|
||||
`POST /api/admin/validate-837` is matrix-gated (ADMIN_ONLY); 401
|
||||
without a session cookie, 403 with a viewer cookie.
|
||||
|
||||
**Pre-upload gate on resubmit-rejected-claims**
|
||||
|
||||
`cyclone resubmit-rejected-claims` runs Edifabric validation
|
||||
before every SFTP `put`. Any file that returns `Status="error"`
|
||||
is skipped (not uploaded, no `Resubmission` row inserted), and the
|
||||
batch continues with the remaining files. After the run:
|
||||
|
||||
```bash
|
||||
.venv/bin/python -m cyclone.cli resubmit-rejected-claims \
|
||||
--ingest-dir /home/tyler/dev/cyclone/ingest/corrected-v2
|
||||
# ...
|
||||
# EDIFABRIC VALIDATION FAIL tp-cyc-deadbeef-1of1.x12: 2 error(s)
|
||||
# [error] PER: PER-04 is required
|
||||
# [error] SBR: SBR-09 is required
|
||||
# ...
|
||||
# DONE uploaded=361 skipped=0 failed=0 validated=363 validate_failed=2 ...
|
||||
#
|
||||
# 2 file(s) blocked by Edifabric gate:
|
||||
# tp-cyc-aaaaaaaa-1of1.x12
|
||||
# [error] PER: PER-04 is required
|
||||
# tp-cyc-bbbbbbbb-1of1.x12
|
||||
# [error] SBR: SBR-09 is required
|
||||
```
|
||||
|
||||
To bypass the gate (dev only — for hot-fix scenarios where
|
||||
Edifabric is the bottleneck):
|
||||
|
||||
```bash
|
||||
.venv/bin/python -m cyclone.cli resubmit-rejected-claims \
|
||||
--ingest-dir ... \
|
||||
--skip-edifabric-validate
|
||||
```
|
||||
|
||||
**Expected OperationResult shape** (verbatim from EdiNation docs):
|
||||
|
||||
```json
|
||||
{
|
||||
"Status": "success" | "warning" | "error",
|
||||
"Details": [
|
||||
{"Index": 5, "SegmentId": "PER", "Value": "...",
|
||||
"Message": "PER-04 is required", "Status": "error"}
|
||||
],
|
||||
"LastIndex": 46
|
||||
}
|
||||
```
|
||||
|
||||
Warnings (`Status="warning"`) never block. Only `Details` items
|
||||
with `Status="error"` fail the gate.
|
||||
|
||||
**Troubleshooting**
|
||||
|
||||
- `API key not configured` → run
|
||||
`cyclone secrets set edifabric.api_key <key>` (or set
|
||||
`CYCLONE_EDIFABRIC_API_KEY_FILE`).
|
||||
- `502 Edifabric upstream failure` → check
|
||||
https://status.edifabric.com/ (or the API status page); the
|
||||
gate treats any 4xx/5xx from Edifabric as a hard fail for that
|
||||
file but continues the batch.
|
||||
- After fixing the byte defect, re-run
|
||||
`cyclone resubmit-rejected-claims` for the blocked file —
|
||||
idempotent on `(claim_id, interchange_control_number)`.
|
||||
|
||||
## Known historical drift — the 804 orphan 999s
|
||||
|
||||
The `acks` table may hold several hundred 999 acks whose source 837
|
||||
|
||||
Reference in New Issue
Block a user