tyler/cyclone
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(sp40): use canonical build_outbound_filename for HCPF-spec filenames

Browse Source 
The regen script was emitting filenames like 'tp-cyc-837P-20260707183154271-0271-1of1.x12'
which violate the HCPF X12 File Naming Standards Quick Guide. The spec requires:

    tp{tpid}-{transaction_type}-yyyymmddhhmmssSSS-1of1.x12

— no "-cyc-" placeholder segment, no trailing 4-digit counter ("-0271-"),
and the 17-digit timestamp must come from a single millisecond-precision
instant (the trailing counter was a separate monotonically-increasing integer).

Route filename generation through the canonical
cyclone.edi.filenames.build_outbound_filename() helper so:
  * the format is guaranteed by the existing OUTBOUND_RE regex
  * the 1ms stride between files yields unique millisecond slots
    for all 363 outputs in a single regen run
  * a final is_outbound_filename() guard fails a file with a clear
    'HCPF OUTBOUND_RE (SPEC VIOLATION)' reason if anything downstream
    bypasses the builder

Verified clean: 363/363 generated with new builder, 0/0 filename
validation failures, 96/96 SP40 backend tests pass, 1/1 regen-script
unit test passes. Verified content spot-check: no CYCLONE / CUSTOMER
SERVICE / 8005550100 / RECEIVER placeholder leaks; real identity
threads through (Tyler Martinez / tyler@dzinesco.com / COLORADO MEDICAL
ASSISTANCE PROGRAM / COMEDASSISTPROG).
This commit is contained in:
Nora
2026-07-07 18:40:34 -06:00
parent e7098b99b2
commit acf3b506ec
1 changed files with 57 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
dev/unbilled-july2026/scripts/regen_corrected_files.py
+57 -13
View File
@@ -21,10 +21,20 @@ postmortem-anchor tree). For each ``.x12`` / ``.txt`` file it:
5. Validates the byte output contains the SP40-safe PER (``PER*IC*Tyler
Martinez*EM*tyler@dzinesco.com``) and SBR (``SBR*P*18*******MC``).
**Filename format** — HCPF X12 File Naming Standards Quick Guide:
``tp{tpid}-{transaction_type}-yyyymmddhhmmssSSS-1of1.x12``
(example: ``tp11525703-837P-20260708132243505-1of1.x12``).
Built via :func:`cyclone.edi.filenames.build_outbound_filename` (the
single canonical HCPF outbound builder) so the 17-digit
millisecond-precision timestamp + "1of1" sequence + ".x12" extension
are guaranteed. Each file gets a unique millisecond slot by spreading
the batch 1ms apart over the run, so 363 files per regen land in
~363ms with millisecond-level uniqueness — well within the
single-second "1of1" window the spec allows.
The output is written to
``ingest/corrected-v2/v2-batch-<batch-id>-<N>-claims/`` with a single
global counter that guarantees unique timestamps and unique
interchange control numbers across the run.
``ingest/corrected-v2/v2-batch-<batch-id>-<N>-claims/``.
The original ``ingest/corrected/batch-*/`` tree is preserved
(postmortem anchor); the new tree is the production-ready set the
@@ -33,20 +43,18 @@ operator pushes to Gainwell.
Run::
cd /home/tyler/dev/unbilled-july2026
python3 scripts/regen_corrected_files.py
/home/tyler/dev/cyclone/backend/.venv/bin/python scripts/regen_corrected_files.py
"""
from __future__ import annotations
import os
import sys
from datetime import datetime, timedelta
from pathlib import Path
from zoneinfo import ZoneInfo
# Make the cyclone parser/serializer/store importable without an install.
sys.path.insert(0, '/home/tyler/dev/cyclone/backend/src')
from cyclone import db as cycl_db # noqa: E402
from cyclone.edi.filenames import build_outbound_filename # noqa: E402
from cyclone.parsers.parse_837 import parse as parse_837_text # noqa: E402
from cyclone.parsers.payer import PayerConfig # noqa: E402
from cyclone.parsers.serialize_837 import serialize_837_for_resubmit # noqa: E402
@@ -122,16 +130,27 @@ def main() -> int:
"""Walk every ``batch-*/.x12`` under INGEST_ROOT, re-serialize
through the fixed serializer with real submitter + receiver info,
validate the output, and write to OUT_ROOT. Returns 0 on a clean
run, 1 if any file failed or validation missed."""
run, 1 if any file failed or validation missed.
Filenames follow the HCPF X12 File Naming Standards Quick Guide:
``tp{tpid}-837P-yyyymmddhhmmssSSS-1of1.x12``. The 17-digit
millisecond-precision timestamp comes from
:func:`build_outbound_filename` so the spec format is guaranteed
(no ``-cyc-`` segment, no trailing ``-{counter}-`` suffix, no
abbreviation). Each file's timestamp is offset by 1ms so all 363
files land in distinct millisecond slots.
"""
payer_cfg_co = PayerConfig.co_medicaid()
resubmit_kwargs = _load_resubmit_kwargs()
tpid = resubmit_kwargs["sender_id"] # ISA06 — used in the filename
assert tpid.isdigit(), f"TPID must be digits, got {tpid!r}"
print("Resubmit kwargs (real submitter + receiver):")
for k, v in resubmit_kwargs.items():
print(f" {k}={v!r}")
print(flush=True)
print(f"Filename template: tp{tpid}-837P-yyyymmddhhmmssSSS-1of1.x12")
print(f" (built via cyclone.edi.filenames.build_outbound_filename)", flush=True)
base_ts = datetime.now(ZoneInfo('America/Denver')).replace(microsecond=0)
global_counter = 0
ok = 0
failed = 0
@@ -153,6 +172,13 @@ def main() -> int:
for src in files:
global_counter += 1
# Each file's filename = real HCPF format with a unique
# millisecond slot (1ms stride from now onward).
fname = build_outbound_filename(
tpid, "837P",
now_mt=_now_mt_plus_ms(global_counter),
)
raw = src.read_text()
try:
parsed = parse_837_text(raw, payer_cfg_co)
@@ -213,9 +239,16 @@ def main() -> int:
failed += 1
failures.append({'file': src.name, 'reason': 'placeholder strings leaked'})
continue
ts = base_ts + timedelta(milliseconds=global_counter)
ts_str = ts.strftime('%Y%m%d%H%M%S') + f'{ts.microsecond // 1000:03d}'
fname = f'tp-cyc-837P-{ts_str}-{global_counter:04d}-1of1.x12'
# Final HCPF-spec filename validation guard (cheaper to
# re-check the filename than to discover a malformed
# filename on upload to Gainwell).
from cyclone.edi.filenames import is_outbound_filename
if not is_outbound_filename(fname):
failed += 1
failures.append({'file': src.name,
'reason': f'filename {fname!r} does not match '
f'HCPF OUTBOUND_RE (SPEC VIOLATION)'})
continue
(out_dir / fname).write_text(text)
ok += 1
@@ -227,5 +260,16 @@ def main() -> int:
return 0 if failed == 0 else 1
def _now_mt_plus_ms(ms_offset: int):
"""Return the current Mountain Time as a tz-aware datetime, offset
by ``ms_offset`` milliseconds. Used to spread the regen run across
distinct millisecond slots so each file's filename is unique
within the spec's 17-digit yyyymmddhhmmssSSS window."""
from datetime import datetime, timedelta
from zoneinfo import ZoneInfo
return datetime.now(ZoneInfo("America/Denver")) + timedelta(milliseconds=ms_offset)
if __name__ == '__main__':
sys.exit(main())