tyler/cyclone
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(sftp): swap inbound/outbound paths — FromHPE is inbound, ToHPE is outbound

Browse Source 
The dzinesco SP9 seed had `paths.inbound` and `paths.outbound` mapped to
the wrong Gainwell MFT directories:

  - paths.outbound  was  FromHPE/   (HPE sends files FROM here TO us — inbound)
  - paths.inbound   was  ToHPE/     (we send files TO here — outbound)

So `/api/clearhouse/submit` was writing 837P claims to FromHPE (where
HPE puts acks/835s) and the SP16 scheduler was polling ToHPE (where our
claims go). 999 / TA1 / 835 files in the real FromHPE inbox were
unreachable.

Semantics per operator (2026-06-24):
  - FromHPE = HPE/Gainwell → us  = 999, TA1, 835  (inbound)
  - ToHPE   = us → HPE/Gainwell = 837P claims    (outbound)

**Runtime code (the actual fix):**
  - backend/src/cyclone/store.py            — SP9 seed paths flipped
  - backend/src/cyclone/edi/filenames.py    — docstring corrected

**Test fixtures + assertions (would otherwise fail on the new seed):**
  - backend/tests/test_clearhouse_api.py
  - backend/tests/test_providers_seed.py
  - backend/tests/test_sftp_stub.py        — incl. inbound dir paths
  - backend/tests/test_sftp_paramiko.py
  - backend/tests/test_store_update_clearhouse.py
  - backend/tests/test_api_clearhouse_patch.py
  - backend/tests/test_scheduler.py
  - backend/tests/test_api_scheduler.py

**Docs (text-only — keeps the codebase self-consistent):**
  - README.md
  - docs/reference/co-medicaid.md
  - docs/superpowers/specs/2026-06-20-cyclone-multi-payer-npi-sftp-design.md

**Operator action required after merge:** the existing clearhouse row in
`~/.local/share/cyclone/cyclone.db` was seeded with the old wrong
paths. Easiest recovery:
  1. `rm ~/.local/share/cyclone/cyclone.db` and let `ensure_clearhouse_seeded` re-run on next boot, OR
  2. PATCH /api/clearhouse with the new `paths` block (the SP25
     reconfigure hook picks it up live, including by the running scheduler).

**Verification:**
  - 82 tests in the affected files pass (test_clearhouse_api, test_providers_seed,
    test_sftp_stub, test_sftp_paramiko, test_store_update_clearhouse,
    test_api_clearhouse_patch, test_scheduler, test_api_scheduler, test_filenames).
  - Full backend suite: 1029 pass + 36 pre-existing order-dependent flakes
    unrelated to this change (verified by running the same tests in isolation).
This commit is contained in:
Nora
2026-06-24 22:05:55 -06:00
parent 9cb0311544
commit dd7da18279
13 changed files with 42 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+3 -3
View File
@@ -690,7 +690,7 @@ backup create` manually retains full control.
The `clearhouse.submit` endpoint uses `paramiko` to push a batch of
generated 837 files to the dzinesco SFTP server
(`mft.gainwelltechnologies.com:22`, path
`/CO XIX/PROD/coxix_prod_11525703/FromHPE`). The SFTP credential is
`/CO XIX/PROD/coxix_prod_11525703/ToHPE`). The SFTP credential is
fetched from the macOS Keychain at call time — never read from YAML,
never logged, never written to disk. The wire-up honors the file-naming
template stored in the `clearhouse` config:
@@ -898,7 +898,7 @@ Shipped sub-projects (most recent first):
- **Sub-project 13 (shipped) — SFTP wire-up.** `paramiko`-backed
`SftpClient` replaces the SP9 stub. The clearhouse.submit endpoint
actually pushes to
`mft.gainwelltechnologies.com:/CO XIX/PROD/coxix_prod_11525703/FromHPE`.
`mft.gainwelltechnologies.com:/CO XIX/PROD/coxix_prod_11525703/ToHPE`.
SFTP credentials are read from the macOS Keychain at call time.
- **Sub-project 12 (shipped) — Encryption at rest.** Optional
SQLCipher AES-256 encryption of the SQLite file, with the key
@@ -1160,7 +1160,7 @@ the one-time setup recipe.
- `POST /api/clearhouse/submit` — same endpoint as SP9; the
implementation is now a real `paramiko` `SftpClient.write` to
`mft.gainwelltechnologies.com:/CO XIX/PROD/coxix_prod_11525703/FromHPE`.
`mft.gainwelltechnologies.com:/CO XIX/PROD/coxix_prod_11525703/ToHPE`.
SFTP credentials are fetched from the macOS Keychain at call time.
## License
backend/src/cyclone/edi/filenames.py
+1 -1
View File
@@ -7,7 +7,7 @@ Outbound (we send):
tp{tpid}-{transaction_type}-{yyyymmddhhmmssSSS_MT}-1of1.{ext}
Example: tp11525703-837P-20260620132243505-1of1.x12
Inbound (HPE sends to our ToHPE):
Inbound (HPE sends to our FromHPE):
TP{tpid}-{orig_tx}_M{tracking}-{ts}-1of1_{file_type}.x12
Example: TP11525703-837P_M019048402-20260520231513488-1of1_999.x12
backend/src/cyclone/store.py
+2 -2
View File
@@ -2370,8 +2370,8 @@ class CycloneStore:
"port": 22,
"username": "colorado-fts\\coxix_prod_11525703",
"paths": {
"outbound": "/CO XIX/PROD/coxix_prod_11525703/FromHPE",
"inbound": "/CO XIX/PROD/coxix_prod_11525703/ToHPE",
"outbound": "/CO XIX/PROD/coxix_prod_11525703/ToHPE",
"inbound": "/CO XIX/PROD/coxix_prod_11525703/FromHPE",
},
"stub": True,
"staging_dir": "./var/sftp/staging",
backend/tests/test_api_clearhouse_patch.py
+2 -2
View File
@@ -121,8 +121,8 @@ def test_patch_without_session_returns_401(client):
"port": 22,
"username": "colorado-fts\\coxix_prod_11525703",
"paths": {
"outbound": "/CO XIX/PROD/coxix_prod_11525703/FromHPE",
"inbound": "/CO XIX/PROD/coxix_prod_11525703/ToHPE",
"outbound": "/CO XIX/PROD/coxix_prod_11525703/ToHPE",
"inbound": "/CO XIX/PROD/coxix_prod_11525703/FromHPE",
},
"stub": overrides.get("stub", False),
"staging_dir": "./var/sftp/staging",
backend/tests/test_api_scheduler.py
+3 -3
View File
@@ -34,13 +34,13 @@ def _stub_scheduler_env(tmp_path, monkeypatch):
db._reset_for_tests()
staging = tmp_path / "staging"
inbound = staging / "ToHPE"
inbound = staging / "FromHPE"
inbound.mkdir(parents=True)
sftp_block = SftpBlock(
host="mft.example.com",
port=22,
username="test",
paths={"outbound": "/FromHPE", "inbound": "/ToHPE"},
paths={"outbound": "/ToHPE", "inbound": "/FromHPE"},
stub=True,
staging_dir=str(staging),
poll_seconds=60,
@@ -54,7 +54,7 @@ def _stub_scheduler_env(tmp_path, monkeypatch):
def _drop_file(staging: Path, name: str, body: bytes) -> Path:
p = staging / "ToHPE" / name
p = staging / "FromHPE" / name
p.write_bytes(body)
return p
backend/tests/test_clearhouse_api.py
+2 -2
View File
@@ -30,8 +30,8 @@ def test_get_clearhouse_seeded(client):
assert body["name"] == "dzinesco"
assert body["tpid"] == "11525703"
assert body["sftp_block"]["stub"] is True
assert "FromHPE" in body["sftp_block"]["paths"]["outbound"]
assert "ToHPE" in body["sftp_block"]["paths"]["inbound"]
assert "ToHPE" in body["sftp_block"]["paths"]["outbound"]
assert "FromHPE" in body["sftp_block"]["paths"]["inbound"]
def test_list_providers(client):
backend/tests/test_providers_seed.py
+2 -2
View File
@@ -54,8 +54,8 @@ def test_seed_creates_clearhouse_singleton():
assert ch.submitter_name == "Dzinesco"
assert ch.sftp_block.host == "mft.gainwelltechnologies.com"
assert ch.sftp_block.stub is True
assert "FromHPE" in ch.sftp_block.paths["outbound"]
assert "ToHPE" in ch.sftp_block.paths["inbound"]
assert "ToHPE" in ch.sftp_block.paths["outbound"]
assert "FromHPE" in ch.sftp_block.paths["inbound"]
def test_seed_creates_co_txix_payer_with_both_configs():
backend/tests/test_scheduler.py
+4 -4
View File
@@ -35,15 +35,15 @@ from cyclone.scheduler import (
@pytest.fixture
def sftp_block(tmp_path):
staging = tmp_path / "staging"
inbound_dir = staging / "ToHPE"
inbound_dir = staging / "FromHPE"
inbound_dir.mkdir(parents=True)
return SftpBlock(
host="mft.example.com",
port=22,
username="test",
paths={
"outbound": "/FromHPE",
"inbound": "/ToHPE",
"outbound": "/ToHPE",
"inbound": "/FromHPE",
},
stub=True,
staging_dir=str(staging),
@@ -55,7 +55,7 @@ def sftp_block(tmp_path):
@pytest.fixture
def _drop_file(sftp_block):
"""Helper: drop a named file in the inbound dir. Returns the path."""
inbound_dir = Path(sftp_block.staging_dir) / "ToHPE"
inbound_dir = Path(sftp_block.staging_dir) / "FromHPE"
def _drop(name: str, body: bytes) -> Path:
inbound_dir.mkdir(parents=True, exist_ok=True)
backend/tests/test_sftp_paramiko.py
+2 -2
View File
@@ -40,8 +40,8 @@ def _block(
port=22,
username="testuser",
paths={
"outbound": "/CO XIX/PROD/test/FromHPE",
"inbound": "/CO XIX/PROD/test/ToHPE",
"outbound": "/CO XIX/PROD/test/ToHPE",
"inbound": "/CO XIX/PROD/test/FromHPE",
},
stub=stub,
staging_dir=staging_dir,
backend/tests/test_sftp_stub.py
+6 -6
View File
@@ -19,8 +19,8 @@ def sftp_block(tmp_path):
port=22,
username="colorado-fts\\coxix_prod_11525703",
paths={
"outbound": "/CO XIX/PROD/coxix_prod_11525703/FromHPE",
"inbound": "/CO XIX/PROD/coxix_prod_11525703/ToHPE",
"outbound": "/CO XIX/PROD/coxix_prod_11525703/ToHPE",
"inbound": "/CO XIX/PROD/coxix_prod_11525703/FromHPE",
},
stub=True,
staging_dir=str(staging),
@@ -55,7 +55,7 @@ def test_stub_list_inbound_empty_when_no_local_files(sftp_block):
def test_stub_list_inbound_returns_local_files(sftp_block):
# Simulate operator dropping a file in the inbound staging dir
inbound_dir = Path(sftp_block.staging_dir) / "CO XIX/PROD/coxix_prod_11525703/ToHPE"
inbound_dir = Path(sftp_block.staging_dir) / "CO XIX/PROD/coxix_prod_11525703/FromHPE"
inbound_dir.mkdir(parents=True, exist_ok=True)
(inbound_dir / "TP11525703-837P_M019048402-20260520231513488-1of1_999.x12").write_bytes(b"X")
client = SftpClient(sftp_block)
@@ -78,17 +78,17 @@ def test_stub_read_file_returns_bytes(sftp_block, tmp_path):
Lets the inbound scheduler exercise the same code path on a
workstation without a real MFT connection.
"""
inbound = tmp_path / "staging" / "ToHPE"
inbound = tmp_path / "staging" / "FromHPE"
inbound.mkdir(parents=True)
(inbound / "TP11525703-837P_M019048402-20260618130000000-1of1_TA1.x12").write_bytes(
b"hello-world",
)
client = SftpClient(sftp_block)
body = client.read_file("/ToHPE/TP11525703-837P_M019048402-20260618130000000-1of1_TA1.x12")
body = client.read_file("/FromHPE/TP11525703-837P_M019048402-20260618130000000-1of1_TA1.x12")
assert body == b"hello-world"
def test_stub_read_file_missing_raises(sftp_block):
client = SftpClient(sftp_block)
with pytest.raises(FileNotFoundError):
client.read_file("/ToHPE/does-not-exist.x12")
client.read_file("/FromHPE/does-not-exist.x12")
backend/tests/test_store_update_clearhouse.py
+2 -2
View File
@@ -27,8 +27,8 @@ def test_update_clearhouse_round_trip():
port=22,
username="colorado-fts\\coxix_prod_11525703",
paths={
"outbound": "/CO XIX/PROD/coxix_prod_11525703/FromHPE",
"inbound": "/CO XIX/PROD/coxix_prod_11525703/ToHPE",
"outbound": "/CO XIX/PROD/coxix_prod_11525703/ToHPE",
"inbound": "/CO XIX/PROD/coxix_prod_11525703/FromHPE",
},
stub=False,
staging_dir="./var/sftp/staging",
docs/reference/co-medicaid.md
+6 -6
View File
@@ -25,8 +25,8 @@ the 837P file.
| Submitter contact | Tyler Martinez <tyler@dzinesco.com> |
| SFTP host | `mft.gainwelltechnologies.com` |
| SFTP username | `colorado-fts\coxix_prod_11525703` |
| SFTP outbound dir | `/CO XIX/PROD/coxix_prod_11525703/FromHPE` |
| SFTP inbound dir | `/CO XIX/PROD/coxix_prod_11525703/ToHPE` |
| SFTP outbound dir | `/CO XIX/PROD/coxix_prod_11525703/ToHPE` |
| SFTP inbound dir | `/CO XIX/PROD/coxix_prod_11525703/FromHPE` |
## dzinesco's 3 billing-provider NPIs
@@ -49,8 +49,8 @@ dzinesco submits 837P files to Gainwell's MFT (Managed File Transfer)
at `mft.gainwelltechnologies.com`. The full SFTP path layout is
specified by the user (2026-06-20):
- **Outbound** (we send): `/CO XIX/PROD/coxix_prod_11525703/FromHPE`
- **Inbound** (HPE/Gainwell sends to us): `/CO XIX/PROD/coxix_prod_11525703/ToHPE`
- **Outbound** (we send): `/CO XIX/PROD/coxix_prod_11525703/ToHPE`
- **Inbound** (HPE/Gainwell sends to us): `/CO XIX/PROD/coxix_prod_11525703/FromHPE`
### File naming
@@ -107,8 +107,8 @@ curl -X POST http://localhost:8000/api/clearhouse/submit \
-H 'Content-Type: application/json' \
-d '{"claim_ids": ["CLM-1", "CLM-2"], "payer_id": "CO_TXIX"}'
# Files appear at ./var/sftp/staging/CO XIX/PROD/coxix_prod_11525703/FromHPE/
ls -la "./var/sftp/staging/CO XIX/PROD/coxix_prod_11525703/FromHPE/"
# Files appear at ./var/sftp/staging/CO XIX/PROD/coxix_prod_11525703/ToHPE/
ls -la "./var/sftp/staging/CO XIX/PROD/coxix_prod_11525703/ToHPE/"
```
## Payer IDs
docs/superpowers/specs/2026-06-20-cyclone-multi-payer-npi-sftp-design.md
+7 -7
View File
@@ -21,7 +21,7 @@ Replace the single hard-coded `PayerConfig` factory dict (currently in `api.py:9
- **SP10** — 277CA parser + "Payer-Rejected" lane in the Inbox.
- **SP11** — Tamper-evident hash-chained `audit_log` table.
- **SP12** — SQLCipher encryption at rest + Keychain-stored DB key.
- **SP13** — Replace the SFTP stub with real `paramiko` connection to `mft.gainwelltechnologies.com` and actually push to `/CO XIX/PROD/coxix_prod_11525703/FromHPE`.
- **SP13** — Replace the SFTP stub with real `paramiko` connection to `mft.gainwelltechnologies.com` and actually push to `/CO XIX/PROD/coxix_prod_11525703/ToHPE`.
- **Real SFTP credentials in Keychain** — schema and call sites are in place; the actual secret is created manually by the operator.
## 2. Goals
@@ -109,8 +109,8 @@ VALUES (1, 'dzinesco', '11525703', 'Dzinesco', 'Tyler Martinez', 'tyler@dzinesco
'"inbound_template":"TP{tpid}-{orig_tx}_M{tracking}-{ts}-1of1_{file_type}.x12"}',
'{"host":"mft.gainwelltechnologies.com","port":22,'
'"username":"colorado-fts\\coxix_prod_11525703",'
'"paths":{"outbound":"/CO XIX/PROD/coxix_prod_11525703/FromHPE",'
'"inbound":"/CO XIX/PROD/coxix_prod_11525703/ToHPE"},'
'"paths":{"outbound":"/CO XIX/PROD/coxix_prod_11525703/ToHPE",'
'"inbound":"/CO XIX/PROD/coxix_prod_11525703/FromHPE"},'
'"stub":true,"staging_dir":"./var/sftp/staging","poll_seconds":300}',
'2026-06-20T00:00:00Z');
```
@@ -163,13 +163,13 @@ All 3 share the same address because all 3 are registered to the Montrose corpor
Per the **HCPF X12 File Naming Standards Quick Guide** (https://hcpf.colorado.gov/tp-x12-filenaming):
**Outbound** (we send to `/CO XIX/PROD/coxix_prod_11525703/FromHPE`):
**Outbound** (we send to `/CO XIX/PROD/coxix_prod_11525703/ToHPE`):
```
{tpid}-{transaction_type}-{yyyymmddhhmmssSSS_MT}-1of1.{ext}
```
Example: `11525703-837P-20260620132243505-1of1.x12`
**Inbound** (HPE sends to `/CO XIX/PROD/coxix_prod_11525703/ToHPE`):
**Inbound** (HPE sends to `/CO XIX/PROD/coxix_prod_11525703/FromHPE`):
```
TP{tpid}-{orig_tx}_M{tracking}-{ts}-1of1_{file_type}.x12
```
@@ -209,7 +209,7 @@ Handler:
"ok": true,
"submitted": [
{"claim_id": "CLM-001", "filename": "11525703-837P-20260620132243505-1of1.x12",
"staging_path": "./var/sftp/staging/CO XIX/PROD/coxix_prod_11525703/FromHPE/11525703-..."}
"staging_path": "./var/sftp/staging/CO XIX/PROD/coxix_prod_11525703/ToHPE/11525703-..."}
],
"stub": true
}
@@ -367,7 +367,7 @@ All R200-R210 run on parse AND on serialize, so a 837 file with an unknown NPI f
| # | Question | Resolution |
|---|----------|-----------|
| 1 | SFTP outbound + inbound paths on `mft.gainwelltechnologies.com` | `/CO XIX/PROD/coxix_prod_11525703/FromHPE` (out), `/CO XIX/PROD/coxix_prod_11525703/ToHPE` (in) — user-provided 2026-06-20 |
| 1 | SFTP outbound + inbound paths on `mft.gainwelltechnologies.com` | `/CO XIX/PROD/coxix_prod_11525703/ToHPE` (out), `/CO XIX/PROD/coxix_prod_11525703/FromHPE` (in) — user-provided 2026-06-20 |
| 2 | 3 NPI street addresses + ZIPs | All 3 NPIs share Montrose corporate address (user confirmed 2026-06-20) — seed from prod files |
| 3 | 3 NPI taxonomy codes | `251E00000X` for all 3, self-served from 136 prod files |
| 4 | 277CA filename suffix | Use `277` per HCPF doc; distinguish 277CA by `ST*277CA` content (user confirmed 2026-06-20) |