readme: document toolchain and sanitized test fixtures
Cargo needs an installed, PATH-visible Rust; record how (rustup via brew) and where build/test commands run (Application/). Python is a fixture-regeneration aid only, not a product dependency. data/fixtures/: committed sanitized copies of all three input formats (see scripts/ and prior commit), safe for any repo consumer and the expected substrate for parser tests. Documents the sanitization choices: values with personal content are replaced, non-PII values (amounts, GL codes, descriptions) kept verbatim, and subfranchise cross-value arithmetic intentionally broken.
This commit is contained in:
@@ -21,6 +21,17 @@ For a few years RPNAB had a daughter company called Recamp Nordic AB that everyt
|
||||
|
||||
Code will eventually be kept on a private gitea instance (for issue handling), but everything is to be considered open source. No expectations of assistance with writing code, but if the project and the company is successful a dream of the maintainer is for other companies and developers to make use of it. However, due to the bespoke nature of the functionality provided this seems unlikely.
|
||||
|
||||
## Toolchain
|
||||
|
||||
- Rust via rustup, stable channel (installed with `brew install rustup-init;
|
||||
rustup default stable` on macOS; rustup's shims live in
|
||||
`/opt/homebrew/opt/rustup/bin` — keep it on PATH).
|
||||
- Build and test from the `Application/` directory: `cargo build`,
|
||||
`cargo test`, `cargo fmt --check`, `cargo clippy`.
|
||||
- Python 3 (stdlib + openpyxl + pypdf) is used only by
|
||||
`Application/scripts/` for regenerating sanitized test fixtures; it is
|
||||
not part of the product toolchain.
|
||||
|
||||
## Technology
|
||||
|
||||
The whole stack is Rust: one language, one toolchain, one test runner,
|
||||
@@ -90,6 +101,19 @@ committed, and their values must never be embedded in committed test files.
|
||||
Tests may *reference* sample files by path (the files stay local), or use
|
||||
synthetic/sanitized values in the repo.
|
||||
|
||||
Sanitized copies of the samples — committed, safe for any repo consumer —
|
||||
live under `Application/data/fixtures/` (epsilon batch files 405-412 with
|
||||
synthetic card/customer ids, batches renumbered 9405+ and dates +2y;
|
||||
a cumulative slice, batches 5001+, dates -6y; tsdrms xlsx files with
|
||||
synthetic R/A / DBR / location ids, dates +2y; subfranchise statements as
|
||||
extracted text with all identity and amounts replaced). Amounts, GL codes
|
||||
and descriptions are kept verbatim where they carry no personal data;
|
||||
cross-value arithmetic in the subfranchise fixtures (EUR × rate = SEK) is
|
||||
deliberately NOT preserved. Regenerate with
|
||||
`python3 Application/scripts/sanitize_samples.py` (reads the local raw
|
||||
samples) and verify sanitization with `check_fixture_leaks.py` (exit 0 =
|
||||
clean). Feature branches should build parser tests against the fixtures.
|
||||
|
||||
The raw formats are canonical: the application parses what the source systems
|
||||
deliver. Filenames carry metadata (batch / month / invoice number) and are
|
||||
the natural keys for idempotent re-ingestion — ingesting the same file twice
|
||||
|
||||
Reference in New Issue
Block a user