readme: document toolchain and sanitized test fixtures
Cargo needs an installed, PATH-visible Rust; record how (rustup via brew) and where build/test commands run (Application/). Python is a fixture-regeneration aid only, not a product dependency. data/fixtures/: committed sanitized copies of all three input formats (see scripts/ and prior commit), safe for any repo consumer and the expected substrate for parser tests. Documents the sanitization choices: values with personal content are replaced, non-PII values (amounts, GL codes, descriptions) kept verbatim, and subfranchise cross-value arithmetic intentionally broken.
This commit is contained in:
@@ -21,6 +21,17 @@ For a few years RPNAB had a daughter company called Recamp Nordic AB that everyt
|
|||||||
|
|
||||||
Code will eventually be kept on a private gitea instance (for issue handling), but everything is to be considered open source. No expectations of assistance with writing code, but if the project and the company is successful a dream of the maintainer is for other companies and developers to make use of it. However, due to the bespoke nature of the functionality provided this seems unlikely.
|
Code will eventually be kept on a private gitea instance (for issue handling), but everything is to be considered open source. No expectations of assistance with writing code, but if the project and the company is successful a dream of the maintainer is for other companies and developers to make use of it. However, due to the bespoke nature of the functionality provided this seems unlikely.
|
||||||
|
|
||||||
|
## Toolchain
|
||||||
|
|
||||||
|
- Rust via rustup, stable channel (installed with `brew install rustup-init;
|
||||||
|
rustup default stable` on macOS; rustup's shims live in
|
||||||
|
`/opt/homebrew/opt/rustup/bin` — keep it on PATH).
|
||||||
|
- Build and test from the `Application/` directory: `cargo build`,
|
||||||
|
`cargo test`, `cargo fmt --check`, `cargo clippy`.
|
||||||
|
- Python 3 (stdlib + openpyxl + pypdf) is used only by
|
||||||
|
`Application/scripts/` for regenerating sanitized test fixtures; it is
|
||||||
|
not part of the product toolchain.
|
||||||
|
|
||||||
## Technology
|
## Technology
|
||||||
|
|
||||||
The whole stack is Rust: one language, one toolchain, one test runner,
|
The whole stack is Rust: one language, one toolchain, one test runner,
|
||||||
@@ -90,6 +101,19 @@ committed, and their values must never be embedded in committed test files.
|
|||||||
Tests may *reference* sample files by path (the files stay local), or use
|
Tests may *reference* sample files by path (the files stay local), or use
|
||||||
synthetic/sanitized values in the repo.
|
synthetic/sanitized values in the repo.
|
||||||
|
|
||||||
|
Sanitized copies of the samples — committed, safe for any repo consumer —
|
||||||
|
live under `Application/data/fixtures/` (epsilon batch files 405-412 with
|
||||||
|
synthetic card/customer ids, batches renumbered 9405+ and dates +2y;
|
||||||
|
a cumulative slice, batches 5001+, dates -6y; tsdrms xlsx files with
|
||||||
|
synthetic R/A / DBR / location ids, dates +2y; subfranchise statements as
|
||||||
|
extracted text with all identity and amounts replaced). Amounts, GL codes
|
||||||
|
and descriptions are kept verbatim where they carry no personal data;
|
||||||
|
cross-value arithmetic in the subfranchise fixtures (EUR × rate = SEK) is
|
||||||
|
deliberately NOT preserved. Regenerate with
|
||||||
|
`python3 Application/scripts/sanitize_samples.py` (reads the local raw
|
||||||
|
samples) and verify sanitization with `check_fixture_leaks.py` (exit 0 =
|
||||||
|
clean). Feature branches should build parser tests against the fixtures.
|
||||||
|
|
||||||
The raw formats are canonical: the application parses what the source systems
|
The raw formats are canonical: the application parses what the source systems
|
||||||
deliver. Filenames carry metadata (batch / month / invoice number) and are
|
deliver. Filenames carry metadata (batch / month / invoice number) and are
|
||||||
the natural keys for idempotent re-ingestion — ingesting the same file twice
|
the natural keys for idempotent re-ingestion — ingesting the same file twice
|
||||||
|
|||||||
Reference in New Issue
Block a user