Roadmap now reflects reality: config done, db tests red awaiting
implementation (with the 'do not alter test intent' guard), and the
epsilon-parser branch open-but-empty pending the step-2 spec.
Guidelines adjusted so an agent can work a feature branch
unattended: fixtures are the explicit exception to the data/ commit
ban, and per-passing-test commits need no feedback on a branch —
main still does.
Cargo needs an installed, PATH-visible Rust; record how (rustup via
brew) and where build/test commands run (Application/). Python is a
fixture-regeneration aid only, not a product dependency.
data/fixtures/: committed sanitized copies of all three input formats
(see scripts/ and prior commit), safe for any repo consumer and the
expected substrate for parser tests. Documents the sanitization
choices: values with personal content are replaced, non-PII values
(amounts, GL codes, descriptions) kept verbatim, and subfranchise
cross-value arithmetic intentionally broken.
Real samples under data/test_input/ stay local and gitignored; these
committed fixtures are format-identical copies with every identifying
value replaced one-way via salted hashes: epsilon contract cards and
customer numbers into synthetic disjoint ranges (batches renumbered
9405+, dates +2y; cumulative slice batches 5001+, dates -6y), tsdrms
R/A / DBR / location ids into ZZ-form synthetic ids (dates +2y,
filenames shifted to match cutoffs), subfranchise statements to
extracted text with sender, partner, invoice numbers and amounts
replaced (EUR x rate = SEK arithmetic deliberately NOT preserved).
Amounts, GL codes, descriptions, and station/terminal/pump/receipt
numbers carry no personal data and are kept verbatim per readme
'Data formats'.
scripts/sanitize_samples.py regenerates fixtures deterministically
from local raw samples; scripts/check_fixture_leaks.py verifies no
real value appears in fixtures (content or filenames), exit 0 =
clean. Verified passing for all sources.
Tests define the db module API:
- connection_url() builds the sqlx URL with percent-encoded
credentials; with/without the database name
- split_statements() splits SQL on semicolons outside strings and
line comments (the embedded migration is applied statement by
statement, since DDL cannot run in a rollbackable transaction)
- the embedded migration is Documentation/schema.sql (version 1),
carrying the seven domain tables only; schema_migrations is
created by the application
- backup_command() shells out to mariadb-dump (mysqldump fallback)
with --single-transaction, password via MYSQL_PWD, never argv
- restore_command() pipes the dump (which carries CREATE DATABASE /
USE) into the mariadb client
- backup_filename() is timestamped per cli.md
- Config::load / load_with read the TOML file; port fields accept
either a TOML int or a string (real configs use both)
- empty strings resolve to defaults: db 3306, web 8080,
$HOME/.config/rpn state dir, <state>/backups backup dir; an
explicit state dir steers the backup default
- show() masks the password; validate_database() enforces
host/user/name for commands that touch the database
- resolve_config_path() implements the documented lookup order
Tests define the API for the config module:
- Config::load_with(path, home) / Config::load(path) read the TOML file
(database/web/state/backup sections per config.template.toml)
- empty strings resolve to built-in defaults: db port 3306, web port
8080, state dir $HOME/.config/rpn, backup dir <state>/backups
(backup follows an explicitly set state dir)
- port may be written as a TOML int or a string in real configs;
non-numeric strings are a ConfigError
- resolve_config_path(): --config flag > $RUSTYRPN_CONFIG >
cwd/config.toml; --env=dev|test only affects that last default slot
- Config::show() renders the effective config with the password masked
- Config::validate_database() requires host/user/name
Also adds the toml crate (forced by the TOML config decision).
First code in the repo. Workspace root is Application/ per readme;
members are src/core (crate rpn-core) and src/cli (binary rpnc).
Crate names are prefixed to avoid clashing with the std `core`
crate; directory names follow the readme layout.
Dependencies (versions pinned in Cargo.lock):
- core: sqlx (mysql, runtime-tokio, chrono, rust_decimal), chrono,
csv, thiserror, serde/serde_json, tracing, plus sha2 (SHA-256
file checksums required by schema.sql), rust_decimal (exact
DECIMAL(10,2) money, forced by the "money is DECIMAL
everywhere" decision), tokio and tempfile as dev deps.
- cli: clap (derive), tokio, tracing-subscriber (env-filter),
anyhow, serde/serde_json, chrono, csv.
The extra crates beyond the readme stack (sha2, rust_decimal,
tracing-subscriber, tempfile-dev) are direct consequences of
already-committed decisions; flag for maintainer review.
No feature logic yet: rpnc is an empty shell, the domain modules
land in the TDD slices that follow.
The transactions table keeps 9 of the 16 source fields; the dropped
list also includes Card type (equals Card number in all samples, per
readme). Header said 10 and omitted Card type.
Move the untracked 'schema draft.sql' to Documentation/schema.sql: the
agreed v1 fuel-domain schema (files, customers, batches, cards,
transactions, invoices, invoice_items), design decisions recorded in the
header: NULL-able cleartext card PINs, slim ledger projection (10 of 16
source fields, the files table is the canonical archive), string customer
business key, unified DECIMAL SEK money, full invoice traceability,
cli.md status values.
cli.md: transaction read takes <date> <receipt> -- the register's
receipt counter repeats across days (9,990 distinct receipts in the
138k-row sample), the day+receipt pair is the ledger dedup key, verified
unique in all samples. File import step 5 clarified as an internal
consistency check, since source files carry no totals of their own.
Invoice business key corrected to invoice number.
readme.md: point the database bullet at Documentation/schema.sql.
- In development: the CLI command structure is specified in
Documentation/cli.md
- Config: daemon state and database backups default to ~/.config/rpn
(backups in its backups/ subdirectory), overridable in config
- [web]: port and public fqdn for the daemon (Caddy proxies from a
separate jail); port defaults to 8080
- [state]: daemon pidfile and logs directory, default ~/.config/rpn
- [backup]: timestamped dumps for rpnc db backup, default
~/.config/rpn/backups; rpnc db restore looks here by default
- cli.md: document the optional positional path and the directory
defaults for db backup/restore
Reframe the implementation notes as design decisions (no code exists
yet) and resolve the open design questions:
- card: contract fuel card only; retail rows carry no card
- customer delete also guards against invoices and cards
- invoice: outgoing fuel only; add explicit send (draft -> sent);
--all fans out to one invoice per customer; document statuses
- VAT: amounts stored inclusive, 25% base/VAT split at invoice time
- file import: v1 = epsilon TSV only; one DB transaction per import;
batch-total mismatch aborts with exit code 1
- file export --format raw is source-faithful; raw errors elsewhere
- batch: document derived nature; update = manual reconciliation;
list gains --from/--to/--year filters
- daemon status is local-only (no external check); pidfile/logs under
a configured state directory
- db: --force on reset/restore; timestamped backups in configured dir;
status reports migration version
- add config show, --version, --config precedence, exit-code table,
business-key ID semantics, and status values
hermes
jakob