DataSync from your FTP server
Cloud-managed PostgreSQL — the system of record
One folder per consumer, scheduled daily
When SAP arrives, it joins as a new export consumer. The catalog hub doesn't change. No re-architecture. No website re-deploy.
A future VPC upgrade for a fully private database adds ~$15/mo and can be done at any later date.
Pipeline and catalog online. IAM-auth wiring complete. First publisher feed flowing end-to-end.
All six publishers ingesting. Inventory management pulling daily from the export area.
Website cut over to the new exports. All current consumers stable. Old catalog retired.
| Entity | Holds | Audit |
|---|---|---|
| products | ISBN-13, title, format, dates, source priority | ingest_audit (per file) and export_audit (per snapshot) record every operation. |
| contributors | authors, illustrators, translators | |
| prices | per-region prices and currencies | |
| subjects | BIC / BISAC / Thema codes | |
| media | cover and content asset references |
| Consumer | Format | Why |
|---|---|---|
| fullybookedonline.com | .sql.gz |
Direct restore into the website's local catalog; fastest path to "live." |
| Inventory management | .csv |
Universal — every ERP / inventory tool accepts CSV. Headers in first row, quoted fields, UTF-8. |
| Point of sale | .json |
Modern POS importers are JSON-friendly. A manifest file lets the POS detect stale exports and skip no-ops. |
| ERP / analytics / mobile / partners | .parquet |
Columnar, compressed, schema-evolution-friendly — the lingua franca of every modern analytics tool. |
sap consumer to the exporter.csv or .idoc-friendly flat files)s3://onix-exports/sap/Public TLS-required database endpoint. Merger and exporter Lambdas authenticate with short-lived IAM tokens. No VPC, no NAT gateway.
Cost: included in the ~$27/mo recurring.
Switch the database to publicly_accessible = false. Add two VPC interface endpoints. Database is reachable only from inside the VPC; security model relies on network-layer isolation in addition to IAM auth.
Cost: ~$15/mo extra. One-Terraform-module change.
.sql exports