This runbook covers the simplified PostgreSQL integration. The active tables
are:
paper_pipeline_statepaper_processing_statestructured_papersstructured_blockspaper_classificationsexperiment_mapscanonical_evidencePostgreSQL is the active query/export sink for scientific outputs and
paper state. Detailed events and artifact manifests remain local JSON records.
Apply the active schema migration:
psql "$DATABASE_URL" -f ops/sql/005_simplified_postgres_schema.sql
Migration 005 drops and recreates every pipeline table except
structured_papers. Back up any other PostgreSQL data that must be retained
before applying it. Do not interrupt the migration after it acquires the
structured_papers lock; all destructive work runs in one transaction. On a
new database the migration creates structured_papers before acquiring the
lock.
DATABASE_URL should be injected through the normal secret path, preferably
Infisical for real environments.
Confirm the active tables exist:
psql "$DATABASE_URL" -c '\d paper_processing_state'
psql "$DATABASE_URL" -c '\d paper_pipeline_state'
psql "$DATABASE_URL" -c '\d structured_papers'
psql "$DATABASE_URL" -c '\d structured_blocks'
psql "$DATABASE_URL" -c '\d paper_classifications'
psql "$DATABASE_URL" -c '\d experiment_maps'
psql "$DATABASE_URL" -c '\d canonical_evidence'
Run the maintained smoke validation:
uv run pytest tests/test_cli_smoke.py -q
When PostgreSQL writes fail, failed deliveries must be retained in:
data/runtime/outbox/postgres_pipeline_records.jsonl
Outbox records must be compact and idempotent. They may reference local payloads
but they must not embed large payloads.
Delivery rules:
idempotency_key;The replay command is:
uv run python -m ops.scripts.sync_postgres_outbox
Export scientific output tables to CSV and Parquet:
uv run victus-postgres-export --output-dir data/reports/exports/postgres
Equivalent module form:
uv run python -m ops.scripts.export_postgres_tables --format csv --format parquet
By default this exports:
paper_processing_statestructured_blockspaper_classificationsexperiment_mapscanonical_evidenceUse repeated --table flags to export a subset or include
paper_pipeline_state and structured_papers.
Create a self-contained judge review sample with random papers, their canonical
evidence, a manifest, and matching PDFs:
uv run victus-canonical-evidence-sample \
--limit 5 \
--seed judge-001 \
--output-dir data/reports/exports/judge-samples/judge-001
All files for that review are written under the chosen --output-dir.
PostgreSQL stores final scientific output records and paper processing state.
It does not store:
Large payloads must remain in artifacts or scientific output rows designed for
that payload.