victus-rag is a modular CLI application for retrieval experimentation over scientific
claims.
The system is shaped as a local retrieval and evaluation pipeline. The CLI coordinates
configuration, indexing, retrieval, evaluation, export, and telemetry modules. Persistent
state lives outside the application process in local files, generated artifacts, and Qdrant.
The central architectural unit is the claim. Sparse, dense, and hybrid retrieval paths all
produce ranked claim results that can be queried directly, evaluated locally, evaluated
through BEIR, or emitted as audit telemetry.
CLI
-> config
-> indexing / retrieval / evaluation / export / telemetry
-> local files, artifacts, Qdrant, OpenTelemetry collector
Path: src/cli.py
The CLI owns command parsing and runtime orchestration. It does not own retrieval logic
directly; it dispatches to indexing, retrieval, evaluation, exporter, and telemetry modules.
Inputs:
Outputs:
Paths: src/config.py, config/default.yaml, config/beir_generation.yaml
Configuration defines runtime paths, Qdrant settings, dense retrieval settings, Parquet
column names, query-provider mode, telemetry settings, and synthetic BEIR generation
defaults.
The configuration layer translates YAML into typed application config objects used by the
CLI and downstream modules.
Path: src/retrieval/sparse.py
Sparse retrieval owns claim loading, local BM25-style index construction, sparse search,
claim serialization, and context matching from available source sections.
Inputs:
Outputs:
Paths: src/indexing/, src/retrieval/dense.py, src/retrieval/query_provider.py
Dense retrieval is split across indexing and querying responsibilities.
Indexing validates embedded Parquet records and upserts claim vectors into Qdrant.
Querying obtains a query vector from the configured query provider and searches the Qdrant
collection.
Query providers are interchangeable:
Path: src/retrieval/hybrid.py
Hybrid retrieval composes sparse and dense retrieval results. It expands the candidate pool,
retrieves from both backends, and merges rankings with Reciprocal Rank Fusion.
The hybrid component depends on both the local sparse index and the Qdrant-backed dense
retriever. It owns fusion behavior, not the underlying retrieval backends.
Paths: src/eval/, src/eval/beir_generation/
Evaluation has three architectural roles:
Local evaluation compares ranked claim IDs against relevant claim IDs. BEIR evaluation
adapts sparse, dense, and hybrid retrievers to the BEIR retrieval contract. Synthetic BEIR
generation builds corpus, queries, qrels, manifest, and status files from the claim universe.
Path: src/exporters/
Exporters produce derived artifacts from repository data. They are separate from retrieval
and evaluation so export behavior does not become hidden retrieval logic.
Path: src/telemetry/
Telemetry is an optional audit layer for retrieval queries. It wraps retrieval execution
with OpenTelemetry spans and records query metadata plus returned claim documents.
Telemetry does not affect ranking, indexing, evaluation metrics, or generated artifacts.
Internal boundaries:
External boundaries:
CLI query sparse
-> load app config
-> load sparse index JSON
-> tokenize query
-> score indexed claims
-> return ranked claim results
-> optionally emit telemetry
CLI query dense
-> load app config
-> build query provider
-> embed query
-> search Qdrant collection
-> normalize Qdrant payloads
-> return ranked claim results
-> optionally emit telemetry
CLI query hybrid
-> load sparse index
-> build dense retriever
-> retrieve sparse candidates
-> retrieve dense candidates
-> fuse rankings with RRF
-> return ranked claim results
-> optionally emit telemetry
evaluation examples or BEIR dataset
-> retriever adapter
-> ranked claim results
-> metric calculation
-> metrics, rankings, and failure-analysis artifacts
The system moves claim-centered data through distinct stages.
claim JSON / embedded claim Parquet
-> sparse index JSON
-> sparse retrieval results
embedded claim Parquet
-> Parquet schema validation
-> Qdrant collection
-> dense retrieval results
sparse results + dense results
-> hybrid fusion
-> hybrid retrieval results
embedded claim Parquet + generated queries + qrels
-> BEIR dataset directory
-> BEIR evaluation
-> metrics and analysis artifacts
Primary persistence locations:
data/claims/ stores claim JSON inputs when available.data/claims_embedded.parquet stores embedded claim inputs.data/indexes/ stores sparse index artifacts.data/beir/ stores generated BEIR-compatible datasets.artifacts/metrics/ stores evaluation metric payloads.artifacts/runs/ stores rankings and failure-analysis outputs.Reproducibility: retrieval and evaluation flows are CLI-driven, config-backed, and artifact-oriented.
Inspectability: generated indexes, datasets, metrics, rankings, manifests, and failure-analysis
files are readable outside the application process.
Modularity: sparse, dense, hybrid, evaluation, export, and telemetry modules have separate
responsibilities and can evolve independently when interfaces remain stable.
Observability: retrieval queries can emit OpenTelemetry spans with query and document-level
audit data.
Recoverability: most local state can be regenerated from source inputs, configuration, and
Qdrant indexing workflows.
Operational simplicity: the architecture relies on local files, a CLI, and a small set of
local services rather than a distributed application runtime.
docs/000-SYSTEM-CONTEXT.md for repository purpose, scope, vocabulary, and navigation.docs/200-OPERATIONS.md for operational workflows and validation.docs/300-CONTRACTS.md for stable CLI, data, telemetry, and artifact expectations.docs/decisions/ for architecture decision records.