Discover candidate papers, screen them by title, and maintain the canonical
metadata lake at data/lake/paper_metadata.jsonl.
uv run victus-processing metadata-extraction explore --mode broad-nutrition
uv run victus-processing metadata-extraction from-doi --doi 10.1000/demo
uv run victus-processing metadata-extraction seed-dois --mode broad-nutrition --limit 200
uv run python -m ops.scripts.data.refresh_paper_metadata_from_dois --limit 10
paper_metadata.jsonl.data/lake/paper_metadata.jsonl.data/lake/paper_metadata.jsonl, when present;data/lake/paper_metadata.jsonl;data/lake/paper_metadata.s2_refreshed.jsonl;metadata-extraction from-doi appends or upserts one normalized record directly
in data/lake/paper_metadata.jsonl.
metadata-extraction explore sends only:
{
"candidates": [
{
"id": "cand_001",
"title": "Paper title"
}
]
}
The LLM must return:
{
"decisions": [
{
"id": "cand_001",
"decision": "keep | drop | uncertain",
"reason": "short reason grounded in the title"
}
]
}
Abstracts are not sent to the LLM selector.
Each line is a JSON object:
{
"metadata_id": "meta:s2:{source_paper_id} | meta:openalex:{document_id} | meta:crossref:{doi}",
"source_metadata": {
"source": "semantic_scholar | openalex",
"source_paper_id": "string | null",
"doi": "string | null",
"arxiv": "string | null",
"title": "string",
"year": "integer | null",
"citation_count": "integer | null",
"pdf_url": "string | null",
"authors": ["string"]
},
"schema_version": "v1",
"discovery": {
"seed_papers": ["doi"],
"is_seed_paper": "boolean"
},
"domain_screening": {
"decision": "keep | drop | uncertain",
"model": "string | null"
},
"created_at": "ISO-8601 UTC timestamp",
"updated_at": "ISO-8601 UTC timestamp"
}
Required fields:
metadata_id;source_metadata.title;schema_version;domain_screening.decision;created_at;updated_at.Rules:
schema_version must be v1;domain_screening.decision must be keep, drop, or uncertain;paper_metadata.jsonl;metadata-extraction explore skips candidates when:
source_metadata.source_paper_id matches the Semantic Scholar paperId;source_metadata.doi matches the candidate DOI.Only data/lake/paper_metadata.jsonl is used for metadata dedupe.
uv run victus-processing metadata-extraction --help
uv run victus-processing metadata-extraction explore --help
uv run pytest tests/test_cli_smoke.py -q