CiteBible — Fixing Pastoral Retrieval: rewrite-first, re-ingest as polish

The headline — proven live, no re-ingest

A single query rewrite flipped the flagship query on the current chapter-chunked corpus. We changed nothing about the index, the embedder, or the chunks — only how we phrase the search. Same engine, same corpus, and the results went from topical noise to genuinely pastoral.

The flagship query“my mother is dying and I'm angry at God”

✕ RAW sentence → retrieval today

the literal sentence, embedded as-is

Acts 2:18–33 Matthew 8:14–30 · Gadarene demoniacs Mark 15:37–47 · Jesus' burial Mark 5:33–43 Revelation 11:13–19

Shared surface vocabulary — death, mother, God — but no comfort for a grieving, angry reader. Topical noise.

✦ THEME-rewritten → retrieval now

step-back to: “grief; lament; feeling abandoned by God; comfort; God's nearness; hope in suffering”

Psalm 88 · the darkest lament 2 Corinthians 1 · the God of all comfort Psalm 42 · why are you cast down, O my soul 2 Corinthians 7 1 Thessalonians 3

Lament, comfort, and God's nearness in suffering — the pastoral function the reader needs. Genuinely pastoral.

Same engine, same corpus, only the query changed.

This is why rewrite-first leads the plan — and why it plausibly demotes the verse-level re-ingest from “required” to “precision polish.” If a one-LLM-call reformulation recovers most of the quality gap for free, we ship it and measure before we spend a re-ingest. The rewrite is being wired into the live /carry endpoint (ollwrite PR #3), reusing the existing oll-model gateway call — verified via curl; end-to-end wiring in progress.

What this is

A build-ready engineering plan, not a design study. The CiteBible engine already exists — the public-domain World English Bible corpus over oll-memory, grounded-only, corpus-agnostic. The branding & design study gave it a face; the Verticalize the Engine build-map put one corpus behind a One Box door. This v2 reorders the fix after a live experiment plus a second research pass: the cheapest lever moved first, and it worked. Everything below is ordered by cost-and-reuse, grounded in the files it touches. Scope stays fixed: study & sermon-prep, a research assistant — not a ghostwriter, not the voice of God, not spiritual counsel.

Part 1The diagnosis — true ≠ relevant

Every reference it returns is real, correctly-addressed, and grounded. And still wrong for the person asking — until the query names the need.

The gap is true ≠ relevant. The engine is honest — it never fabricates, and it addresses Book Chapter:Verse correctly. But relevance for a grieving reader is pastoral function, and the raw pipeline scores topical similarity. Two compounding causes produce this — and the live result above shows the query-side one dominates:

Cause 1 — the raw query scores topical similarity, not pastoral function

The literal sentence embeds next to Jesus' burial because they share words (death, mother, God). Psalm 88 and 2 Corinthians 1 share little surface vocabulary with the sentence — they comfort by function, not by words — so they score low until the query is abstracted to the need. This is the cause the rewrite fixes directly, live, for free.

Fix path — the query, not the index: ollwrite BFF /api/citebible/ask → handleSearch, one oll-model call before retrieval.

Cause 2 — chapter chunking dilutes the target verse (a precision ceiling, not the floor)

Psalm 34:18 (“Yahweh is near to the brokenhearted”) is averaged into a ~20-verse chapter vector, so a single comforting verse can't surface on its own merits. This still caps how sharp retrieval can get — but the live flip proves it is not the thing blocking a good answer today. It becomes worth fixing only once the cheaper levers are measured.

Fix path — the index: services/oll-memory/engine.py chunks whole-chapter text. The unit of retrieval is the chapter, not the verse. Demoted to step 4.

The framing that still orders everything: recall before precision

A reranker cannot fix what retrieval missed. If a comfort verse is not in the candidate pool, no amount of reordering can surface it. The rewrite is a recall move — it pulls the right passages into the pool by aiming at the need. Only then do precision moves (reranker, genre boost) reorder them to the top. That order is why the steps are numbered the way they are.

Phase A · first

Recall — get the right verse INTO the pool

Task-prefixes (correctness) · query rewrite into the need (the proven lever) · verse-level chunks only if a gap remains · a curated human-vote prior.

Goal: the comfort passages are somewhere in the top-50 candidates.

→

Phase B · then

Precision — reorder it to the top

Cross-encoder reranker over a wide pool · a soft genre boost conditioned on pastoral intent.

Goal: the best comfort verse moves to top-3.

Part 2The reordered build — cheapest-first, reuse-maximizing

Numbered by cost-and-reuse, not by effort. Each step: the change · the file it touches · impact · cost · confidence · anti-pattern · sources. Steps 1–3 need no re-ingest; step 4 is the only one that does, and it's conditional.

Fix nomic task-prefixes FREE · precondition

services/oll-memory/embeddings.py

Ingest every chunk as search_document: <text>; embed every query as search_query: <query>. Nomic REQUIRES these prefixes — it was trained asymmetrically — and Ollama does NOT add them automatically. Mixing prefixed and unprefixed vectors distorts cosine similarity, so today we're comparing points from two different spaces.

Impact: load-bearing, not polish. It moves every vector into the space the model was actually trained for — and it's the precondition that makes reusing the old chapter vectors valid (step 4 re-embeds them with the document prefix into one clean space).

Ordering: a prefix change forces a re-embed, so land it first and re-embed once.

Cost: FREEConfidence: HIGH

Sources: HF nomic-ai/nomic-embed-text-v1 & v1.5 model cards · arXiv 2402.01613 · ollama.com/library/nomic-embed-text

Ship Step-Back query rewrite on the CURRENT corpus Recall · PROVEN

ollwrite BFF — /api/citebible/ask → handleSearch, before the oll-memory query

One LLM call maps the raw emotional sentence to its abstract biblical themes / underlying need before retrieval: “my mother is dying and I'm angry at God” → “grief; lament; feeling abandoned by God; comfort; God's nearness; hope in suffering.” Then layer a capped multi-query union — N=3–5 reformulations plus ALWAYS the original (so a true literal match is never lost) — fused by RRF. Wired server-side in the /carry BFF, reusing the existing oll-model gateway call.

Impact: the proven flip above. Plausibly recovers most of the quality gap with no re-ingest — which is exactly what demotes step 4 to optional. Build a small emotional-query eval set to measure it (don't eyeball).

Anti-patterns: (a) HyDE that renders a fabricated verse — HyDE stays strictly internal: the generated draft feeds only the embedder, is discarded, and no generated verse is ever shown or cited. (b) Over-rewriting that drifts from intent — mitigate by keeping+retrieving the original, a relevance floor, and honest abstention. (c) Skip the RL-trained rewriter (Rewrite-Retrieve-Read) — a zero-shot LLM call is enough.

Cost: low — one LLM call, reuses oll-modelConfidence: HIGH (right lever)Confidence: MEDIUM (exact magnitude)

Sources: Step-Back Prompting arXiv 2310.06117 (TimeQA +27%) · Rewrite-Retrieve-Read 2305.14283 · HyDE 2212.10496 · LangChain MultiQueryRetriever · RAG-Fusion

Cross-encoder reranker over a WIDE pool Precision

oll-memory query path — after retrieval, before the relevance floor

Retrieve top-30–50, then rerank to top-5 with bge-reranker-v2-m3 (self-host, ~90ms), inserted after retrieval and before the floor. A cross-encoder reads query and candidate together — and it is the only step that yields one comparable relevance score across granularities, so it can honestly rank a verse against a chapter (raw cosine can't).

Impact: +5–15 NDCG@10 — roughly 5–10× the lift of a base-model swap — provided the right passage is already in the pool (hence the wide top-50, and steps 2/4/5 feeding recall first).

Anti-pattern: reranking a narrow top-5 — you can't reorder what wasn't retrieved. Measure Recall@50 first.

Cost: low — self-host, ~90msConfidence: HIGH

Sources: calebyhan/bible-rag · machinelearningmastery.com reranking roundup

ONLY IF a gap remains: multi-granularity re-ingest Conditional · precision polish

services/oll-memory/engine.py · upsert_document()

Reuse, don't rebuild. Keep the chapter vectors, re-embed them with search_document: for one clean space, and ADD verse vectors. Store level = 'verse' | 'chapter' plus book/chapter/verse metadata in ONE pgvector table, one embedding column, same model (mandatory — mixed spaces across levels are incomparable).

Retrieve top-K PER level (two WHERE level= queries — which also dodges pgvector's HNSW post-filter overfilter; enable hnsw.iterative_scan='relaxed_order' on 0.8+ if you filter). RRF-fuse (k=60) the two per-level lists. Then AutoMerge-collapse (LlamaIndex simple_ratio_thresh default 0.5): collapse retrieved verses up to their parent chapter only when >50% of that chapter's retrieved children hit; the merged parent inherits the average child score; otherwise keep the individual verses. Then cross-encoder rerank the deduped union.

Our engine already supports the storage: upsert_document(collection, document, chunks, acl_tags) is idempotent and takes pre-built chunks with per-chunk metadata (services/oll-memory/engine.py) — so this is ~30 lines over the existing pgvector+reranker, or the turnkey LlamaIndex HierarchicalNodeParser ([2048,512,128], embed leaves only) + AutoMergingRetriever (a bigger commitment).

Impact: the precision ceiling — an addressable, undiluted verse that can rank and cite exactly. Reach for it only if steps 2–3 leave a measured gap.

Anti-patterns: a flat index of parents + children (near-dupes crowd results) · RRF/hash-dedup does NOT merge nested granularities (a verse and its chapter are different nodes) · mixed embedding spaces across levels · never return both a verse and its containing chapter (double-counts text, wastes slots).

Cost: moderate — one re-ingest, conditionalConfidence: HIGH (mechanics)

Sources: LlamaIndex AutoMergingRetriever (ratio 0.5, avg-merge) + HierarchicalNodeParser · QueryFusionRetriever (RRF k=60, dedup by node.hash — opt into reciprocal_rerank) · RRF (Cormack, SIGIR 2009) · RAPTOR arXiv 2401.18059 · Dense Hierarchical Retrieval arXiv 2110.15439 · pgvector 0.8 iterative scan

Curated OpenBible topical prior + cross-refs — and the eval gold set Recall + Proof

new ingest of OpenBible.info topical votes + cross-references (CC-BY)

Ingest OpenBible.info's topical vote data (topic → ranked verses, refreshed weekly) plus their 340k cross-references — ~20 years of human votes encoding “a Psalm comforts better than a genealogy,” a signal no embedding captures. Two uses: a high-precision fast path for confident pastoral queries, and a rerank prior.

Crucial dual use: the same data is the eval gold set that PROVES the flagship query is fixed and stays fixed — how “grounded is non-negotiable” gets measured instead of eyeballed.

Cost: moderate — one-time ingest, CC-BYConfidence: HIGH

Sources: openbible.info/topics · openbible.info/labs/cross-references

Genre metadata + SOFT boost — never a hard filter Precision

verse metadata + rerank stage

Store genre per verse. At rerank, conditioned on detected pastoral intent, apply small multiplicative boosts — never additive, never zero:

Psalms                              × 1.30
wisdom / gospel-comfort / epistle   × 1.15
narrative                           × 1.00
genealogy / census / ceremonial-law × 0.70   ← down-weight, never ×0

Semantic similarity stays dominant; genre only tips ties. This quiets the Gadarene-demoniac / genealogy noise without hiding apt narrative (John 11 · Num 6:24–26).

Anti-pattern: hard-excluding genres or books hides the best grief verse when it sits in “narrative” or “law.” Down-weight, don't delete. And genre at book granularity is lossy — Psalm 78 is narrative — so tag per verse.

Confidence: HIGH

Sources: soft-vs-hard weighting (medium.com/logspace) · pgvector recall (dev.to/franckpachot)

Part 3Display — show the most relevant part, expand for context

An independent track (Sam's design call) that ships anytime, orthogonal to the retrieval steps: small-to-big at the UI. Progressive disclosure — show the answer, hide only the context.

Psalm 34:18 · World English Bible

“Yahweh is near to the brokenhearted, and saves those who have a crushed spirit.”

Show in context

¹⁷ The righteous cry, and Yahweh hears, and delivers them out of all their troubles. ¹⁸ Yahweh is near to the brokenhearted, and saves those who have a crushed spirit. ¹⁹ Many are the afflictions of the righteous, but Yahweh delivers him out of them all.

Pin the single best verse — and keep the reference (Book Chapter:Verse) visible in BOTH collapsed and expanded states. Non-negotiable for a citation product: losing provenance on expand is the top anti-pattern.
Highlight query / theme terms in the snippet (KWIC / query-biased).
Wrap the surrounding ±2–3 verse window (or parent chapter) in a native <details>/<summary> disclosure — default collapsed, accessible for free (role/button, aria-expanded, Enter+Space).
Define “the most relevant part” of a coarse chapter by having the existing cross-encoder score each verse and pinning the top one — reuse the reranker, the cheapest correct option. At scale, note AGRaME (arXiv 2405.15028).

Governing principle — progressive disclosure

Show the answer (the matched verse) up front; hide only the context behind one tap. The reference travels with the verse in every state.

Sources: NN/g progressive disclosure · W3C WAI-ARIA APG disclosure pattern · Elastic highlighting.

Part 4Build order, scope, and what NOT to do

The build order — one line

(1) fix prefixes → (2) ship rewrite on the current corpus + measure → (3) reranker → (4) only if a gap remains, multi-granularity verse+chapter fused → (5) OpenBible prior / eval gold set → (6) genre boost. Display ships independently, anytime.

Metadata schema — stored per node

Tier	Fields	When
Tier 1	`book` · `chapter` · `verse_start` · `verse_end` · `ref` · `level` · `translation`	Always — the citation contract + per-level retrieval
Tier 2	`testament` · `genre`	Cheap & high-value — enables step 6
Tier 3	`themes[]` · `cross_refs`	When a feature needs it — from step 5

Reference implementation to mirror

github.com/calebyhan/bible-rag — per-verse chunks, pgvector, RRF + bge-reranker-v2-m3, LLM query expansion. It is steps 1–4 already proven end-to-end. Its one gap — no genre boost, no human-vote prior — is exactly our differentiator (steps 5 and 6).

The net anti-patterns — what NOT to do

Chapter / fixed-token chunkingdilutes the target verse into its neighbours.

Pure dense for intentscores vocabulary overlap, not pastoral function — rewrite the query.

Reranking before fixing recallcan't reorder what was never retrieved.

Hard-excluding genres / bookshides the best grief verse when it sits in “narrative.”

Genre tagging at book granularitylossy — Psalm 78 is narrative, not poetry.

Assuming pgvector filters “just work”metadata post-filtering on HNSW silently craters recall.

Flat parent + child indexnear-dupes crowd results — retrieve per level, then AutoMerge.

RRF / hash-dedup to merge granularitiesit won't — a verse and its chapter are distinct nodes.

Mixed embedding spaces across levelsverse and chapter vectors become incomparable.

HyDE that renders a fabricated versethe draft is internal-only, discarded — never shown or cited.

Losing the reference on expandprovenance must travel with the verse in every state.

→The verdict

Rewrite-first is proven, and it likely makes the re-ingest optional precision polish rather than a prerequisite. Ship the free prefix fix, ship the rewrite, measure on a real emotional-query eval set, add the reranker — and reach for the multi-granularity re-ingest only if a gap remains. The aim does not move: study & sermon-prep, a research assistant — not a ghostwriter, the librarian, not the voice of God. Better retrieval and honest abstention serve that aim.

Rewrite the query into the need, then reorder it to the top.

Status: ENGINEERING PLAN v2 — build-ready, rewrite-first, 2026-07-05. A live experiment flipped the flagship query on the current chapter-chunked corpus with a single step-back rewrite and NO re-ingest (Acts 2 / Gadarene demoniacs / Jesus' burial → Psalm 88 / 2 Corinthians 1 / Psalm 42) — so the build now leads with the cheapest lever and demotes verse-level re-ingest to conditional precision polish. Order: (1) task-prefixes → (2) rewrite + measure → (3) reranker → (4) only-if-gap multi-granularity re-ingest → (5) OpenBible prior/eval → (6) genre boost; display ships anytime. Rewrite wiring into /carry = ollwrite PR #3 (curl-verified, in progress). Scope is Sam's call; aim stays study/sermon-prep. Face & naming: the design study. Thesis & build-map: Verticalize the Engine.