Search

Full-text search

The lexical leg of the index is a BM25 full-text index over each document's data field. It needs no embedder, no model download and no network, and it is the leg that answers first: a write is searchable within the indexer's 250 ms debounce.

Pinning the mode

hybrid is the default; pass "mode": "fts" when you want lexical ranking only — exact terms, identifiers, error strings, names.

curl -s http://127.0.0.1:7001/v1/spaces/$SPACE/search \
  -H 'content-type: application/json' \
  -d '{"query": "index.embedder_unavailable", "mode": "fts"}'

any search $SPACE "index.embedder_unavailable" --mode fts

The reply's vectorStatus is skipped — the echo of your own choice — and score is the BM25 score.

Query operators

The query string is parsed by the full-text engine, so two operators work inline; two more arrive as separate arrays.

curl -s http://127.0.0.1:7001/v1/spaces/$SPACE/search \
  -H 'content-type: application/json' \
  -d '{"query": "\"zeppelin disaster\" hind*",
       "require": ["1937"],
       "exclude": ["fiction"]}'

any search $SPACE '"zeppelin disaster" hind*' --require 1937 --exclude fiction

Bare terms combine with OR by default: a hit needs any of them, and more matching terms rank higher. A bare term next to a require is an optional boost, not a filter. The operators apply to the lexical leg only — in pure vector mode they are ignored.

Note. index.search.defaultOperator: and makes every bare term required. It is precise for short keyword input a client controls, but on natural-language queries it collapses recall (few documents contain every word — measured on BEIR, nDCG@10 fell from 0.66 to 0.02). Prefer phrases, require and exclude, which add precision without the cliff.

Stop words

A small English stop-word list is stripped from the FTS-leg query before it runs (index.search.stopWords, default on). On a bag-of-words OR engine every common word matches a large share of the corpus and drags BM25 toward length and frequency noise; dropping them is a precision win for conversational queries and neutral on terse ones. Two safeguards: an all-stop-words query is left unchanged rather than emptied, and stripping is skipped whenever query contains a ", so phrases survive intact. The vector leg always sees the full query.

Scopes

scopes narrows the search to a set of slugs; empty means all. Scopes are an open set ([a-z0-9_-], up to 64 characters) — an unknown but well-formed scope returns no hits, a malformed one is 400 search.bad_scope.

Property values are self-describing entries — "Publisher: Gollancz", "Score: 9" — so searching by property name works too, and bare numbers carry context. A content-only search passes scopes without props.

any search $SPACE "gollancz" --scopes props --mode fts

BM25 tuning

Three engine dials are exposed under index.search.*; all default to 0, meaning "engine default", so an absent block changes nothing.

The measured sweep found no non-zero value that helped on the reference corpus, so the knobs stay neutral; see Evaluation.

Reading the scores

BM25 scores are comparable only within one response, and never with the cosine or RRF scores of the other modes. Rank on them, don't threshold.

Errors

See Hybrid ranking for how this leg is fused with the vector leg, and How indexing works for what makes it into the index in the first place.