API reference

The core is importable from wardowl and has no dependencies. Adapters live in their own modules and need their extra.

Retrieval

class wardowl.PermissionedRetriever(backend, authorizer, *, strategy='auto', action='read', overfetch=4, max_candidates=1000, recheck=True, observers=(), audit_key=None, reranker=None, rerank_depth=50, redactor=None)

Searches a backend and returns only the hits the subject may read.

strategy chooses where permissions are applied: "prefilter" inside the search, with a filter from the authorizer; "postfilter" after it, checking each candidate; "auto" (the default) pre-filters when the backend supports filters and the authorizer returns one. action is passed to the authorizer on every call.

Post-filtering fetches k * overfetch candidates and multiplies the fetch by overfetch until k permitted hits are found, the backend has no more, or max_candidates is reached; it never checks more candidates than it asked for, whatever the backend returns. max_candidates bounds every retrieval: a k above it raises ValueError. recheck checks pre-filtered hits again with the authorizer, so a backend that ignores the filter cannot leak; with recheck=False they are matched against the filter locally instead, which needs the backend to return each chunk’s metadata as stored (the conformance kit checks that). observers receive a RetrievalEvent for every call, and audit_key (at least 16 random bytes) adds an HMAC of the query to those events.

reranker reorders the result after the permission check: the retriever finds the top rerank_depth hits the subject may read (or k, if larger, and at most max_candidates), the reranker orders them, and the first k are returned. The reranker never sees a chunk the subject may not read.

redactor (see wardowl.redaction) hides parts of the permitted hits, after the check and before the reranker: it may change a hit’s text and metadata but must return the same hits in the same order, and if it fails the retrieval raises AuthorizationError. aretrieve runs it in a thread, off the event loop.

Parameters:

• backend (SearchBackend)

• authorizer (Authorizer)

• strategy (Strategy)

• action (str)

• overfetch (int)

• max_candidates (int)

• recheck (bool)

• observers (Sequence[Observer])

• audit_key (bytes | None)

• reranker (Reranker | RerankFunction | None)

• rerank_depth (int)

• redactor (Redactor | RedactFunction | None)

retrieve(query, subject, k=5)

Return at most k hits the subject may read, best first.

query is text, or a Query carrying a vector you computed. Raises AuthorizationError when the authorizer fails or gives a malformed answer, and ValueError or TypeError for invalid arguments: InvalidQuery, a ValueError, when the query or k is at fault rather than the configuration. Nothing is returned in any of these cases.

Parameters:

• query (str | Query)

• subject (Subject)

• k (int)

Return type:

RetrievalResult

async aretrieve(query, subject, k=5)

The async form of retrieve, with the same arguments and guarantees.

It uses the backend’s asearch and the authorizer’s afilter_for and acheck when they exist, and runs the sync methods in a thread otherwise. The redactor runs in a thread.

Parameters:

• query (str | Query)

• subject (Subject)

• k (int)

Return type:

RetrievalResult

class wardowl.RetrievalResult(hits, strategy, examined, denied, violations, complete)

What one retrieval returned, with the counts needed to monitor it.

Parameters:

• hits (list[Hit])

• strategy (Literal['prefilter', 'postfilter'])

• examined (int)

• denied (int)

• violations (int)

• complete (bool)

hits: list[Hit]

Permitted hits, best first, at most k.

strategy: Literal['prefilter', 'postfilter']

The strategy that was actually used.

examined: int

Candidates the authorizer checked.

denied: int

Post-filtered candidates the subject may not read.

violations: int

Pre-filtered hits that failed the re-check. Should always be 0; anything else means the backend did not apply the filter and deserves an alert.

complete: bool

False when wardowl knows results may be missing: post-filtering stopped at max_candidates, or the re-check removed pre-filtered hits. It trusts the backend’s ranking, so it cannot see results an approximate index failed to return.

class wardowl.SearchBackend(*args, **kwargs)

A vector store or search index. Return hits ordered best first.

supports_filters says whether search honours the filter argument. Backends may also provide async def asearch(...) with the same signature; otherwise search runs in a thread.

query is what the caller passed to retrieve, unchanged: a string, or a Query when the caller supplies a vector. Query.of(query) gives a Query either way. A backend that takes only strings still works with every string query.

class wardowl.Redactor(*args, **kwargs)

Changes the text or metadata of hits the subject may read, keeping the same hits in order.

class wardowl.SpanRedactor(field='redactions', replacement='[REDACTED]', *, required=True)

Hides spans of a chunk’s text that the subject may not read.

Each chunk carries, under field in its metadata, a list of spans: {"start": 120, "end": 340, "allowed_groups": ["hr"], "allowed_users": []}, character offsets into the text. A span the subject may not read (not a listed user, not in a listed group, and "*" not listed) is replaced by replacement; overlapping spans are replaced together. The field itself is removed from the metadata returned, since it names who may read what. A malformed list of spans raises ValueError, which the retriever turns into AuthorizationError: a field that is not a list, a span that is not a mapping, offsets that are not two integers with 0 <= start < end, a sha256 that is not a string, or readers that are not lists of strings. Those are wrong whatever the text.

A chunk without the field is hidden whole, its text replaced by replacement: a list lost when the metadata was replaced cannot be told from one never written. Store [] for a chunk with nothing to hide, or pass required=False to show chunks without the field in full.

A chunk whose spans no longer fit its text is hidden whole too: a span that ends past the text, because the text was shortened after the spans were written, or a span whose sha256 (the span_digest of the text it covers) no longer matches the text at its offsets. The offsets may no longer cover what they were written to protect.

Parameters:

• field (str)

• replacement (str)

• required (bool)

wardowl.redaction.span_digest(text)

The sha256 a span may carry: the hex SHA-256 of the UTF-8 text it covers, text[start:end].

Parameters:

text (str)

Return type:

str

class wardowl.PatternRedactor(patterns, *, exempt_groups=None, replacement='[REDACTED {name}]')

Hides text matching patterns, unless the subject belongs to a group exempt from a pattern.

patterns maps a name to a regular expression. exempt_groups maps a name to the groups whose members see that pattern’s matches. Matches are replaced by replacement, in which {name} is the pattern’s name.

Parameters:

• patterns (Mapping[str, str | re.Pattern[str]])

• exempt_groups (Mapping[str, Collection[str]] | None)

• replacement (str)

class wardowl.Reranker(*args, **kwargs)

Reorders the hits a subject may read: a cross-encoder, a hosted rerank API, any model.

rerank receives the query as it was given to retrieve and the permitted hits, best first, and returns some or all of those hits in its order, with its scores. It may also provide async def arerank(...) with the same signature. A plain function with the same signature can be passed instead.

class wardowl.RankFusion(*backends, weights=None, rank_constant=60.0, depth=None)

A backend that searches several backends and fuses their rankings (reciprocal rank fusion).

weights (one per backend, default 1 each) scale each backend’s contribution. rank_constant (default 60, as in the original RRF paper) damps the advantage of the very top ranks. depth is how many results each backend is asked for (default: the k requested). Filters are supported when every backend supports them. Every backend gets the same query: a Query with both text and vector gives keyword backends the text and vector backends the vector, so the query is embedded once, by you, however many vector backends there are.

Parameters:

• backends (SearchBackend)

• weights (Sequence[float] | None)

• rank_constant (float)

• depth (int | None)

Strategy is Literal["auto", "prefilter", "postfilter"].

Subjects, documents, hits and queries

class wardowl.Subject(id, groups=(), attributes=None)

Who is asking: a user id, the groups they belong to, and any attributes your policy needs.

The id and every group are strings. attributes is copied when the subject is made, and the copy cannot be changed: mappings in it are read-only, lists become tuples and sets frozensets. Decisions are cached per subject, so a subject changed afterwards could otherwise be given an answer cached for what it was before.

Parameters:

• id (str)

• groups (frozenset[str])

• attributes (Mapping[str, Any])

id: str

groups: frozenset[str] = frozenset({})

attributes: Mapping[str, Any]

class wardowl.Document(id, text, metadata=<factory>)

A chunk as it is stored: an id, its text, and metadata that may carry permissions.

Parameters:

• id (str)

• text (str)

• metadata (Mapping[str, Any])

id: str

text: str

metadata: Mapping[str, Any]

class wardowl.Hit(id, text, score, metadata=<factory>)

A search result: the chunk plus the backend’s relevance score (higher is better).

Parameters:

• id (str)

• text (str)

• score (float)

• metadata (Mapping[str, Any])

id: str

text: str

score: float

metadata: Mapping[str, Any]

class wardowl.Query(text=None, vector=None)

What to search for: text, a vector you computed, or both.

A plain string is a text query. Pass a Query with a vector to search with an embedding you made yourself: an image’s, one your embedding service returned, or one computed once and shared by several backends. Vector search uses vector when it is given and embeds text otherwise; keyword search uses text. Give both to fuse vector and keyword results with RankFusion.

Parameters:

• text (str | None)

• vector (tuple[float, ...] | None)

classmethod of(query)

query as a Query: a string becomes a text query.

Parameters:

query (str | Query)

Return type:

Query

embedded(embed)

The vector to search with: vector if given, otherwise embed(text).

Parameters:

embed (Callable[[str], Sequence[float]] | None)

Return type:

list[float]

keywords()

The text to match words against; keyword search cannot use a vector.

Return type:

str

class wardowl.InvalidQuery

A query that cannot be searched as asked, through a fault of the query, not the system.

Raised for a k outside what the retriever allows, a query with only a vector for keyword search, only text for a backend with no embed function, or a vector whose length the backend knows is wrong. It is a ValueError, so code catching that keeps working; the HTTP service answers it with 422. A backend of your own can raise it for queries it cannot search.

Filters

Filter is In | And | Or.

class wardowl.In(field, values)

True when the metadata field equals one of values.

For list-valued fields (for example allowed_groups: ["sales", "hr"]) it is true when the field and values share at least one element. A missing field never matches, and neither do metadata values that are not strings or integers.

Parameters:

• field (str)

• values (frozenset[str | int])

class wardowl.And(*clauses)

True when every clause is true.

Parameters:

clauses (tuple[In | And | Or, ...])

class wardowl.Or(*clauses)

True when at least one clause is true.

Parameters:

clauses (tuple[In | And | Or, ...])

wardowl.matches(f, metadata)

Evaluate a filter against one chunk’s metadata.

Parameters:

• f (In | And | Or)

• metadata (Mapping[str, Any])

Return type:

bool

wardowl.filter_to_dict(f)

JSON-friendly form: {“in”: {“field”: …, “values”: […]}}, {“and”: […]}, {“or”: […]}.

Parameters:

f (In | And | Or)

Return type:

dict[str, Any]

wardowl.filter_from_dict(data)

Parse the form produced by filter_to_dict. Anything unexpected raises ValueError, including filters nested more than 100 levels deep.

Parameters:

data (Any)

Return type:

In | And | Or

Authorizers

class wardowl.Authorizer(*args, **kwargs)

check(subject, hits, action='read')

One decision per hit, in the same order.

Parameters:

• subject (Subject)

• hits (Sequence[Hit])

• action (str)

Return type:

list[bool]

filter_for(subject, action='read')

A filter matching exactly the chunks the subject may access, or None.

Parameters:

• subject (Subject)

• action (str)

Return type:

In | And | Or | None

exception wardowl.AuthorizationError

The authorizer could not make a decision. No results are returned.

class wardowl.MetadataACLAuthorizer(*, users_field='allowed_users', groups_field='allowed_groups', everyone='*', actions=('read',))

Permissions stored on each chunk, in two metadata fields.

allowed_users lists user ids and allowed_groups lists group names. A chunk whose allowed_groups contains the everyone marker (default "*") is readable by every subject. Chunks without either field are readable by nobody: missing permissions mean no access.

This is the simplest model and needs no external service. It suits documents whose permissions are copied onto the chunks at ingestion time.

Parameters:

• users_field (str)

• groups_field (str)

• everyone (str)

• actions (Sequence[str])

class wardowl.FunctionAuthorizer(check, filter_for=None)

Adapts your own permission logic.

check(subject, hits, action) -> list[bool] is required. Pass filter_for as well if your rules can be turned into a filter; otherwise results are post-filtered.

Parameters:

• check (Callable[[Subject, Sequence[Hit], str], Sequence[bool]])

• filter_for (Callable[[Subject, str], Filter | None] | None)

class wardowl.TenantAuthorizer(field='tenant', attribute=None)

Chunks are readable only within the subject’s tenant.

A chunk is readable when its metadata field field holds the subject’s attribute attribute (both "tenant" by default). A subject without that attribute raises AuthorizationError, so a request that lost its tenant returns nothing rather than everything. Combine it with fine-grained permissions: AllOf(TenantAuthorizer(), OpenFGAAuthorizer(...)).

Parameters:

• field (str)

• attribute (str | None)

class wardowl.AllOf(*authorizers)

Allows a chunk only when every authorizer allows it.

For example tenant isolation plus fine-grained permissions: AllOf(TenantAuthorizer(), OpenFGAAuthorizer(...)). Authorizers are asked in order, each only about the chunks every earlier one allowed, so put cheap ones first. The pre-filter is the conjunction of every authorizer’s filter, or none when one of them has none; scope_for then gives the conjunction of the filters that do exist, so post-filtering searches only inside them (for example only the subject’s tenant).

Parameters:

authorizers (Authorizer)

class wardowl.AnyOf(*authorizers)

Allows a chunk when at least one authorizer allows it.

For example documents shared directly or through a policy: AnyOf(MetadataACLAuthorizer(), OPAAuthorizer(...)). Authorizers are asked in order, each only about the chunks every earlier one denied. The pre-filter is the disjunction of every authorizer’s filter, or none when one of them has none; scope_for is the disjunction of every authorizer’s filter or scope, or none when one has neither.

Parameters:

authorizers (Authorizer)

class wardowl.CachingAuthorizer(authorizer, *, ttl=30.0, maxsize=100000, shared=None, namespace=None, local_ttl=1.0, clock=<built-in function monotonic>)

Caches another authorizer’s answers per subject and action for ttl seconds.

ttl is the revocation bound: a permission revoked at the source stops granting access at most ttl seconds later. maxsize caps the number of entries in this process, dropping the least recently used first. shared adds a cache all processes share (for example RedisCache), under keys starting with namespace, which it requires; this process then keeps its own copy of each answer for at most local_ttl seconds, which bounds how long an invalidate() made in another process takes to reach this one. Errors and malformed answers are never cached. clock is for tests.

Every CachingAuthorizer with the same shared cache and namespace uses the same answers, whatever authorizer it wraps: give each differently configured authorizer (another policy, store, tenant model or action mapping) a namespace of its own. In one process, a second CachingAuthorizer that wraps an authorizer of another type under a shared cache and namespace already in use raises ValueError; it cannot tell two configurations of one type apart, nor see other processes.

Parameters:

• authorizer (Authorizer)

• ttl (float)

• maxsize (int)

• shared (SharedCache | None)

• namespace (str | None)

• local_ttl (float)

• clock (Callable[[], float])

stats()

What this cache has done since it was created, and how many answers it holds now.

Return type:

CacheStats

invalidate(subject=None)

Drop every cached decision, or those for one subject id, here and in the shared cache.

subject is a Subject or its id; the answers for every subject with that id are dropped, whatever groups and attributes they came with. An answer the authorizer gave before this call, to a retrieval still in progress, is not cached afterwards, here or in the shared cache. Other processes stop using their own copies within local_ttl seconds. A failure to reach the shared cache is raised: the caller asked for the revocation now.

Parameters:

subject (Subject | str | None)

Return type:

None

class wardowl.caching.CacheStats(hits, shared_hits, misses, shared_failures, entries)

What a CachingAuthorizer has done since it was created, from stats(), for monitoring.

Parameters:

• hits (int)

• shared_hits (int)

• misses (int)

• shared_failures (int)

• entries (int)

hits: int

Answers served from this process’s own cache.

shared_hits: int

Answers served from the shared cache.

misses: int

Answers the wrapped authorizer was asked for.

shared_failures: int

Reads and writes of the shared cache that failed; each failed read was treated as a miss.

entries: int

Answers this process holds now.

class wardowl.caching.SharedCache(*args, **kwargs)

A cache every process shares, such as Redis. Keys are strings and values opaque bytes.

get_many(keys)

The value of each key, or None when it is absent.

Parameters:

keys (Sequence[str])

Return type:

list[bytes | None]

set_many(entries, ttl)

Store each entry for ttl seconds.

Parameters:

• entries (Mapping[str, bytes])

• ttl (float)

Return type:

None

delete_prefix(prefix)

Delete every key that starts with prefix.

Parameters:

prefix (str)

Return type:

None

class wardowl.caching.RedisCache(client, *, signing_key=None)

A SharedCache on Redis or a Redis-compatible store (Valkey, Dragonfly), with redis-py.

Install with pip install "wardowl[redis]" and pass a redis.Redis (or RedisCluster) client. Entries expire in Redis after the cache’s ttl.

Anyone who can write to this Redis could otherwise plant a decision. Pass signing_key (random bytes known only to your application, for example 32 from secrets.token_bytes) and every value is signed with HMAC-SHA256 over its key and content: an entry without a valid signature is ignored, so the authorizer is asked instead.

Parameters:

• client (Any)

• signing_key (bytes | None)

class wardowl.authorizers.opa.OPAAuthorizer(url, *, decisions_path='wardowl/decisions', filter_path=None, timeout=2.0, headers=None, retries=1, circuit_breaker=None, ssl_context=None, max_response_bytes=10485760, allow_insecure=False)

Asks an Open Policy Agent server for decisions over its HTTP API.

decisions_path is the rule returning one boolean per resource and filter_path, if set, the rule returning a filter or null. headers are sent with every request, for example an Authorization header. timeout bounds each call in total, retries included; only the DNS lookup of the host is left to the system resolver’s own timeout. retries is how many times a transient failure is retried, circuit_breaker pauses requests to a server that keeps failing, and ssl_context sets the TLS trust and client certificates (your own CA, mutual TLS) of an https:// url; it is refused with http://. An answer larger than max_response_bytes (10 MiB by default) is refused. An Authorization header is refused over http:// to any host but this machine unless allow_insecure=True. Every failure raises AuthorizationError.

Parameters:

• url (str)

• decisions_path (str)

• filter_path (str | None)

• timeout (float)

• headers (Mapping[str, str] | None)

• retries (int)

• circuit_breaker (CircuitBreaker | None)

• ssl_context (ssl.SSLContext | None)

• max_response_bytes (int)

• allow_insecure (bool)

class wardowl.authorizers.openfga.OpenFGAAuthorizer(url, store_id, *, authorization_model_id=None, relations=None, object_type='document', user_type='user', resource_field='doc_id', group_type='group', group_relation='member', contextual_groups=True, consistency=None, list_objects_limit=1000, list_objects_deadline=3.0, batch_size=50, timeout=2.0, headers=None, retries=1, circuit_breaker=None, ssl_context=None, max_response_bytes=10485760, allow_insecure=False)

Asks an OpenFGA store for decisions over its HTTP API.

relations maps actions to relations (default {"read": "viewer"}). Users are <user_type>:<subject id> and objects <object_type>:<resource id>, where the resource id is the metadata field resource_field (or the chunk id when it is None). consistency is "HIGHER_CONSISTENCY", "MINIMIZE_LATENCY" or None (the server’s default). list_objects_limit and list_objects_deadline are the server’s ListObjects result limit and deadline in seconds (None for a server without a deadline): an answer that reaches either may be partial, so it gives no filter. timeout, retries, circuit_breaker, ssl_context, max_response_bytes and allow_insecure work as in OPAAuthorizer. Every failure raises AuthorizationError.

Parameters:

• url (str)

• store_id (str)

• authorization_model_id (str | None)

• relations (Mapping[str, str] | None)

• object_type (str)

• user_type (str)

• resource_field (str | None)

• group_type (str)

• group_relation (str)

• contextual_groups (bool)

• consistency (str | None)

• list_objects_limit (int)

• list_objects_deadline (float | None)

• batch_size (int)

• timeout (float)

• headers (Mapping[str, str] | None)

• retries (int)

• circuit_breaker (CircuitBreaker | None)

• ssl_context (ssl.SSLContext | None)

• max_response_bytes (int)

• allow_insecure (bool)

class wardowl.authorizers.spicedb.SpiceDBAuthorizer(url, token=None, *, permissions=None, resource_type='document', subject_type='user', resource_field='doc_id', group_type='group', group_relation='member', group_subjects=False, consistency=None, caveat_context=None, lookup_limit=1000, batch_size=100, timeout=2.0, headers=None, retries=1, circuit_breaker=None, ssl_context=None, max_response_bytes=10485760, allow_insecure=False)

Asks SpiceDB for decisions over its HTTP API.

token is SpiceDB’s preshared key (sent as a bearer token). permissions maps actions to SpiceDB permissions (default {"read": "view"}). Resources are <resource_type>:<resource id> and subjects <subject_type>:<subject id>, plus <group_type>:<group>#<group_relation> for each of the subject’s groups when group_subjects is true (False by default; never with a permission that excludes or intersects, as the module docstring explains). caveat_context(subject) returns the context for caveated relationships. timeout, retries, circuit_breaker, ssl_context, max_response_bytes and allow_insecure work as in OPAAuthorizer: a token sent over http:// to a host other than this machine needs allow_insecure=True. Every failure raises AuthorizationError.

Parameters:

• url (str)

• token (str | None)

• permissions (Mapping[str, str] | None)

• resource_type (str)

• subject_type (str)

• resource_field (str | None)

• group_type (str)

• group_relation (str)

• group_subjects (bool)

• consistency (Mapping[str, Any] | None)

• caveat_context (Callable[[Subject], Mapping[str, Any]] | None)

• lookup_limit (int)

• batch_size (int)

• timeout (float)

• headers (Mapping[str, str] | None)

• retries (int)

• circuit_breaker (CircuitBreaker | None)

• ssl_context (ssl.SSLContext | None)

• max_response_bytes (int)

• allow_insecure (bool)

class wardowl.authorizers.cerbos.CerbosAuthorizer(url, *, actions=None, resource_kind='document', resource_field=None, roles=('user',), policy_version=None, scope=None, batch_size=50, timeout=2.0, headers=None, retries=1, circuit_breaker=None, ssl_context=None, max_response_bytes=10485760, allow_insecure=False)

Asks a Cerbos PDP for decisions over its HTTP API.

actions maps wardowl actions to Cerbos actions (default {"read": "read"}). roles is a list of roles for every principal, or a function of the subject (default ["user"]). Resources are of kind resource_kind, with the chunk id as id (or resource_field from the metadata) and the chunk’s metadata as attributes. policy_version and scope are passed to both principal and resources. timeout, retries, circuit_breaker, ssl_context, max_response_bytes and allow_insecure work as in OPAAuthorizer. Every failure raises AuthorizationError.

Parameters:

• url (str)

• actions (Mapping[str, str] | None)

• resource_kind (str)

• resource_field (str | None)

• roles (Sequence[str] | Callable[[Subject], Sequence[str]])

• policy_version (str | None)

• scope (str | None)

• batch_size (int)

• timeout (float)

• headers (Mapping[str, str] | None)

• retries (int)

• circuit_breaker (CircuitBreaker | None)

• ssl_context (ssl.SSLContext | None)

• max_response_bytes (int)

• allow_insecure (bool)

wardowl.authorizers.cerbos.plan_to_filter(plan)

Translate a PlanResources filter into a wardowl filter, or None when it cannot be expressed exactly (the retriever then post-filters).

Parameters:

plan (Mapping[str, Any])

Return type:

In | And | Or | None

class wardowl.authorizers.cedar.CedarAuthorizer(policies, *, entities=(), actions=None, principal_type='User', group_type='Group', resource_type='Document', schema=None, batch_size=100)

Evaluates Cedar policies in this process, with cedarpy.

policies is the policy text. entities adds entities to every request, such as a group hierarchy or the folders documents belong to: a list of Cedar entity JSON objects, or a function of the subject returning them. An entity for one of the subject’s groups, with its parent groups say, replaces the empty one wardowl adds; entities must not repeat the principal or the chunks. actions maps wardowl actions to Cedar action ids (default {"read": "read"}, so Action::"read"). schema, when given, is passed to Cedar, which then validates requests and entities against it. batch_size is how many chunks one Cedar call decides.

Parameters:

• policies (str)

• entities (Sequence[Entity] | Callable[[Subject], Sequence[Entity]])

• actions (Mapping[str, str] | None)

• principal_type (str)

• group_type (str)

• resource_type (str)

• schema (str | Mapping[str, Any] | None)

• batch_size (int)

class wardowl.authorizers.casbin.CasbinAuthorizer(enforcer, *, actions=None, resource_field='doc_id', group_subjects=False, group_prefix='group:', request=None)

Decides each chunk with a Casbin enforcer.

enforcer is a casbin.Enforcer (or SyncedEnforcer, to share one across threads that also change policies). actions maps wardowl actions to Casbin actions (default {"read": "read"}). With group_subjects=True, groups are asked about as group_prefix + the group, so policies for groups must use that prefix.

Parameters:

• enforcer (Any)

• actions (Mapping[str, str] | None)

• resource_field (str | None)

• group_subjects (bool)

• group_prefix (str)

• request (Request | None)

class wardowl.authorizers.keto.KetoAuthorizer(url, *, namespace='Document', actions=None, resource_field='doc_id', group_subjects=False, group_namespace='Group', group_relation='members', batch_size=10, timeout=2.0, headers=None, retries=1, circuit_breaker=None, ssl_context=None, max_response_bytes=10485760, allow_insecure=False)

Asks an Ory Keto server for relationship checks over its HTTP read API.

actions maps wardowl actions to Keto relations (default {"read": "view"}). Objects live in namespace and come from resource_field. batch_size is how many tuples one request checks: 10 by default, because Keto refuses a batch check of more than its limit.max_batch_check_size tuples, 10 unless configured otherwise. Raise both together. timeout, retries, circuit_breaker, ssl_context, max_response_bytes and allow_insecure work as in OPAAuthorizer. Every failure raises AuthorizationError.

Parameters:

• url (str)

• namespace (str)

• actions (Mapping[str, str] | None)

• resource_field (str | None)

• group_subjects (bool)

• group_namespace (str)

• group_relation (str)

• batch_size (int)

• timeout (float)

• headers (Mapping[str, str] | None)

• retries (int)

• circuit_breaker (CircuitBreaker | None)

• ssl_context (ssl.SSLContext | None)

• max_response_bytes (int)

• allow_insecure (bool)

class wardowl.authorizers.CircuitBreaker(failures=5, reset_after=30.0, *, clock=<built-in function monotonic>)

Pauses requests to a permission service that keeps failing.

After failures consecutive failed requests the breaker opens, and every call raises AuthorizationError, caused by CircuitOpenError, at once, without contacting the service, for reset_after seconds. Then one request is let through as a probe: if it succeeds the breaker closes, if it fails the breaker opens again. A failure is a request that got no HTTP response, or a 5xx one; any other response shows that the service is up. Retrieval fails closed while the breaker is open, as it would if every request failed, but a struggling service is not flooded with requests.

One breaker can be shared by several authorizers that call the same service.

Parameters:

• failures (int)

• reset_after (float)

• clock (Callable[[], float])

property is_open: bool

True while requests are paused, for monitoring.

class wardowl.authorizers.CircuitOpenError

The cause of the AuthorizationError an open CircuitBreaker raises: the request was refused without contacting the service. RetrievalEvent.cause is then “CircuitOpenError”.

Permission sync

See Google Drive.

class wardowl.sync.Access(users=(), groups=(), everyone=False, expires_at=None)

Who may read a document: these users, the members of these groups, or everyone.

Access() grants nobody: the safe answer when a source says nothing. expires_at, a timezone-aware time, is when the earliest grant in the answer ends: from then on the answer may grant more than the source does, so the document must be synced again before it.

Parameters:

• users (frozenset[str])

• groups (frozenset[str])

• everyone (bool)

• expires_at (datetime | None)

metadata(*, users_field='allowed_users', groups_field='allowed_groups', everyone='*', expires_field='access_expires_at')

The permission fields for MetadataACLAuthorizer, with its default field names and marker.

Both fields are always present, so writing them replaces every earlier grant. A group named like the everyone marker raises ValueError: stored, it would let everyone read. When expires_at is set, expires_field holds it too, in UTC to the second, rounded down ("2026-10-17T12:00:00Z"), so stored expiries sort as text.

Parameters:

• users_field (str)

• groups_field (str)

• everyone (str)

• expires_field (str)

Return type:

dict[str, Any]

wardowl.sync.with_access(metadata, access, *, users_field='allowed_users', groups_field='allowed_groups', everyone='*', expires_field='access_expires_at')

A chunk’s metadata with its permission fields replaced by access, and every other key kept.

An expiry written by an earlier sync is removed when access has none.

Parameters:

• metadata (Mapping[str, Any])

• access (Access)

• users_field (str)

• groups_field (str)

• everyone (str)

• expires_field (str)

Return type:

dict[str, Any]

class wardowl.sync.google_drive.GoogleDriveACL(service, *, link_sharing=False, domain_prefix='domain:', domains=None, expiring_grants=False)

Reads who may read Drive files, through a Drive v3 service.

service is googleapiclient.discovery.build("drive", "v3", credentials=...). Its credentials need the drive.readonly scope (shared drives’ settings need more than drive.metadata.readonly) and must see each file’s sharing and each shared drive’s settings.

domains lists your organization’s domains. When given, grants outside them count for nothing. A file in a shared drive that is open only to its domain’s users needs them: reading its access without them raises ValueError. expiring_grants=True counts grants with an expiry time until they expire (see drive_access).

A file that is gone, or that the credentials can no longer see (HTTP 404, or 403 with the reason notFound or insufficientFilePermissions), is readable by nobody: access returns Access(), so deleting a file or unsharing it from the indexer revokes access. Other errors from the API, such as rate limits and server errors, are raised: a document whose permissions could not be read should keep the permissions it has, not lose or gain any.

Parameters:

• service (Any)

• link_sharing (bool)

• domain_prefix (str)

• domains (Iterable[str] | None)

• expiring_grants (bool)

permissions(file_id)

Every permission on the file, across all pages.

Parameters:

file_id (str)

Return type:

list[Mapping[str, Any]]

access(file_id)

Who may read the file, within its shared drive’s restrictions; nobody when it is gone.

Parameters:

file_id (str)

Return type:

Access

wardowl.sync.google_drive.drive_access(permissions, *, link_sharing=False, domain_prefix='domain:', members_only=False, domains=None, expiring_grants=False, now=None)

Who may read a file, from the permission resources permissions.list returns for it.

Ask for the fields in FIELDS. link_sharing=True counts sharing by link as access for the whole domain or for everyone. A domain grant gives the group domain_prefix + the domain. members_only=True counts only grants to a shared drive’s members. When domains is given, only grants inside those domains count, and an “anyone” grant gives each of their domain groups instead of everyone. A grant with an expiry time counts only with expiring_grants=True, until it expires, and the answer’s expires_at is then the earliest expiry among the grants that count. Expiry times are compared with now, the current time by default.

Parameters:

• permissions (Iterable[Mapping[str, Any]])

• link_sharing (bool)

• domain_prefix (str)

• members_only (bool)

• domains (Iterable[str] | None)

• expiring_grants (bool)

• now (datetime | None)

Return type:

Access

wardowl.sync.google_drive.FIELDS = 'nextPageToken,permissions(id,type,role,emailAddress,domain,allowFileDiscovery,deleted,expirationTime,view,permissionDetails,inheritedPermissionsDisabled)'

The fields GoogleDriveACL asks Drive for: everything drive_access reads.

wardowl.sync.google_drive.READ_ROLES = frozenset({'commenter', 'fileOrganizer', 'organizer', 'owner', 'reader', 'writer'})

Every Drive role that can read a file’s content.

Events, audit and tracing

class wardowl.RetrievalEvent(subject_id, action, k, strategy, returned_ids=(), examined=0, denied=0, violations=0, complete=False, duration_ms=0.0, error=None, query_hmac=None, started_at=0.0, search_ms=0.0, authorize_ms=0.0, searches=0, rerank_ms=0.0, reranked=0, cause=None)

One retrieval, for audit logs and monitoring. Never contains query or chunk text.

Parameters:

• subject_id (str)

• action (str)

• k (int)

• strategy (str | None)

• returned_ids (tuple[str, ...])

• examined (int)

• denied (int)

• violations (int)

• complete (bool)

• duration_ms (float)

• error (str | None)

• query_hmac (str | None)

• started_at (float)

• search_ms (float)

• authorize_ms (float)

• searches (int)

• rerank_ms (float)

• reranked (int)

• cause (str | None)

subject_id: str

action: str

k: int

strategy: str | None

The strategy that ran, or None if the call failed before choosing one.

returned_ids: tuple[str, ...] = ()

examined: int = 0

denied: int = 0

violations: int = 0

complete: bool = False

duration_ms: float = 0.0

error: str | None = None

The exception type when the call failed, for example “AuthorizationError”.

query_hmac: str | None = None

started_at: float = 0.0

Unix time when the call started.

search_ms: float = 0.0

Time spent in the backend, over every search of this call.

authorize_ms: float = 0.0

Time spent in the authorizer: filters, scopes and checks.

searches: int = 0

Backend searches made: one when pre-filtering, more when post-filtering fetched again.

rerank_ms: float = 0.0

Time spent in the reranker.

reranked: int = 0

Permitted chunks given to the reranker; 0 when there is none.

cause: str | None = None

The type of the exception behind error, when there is one: for an AuthorizationError, what the call to the permission service raised, such as “TimeoutError” or “ConnectionRefusedError”, or “CircuitOpenError” when an open CircuitBreaker refused the call. None when the error has no cause, as for a malformed answer.

to_dict()

Return type:

dict[str, Any]

class wardowl.JsonLinesAudit(stream)

Writes one JSON object per retrieval to a text stream (a file, stdout, a socket wrapper).

It records who asked, what came back and what was denied, never the query or chunk text.

Parameters:

stream (IO[str])

wardowl.events.query_fingerprint(query, key)

HMAC-SHA256 of the query under key, in hex. Equal queries give equal fingerprints.

A Query with text is fingerprinted by its text, so the same question matches whether or not a vector came with it. A vector-only query is fingerprinted by its vector.

Parameters:

• query (str | Query)

• key (bytes)

Return type:

str

class wardowl.otel.OpenTelemetryObserver(tracer=None, *, record_subject=False)

Records each RetrievalEvent as a span named wardowl.retrieve.

tracer defaults to the wardowl tracer of the global tracer provider. Spans carry counts, outcomes and the time spent in the backend and the authorizer, never query or chunk text; the subject id only with record_subject=True.

Parameters:

• tracer (Any)

• record_subject (bool)

class wardowl.otel.OpenTelemetryMetrics(meter=None)

Records each RetrievalEvent as OpenTelemetry metrics.

meter defaults to the wardowl meter of the global meter provider. Every instrument is broken down by wardowl.action and wardowl.strategy only, plus the outcome and error.type where they apply. These have a handful of values each, so the metrics stay cheap to store; subjects, queries and chunks are never recorded.

• gen_ai.client.operation.duration (histogram, s): each retrieval, with gen_ai.operation.name = "retrieval".

• wardowl.retrievals (counter): retrievals by wardowl.outcome, which is complete, incomplete or error.

• wardowl.search.duration, wardowl.authorize.duration (histograms, s): time spent in the backend and in the authorizer during each retrieval.

• wardowl.rerank.duration (histogram, s): time spent in the reranker, for retrievals that reranked.

• wardowl.chunks.returned, wardowl.chunks.examined, wardowl.chunks.denied (counters): chunks returned, checked by the authorizer, and withheld.

• wardowl.violations (counter): pre-filtered chunks the re-check rejected. Alert on any increase.

Parameters:

meter (Any)

An observer is any callable that takes a RetrievalEvent.

Backends

class wardowl.InMemoryBackend(documents=())

Keeps documents in memory and ranks them by keyword overlap.

For tests, examples and small corpora. Its filtering follows matches exactly, which makes it the reference the other backends are tested against.

Parameters:

documents (Iterable[Document])

add(documents)

Store chunks, replacing any with the same id.

Unlike the vector backends, it takes no embeddings: it ranks by the words a chunk shares with the query and never uses a vector, so it has nowhere to keep one.

Parameters:

documents (Iterable[Document])

Return type:

None

set_metadata(metadata)

Replace the metadata of stored chunks, keeping their text. Ids not stored are ignored.

Parameters:

metadata (Mapping[str, Mapping[str, Any]])

Return type:

None

delete(ids)

Remove chunks by id. Ids not stored are ignored.

Parameters:

ids (Iterable[str])

Return type:

None

exception wardowl.backends.NothingMatches(message='the filter matches nothing, so there is no query to send')

The filter can match no chunk (an In with no values, or an And holding one), so there is no query to send.

The public filter translators (filter_to_where, filter_to_query, filter_to_qdrant, …) raise it rather than return None or {}, which store clients read as “no filter”, that is, as “match everything”. The backends catch it and return no hits without asking the store; code that sends a translated filter itself should do the same.

Parameters:

message (str)

Return type:

None

class wardowl.backends.pgvector.PgVectorBackend(pool, embed=None, *, table='wardowl_chunks', iterative_scan='relaxed_order', ef_search=None, mode='vector', text_search='english', async_pool=None)

Cosine-similarity search over a pgvector table, with the filter in the WHERE clause.

pool is a psycopg ConnectionPool; embed turns text into a vector (your embedding model). Without embed, search with Query(vector=...) and pass embeddings to add. Scores are cosine similarity, 1 - cosine distance. Pass async_pool (an AsyncConnectionPool) for a native asearch; otherwise async calls run search in a thread. ef_search sets pgvector’s hnsw.ef_search (1 to 1000) for each search: higher finds more of the true nearest neighbours and takes longer. None leaves the server’s setting (40 by default).

mode="keyword" searches the text with PostgreSQL full-text search instead (websearch_to_tsquery in the text_search configuration, ranked by ts_rank_cd), under the same permission condition, for hybrid search with RankFusion; embed may then be None. Create the matching index with create_schema(..., text_search=...).

Parameters:

• pool (Any)

• embed (Embed | None)

• table (str)

• iterative_scan (str | None)

• ef_search (int | None)

• mode (str)

• text_search (str)

• async_pool (Any)

add(documents, embeddings=None)

Insert or replace chunks. Embeddings are computed with embed unless given.

Parameters:

• documents (Iterable[Document])

• embeddings (Iterable[Sequence[float]] | None)

Return type:

None

set_metadata(metadata)

Replace the metadata of stored chunks, keeping their text and vectors. Ids not stored are ignored.

Parameters:

metadata (Mapping[str, Mapping[str, Any]])

Return type:

None

wardowl.backends.pgvector.create_schema(pool, dim, *, table='wardowl_chunks', hnsw=True, text_search=None)

Create the extension, table and indexes if they do not exist. pool is a psycopg ConnectionPool.

text_search (a text search configuration, such as "english") also creates the full-text GIN index that PgVectorBackend(mode="keyword", text_search=...) uses; pass the same name to both.

Parameters:

• pool (Any)

• dim (int)

• table (str)

• hnsw (bool)

• text_search (str | None)

Return type:

None

wardowl.backends.pgvector.filter_to_sql(f)

Translate a filter into a condition on the metadata jsonb column, with bound parameters.

Each value becomes two containment tests: scalar equality ({"field": v}) and array membership ({"field": [v]}), which the GIN index on metadata serves. Above MANY_VALUES values of one type (a subject in thousands of groups) the test takes two array parameters instead of two per value, so the query stays small and far below PostgreSQL’s limit of 65,535 parameters. jsonb compares strings, and keeps strings, numbers and booleans apart, as the reference semantics in wardowl.filters.matches do; but it compares numbers by value, so 1.0 would contain 1. Integer values therefore also require the stored number to read as that integer (_EXACT_INTEGERS), so a float in the metadata never matches, as in the reference.

Parameters:

f (In | And | Or | None)

Return type:

tuple[str, list[Any]]

class wardowl.backends.chroma.ChromaBackend(collection, embed=None, *, batch_size=None)

Search a Chroma collection with the permission filter applied by Chroma.

If embed is given, queries and added documents are embedded with it; otherwise the collection’s own embedding function is used. A Query with a vector is searched with that vector. Scores are 1 - distance, so create the collection with cosine distance for scores in [-1, 1].

add, delete and set_metadata send at most batch_size records per call, since Chroma refuses larger batches. By default it is the client’s own maximum (get_max_batch_size(), about 5,400 for a local client), or 1,000 when the collection does not report one. Each write is checked whole before the first batch is sent; a store that fails part way leaves the batches before it written.

Parameters:

• collection (Any)

• embed (Embed | None)

• batch_size (int | None)

set_metadata(metadata)

Replace the metadata of stored chunks, keeping their text and vectors. Ids not stored are ignored.

An update, which Chroma applies to stored records only: a chunk deleted meanwhile is not recreated, and its text and vector are never read and written back. Keys of the stored metadata that the new one lacks are removed (set to None), so no old list key survives. The read and the update are two calls, so an add between them can leave list keys the update does not know of; search checks each hit’s JSON copy against the filter, so such a key never makes a chunk match.

Parameters:

metadata (Mapping[str, Mapping[str, Any]])

Return type:

None

wardowl.backends.chroma.filter_to_where(f)

Translate a filter into a Chroma where document.

In becomes $in on the scalar field (one clause per value type, since Chroma’s $in lists are single-typed) plus an equality test on each value’s list key. Raises NothingMatches when nothing can match, as Chroma reads an empty where as “match everything”, and ValueError for a field name containing :: or equal to wardowl_metadata, which would address the stored list keys or the JSON copy rather than a metadata field.

Parameters:

f (In | And | Or)

Return type:

dict[str, Any]

class wardowl.backends.elasticsearch.ElasticsearchBackend(client, index, embed=None, *, vector_field='embedding', content_field='content', metadata_field='metadata', num_candidates=100, refresh='wait_for', mode='vector', async_client=None)

Search an Elasticsearch index with the permission filter applied during the kNN search.

embed turns text into a vector; without it, search with Query(vector=...) and pass embeddings to add. num_candidates is how many nearest neighbours each shard considers (at least k, at most 10,000): higher finds more of the true nearest chunks and takes longer. Scores are Elasticsearch’s: (1 + cosine) / 2 for a cosine field. refresh is passed to bulk writes ("wait_for" by default, so written chunks are searchable when add returns). mode="keyword" searches the text with BM25 instead, under the same permission filter, for hybrid search with RankFusion; embed may then be None.

Parameters:

• client (Any)

• index (str)

• embed (Embed | None)

• vector_field (str)

• content_field (str)

• metadata_field (str)

• num_candidates (int)

• refresh (bool | str)

• mode (str)

• async_client (Any)

add(documents, embeddings=None)

Insert or replace chunks. Embeddings are computed with embed unless given.

Parameters:

• documents (Iterable[Document])

• embeddings (Iterable[Sequence[float]] | None)

Return type:

None

set_metadata(metadata)

Replace the metadata of stored chunks, keeping their text and vectors. Ids not stored are ignored.

Parameters:

metadata (Mapping[str, Mapping[str, Any]])

Return type:

None

wardowl.backends.elasticsearch.create_index(client, index, dim, *, index_options=None, settings=None, vector_field='embedding', content_field='content', metadata_field='metadata')

Create the index if it does not exist, with a cosine dense_vector field.

Elasticsearch quantizes dense vectors by default (int8_hnsw below 384 dimensions, bbq_hnsw from 384 in 9.1 and later), which saves memory and costs some recall. Pass index_options={"type": "hnsw"} to keep full-precision vectors.

Parameters:

• client (Any)

• index (str)

• dim (int)

• index_options (Mapping[str, Any] | None)

• settings (Mapping[str, Any] | None)

• vector_field (str)

• content_field (str)

• metadata_field (str)

Return type:

None

wardowl.backends.elasticsearch.permission_terms(metadata)

The value of wardowl_terms for a chunk with this metadata.

Parameters:

metadata (Mapping[str, Any])

Return type:

list[str]

class wardowl.backends.opensearch.OpenSearchBackend(client, index, embed=None, *, vector_field='embedding', content_field='content', metadata_field='metadata', num_candidates=100, refresh='wait_for', mode='vector', async_client=None)

Search an OpenSearch index with the permission filter applied during the k-NN search.

embed turns text into a vector; without it, search with Query(vector=...) and pass embeddings to add. Scores are OpenSearch’s: (1 + cosine) / 2 for a cosinesimil field. refresh is passed to bulk writes ("wait_for" by default, so written chunks are searchable when add returns). num_candidates is not used by OpenSearch’s knn query. With the Lucene engine (the default), the query’s k sets how widely the graph is searched; the index setting index.knn.algo_param.ef_search applies to the Faiss engine only. mode="keyword" searches the text with BM25 instead, under the same permission filter, for hybrid search with RankFusion; embed may then be None.

Parameters:

• client (Any)

• index (str)

• embed (Embed | None)

• vector_field (str)

• content_field (str)

• metadata_field (str)

• num_candidates (int)

• refresh (bool | str)

• mode (str)

• async_client (Any)

add(documents, embeddings=None)

Insert or replace chunks. Embeddings are computed with embed unless given.

Parameters:

• documents (Iterable[Document])

• embeddings (Iterable[Sequence[float]] | None)

Return type:

None

set_metadata(metadata)

Replace the metadata of stored chunks, keeping their text and vectors. Ids not stored are ignored.

Parameters:

metadata (Mapping[str, Mapping[str, Any]])

Return type:

None

wardowl.backends.opensearch.create_index(client, index, dim, *, engine='lucene', parameters=None, settings=None, vector_field='embedding', content_field='content', metadata_field='metadata')

Create the index if it does not exist, with a cosine knn_vector field (HNSW).

engine is "lucene" (the default) or "faiss"; both apply filters during the search. parameters are the HNSW method parameters, for example {"m": 16, "ef_construction": 128}.

Parameters:

• client (Any)

• index (str)

• dim (int)

• engine (str)

• parameters (Mapping[str, Any] | None)

• settings (Mapping[str, Any] | None)

• vector_field (str)

• content_field (str)

• metadata_field (str)

Return type:

None

wardowl.backends.opensearch.permission_terms(metadata)

The value of wardowl_terms for a chunk with this metadata.

Parameters:

metadata (Mapping[str, Any])

Return type:

list[str]

class wardowl.backends.qdrant.QdrantBackend(client, collection, embed=None, *, mode='vector', text_search=False, content_key='page_content', metadata_key='metadata', vector_name=None, search_params=None, async_client=None)

Search a Qdrant collection with the permission filter applied by Qdrant.

embed turns text into a vector (your embedding model). vector_name selects a named vector, search_params (a qdrant_client.models.SearchParams) tunes the search, for example SearchParams(hnsw_ef=128) for higher recall or SearchParams(exact=True) for brute force. Scores are Qdrant’s: cosine similarity for a cosine collection. Pass async_client (an AsyncQdrantClient) for a native asearch; otherwise async calls run search in a thread. Without embed, search with Query(vector=...) and pass embeddings to add.

mode="keyword" ranks by BM25 instead, on the sparse vector wardowl_bm25 of a collection created with text_search=True, under the same query_filter. Scores are BM25, higher for better matches. A vector backend writes that sparse vector too when given text_search=True, so one collection serves both halves of hybrid search. A keyword backend without embed writes chunks with only the sparse vector, which keyword search finds and vector search does not.

Parameters:

• client (Any)

• collection (str)

• embed (Embed | None)

• mode (str)

• text_search (bool)

• content_key (str)

• metadata_key (str)

• vector_name (str | None)

• search_params (Any)

• async_client (Any)

add(documents, embeddings=None)

Insert or replace chunks. Embeddings are computed with embed unless given.

Parameters:

• documents (Iterable[Document])

• embeddings (Iterable[Sequence[float]] | None)

Return type:

None

set_metadata(metadata)

Replace the metadata of stored chunks, keeping their text and vectors. Ids not stored are ignored.

Each change sets only the metadata key of the payload (Qdrant replaces a top-level key’s whole value, so no key of the old metadata survives), on the points that hold that id. It never writes the text back, and a point deleted meanwhile is not recreated.

Parameters:

metadata (Mapping[str, Mapping[str, Any]])

Return type:

None

delete(ids)

Remove chunks by id: the points holding each id, and no other. Ids not stored are ignored.

Parameters:

ids (Iterable[str])

Return type:

None

wardowl.backends.qdrant.create_collection(client, name, dim, *, keyword_fields=('allowed_users', 'allowed_groups'), metadata_key='metadata', vector_name=None, text_search=False)

Create a cosine-distance collection if it does not exist, and keyword indexes on the permission fields (metadata.allowed_users and metadata.allowed_groups by default) and on wardowl_id, which set_metadata and delete filter on (a collection in strict mode refuses filtered updates on fields without an index).

text_search=True also creates the sparse vector wardowl_bm25 with Qdrant’s IDF modifier, for mode="keyword". Qdrant can add it to an existing collection only by recreating it.

Parameters:

• client (Any)

• name (str)

• dim (int)

• keyword_fields (Sequence[str])

• metadata_key (str)

• vector_name (str | None)

• text_search (bool)

Return type:

None

wardowl.backends.qdrant.filter_to_qdrant(f, metadata_key='metadata')

Translate a filter into a Qdrant Filter. Raises NothingMatches when nothing can match.

In becomes MatchAny on metadata.<field>, one condition per value type (strings and integers are never mixed in one list), joined with should. An In with no values matches nothing; that is decided here instead of being left to Qdrant’s handling of empty lists, and a filter that can match nothing raises rather than return something Qdrant reads as no filter.

Parameters:

• f (Filter)

• metadata_key (str)

Return type:

models.Filter

wardowl.backends.qdrant.point_id(document_id)

The Qdrant point id add writes a document id to: a UUID derived from its SHA-256.

Every id is hashed, one that is already a UUID too, so no two document ids share a point.

Parameters:

document_id (str)

Return type:

str

wardowl.backends.qdrant.bm25_terms(text)

A chunk’s BM25 term weights, as the indices and values of the sparse vector wardowl_bm25.

Words are runs of letters, digits and underscores, lowercased; every word is kept. Each word’s index is its CRC-32, and its value BM25’s term-frequency part, tf * (k1 + 1) / (tf + k1 * (1 - b + b * length / 256)) with k1 = 1.2 and b = 0.75. Qdrant multiplies in the inverse document frequency at search time. Use it to fill the field for points written by other tools.

Parameters:

text (str)

Return type:

tuple[list[int], list[float]]

class wardowl.backends.milvus.MilvusBackend(client, collection, embed=None, *, mode='vector', vector_field='embedding', content_field='content', metadata_field='metadata', ef=100, search_params=None, consistency_level='Strong', async_client=None)

Vector search on Milvus, with the permission filter applied inside the search.

embed turns text into a vector (your embedding model). ef is HNSW’s search breadth: higher finds more of the true nearest neighbours, more slowly; it is raised to k when k is larger, as Milvus requires. search_params replaces the search parameters entirely, for another index type. consistency_level is passed to every search (“Strong” by default, so writes are visible at once; “Bounded” is faster when a short delay is acceptable). Pass async_client (an AsyncMilvusClient) for a native asearch; otherwise async calls run search in a thread. Without embed, search with Query(vector=...) and pass embeddings to add.

mode="keyword" searches the BM25 sparse vector of a collection created with text_search=True instead, under the same permission filter, with the query’s text. Scores are Milvus’s BM25, higher for better matches; search_params is passed only when given. Milvus requires every chunk’s vector, so add still needs embed or embeddings in this mode.

Parameters:

• client (Any)

• collection (str)

• embed (Embed | None)

• mode (str)

• vector_field (str)

• content_field (str)

• metadata_field (str)

• ef (int)

• search_params (dict[str, Any] | None)

• consistency_level (str)

• async_client (Any)

add(documents, embeddings=None)

Insert or replace chunks. Embeddings are computed with embed unless given.

Parameters:

• documents (Iterable[Document])

• embeddings (Iterable[Sequence[float]] | None)

Return type:

None

set_metadata(metadata)

Replace the metadata of stored chunks, keeping their text and vectors. Ids not stored are ignored.

The rows found are changed with a partial update (Milvus 2.6.2 or newer) of the metadata and terms alone, which replaces the value of each field it names: the text and vector are never written back. A row deleted between the lookup and the update is not recreated, since Milvus refuses to insert a row without its vector; the call then raises, and can be repeated.

Parameters:

metadata (Mapping[str, Mapping[str, Any]])

Return type:

None

wardowl.backends.milvus.create_collection(client, name, dim, *, metric='COSINE', vector_field='embedding', content_field='content', metadata_field='metadata', max_id_length=512, max_content_length=65535, max_terms=4096, max_term_length=1024, hnsw=None, text_search=False)

Create a collection MilvusBackend can use, with an HNSW index on the vector and an inverted index on the permission terms.

max_terms caps the tagged terms per chunk: one per string or integer in its metadata, so a chunk shared with many groups needs more. max_term_length caps one term, which is the field name, two characters and the value.

text_search=True analyzes the text with Milvus’s standard analyzer, which keeps every word, and adds a BM25 function that fills a sparse vector, wardowl_sparse, with an inverted index, for mode="keyword". Milvus can only add it when the collection is created.

Parameters:

• client (Any)

• name (str)

• dim (int)

• metric (str)

• vector_field (str)

• content_field (str)

• metadata_field (str)

• max_id_length (int)

• max_content_length (int)

• max_terms (int)

• max_term_length (int)

• hnsw (dict[str, int] | None)

• text_search (bool)

Return type:

None

wardowl.backends.milvus.filter_to_expression(f)

Translate a filter into a Milvus filter template and its parameters.

Returns (expression, params) for search(filter=expression, filter_params=params). Each In becomes ARRAY_CONTAINS_ANY(wardowl_terms, {pN}) with the tagged terms in params["pN"]. Raises NothingMatches when nothing can match, as Milvus reads an empty filter as “match everything”.

Parameters:

f (In | And | Or)

Return type:

tuple[str, dict[str, list[str]]]

wardowl.backends.milvus.permission_terms(metadata)

The value of wardowl_terms for a chunk with this metadata, for rows written by other tools.

Parameters:

metadata (Mapping[str, Any])

Return type:

list[str]

class wardowl.backends.mongodb.MongoDBBackend(collection, embed=None, *, mode='vector', index='wardowl', text_index=None, vector_field='embedding', content_field='content', metadata_field='metadata', num_candidates=100, exact=False, wait_timeout=60.0, async_collection=None)

Vector search on MongoDB Atlas Vector Search, with the permission filter applied inside the search.

embed turns text into a vector (your embedding model). index names the vector search index. num_candidates is how many neighbours the approximate search considers (raised to k, at most 10,000): higher finds more of the true nearest chunks and takes longer. exact=True searches every chunk instead. wait_timeout bounds how long add and delete wait for the index. Pass async_collection (the same collection from an AsyncMongoClient) for a native asearch; otherwise async calls run search in a thread. Without embed, search with Query(vector=...) and pass embeddings to add.

mode="keyword" searches the text with Atlas Search (BM25) instead, in the index named text_index ("wardowl_text" by default; see create_text_index), under the same permission filter. Scores are Atlas Search’s, higher for better matches. Such a backend needs no embed: without one, add writes chunks with no vector, which keyword search still finds. A vector backend given text_index also waits for that index after each write, so writes through it are searchable by both halves of hybrid search when they return.

Parameters:

• collection (Any)

• embed (Embed | None)

• mode (str)

• index (str)

• text_index (str | None)

• vector_field (str)

• content_field (str)

• metadata_field (str)

• num_candidates (int)

• exact (bool)

• wait_timeout (float)

• async_collection (Any)

add(documents, embeddings=None, *, wait=True)

Insert or replace chunks. Embeddings are computed with embed unless given.

With wait (the default), returns once the search index shows every chunk as written, or raises TimeoutError after wait_timeout seconds.

Parameters:

• documents (Iterable[Document])

• embeddings (Iterable[Sequence[float]] | None)

• wait (bool)

Return type:

None

set_metadata(metadata, *, wait=True)

Replace the metadata of stored chunks, keeping their text and vectors. Ids not stored are ignored.

With wait (the default), returns once the search index shows the new metadata, or raises TimeoutError after wait_timeout seconds.

Parameters:

• metadata (Mapping[str, Mapping[str, Any]])

• wait (bool)

Return type:

None

delete(ids, *, wait=True)

Delete chunks by id; with wait, returns once the search index no longer returns them.

Parameters:

• ids (Iterable[str])

• wait (bool)

Return type:

None

wardowl.backends.mongodb.create_search_index(collection, dim, *, name='wardowl', vector_field='embedding', similarity='cosine', wait=True, timeout=120.0)

Create the vector search index MongoDBBackend uses, creating the collection if needed.

The index holds the vector and, as filter fields, the permission terms, the write marker and _id. With wait, it returns once the index is queryable, or raises TimeoutError.

Parameters:

• collection (Any)

• dim (int)

• name (str)

• vector_field (str)

• similarity (str)

• wait (bool)

• timeout (float)

Return type:

None

wardowl.backends.mongodb.create_text_index(collection, *, name='wardowl_text', content_field='content', wait=True, timeout=120.0)

Create the Atlas Search index that mode="keyword" searches, creating the collection if needed.

The index holds the text, with the lucene.standard analyzer, which keeps every word, and as tokens the permission terms, the write marker and _id. With wait, it returns once the index is queryable, or raises TimeoutError.

Parameters:

• collection (Any)

• name (str)

• content_field (str)

• wait (bool)

• timeout (float)

Return type:

None

wardowl.backends.mongodb.filter_to_mongo(f)

Translate a filter into a $vectorSearch filter on the tagged terms.

Raises NothingMatches when nothing can match, as $vectorSearch reads a missing filter as “match everything”.

Parameters:

f (In | And | Or)

Return type:

dict[str, Any]

wardowl.backends.mongodb.filter_to_search(f)

Translate a filter into an Atlas Search operator on the tagged terms, for $search: in for each In, and compound for And and Or. Raises NothingMatches when nothing can match.

Parameters:

f (In | And | Or)

Return type:

dict[str, Any]

wardowl.backends.mongodb.permission_terms(metadata)

The value of wardowl_terms for a chunk with this metadata, for documents written by other tools.

Parameters:

metadata (Mapping[str, Any])

Return type:

list[str]

class wardowl.backends.weaviate.WeaviateBackend(client, collection, embed=None, *, vector_name='embedding', content_property='content', metadata_property='metadata_json', mode='vector', async_client=None)

Vector search on Weaviate, with the permission filter applied inside the search.

embed turns text into a vector (your embedding model). vector_name is the collection’s named vector. Scores are cosine similarity, 1 minus Weaviate’s cosine distance. Pass async_client (a connected WeaviateAsyncClient) for a native asearch; otherwise async calls run search in a thread. Without embed, search with Query(vector=...) and pass embeddings to add.

mode="keyword" searches the text with Weaviate’s BM25 instead, under the same permission filter, for hybrid search with RankFusion; embed may then be None, and scores are BM25’s.

Parameters:

• client (Any)

• collection (str)

• embed (Embed | None)

• vector_name (str)

• content_property (str)

• metadata_property (str)

• mode (str)

• async_client (Any)

add(documents, embeddings=None)

Insert or replace chunks. Embeddings are computed with embed unless given.

Parameters:

• documents (Iterable[Document])

• embeddings (Iterable[Sequence[float]] | None)

Return type:

None

set_metadata(metadata)

Replace the metadata of stored chunks, keeping their text and vectors. Ids not stored are ignored.

An object is changed only when its wardowl_id is the id given, or, for an object written by another tool, when it has no wardowl_id and the id is its UUID.

Parameters:

metadata (Mapping[str, Mapping[str, Any]])

Return type:

None

delete(ids)

Remove chunks by id: the objects holding each id, and no other. Ids not stored are ignored.

Parameters:

ids (Iterable[str])

Return type:

None

wardowl.backends.weaviate.create_collection(client, name, *, vector_name='embedding', content_property='content', metadata_property='metadata_json')

Create a collection WeaviateBackend can use: a self-provided cosine HNSW vector, the text, the metadata as unindexed JSON, and the id and permission terms with field tokenization.

Parameters:

• client (Any)

• name (str)

• vector_name (str)

• content_property (str)

• metadata_property (str)

Return type:

None

wardowl.backends.weaviate.filter_to_weaviate(f)

Translate a filter into a Weaviate filter on the tagged terms. Raises NothingMatches when nothing can match.

Parameters:

f (In | And | Or)

Return type:

Any

wardowl.backends.weaviate.permission_terms(metadata)

The value of wardowl_terms for a chunk with this metadata, for objects written by other tools.

Parameters:

metadata (Mapping[str, Any])

Return type:

list[str]

wardowl.backends.weaviate.point_id(document_id)

The Weaviate object id add writes a document id to: a UUID derived from its SHA-256.

Every id is hashed, one that is already a UUID too, so no two document ids share an object.

Parameters:

document_id (str)

Return type:

str

class wardowl.backends.redis.RedisBackend(client, index, embed=None, *, mode='vector', prefix='wardowl:', vector_field='embedding', content_field='content', metadata_field='metadata', ef_runtime=None, async_client=None)

Vector search on the Redis Query Engine, with the permission filter applied inside the search.

embed turns text into a vector (your embedding model). Chunks are hashes under prefix (the document id appended). ef_runtime is HNSW’s search breadth (Redis’s default when None). Scores are cosine similarity, 1 minus Redis’s cosine distance. Pass async_client (a redis.asyncio.Redis) for a native asearch; otherwise async calls run search in a thread. Without embed, search with Query(vector=...) and pass embeddings to add.

mode="keyword" searches the text with BM25 instead, under the same permission filter, in an index created with content_field=. Each word of the query (letters, digits and underscores) becomes a parameter, so none can act as query syntax, and up to 64 distinct words are kept. Scores are Redis’s BM25, higher for better matches. Such a backend needs no embed: without one, add writes chunks with no vector, which keyword search still finds.

Parameters:

• client (Any)

• index (str)

• embed (Embed | None)

• mode (str)

• prefix (str)

• vector_field (str)

• content_field (str)

• metadata_field (str)

• ef_runtime (int | None)

• async_client (Any)

add(documents, embeddings=None)

Insert or replace chunks. Embeddings are computed with embed unless given.

Parameters:

• documents (Iterable[Document])

• embeddings (Iterable[Sequence[float]] | None)

Return type:

None

set_metadata(metadata)

Replace the metadata of stored chunks, keeping their text and vectors. Ids not stored are ignored.

Parameters:

metadata (Mapping[str, Mapping[str, Any]])

Return type:

None

wardowl.backends.redis.create_index(client, index, dim, *, prefix='wardowl:', vector_field='embedding', content_field=None, algorithm='HNSW', m=16, ef_construction=200)

Create the index if it does not exist: the permission terms as a TAG field and a cosine vector index, over the hashes whose keys start with prefix.

algorithm="HNSW" (the default, tuned by m and ef_construction) is approximate. Redis indexes a hash again whenever any of its fields changes, so set_metadata removes and re-adds the chunk’s vector in the graph, and a small, often-changed graph can then miss a chunk. algorithm="FLAT" compares the query with every vector: exact, and fast enough for indexes of up to some tens of thousands of chunks.

With content_field (the backend’s, "content" by default) the chunks’ text is indexed as well, for mode="keyword", keeping every word: the index has no stop words. An index created without it can add it with FT.ALTER <index> SCHEMA ADD content TEXT.

Parameters:

• client (Any)

• index (str)

• dim (int)

• prefix (str)

• vector_field (str)

• content_field (str | None)

• algorithm (str)

• m (int)

• ef_construction (int)

Return type:

None

wardowl.backends.redis.filter_to_query(f)

Translate a filter into a Query Engine expression on the hex-encoded terms.

Raises NothingMatches when nothing can match, as a query without the expression matches everything.

Parameters:

f (In | And | Or)

Return type:

str

wardowl.backends.redis.permission_terms(metadata)

The value of wardowl_terms for a chunk with this metadata, for hashes written by other tools.

Parameters:

metadata (Mapping[str, Any])

Return type:

str

class wardowl.backends.lancedb.LanceDBBackend(table, embed=None, *, mode='vector', vector_field='vector', content_field='content', metadata_field='metadata', nprobes=None, refine_factor=None)

Vector search on a LanceDB table, with the permission filter applied before ranking.

embed turns text into a vector (your embedding model); without it, search with Query(vector=...) and pass embeddings to add. Scores are cosine similarity, 1 minus LanceDB’s cosine distance. nprobes and refine_factor tune an IVF index if the table has one. Async calls run search in a thread.

mode="keyword" searches the text with the table’s full-text index instead (build it with create_indexes(table, content_field="content")), under the same permission filter. Scores are LanceDB’s BM25, higher for better matches. Such a backend needs no embed: without one, add writes chunks with no vector, which keyword search still finds.

Parameters:

• table (Any)

• embed (Embed | None)

• mode (str)

• vector_field (str)

• content_field (str)

• metadata_field (str)

• nprobes (int | None)

• refine_factor (int | None)

add(documents, embeddings=None)

Insert or replace chunks. Embeddings are computed with embed unless given.

Parameters:

• documents (Iterable[Document])

• embeddings (Iterable[Sequence[float]] | None)

Return type:

None

set_metadata(metadata)

Replace the metadata of stored chunks, keeping their text and vectors. Ids not stored are ignored.

An update-only merge of the metadata and terms columns: rows whose id is not stored are not inserted, so a chunk deleted meanwhile is not recreated, and the text and vector are never read and written back. No metadata value is ever part of the SQL.

Parameters:

metadata (Mapping[str, Mapping[str, Any]])

Return type:

None

wardowl.backends.lancedb.create_table(db, name, dim, *, vector_field='vector', content_field='content', metadata_field='metadata')

Create the table if it does not exist, and return it. db is a lancedb.connect(...) connection.

Parameters:

• db (Any)

• name (str)

• dim (int)

• vector_field (str)

• content_field (str)

• metadata_field (str)

Return type:

Any

wardowl.backends.lancedb.create_indexes(table, *, content_field=None)

Index the permission terms (LABEL_LIST) and the ids, which filters and upserts use. Run it once the table holds data; rows added later are searched correctly before they are indexed.

With content_field (the backend’s, "content" by default) it also builds the full-text index that mode="keyword" searches, keeping every word: no stop words are removed. Rows added later are found by a scan until table.optimize() adds them to the index.

Parameters:

• table (Any)

• content_field (str | None)

Return type:

None

wardowl.backends.lancedb.filter_to_sql(f)

Translate a filter into a where predicate on the hex-encoded terms.

Raises NothingMatches when nothing can match, as LanceDB reads a missing predicate as “match everything”.

Parameters:

f (In | And | Or)

Return type:

str

wardowl.backends.lancedb.permission_terms(metadata)

The value of wardowl_terms for a chunk with this metadata, for rows written by other tools.

Parameters:

metadata (Mapping[str, Any])

Return type:

list[str]

class wardowl.backends.faiss.FaissBackend(embed, dim, *, index=None)

Chunks in a FAISS index, searched with the permission filter as an ID selector.

embed turns text into a vector of dim numbers (it may be None when every chunk and query carries its own vector). Thread-safe: a lock serializes changes and searches.

Parameters:

• embed (Embed | None)

• dim (int)

• index (Any)

add(documents, embeddings=None)

Store chunks, replacing any with the same id. embeddings gives their vectors, in order.

When one call repeats an id, the last of its documents is stored, as if they were added in turn.

Parameters:

• documents (Iterable[Document])

• embeddings (Iterable[Sequence[float]] | None)

Return type:

None

delete(ids)

Remove chunks by id. Ids not stored are ignored.

Parameters:

ids (Iterable[str])

Return type:

None

set_metadata(metadata)

Replace the metadata of stored chunks, keeping their text and vectors. Ids not stored are ignored.

Parameters:

metadata (Mapping[str, Mapping[str, Any]])

Return type:

None

save(path)

Write the index and the chunks to the directory path, creating it if needed.

Parameters:

path (str | Path)

Return type:

None

classmethod load(path, embed)

A backend with the index and chunks save wrote to the directory path.

Parameters:

• path (str | Path)

• embed (Callable[[str], Sequence[float]] | None)

Return type:

FaissBackend

class wardowl.backends.vespa.VespaBackend(url, embed, *, schema='wardowl', namespace='wardowl', mode='vector', approximate=True, client=None, timeout=10.0, concurrency=8)

Chunks in a Vespa content cluster, searched with the permission filter in the YQL query.

url is the container’s endpoint (http://localhost:8080). mode="keyword" ranks by BM25 on the text instead, under the same filter. Pass client, an httpx.Client, for TLS client certificates, tokens or proxies (Vespa Cloud, say). Writes go concurrency at a time. Thread-safe.

Parameters:

• url (str)

• embed (Embed | None)

• schema (str)

• namespace (str)

• mode (Literal['vector', 'keyword'])

• approximate (bool)

• client (Any)

• timeout (float)

• concurrency (int)

close()

Close the HTTP client, if the backend created it.

Return type:

None

add(documents, embeddings=None)

Store chunks, replacing any with the same id. embeddings gives their vectors, in order.

Parameters:

• documents (Iterable[Document])

• embeddings (Iterable[Sequence[float]] | None)

Return type:

None

delete(ids)

Remove chunks by id. Ids not stored are ignored.

Parameters:

ids (Iterable[str])

Return type:

None

set_metadata(metadata)

Replace the metadata of stored chunks, keeping their text and vectors. Ids not stored are ignored.

Parameters:

metadata (Mapping[str, Mapping[str, Any]])

Return type:

None

wardowl.backends.vespa.schema(dim, name='wardowl')

A Vespa schema (schemas/<name>.sd) for chunks with dim-dimensional vectors.

Parameters:

• dim (int)

• name (str)

Return type:

str

wardowl.backends.vespa.filter_to_yql(f)

Translate a filter into a YQL condition on the hex-encoded terms.

Raises NothingMatches when nothing can match, as a query without the condition matches everything.

Parameters:

f (In | And | Or)

Return type:

str

class wardowl.backends.cassandra.CassandraBackend(session, keyspace, table, embed, *, similarity='cosine', max_queries=256, concurrency=32, write_concurrency=4)

Chunks in a Cassandra table, searched by ANN with the permission filter in each query’s WHERE.

session is a connected cassandra.cluster.Session; its execution profile sets the consistency level (LOCAL_QUORUM for both reads and writes gives read-your-writes). The similarity must match the index: the score is that similarity function’s value. Thread-safe, as the driver’s session is.

Parameters:

• session (Any)

• keyspace (str)

• table (str)

• embed (Embed | None)

• similarity (str)

• max_queries (int)

• concurrency (int)

• write_concurrency (int)

add(documents, embeddings=None)

Store chunks, replacing any with the same id. embeddings gives their vectors, in order.

Parameters:

• documents (Iterable[Document])

• embeddings (Iterable[Sequence[float]] | None)

Return type:

None

delete(ids)

Remove chunks by id. Ids not stored are ignored.

Parameters:

ids (Iterable[str])

Return type:

None

set_metadata(metadata)

Replace the metadata of stored chunks, keeping their text and vectors. Ids not stored are ignored.

The ids are looked up first, and only the rows found are updated, with plain writes like those of add and delete. A lightweight transaction (IF EXISTS) would take its timestamp from Paxos rather than from the client, and Cassandra orders writes by timestamp, so mixed with plain writes to the same rows it could apply an older change over a newer one, bringing a revoked permission back. A chunk deleted between the lookup and the update is left as a row holding only its metadata and terms, with no text and no vector, which an ANN search never returns; a later add of that id replaces it.

Parameters:

metadata (Mapping[str, Mapping[str, Any]])

Return type:

None

wardowl.backends.cassandra.create_table(session, keyspace, table, dim, *, similarity='cosine')

Create the table for chunks with dim-dimensional vectors, and its SAI indexes, if missing.

Parameters:

• session (Any)

• keyspace (str)

• table (str)

• dim (int)

• similarity (str)

Return type:

None

wardowl.backends.cassandra.filter_to_conjunctions(f, limit=10000)

The filter as an OR of ANDs: a chunk matches when its terms include every term of one set.

An empty list means nothing can match. ANDs implied by others are dropped, and a filter that needs more than limit ANDs raises ValueError.

Parameters:

• f (In | And | Or)

• limit (int)

Return type:

list[frozenset[str]]

wardowl.backends.cassandra.required_terms(f)

Terms every chunk the filter matches must have (possibly none).

Parameters:

f (In | And | Or)

Return type:

frozenset[str]

exception wardowl.backends.cassandra.TooManyQueries

The filter expands to more ANDs than allowed.

class wardowl.backends.pinecone.PineconeBackend(host, api_key, embed, *, namespace='', wait=30.0, client=None, timeout=10.0, concurrency=8)

Chunks in a Pinecone index, searched with the permission filter in the query.

host is the index’s host (from the console, or describe_index), and api_key a Pinecone API key, kept out of every message and representation. namespace keeps tenants or corpora apart. wait is how many seconds add, delete and set_metadata wait for a search to see the change (raising TimeoutError after that); None does not wait. Pass client, an httpx.Client, for proxies or custom TLS. Thread-safe.

Parameters:

• host (str)

• api_key (str)

• embed (Embed | None)

• namespace (str)

• wait (float | None)

• client (Any)

• timeout (float)

• concurrency (int)

close()

Close the HTTP client, if the backend created it.

Return type:

None

add(documents, embeddings=None)

Store chunks, replacing any with the same id. embeddings gives their vectors, in order.

Parameters:

• documents (Iterable[Document])

• embeddings (Iterable[Sequence[float]] | None)

Return type:

None

delete(ids)

Remove chunks by id. Ids not stored are ignored, and so is a record that holds another id.

Parameters:

ids (Iterable[str])

Return type:

None

clear()

Remove every chunk in the backend’s namespace.

Return type:

None

set_metadata(metadata)

Replace the metadata of stored chunks, keeping their text and vectors. Ids not stored are ignored, and so is a record that holds another id.

Parameters:

metadata (Mapping[str, Mapping[str, Any]])

Return type:

None

wardowl.backends.pinecone.filter_to_pinecone(f)

Translate a filter into a Pinecone metadata filter on the hex-encoded terms.

Raises NothingMatches when nothing can match: Pinecone reads a missing filter as “match everything”.

Parameters:

f (In | And | Or)

Return type:

dict[str, Any]

wardowl.backends.pinecone.record_id(doc_id)

The Pinecone record id for a chunk id: itself when it is printable ASCII of 1 to 512 characters that does not start with sha256-, otherwise sha256- and its SHA-256.

No two chunk ids share a record id: one kept as it is never starts with sha256-, and a hashed one always does.

Parameters:

doc_id (str)

Return type:

str

Frameworks

class wardowl.langchain.WardowlRetriever(*args, name=None, tags=None, metadata=None, retriever, k=4, subject=None, metadata_fields=())

Returns only the chunks the subject may read.

Each Document carries relevance_score and the chunk’s metadata keys named in metadata_fields, none by default: permission fields such as allowed_users, allowed_groups and tenant say who may read what, and documents reach the model, prompts and traces.

Parameters:

• args (Any)

• name (str | None)

• tags (list[str] | None)

• metadata (dict[str, Any] | None)

• retriever (PermissionedRetriever)

• k (int)

• subject (Subject | None)

• metadata_fields (tuple[str, ...])

class wardowl.langchain.LangChainVectorStoreBackend(vectorstore, *, id_key=None)

Any LangChain VectorStore as a wardowl backend (post-filtered).

Documents need an id: Document.id, or the metadata field named by id_key. Scores follow the store’s ranking (best first), since stores disagree on what their raw scores mean. A Query with a vector is searched with similarity_search_by_vector.

Parameters:

• vectorstore (Any)

• id_key (str | None)

class wardowl.llamaindex.WardowlRetriever(retriever, subject, *, k=4, metadata_fields=(), **kwargs)

Returns only the nodes subject may read.

Each node carries the chunk’s metadata keys named in metadata_fields, none by default: permission fields such as allowed_users, allowed_groups and tenant say who may read what, and nodes reach the model, prompts and traces.

Parameters:

• retriever (PermissionedRetriever)

• subject (Subject)

• k (int)

• metadata_fields (Collection[str])

• kwargs (Any)

class wardowl.llamaindex.PermissionNodePostprocessor(*, callback_manager=<factory>, authorizer, subject, action='read', metadata_fields=())

Keeps only the nodes subject may read, as decided by authorizer.

The nodes it keeps are copies that carry the metadata keys named in metadata_fields, none by default, as the retriever’s nodes do: LlamaIndex puts a node’s metadata into prompts by default, and permission fields such as allowed_users, allowed_groups and tenant say who may read what. The authorizer sees all the metadata, and the nodes passed in are not changed.

Parameters:

• callback_manager (CallbackManager)

• authorizer (Any)

• subject (Any)

• action (str)

• metadata_fields (tuple[str, ...])

class wardowl.llamaindex.LlamaIndexBackend(index)

A LlamaIndex index (anything with as_retriever(similarity_top_k=...)) as a backend.

Parameters:

index (Any)

class wardowl.haystack.WardowlRetriever(*args, **kwargs)

Returns at most top_k documents the run’s subject may read, best first.

subject is the authenticated user’s Subject, or a function returning it, called on every run. Anything else raises TypeError. The input is typed Callable[[], Subject] | Any on purpose. ComponentTool leaves an input whose type holds a function out of the tool’s parameters, but it passes on every argument a model sends, converted to the input’s type with pydantic, and an input typed Subject would turn a JSON object into one. With Any, the JSON stays as it is and is refused, so a model cannot name another user.

Documents carry the chunk’s metadata keys named in metadata_fields, none by default: permission fields such as allowed_users, allowed_groups and tenant say who may read what, and documents reach the model, prompts and traces.

Parameters:

• args (Any)

• kwargs (Any)

Return type:

Any

run(query, subject, top_k=None, query_embedding=None)

Retrieve for subject. top_k overrides the component’s default for this run.

query_embedding (from a text embedder, for example) is searched with instead of embedding query again; query may then be empty.

Parameters:

• query (str)

• subject (Callable[[], Subject] | Any)

• top_k (int | None)

• query_embedding (list[float] | None)

Return type:

dict[str, Any]

async run_async(query, subject, top_k=None, query_embedding=None)

The async form of run, with the same inputs and outputs.

Parameters:

• query (str)

• subject (Callable[[], Subject] | Any)

• top_k (int | None)

• query_embedding (list[float] | None)

Return type:

dict[str, Any]

class wardowl.haystack.BoundWardowlRetriever(*args, **kwargs)

Returns at most top_k documents the subject from subject_provider() may read, best first.

The subject is not an input: subject_provider is called on every run, where the run happens (in the run’s thread, or its task for run_async), and must return the authenticated user’s Subject, for example by reading a contextvar the web layer set for the request. Neither a pipeline’s caller nor a model calling the component as a tool can name another user. A provider that fails, or returns anything but a Subject, stops the run. Inputs, outputs and metadata_fields are as for WardowlRetriever, without subject.

Parameters:

• args (Any)

• kwargs (Any)

Return type:

Any

run(query, top_k=None, query_embedding=None)

Retrieve for the subject subject_provider returns. top_k overrides the default for this run.

Parameters:

• query (str)

• top_k (int | None)

• query_embedding (list[float] | None)

Return type:

dict[str, Any]

async run_async(query, top_k=None, query_embedding=None)

The async form of run, with the same inputs and outputs.

Parameters:

• query (str)

• top_k (int | None)

• query_embedding (list[float] | None)

Return type:

dict[str, Any]

wardowl.mcp.add_search_tool(server, retriever, subject, *, name='search', description="Search the organisation's documents. Returns only passages the calling user is allowed to read, most relevant first, each with its id, text and relevance score.", limit=5, max_limit=20, max_query_length=4000, metadata=())

Register a permission-aware search tool on server.

subject is a function from the request’s verified AccessToken to a Subject, or one fixed Subject for a server only that person can reach; an async function is awaited, and a synchronous one runs in a worker thread. The tool takes query (at most max_query_length characters) and limit (default limit, at most max_limit); metadata names the metadata keys included in each result.

Parameters:

• server (MCPServer[Any])

• retriever (PermissionedRetriever)

• subject (Subject | Callable[[AccessToken], Subject | Awaitable[Subject]])

• name (str)

• description (str)

• limit (int)

• max_limit (int)

• max_query_length (int)

• metadata (Sequence[str])

Return type:

None

wardowl.server.create_app(retriever, authenticate, *, max_k=50, max_query_length=4000, max_vector_length=4096, max_body_bytes=262144, metadata_fields=(), title='wardowl', docs=False)

A FastAPI application serving retriever to callers that authenticate identifies.

authenticate receives the bearer token and returns the Subject, or raises InvalidToken; OIDCAuthenticator is one. An async authenticator is awaited, and a synchronous one runs in a worker thread. metadata_fields lists the metadata returned with each hit (none by default). A request body over max_body_bytes is refused with 413. docs serves the interactive API pages at /docs.

Parameters:

• retriever (PermissionedRetriever)

• authenticate (Callable[[str], Subject | Awaitable[Subject]])

• max_k (int)

• max_query_length (int)

• max_vector_length (int)

• max_body_bytes (int)

• metadata_fields (Collection[str])

• title (str)

• docs (bool)

Return type:

FastAPI

class wardowl.server.OIDCAuthenticator(issuer, audience, jwks_url, *, algorithms=('RS256', 'ES256'), groups_claim='groups', tenant_claim=None, leeway=0, lifespan=300, cooldown=30, timeout=5, ssl_context=None)

Verifies JWT access tokens from an OpenID Connect provider against its published keys.

The token must name (kid) a key in jwks_url whose algorithm is one of algorithms (asymmetric only, and never taken from the token), be signed by it, be issued by issuer for audience, and be unexpired. The subject is the sub claim, its groups the list in groups_claim, and, when tenant_claim is set, its tenant attribute the string in that claim, which must then be present. A token that fails any check raises InvalidToken.

The key set is fetched with a timeout (seconds) and kept for lifespan seconds, so a key the provider has rotated out stops verifying within that time. A token naming a key that is not in the set fetches it again, at most once every cooldown seconds; until then such a token is invalid at once. A key endpoint that cannot be reached, or that returns no usable keys, raises AuthorizationError. Calls are synchronous: create_app runs them in a worker thread.

The key set is fetched over HTTPS with the system’s trust store; ssl_context replaces it, for an identity provider whose certificate a private CA signed.

Parameters:

• issuer (str)

• audience (str | Sequence[str])

• jwks_url (str)

• algorithms (Sequence[str])

• groups_claim (str)

• tenant_claim (str | None)

• leeway (float)

• lifespan (float)

• cooldown (float)

• timeout (float)

• ssl_context (SSLContext | None)

subject(claims)

The subject a verified token’s claims describe.

Parameters:

claims (Mapping[str, Any])

Return type:

Subject

class wardowl.server.InvalidToken

The bearer token is missing, malformed, expired or not issued for this service.

Conformance testing

See Extending wardowl.

wardowl.testing.check_backend(factory, *, examples=25, seed=0, max_documents=20, filters_per_example=4, exact=True, query='doc')

Check a backend against the reference semantics. Raises ConformanceError on the first failure.

factory(documents) must return a new backend holding exactly those documents. For each of examples random corpora (up to max_documents each) it checks that:

• a search without a filter returns stored documents only, min(k, stored) of them for k up to the whole corpus, best first, with text and metadata exactly as stored;

• a search with a filter (when supports_filters is true) returns only documents matches selects, and min(k, matching) of them, for filters_per_example random nested filters;

• asearch, if the backend has one, returns the same hits as search for every one of these searches, with and without a filter: the same ids in the same order, with the same text, metadata and scores, and refuses with ValueError every filter that search refuses;

• retrieval through PermissionedRetriever with MetadataACLAuthorizer, through retrieve and aretrieve alike, never returns a document the subject may not read, reports no violations, and returns as many readable documents as were asked for (with "prefilter" and "postfilter" where available);

• if the backend has set_metadata, replacing the metadata of some documents (and of an id that is not stored) leaves their text as it was, changes what filters select as matches says it should, keeps no key of the old metadata, and stores nothing new.

The data holds the values stores tend to confuse: 1 and “1”, “a” and “A”, one letter composed and decomposed, and field names with a dot or letters outside ASCII. A backend may refuse a filter on such a field name by raising ValueError, which returns nothing, from search and asearch alike; it may not return wrong hits.

exact=False suits approximate indexes, which may miss matching documents: results must still never include a document that does not match, but may contain fewer than expected.

Every search uses query. Pass Query(vector=...) to check a backend searched with vectors you compute, for example Query(QUERY, HashingEmbedder()(QUERY)) against a backend built without embed and filled with add(documents, embeddings=...).

Parameters:

• factory (Callable[[Sequence[Document]], SearchBackend])

• examples (int)

• seed (int)

• max_documents (int)

• filters_per_example (int)

• exact (bool)

• query (str | Query)

Return type:

None

wardowl.testing.check_authorizer(authorizer, subjects=None, hits=None, *, action='read', seed=0)

Check that an authorizer’s answers are well formed and agree with each other.

Without subjects and hits, random users and groups and allowed_users/allowed_groups metadata are used, which suits authorizers that read permissions from the chunks (MetadataACLAuthorizer, a FunctionAuthorizer over metadata, CachingAuthorizer around them). For an authorizer backed by a service, pass subjects and hits from your test data. Every CachingAuthorizer in it is emptied before each call, so the authorizer it wraps answers. For every subject it checks that:

• check returns one boolean per hit, and [] for no hits;

• the answer for a hit does not depend on the other hits or their order;

• when filter_for returns a filter, check agrees with matches(filter, hit.metadata) for every hit. If they differ, pre-filtered and post-filtered results differ, and the re-check of pre-filtered hits reports violations;

• acheck, afilter_for and ascope_for, where it has them, give the same answers as check, filter_for and scope_for: aretrieve uses them.

Parameters:

• authorizer (Authorizer)

• subjects (Sequence[Subject] | None)

• hits (Sequence[Hit] | None)

• action (str)

• seed (int)

Return type:

None

exception wardowl.testing.ConformanceError

A backend or authorizer disagreed with wardowl’s reference semantics.

class wardowl.testing.HashingEmbedder(dim=16)

Deterministic bag-of-words embeddings for tests: no model, no network.

Texts that share words get similar vectors, so search results are meaningful enough for tests, and the same text always gets the same vector.

Parameters:

dim (int)

wardowl.testing.random_documents(rng, n)

n documents containing QUERY, with metadata that mixes strings, integers, lists of either, booleans and missing fields: the cases where backends’ filter translations tend to differ from the reference.

Parameters:

• rng (Random)

• n (int)

Return type:

list[Document]

wardowl.testing.random_filter(rng, depth=2)

A filter over the fields random_documents uses, nested up to depth levels of And/Or.

Parameters:

• rng (Random)

• depth (int)

Return type:

In | And | Or