Vectors

Declaring a vector field

Vector fields are declared per collection from the dashboard or CLI. Open a collection, click Vector Fields, enter a field name (e.g. embedding), pick a dimension, and add it. From that point, every record in the collection has a slot for that field — set it on the records that need it, leave it unset on the rest.

The dimension is the length of every vector you'll store on that field. It must match exactly what your embedding model produces — see Supported dimensions below for the values Koolbase supports.

Writing and reading vectors

Set a vector on a record from the SDK once you have an embedding from your model:

final embedding = await myEmbeddingModel.encode(article.content);

await Koolbase.db.doc(article.id).setVector(
  'embedding',
  embedding,
);

Read it back later — useful for inspection or for re-running search on a known anchor:

final v = await Koolbase.db.doc(article.id).getVector('embedding');
print('${v.vector.length}-dim, updated ${v.updatedAt}');

And remove a record's vector when you no longer need it. The field declaration stays — it's still settable on other records.

await Koolbase.db.doc(article.id).deleteVector('embedding');

Auto-embedding on write

You don't need a client-side embedding model. Configure a vector field with an embedding provider + model + source field, and Koolbase will embed records automatically on insert and update.

Configure the field from the dashboard (Database → your collection → Vector Fields → field settings) or via direct SQL:

UPDATE collection_vector_fields
SET embedding_provider = 'gemini',
    embedding_model    = 'gemini-embedding-001',
    source_field       = 'content'
WHERE collection_id = '<your collection id>'
  AND field_name     = 'content_embedding';

From that point on, every record inserted or updated with a non-empty content field automatically gets a vector queued in embedding_jobs. A background worker processes the queue with 1s/2s/4s/8s/16s exponential backoff and writes the vector to record_vectors_<dim>. Same-text deduplication via content hash means re-embedding identical content from different records skips the API call entirely.

On the same write, Koolbase also populates a lexical (BM25) row in db_record_lexical for the configured source field. This makes lexical and hybrid search work out of the box — no extra configuration, no extra worker. The lexical index is a Postgres tsvector tokenized with the vector field's configured language (see Language for lexical indexing), so it's ready as soon as the row is written.

Setting up an AI provider

Koolbase supports two embedding providers in v1: Google Gemini (free tier, no payment method required) and OpenAI (requires billing). One config per project per provider.

Get a Gemini key at aistudio.google.com/apikey, then register it:

curl -X POST "https://api.koolbase.com/v1/projects/$PROJECT_ID/ai/providers" \
  -H "Authorization: Bearer $DASHBOARD_JWT" \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "gemini",
    "api_key": "AIza...",
    "default_model": "gemini-embedding-001"
  }'

Validation happens inline — the response includes status: "valid" if the key works, or status: "invalid" with a validation_error message. Rate-limit errors (429) return status: "unvalidated"; click Re-test from the dashboard once the limit clears.

Searching: semantic, lexical, hybrid

Koolbase ships three retrieval modes from a single endpoint. Pick the one that matches your query characteristics, or use hybrid as a strong production default.

• Semantic — pure vector search via HNSW + cosine. Best for fuzzy / conceptual queries where exact words don't have to match.
• Lexical — pure BM25 over the field's source text via Postgres ts_rank_cd. Best for exact terms, codes, names, acronyms.
• Hybrid — vector + lexical fused with reciprocal rank fusion (k=60). Generally the strongest default; both rankers vote and the fused score promotes records that score well on either signal.

Flutter

// Semantic (default) — pure vector via HNSW + cosine.
final semantic = await Koolbase.db.collection('articles').searchSemantic(
  field: 'content_embedding',
  queryText: 'how do I configure CI/CD?',
  limit: 10,
);

// Lexical — pure BM25 over the field's source text.
final lexical = await Koolbase.db.collection('articles').searchSemantic(
  field: 'content_embedding',
  queryText: 'CVE-2024-1234',
  mode: KoolbaseSearchMode.lexical,
  limit: 10,
);

// Hybrid — vector + lexical, RRF-fused (k=60).
final hybrid = await Koolbase.db.collection('articles').searchSemantic(
  field: 'content_embedding',
  queryText: 'production deploy pipeline',
  mode: KoolbaseSearchMode.hybrid,
  limit: 10,
);

for (final hit in hybrid.hits) {
  print('${hit.record['title']}  (${hit.distance.toStringAsFixed(3)})');
}

React Native & Web

// Semantic (default) — pure vector via HNSW + cosine.
const semantic = await Koolbase.db.searchSemantic({
  collection: 'articles',
  field: 'content_embedding',
  queryText: 'how do I configure CI/CD?',
  limit: 10,
});

// Lexical — pure BM25 over the field's source text.
const lexical = await Koolbase.db.searchSemantic({
  collection: 'articles',
  field: 'content_embedding',
  queryText: 'CVE-2024-1234',
  mode: 'lexical',
  limit: 10,
});

// Hybrid — vector + lexical, RRF-fused (k=60).
const hybrid = await Koolbase.db.searchSemantic({
  collection: 'articles',
  field: 'content_embedding',
  queryText: 'production deploy pipeline',
  mode: 'hybrid',
  limit: 10,
});

for (const hit of hybrid.hits) {
  console.log(`${hit.record.title}  ${hit.distance.toFixed(3)}`);
}

The pre-existing queryVector path still works for semantic mode — useful when you've already encoded the query client-side or want to use a different embedding model for the query than for the corpus. Lexical and hybrid modes require queryText (BM25 has no notion of vector queries).

Language for lexical indexing

Lexical and hybrid search use a Postgres text-search config to tokenize the source field — the config controls stemming (so running, runs, and ran can all match a query for run) and stop-word handling. Each vector field carries its own lexical_config, defaulting to simple.

Set it from the dashboard (Database → your collection → Vector Fields → the 🌐 Languages icon next to a field) or via the API:

curl -X PATCH "$API/v1/projects/$PROJECT_ID/collections/$COLLECTION/vector-fields/$FIELD" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"lexical_config": "english"}'

Or with the CLI:

koolbase vector-fields update articles content_embedding \
  --lexical-config english \
  --project $PROJECT_ID

Supported configs in v1 — pick the one that matches the dominant language of your source data:

Filtering weak matches

Pass minSimilarity (0..100) to drop results below a similarity threshold server-side. Saves bandwidth on weak matches and keeps client code simpler — no post-filtering needed.

final result = await Koolbase.db.collection('articles').searchSemantic(
  field: 'content_embedding',
  queryText: 'how do I configure CI/CD?',
  mode: KoolbaseSearchMode.hybrid,
  minSimilarity: 70, // only matches at least 70% similar
  limit: 10,
);

const result = await Koolbase.db.searchSemantic({
  collection: 'articles',
  field: 'content_embedding',
  queryText: 'how do I configure CI/CD?',
  mode: 'hybrid',
  minSimilarity: 70,
  limit: 10,
});

Backfilling existing records

Records that pre-date auto-embed or lexical indexing don't have vectors or lexical rows yet. Two backfill paths cover the two indexes — both are idempotent, both are exposed from the dashboard's Vector Fields modal.

Embedding backfill

For a single record, use embedText to queue an embedding job. Returns as soon as the job is queued — typically <100ms. The vector lands within ~1 second once the worker picks it up.

// Embed using the record's source_field value (whatever you configured):
await Koolbase.db.collection('articles').embedText(
  recordId: article.id,
  vectorField: 'content_embedding',
);

// Or override with custom text — useful for combining fields:
await Koolbase.db.collection('articles').embedText(
  recordId: article.id,
  vectorField: 'content_embedding',
  text: '${article.title}\n\n${article.summary}',
);

For a full collection at once, hit the admin endpoint (also available from the dashboard's Vector Fields modal via the ✨ wand icon):

curl -X POST "$API/v1/projects/$PROJECT_ID/collections/$COLLECTION/vector-fields/$FIELD/embed-all" \
  -H "Authorization: Bearer $TOKEN"

Lexical backfill

When you enable hybrid or lexical search on a collection that already has records, lexical rows need to be populated for the pre-existing data. (The write-path hook only covers writes that happen after the hook was wired.) The dashboard's Vector Fields modal has a 📖 book icon next to the ✨ wand for this; the corresponding endpoint:

curl -X POST "$API/v1/projects/$PROJECT_ID/collections/$COLLECTION/vector-fields/$FIELD/lexical-backfill" \
  -H "Authorization: Bearer $TOKEN"

Combining with structured filters

All three modes accept a where filter — e.g. “find the 10 closest articles in the ‘tech’ category”:

final scoped = await Koolbase.db.collection('articles').searchSemantic(
  field: 'content_embedding',
  queryText: 'production deploy pipeline',
  mode: KoolbaseSearchMode.hybrid,
  limit: 10,
  where: {'category': 'tech'},
);

Supported dimensions

Vector dimensions match the output size of your embedding model. Koolbase supports these dimensions in v1, covering the most common open-source and commercial models:

Behavior & errors

Online-only

Vector operations don't cache locally and aren't queued offline — HNSW similarity search and BM25 ranking have no useful offline semantics, so deferring writes could corrupt your view of what's stored. Calls throw on network failure rather than queuing for sync.

Read rules apply post-search

All three search modes respect the collection's read rule the same way get() does. For owner, scoped, or conditional rules, records are filtered to the caller after the index lookup — so very strict rules may return fewer than limit hits.

Errors to handle

try {
  await Koolbase.db.doc(id).setVector('embedding', vec);
} on KoolbaseVectorDimensionMismatchException catch (e) {
  // vector.length didn't match the field's declared dimension
  showError(e.message);  // includes both expected and actual dims
} on KoolbaseNotFoundException {
  // record_not_found, vector_field_not_found, or vector_not_found
  showError('Record or field is missing.');
} on KoolbasePermissionException {
  // The collection's write rule denied the operation
  showError('Not allowed.');
}