Database Vector Stores

Overview

Database Vector Stores provide persistent indexing, metadata filtering, and approximate nearest neighbor (ANN) retrieval capabilities across relational databases, distributed search engines, in-memory data structures, and graph analytics systems. Within Mem0, these database backends handle the storage and querying of long-term memory embeddings alongside associated payloads, user tenant contexts, and timestamps. Unlike standalone Embedded Vector Stores or managed Cloud Vector Backends, database vector stores integrate embedding storage directly into existing enterprise data systems like PostgreSQL via pgvector, Elasticsearch, OpenSearch, Redis, Valkey, Apache Cassandra, Amazon Neptune Analytics, Oracle AI Vector Search, MongoDB, Azure AI Search, and Baidu Mochow.

Sources: mem0-ts/src/oss/src/vector_stores/pgvector.ts:217-245

The primary design challenge solved by these drivers is bridging multi-dimensional vector math with structured relational tables, document schemas, or inverted indices. Because different engines handle vector indexing via distinct internal mechanisms—such as PostgreSQL native vector data types, Elasticsearch dense_vector fields with HNSW graphs, or Redis Hash keys with RediSearch modules—each backend abstracts the connection lifecycle, schema initialization, parameter binding, and query filtering into a unified VectorStore interface.

Sources: mem0-ts/src/oss/src/vector_stores/pgvector.ts:217-245

Sources: mem0-ts/src/oss/src/vector_stores/pgvector.ts:217-245

Relational Database Integrations: pgvector, Azure MySQL, and Oracle DB

Relational database integrations extend traditional SQL tables with native or JSON-serialized vector storage and similarity operators.

In PGVector, tables are initialized with a UUID primary key, a vector data type column sized to embeddingModelDims, and a JSONB payload column. During nearest neighbor search, pgvector executes queries using the <-> cosine distance operator, sorting results and bounding output sizes with LIMIT $2. Parameterized filters are dynamically constructed via buildFilterConditions(), translating metadata JSON path lookups into SQL expressions with strict parameter indexing to prevent injection.

Sources: mem0-ts/src/oss/src/vector_stores/pgvector.ts:284-446

For Azure MySQL, embeddings and JSON payloads are stored as JSON columns and evaluated via application-level cosineSimilarity() loops over fetched rows, since MySQL handles vector distance sorting in memory after fetching candidate sets.

Sources: mem0-ts/src/oss/src/vector_stores/azure_mysql.ts:123-211

Oracle AI Vector Search utilizes Oracle 23.4+ native VECTOR data types and JSON payload columns inside relational tables. Search operations invoke VECTOR_DISTANCE(vector, :query_vec, ...) combined with ORDER BY distance FETCH APPROX FIRST :max_rows ROWS ONLY. When executing filtered searches, Oracle conditionally omits index hints or injects VECTOR_INDEX_TRANSFORM depending on filter presence.

Sources: mem0/vector_stores/oracledb.py:350-402

Sources: mem0-ts/src/oss/src/vector_stores/pgvector.ts:414-446

Sources: mem0-ts/src/oss/src/vector_stores/pgvector.ts:315-349, mem0-ts/src/oss/src/vector_stores/azure_mysql.ts:124-189, mem0/vector_stores/oracledb.py:321-364

Search Engines: Elasticsearch, OpenSearch, and Azure AI Search

Distributed search and retrieval engines leverage inverted indices alongside vector quantization and HNSW graphs to execute hybrid keyword and semantic searches.

Sources: mem0/vector_stores/opensearch.py:83-135

Elasticsearch and OpenSearch store embeddings in dense_vector or knn_vector fields configured with nmslib or native engines using cosine similarity. When inserting documents, payloads are flattened into metadata objects. During vector search, queries construct a knn block specifying query_vector, k, and num_candidates (typically topK * 2). Metadata filters are validated against a strict regex (/^[a-zA-Z_][a-zA-Z0-9_]*$/) and injected into the KNN filter block as boolean term queries.

Sources: mem0-ts/src/oss/src/vector_stores/elasticsearch.ts:134-164, mem0-ts/src/oss/src/vector_stores/opensearch.ts:160-194

Azure AI Search configures indices with SearchField vector definitions (supporting Collection(Edm.Single) or Collection(Edm.Half) precision) and optional Scalar or Binary Quantization compression profiles (ScalarQuantizationCompression, BinaryQuantizationCompression). Filtering is performed via OData filter expressions generated by _build_filter_expression().

Sources: mem0/vector_stores/azure_ai_search.py:105-163

Sources: mem0/vector_stores/elasticsearch.py:72-95

Sources: mem0/vector_stores/elasticsearch.py:72-95, mem0/vector_stores/opensearch.py:83-107, mem0/vector_stores/azure_ai_search.py:135-160

In-Memory & Key-Value Stores: Redis and Valkey

In-memory data stores like Redis and Valkey employ RediSearch and RedisVL modules to index hash structures containing binary-encoded floating-point vector embeddings.

Sources: mem0/vector_stores/redis.py:48-78

The RedisDB and ValkeyDB drivers store entries as Redis Hashes using keys prefixed with mem0:{collection_name}:{id}. Float32 arrays are converted to raw byte buffers (np.array(vector, dtype=np.float32).tobytes() in Python or Buffer.from(new Float32Array(query).buffer) in TypeScript) before persistence.

Sources: mem0/vector_stores/redis.py:123-147, mem0-ts/src/oss/src/vector_stores/valkey.ts:432-439

Schema definitions declare fields such as memory_id, hash, agent_id, run_id, user_id as TAG types, memory and metadata as TEXT or TAG, created_at as NUMERIC, and embedding as a VECTOR attribute using FLAT or HNSW algorithms.

Sources: mem0/vector_stores/valkey.py:18-34, mem0/vector_stores/redis.py:20-36

Copy// Example: Building a Valkey / Redis pre-filter expression
export function buildRedisFilterExpr(filters?: SearchFilters): string {
  if (!filters) return "*";
  const conditions = Object.entries(toSnakeCase(filters))
    .filter(([, value]) => value !== null && value !== undefined)
    .map(([key, value]) => `@${key}:{${escapeRedisTagValue(value)}}`);
  return conditions.length > 0 ? conditions.join(" ") : "*";
}

Sources: mem0-ts/src/oss/src/vector_stores/redis.ts:141-147

Sources: mem0/vector_stores/valkey.py:48-57, mem0-ts/src/oss/src/vector_stores/valkey.ts:22-27

Distributed & Graph Backends: Cassandra, Neptune Analytics, Milvus, and Baidu Mochow

Distributed and specialized graph stores handle massive memory throughput using custom partition keys and vector-optimized storage engines.

Sources: mem0-ts/src/oss/src/vector_stores/cassandra.ts:41-80

• Apache Cassandra: Stores vectors as list<float> and payloads as JSON text inside tables structured under configurable keyspaces (default mem0). Search queries scan rows in pages (PAGE_SIZE = 500), normalize vectors, apply in-memory metadata filters, and maintain top-K results via pushTopResult().
• Amazon Neptune Analytics: Operates as a serverless graph database where memory entities are stored as graph vertices with label MEM0_VECTOR_{collection_name} and identifier ~id. Vector upserts and similarity searches execute Cypher procedures such as CALL neptune.algo.vectors.upsert(n, embedding) and CALL neptune.algo.vectors.topKByEmbeddingWithFiltering(...).
• Milvus / Zilliz: Configures collections with AUTOINDEX for float vectors and SPARSE_INVERTED_INDEX with built-in BM25 functions for hybrid sparse-dense retrieval over auto-tokenized text fields.
• Baidu Mochow: Utilizes typed tables (STRING id, FLOAT_VECTOR vector, JSON metadata) with HNSW vector indices and BM25SearchRequest full-text capabilities.

Sources: mem0-ts/src/oss/src/vector_stores/cassandra.ts:143-185, mem0/vector_stores/neptune_analytics.py:41-61, mem0/vector_stores/milvus.py:33-65, mem0-ts/src/oss/src/vector_stores/baidu.ts:352-391

Sources: mem0-ts/src/oss/src/vector_stores/cassandra.ts:143-185, mem0/vector_stores/neptune_analytics.py:159-198, mem0/vector_stores/milvus.py:66-125

Copyasync search(
  query: number[],
  topK: number = 5,
  filters?: SearchFilters,
): Promise<VectorStoreResult[]> {
  await this.initialize();
  this.assertVectorDimension(query, "Query");
 
  const scored: VectorStoreResult[] = [];
  await this.scanRows(
    `
      SELECT id, vector, payload
      FROM ${this.keyspace}.${this.collectionName}
    `,
    async (row) => {
      const vector = this.normalizeVector(row.vector);
      const payload = this.parsePayload(row.payload);
      if (!vector || vector.length !== this.dimension) {
        return;
      }
      const item: CassandraVector = {
        id: String(row.id),
        vector,
        payload,
      };
      if (!this.filterVector(item, filters)) {
        return;
      }
      this.pushTopResult(
        scored,
        {
          id: item.id,
          payload: item.payload,
          score: this.cosineSimilarity(query, item.vector),
        },
        topK,
      );
    },
    Math.max(topK, CassandraDB.PAGE_SIZE),
  );
 
  return scored;
}

Sources: mem0-ts/src/oss/src/vector_stores/cassandra.ts:143-185

Configuration, Connection Management, and Lazy Loading

Database vector store initializers implement lazy connection loading to prevent optional peer dependencies (such as @elastic/elasticsearch, pymongo, cassandra-driver, redis, or oracledb) from breaking base package imports when a provider is not actively in use.

Sources: mem0-ts/src/oss/src/vector_stores/elasticsearch.ts:63-65, mem0-ts/src/oss/src/vector_stores/mongodb.ts:35-37

Clients and connection pools are initialized on demand during the first call to initialize(), which caches its execution promise (_initPromise) to prevent race conditions during concurrent startup sequences.

Sources: mem0-ts/src/oss/src/vector_stores/opensearch.ts:120-126

Copy// Example: Lazy peer loading pattern in OpenSearch and Elasticsearch
async initialize(): Promise<void> {
  if (!this._initPromise) {
    this._initPromise = this._doInitialize();
  }
  return this._initPromise;
}

Sources: mem0-ts/src/oss/src/vector_stores/opensearch.ts:120-126

Sources: mem0/configs/vector_stores/oracledb.py:41-69, mem0/vector_stores/opensearch.py:62-81, mem0-ts/src/oss/src/vector_stores/azure_mysql.ts:34-46

Error Handling, Sanitization, and Security Invariants

Database vector stores enforce strict input sanitization and parameter binding guards to protect against injection attacks and malformed embeddings.

Sources: mem0/vector_stores/mongodb.py:150-165

• Filter Key Validation: Drivers such as Elasticsearch, OpenSearch, pgvector, and Baidu enforce strict regular expression validation on filter keys (/^[a-zA-Z_][a-zA-Z0-9_]*$/) to prevent arbitrary query injection through metadata keys.
• Operator Blocking: MongoDB drivers reject filter values that contain dictionaries ($ne, $gt, etc.) to prevent NoSQL query operator injection.
• Dimension Checking: Insert and search operations validate that input vector lengths match configured embedding dimensions before issuing database commands.

Sources: mem0/vector_stores/mongodb.py:150-165, mem0-ts/src/oss/src/vector_stores/elasticsearch.ts:26-44, mem0/vector_stores/opensearch.py:25-53

Sources: mem0/vector_stores/mongodb.py:150-165

Related

• Vector Store Architecture
• Cloud Vector Backends