Skip to main content
doc0blade47/mem0documentation home
Sign in

OSS to Platform Migration

Last updated Oct 11, 2026100% grounded
Relevant source files

The following files were used as context for generating this wiki page:

Overview

The OSS to Platform Migration utility facilitates the transition of locally stored vector collections and memory artifacts from open-source Mem0 deployments to hosted platform backends. By automating collection extraction, schema normalization, and bulk API ingestion, it addresses the data fragmentation and operational overhead of moving between self-hosted vector engines and managed cloud infrastructure. Key design decisions include robust pagination, idempotent import key generation, and preflight validation checks that ensure metadata integrity and entity mapping are preserved across boundaries.

Sources: scripts/oss-to-platform-migrate.sh:711-729, scripts/oss-to-platform-migrate.sh:738-750, scripts/oss-to-platform-migrate.sh:825-835

Migration script invocation and flags

CLI Interface and Options

The migration script exposes a command-line interface powered by standard argument parsing. It supports options for configuring Qdrant connections, setting export output paths, filtering memories by scope identifiers, and controlling pagination and import-only behaviors.

Flag / OptionArgument TypeDefault ValueDescription
--qdrant-urlstringhttp://localhost:6333Base URL for the Qdrant instance.
--qdrant-api-keystringNoneAPI key for authenticating with Qdrant.
--qdrant-collectionstringmem0Name of the Qdrant collection to export from.
--qdrant-page-sizeinteger100Page size for scrolling through Qdrant points. Must be greater than 0.
--user-idstringNoneFilter exported memories by specific user ID.
--agent-idstringNoneFilter exported memories by specific agent ID.
--run-idstringNoneFilter exported memories by specific run ID.
--allflagFalseExport all records without scope filtering.
--outputpathAuto-generatedTarget file path for the exported JSON artifact.
--import-onlyflagFalseSkip export and run platform ingestion on an existing input file.
--inputpathNoneSource JSON file path required when using --import-only.

Sources: scripts/oss-to-platform-migrate.sh:739-746, scripts/oss-to-platform-migrate.sh:762-767, scripts/oss-to-platform-migrate.sh:796-799

Warning

Setting --qdrant-page-size to zero or a negative integer raises a MigrationError during initialization, halting the export process before any network requests are dispatched.

Sources: scripts/oss-to-platform-migrate.sh:743-745

Environment Options and Configuration

Authentication credentials and connection endpoints can be supplied via runtime environment variables or overridden explicitly through command-line flags. When platform requests return authentication failures (401 or 403), the script raises an AuthError directing operators to verify their MEM0_API_KEY environment variable or re-authenticate via email login.

Sources: scripts/oss-to-platform-migrate.sh:791-792

Note

The import routine strictly requires the --input flag when --import-only is invoked; omitting it triggers an immediate validation failure.

Sources: scripts/oss-to-platform-migrate.sh:797-798

Extracting open source vector collections

Qdrant Collection Export and Normalization

Dumping and reading memory payloads and vectors from open source vector store backends Database Vector Stores involves establishing collection connections, scrolling through stored points with pagination constraints, and normalizing payload structures into standardized export artifacts. The migration utility queries vector engines such as Qdrant to extract point records, isolating core fields and promoting identifiers into structured dictionaries.

The export process reads points from a designated collection using configurable page sizes and optional scope filters. Each retrieved point is normalized by mapping core properties—such as the memory text, content hash, creation timestamp, and update timestamp—while promoting specific identifier keys and collecting unhandled properties into a separate metadata dictionary.

promoted_payload_keys = ("user_id", "agent_id", "run_id", "actor_id", "role")
core_and_promoted_keys = {
    "data",
    "hash",
    "created_at",
    "updated_at",
    "id",
    "text_lemmatized",
    "attributed_to",
    *promoted_payload_keys,
}
 
record: Dict[str, Any] = {
    "id": str(point.get("id")),
    "memory": payload.get("data", ""),
    "hash": payload.get("hash"),
    "created_at": payload.get("created_at"),
    "updated_at": payload.get("updated_at"),
    "source": "python_oss_qdrant",
    "source_payload": payload,
}
 
for key in promoted_payload_keys:
    if key in payload:
        record[key] = payload[key]
 
metadata = {key: value for key, value in payload.items() if key not in core_and_promoted_keys}
if metadata:
    record["metadata"] = metadata
 
return record

Sources: scripts/oss-to-platform-migrate.sh:699-729

The resulting export artifact bundles metadata about the source SDK, vector store type, storage configuration, collection name, server URL, active filters, local anonymous identifiers, and the complete list of normalized records before writing the payload to disk with restricted file permissions.

artifact = {
    "version": 1,
    "kind": "mem0_oss_qdrant_export",
    "exported_at": utc_now(),
    "source": {
        "sdk": "python",
        "vector_store": "qdrant",
        "storage": "hosted",
        "collection": collection_name,
        "url": base_url,
        "filters": {
            "user_id": args.user_id,
            "agent_id": args.agent_id,
            "run_id": args.run_id,
            "all": args.all,
        },
    },
    "local_anonymous_id": primary_anon_id(anon_ids),
    "anonymous_ids": {
        "oss": anon_ids.get("oss"),
        "cli": anon_ids.get("cli"),
    },
    "record_count": len(records),
    "records": records,
}
write_export_file(output_path, artifact)
return output_path, len(records)

Sources: scripts/oss-to-platform-migrate.sh:752-778

Caution

Export output files are explicitly written with read-write permissions restricted to the file owner (stat.S_IRUSR | stat.S_IWUSR) to protect sensitive memory payloads and metadata exposed during the dump.

Sources: scripts/oss-to-platform-migrate.sh:735, scripts/oss-to-platform-migrate.sh:958

Vector Store Abstractions and Graph Backends

Beyond Qdrant, TypeScript vector store implementations such as the NeptuneAnalyticsVectorStore interact with graph databases by structuring collections around specific labels, node identifiers, and vector execution commands. These stores initialize collection labels (MEM0_VECTOR_${this.collectionName}), user migration labels (MEM0_VECTOR_memory_migrations), and default node references (mem0-user) to execute batch insertions and top-K vector similarity searches.

constructor(config: NeptuneAnalyticsConfig) {
  this.graphIdentifier = this.resolveGraphIdentifier(config);
  this.collectionName = config.collectionName || "memories";
  this.collectionLabel = `MEM0_VECTOR_${this.collectionName}`;
  this.collectionLabelExpr = this.escapeLabel(this.collectionLabel);
  this.userLabel = "MEM0_VECTOR_memory_migrations";
  this.userLabelExpr = this.escapeLabel(this.userLabel);
  this.userNodeId = "mem0-user";
  this.dimension = config.dimension || 1536;
  this.clientConfig = this.buildClientConfig(config);
  this.clientOverride = config.client;
 

Sources: mem0-ts/src/oss/src/vector_stores/neptune_analytics.ts:58-69

Warning

Neptune Analytics vector indices are non-transactional; combined updates that write payloads before vector embeddings employ best-effort compensation reads to capture prior node states, but concurrent same-vertex writes can interleave and desync stored payloads from their vectors.

Sources: mem0-ts/src/oss/src/vector_stores/neptune_analytics.ts:184-189

Platform ingestion and schema alignment

Payload Parsing and Record Normalization

Transforming open source records and streaming them into Mem0 platform API endpoints requires reading export files, validating record structures, and transforming local schema fields into hosted platform payloads. The migration script reads export artifacts containing either a raw list of records or a dictionary wrapper with a records array, validating that each entry conforms to an object structure before mapping.

def read_import_records(input_path: Path) -> Tuple[Dict[str, Any], List[Dict[str, Any]]]:
    try:
        parsed = json.loads(input_path.read_text(encoding="utf-8"))
    except FileNotFoundError as exc:
        raise MigrationError(f"Import file not found: {input_path}") from exc
    except json.JSONDecodeError as exc:
        raise MigrationError(f"Import file is not valid JSON: {input_path}") from exc
 
    if isinstance(parsed, list):
        records = parsed
        artifact: Dict[str, Any] = {}
    elif isinstance(parsed, dict):
        artifact = parsed
        raw_records = parsed.get("records")
        if not isinstance(raw_records, list):
            raise MigrationError("Import artifact must contain a records array.")
        records = raw_records
    else:
        raise MigrationError("Import file must be a JSON object or array.")
 
    return artifact, [record for record in records if isinstance(record, dict)]
 

Sources: scripts/oss-to-platform-migrate.sh:802-823

Each record undergoes schema alignment by extracting memory text, constructing platform-compatible messages, and injecting migration metadata. A deterministic migration import key is computed using a SHA-256 hash derived from the source SDK, vector store, collection name, and local record identifier.

def migration_import_key(source: Dict[str, Any], record: Dict[str, Any]) -> str:
    raw = ":".join(
        [
            str(source.get("sdk") or "unknown"),
            str(source.get("vector_store") or "unknown"),
            str(source.get("collection") or "unknown"),
            str(record.get("id") or ""),
        ]
    )
    return sha256(raw.encode("utf-8")).hexdigest()
 

Sources: scripts/oss-to-platform-migrate.sh:825-835

Note

The import payload builder enforces that every record includes non-empty memory text and at least one scoping identifier (user_id, agent_id, or run_id), raising a MigrationError if these prerequisites are unmet.

Sources: scripts/oss-to-platform-migrate.sh:861-884

Migration Metadata Mapping

The metadata structure built for imported records embeds traceability fields into the hosted platform items. These properties link platform memories directly back to their open source origin collections, local IDs, and content hashes.

Metadata KeySource / LogicDescription
mem0_migration_sourcerecord.get("source") or "python_oss_qdrant"Identifies the originating open source client driver or store.
mem0_migration_collectionsource.get("collection")Captures the source vector collection name.
mem0_migration_local_idrecord.get("id")Preserves the original vector point or record identifier.
mem0_migration_local_hashrecord.get("hash")Stores the content hash from the local database.
mem0_migration_import_keymigration_import_key(source, record)Holds the SHA-256 deduplication hash for idempotency checking.
def build_import_metadata(source: Dict[str, Any], record: Dict[str, Any]) -> Dict[str, Any]:
    existing_metadata = record.get("metadata") if isinstance(record.get("metadata"), dict) else {}
    metadata = dict(existing_metadata)
    metadata.update(
        {
            "mem0_migration_source": record.get("source") or "python_oss_qdrant",
            "mem0_migration_collection": source.get("collection"),
            "mem0_migration_local_id": record.get("id"),
            "mem0_migration_local_hash": record.get("hash"),
            "mem0_migration_import_key": migration_import_key(source, record),
        }
    )
    return {key: value for key, value in metadata.items() if value is not None}
 

Sources: scripts/oss-to-platform-migrate.sh:846-859

Streaming and Scope Pagination

When streaming records to the platform, the script queries existing remote memories across affected scopes to prevent duplicate imports. Scopes are extracted from payloads and paginated against the platform API endpoint using designated page sizes and filter parameters.

def platform_get_all_scope(base_url: str, api_key: str, scope_field: str, scope_value: str) -> List[Dict[str, Any]]:
    memories: List[Dict[str, Any]] = []
    page = 1
    while True:
        response = platform_request_json(
            "POST",
            f"{base_url}/v3/memories/?page={page}&page_size={DEFAULT_PLATFORM_PAGE_SIZE}",
            api_key=api_key,
            payload={"filters": {scope_field: scope_value}, "source": "migration"},
        )
        page_results = response.get("results")
        if not isinstance(page_results, list):
            raise MigrationError("Platform get_all response did not include a results list.")
        memories.extend(memory for memory in page_results if isinstance(memory, dict))
        if not response.get("next"):
            return memories
        page += 1
 

Sources: scripts/oss-to-platform-migrate.sh:896-913

Warning

Platform request functions intercept HTTP status codes 401 and 403 to raise an AuthError, instructing operators to verify their MEM0_API_KEY or re-authenticate before retrying the data stream.

Sources: scripts/oss-to-platform-migrate.sh:791-792

Integration provider transition

Mode Selection and Initialization

Switching active client provider implementations between open source engines and hosted platform backends is governed by the createProvider factory function and runtime configuration loading. The plugin inspects the configuration mode flag and instantiates either the self-hosted OSSProvider or the hosted PlatformProvider.

export function createProvider(
  cfg: Mem0Config,
  api: OpenClawPluginApi,
): Mem0Provider {
  if (cfg.mode === "open-source") {
    // v3.0.0: use customInstructions (was customPrompt)
    return new OSSProvider(cfg.oss, cfg.customInstructions, (p) =>
      api.resolvePath(p),
    );
  }
 
  return new PlatformProvider(cfg.apiKey!, cfg.baseUrl);
}

Sources: integrations/openclaw/providers.ts:530-542

const provider = createProvider(cfg, api);
const lifecycle = createMemoryLifecycle();
lifecycle.beginSession();
 
// Create Backend instance — PlatformBackend for platform mode, providerToBackend adapter for OSS
let backend: Backend;
if (cfg.mode === "platform") {
  backend = new PlatformBackend({
    apiKey: cfg.apiKey!,
    baseUrl: cfg.baseUrl ?? "https://api.mem0.ai",
  });
} else {
  backend = providerToBackend(provider, cfg.userId);
}

Sources: integrations/openclaw/index.ts:170-183

Important

When cfg.needsSetup evaluates to true during plugin initialization, the plugin suppresses provider instantiation and enters a setup-waiting state, registering only the CLI and a placeholder service until an API key is configured via openclaw mem0 init.

Sources: integrations/openclaw/index.ts:137-168

Backend Protocol Adapter

Because open source mode uses client-side storage drivers while platform mode communicates directly with hosted API endpoints, the providerToBackend function wraps any Mem0Provider implementation into a uniform Backend interface. This adapter translates backend calls into provider methods while restricting platform-only capabilities like entity management and event tracking.

Backend MethodUnderlying Provider ActionRestriction / Fallback
addprovider.add(msgs, opts)Forces source to "OPENCLAW" and respects inference options.
searchprovider.search(query, opts)Passes filters, top_k, and threshold with source scope.
getprovider.get(memoryId)Directly retrieves single memory items.
listMemoriesprovider.getAll(opts)Paginates local records via page size constraints.
updateprovider.update(memoryId, content)Applies text changes; logs a warning if metadata-only updates are attempted.
deleteprovider.delete / deleteAllClears individual items or all records for a user ID.
deleteEntities—Throws an error: "Entity management is only available in platform mode."
status—Returns { connected: true, backend: "oss" }.
entities—Throws an error: "Entity management is only available in platform mode."
listEvents / getEvent—Throws an error: "Event management is only available in platform mode."
export function providerToBackend(
  provider: Mem0Provider,
  userId: string,
): Backend {
  return {
    async add(content, messages, opts = {}) {
      const msgs = messages ?? (content ? [{ role: "user", content }] : []);
      // v3.0.0: removed immutable, expiration_date
      const result = await provider.add(
        msgs as Array<{ role: string; content: string }>,
        {
          user_id: opts.userId ?? userId,
          source: "OPENCLAW",
          ...(opts.runId && { run_id: opts.runId }),
          ...(opts.metadata && { metadata: opts.metadata }),
          ...(opts.infer === false && { infer: false }),
        },
      );
      return result as unknown as Record<string, unknown>;
    },
 
    async search(query, opts = {}) {
      // v3.0.0: removed keyword_search, reranking
      const results = await provider.search(query, {
        user_id: opts.userId ?? userId,
        top_k: opts.topK,
        threshold: opts.threshold,
        filters: opts.filters,
        source: "OPENCLAW",
      });
      return results as unknown as Record<string, unknown>[];
    },
 
    async get(memoryId) {
      const item = await provider.get(memoryId);
      return item as unknown as Record<string, unknown>;
    },
 
    async listMemories(opts = {}) {
      const items = await provider.getAll({
        user_id: opts.userId ?? userId,
        page_size: opts.pageSize,
        source: "OPENCLAW",
      });
      return items as unknown as Record<string, unknown>[];
    },
 
    async update(memoryId, content, metadata) {
      if (content) await provider.update(memoryId, content);
      if (metadata) {
        // OSS provider doesn't support metadata-only updates — log warning
        console.warn(
          "providerToBackend: metadata updates are not supported in OSS mode, only text updates are applied",
        );
      }
      return { id: memoryId, updated: true };
    },
 
    async delete(memoryId, opts = {}) {
      if (opts.all) {
        await provider.deleteAll(opts.userId ?? userId);
        return { deleted: "all" };
      }
      if (memoryId) {
        await provider.delete(memoryId);
        return { deleted: memoryId };
      }
      throw new Error("Either memoryId or all is required");
    },
 
    async deleteEntities() {
      throw new Error("Entity management is only available in platform mode.");
    },
    async status() {
      return { connected: true, backend: "oss" };
    },
    async entities() {
      throw new Error("Entity management is only available in platform mode.");
    },
    async listEvents() {

Sources: integrations/openclaw/providers.ts:555-641

Warning

Attempting to execute entity management or event inspection through the provider-to-backend adapter in open-source mode will immediately throw execution errors, as those features require a hosted platform backend connection.

Sources: integrations/openclaw/providers.ts:625-640

Validation and post-migration verification

Review Artifact Generation

Post-migration verification ensures that uploaded records match their source counterparts and maintain metadata integrity across the platform boundary. During the import execution loop, the migration routine tracks import outcomes, statuses, and validation failures by compiling review records. When the import run completes, failure states and skipped records are serialized into a timestamped review file to assist operators in auditing data parity.

def review_file_path(input_path: Path) -> Path:
    timestamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
    return input_path.with_name(f"{input_path.stem}-import-review-{timestamp}.json")
 
 
def write_import_review_file(input_path: Path, review_records: List[Dict[str, Any]], summary: Dict[str, Any]) -> Optional[Path]:
    if not review_records:
        return None
    path = review_file_path(input_path)
    payload = {
        "version": 1,
        "kind": "mem0_platform_import_review",
        "created_at": utc_now(),
        "summary": summary,
        "records": review_records,
    }
    path.write_text(json.dumps(payload, indent=2) + "\n", encoding="utf-8")
    os.chmod(path, stat.S_IRUSR | stat.S_IWUSR)
    return path
 

Sources: scripts/oss-to-platform-migrate.sh:941-960

Note

Review files are generated using the local input filename suffixed with -import-review- and an explicit UTC timestamp to prevent concurrent migration runs from overwriting audit logs.

Sources: scripts/oss-to-platform-migrate.sh:941-943

Metadata and Entity Scope Validation

To verify that entity relationships and local provenance survive migration intact, imported records preserve structural linkage through custom metadata fields. Each imported payload embeds migration tracking attributes derived from the source collection and point identifiers.

Metadata FieldSource PropertyPurpose in Platform
mem0_migration_sourcerecord.get("source")Tracks the originating OSS store driver (default "python_oss_qdrant").
mem0_migration_collectionsource.get("collection")Records the source vector collection name.
mem0_migration_local_idrecord.get("id")Preserves the original point ID from the open source database.
mem0_migration_local_hashrecord.get("hash")Retains content hash for collision and duplication checks.
mem0_migration_import_keymigration_import_key(source, record)Stores a SHA-256 digest identifying the unique import stream key.
def build_import_metadata(source: Dict[str, Any], record: Dict[str, Any]) -> Dict[str, Any]:
    existing_metadata = record.get("metadata") if isinstance(record.get("metadata"), dict) else {}
    metadata = dict(existing_metadata)
    metadata.update(
        {
            "mem0_migration_source": record.get("source") or "python_oss_qdrant",
            "mem0_migration_collection": source.get("collection"),
            "mem0_migration_local_id": record.get("id"),
            "mem0_migration_local_hash": record.get("hash"),
            "mem0_migration_import_key": migration_import_key(source, record),
        }
    )
    return {key: value for key, value in metadata.items() if value is not None}
 

Sources: scripts/oss-to-platform-migrate.sh:846-859

Warning

Verification queries against the platform API must ensure that at least one entity scope (user_id, agent_id, or run_id) is correctly mapped; payloads lacking all three entity scopes will fail payload building and trigger an import error.

Sources: scripts/oss-to-platform-migrate.sh:882-884

Was this page helpful?