Python LLM Adapters

Overview

Python language model adapters power the memory extraction, context evaluation, and tool execution pipelines across the memory engine. By providing a uniform interface through Memory Lifecycle integrations, these adapters handle parameter filtering, reasoning model detection, and provider-specific payload formatting for diverse backend providers.

Sources: mem0/llms/base.py:7-11, mem0/memory/main.py:958-964

LLMBase abstraction and lifecycle

Parameter Filtering and Reasoning Model Detection

The LLMBase abstract base class provides the foundational interface and configuration lifecycle for all language model providers in the engine. When an adapter is instantiated, the __init__ method accepts either a BaseLlmConfig instance or a raw configuration dictionary, falling back to a default BaseLlmConfig if no argument is provided, and subsequently executes _validate_config to guarantee that the required model attribute is present on the configuration object.

Sources: mem0/llms/base.py:7-37

The base class implements name-based heuristics and explicit configuration overrides to classify reasoning models and adjust parameter payloads accordingly via _is_reasoning_model and _get_supported_params. An explicit is_reasoning_model boolean on the configuration takes precedence over string matching, allowing custom or versioned model deployments to opt in or out directly. When this configuration attribute is None, the model name is evaluated against a predefined set of base identifiers and prefix families.

Sources: mem0/llms/base.py:38-75

Copyreasoning_models = {
    "o1", "o1-preview", "o3-mini", "o3",
    "gpt-5", "gpt-5o", "gpt-5o-mini", "gpt-5o-micro",
}
 
model_lower = model.lower()
# Strip provider prefixes (e.g. "openai/o3-mini" -> "o3-mini")
base_model = model_lower.rsplit("/", 1)[-1]
 
if base_model in reasoning_models:
    return True
 
# Match o1/o3 family with prefixes (o1-2024-12-17, o3-2025-04-16)
# but NOT gpt-5.x variants (gpt-5.4-mini supports temperature)
if any(base_model.startswith(prefix) for prefix in ["o1-", "o1.", "o3-", "o3."]):
    return True
 
return False

Sources: mem0/llms/base.py:58-75

When parameter filtering executes in _get_supported_params, reasoning models and the gpt-5 family restrict outbound parameters to a strict subset of keys, while regular models incorporate common parameters such as temperature, top_p, and token limits through _get_common_params. Furthermore, _uses_max_completion_tokens checks whether the model starts with gpt-5 to substitute max_tokens with max_completion_tokens on the chat completions payload.

Sources: mem0/llms/base.py:77-170

Structured output adapters

Native OpenAI and Azure OpenAI Structured Classes

The structured output adapters implement strict schema parsing and specialized API interactions for OpenAI and Azure OpenAI deployments. OpenAIStructuredLLM inherits from LLMBase and initializes an OpenAI client using api_key and base_url resolution, falling back to "gpt-5-mini" if the configuration model is empty. Its generate_response method delegates payload assembly to _get_supported_params, injects response_format and tool parameters when present, and executes parsed extractions using client.beta.chat.completions.parse.

Sources: mem0/llms/openai_structured.py:10-49

AzureOpenAIStructuredLLM provides an equivalent interface tailored for Azure deployments, resolving credentials through DefaultAzureCredential and bearer token providers when the api_key is absent, empty, or set to the "your-api-key" placeholder. It manages Azure endpoints, deployments, and API versions through self.config.azure_kwargs or matching environment variables (LLM_AZURE_OPENAI_API_KEY, LLM_AZURE_DEPLOYMENT, LLM_AZURE_ENDPOINT, LLM_AZURE_API_VERSION).

Sources: mem0/llms/azure_openai_structured.py:16-124

Copy@staticmethod
def _rewrite_assistant_keyword(messages):
    """
    Return a copy of ``messages`` with the word "assistant" replaced by "ai"
    in the last message's textual content.
 
    Azure's content management policy can flag the literal word "assistant",
    which makes ``add`` fail (see issue #2636). The rewrite targets that
    trigger without mutating the caller's messages and without assuming the
    content is a string, so multimodal (list) content passes through untouched.
    """
    if not messages:
        return messages
 
    messages = copy.deepcopy(messages)
    last_content = messages[-1].get("content")
    if isinstance(last_content, str):
        messages[-1]["content"] = last_content.replace("assistant", "ai")
    return messages
 

Sources: mem0/llms/azure_openai_structured.py:105-124

Parameter Formatting and Response Parsing

When generating responses under AzureOpenAIStructuredLLM, reasoning models bypass standard sampling parameters like temperature and top_p, while max_tokens is dynamically mapped to max_completion_tokens for reasoning models and gpt-5 series variants. Optional reasoning_effort configurations are appended when specified on the configuration object.

Sources: mem0/llms/azure_openai_structured.py:76-101

Sources: mem0/llms/azure_openai_structured.py:77-101

Response parsing in AzureOpenAIStructuredLLM branches based on whether tools were supplied in the request. If tools are active, _parse_response extracts content from response.choices[0].message.content and iterates over any returned tool_calls, parsing function arguments safely using extract_json and json.loads. Without tools, it returns the raw message content string directly.

Sources: mem0/llms/azure_openai_structured.py:125-153

OpenAI and drop-in compatible providers

OpenAI API Client and Drop-In Integrations

The OpenAI adapter (OpenAILLM) extends LLMBase to manage completions and function calling through the standard OpenAI Python client. If OPENROUTER_API_KEY is present in the environment, it routes requests through OpenRouter using custom base URLs and headers (HTTP-Referer, X-Title). Otherwise, it resolves credentials via self.config.api_key or OPENAI_API_KEY, defaulting to "gpt-5-mini" when no model is specified.

Sources: mem0/llms/openai.py:14-54

Sources: mem0/llms/openai.py:129-135

Compatible providers implement identical initialization and parsing patterns while targeting custom endpoints. DeepSeekLLM defaults to the "deepseek-chat" model, resolving its API key and base URL from DEEPSEEK_API_KEY and DEEPSEEK_API_BASE (falling back to https://api.deepseek.com). XAILLM defaults to "grok-4.3", utilizing XAI_API_KEY and XAI_API_BASE with a default URL of https://api.x.ai/v1. VllmLLM defaults to Qwen/Qwen2.5-32B-Instruct and requires explicit vllm_base_url configuration with a fallback API key of "vllm-api-key". LMStudioLLM defaults to a local GGUF model path and uses "lm-studio" as its default API key.

Sources: mem0/llms/deepseek.py:13-42, mem0/llms/xai.py:13-42, mem0/llms/vllm.py:13-40, mem0/llms/lmstudio.py:12-42

Copyif self.config.lmstudio_response_format:
    params["response_format"] = self.config.lmstudio_response_format
elif response_format:
    params["response_format"] = response_format
else:
    params["response_format"] = {"type": "json_object"}
 

Sources: mem0/llms/lmstudio.py:102-108

Response Parsing and Tool Execution

All adapters in this family share a common _parse_response implementation. When tools are provided in the generation request, the method constructs a dictionary containing the message content and an array of executed tool_calls, where function arguments are parsed from JSON strings using extract_json. When no tools are requested, it returns the raw response content string directly.

Sources: mem0/llms/deepseek.py:43-71, mem0/llms/lmstudio.py:43-71, mem0/llms/xai.py:43-71, mem0/llms/openai.py:55-83, mem0/llms/vllm.py:42-70

Sources: mem0/llms/openai.py:39-51, mem0/llms/deepseek.py:36-40, mem0/llms/xai.py:36-40, mem0/llms/vllm.py:36-39, mem0/llms/lmstudio.py:35-41

Cloud provider abstractions

AWS Bedrock Multi-Model Input Formatting and Parameter Mappings

AWS Bedrock supports diverse model families through BedrockLLM, which normalizes generation requests and translates configuration parameters based on the target provider. Provider parameter mapping dictionaries explicitly map generic configuration keys to model-specific keys:

Sources: mem0/llms/aws_bedrock.py:261-276

Depending on the specific AWS model family selected, _prepare_input constructs distinct body structures:

• Amazon Nova Models: Uses the Converse API format containing a messages array, max_tokens (defaulting to 5000), temperature (defaulting to 0.1), and optional nucleus sampling (top_p).
• Legacy Amazon Models: Populates inputText alongside a textGenerationConfig object containing maxTokenCount and temperature.
• Anthropic Models: Formats messages into a structured block with anthropic_version set to "bedrock-2023-05-31", max_tokens defaulting to 2000, and temperature to 0.1.
• Cohere Command Models: Replaces the standard prompt key with message when "cohere.command" is present in the model identifier.

Sources: mem0/llms/aws_bedrock.py:277-324

Sources: mem0/llms/aws_bedrock.py:326-361

Gemini Message Reformatting and Tool Integration

GeminiLLM instantiates clients using either the standard Google GenAI client or Vertex AI (vertexai=True), defaulting to "gemini-2.0-flash" when no model name is supplied.

Sources: mem0/llms/gemini.py:36-44

The adapter separates incoming chat history via _reformat_messages, isolating system prompts into system_instruction while converting remaining entries into types.Content structures with part lists and role assignments.

Sources: mem0/llms/gemini.py:92-115

Copydef remove_additional_properties(data):
    """Recursively removes 'additionalProperties' from nested dictionaries."""
    if isinstance(data, dict):
        filtered_dict = {
            key: remove_additional_properties(value)
            for key, value in data.items()
            if not (key == "additionalProperties")
        }
        return filtered_dict
    else:
        return data
 

Sources: mem0/llms/gemini.py:128-139

Tool definitions undergo recursive purging of additionalProperties constraints before being wrapped into types.FunctionDeclaration arrays and encapsulated within types.Tool objects.

Sources: mem0/llms/gemini.py:117-156

Tool choice configurations map API parameters directly to Gemini execution modes using types.FunctionCallingConfigMode options:

• auto maps to types.FunctionCallingConfigMode.AUTO
• any maps to types.FunctionCallingConfigMode.ANY, restricting execution via allowed_function_names
• Any other value defaults to types.FunctionCallingConfigMode.NONE

Sources: mem0/llms/gemini.py:204-221

Local and universal runtime bridges

Local Deployment via Ollama

OllamaLLM integrates local models by wrapping the ollama Python library and accepting configuration variants including BaseLlmConfig, OllamaConfig, or raw dictionaries, defaulting to "llama3.1:70b" when model selection is omitted.

Sources: mem0/llms/ollama.py:16-42

Copy# Build parameters for Ollama
params = {
    "model": self.config.model,
    "messages": messages,
}
 
# Handle JSON response format by using Ollama's native format parameter
if response_format and response_format.get("type") == "json_object":
    params["format"] = "json"
    messages = [dict(m) for m in messages]
    if messages and messages[-1]["role"] == "user":
        messages[-1]["content"] += "\n\nPlease respond with valid JSON only."
    else:
        messages.append({"role": "user", "content": "Please respond with valid JSON only."})
    params["messages"] = messages
 
# Add options for Ollama (temperature, num_predict, top_p)
options = {
    "temperature": self.config.temperature,
    "num_predict": self.config.max_tokens,
    "top_p": self.config.top_p,
}
params["options"] = options
 
# Remove OpenAI-specific parameters that Ollama doesn't support
params.pop("max_tokens", None)  # Ollama uses different parameter names
 
if tools:
    params["tools"] = tools
 
response = self.client.chat(**params)
return self._parse_response(response, tools)

Sources: mem0/llms/ollama.py:113-144

Sources: mem0/llms/ollama.py:129-138

Universal Proxying via LiteLLM

LiteLLM routes chat completion requests across diverse provider APIs through the litellm proxy package, defaulting to "gpt-5-mini" if no model string is specified.

Sources: mem0/llms/litellm.py:16-22

Copyif tools and not litellm.supports_function_calling(self.config.model):
    raise ValueError(f"Model '{self.config.model}' in litellm does not support function calling.")
 
params = {
    "model": self.config.model,
    "messages": messages,
    "temperature": self.config.temperature,
    "top_p": self.config.top_p,
}
if self._uses_max_completion_tokens(self.config.model):
    params["max_completion_tokens"] = self.config.max_tokens
else:
    params["max_tokens"] = self.config.max_tokens
if response_format:
    params["response_format"] = response_format
if tools:  # TODO: Remove tools if no issues found with new memory addition logic
    params["tools"] = tools
    params["tool_choice"] = tool_choice
 
response = litellm.completion(**params)

Sources: mem0/llms/litellm.py:72-91

Sources: mem0/llms/litellm.py:72-73

LangChain Model Wrapping

LangchainLLM wraps an instantiated LangChain chat model (BaseChatModel), requiring the model attribute to be supplied directly within the configuration object.

Sources: mem0/llms/langchain.py:13-24

Copy# Convert the messages to LangChain's tuple format
langchain_messages = []
for message in messages:
    role = message["role"]
    content = message["content"]
 
    if role == "system":
        langchain_messages.append(("system", content))
    elif role == "user":
        langchain_messages.append(("human", content))
    elif role == "assistant":
        langchain_messages.append(("ai", content))
 
if not langchain_messages:
    raise ValueError("No valid messages found in the messages list")
 
langchain_model = self.langchain_model
if tools:
    langchain_model = langchain_model.bind_tools(tools=tools, tool_choice=tool_choice)
 
response: AIMessage = langchain_model.invoke(langchain_messages, **kwargs)

Sources: mem0/llms/langchain.py:76-96

Related

• Memory Lifecycle
• TypeScript LLM Adapters