39.2. Models

39.2.1. Model Loading

class memories.models.load_model.LoadModel

Bases: object

__init__(use_gpu=True, model_provider=None, deployment_type=None, model_name=None, api_key=None, endpoint=None, device=None)

Initialize model loader with configuration.

Parameters:

• use_gpu (bool) – Whether to use GPU if available

• model_provider (str) – The model provider (e.g., “deepseek”, “azure-ai”, “mistral”)

• deployment_type (str) – Either “local” or “api”

• model_name (str) – Short name of the model from BaseModel.MODEL_MAPPINGS

• api_key (str) – API key for the model provider (required for API deployment type)

• endpoint (str) – Endpoint URL for the model provider (optional)

• device (str) – Specific GPU device to use (e.g., “cuda:0”, “cuda:1”)

get_response(prompt, **kwargs)

Generate a response using either local model or API.

Parameters:

• prompt (str) – The input prompt

• **kwargs – Additional generation parameters including: max_length: Maximum length of generated response temperature: Sampling temperature (0.0 to 1.0) top_p: Nucleus sampling parameter top_k: Top-k sampling parameter num_beams: Number of beams for beam search

Returns:

Response dictionary containing:

text: The generated response text metadata: Generation metadata (tokens, time, etc) error: Error message if generation failed

Return type:

Dict[str, Any]

cleanup()

Clean up model resources.

get_response_with_context(prompt, context_data, **kwargs)

Generate a response using context-aware prompting.

Parameters:

• prompt (str) – The input prompt

• context_data (Dict[str, Any]) – Dictionary containing contextual information including: - location_info: Location details - raw_data_summary: Summary of raw data from different sources - analysis_results: Results of various analyses - scenario_projections: Future scenario projections - historical_trends: Historical trend analysis

• **kwargs – Additional generation parameters including: max_length: Maximum length of generated response temperature: Sampling temperature (0.0 to 1.0) top_p: Nucleus sampling parameter top_k: Top-k sampling parameter num_beams: Number of beams for beam search

Returns:

Response dictionary containing:

text: The generated response text metadata: Generation metadata including context usage error: Error message if generation failed

Return type:

Dict[str, Any]

chat_completion(messages, tools=None, tool_choice='auto', **kwargs)

Generate a chat completion response using either local model or API.

Parameters:

• messages (List[Dict[str, str]]) – List of message dictionaries with ‘role’ and ‘content’ keys. Roles can be ‘user’, ‘assistant’, ‘system’, or ‘function’.

• tools (List[Dict[str, Any]] | None) – Optional list of tool/function definitions that the model can use. Each tool should have a ‘type’, ‘function’ with ‘name’, ‘description’, ‘parameters’.

• tool_choice (str) – How to handle tool selection. Options: - “auto”: Let the model decide if it should call a function - “none”: Don’t call any functions - Dict with specific function to call

• **kwargs – Additional parameters including: temperature: Sampling temperature (0.0 to 1.0) max_tokens: Maximum tokens in the response top_p: Nucleus sampling parameter frequency_penalty: Frequency penalty parameter presence_penalty: Presence penalty parameter

Returns:

Response dictionary containing:

message: The assistant’s message tool_calls: List of tool calls if any metadata: Generation metadata error: Error message if generation failed

Return type:

Dict[str, Any]

39.2.2. Base Model

class memories.models.base_model.BaseModel

Bases: object

Base model class that can be shared across modules

__init__()

Initialize the base model.

classmethod get_instance()

Get singleton instance of BaseModel.

get_model_config(model_name)

Get configuration for a specific model.

Parameters:

model_name (str) –

Return type:

Dict[str, Any]

initialize_model(model, use_gpu=True, device=None)

Initialize a model with the specified configuration.

Parameters:

• model (str) – Model identifier from config

• use_gpu (bool) – Whether to use GPU if available

• device (str) – Specific GPU device to use (e.g., “cuda:0”, “cuda:1”)

Returns:

True if initialization successful, False otherwise

Return type:

bool

generate(prompt, **kwargs)

Generate text using the model with configured parameters.

Parameters:

• prompt (str) – Input prompt

• **kwargs – Override default generation parameters

Returns:

Generated text

Return type:

str

cleanup()

Clean up model resources.

classmethod get_model_path(provider, model_key)

Get the full model path/identifier for a given provider and model key

Parameters:

• provider (str) –

• model_key (str) –

Return type:

str

classmethod list_providers()

List all available providers

Return type:

List[str]

classmethod list_models(provider=None)

List all available models, optionally filtered by provider

Parameters:

provider (str) –

Return type:

List[str]

39.2.3. API Connectors

API connectors for various model providers.

class memories.models.api_connector.APIConnector

Bases: ABC

Base class for API connectors.

__init__(api_key=None)

Initialize the API connector.

Parameters:

api_key (str | None) –

abstract generate(prompt, **kwargs)

Generate text using the API.

Parameters:

prompt (str) –

Return type:

str

abstract chat_completion(messages, tools=None, tool_choice='auto', **kwargs)

Generate a chat completion from messages and optional tools.

Parameters:

• messages (List[Dict[str, str]]) –

• tools (List[Dict[str, Any]] | None) –

• tool_choice (str) –

Return type:

Dict[str, Any]

class memories.models.api_connector.OpenAIConnector

Bases: APIConnector

Connector for OpenAI API.

__init__(api_key=None)

Initialize the API connector.

Parameters:

api_key (str) –

generate(prompt, model=None, **kwargs)

Generate text using the API.

Parameters:

• prompt (str) –

• model (str) –

Return type:

str

chat_completion(messages, tools=None, tool_choice='auto', **kwargs)

Generate a chat completion using the OpenAI API.

Parameters:

• messages (List[Dict[str, str]]) – List of message dictionaries with ‘role’ and ‘content’ keys

• tools (List[Dict[str, Any]] | None) – Optional list of tools/functions the model can use

• tool_choice (str) – How to handle tool selection (“auto”, “none”, or specific)

• **kwargs – Additional parameters for the API call

Returns:

Dict containing the response message, tool calls, and metadata

Return type:

Dict[str, Any]

class memories.models.api_connector.DeepseekConnector

Bases: APIConnector

Connector for Deepseek API.

__init__(api_key=None)

Initialize the API connector.

Parameters:

api_key (str) –

generate(prompt, model=None, **kwargs)

Generate text using the API.

Parameters:

• prompt (str) –

• model (str) –

Return type:

str

chat_completion(messages, tools=None, tool_choice='auto', **kwargs)

Generate a chat completion using the Deepseek API. Note: Tool usage may not be supported by all Deepseek models.

Parameters:

• messages (List[Dict[str, str]]) –

• tools (List[Dict[str, Any]] | None) –

• tool_choice (str) –

Return type:

Dict[str, Any]

class memories.models.api_connector.AnthropicConnector

Bases: APIConnector

Connector for Anthropic API.

__init__(api_key=None)

Initialize the API connector.

Parameters:

api_key (str) –

generate(prompt, model=None, **kwargs)

Generate text using the API.

Parameters:

• prompt (str) –

• model (str) –

Return type:

str

chat_completion(messages, tools=None, tool_choice='auto', **kwargs)

Generate a chat completion using the Anthropic API. Note: Tool usage may not be supported by all Anthropic models.

Parameters:

• messages (List[Dict[str, str]]) –

• tools (List[Dict[str, Any]] | None) –

• tool_choice (str) –

Return type:

Dict[str, Any]

class memories.models.api_connector.AzureAIConnector

Bases: APIConnector

Connector for Azure AI API.

__init__(api_key=None, endpoint=None)

Initialize the API connector.

Parameters:

• api_key (str) –

• endpoint (str) –

chat_completion(messages, tools=None, tool_choice='auto', **kwargs)

Generate chat completion using Azure AI API.

Parameters:

• messages (List[Dict[str, str]]) –

• tools (List[Dict[str, Any]] | None) –

• tool_choice (str) –

Return type:

Dict[str, Any]

generate(prompt, model=None, **kwargs)

Generate text using Azure AI.

Parameters:

• prompt (str) – Input prompt

• model (str) – Model name (optional)

• **kwargs – Additional arguments

Returns:

Generated text

Return type:

str

memories.models.api_connector.get_connector(provider, api_key=None, endpoint=None)

Get the appropriate API connector based on provider.

Parameters:

• provider (str) – Name of the API provider (openai, azure-ai, anthropic, deepseek)

• api_key (str | None) – Optional API key

• endpoint (str | None) – Optional endpoint URL (for Azure)

Returns:

APIConnector instance

Raises:

ValueError – If provider is not supported

Return type:

APIConnector

class memories.models.api_connector.APIConnector

Bases: ABC

Base class for API connectors.

__init__(api_key=None)

Initialize the API connector.

Parameters:

api_key (str | None) –

abstract generate(prompt, **kwargs)

Generate text using the API.

Parameters:

prompt (str) –

Return type:

str

abstract chat_completion(messages, tools=None, tool_choice='auto', **kwargs)

Generate a chat completion from messages and optional tools.

Parameters:

• messages (List[Dict[str, str]]) –

• tools (List[Dict[str, Any]] | None) –

• tool_choice (str) –

Return type:

Dict[str, Any]

class memories.models.api_connector.OpenAIConnector

Bases: APIConnector

Connector for OpenAI API.

__init__(api_key=None)

Initialize the API connector.

Parameters:

api_key (str) –

generate(prompt, model=None, **kwargs)

Generate text using the API.

Parameters:

• prompt (str) –

• model (str) –

Return type:

str

chat_completion(messages, tools=None, tool_choice='auto', **kwargs)

Generate a chat completion using the OpenAI API.

Parameters:

• messages (List[Dict[str, str]]) – List of message dictionaries with ‘role’ and ‘content’ keys

• tools (List[Dict[str, Any]] | None) – Optional list of tools/functions the model can use

• tool_choice (str) – How to handle tool selection (“auto”, “none”, or specific)

• **kwargs – Additional parameters for the API call

Returns:

Dict containing the response message, tool calls, and metadata

Return type:

Dict[str, Any]

class memories.models.api_connector.AnthropicConnector

Bases: APIConnector

Connector for Anthropic API.

__init__(api_key=None)

Initialize the API connector.

Parameters:

api_key (str) –

generate(prompt, model=None, **kwargs)

Generate text using the API.

Parameters:

• prompt (str) –

• model (str) –

Return type:

str

chat_completion(messages, tools=None, tool_choice='auto', **kwargs)

Generate a chat completion using the Anthropic API. Note: Tool usage may not be supported by all Anthropic models.

Parameters:

• messages (List[Dict[str, str]]) –

• tools (List[Dict[str, Any]] | None) –

• tool_choice (str) –

Return type:

Dict[str, Any]

class memories.models.api_connector.DeepseekConnector

Bases: APIConnector

Connector for Deepseek API.

__init__(api_key=None)

Initialize the API connector.

Parameters:

api_key (str) –

generate(prompt, model=None, **kwargs)

Generate text using the API.

Parameters:

• prompt (str) –

• model (str) –

Return type:

str

chat_completion(messages, tools=None, tool_choice='auto', **kwargs)

Generate a chat completion using the Deepseek API. Note: Tool usage may not be supported by all Deepseek models.

Parameters:

• messages (List[Dict[str, str]]) –

• tools (List[Dict[str, Any]] | None) –

• tool_choice (str) –

Return type:

Dict[str, Any]

39.2.4. Deployment Types

The system supports two deployment types:

1. Local Deployment: Models are loaded and run locally on your machine

2. API Deployment: Models are accessed through provider APIs (OpenAI, Anthropic, Deepseek)

39.2.5. Supported Providers

• deepseek-ai: Models from Deepseek AI

• openai: Models from OpenAI

• anthropic: Models from Anthropic

• meta: Models from Meta

• mistral: Models from Mistral AI