What Are Memory Templates?

Memory Templates are reusable, customizable structures that define how information is stored, retrieved, and managed within Palo Bloom, Mpalo's advanced AI memory system. They serve as blueprints for memory, enabling standardized data representation for common use cases like user profiles, complex project tracking, conversational histories, and much more. By utilizing templates, developers and users can significantly accelerate AI application development, ensure data consistency, and foster interoperability.

Beyond individual utility, Memory Templates are foundational to the upcoming Palo Marketplace. Well-designed templates can be shared, discovered, and utilized by the broader community, with future opportunities for creators to participate in profit-sharing models based on template adoption and utility. They are key to unlocking collaborative AI development and democratizing access to sophisticated memory structures.

Why Use Memory Templates?

• Accelerate development with pre-built structures for common memory needs.
• Ensure consistency and best practices across your AI applications.
• Enable rapid prototyping and iteration for new use cases.
• Reduce errors by using tested, community-reviewed templates.

How to Use Memory Templates

1. Choose a template that matches your use case (e.g., user profile, chat session, project task).
2. Customize fields and structure as needed for your application.
3. Integrate the template with your API or SDK calls (see below).
4. Store, retrieve, and update memory using the standardized template format.

Memory Templates & The Palo Marketplace

Memory Templates are not just individual tools; they are a cornerstone of the upcoming Palo Marketplace. This ecosystem will empower developers and domain experts to share, discover, and leverage powerful, pre-built memory structures.

Profit-Sharing Opportunities

Mpalo is committed to recognizing and rewarding valuable contributions. High-quality, widely adopted Memory Templates in the Palo Marketplace will be eligible for profit-sharing programs. This means your innovative template designs can generate ongoing revenue as they are utilized by other users and integrated into applications.

• Monetize Your Expertise: Turn your understanding of data structuring into a valuable asset.
• Drive Adoption: Templates that solve real-world problems effectively will naturally gain traction.
• Fair Compensation: Our model will aim to transparently reward creators based on usage and impact. (Further details on the profit-sharing model will be announced closer to the Marketplace launch.)

Start thinking about how your unique insights into data organization can translate into powerful, shareable Memory Templates. The Marketplace will provide the platform; your creativity will fuel its success.

Designing Effective Memory Templates

Creating a good Memory Template goes beyond just listing fields. An effective template is intuitive, robust, and adaptable. Whether for personal use or for sharing on the Palo Marketplace, consider these design principles:

{
  "template_name": "BasicProductListing",
  "description": "A simple template for e-commerce product listings.",
  "version": "1.0.0",
  "fields": [
    { "name": "product_id", "type": "string", "description": "Unique identifier for the product.", "required": true },
    { "name": "product_name", "type": "string", "description": "Display name of the product.", "required": true },
    { "name": "price", "type": "number", "description": "Retail price in USD.", "required": true },
    { "name": "short_description", "type": "string", "description": "A brief summary for product cards." }
  ]
}

By following these guidelines, you can create Memory Templates that are not only functional but also valuable assets for yourself and the wider Mpalo community.

Example Memory Templates

{
  "user_id": "string",
  "name": "string",
  "preferences": {
    "theme": "string",
    "language": "string"
  },
  "history": [
    {"date": "string", "event": "string"}
  ]
}

{
  "task_id": "string",
  "title": "string",
  "status": "string",
  "assigned_to": "string",
  "history": [
    {"date": "string", "update": "string"}
  ]
}

Populating Memories via Document Migration

A powerful way to populate memories in Mpalo is by uploading your existing data files. This feature allows you to transform structured data (like CSV, Excel, or JSON files) into rich, semantic memories that Palo can effectively use. You can leverage Memory Templates – either your own custom designs or those discovered on the Palo Marketplace – to define the structure and ensure consistency for your ingested data. This process is designed to be transparent, user-friendly, and give you maximum control over how your data is interpreted.

The Upload & Ingestion Flow

Here's a step-by-step breakdown of how you can create memories from your documents:

1. User Uploads File: You can directly upload files in common formats such as CSV (Comma Separated Values), Excel (XLSX, XLS), or JSON (JavaScript Object Notation). This is ideal for users with existing datasets.
2. Mpalo Reads Rows & Estimates Price: Upon upload, Mpalo analyzes the file structure and content. We provide an upfront estimate of the processing cost. This estimation is typically based on the total token count that would be generated from all rows after being processed with your defined format string. This transparency helps you manage expectations and trust. For very large files, Mpalo might process a sample to estimate or stream chunks.
3. User Chooses a Name for the Memory/Dataset: You'll be prompted to give your new memory set a descriptive name. This helps in organizing and easily identifying different memory sets within your Workbench.
4. Define Data Structure (Using a Template or Format String): This is a key step for enriching and structuring your data for Palo. You have a couple of powerful options:
   • Select a Memory Template: Browse and select from your existing Memory Templates or discover suitable ones on the Palo Marketplace. A chosen template can pre-define the target structure, expected fields, and may even suggest a format string or column mappings, significantly streamlining the process.
   • Define a Custom Format String: Alternatively, or for further customization, you can define a "format string" that dictates how each row from your file is converted into a natural language sentence or structured text entry.
   If you opt to define a custom format string, for example, if your CSV has columns name, id, role, and email, you might use: "{name} with the id {id} has the role {role} under the email alias {email}."
   • Benefit - Semantic Richness: This transforms raw data fields into meaningful sentences, making the resulting embeddings far more effective for Palo's retrieval and understanding.
   • Benefit - User Control: You can tailor the "narrative" of each memory entry, prioritize important fields, or add contextual explanations.
   • Benefit - Flexibility: This approach adapts to various dataset schemas without requiring rigid, predefined templates from Mpalo.

   Another example, for a product catalog with product_name, category, and description:

   "Product: {product_name}, Category: {category}. Description: {description}"

   Or for Q&A pairs with question and answer:

   "Question: {question} Answer: {answer}"

Key Technical & UX Considerations

To ensure a smooth and effective experience, consider the following:

• File Parsing Robustness: Mpalo aims for robust parsing of different CSV delimiters, Excel sheets, and JSON structures (e.g., arrays of objects, nested objects). We provide error handling for malformed files.
• Format String Parsing: The system accurately identifies and replaces {field_name} placeholders from your file's columns/keys. If a field_name in your format string doesn't exist in a row, Mpalo typically skips that field or allows for default values if specified (e.g., {email|N/A} or syntax like {notes?:\'\'\'No notes provided.\'\'\'}).
• Interactive Preview (Recommended UX): Where possible, the Workbench will show a real-time preview of how the first few rows of your data will be processed based on your chosen format string or selected template. This allows you to iterate and refine your configuration before committing to the full upload and processing.
• Column Mapping (CSV/Excel): For CSV and Excel files, you may need to confirm or map which uploaded column corresponds to which field_name you intend to use in your format string or template. This is often handled via simple dropdowns or an intuitive mapping interface in the Workbench.
• Background Processing & Notifications: For large files, the embedding creation and vector store insertion happen as a background process. The Workbench will provide status updates (e.g., "Processing 500/2000 rows...", "Memory creation complete!"). You'll be notified when processing is finished or if any critical errors occurred.
• Leveraging Memory Templates during Ingestion: While you have full control with custom format strings, the Workbench will facilitate using existing Memory Templates to guide ingestion:
  • Template Selection: You'll be able to select from your personal library of Memory Templates or browse suitable options from the Palo Marketplace.
  • Automatic Configuration: Choosing a template can automatically configure column mappings (if field names match or are mappable) and suggest an optimized format string based on the template's structure.
  • Guided Structuring: This significantly simplifies creating well-structured memories, ensuring they conform to best practices and are ready for use with Palo's advanced features. You can always customize the suggested configurations.

Understanding Potential Errors

During the document upload and memory creation process, you might encounter some issues. Here's a guide to common errors and their potential meanings:

• Malformed File: This error indicates that the uploaded file doesn't conform to the expected structure for its type (e.g., a CSV with inconsistent delimiters, or a badly structured JSON). Solution: Check your file for formatting errors, ensure consistent use of delimiters (for CSV), valid JSON syntax, or correct Excel formatting. Try saving the file again from its source program.
• Missing Fields in Format String: If your format string uses a placeholder like {product_sku} but a row in your data doesn't have a 'product_sku' column or key, that specific part of the string might be empty or cause a row-level warning. Solution: Ensure all fields referenced in your format string exist in your data. Use default value syntax if some fields are optional (e.g., {product_sku|NOT_AVAILABLE}). The interactive preview can help catch these.
• Processing Errors / Embedding Failures: Sometimes, individual rows might fail during the embedding process. This could be due to excessively long text generated by the format string (exceeding model context limits) or other transient issues with the embedding service. Solution: Mpalo usually attempts to process as much of the file as possible and will report on rows that failed. Review the failed rows. You might need to shorten the content for those rows or simplify your format string if it generates overly verbose outputs.
• Unsupported File Type: If you upload a file type not currently supported by the uploader. Solution: Convert your file to one of the supported formats (CSV, Excel, JSON).
• Upload Timeout / Size Limit Exceeded: Very large files might exceed upload size limits or timeout during transfer. Solution: Try breaking down very large files into smaller chunks. Check documentation for current size limits.
• Hugging Face - Trust Remote Code Error: When attempting to load certain models or datasets from Hugging Face, you might encounter an error requiring you to `trust_remote_code=True`. This means the model or dataset requires custom code to be executed. Solution: If using the Mpalo API or SDK for this, there might be a parameter (e.g., `allow_remote_code=True`) or a setting in the Workbench to enable this. Be cautious and only enable this for content from trusted sources, as executing arbitrary remote code can be a security risk.
• Hugging Face - Split Not Available / Invalid Split: When loading a dataset from Hugging Face, you might request a specific split (e.g., 'train', 'test', 'validation') that doesn't exist for that particular dataset, or the dataset might not be structured with standard splits. Solution: First, check the dataset's page on Hugging Face to see what splits are available. You may need to specify a different split name. Some datasets might not have predefined splits, or they might use different naming conventions. If no suitable split is found, you might need to process the entire dataset if appropriate or look for alternative datasets.
• Hugging Face - Model/Dataset Requires Authentication: Some Hugging Face models or datasets are gated and require authentication (e.g., accepting terms of service or needing a Hugging Face Hub token). Solution: Ensure you are logged into your Hugging Face account if using their direct tools. If accessing via Mpalo's Workbench or API, there might be a section to securely provide your Hugging Face API token or a mechanism to handle such authentications. Refer to Mpalo's specific documentation for connecting to private/gated Hugging Face resources.

This sophisticated yet intuitive approach to data ingestion is designed to significantly elevate your ability to create powerful, custom memories for Palo. It empowers you to extract maximum value from your existing data and put Palo to work effectively and immediately.

API & SDK Integration

Memory Templates, whether custom-defined or sourced from the Palo Marketplace, are seamlessly integrated into your workflows via the Mpalo API and SDKs. You can reference templates by name (for your privately defined templates or globally recognized ones) or by a unique Marketplace ID.

SDK Examples (Python)

Interacting with memories using templates is straightforward with our Python SDK.

# Example: Storing data using a private/named Memory Template
from mpalo import Client, MemoryData

# Initialize the client (ensure API key is configured)
client = Client(api_key="YOUR_API_KEY")

# Data conforming to your 'user_profile_v2' template
user_data = MemoryData(
    template_name="user_profile_v2", # Your custom template name
    data={
        "user_id": "user_001",
        "displayName": "AliceWonder",
        "email": "alice@example.com",
        "preferences": {
            "theme": "dark",
            "notifications": {"email": True, "push": False}
        },
        "last_login": "2024-03-15T10:00:00Z"
    }
)
try:
    memory_id = client.memory.store(user_data)
    print(f"Successfully stored memory with ID: {memory_id} using template: {user_data.template_name}")
except Exception as e:
    print(f"Error storing memory: {e}")

# Example: Storing data using a Memory Template from the Palo Marketplace
marketplace_user_profile_data = MemoryData(
    marketplace_template_id="mpalo/user-profile-standard-v1.2", # Hypothetical Marketplace ID
    data={
        "userId": "user_002", # Field names as per the Marketplace template
        "username": "BobTheBuilder",
        "contact": {"emailAddress": "bob@example.com"},
        "settings": {"interfaceLanguage": "en_GB"}
    }
)
try:
    memory_id_market = client.memory.store(marketplace_user_profile_data)
    print(f"Successfully stored memory with ID: {memory_id_market} using Marketplace template: {marketplace_user_profile_data.marketplace_template_id}")
except Exception as e:
    print(f"Error storing memory with Marketplace template: {e}")

# Discovering templates (conceptual)
# marketplace_templates = client.marketplace.list_templates(category="productivity", sort_by="popularity")
# my_templates = client.templates.list()

The template_name parameter is used for your organization's private templates, while marketplace_template_id would be used to reference templates acquired or available from the Palo Marketplace. The actual SDK methods for template discovery and management will be detailed in the official SDK documentation.

API Endpoints

Our REST API provides comprehensive endpoints for managing Memory Templates and interacting with memories based on these templates. Key functionalities include:

• POST /v1/memories: Create new memory entries, specifying a template_name or marketplace_template_id along with the conforming data payload.
• GET /v1/memories/{memory_id}: Retrieve memory entries, which will include metadata about the template used.
• GET /v1/templates: List your organization's private Memory Templates.
• POST /v1/templates: Create new private Memory Templates.
• GET /v1/templates/{template_name_or_id}: Retrieve a specific private Memory Template definition.
• PUT /v1/templates/{template_name_or_id}: Update an existing private Memory Template.
• GET /v1/marketplace/templates: Discover and search for Memory Templates available on the Palo Marketplace (e.g., filter by category, search by keyword).
• GET /v1/marketplace/templates/{marketplace_template_id}: Fetch the definition of a specific Marketplace template.

For detailed request/response schemas, authentication, and advanced usage, please consult the full SDK Documentation and the comprehensive API Reference.