Indexing

The aiXplain indexing service allows you to store, search, filter, and retrieve your private collections of documents with vector-based semantic search. It is the foundation for building AI search, RAG (retrieval-augmented generation), and domain-specific knowledge systems.

This guide focuses on text-based vector databases. Support for image embeddings and graph-structured retrieval for advanced RAG will be added soon.

The default indexing and retrieval model on aiXplain is aiR by aiXplain. Your data is securely stored in aiXplain's managed vector database, powered by Quadrant, and is accessible only through your API key. The system supports filtering and ranked results. It also supports multiple types of embeddings for both text and image data.

How it works

You create an index to store your documents.
You can upsert (insert or update) documents into your index.
You can search documents using natural language queries.
You can apply filters based on metadata fields.

Lifecycle

create() → upsert() → count() → search() → get_document() → delete_document() → delete()

note

Use IndexFactory.list() to list all your accessible indexes, based your API key, and IndexFactory.get(<index_id>) to retrieve a specific index.

Creating an index

Creating an Index asset initializes an empty vector database. The private Index will appear in your dashboard and can be accessed at: https://platform.aixplain.com/discover/model/<index.id> You can use this asset to index additional documents using upsert(), or retrieve results using search().

from aixplain.factories import IndexFactory

index = IndexFactory.create(
    name="My First Index",
    description="An index storing random scientific facts",
)

index.id

Parameters:

name (str): Index name.
description (str): Short description.
embedding_model (EmbeddingModel, optional): Selects the model used to embed your data into vector space (default: OPENAI_ADA002).

note

Assets' name and description help agents reason about tool usage and are displayed on asset cards in Discover.

Supported embedding models

When creating an index, you can specify which embedding model to use. Each model is optimized for different needs:

Model	Description	Recommended for
`OPENAI_ADA002`	OpenAI’s Ada v2 model for semantic text embeddings.	General-purpose text search and RAG applications.
`SNOWFLAKE_ARCTIC_EMBED_L_V2_0`	Snowflake Arctic large embedding model optimized for long documents.	Long-form document retrieval and deep semantic search.
`SNOWFLAKE_ARCTIC_EMBED_M_LONG`	Snowflake Arctic medium model tuned for long inputs with efficiency.	Balanced performance for moderately long documents.
`JINA_CLIP_V2_MULTIMODAL`	Jina’s CLIP v2 model supporting text, image, and cross-modal embeddings.	Multimodal datasets where you search across images and text.

Example usage:

from aixplain.factories import IndexFactory
from aixplain.enums import EmbeddingModel

index = IndexFactory.create(
    name="Long Technical Documents",
    description="Index optimized for in-depth technical papers and reports",
    embedding_model=EmbeddingModel.SNOWFLAKE_ARCTIC_EMBED_M_LONG
)

index.id

Inserting documents (upsert)

You can insert documents into your index using the Record class.

from aixplain.modules.model.record import Record

records = [
    Record(id="doc1", value="The Earth orbits the Sun once every 365.25 days, creating the calendar year.", attributes={"category": "science"}),
    Record(id="doc2", value="Quantum computing promises to solve complex problems that are currently intractable for classical computers.", attributes={"category": "technology"}),
]

# Upsert documents
index.upsert(records)

Parameters:

id (optional): Document ID. Auto-generated if not provided. Must be unique per document.
value (str): Text content (or extracted content from files).
value_type: Defines the type of the value contained in the Record: DataType.TEXT or DataType.IMAGE (default: text).
attributes (dict): Optional metadata (e.g., author, category, date) for filtering.
uri (optional): If uploading files (e.g., images), you can use uri instead of value, and the file will be processed.

Notes:

When value_type is set to DataType.TEXT, the value field should contain a text string.
When value_type is set to DataType.IMAGE, the uri field must be provided, pointing to an image file or URL.
If a file path or URL is provided for an image and it is valid, the Record automatically adjusts the value_type to DataType.IMAGE.

Searching an index

Perform a semantic search using natural language queries:

response = index.search("What is AI?")

Parameters:

query (str): Natural language query.
top_k (int, optional): Number of rankedresults (default = 10).
filters (list, optional): List of IndexFilter to filter based on metadata.

response.details # List of top results with scores and metadata

response.data # Top ranking data

response # List of top results with scores and metadata

Show output

ModelResponse fields:

status: The final status of the request (e.g., SUCCESS, FAILURE).
data: The top-ranked retrieved result or output, typically the most relevant match.
details
A list of matched documents with associated information:
- score: Relevance score indicating how closely the document matches the query.
- data: Text snippet or content retrieved from the document.
- document: ID of the document where the match was found.
- metadata: Additional metadata about the document (e.g., category).
completed: Boolean indicating whether the operation completed successfully (true or false).
used_credits: The amount of credits consumed by the operation.
run_time: The total time (in seconds) it took to complete the operation.

Filterng an index

You can also filter your search results by metadata:

from aixplain.modules.model.index_model import IndexFilter, IndexFilterOperator

filters = [
    IndexFilter(field="category", value="technology", operator=IndexFilterOperator.EQUALS),
]

response = index.search(
    query="Artificial Intelligence",
    filters=filters
)

response.details

Available filter operators:

Operator	Meaning
EQUALS (`==`)	Field equals value
NOT_EQUALS (`!=`)	Field does not equal value
CONTAINS (`in`)	Field contains value
NOT_CONTAINS (`not in`)	Field does not contain value
GREATER_THAN (`>`)	Field greater than value
LESS_THAN (`<`)	Field less than value
GREATER_THAN_OR_EQUALS (`>=`)	Field greater than or equal
LESS_THAN_OR_EQUALS (`<=`)	Field less than or equal

note

Filtering by metadata is supported in the IndexModel. Agent-based filters are coming soon.

Retrieving information

Get document count:

index.count()

Show output

Get a specific document by ID:

index.get_document(document_id="doc1")

Show output

Delete a document by ID:

index.delete_document(document_id="doc1")

Index usage with agents

Indexes can be used directly as tools inside aiXplain agents for retrieval-augmented generation (RAG).

from aixplain.factories import AgentFactory

agent = AgentFactory.create(
    name="Knowledge Assistant",
    description="Answers based on the provided index",
    tools=[
        AgentFactory.create_model_tool(index.id)
    ]
)

agent_response = agent.run("What is quantum computing promising?")

agent_response

Show output

The agent will now search the index automatically when answering questions.

Learn more about agent tools →

Full example

from aixplain.factories import IndexFactory
from aixplain.modules.model.record import Record
from aixplain.modules.model.index_model import IndexFilter, IndexFilterOperator

# Create
index = IndexFactory.create(name="Knowledge Base", description="General knowledge across technology, art, and science.")

# Prepare data
data = [
    {
        "id": "doc1",
        "text": "Artificial intelligence is transforming industries worldwide, from healthcare to finance.",
        "category": "technology"
    },
    {
        "id": "doc2",
        "text": "The Mona Lisa, painted by Leonardo da Vinci, is one of the most famous artworks in history.",
        "category": "art"
    },
    {
        "id": "doc3",
        "text": "Machine learning algorithms are being used to predict patient outcomes in hospitals.",
        "category": "technology"
    },
    {
        "id": "doc4",
        "text": "The Earth orbits the Sun once every 365.25 days, creating the calendar year.",
        "category": "science"
    },
    {
        "id": "doc5",
        "text": "Quantum computing promises to solve complex problems that are currently intractable for classical computers.",
        "category": "technology"
    },
]

# Upsert
records = []
for item in data:
    record = Record(
        id=item["id"],
        value=item["text"],
        attributes={"category": item["category"]}
    )
    records.append(record)

index.upsert(records)

# Search
response = index.search("artificial intelligence")

# Filtered search
filters = [IndexFilter(field="category", value="technology", operator=IndexFilterOperator.EQUALS)]
filtered_response = index.search("computing", filters=filters)

filtered_response

Show output

How it works​

Lifecycle​

Creating an index​

Supported embedding models​

Inserting documents (upsert)​

Searching an index​

Filterng an index​

Retrieving information​

Index usage with agents​

Full example​

How it works

Lifecycle

Creating an index

Supported embedding models

Inserting documents (upsert)

Searching an index

Filterng an index

Retrieving information

Index usage with agents

Full example