Skip to main content

How to call an asset

This guide will walk you through calling your deployed models, pipelines and agents.

Models

The aiXplain SDK allows you to run models synchronously (Python) or asynchronously (Python and Swift). You can also process Data assets (Python).

Let's use Groq's Llama 70B as an example.

Docusaurus themed imageDocusaurus themed image
from aixplain.factories import ModelFactory
from aixplain.enums import Supplier

model_list = ModelFactory.list(suppliers=Supplier.GROQ)["results"]
for model in model_list:
print(model.__dict__)
Show output
model = ModelFactory.get("6626a3a8c8f1d089790cf5a2")
note

Model (and pipeline) inputs can be URLs, file paths, or direct text/labels (if applicable).
The examples below use only direct text.

Synchronous

model.run("Tell me a joke about dogs.")
Show output

Use a dictionary to specify additional parameters or if the model takes multiple inputs.

model.run(
{
"text": "Tell me a joke about dogs.",
"max_tokens": 10,
"temperature": 0.5,
}
)
Show output

Asynchronous

start_response = model.run_async("Tell me a joke about dogs.")
start_response
Show output

Use the poll method to monitor inference progress.

while True:
status = model.poll(start_response['url'])
print(status)
if status['status'] != 'IN_PROGRESS':
break
time.sleep(1) # wait for 1 second before checking again
Show output

Process Data Assets

You can also perform inference on Data assets (Corpora or Datasets). You will need to onboard a data asset to use it.

note

Inference on Data assets is only available in Python.

Each data asset has an ID, and each column in that data asset has an ID, too. Specify both to perform inference:

Run

result = model.run(
data="64acbad666608858f693a3a0",
data_asset="64acbad666608858f693a39f"
)

Run Async

start_response = model.run_async(
data="64acbad666608858f693a3a0",
data_asset="64acbad666608858f693a39f"
)