Skip to main content

module aixplain.modules.model.llm_model

class LLM​

Ready-to-use LLM model. This model can be run in both synchronous and asynchronous manner.

Attributes:

  • id (Text): ID of the Model
  • name (Text): Name of the Model
  • description (Text, optional): description of the model. Defaults to "".
  • api_key (Text, optional): API key of the Model. Defaults to None.
  • url (Text, optional): endpoint of the model. Defaults to config.MODELS_RUN_URL.
  • supplier (Union[Dict, Text, Supplier, int], optional): supplier of the asset. Defaults to "aiXplain".
  • version (Text, optional): version of the model. Defaults to "1.0".
  • function (Text, optional): model AI function. Defaults to None.
  • url (str): URL to run the model.
  • backend_url (str): URL of the backend.
  • pricing (Dict, optional): model price. Defaults to None.
  • **additional_info: Any additional Model info to be saved

method __init__​

__init__(
    id: str,
    name: str,
    description: str = '',
    api_key: Optional[str] = None,
    supplier: Union[Dict, str, Supplier, int] = 'aiXplain',
    version: Optional[str] = None,
    function: Optional[Function] = None,
    is_subscribed: bool = False,
    cost: Optional[Dict] = None,
    temperature: float = 0.001,
    **additional_info
) → None

LLM Init

Args:

  • id (Text): ID of the Model
  • name (Text): Name of the Model
  • description (Text, optional): description of the model. Defaults to "".
  • api_key (Text, optional): API key of the Model. Defaults to None.
  • supplier (Union[Dict, Text, Supplier, int], optional): supplier of the asset. Defaults to "aiXplain".
  • version (Text, optional): version of the model. Defaults to "1.0".
  • function (Function, optional): model AI function. Defaults to None.
  • is_subscribed (bool, optional): Is the user subscribed. Defaults to False.
  • cost (Dict, optional): model price. Defaults to None.
  • **additional_info: Any additional Model info to be saved

method run​

run(
    data: str,
    context: Optional[str] = None,
    prompt: Optional[str] = None,
    history: Optional[List[Dict]] = None,
    temperature: Optional[float] = None,
    max_tokens: int = 128,
    top_p: float = 1.0,
    name: str = 'model_process',
    timeout: float = 300,
    parameters: Optional[Dict] = None,
    wait_time: float = 0.5
) → ModelResponse

Synchronously running a Large Language Model (LLM) model.

Args:

  • data (Union[Text, Dict]): Text to LLM or last user utterance of a conversation.
  • context (Optional[Text], optional): System message. Defaults to None.
  • prompt (Optional[Text], optional): Prompt Message which comes on the left side of the last utterance. Defaults to None.
  • history (Optional[List[Dict]], optional): Conversation history in OpenAI format ([{ "role": "assistant", "content": "Hello, world!"}]). Defaults to None.
  • temperature (Optional[float], optional): LLM temperature. Defaults to None.
  • max_tokens (int, optional): Maximum Generation Tokens. Defaults to 128.
  • top_p (float, optional): Top P. Defaults to 1.0.
  • name (Text, optional): ID given to a call. Defaults to "model_process".
  • timeout (float, optional): total polling time. Defaults to 300.
  • parameters (Dict, optional): optional parameters to the model. Defaults to None.
  • wait_time (float, optional): wait time in seconds between polling calls. Defaults to 0.5.

Returns:

  • Dict: parsed output from model

method run_async​

run_async(
    data: str,
    context: Optional[str] = None,
    prompt: Optional[str] = None,
    history: Optional[List[Dict]] = None,
    temperature: Optional[float] = None,
    max_tokens: int = 128,
    top_p: float = 1.0,
    name: str = 'model_process',
    parameters: Optional[Dict] = None
) → ModelResponse

Runs asynchronously a model call.

Args:

  • data (Union[Text, Dict]): Text to LLM or last user utterance of a conversation.
  • context (Optional[Text], optional): System message. Defaults to None.
  • prompt (Optional[Text], optional): Prompt Message which comes on the left side of the last utterance. Defaults to None.
  • history (Optional[List[Dict]], optional): Conversation history in OpenAI format ([{ "role": "assistant", "content": "Hello, world!"}]). Defaults to None.
  • temperature (Optional[float], optional): LLM temperature. Defaults to None.
  • max_tokens (int, optional): Maximum Generation Tokens. Defaults to 128.
  • top_p (float, optional): Top P. Defaults to 1.0.
  • name (Text, optional): ID given to a call. Defaults to "model_process".
  • parameters (Dict, optional): optional parameters to the model. Defaults to None.

Returns:

  • dict: polling URL in response