module aixplain.modules.benchmark

---

class Benchmark

Benchmark is a powerful tool for benchmarking machine learning models and evaluating their performance on specific tasks. It represents a collection of Models, Datasets and Metrics to run associated Benchmark Jobs.

Attributes:

• id (str): ID of the Benchmark.
• name (str): Name of the Benchmark.
• model_list (List[Model]): List of Models to be used for benchmarking.
• dataset_list (List[Dataset]): List of Datasets to be used for benchmarking.
• metric_list (List[Metric]): List of Metrics to be used for benchmarking.
• job_list (List[BenchmarkJob]): List of associated Benchmark Jobs.
• additional_info (dict): Any additional information to be saved with the Benchmark.

method __init__

__init__(
    id: str,
    name: str,
    dataset_list: List[Dataset],
    model_list: List[Model],
    metric_list: List[Metric],
    job_list: List[BenchmarkJob],
    description: str = '',
    supplier: str = 'aiXplain',
    version: str = '1.0',
    **additional_info
) → None

Create a Benchmark with the necessary information.

Args:

• id (Text): ID of the Benchmark.
• name (Text): Name of the Benchmark.
• model_list (List[Model]): List of Models to be used for benchmarking
• dataset_list (List[Dataset]): List of Datasets to be used for benchmarking
• metric_list (List[Metric]): List of Metrics to be used for benchmarking
• job_list (List[BenchmarkJob]): List of associated Benchmark Jobs
• supplier (Text, optional): author of the Benchmark. Defaults to "aiXplain".
• version (Text, optional): Benchmark version. Defaults to "1.0".
• **additional_info: Any additional Benchmark info to be saved

---

method start

start() → BenchmarkJob

Starts a new benchmark job(run) for the current benchmark

Returns:

• BenchmarkJob: Benchmark Job that just got started