Skip to main content

module aixplain.modules.benchmark_job

class BenchmarkJob​

Benchmark Job Represents a single run of an already created Benchmark.

Attributes:

  • id (str): ID of the Benchmark Job.
  • status (str): Status of the Benchmark Job.
  • benchmark_id (str): ID of the associated parent Benchmark.
  • additional_info (dict): Any additional information to be saved with the Benchmark Job.

method __init__​

__init__(id: str, status: str, benchmark_id: str, **additional_info) → None

Create a Benchmark Job with the necessary information. Each Job is a run of a parent Benchmark

Args:

  • id (Text): ID of the Benchmark Job
  • status (Text): Status of the Benchmark Job
  • benchmark_id (Text): ID of the associated parent Benchmark
  • **additional_info: Any additional Benchmark Job info to be saved

method check_status​

check_status()

method download_results_as_csv​

download_results_as_csv(
    save_path: Optional[str] = None,
    return_dataframe: bool = False
)

Get the results of the benchmark job in a CSV format. The results can either be downloaded locally or returned in the form of pandas.DataFrame.

Args:

  • save_path (Text, optional): Path to save the CSV if return_dataframe is False. If None, a ranmdom path is generated. defaults to None.
  • return_dataframe (bool): If True, the result is returned as pandas.DataFrame else saved as a CSV file. defaults to False.

Returns:

  • str/pandas.DataFrame: results as path of locally saved file if return_dataframe is False else as a pandas dataframe

method get_all_explanations​

get_all_explanations()

method get_failuire_rate​

get_failuire_rate(return_as_dataframe=True)

method get_localized_explanations​

get_localized_explanations(metric_dependant: bool, group_by_task: bool = False)

method get_scores​

get_scores(return_simplified=True, return_as_dataframe=True)