Skip to main content

module aixplain.modules.metric

class Metric​

Represents a metric to be computed on one or more peices of data. It is usually linked to a machine learning task.

Attributes:

  • id (Text): ID of the Metric
  • name (Text): Name of the Metric
  • description (Text): Description of the Metric
  • supplier (Text, optional): author of the Metric. Defaults to "aiXplain".
  • version (Text, optional): Metric version. Defaults to "1.0".
  • additional_info: Any additional Metric info to be saved

method __init__​

__init__(
    id: str,
    name: str,
    supplier: str,
    is_reference_required: bool,
    is_source_required: bool,
    cost: float,
    function: str,
    normalization_options: list = [],
    **additional_info
) → None

Create a Metric with the necessary information

Args:

  • id (Text): ID of the Metric
  • name (Text): Name of the Metric
  • supplier (Text): author of the Metric
  • is_reference_required (bool): does the metric use reference
  • is_source_required (bool): does the metric use source
  • cost (float): price of the metric normalization_options(list, [])
  • **additional_info: Any additional Metric info to be saved

method add_normalization_options​

add_normalization_options(normalization_options: List[str])

Add a given set of normalization options to be used while benchmarking

Args:

  • normalization_options (List[str]): List of normalization options to be added

method run​

run(
    hypothesis: Optional[str, List[str]] = None,
    source: Optional[str, List[str]] = None,
    reference: Optional[str, List[str]] = None
)

Run the metric to calculate the scores.

Args:

  • hypothesis (Optional[Union[str, List[str]]], optional): Can give a single hypothesis or a list of hypothesis for metric calculation. Defaults to None.
  • source (Optional[Union[str, List[str]]], optional): Can give a single source or a list of sources for metric calculation. Defaults to None.
  • reference (Optional[Union[str, List[str]]], optional): Can give a single reference or a list of references for metric calculation. Defaults to None.