aixplain.modules.finetune.hyperparameters

SchedulerType Objects

class SchedulerType(Text, Enum)

[view_source]

Enum representing different learning rate schedulers.

This enum defines the possible learning rate schedulers that can be used in the fine-tuning process. Each scheduler is represented by a string constant.

Attributes:

• LINEAR Text - Linear learning rate scheduler.
• COSINE Text - Cosine learning rate scheduler.
• COSINE_WITH_RESTARTS Text - Cosine with restarts learning rate scheduler.
• POLYNOMIAL Text - Polynomial learning rate scheduler.
• CONSTANT Text - Constant learning rate scheduler.
• CONSTANT_WITH_WARMUP Text - Constant with warmup learning rate scheduler.
• INVERSE_SQRT Text - Inverse square root learning rate scheduler.
• REDUCE_ON_PLATEAU Text - Reduce learning rate on plateau learning rate scheduler.

Hyperparameters Objects

@dataclass_json

@dataclass
class Hyperparameters(object)

[view_source]

Configuration for the fine-tuning process.

This class encapsulates the hyperparameters for training a model using a fine-tuning approach. It includes settings for epochs, batch sizes, learning rates, sequence lengths, and learning rate schedulers.

Attributes:

• epochs int - Number of training epochs.
• train_batch_size int - Batch size for training.
• eval_batch_size int - Batch size for evaluation.
• learning_rate float - Learning rate for training.
• max_seq_length int - Maximum sequence length for model inputs.
• warmup_ratio float - Warmup ratio for learning rate scheduler.
• warmup_steps int - Number of warmup steps for learning rate scheduler.
• lr_scheduler_type SchedulerType - Type of learning rate scheduler.

__post_init__

def __post_init__()

[view_source]

Post-initialization validation for the hyperparameters.

This method performs validation checks on the hyperparameters after initialization. It ensures that the provided values are of the correct types and within the allowed ranges.

Raises:

• TypeError - If the provided values are not of the correct types.
• ValueError - If the provided values are outside the allowed ranges.