AdaptiveLinear

Our framework BeGin provides the linear layer, which supports variable number of classes and masking the outputs based on the task ids or ideal outputs of the training data.

class AdaptiveLinear(*args: Any, **kwargs: Any)

The linear layer for helping the continual learning procedure, by masking the outputs.

Parameters:

• in_channels (int) – the number of input channels (in_features of torch.nn.Linear).

• out_channels (int) – the number of output channels (out_features of torch.nn.Linear).

• bias (bool) – If set to False, the layer will not learn an additive bias.

• accum (bool) – If set to True, the layer can also provide the logits for the previously seen outputs.

forward(x, task_masks=None)

Returns the masked results of the inner linear layer.

Parameters:

• x (torch.Tensor) – the input features.

• task_masks (torch.Tensor or None) – If task_masks is provided, the layer uses the tensor for masking the outputs. Otherwise, the layer uses the mask managed by the layer.

get_output_mask(task_ids=None)

Returns the mask managed by the layer.

Parameters:

task_ids (torch.Tensor or None) – If task_ids is provided, the layer returns the mask for each index of the task. Otherwise, it returns the mask for the last task it observed.

observe_outputs(new_outputs, verbose=True)

Observes the ideal outputs in the training dataset. By observing the outputs, the layer determines which outputs (logits) will be masked or not.

Parameters:

new_outputs (torch.Tensor) – the ideal outputs in the training dataset.