LwF

Learning without Forgetting (LwF) is a continual learning method that minimizes the difference between outputs of a previous model and a new model. For the details, see the original paper.

Node-level Problems

class NCClassILLwFMinibatchTrainer(model, scenario, optimizer_fn, loss_fn, device, **kwargs)[source]

afterInference(results, model, optimizer, _curr_batch, training_states)[source]

The event function to execute some processes right after the inference step (for training). We recommend performing backpropagation in this event function.

LwF performs knowledge distillation process in this function.

Parameters:

results (dict) – the returned dictionary from the event function inference.
model (torch.nn.Module) – the current trained model.
optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_batch (object) – the data (or minibatch) for the current iteration.
curr_training_states (dict) – the dictionary containing the current training states.

Returns:

A dictionary containing the information from the results.

processAfterTraining(task_id, curr_dataset, curr_model, curr_optimizer, curr_training_states)[source]

The event function to execute some processes after training.

We need to store previously learned weights for the knowledge distillation process in afterInference().

Parameters:

task_id (int) – the index of the current task
curr_dataset (object) – The dataset for the current task.
curr_model (torch.nn.Module) – the current trained model.
curr_optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_training_states (dict) – the dictionary containing the current training states.

class NCClassILLwFTrainer(model, scenario, optimizer_fn, loss_fn, device, **kwargs)[source]

afterInference(results, model, optimizer, _curr_batch, training_states)[source]

The event function to execute some processes right after the inference step (for training). We recommend performing backpropagation in this event function.

LwF performs knowledge distillation process in this function.

Parameters:

results (dict) – the returned dictionary from the event function inference.
model (torch.nn.Module) – the current trained model.
optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_batch (object) – the data (or minibatch) for the current iteration.
curr_training_states (dict) – the dictionary containing the current training states.

Returns:

A dictionary containing the information from the results.

processAfterTraining(task_id, curr_dataset, curr_model, curr_optimizer, curr_training_states)[source]

The event function to execute some processes after training.

We need to store previously learned weights for the knowledge distillation process in afterInference().

Parameters:

task_id (int) – the index of the current task
curr_dataset (object) – The dataset for the current task.
curr_model (torch.nn.Module) – the current trained model.
curr_optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_training_states (dict) – the dictionary containing the current training states.

class NCDomainILLwFTrainer(model, scenario, optimizer_fn, loss_fn, device, **kwargs)[source]: This trainer has the same behavior as NCClassILLwFTrainer.

class NCTaskILLwFTrainer(model, scenario, optimizer_fn, loss_fn, device, **kwargs)[source]

afterInference(results, model, optimizer, _curr_batch, training_states)[source]

The event function to execute some processes right after the inference step (for training). We recommend performing backpropagation in this event function.

LwF performs knowledge distillation process in this function.

Parameters:

results (dict) – the returned dictionary from the event function inference.
model (torch.nn.Module) – the current trained model.
optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_batch (object) – the data (or minibatch) for the current iteration.
curr_training_states (dict) – the dictionary containing the current training states.

Returns:

A dictionary containing the information from the results.

inference(model, _curr_batch, training_states)[source]

The event function to execute inference step.

For task-IL, we need to additionally consider task information for the inference step.

Parameters:

model (torch.nn.Module) – the current trained model.
curr_batch (object) – the data (or minibatch) for the current iteration.
curr_training_states (dict) – the dictionary containing the current training states.

Returns:

A dictionary containing the inference results, such as prediction result and loss.

processBeforeTraining(task_id, curr_dataset, curr_model, curr_optimizer, curr_training_states)[source]

The event function to execute some processes before training.

We need to store previously learned weights for the knowledge distillation process in afterInference().

Parameters:

task_id (int) – the index of the current task
curr_dataset (object) – The dataset for the current task.
curr_model (torch.nn.Module) – the current trained model.
curr_optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_training_states (dict) – the dictionary containing the current training states.

class NCTimeILLwFTrainer(model, scenario, optimizer_fn, loss_fn, device, **kwargs)[source]: This trainer has the same behavior as NCClassILLwFTrainer.

Link-level Problems

class LCClassILLwFTrainer(model, scenario, optimizer_fn, loss_fn, device, **kwargs)[source]

afterInference(results, model, optimizer, _curr_batch, training_states)[source]

The event function to execute some processes right after the inference step (for training). We recommend performing backpropagation in this event function.

LwF performs knowledge distillation process in this function.

Parameters:

results (dict) – the returned dictionary from the event function inference.
model (torch.nn.Module) – the current trained model.
optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_batch (object) – the data (or minibatch) for the current iteration.
curr_training_states (dict) – the dictionary containing the current training states.

Returns:

A dictionary containing the information from the results.

processAfterTraining(task_id, curr_dataset, curr_model, curr_optimizer, curr_training_states)[source]

The event function to execute some processes after training.

We need to store previously learned weights for the knowledge distillation process in afterInference().

Parameters:

task_id (int) – the index of the current task
curr_dataset (object) – The dataset for the current task.
curr_model (torch.nn.Module) – the current trained model.
curr_optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_training_states (dict) – the dictionary containing the current training states.

class LCTaskILLwFTrainer(model, scenario, optimizer_fn, loss_fn, device, **kwargs)[source]

afterInference(results, model, optimizer, _curr_batch, training_states)[source]

The event function to execute some processes right after the inference step (for training). We recommend performing backpropagation in this event function.

LwF performs knowledge distillation process in this function.

Parameters:

results (dict) – the returned dictionary from the event function inference.
model (torch.nn.Module) – the current trained model.
optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_batch (object) – the data (or minibatch) for the current iteration.
curr_training_states (dict) – the dictionary containing the current training states.

Returns:

A dictionary containing the information from the results.

inference(model, _curr_batch, training_states)[source]

The event function to execute inference step.

For task-IL, we need to additionally consider task information for the inference step.

Parameters:

model (torch.nn.Module) – the current trained model.
curr_batch (object) – the data (or minibatch) for the current iteration.
curr_training_states (dict) – the dictionary containing the current training states.

Returns:

A dictionary containing the inference results, such as prediction result and loss.

prepareLoader(_curr_dataset, curr_training_states)[source]

The event function to generate dataloaders from the given dataset for the current task.

For task-IL, we need to additionally consider task information for the inference step.

Parameters:

curr_dataset (object) – The dataset for the current task. Its type is dgl.graph for node-level and link-level problem, and dgl.data.DGLDataset for graph-level problem.
curr_training_states (dict) – the dictionary containing the current training states.

Returns:

A tuple containing three dataloaders. The trainer considers the first dataloader, second dataloader, and third dataloader as dataloaders for training, validation, and test, respectively.

processBeforeTraining(task_id, curr_dataset, curr_model, curr_optimizer, curr_training_states)[source]

The event function to execute some processes before training.

We need to store previously learned weights for the knowledge distillation process in afterInference().

Parameters:

task_id (int) – the index of the current task
curr_dataset (object) – The dataset for the current task.
curr_model (torch.nn.Module) – the current trained model.
curr_optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_training_states (dict) – the dictionary containing the current training states.

class LCTimeILLwFTrainer(model, scenario, optimizer_fn, loss_fn, device, **kwargs)[source]

afterInference(results, model, optimizer, _curr_batch, training_states)[source]

The event function to execute some processes right after the inference step (for training). We recommend performing backpropagation in this event function.

LwF performs knowledge distillation process in this function.

Parameters:

results (dict) – the returned dictionary from the event function inference.
model (torch.nn.Module) – the current trained model.
optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_batch (object) – the data (or minibatch) for the current iteration.
curr_training_states (dict) – the dictionary containing the current training states.

Returns:

A dictionary containing the information from the results.

processAfterTraining(task_id, curr_dataset, curr_model, curr_optimizer, curr_training_states)[source]

The event function to execute some processes after training.

We need to store previously learned weights for the knowledge distillation process in afterInference().

Parameters:

task_id (int) – the index of the current task
curr_dataset (object) – The dataset for the current task.
curr_model (torch.nn.Module) – the current trained model.
curr_optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_training_states (dict) – the dictionary containing the current training states.

processBeforeTraining(task_id, curr_dataset, curr_model, curr_optimizer, curr_training_states)[source]

The event function to execute some processes before training. We need to extend the base function since the output format is slightly different from the base trainer.

Parameters:

task_id (int) – the index of the current task
curr_dataset (object) – The dataset for the current task.
curr_model (torch.nn.Module) – the current trained model.
curr_optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_training_states (dict) – the dictionary containing the current training states.

processEvalIteration(model, _curr_batch)[source]

The event function to handle every evaluation iteration. We need to extend the base function since the output format is slightly different from the base trainer.

Parameters:

model (torch.nn.Module) – the current trained model.
curr_batch (object) – the data (or minibatch) for the current iteration.

Returns:

A dictionary containing the outcomes (stats) during the evaluation iteration.

class LPDomainILLwFTrainer(model, scenario, optimizer_fn, loss_fn, device, **kwargs)[source]: This trainer has the same behavior as LPTimeILLwFTrainer.

class LPTimeILLwFTrainer(model, scenario, optimizer_fn, loss_fn, device, **kwargs)[source]

processAfterTraining(task_id, curr_dataset, curr_model, curr_optimizer, curr_training_states)[source]

The event function to execute some processes after training the current task.

We need to store previously learned weights for the knowledge distillation process in processTrainIteration().

Parameters:

task_id (int) – the index of the current task.
curr_dataset (object) – The dataset for the current task.
curr_model (torch.nn.Module) – the current trained model.
curr_optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_training_states (dict) – the dictionary containing the current training states.

processTrainIteration(model, optimizer, _curr_batch, training_states)[source]

The event function to handle every training iteration.

LwF performs inference and knowledge distillation process in this function.

Parameters:

model (torch.nn.Module) – the current trained model.
optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_batch (object) – the data (or minibatch) for the current iteration.
curr_training_states (dict) – the dictionary containing the current training states.

Returns:

A dictionary containing the outcomes (stats) during the training iteration.

Graph-level Problems

class GCClassILLwFTrainer(model, scenario, optimizer_fn, loss_fn, device, **kwargs)[source]

afterInference(results, model, optimizer, _curr_batch, training_states)[source]

The event function to execute some processes right after the inference step (for training). We recommend performing backpropagation in this event function.

LwF performs knowledge distillation process in this function.

Parameters:

results (dict) – the returned dictionary from the event function inference.
model (torch.nn.Module) – the current trained model.
optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_batch (object) – the data (or minibatch) for the current iteration.
curr_training_states (dict) – the dictionary containing the current training states.

Returns:

A dictionary containing the information from the results.

processAfterTraining(task_id, curr_dataset, curr_model, curr_optimizer, curr_training_states)[source]

The event function to execute some processes after training.

We need to store previously learned weights for the knowledge distillation process in afterInference().

Parameters:

task_id (int) – the index of the current task
curr_dataset (object) – The dataset for the current task.
curr_model (torch.nn.Module) – the current trained model.
curr_optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_training_states (dict) – the dictionary containing the current training states.

class GCDomainILLwFTrainer(model, scenario, optimizer_fn, loss_fn, device, **kwargs)[source]: This trainer has the same behavior as GCClassILLwFTrainer.

class GCTaskILLwFTrainer(model, scenario, optimizer_fn, loss_fn, device, **kwargs)[source]

afterInference(results, model, optimizer, _curr_batch, training_states)[source]

The event function to execute some processes right after the inference step (for training). We recommend performing backpropagation in this event function.

LwF performs knowledge distillation process in this function.

Parameters:

results (dict) – the returned dictionary from the event function inference.
model (torch.nn.Module) – the current trained model.
optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_batch (object) – the data (or minibatch) for the current iteration.
curr_training_states (dict) – the dictionary containing the current training states.

Returns:

A dictionary containing the information from the results.

inference(model, _curr_batch, training_states)[source]

The event function to execute inference step.

For task-IL, we need to additionally consider task information for the inference step.

Parameters:

model (torch.nn.Module) – the current trained model.
curr_batch (object) – the data (or minibatch) for the current iteration.
curr_training_states (dict) – the dictionary containing the current training states.

Returns:

A dictionary containing the inference results, such as prediction result and loss.

processBeforeTraining(task_id, curr_dataset, curr_model, curr_optimizer, curr_training_states)[source]

The event function to execute some processes before training.

We need to store previously learned weights for the knowledge distillation process in afterInference().

Parameters:

task_id (int) – the index of the current task
curr_dataset (object) – The dataset for the current task.
curr_model (torch.nn.Module) – the current trained model.
curr_optimizer (torch.optim.Optimizer) – the current optimizer function.
curr_training_states (dict) – the dictionary containing the current training states.

class GCTimeILLwFTrainer(model, scenario, optimizer_fn, loss_fn, device, **kwargs)[source]: This trainer has the same behavior as GCClassILLwFTrainer.