trainer.ml package

Submodules

trainer.ml.augmentation module

Augmentations can artificially increase dataset size.

Augmenting using adversarial attacks can work as a method of regularization.

trainer.ml.augmentation.fgsm_attack(image: torch.Tensor, epsilon: float, data_grad: torch.Tensor, data_range=- 1.0, 1.0)

Fast gradient sign method attack as proposed by: https://arxiv.org/pdf/1712.07107.pdf

FGSM is an attack for an infinity-norm bounded adversary.

trainer.ml.augmentation.oneshot_attack(image: torch.Tensor, max_change_percentage: float, data_grad: torch.Tensor, data_range=- 1.0, 1.0)

Oneshot attack, takes one step against the gradient direction to increase the error.

trainer.ml.layers module

class trainer.ml.layers.ConvGRUCell(input_dim, hidden_dim, kernel_size, bias)

Bases: torch.nn.modules.module.Module

Adapted from https://github.com/happyjin/ConvGRU-pytorch

forward(input_tensor: torch.Tensor)
init_hidden(input_batch: torch.Tensor)
reset_hidden()
class trainer.ml.layers.ResidualBlock(in_channels, out_channels, stride=1)

Bases: torch.nn.modules.module.Module

forward(x)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

trainer.ml.losses module

class trainer.ml.losses.FocalLoss(alpha=1.0, gamma=2.0, logits=False, reduce=True)

Bases: torch.nn.modules.module.Module

Loss based on cross entropy that emphasizes those outputs that have a large difference to the targets.

Focal loss is a simple trick which can be used to train networks when class imbalance is present.

Focusing parameter gamma: Increase to emphasize hard examples and put less effort into optimizing easy ones.

forward(inputs: torch.Tensor, targets: torch.Tensor)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class trainer.ml.losses.SegCrit(alpha, beta, loss_weights: Tuple)

Bases: torch.nn.modules.module.Module

Criterion which is optimized for semantic segmentation tasks.

Expects targets in the range between 0 and #C with shape [#B, W, H] The logits of the predictions for each class: [#B, #C, W, H].

>>> import trainer.ml as ml
>>> import trainer.lib as lib
>>> import trainer.demo_data as dd
>>> import numpy as np
>>> import torch
>>> np.random.seed(0)
>>> alpha, beta, loss_weights = 1., 2., (0.5, 0.5)
>>> sc = ml.SegCrit(alpha, beta, loss_weights)
>>> preds, target = dd.get_test_logits(shape=(8, 1, 3, 3)), np.random.randint(size=(8, 1, 3, 3), low=0, high=2)
>>> preds, target = torch.from_numpy(preds.astype(np.float32)), torch.from_numpy(target.astype(np.float32))
>>> sc.forward(preds, target)
tensor(6.8156)
forward(logits, target)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

trainer.ml.losses.dice_loss(logits: torch.FloatTensor, true: torch.LongTensor, eps=1e-07)

Computes the Sørensen–Dice loss. Note that PyTorch optimizers minimize a loss. In this case, we would like to maximize the dice loss so we return the negated dice loss.

Parameters

  • logits – a tensor of shape [B, C, H, W]. Corresponds to the raw output or logits of the model.

  • true – a tensor of shape [B, 1, H, W].

  • eps – added to the denominator for numerical stability.

Returns dice_loss

the Sørensen–Dice loss.

trainer.ml.metrics module

class trainer.ml.metrics.SegmentationMetric(x: Union[numpy.ndarray, torch.Tensor], y: Union[numpy.ndarray, torch.Tensor])

Bases: object

The class takes two numpy arrays or torch tensors as input and computes various segmentation metrics.

compute_dice()
compute_hausdorff()
compute_iou()

trainer.ml.predictor module

trainer.ml.torch_utils module

class trainer.ml.torch_utils.AccuracyMetric

Bases: trainer.ml.torch_utils.TrainerMetric

get_result()
update(prediction: numpy.ndarray, target: numpy.ndarray)
class trainer.ml.torch_utils.InMemoryDataset(ds_name: str, split_name: str, f: Union[Callable[[trainer.lib.data_model.Subject, trainer.ml.torch_utils.ModelMode], V], functools.partial], mode: trainer.ml.torch_utils.ModelMode = <ModelMode.Train: 'Train'>, subject_filter: Optional[Callable[[trainer.lib.data_model.Subject], bool]] = None)

Bases: torch.utils.data.dataset.Dataset

Wrapper around one dataset split to work with the torch.utils.data.Dataloader. This dataloader can be used to perform augmentations on multiple processes on the CPU and train on the GPU.

get_by_subject_name(s_name: str) → V

Searches for the subject with a given name in the preloaded data using a loop.

Applies the preprocessor function.

get_random_batch()
get_torch_dataloader(**kwargs) → torch.utils.data.dataloader.DataLoader
class trainer.ml.torch_utils.ModelMode(value)

Bases: enum.Enum

Used to determine what the model is currently doing.

The following guidelines apply for the semantics of this enum:

  • Train asks for augmentation and other tricks during training (batch normalization, …)

  • Eval does not require augmentation and is used for evaluation

  • Usage indicates that ground truths are not only not necessary, they may not be available

Eval = 'Eval'
Train = 'Train'
Usage = 'Usage'
class trainer.ml.torch_utils.TrainerMetric

Bases: abc.ABC

Base class of trainer metrics

evaluate(preds: numpy.ndarray, targets: numpy.ndarray)
abstract get_result()
abstract update(prediction: numpy.ndarray, target: numpy.ndarray)
trainer.ml.torch_utils.bench_mark_dataset(ds: trainer.ml.torch_utils.InMemoryDataset, extractor: Callable)
trainer.ml.torch_utils.get_capacity(model: torch.nn.modules.module.Module) → int

Computes the number of parameters of a network.

trainer.ml.torch_utils.init_weights(layer: torch.nn.modules.module.Module) → None
trainer.ml.torch_utils.plot_grad_flow(named_parameters: Iterator[Tuple[str, torch.nn.parameter.Parameter]]) → matplotlib.figure.Figure

Plots the gradients flowing through different layers in the net during training. Can be used for checking for possible gradient vanishing / exploding problems.

Usage: Plug this function in Trainer class after loss.backwards() as “plot_grad_flow(self.model.named_parameters())” to visualize the gradient flow

trainer.ml.utils module

class trainer.ml.utils.ImageNormalizations(value)

Bases: enum.Enum

An enumeration.

UnitRange = 1
trainer.ml.utils.cont_to_ont_hot(arr: numpy.ndarray, n_values=- 1) → numpy.ndarray
trainer.ml.utils.distance_transformed(mask: numpy.ndarray) → numpy.ndarray
trainer.ml.utils.duplicate_columns(data, minoccur=2)
trainer.ml.utils.insert_np_at(a1: numpy.ndarray, a2: numpy.ndarray, pos: Tuple[int, int], filter_arr=None) → numpy.ndarray
trainer.ml.utils.normalize_im(im: numpy.ndarray, norm_type=<ImageNormalizations.UnitRange: 1>) → numpy.ndarray

Currently just normalizes an image with pixel intensities in range [0, 255] to [-1, 1] :return: The normalized image

trainer.ml.utils.one_hot_to_cont(x: numpy.ndarray) → numpy.ndarray

Convert a one hot encoded image into the same image with integer representations.

Parameters

x – np.ndarray with (C, W, H)

Returns

np.ndarray with (W, H)

trainer.ml.utils.pad(small_arr: numpy.ndarray, size=30, 30) → numpy.ndarray
trainer.ml.utils.pair_augmentation(g: Iterable[Tuple[numpy.ndarray, numpy.ndarray]], aug_ls) → Iterable[Tuple[numpy.ndarray, numpy.ndarray]]
trainer.ml.utils.reduce_by_attention(arr: numpy.ndarray, att: numpy.ndarray)

Reduce an array by a field of attention, such that the result is a rectangle with the empty borders cropped.

Parameters

  • arr – Target array. The last two dimensions need to be of the same shape as the attention field

  • att – field of attention

Returns

cropped array

trainer.ml.utils.split_into_regions(arr: numpy.ndarray, mode=0) → List[numpy.ndarray]

Splits an array into its coherent regions.

Parameters

  • mode – 0 for orthogonal connection, 1 for full connection

  • arr – Numpy array with shape [W, H]

Returns

A list with length #NumberOfRegions of arrays with shape [W, H]

Module contents

Machine Learning Package

The machine learning package bundles everything directly related to learning from the data. For internal models, torch is tightly integrated into trainer.