trainer.demo_data package

Submodules

trainer.demo_data.DemoDataset module

class trainer.demo_data.DemoDataset.DemoDataset(data_path: str, ds_name: str)

Bases: abc.ABC

Intended to be used to load standard machine learning datasets as mock-up data for the trainer dataset format.

abstract build_dataset(sess: sqlalchemy.orm.session.Session = <sqlalchemy.orm.session.Session object>) → trainer.lib.data_model.Dataset

Builds a new dataset if it does not yet exists in the database.

trainer.demo_data.DemoDataset.build_test_subject() → trainer.lib.data_model.Subject
trainer.demo_data.DemoDataset.get_test_logits(shape=50, 50, bounds=- 50, 20) → numpy.ndarray

Returns a demo array for testing functionality with uniformly distributed logits.

>>> import trainer.demo_data as dd
>>> import numpy as np
>>> np.random.seed(0)
>>> dd.get_test_logits(shape=(2,))
array([-5.28481063, -2.39723662])
Parameters

  • shape – Shape of the test data. For one-dimensional data use (w, ).

  • bounds – Optional to specify the ceiling and floor of the output using a 2-Tuple (floor, ceiling)

Returns

Demo logits

trainer.demo_data.arc module

Provides utilities and the ARC-dataset in the trainer-format.

See https://github.com/fchollet/ARC for details.

class trainer.demo_data.arc.ArcDataset(data_path: str)

Bases: trainer.demo_data.DemoDataset.DemoDataset

build_dataset(sess=<sqlalchemy.orm.session.Session object>) → trainer.lib.data_model.Dataset

Builds a new dataset if it does not yet exists in the database.

create_arc_split(d: trainer.lib.data_model.Dataset, ss_tpl: trainer.lib.data_model.SemSegTpl, split_name='training')
class trainer.demo_data.arc.Game(train_pairs: List[trainer.demo_data.arc.Pair], test_pairs: List[trainer.demo_data.arc.Pair])

Bases: object

compute_boxes()
classmethod extract_from_subject(s: trainer.lib.data_model.Subject)
get_mults()
get_states() → Tuple[List[Dict[str, Any]], List[Dict[str, Any]]]
get_train_colors()
class trainer.demo_data.arc.Pair(situation: numpy.ndarray, target: numpy.ndarray)

Bases: object

get_situation()
get_target()
visualize(ax1, ax2)
class trainer.demo_data.arc.Value(value)

Bases: enum.Enum

An enumeration.

Black = 1
Blue = 2
Cyan = 9
Empty = 0
Green = 4
Grey = 6
Magenta = 10
Orange = 8
Pink = 7
Red = 3
Yellow = 5
static from_ind(c_ind: int) → trainer.demo_data.arc.Value
trainer.demo_data.arc.array_from_json(list_repr: List[List], depth=10) → numpy.ndarray
trainer.demo_data.arc.encode_depthmap(x: numpy.ndarray, n_classes=11, max_grid=30) → numpy.ndarray
trainer.demo_data.arc.extract_objts(grid: numpy.ndarray, indices: List[numpy.ndarray]) → List[numpy.ndarray]

Given an ARC field and

trainer.demo_data.arc.extract_train_test(s: trainer.lib.data_model.Subject)
trainer.demo_data.arc.game_from_subject(s: trainer.lib.data_model.Subject, logging=True) → trainer.demo_data.arc.Game
trainer.demo_data.arc.pair_from_imstack(im: trainer.lib.data_model.ImStack) → trainer.demo_data.arc.Pair
trainer.demo_data.arc.plot_as_heatmap(arc_field: numpy.ndarray, ax=None, title='title') → None
trainer.demo_data.arc.plot_game(g: trainer.demo_data.arc.Game) → Tuple[matplotlib.figure.Figure, matplotlib.figure.Figure]
trainer.demo_data.arc.plot_pairs(pairs: List[trainer.demo_data.arc.Pair], title='training', magnification=3) → matplotlib.figure.Figure

trainer.demo_data.mnist module

class trainer.demo_data.mnist.MnistDataset(data_path: str, max_training_examples=- 1)

Bases: trainer.demo_data.DemoDataset.DemoDataset

>>> import tempfile
>>> import trainer.demo_data as dd
>>> dir_path = tempfile.gettempdir()
>>> mnist_dataset = dd.MnistDataset(dir_path)
>>> x, y = mnist_dataset.sample_digit(digit=2)
>>> y
2
>>> x.shape
(28, 28)
build_dataset(sess=None) → trainer.lib.data_model.Dataset

Builds a new dataset if it does not yet exists in the database.

refill_mnist_indices()
sample_digit(digit=0)

Module contents

Produces (smallish) datasets for testing the functionality of the annotator and the machine learning capabilities.

Uses synthetic data that uses tasks solvable by a human to enable simple demonstration of trainer functionality.

To fill your database with one of the datasets, just execute corresponding file. The relevant code is always at the bottom in the __main__ statement.