Callbacks¶
Callbacks can be used with PySHAC for custom code execution, such as monitoring the improvement of the engine, maintaining a history of the training session or simply writing the logs to files.
Callbacks allow you the flexibility to prepare your state prior to training or evaluation, and in doing so can allow one to perform stateful evaluation if necessary using concurrent evaluators.
Class Information¶
Callback¶
pyshac.config.callbacks.Callback()
Abstract base class used to build new callbacks.
Properties
- engine: instance of a PySHAC Engine. Reference of the model being trained.
The logs dictionary that callback methods
take as argument will contain keys for quantities relevant to
the current batch or epoch.
Callback methods¶
on_dataset_changed¶
on_dataset_changed(dataset, logs=None)
Called with the dataset maintained by the engine is updated with new samples or data.
Arguments:
- dataset (Dataset): A Dataset object which contains the history of sampled parameters and their corresponding evaluation values.
- logs (dict | None): dictionary of logs.
on_epoch_begin¶
on_epoch_begin(epoch, logs=None)
Called at the start of an epoch.
Arguments
- epoch (int): index of epoch.
- logs (dict | None): dictionary of logs.
on_epoch_end¶
on_epoch_end(epoch, logs=None)
Called at the end of an epoch.
Arguments
- epoch (int): index of epoch.
- logs (dict | None): dictionary of logs.
on_evaluation_begin¶
on_evaluation_begin(params, logs=None)
Called before the generated parameters are evaluated.
Arguments:
params (list(OrderedDict)): A list of OrderedDicts, such that each item is a dictionary of the names and sampled values of a HyperParemeterList. - logs (dict | None): dictionary of logs.
on_evaluation_ended¶
on_evaluation_ended(evaluations, logs=None)
Called after the generated parameters are evaluated.
Arguments:
evaluations (list(float)): A list of floating point values, corresponding to the provided parameter settings. - logs (dict | None): dictionary of logs.
on_train_begin¶
on_train_begin(logs=None)
Called at the beginning of training.
Arguments
- logs (dict | None): dictionary of logs.
on_train_end¶
on_train_end(logs=None)
Called at the end of training.
Arguments
- logs (dict | None): dictionary of logs.
set_engine¶
set_engine(engine)
Sets an instance of a PySHAC Engine.
Arguments:
- engine (AbstractSHAC): A concrete implementation of the SHAC engine.
History¶
pyshac.config.callbacks.History()
Callback that records events into a History object.
This callback is automatically applied to
every engine. The History object
gets returned by the fit or fit_dataset methods.
CSVLogger¶
pyshac.config.callbacks.CSVLogger(filename, separator=',', append=False)
Callback that streams epoch results to a csv file.
Supports all values that can be represented as a string, including 1D iterables such as np.ndarray.
Example
csv_logger = CSVLogger('training.log') shac.fit(evaluation_function, callbacks=[csv_logger])
Arguments
- filename (str): filename of the csv file, e.g. 'run/log.csv'.
- separator (str): string used to separate elements in the csv file.
- append (bool): True: append if file exists (useful for continuing training). False: overwrite existing file,
get_history¶
get_history(callbacks)
Gets the History callback from a list of callbacks.
Argumetns:
- callbacks (list | CallbackList): a list of callbacks
Returns:
A History callback object or None if it was not found.