

Hyper Parameters¶

There are 3 primary hyper parameters :

DiscreteHyperParameter (DiscreteHP)
UniformContinuousHyperParameter (UniformHP)
NormalContinuousHyperParameter (NormalHP)

As their names suggest, each hyper paramter takes specific parameters or value ranges.

Discrete Hyper Parmeters takes lists of any python data type that can be serialized.
- This includes numpy values, and even numpy arrays
Continuous Hyper Parameters take ranges of values
- Uniform Continuous Hyper Parameters will sample uniformly between the min and max values
- Normal Continuous Hyper Parameters will sample from the normal distribution with the provided mean and standard deviation

A special composite hyper parameter, called HyperParameterList is used to provide a convenient interface to several hyper parameters at once, and is used internally by the training algorithm.

Class Information¶

[source]

AbstractHyperParameter ¶

pyshac.config.hyperparameters.AbstractHyperParameter(name, values, seed)

Abstract Hyper Parameter that defines the methods that all hyperparameters need to supply

Arguments:

name (str): Name of the hyper parameter
values (List, None): A list of values (must all be pickle-able and hashable) values or None. If None, it is assumed to be a continuous value generator.

Raises:

ValueError: If the name is not specified.

AbstractHyperParameter methods¶

decode¶

decode(x)

Abstract method that defines how the parameter is decoded so that the model can be properly trained.

Arguments:

x (int | float): an encoded value that needs to be decoded.

Raises:

NotImplementedError: Must be overridden by the subclass.

Returns:

a decoded value for the encoded input x.

encode¶

encode(x)

Abstract method that defines how the parameter is encoded so that the model can properly be trained.

Arguments:

x (int | float | str): a single value that needs to be encoded.

Raises:

NotImplementedError: Must be overridden by the subclass.

Returns:

an encoded representation of the value of x.

get_config¶

get_config()

Creates the config of the class with all of its values.

Returns:

a dictionary with the config of the class.

load_from_config¶

load_from_config(config)

Utilizes the provided config to instantiate a new instance of the class with the same arguments.

Arguments:

config (dict): A dictionary having keys as the argument names and values as the values specified to the class using its constructor.

Returns:

A new instance of this class with the correct arguments.

sample¶

sample()

Abstract method that defines how parameters are sampled.

Raises:

NotImplementedError: Must be overridden by the subclass.

Returns:

a singular value sampled from possible values.

set_seed¶

set_seed(seed)

Sets the random seed of the local RNG.

Arguments:

seed (int | None): Random seed value.

[source]

AbstractContinuousHyperParameter ¶

pyshac.config.hyperparameters.AbstractContinuousHyperParameter(name, val1, val2, log_encode=False, seed=None)

An abstract hyper parameter that represents a parameter that can take a range of values from a certain distribution.

Arguments:

name (str): Name of the parameter.
val1 (float): A symbolic value that is used by subclasses.
val2 (float): A symbolic value that is used by subclasses.
log_encode (bool): Determines whether the encoding must be in natural log-space or not.

Raises:

NotImplementedError: If sample() is called.

AbstractContinuousHyperParameter methods¶

decode¶

decode(x)

Decodes the floating point value into normal space if log_space was set in the constructor, else returns its original value.

Arguments:

x (float): a single encoded sample.

Returns:

float.

encode¶

encode(x)

Encodes the floating point value into log space if log_space was set in the constructor, else returns its original value.

Arguments:

x (float): a single sample.

Returns:

float.

get_config¶

get_config()

Creates the config of the class with all of its values.

Returns:

a dictionary with the config of the class.

load_from_config¶

load_from_config(config)

Utilizes the provided config to instantiate a new instance of the class with the same arguments.

Arguments:

config (dict): A dictionary having keys as the argument names and values as the values specified to the class using its constructor.

Returns:

A new instance of this class with the correct arguments.

sample¶

sample()

Abstract method that must be redefined by base classes.

Returns:

a float value.

set_seed¶

set_seed(seed)

Sets the random seed of the local RNG.

Arguments:

seed (int | None): Random seed value.

[source]

AbstractMultiContinuousHyperParameter ¶

pyshac.config.hyperparameters.AbstractMultiContinuousHyperParameter(name, val1, val2, log_encode=False, sample_count=1, seed=None)

An abstract hyper parameter that represents a parameter that can take a range of values from a certain distribution, sampled multiple times.

Arguments:

name (str): Name of the parameter.
val1 (float): A symbolic value that is used by subclasses.
val2 (float): A symbolic value that is used by subclasses.
log_encode (bool): Determines whether the encoding must be in natural log-space or not.
sample_count (int): Number of samples that are required from this hyper parameter.

Raises:

NotImplementedError: If sample() is called.
ValueErroe: If sample count is less than 1.

AbstractMultiContinuousHyperParameter methods¶

decode¶

decode(x)

Decodes a list of floating point values into normal space if log_space was set in the constructor, else returns its original value.

Arguments:

x (float): a list of encoded samples.

Returns:

list of floats.

encode¶

encode(x)

Encodes a list of floating point values into log space if log_space was set in the constructor, else returns its original value.

Arguments:

x (float): a list of samples.

Returns:

list of floats.

get_config¶

get_config()

Creates the config of the class with all of its values.

Returns:

a dictionary with the config of the class.

load_from_config¶

load_from_config(config)

Utilizes the provided config to instantiate a new instance of the class with the same arguments.

Arguments:

config (dict): A dictionary having keys as the argument names and values as the values specified to the class using its constructor.

Returns:

A new instance of this class with the correct arguments.

sample¶

sample()

Abstract method that must be redefined by base classes.

Returns:

a float value.

set_seed¶

set_seed(seed)

Sets the random seed of the local RNG.

Arguments:

seed (int | None): Random seed value.

[source]

DiscreteHyperParameter ¶

pyshac.config.hyperparameters.DiscreteHyperParameter(name, values, seed=None)

Discrete Hyper Parameter that defines a set of discrete values that it can take.

Arguments:

name (str): Name of the hyper parameter.
values (list): A list of values (must all be pickle-able and hashable) values or None.

Raises:

ValueError: If the name is not specified. .

Raises:

ValueError: If the name is not specified.

DiscreteHyperParameter methods¶

decode¶

decode(x)

Decodes a single encoded integer into its original value.

Args:

x (int): an integer encoded value.

Returns:

(int | float | str) representing the actual decoded value.

encode¶

encode(x)

Encodes a single value into an integer index.

Arguments:

x (int | float | str): A value sampled from its possible values.

Returns:

int value representing its encoded index.

get_config¶

get_config()

Creates the config of the class with all of its values.

Returns:

a dictionary with the config of the class.

load_from_config¶

load_from_config(config)

Utilizes the provided config to instantiate a new instance of the class with the same arguments.

Arguments:

config (dict): A dictionary having keys as the argument names and values as the values specified to the class using its constructor.

Returns:

A new instance of this class with the correct arguments.

sample¶

sample()

Samples a single value from its set of discrete values.

Returns:

a single value from its list of possible values.

set_seed¶

set_seed(seed)

Sets the random seed of the local RNG.

Arguments:

seed (int | None): Random seed value.

[source]

UniformContinuousHyperParameter ¶

pyshac.config.hyperparameters.UniformContinuousHyperParameter(name, min_value, max_value, log_encode=False, seed=None)

A hyper parameter that represents a parameter that can take a range of values from a uniform distribution.

Arguments:

name (str): Name of the parameter.
min_value (float): The minimum value (inclusive) that the uniform distribution can take.
max_value (float): The maximum value (exclusive) that the uniform distribution can take.
log_encode (bool): Determines whether the encoding must be in natural log-space or not.

UniformContinuousHyperParameter methods¶

decode¶

decode(x)

Decodes the floating point value into normal space if log_space was set in the constructor, else returns its original value.

Arguments:

x (float): a single encoded sample.

Returns:

float.

encode¶

encode(x)

Encodes the floating point value into log space if log_space was set in the constructor, else returns its original value.

Arguments:

x (float): a single sample.

Returns:

float.

get_config¶

get_config()

Creates the config of the class with all of its values.

Returns:

a dictionary with the config of the class.

load_from_config¶

load_from_config(config)

Utilizes the provided config to instantiate a new instance of the class with the same arguments.

Arguments:

config (dict): A dictionary having keys as the argument names and values as the values specified to the class using its constructor.

Returns:

A new instance of this class with the correct arguments.

sample¶

sample()

Samples uniformly from the range [min_value, max_value).

Returns:

float.

set_seed¶

set_seed(seed)

Sets the random seed of the local RNG.

Arguments:

seed (int | None): Random seed value.

[source]

NormalContinuousHyperParameter ¶

pyshac.config.hyperparameters.NormalContinuousHyperParameter(name, mean, std, seed=None)

A hyper parameter that represents a parameter that can take a range of values from a normal distribution.

Arguments:

name (str): Name of the parameter.
mean (float): The mean of the normal distribution.
std (float): The standard deviation of the normal distribution.

NormalContinuousHyperParameter methods¶

decode¶

decode(x)

Decodes the floating point value into normal space if log_space was set in the constructor, else returns its original value.

Arguments:

x (float): a single encoded sample.

Returns:

float.

encode¶

encode(x)

Encodes the floating point value into log space if log_space was set in the constructor, else returns its original value.

Arguments:

x (float): a single sample.

Returns:

float.

get_config¶

get_config()

Creates the config of the class with all of its values.

Returns:

a dictionary with the config of the class.

load_from_config¶

load_from_config(config)

Utilizes the provided config to instantiate a new instance of the class with the same arguments.

Arguments:

config (dict): A dictionary having keys as the argument names and values as the values specified to the class using its constructor.

Returns:

A new instance of this class with the correct arguments.

sample¶

sample()

Samples from the normal distribution with a mean and standard deviation as specified in the constructor.

Returns:

float.

set_seed¶

set_seed(seed)

Sets the random seed of the local RNG.

Arguments:

seed (int | None): Random seed value.

[source]

MultiDiscreteHyperParameter ¶

pyshac.config.hyperparameters.MultiDiscreteHyperParameter(name, values, sample_count=1, seed=None)

Discrete Hyper Parameter that defines a set of discrete values that it can take, and acts upon a list of samples.

Arguments:

name (str): Name of the hyper parameter.
values (list): A list of values (must all be pickle-able and hashable) values or None.
sample_count (int): Number of samples that are required from this hyper parameter.

Raises:

ValueError: If the name is not specified or if sample_count is less than 1. .

Raises:

ValueError: If the name is not specified or if sample_count is less than 1.

MultiDiscreteHyperParameter methods¶

decode¶

decode(x)

Decodes a list of encoded integers into their original value.

Args:

x (int): a list of integer encoded values.

Returns:

list of (int | float | str) representing the actual decoded values.

encode¶

encode(x)

Encodes a list of values into a list of the corresponding integer index.

Arguments:

x (int | float | str): A list of values sampled from its possible values.

Returns:

list of int values representing their encoded index.

get_config¶

get_config()

Creates the config of the class with all of its values.

Returns:

a dictionary with the config of the class.

load_from_config¶

load_from_config(config)

Utilizes the provided config to instantiate a new instance of the class with the same arguments.

Arguments:

config (dict): A dictionary having keys as the argument names and values as the values specified to the class using its constructor.

Returns:

A new instance of this class with the correct arguments.

sample¶

sample()

Samples a number of values from its set of discrete values.

Returns:

a list of values from its set of possible values.

set_seed¶

set_seed(seed)

Sets the random seed of the local RNG.

Arguments:

seed (int | None): Random seed value.

[source]

MultiUniformContinuousHyperParameter ¶

pyshac.config.hyperparameters.MultiUniformContinuousHyperParameter(name, min_value, max_value, log_encode=False, sample_count=1, seed=None)

A hyper parameter that represents a parameter that can take a range of values from a uniform distribution, sampled multiple times.

Arguments:

name (str): Name of the parameter.
min_value (float): The minimum value (inclusive) that the uniform distribution can take.
max_value (float): The maximum value (exclusive) that the uniform distribution can take.
log_encode (bool): Determines whether the encoding must be in natural log-space or not.
sample_count (int): Number of samples that are required from this hyper parameter.

Raises:

ValueErroe: If sample count is less than 1.

MultiUniformContinuousHyperParameter methods¶

decode¶

decode(x)

Decodes a list of floating point values into normal space if log_space was set in the constructor, else returns its original value.

Arguments:

x (float): a list of encoded samples.

Returns:

list of floats.

encode¶

encode(x)

Encodes a list of floating point values into log space if log_space was set in the constructor, else returns its original value.

Arguments:

x (float): a list of samples.

Returns:

list of floats.

get_config¶

get_config()

Creates the config of the class with all of its values.

Returns:

a dictionary with the config of the class.

load_from_config¶

load_from_config(config)

Utilizes the provided config to instantiate a new instance of the class with the same arguments.

Arguments:

config (dict): A dictionary having keys as the argument names and values as the values specified to the class using its constructor.

Returns:

A new instance of this class with the correct arguments.

sample¶

sample()

Samples uniformly from the range [min_value, max_value).

Returns:

list of floats.

set_seed¶

set_seed(seed)

Sets the random seed of the local RNG.

Arguments:

seed (int | None): Random seed value.

[source]

MultiDiscreteHyperParameter ¶

pyshac.config.hyperparameters.MultiDiscreteHyperParameter(name, values, sample_count=1, seed=None)

Discrete Hyper Parameter that defines a set of discrete values that it can take, and acts upon a list of samples.

Arguments:

name (str): Name of the hyper parameter.
values (list): A list of values (must all be pickle-able and hashable) values or None.
sample_count (int): Number of samples that are required from this hyper parameter.

Raises:

ValueError: If the name is not specified or if sample_count is less than 1. .

Raises:

ValueError: If the name is not specified or if sample_count is less than 1.

MultiDiscreteHyperParameter methods¶

decode¶

decode(x)

Decodes a list of encoded integers into their original value.

Args:

x (int): a list of integer encoded values.

Returns:

list of (int | float | str) representing the actual decoded values.

encode¶

encode(x)

Encodes a list of values into a list of the corresponding integer index.

Arguments:

x (int | float | str): A list of values sampled from its possible values.

Returns:

list of int values representing their encoded index.

get_config¶

get_config()

Creates the config of the class with all of its values.

Returns:

a dictionary with the config of the class.

load_from_config¶

load_from_config(config)

Utilizes the provided config to instantiate a new instance of the class with the same arguments.

Arguments:

config (dict): A dictionary having keys as the argument names and values as the values specified to the class using its constructor.

Returns:

A new instance of this class with the correct arguments.

sample¶

sample()

Samples a number of values from its set of discrete values.

Returns:

a list of values from its set of possible values.

set_seed¶

set_seed(seed)

Sets the random seed of the local RNG.

Arguments:

seed (int | None): Random seed value.

[source]

HyperParameterList ¶

pyshac.config.hyperparameters.HyperParameterList(hyper_parameter_list=None, seed=None)

A composite hyper parameter, that encloses a list of hyper parameters (either discrete or continuous) and provides utility methods for efficient handling by the engine.

Arguments:

hyper_parameter_list (list(AnstractHyperParameter) | None): A list of hyper parameters or None (which initializes this with 0 elements).

HyperParameterList methods¶

add_hyper_parameter¶

add_hyper_parameter(parameter)

Adds a single hyper parameter (discrete or continuous) to the list of hyper parameters managed by this HyperParameterList.

Arguments:

parameter (AbstractHyperParameter): a subclass of AbstractHyperParameter, which will be embedded into this composite class.

Raises:

ValueError: If the passed parameter is None, or the name already exists in the list of managed parameters.

remove_hyper_parameter¶

remove_hyper_parameter(parameter)

Removes a single hyper parameter (discrete or continuous) from the list of hyper parameters managed by this HyperParameterList.

Arguments:

parameter (AbstractHyperParameter, str): A string name or a subclass of AbstractHyperParameter which needs to be removed.

Raises:

ValueError: If the passed parameter is None.

sample¶

sample()

Samples all of its component parameters and returns a list of the samples.

Returns:

list of sampled parameters.

encode¶

encode(x)

Encodes a list of sampled hyper parameters.

Arguments:

x (list | np.ndarray): A python list or numpy array of samples from the list of hyper parameters.

Raises:

ValueError: If a numpy array of more than 1 dimension is provided.

Returns:

ndarray(float).

decode¶

decode(x)

Decodes a list of sampled hyper parameters.

Arguments:

x (list(int | float)): a list of encoded integer or floating point values that are to be decoded.

Returns:

list of decoded samples.

get_config¶

get_config()

Creates the config of the class with all of its values.

Returns:

an ordered dictionary with the config of the class.

load_from_config¶

load_from_config(config)

get_parameter_names¶

get_parameter_names()

Gets a list of all the parameter names managed by this class.

Returns:

a list(str) with the names of the parameters.

set_seed¶

set_seed(seed)

Sets the seed of all the parameters held by the container.

Arguments:

seed (int | None): Seed value for the random state.

get_parameter¶

get_parameter(name)

Utility method to get the hyper parameter class by its name.

Arguments:

name (str): Name of the class or its alias.

Raises:

ValueError: If the class with the provided name does not exists in the set of available parameters.

Returns:

The hyper parameter class.

set_custom_parameter_class¶

set_custom_parameter_class(cls)

Utility function to dynamically add a custom hyper parameter to the set of available hyper parameters.

Arguments:

cls (cls): A class which extends AbstractHyperParameter in some way and implements the abstract methods.