Schedule

The schedule module houses Schedule class that produces sequences that can be implemented with proprietary and open source optimizers and algorithms.

raises ValueError:

Error if there is more or less than exactly one more element of values that boundaries

return:

Sequence of values with each element being a value (e.g. learning rate or difference) for each epoch

rtype:

ndarray

class optschedule.schedule.Schedule(n_steps)

Bases: object

__init__(n_steps) → None

Initializes Schedule Object.

Object properties:

• n_steps - number of steps

• steps - sequence of steps

Parameters:

n_steps (int) – Number of decay steps. Must be equal to the number of epochs of the algorithm

arithmetic_decay(initial_value, decay_rate, minimum_value)

Sequence with arithmetic decay given by

\[\text{value} = \max(\text{initial_value} - \text{decay_rate} \times \text{i}, \text{minimum_value})\]

Parameters:

• initial_value (float) – Initial value of the sequence

• decay_rate (float) – Rate of arithmetic decay

• minimum_value (float) – Minimum value of the sequence

Returns:

Sequence of values with each element being a value (e.g. learning rate or difference) for each epoch

Return type:

ndarray

constant(value)

Constant sequence

Parameters:

value (float) – Value for each epoch

Returns:

Sequence of values with each element being a value (e.g. learning rate or difference) for each epoch

Return type:

ndarray

cosine_decay(initial_value, alpha)

Sequence with cosine decay.

Parameters:

• initial_value (float) – Initial value of the sequence

• alpha (float) – Minimum sequence value as a fraction of initial_value

Returns:

Sequence of values with each element being a value (e.g. learning rate or difference) for each epoch

Return type:

ndarray

exponential_decay(initial_value, decay_rate, staircase=False)

Sequence with exponential decay.

Parameters:

• initial_value (float) – Initial value of the sequence

• decay_rate (float) – Rate of decay

• staircase (bool, optional) – If True decay the sequence at discrete intervals, defaults to False

Returns:

Sequence of values with each element being a value (e.g. learning rate or difference) for each epoch

Return type:

ndarray

geometric_decay(initial_value, decay_rate, minimum_value)

Sequence with geometric decay given by

\[\text{value} = \max(\text{initial_value} \times \text{decay_rate}^{\text{i}}, \text{minimum_value})\]

Parameters:

• initial_value (float) – Initial value of the sequence

• decay_rate (float) – Rate of geometric decay

• minimum_value (float) – Minimum value of the sequence

Returns:

Sequence of values with each element being a value (e.g. learning rate or difference) for each epoch

Return type:

ndarray

inverse_time_decay(initial_value, decay_rate, staircase=False)

Sequence with inverse time decay

Parameters:

• initial_value (float) – Initial value of the sequence

• decay_rate (float) – Rate of decay

• staircase (bool, optional) – If True decay the sequence at discrete intervals, defaults to False

Returns:

Sequence of values with each element being a value (e.g. learning rate or difference) for each epoch

Return type:

ndarray

piecewise_constant_decay(boundaries, values)

Sequence with piecewise constant decay.

Parameters:

• boundaries (list) – Boundaries of the pieces

• values (list) – list of values in sequence in each of the pieces

Raises:

ValueError – Error if there is more or less than exactly one more element of values that boundaries

Returns:

Sequence of values with each element being a value (e.g. learning rate or difference) for each epoch

Return type:

ndarray

polynomial_decay(initial_value, end_value, power, cycle=False)

Sequence with polynomial decay.

Parameters:

• initial_value (float) – Initial value of the sequence

• end_value (float) – The minimal end sequence value

• power (float) – The power of the polynomial

• cycle (bool, optional) – Whether or not it should cycle beyond self.n_steps, defaults to False

Returns:

Sequence of values with each element being a value (e.g. learning rate or difference) for each epoch

Return type:

ndarray

step_decay(initial_value, decay_value, decay_every)

Sequence with step decay given by

\[\text{value} = \text{initial_value} \times \text{decay_value}^{\lfloor \frac{i}{\text{decay_every}} \rfloor}\]

where i is the current value in the sequence.

Parameters:

• initial_value (float) – Initial value of the sequence

• decay_value (float) – Drop size

• decay_every (int) – Drop is performed every decay_every values

Returns:

Sequence of values with each element being a value (e.g. learning rate or difference) for each epoch

Return type:

ndarray

time_decay(initial_value, decay_rate)

Sequence with time-based decay given by

\[\text{value} = \frac{\text{prev_value}}{1+\text{decay_rate} \times \text{i}}\]

where prev_value is the previous value in the sequence and decay_rate is the decay rate parameter.

Parameters:

• initial_value (float) – Initial value of the sequence

• decay_rate (float) – Decay rate

Returns:

Sequence of values with each element being a value (e.g. learning rate or difference) for each epoch

Return type:

ndarray