autogluon.space

Search space of possible hyperparameter values to consider.

Example

Define a dummy training function with searchable spaces for hyperparameters lr and wd:

>>> import numpy as np
>>> import autogluon as ag
>>> @ag.args(
>>>     lr=ag.space.Real(1e-3, 1e-2, log=True),
>>>     wd=ag.space.Real(1e-3, 1e-2))
>>> def train_fn(args, reporter):
...     print('lr: {}, wd: {}'.format(args.lr, args.wd))
...     for e in range(10):
...         dummy_accuracy = 1 - np.power(1.8, -np.random.uniform(e, 2*e))
...         reporter(epoch=e, accuracy=dummy_accuracy, lr=args.lr, wd=args.wd)

Create a scheduler to manage training jobs and begin hyperparameter tuning with the provided search space:

>>> scheduler = ag.scheduler.HyperbandScheduler(train_fn,
>>>                                             resource={'num_cpus': 2, 'num_gpus': 0},
>>>                                             num_trials=10,
>>>                                             reward_attr='accuracy',
>>>                                             time_attr='epoch',
>>>                                             grace_period=1)
>>> scheduler.run()
>>> scheduler.join_jobs()

Visualize the results:

>>> scheduler.get_training_curves(plot=True)
https://raw.githubusercontent.com/zhanghang1989/AutoGluonWebdata/master/doc/api/autogluon.1.png

Search Space

Real

Search space for numeric hyperparameter that takes continuous values.

Int

Search space for numeric hyperparameter that takes integer values.

Bool

Search space for hyperparameter that is either True or False.

Categorical

Nested search space for hyperparameters which are categorical.

List

Nested search space corresponding to an ordered list of hyperparameters.

Dict

Nested search space for dictionary containing multiple hyperparameters.

AutoGluonObject

Searchable objects, created by decorating a custom Python class or function using the autogluon.obj() or autogluon.func() decorators.

Real

class autogluon.space.Real(lower, upper, default=None, log=False)

Search space for numeric hyperparameter that takes continuous values.

Parameters
lowerfloat

the lower bound of the search space

upperfloat

the upper bound of the search space

defaultfloat (optional)

default value

log(True/False)

Whether to search the values on a logarithmic rather than linear scale. This is useful for numeric hyperparameters (such as learning rates) whose search space spans many orders of magnitude.

Examples

>>> learning_rate = ag.Real(0.01, 0.1, log=True)
Attributes
default

Return default value of hyperparameter corresponding to this search space.

hp

Return hyperparameter corresponding to this search space.

rand

Return randomly sampled (but valid) value from this search space.

Methods

get_hp(self, name)

Fetch particular hyperparameter based on its name.

get_hp(self, name)

Fetch particular hyperparameter based on its name.

Int

class autogluon.space.Int(lower, upper, default=None)

Search space for numeric hyperparameter that takes integer values.

Parameters
lowerint

The lower bound of the search space

upperint

The upper bound of the search space

defaultint (optional)

Default value

Examples

>>> range = ag.space.Int(0, 100)
Attributes
default

Return default value of hyperparameter corresponding to this search space.

hp

Return hyperparameter corresponding to this search space.

rand

Return randomly sampled (but valid) value from this search space.

Methods

get_hp(self, name)

Fetch particular hyperparameter based on its name.

get_hp(self, name)

Fetch particular hyperparameter based on its name.

Bool

class autogluon.space.Bool
Search space for hyperparameter that is either True or False.

ag.Bool() serves as shorthand for: ag.space.Categorical(True, False)

Examples

pretrained = ag.space.Bool()

Attributes
default

Return default value of hyperparameter corresponding to this search space.

hp

Return hyperparameter corresponding to this search space.

rand

Return randomly sampled (but valid) value from this search space.

Methods

get_hp(self, name)

Fetch particular hyperparameter based on its name.

Categorical

class autogluon.space.Categorical(*data)

Nested search space for hyperparameters which are categorical. Such a hyperparameter takes one value out of the discrete set of provided options.

Parameters
dataSpace or python built-in objects

the choice candidates

Examples

a = ag.space.Categorical(‘a’, ‘b’, ‘c’, ‘d’) b = ag.space.Categorical(‘resnet50’, autogluon_obj())

Attributes
cs

ConfigSpace representation of this search space.

default

Return default value for hyperparameter corresponding to this search space.

kwspaces

OrderedDict representation of this search space.

rand

Randomly sample configuration from this nested search space.

Methods

sample(self, \*\*config)

Sample a configuration from this search space.

property cs

ConfigSpace representation of this search space.

property kwspaces

OrderedDict representation of this search space.

sample(self, **config)

Sample a configuration from this search space.

List

class autogluon.space.List(*args)

Nested search space corresponding to an ordered list of hyperparameters.

Parameters
argslist

a list of search spaces.

Examples

>>> sequence = ag.List(
>>>     ag.space.Categorical('conv3x3', 'conv5x5', 'conv7x7'),
>>>     ag.space.Categorical('BatchNorm', 'InstanceNorm'),
>>>     ag.space.Categorical('relu', 'sigmoid'),
>>> )
Attributes
cs

ConfigSpace representation of this search space.

default

Return default value for hyperparameter corresponding to this search space.

kwspaces

OrderedDict representation of this search space.

rand

Randomly sample configuration from this nested search space.

Methods

sample(self, \*\*config)

Sample a configuration from this search space.

property cs

ConfigSpace representation of this search space.

property kwspaces

OrderedDict representation of this search space.

sample(self, **config)

Sample a configuration from this search space.

Dict

class autogluon.space.Dict(**kwargs)

Nested search space for dictionary containing multiple hyperparameters.

Examples

>>> g = ag.space.Dict(
>>>         hyperparam1 = ag.space.Categorical('alpha', 'beta'),
>>>         hyperparam2 = ag.space.Int(0, 3)
>>>     )
>>> print(g)
Attributes
cs

ConfigSpace representation of this search space.

default

Return default value for hyperparameter corresponding to this search space.

kwspaces

OrderedDict representation of this search space.

rand

Randomly sample configuration from this nested search space.

Methods

sample(self, \*\*config)

Sample a configuration from this search space.

property cs

ConfigSpace representation of this search space.

property kwspaces

OrderedDict representation of this search space.

sample(self, **config)

Sample a configuration from this search space.

AutoGluonObject

class autogluon.space.AutoGluonObject

Searchable objects, created by decorating a custom Python class or function using the autogluon.obj() or autogluon.func() decorators.

Attributes
cs

ConfigSpace representation of this search space.

default

Return default value for hyperparameter corresponding to this search space.

rand

Randomly sample configuration from this nested search space.

Methods

__call__(self, \*args, \*\*kwargs)

Convenience method for interacting with AutoGluonObject.

init(self)

Instantiate an actual instance of this AutoGluonObject.

sample(self)

Sample a configuration from this search space.

property cs

ConfigSpace representation of this search space.

init(self)

Instantiate an actual instance of this AutoGluonObject. In order to interact with such an object, you must always first call: object.init().

sample(self)

Sample a configuration from this search space.