mxnet.optimizer.RMSProp¶

class mxnet.optimizer.RMSProp(learning_rate=0.001, gamma1=0.9, gamma2=0.9, epsilon=1e-08, centered=False, clip_weights=None, **kwargs)[source]¶

The RMSProp optimizer.

Two versions of RMSProp are implemented:

If centered=False, we follow http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf by Tieleman & Hinton, 2012. For details of the update algorithm see rmsprop_update.

If centered=True, we follow http://arxiv.org/pdf/1308.0850v5.pdf (38)-(45) by Alex Graves, 2013. For details of the update algorithm see rmspropalex_update.

This optimizer accepts the following parameters in addition to those accepted by Optimizer.

Parameters

gamma1 (float, optional) – A decay factor of moving average over past squared gradient.
gamma2 (float, optional) – A “momentum” factor. Only used if centered`=``True`.
epsilon (float, optional) – Small value to avoid division by 0.

centered (bool, optional) –

Flag to control which version of RMSProp to use.:

True: will use Graves's version of `RMSProp`,
False: will use Tieleman & Hinton's version of `RMSProp`.

clip_weights (float, optional) – Clips weights into range [-clip_weights, clip_weights].

__init__(learning_rate=0.001, gamma1=0.9, gamma2=0.9, epsilon=1e-08, centered=False, clip_weights=None, **kwargs)[source]¶: Initialize self. See help(type(self)) for accurate signature.

Methods

`__init__`([learning_rate, gamma1, gamma2, …])	Initialize self.
`create_optimizer`(name, **kwargs)	Instantiates an optimizer with a given name and kwargs.
`create_state`(index, weight)	Creates auxiliary state for a given weight.
`create_state_multi_precision`(index, weight)	Creates auxiliary state for a given weight, including FP32 high precision copy if original weight is FP16.
`register`(klass)	Registers a new optimizer.
`set_learning_rate`(lr)	Sets a new learning rate of the optimizer.
`set_lr_mult`(args_lr_mult)	Sets an individual learning rate multiplier for each parameter.
`set_lr_scale`(args_lrscale)	[DEPRECATED] Sets lr scale.
`set_wd_mult`(args_wd_mult)	Sets an individual weight decay multiplier for each parameter.
`update`(index, weight, grad, state)	Updates the given parameter using the corresponding gradient and state.
`update_multi_precision`(index, weight, grad, …)	Updates the given parameter using the corresponding gradient and state.

Attributes

`learning_rate`
`opt_registry`