Quickstart Guide¶
This quickstart guide tries to show the basic mechanisms of how to use pySMAC to find good input parameters to a python function minimizing the return value.
To gain familiarity with the workflow, we shall dissect the file “rosenbrock_example.py” which can be found in the example folder. The full script looks like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | from __future__ import division, print_function
import pysmac
def rosenbrock_4d (x1,x2,x3,x4):
""" The 4 dimensional Rosenbrock function as a toy model
The Rosenbrock function is well know in the optimization community and
often serves as a toy problem. It can be defined for arbitrary
dimensions. The minimium is always at x_i = 1 with a function value of
zero. All input parameters are continuous, but we will pretent that
x2, x3, and x4 can only take integral values. The search domain for
all x's is the interval [-5, 5].
"""
val=( 100*(x2-x1**2)**2 + (x1-1)**2 +
100*(x3-x2**2)**2 + (x2-1)**2 +
100*(x4-x3**2)**2 + (x3-1)**2)
return(val)
parameters=dict(\
# x1 is a continuous ('real') parameter between -5 and 5.
# The default value/initial guess is 5.
x1=('real', [-5, 5], 5),
# x2 can take only integral values, but range and initial
# guess are identical to x1.
x2=('integer', [-5, 5], 5),
# x3 is encoded as a categorical parameter. Variables of
# this type can only take values from a finite set.
# The actual values can be numeric or strings.
x3=('categorical',[5, 2, 0, 1, -1, -2, 4, -3, 3, -5, -4], 5),
# x3 is defined as a so called ordinal parameter. This type
# is similar to 'categorical', but additionally there exists
# an ordering among the elements.
x4=('ordinal', [-5,-4,-3,-2,-1,0,1,2,3,4,5] , 5),
)
# Note: the definition of x3 and x4 is only to demonstrate the different
# types of variables pysmac supports. Here these definitions are overly
# complicated for this toy model. For example, the definitions of x2 and
# x3 are equivalent, but the purpose of this example is not to show a
# realistic use case
# The next step is to create a SMAC_optimizer object
opt = pysmac.SMAC_optimizer()
# Then, call its minimize method with (at least) the three mandatory parameters
value, parameters = opt.minimize(
rosenbrock_4d, # the function to be minimized
1000, # the number of function calls allowed
parameters) # the parameter dictionary
# the return value is a tuple of the lowest function value and a dictionary
# containing corresponding parameter setting.
print(('Lowest function value found: %f'%value))
print(('Parameter setting %s'%parameters))
|
After the necessary import, the function ‘’rosenbrock_4d’’ is defined. This function function is well know and often used test function in the optimization community. Normally, all parameters are continuous, but we will declare three of them to be integer valued to demonstrate the different types that (py)SMAC supports.
The next step is to specify the type, range and default value of every
parameter in the pySMAC way
. In our example, this is done by defining
the dictionary
parameters=dict(\
x1=('real', [-5, 5], 5),
x2=('integer', [-5, 5], 5),
x3=('categorical',[5, 2, 0, 1, -1, -2, 4, -3, 3, -5, -4], 5),
x4=('ordinal', [-5,-4,-3,-2,-1,0,1,2,3,4,5] , 5),
)
Every entry defines a parameter with its name as the key and a tuple
as the corresponding value. The first entry of the tuple defines the type,
here x1
is a real, i.e. a continuous parameter, while x2
is an integer.
Categorical types, like x3
, can only take a value from a user-provided set.
There is no specific order among the elements, which is why in the example the
order of the values is shuffled. In contrast, ordinal parameters posses an
inherent ordering between their values. In the defining list, elements are
assumed to be in increasing order.
The second entry of the tuple defines the possible value. For numerical parameters, this has to be a 2-element list containing the smallest and the largest allowed value. For categorical and ordinal values this list contains all allowed values.
The third tuple entry is the default value. This has to be inside the specified range for real and integer, or an element of the previous list for ordinal and categorical. Please refer to the section Parameter Definitions for more detail.
The example here is cooked up and overly complicated for the actual optimization
problem. For instance, the definitions for x2
and x4
are identical, and
defining x3
as a categorical with only integral values is not exactly the
same as a integer type. But the purpose of this guide was to introduce pySMAC’s
syntax, not solving an interesting problem in the most efficient way possible.
To start the actual minimization, we instantiate a SMAC_optimizer object, and call its minimize method with at least 3 argument:
opt = pySMAC.SMAC_optimizer()
value, parameters = opt.minimize(rosenbrock_4d, 1000, parameters)
The arguments of minimze are the function we which to minimize, the budged of function calls, and the parameter definition. It returns the lowest function value encountered, and the corresponding parameter configuration as a dictionary with parameter name - parameter value as key-value pairs.
The reason why an object is created before a minimization function is called will become clear in the section Advanced configuration of pySMAC.
The output of the code could look like this:
Lowest function value found: 1.691811
Parameter setting {'x3': '1', 'x1': '0.9327937187899629', 'x4': '1', 'x2': '1'}
with the true minimum at (1,1,1,1) having a function value of 0. Given 1000
function evaluations, this result is not competitive to continuous optimizers
if we would drop the integral restriction for x2
, x3
, and x4
. Continuous
optimization allows to estimate and exploit gradients which can lead to a much
faster convergence towards a local optimum. But the purpose of this quickstart guide
is solely to introduce how to use pySMAC rather than showing an interesting example.
The strength of pySMAC become more visible in more complicated cases that are usually
infeasible for standard optimizers.