Dual Annealing Optimization Example in Python

     Dual annealing is a stochastic global optimization algorithm based on combined Classical Simulated Annealing and Fast Simulated Annealing algorithms. Simulated annealing is an optimization algorithm for approximating the global optima of a given function.

    SciPy provides dual_annealing() function to implement dual annealing method in Python. In this tutorial, we'll briefly learn how to implement and solve optimization problem with dual annealing by using this SciPy function.

    The tutorial covers:

1. Dual annealing with 2D function
2. Dual annealing with 3D function
3. Source code listing
   

 We'll start by loading the required libraries.

from scipy.optimize import dual_annealing
import matplotlib.pyplot as plt
import numpy as np

 

Dual annealing with 2D function

 

   We'll start creating a 2-dimensional objective function to search its global optimum and visualize it in a plot.  Below code shows how to define a function in a range and its visualization in a plot.

 

# define function 

def func(x):
    x = x**2+2*np.sin(x*np.pi)
    return x

# define ranges

x = np.arange(-2, 2, 0.01)
y = func(x)

# visualize in a plot

plt.title("Given function in x range")
plt.plot(x, y)
plt.grid()
plt.show()  

 

Our task is to search a global minimum of the above function in given range. 

 

bounds = [(-2, 2)] 
result = dual_annealing(func, bounds=bounds)
print(result)

 fun: array([-1.77303647])
 message: ['Maximum number of iteration reached']
    nfev: 2025
    nhev: 0
     nit: 1000
    njev: 12
  status: 0
 success: True
       x: array([-0.4538536])   

 

Here, the result contains the following attributes:

    fun - value of objective function 

    nfev -  the number of evaluation, 

    nit - the number of iterations, 

    success - the existence of optimizer, 

    x -  the solution of the optimization

 

Now we can visualize the minimum of function in a plot. 

plt.title("Describing the function minimum")
plt.plot(x, y, label="y")
plt.plot(result['x'], result['fun'], 'sr', label="global minimum")
plt.legend(loc='best', fancybox=True, shadow=True)
plt.grid()
plt.show()
 

 

 

 

 

Dual annealing with 3D function

 

   Next, we'll apply dual annealing for 3-dimensional function. Below code shows how to define objective function, create mesh grid, and visualize it in a plot.

 

# define ranges
x_range = np.arange(-4, 4, 0.1)
y_range = np.arange(-4, 4, 0.1)

# create meshgrid
x, y = np.meshgrid(x_range, y_range)

# define the function 
z = np.sqrt(np.sqrt(x**2+y**2))

# Plot the surface.
fig, ax = plt.subplots(subplot_kw={"projection": "3d"})
surf = ax.plot_surface(x, y, z, cmap=cm.jet,
                       linewidth=0, antialiased=False)

fig.colorbar(surf, shrink=0.5, aspect=5)
plt.show()

 

    Our task is to search a global minimum of the above function in given range by using dual_annealing() function. The function requires search bounds and callable objective function. We'll redefine function without changing the formula above. 

 

 
# define function
def func(p):
    x, y = p
    r = np.sqrt(x**2+y**2)
    return np.sqrt(r)

We'll set variable bounds that can be specified by defining the max and min values.

bounds = [[-4, 4], [-4, 4]]

Then execute the dual_annealing() function.

# execute dual annealing search

result = dual_annealing(func, bounds)  

print(result)

     fun: 8.386204887291893e-05
 message: ['Maximum number of iteration reached']
    nfev: 4310
    nhev: 0
     nit: 1000
    njev: 103
  status: 0
 success: True
       x: array([-4.99074449e-09, -4.95513405e-09]) 

The result shows that minimum of the function is located in a point of (-4.99074449e-09, -4.95513405e-09). 

Now we can visualize the result in a plot.

fig, ax = plt.subplots(subplot_kw={"projection": "3d"})
surf = ax.plot_surface(x, y, z, cmap=cm.jet,
                       linewidth=0, antialiased=False)
plt.plot(result['x'][0], result['x'][1], 'o', color="black", label="global minimum")
plt.legend()
plt.show()

 

    In this tutorial, we've briefly learned how to search global optimum by using dual annealing  method in Python. The full source code is listed below. 

 

 

Source code listing

from scipy.optimize import dual_annealing
import matplotlib.pyplot as plt
import numpy as np 

# applying the method for 2-dimensional function  

def func(x):
    x = x**2+2*np.sin(x*np.pi)
    return x

x = np.arange(-2, 2, 0.01)
y = func(x)

plt.title("Given function in x range")
plt.plot(x, y)
plt.grid()
plt.show() 

bounds = [(-2, 2)] 
result = dual_annealing(func, bounds=bounds)
print(result)

plt.title("Describing the function minimum")
plt.plot(x, y, label="y")
plt.plot(result['x'], result['fun'], 'sr', label="global minimum")
plt.legend(loc='best', fancybox=True, shadow=True)
plt.grid()
plt.show() 

# applying the method for 3-dimensional function 

# define ranges
x_range = np.arange(-4, 4, 0.1)
y_range = np.arange(-4, 4, 0.1)

# create meshgrid
x, y = np.meshgrid(x_range, y_range)

# define the function
z = np.sqrt(np.sqrt(x**2+y**2))

# plot the surface
fig, ax = plt.subplots(subplot_kw={"projection": "3d"})
surf = ax.plot_surface(x, y, z, cmap=cm.jet,
                       linewidth=0, antialiased=False)

fig.colorbar(surf, shrink=0.5, aspect=5)
plt.show()

# set bounds
bounds = [[-4, 4], [-4, 4]]

# define function
def func(p):
    x,y = p
    r = np.sqrt(x**2+y**2)
    return np.sqrt(r)

# execute differentian evolution search 
result = dual_annealing(func, bounds)
print(result) 

fig, ax = plt.subplots(subplot_kw={"projection": "3d"})
surf = ax.plot_surface(x, y, z, cmap=cm.jet,
                       linewidth=0, antialiased=False)
plt.plot(result['x'][0], result['x'][1], 'o', color="black", label="global minimum")
plt.legend()
plt.show() 

 

 

References:

1. SciPy Dual Annealing