Explanation of Meta-optimization, Tuning Optimizers with AI

Meta-optimization from numerical optimization is the use of one optimization method to tune another optimization method. This sophisticated approach is reported to have been used as early as in the late 1970s by Mercer and Sampson for finding optimal parameter settings of a genetic algorithm.

Optimization methods such as a genetic algorithm and differential evolution have several parameters that govern their behaviour and efficiency in optimizing a given problem. Selecting these behavioural parameters by hand is a laborious task that is susceptible to human misconceptions of what makes the optimizer perform well.

Get the Latest Sports News from Seymour's Bird

Receive the latest updates on football, basketball,and technology news directly in your inbox. It's free!

Subscribe

The behavioural parameters of an optimizer can be varied and the optimization performance plotted as a landscape. This is computationally feasible for optimizers with few behavioural parameters, but when the number of parameters increases, the time usage increases exponentially due to the curse of dimensionality.

A simple way of finding good behavioural parameters for an optimizer is to employ another overlaying optimizer, called the meta-optimizer. There are different ways of doing this depending on whether the parameters to be tuned are real-valued or discrete-valued.

Methods and Evolution of Meta-optimization

Meta-optimization and related concepts are also known in the literature as meta-evolution, super-optimization, automated parameter calibration, and hyper-heuristics. Researchers have continuously explored these techniques across various computational frameworks.

Meta-optimizing the parameters of a genetic algorithm was done by Grefenstette and Keane, amongst others, while experiments with meta-optimizing both parameters and genetic operators were reported by Bäck.

Further developments include meta-optimization of particle swarm optimization by Meissner, Pedersen, and Chipperfield, as well as applications to differential evolution and ant colony optimization by other leading computational scientists.

Statistical models have also been used to reveal more about the relationship between choices of behavioural parameters and optimization performance, helping researchers compare various meta-optimization techniques effectively.