-
-
CMA-ES Youtube Series
Finally finished the CMA-ES Youtube series after much procrastination! [Read More] -
Estimating gradients with TD3 (Differentiable Quality-Diversity part 2)
Welcome back! Previously we talked about how Fontaine, Matthew, and Stefanos Nikolaidis (2021) and Tjanaka, Bryon, et al. (2022) used gradients to simultaneously optimize fitnesses and explore diverse behavior descriptors in MAP-Elites, as well as how to estimate gradients using noisy samples (see Differentiable Quality-Diversity part 1). Towards the end... [Read More] -
How to use and estimate gradient in MAP-Elites (Differentiable Quality-Diversity part 1)
Hi there! This and the next blog post aim at intuitively explaining CMA-MEGA and some of its variants. “CMA” stands for CMA-ES, and “MEGA” stands for MAP-Elites via a Gradient Arborescence, so the name gives away two constituent parts of the algorithm. If you are unfamiliar with CMA-ES, you can... [Read More] -
Intelligent Trial and Error: MAP-Elites for adaptive robot
Hi there! This blog showcases one of the many interesting applications of MAP-Elites algorithm. Namely, we’ll look at the paper Robots that can adapt like animals where the MAP-Elites archive is used for keeping “backup solutions” in case the optimum solution doesn’t perform as well as expected, thus allowing the... [Read More] -
CMA-ME: Using CMA-ES to guide MAP-Elites
Hi there! This blog follows up the previous blog on MAP-Elites, where we explored the idea of maintaining an archive of solutions that accomplish the target objective through a diverse set of behaviors. This was primarily motivated by the need to overcome local optima, but the archive could also be... [Read More] -
MAP-Elites Introduction
Hi there! This blog post aims to be a gentle introduction to MAP-Elites algorithm. We’ll look at how MAP-Elites works, what motivated its development, why it might be a promising direction of research, and we’ll end with a demo where we implement a simple MAP-Elites and apply it on the... [Read More] -
Adapting update step size and direction (CMA-ES part 3)
Welcome back! Previously in part 2 we looked at why and how CMA-ES de-coupled step size $\sigma$ and direction $C$. Here in part 3 we will discuss the mechanisms by which each is adapted independently. [Read More] -
Sampling and recombination (CMA-ES part 2)
Welcome back! Part 2 & 3 will go over the pseudocode from The CMA Evolution Strategy: A Tutorial page 29. I try to stay on the intuition level and focus on the motivations behind each line. Rest assured! I will not dive too deep into the exact mathematical derivations. Here’s... [Read More] -
Simple Evolution Strategy and Covariance Matrix Adaptation (CMA-ES part 1)
Hi there! This blog series is intended as an intuitive explanation on Covariance Matrix Adaptation Evolution Strategy(CMA-ES) algorithm. I’m planning to divide the series into three parts. Part 1 skims over some background knowledge about Evolution Strategy(ES) in general. I used the otoro tutorial as a base when making part... [Read More]