Figure 12

Convergence of the GMR search process. (a-f) A sample search in the augmented goal-policy space ζ is plotted for six selected self-refinement iterations. Here, only one dimension for ζ ^O and ζ ^I is depicted, but we have ${\mathit{\zeta }}^O\in {ℝ}^{36}$ and ${\mathit{\zeta }}^I\in {ℝ}^9$. The gray dots show the initial set of iterations and the black dots are the new estimated policy. The blue line represents the known goal of the task with highest reward on which the red cross shows the best achieved policy after convergence.