This repo is dedicated to code which involves the following concepts:
DFO: Derivative Free Optimization
ZOO: Zeroth-Order Optimization
Derivative free optimization (DFO) constitutes a wide class of methods that do not have access to the gradient or any other higher order information. Zeroth-order optimization (ZOO) typically describes methods that aim to approximate the gradient in some way (such as finite difference), and is often considered to be a subset of DFO.
This repo currently contains a notebook playing around with a few simple ZOO methods, and even a few direct search DFO-type methods (although we do not explore any "model based" methods"). These methods are applied to a simple but hard to optimize 2d-function, and their paths, convergence properties, and error are all looked into.