Create scoring algorithm

[JamesPHoughton]

Given a stream of participants, choose the stream that

Want the scoring function itself to be independent of the algorithm used to assign points.

For now we can try a density comparison algorithm:

Density comparison

1. Define a function over the space we want to sample: maybe uniform or otherwise
2. Take a KDE of the points we sampled, this gives us
3. Compare the KDE to the desired density function at a number of points
4. Integrate difference over the spaceusing a monte carlo integration (or a grid)?

• A gaussian kernel minimizes the number of assumptions (Statistical Rethinking page 403)
• We can start with a fairly wide kernel, assuming that the landscape will be reasonably flat in any given dimension (ie, likely to be linear or u-shaped in any one dimension)
• As we take data, we can look at a different kernel size, which might suggest a different algorithm.