BFGS: Performance Improvements

[dwmunster]

Overview

Improves performance by reducing allocations in gradient gathering by returning views into the data/gradient in each primitive. BFGS performance is significantly improved by avoiding the formation of large matrix products.

Details

The original version of the BFGS update repeatedly formed intermediate products for terms such as h * y. That product is computed once and reused. Additionally, updates of the form alpha * x * y^T + beta * A (for vectors x, y, matrix A, and scalars alpha, beta) can be computed without forming the intermediate product x * y^T. This is implemented using the ger function in nalgebra.

Benchmarks

On my machine:

Benchmark: CirclesWithLines
n:   30,        primitives:   59,       constraints:  88,       solver: BFGSSolver         ,    solved: true,   error: 0.00,    duration: 0ms
n:   50,        primitives:   99,       constraints: 148,       solver: BFGSSolver         ,    solved: true,   error: 0.00,    duration: 1ms
n:  100,        primitives:  199,       constraints: 298,       solver: BFGSSolver         ,    solved: true,   error: 0.00,    duration: 6ms
n:  300,        primitives:  599,       constraints: 898,       solver: BFGSSolver         ,    solved: true,   error: 0.00,    duration: 48ms
Benchmark: StairsWithLines
n:   30,        primitives:   59,       constraints:  59,       solver: BFGSSolver         ,    solved: true,   error: 0.00,    duration: 0ms
n:   50,        primitives:   99,       constraints:  99,       solver: BFGSSolver         ,    solved: true,   error: 0.00,    duration: 1ms
n:  100,        primitives:  199,       constraints: 199,       solver: BFGSSolver         ,    solved: true,   error: 0.00,    duration: 10ms
n:  300,        primitives:  599,       constraints: 599,       solver: BFGSSolver         ,    solved: true,   error: 0.00,    duration: 245ms

[jplatte]

I see you posted some benchmark results, but I don't see any benchmarks in this repo or in your PR.

Maybe this is not helpful, but just wanted to drop the info that Cargo has (limited) built-in support for running benchmarks. You still need to pull in an external library to actually define the benchmarks, but there's two pretty good crates to choose from: criterion, divan.

[dwmunster]

but I don't see any benchmarks in this repo or in your PR.

The benchmarks have currently been implemented as a set of (ignored) tests. I've done some tire kicking of criterion locally but haven't had a chance to look at pulling that in and/or integration with actions, though I agree that would be valuable.