badger "cannot use generic type ristretto.Cache[K z.Key, V any] without instantiation"

[carsonip]

update-beats #14296 is pulling in latest beats (with badgerv4 dep after elastic/beats#40939 ) which pulls in github.com/dgraph-io/ristretto v0.1.2-0.20240116140435-c67e07994f91 which contains a breaking change and compile error. See issue in ristretto dgraph-io/ristretto#394 and dgraph-io/ristretto#386

 Error: #25 16.09 /go/pkg/mod/github.com/dgraph-io/badger/[email protected]/table/table.go:79:14: cannot use generic type ristretto.Cache[K z.Key, V any] without instantiation
Error: #25 16.09 /go/pkg/mod/github.com/dgraph-io/badger/[email protected]/table/table.go:80:14: cannot use generic type ristretto.Cache[K z.Key, V any] without instantiation

ristretto has cut a v0.2.0 release before the breaking change, and v1 after. But this does not fix our use case.

Potential solutions

1. APM server has to bump to badgerv4 (e.g. v4.3.1 so use ristretto v1). Work to update badger is tracked in Update badger to latest version #11546.
2. Revert beats badgerv4 bump feat: bump badger to v4 beats#40939
3. I don't think there's a way to use both badgerv4 (via beats) and badgerv2 in apm-server at the same time as v1 ristretto will be picked over v0 as they share the same import path, and badgerv2 code will get a compile error. Unless we convince ristretto to release a v2 and make badgerv4 use it. See discussion.
4. Revert beats bump in apm-server

APM Server version (apm-server version): in PR branches e.g. #14296

Provide logs (if relevant):

https://github.com/elastic/apm-server/actions/runs/11232054143/job/31222853639

[kruskall]

This will be fixed once #11546 is fixed.

I'll keep this open since this is about bumping beats but will post updates in the other issue

[kruskall]

Can be closed by pinning to an older ristretto version #14469