20210805 developer call notes #648

tomwhite · 2021-08-05T16:18:21Z

tomwhite
Aug 5, 2021
Maintainer

sgkit PR#647 [WIP] Weir-Goudet kinship estimator

sgkit#646 Presentation for GA4GH "Future of VCF" meeting

TW: slides are ready for Monday. JK has created simulated data.
JK: surprised that SAV compression is not as good as Zarr. Especially as PBWT is state of the art.
TW: not sure PBWT is enabled by default. For 1000 genomes chr22 data, SAV is slightly better than Zarr.
TW: any use in generating 100Mb?
JK: no - just multiply numbers by 10 since number of variants is a function of number of samples, not size of genome.
JH: spoke to Ryan Williams about Zarr and CZI grant. Josh Moore is doing a lot of the day-to-day work on Zarr. Not sure about status of v3. Might be worth having a slide about it.
TW: everything we have done is with v2, and the only thing v3 would enable is pluggable storage for sparse data, which arguable we don’t need given JK’s numbers compared to SAV.
JH: genetic bit packing?
TW: EC wrote something in his prototype which we could revive (but needs changing from 2D to 3D) - but doesn’t need Zarr v3.
EC: how important is decompression perf? Zarr should do well there.
JK: it’s important. I would like to see benchmarking going further. Get some standard files (using msprime/tskit), then have various benchmarks: sequential and random access, for both variants and samples.

Sckit-allel note?

CZI funding?

JK: more upset about not getting feedback. Would like to know if not essential vs need better application.
JK: can use in future apps
JH: have NumFOCUS backing now.

Reach out to academic and industry users?

JH: with release, good time to send link over. Put names on old discourse (with login).
JK: happy to reach out to some.
JK: are we coming out of stealth mode?
JH: semi stealth - paper first. “Friends and family”

Time to revive the paper?

JK: thoughts on timeline?
JH: thought about positioning. EC/TW thoughts? Push a bit on summary stats - good paper topic. PRS would consume summary stats. Good hook for publication. Not a lot of urgency - do more dev on sumstats, PRS first?
JK: benchmarking - are they synced up with plans for paper?
JH: don’t know if we’ll get anything. When school year returns can reach out to Arun. Would be helpful, but had a bit of a false start.

Oxford hiring?

JK: been chasing AM and people in his hierarchy. Will keep chasing. Decision makers are busy (COVID). Still hope to, but out of hands.