20241014 developer call notes

[tomwhite]

20241014

Pre-notes

• Notes from previous meeting

PRs

Issues

• Windowing fails for datasets where contigs lack variant sites Windowing fails for datasets where contigs lack variant sites #1268
• Benchmarking Numba vs JAX Benchmarking Numba vs JAX #1266
• Dask 2024.8.1 and later is very slow Dask 2024.8.1 and later is very slow #1267

Discussions

• Releases
• Plan for moving plink, bgen to bio2zarr

Notes

Attendees

1. Tom White
2. Will Tyler
3. Jerome Kelleher
4. Ben Jeffery
5. Per Unneberg

Discussion

• Benchmarking Numba vs JAX Benchmarking Numba vs JAX #1266
• Upstream
  • Python 3.12 is supported for bio2zarr, vcztools, sgkit now
  • Dask slowdown is fixed upstream now - waiting for a release before we can unpin
• Per is going to give a talk about new methods in popgen - will mention paper
• Spruce data
  • Hitting a heisenbug - will open issues
  • Windowing bug - needs a corner case fix, could possibly workaround by filtering out contigs with no variant sites?
• Bed2zarr - ready to go now, edge cases fixed
• Vcztools performance work by Will
  • JK: next step is to do 1000 genomes, which has lots of columns. Use makefile in the paper repo that downloads the data.
• Plan for moving plink, bgen to bio2zarr
  • JK: 80% done. Need to discuss (e.g. with JH) about where plink and bgen code lives. They are different to VCF as they can be read without being converted to Zarr first
  • TW: want it in one place - bio2zarr would be good as it improves dependencies
• Releases:
  • JK: moving back to the paper - will get releases out in a couple of weeks hopefully (bio2zarr, vcztools)