Code

(feat): read_elem_as_dask method#1469

Code

Merged

ilan-gold merged 160 commits intomainscverse/anndata:mainfrom ig/read_dask_elemscverse/anndata:ig/read_dask_elemCopy head branch name to clipboardJul 23, 2024

+459-69

Commits

Commits on Apr 11, 2024

• (feat): read_elem_lazy method

  ilan-gold

  committed
  d111f04

  Copy full SHA for d111f04

• (revert): error message

  ilan-gold

  committed
  00be7f0

  Copy full SHA for 00be7f0

• (refactor): declare is_csc reading elem directly in h5

  ilan-gold

  committed
  fd635d7

  Copy full SHA for fd635d7

Commits on Apr 12, 2024

• (chore): read_elem_lazy -> read_elem_as_dask

  ilan-gold

  committed
  f5e7fda

  Copy full SHA for f5e7fda

• (chore): remove string handling

  ilan-gold

  committed
  ae5396c

  Copy full SHA for ae5396c

• (refactor): use elem for h5 where posssble

  ilan-gold

  committed
  664336a

  Copy full SHA for 664336a

Commits on Apr 17, 2024

• Merge branch 'main' into ig/read_dask_elem

  ilan-gold

  authored
  2370215

  Copy full SHA for 2370215

• (chore): remove invlaud syntax

  ilan-gold

  committed
  52002b6

  Copy full SHA for 52002b6

• Merge branch 'ig/read_dask_elem' of github.com:scverse/anndata into ig/read_dask_elem

  ilan-gold

  committed
  5ab1ad1

  Copy full SHA for 5ab1ad1

• (fix): put dask import inside function

  ilan-gold

  committed
  aa1006e

  Copy full SHA for aa1006e

• (refactor): try maybe open?

  ilan-gold

  committed
  dda7d83

  Copy full SHA for dda7d83

Commits on May 27, 2024

• Merge branch 'main' into ig/read_dask_elem

  ilan-gold

  committed
  fd418f0

  Copy full SHA for fd418f0

• Merge branch 'main' into ig/read_dask_elem

  ilan-gold

  authored
  23b0bfd

  Copy full SHA for 23b0bfd

Commits on Jun 3, 2024

• Merge branch 'main' into ig/read_dask_elem

  ilan-gold

  authored
  97b8031

  Copy full SHA for 97b8031

• (fix): revert encoding-version

  ilan-gold

  committed
  1fc4cc3

  Copy full SHA for 1fc4cc3

• (chore): document create_sparse_store test function

  ilan-gold

  committed
  5ca71ea

  Copy full SHA for 5ca71ea

• (chore): sort indices to prevent warning

  ilan-gold

  committed
  3672c18

  Copy full SHA for 3672c18

• (fix): remove utility function make_dask_array

  ilan-gold

  committed
  33c3599

  Copy full SHA for 33c3599

• (chore): read_sparse_as_dask_h5 -> read_sparse_as_dask

  ilan-gold

  committed
  157e710

  Copy full SHA for 157e710

• (feat): make params of h5_chunks and stride

  ilan-gold

  committed
  375000d

  Copy full SHA for 375000d

• (chore): add distributed test

  ilan-gold

  committed
  241904a

  Copy full SHA for 241904a

• (fix): TypeVar bind

  ilan-gold

  committed
  42d0d22

  Copy full SHA for 42d0d22

Commits on Jun 4, 2024

• (chore): release note

  ilan-gold

  committed
  0bba2c0

  Copy full SHA for 0bba2c0

Commits on Jun 5, 2024

• (chore): 0.10.8 -> 0.11.0

  ilan-gold

  committed
  0d0b43a

  Copy full SHA for 0d0b43a

Commits on Jun 26, 2024

• Merge branch 'main' into ig/read_dask_elem

  ilan-gold

  authored
  762d4c6

  Copy full SHA for 762d4c6

• (fix): ruff for default pytest.fixture scope

  ilan-gold

  committed
  c935fe0

  Copy full SHA for c935fe0

Commits on Jul 1, 2024

• Apply suggestions from code review

  Show description for 23e0ea2
  

  ilan-gold

  and

  flying-sheep

  authored
  23e0ea2

  Copy full SHA for 23e0ea2

• (fix): Any to DaskArray

  ilan-gold

  committed
  5b96c77

  Copy full SHA for 5b96c77

• (fix): type make_index + fix undeclared

  ilan-gold

  committed
  0907a4e

  Copy full SHA for 0907a4e

• [pre-commit.ci] auto fixes from pre-commit.com hooks

  Show description for 20ced16
  pre-commit-ci[bot]

  committed
  20ced16

  Copy full SHA for 20ced16

• Merge branch 'main' into ig/read_dask_elem

  ilan-gold

  authored
  36ae8f2

  Copy full SHA for 36ae8f2

• fix rest

  flying-sheep

  committed
  bb6607e

  Copy full SHA for bb6607e

Commits on Jul 2, 2024

• (fix): use chunks kwarg

  ilan-gold

  committed
  419691b

  Copy full SHA for 419691b

• Merge branch 'main' into ig/read_dask_elem

  ilan-gold

  authored
  a23df34

  Copy full SHA for a23df34

• (feat): expose chunks as an option to read_elem_as_dask via dataset_kwargs

  ilan-gold

  committed
  fd2376a

  Copy full SHA for fd2376a

• Merge branch 'ig/read_dask_elem' of github.com:scverse/anndata into ig/read_dask_elem

  ilan-gold

  committed
  ae723d0

  Copy full SHA for ae723d0

• (fix): test_read_dispatched_null_case test

  ilan-gold

  committed
  42b1093

  Copy full SHA for 42b1093

• (fix): disallowed spread syntax?

  ilan-gold

  committed
  78de057

  Copy full SHA for 78de057

• (refactor): reuse compute_chunk_layout_for_axis_shape functionality

  ilan-gold

  committed
  717b997

  Copy full SHA for 717b997

Commits on Jul 3, 2024

• (fix): remove unneeded slice arguments

  ilan-gold

  committed
  2b86293

  Copy full SHA for 2b86293

• (fix): revert message

  ilan-gold

  committed
  8d5a9df

  Copy full SHA for 8d5a9df

• (refactor): make_index -> make_block_indexer

  ilan-gold

  committed
  449fc1a

  Copy full SHA for 449fc1a

• (fix): export from experimental

  ilan-gold

  committed
  1522de3

  Copy full SHA for 1522de3

• (fix): callback signature for `test_read_dispatched_null_case

  ilan-gold

  committed
  71c150d

  Copy full SHA for 71c150d

• (chore): get_elem_name helper

  ilan-gold

  committed
  b441366

  Copy full SHA for b441366

• (chore): use H5Group consistently

  ilan-gold

  committed
  0307a1d

  Copy full SHA for 0307a1d

• (refactor): make chunks public facing API instead of dataset_kwargs

  ilan-gold

  committed
  ee075cd

  Copy full SHA for ee075cd

• (fix): regsiter for group not array

  ilan-gold

  committed
  89acec4

  Copy full SHA for 89acec4

• (chore): add warning test

  ilan-gold

  committed
  48b7630

  Copy full SHA for 48b7630

• (chore): make arg order consistent

  ilan-gold

  committed
  8712582

  Copy full SHA for 8712582

Commits on Jul 5, 2024

• (feat): add callback typing for read_dispatched

  ilan-gold

  committed
  cda8aa7

  Copy full SHA for cda8aa7

• (chore): use npt.NDArray

  ilan-gold

  committed
  e8f62f4

  Copy full SHA for e8f62f4

• (fix): remove uneceesary union

  ilan-gold

  committed
  f6e48ac

  Copy full SHA for f6e48ac

• (chore): release note

  ilan-gold

  committed
  4de3246

  Copy full SHA for 4de3246

• (fix); try protocol docs

  ilan-gold

  committed
  ba817e0

  Copy full SHA for ba817e0

• (feat): create InMemoryElem + DictElemType to remove Any

  ilan-gold

  committed
  438d28d

  Copy full SHA for 438d28d

• (chore): refactor DictElemType -> InMemoryArrayOrScalarType for reuse

  ilan-gold

  committed
  296ea3f

  Copy full SHA for 296ea3f

• (fix): use Union

  ilan-gold

  committed
  cf13a57

  Copy full SHA for cf13a57

• (fix): more Union

  ilan-gold

  committed
  d02ba49

  Copy full SHA for d02ba49

• (refactor): InMemoryElem -> InMemoryReadElem

  ilan-gold

  committed
  6970a97

  Copy full SHA for 6970a97

• (chore): add needed types to public export + docs fix

  ilan-gold

  committed
  2282351

  Copy full SHA for 2282351

Commits on Jul 8, 2024

• Merge branch 'main' into ig/read_dask_elem

  flying-sheep

  committed
  810cd0a

  Copy full SHA for 810cd0a

• (chore): type write_elem functions

  ilan-gold

  committed
  a996081

  Copy full SHA for a996081

• (chore): create write_callback protocol

  ilan-gold

  committed
  f6e457b

  Copy full SHA for f6e457b

• Merge branch 'main' into ig/protocol_for_callback

  ilan-gold

  committed
  a0b4057

  Copy full SHA for a0b4057

• (chore): export + docs

  ilan-gold

  committed
  4416526

  Copy full SHA for 4416526

• (fix): add string descriptions

  ilan-gold

  committed
  fbe44f0

  Copy full SHA for fbe44f0

• (fix): try sphinx protocol doc

  ilan-gold

  committed
  8c1f01d

  Copy full SHA for 8c1f01d

• (fix): try ignoring exports

  ilan-gold

  committed
  a7d412a

  Copy full SHA for a7d412a

• (fix): remap callback internal usages

  ilan-gold

  committed
  4d56396

  Copy full SHA for 4d56396

• (fix): add docstring

  ilan-gold

  committed
  2012ee5

  Copy full SHA for 2012ee5

Commits on Jul 9, 2024

• Discard changes to pyproject.toml

  flying-sheep

  authored
  f65f065

  Copy full SHA for f65f065

• re-add dep

  flying-sheep

  committed
  8f6ea49

  Copy full SHA for 8f6ea49

• Fix docs

  flying-sheep

  committed
  155a21e

  Copy full SHA for 155a21e

• Almost works

  flying-sheep

  committed
  daae3e5

  Copy full SHA for daae3e5

• works!

  flying-sheep

  committed
  c415ae4

  Copy full SHA for c415ae4

• (chore): use pascal-case

  ilan-gold

  committed
  00010b8

  Copy full SHA for 00010b8

• (feat): type read/write funcs in callback

  ilan-gold

  committed
  0bd87fc

  Copy full SHA for 0bd87fc

• (fix): use generic for Read as well.

  ilan-gold

  committed
  5997678

  Copy full SHA for 5997678

• (fix): need more aliases

  ilan-gold

  committed
  f208332

  Copy full SHA for f208332

• Split table, format

  flying-sheep

  committed
  eb69fcb

  Copy full SHA for eb69fcb

• (refactor): move to _types file

  ilan-gold

  committed
  477bbef

  Copy full SHA for 477bbef

• Merge branch 'ig/protocol_for_callback' of github.com:scverse/anndata into ig/protocol_for_callback

  ilan-gold

  committed
  103cad6

  Copy full SHA for 103cad6

• bump scanpydoc

  flying-sheep

  committed
  8d23f6f

  Copy full SHA for 8d23f6f

• Some basic syntax fixes

  flying-sheep

  committed
  9b647c2

  Copy full SHA for 9b647c2

• Merge branch 'ig/protocol_for_callback' into ig/read_dask_elem

  ilan-gold

  committed
  d6d01bc

  Copy full SHA for d6d01bc

• (fix): change Read{Callback} type for kwargs

  ilan-gold

  committed
  5ef93e1

  Copy full SHA for 5ef93e1

• (chore): test chunks argument

  ilan-gold

  committed
  9cfe908

  Copy full SHA for 9cfe908

• (fix): type read_recarray

  ilan-gold

  committed
  99fc6db

  Copy full SHA for 99fc6db

• (fix): GroupyStorageType not StorageType

  ilan-gold

  committed
  b5bccc3

  Copy full SHA for b5bccc3

• (fix): little type fixes

  ilan-gold

  committed
  e5ea2b0

  Copy full SHA for e5ea2b0

• (fix): clarify H5File typing

  ilan-gold

  committed
  6ac72d6

  Copy full SHA for 6ac72d6

• (fix): dask doc

  ilan-gold

  committed
  989dc65

  Copy full SHA for 989dc65

• (fix): dask docs

  ilan-gold

  committed
  36b0207

  Copy full SHA for 36b0207

• Merge branch 'ig/protocol_for_callback' into ig/read_dask_elem

  ilan-gold

  committed
  dadfb4d

  Copy full SHA for dadfb4d

• (fix): typing

  ilan-gold

  committed
  ca6cf66

  Copy full SHA for ca6cf66

• (fix): handle case when chunks is None

  ilan-gold

  committed
  eabaf35

  Copy full SHA for eabaf35

• (feat): add string-array reading

  ilan-gold

  committed
  4c398c3

  Copy full SHA for 4c398c3

• (fix): remove string-array because it is not tested

  ilan-gold

  committed
  d6fc8a4

  Copy full SHA for d6fc8a4

Commits on Jul 10, 2024

• (refactor): clean up tests

  ilan-gold

  committed
  33aebb2

  Copy full SHA for 33aebb2

• (fix): overfetching problem

  ilan-gold

  committed
  701cd85

  Copy full SHA for 701cd85

Commits on Jul 11, 2024

• Fix circular import

  flying-sheep

  committed
  43b21a2

  Copy full SHA for 43b21a2

• add some typing

  flying-sheep

  committed
  0e22449

  Copy full SHA for 0e22449

• fix mapping types

  flying-sheep

  committed
  ec546f4

  Copy full SHA for ec546f4

• Fix Read/Write

  flying-sheep

  committed
  7c2e4da

  Copy full SHA for 7c2e4da

• Fix one more

  flying-sheep

  committed
  1ba5b99

  Copy full SHA for 1ba5b99

• unify names

  flying-sheep

  committed
  49c0d49

  Copy full SHA for 49c0d49

• claift ReadCallback signature

  flying-sheep

  committed
  3666735

  Copy full SHA for 3666735

• Fix type aliases

  flying-sheep

  committed
  3a332ad

  Copy full SHA for 3a332ad

• (fix): clean up typing to use RWAble

  ilan-gold

  committed
  d0f4d13

  Copy full SHA for d0f4d13

• Merge branch 'main' into ig/protocol_for_callback

  ilan-gold

  authored
  6e89e14

  Copy full SHA for 6e89e14

• (fix): use Union

  ilan-gold

  committed
  ea29cfa

  Copy full SHA for ea29cfa

• (fix): add qualname override

  ilan-gold

  committed
  f4ff236

  Copy full SHA for f4ff236

• (fix): ignore dask and masked array

  ilan-gold

  committed
  f50b286

  Copy full SHA for f50b286

• (fix): ignore erroneous class warning

  ilan-gold

  committed
  712e085

  Copy full SHA for 712e085

• (fix): upgrade scanpydoc

  ilan-gold

  committed
  24dd18b

  Copy full SHA for 24dd18b

• (fix): use MutableMapping instead of dict due to broken docstring

  ilan-gold

  committed
  79d3fdc

  Copy full SHA for 79d3fdc

• Merge branch 'ig/protocol_for_callback' into ig/read_dask_elem

  ilan-gold

  committed
  9a2be00

  Copy full SHA for 9a2be00

• Add data docs

  flying-sheep

  committed
  d3bcddf

  Copy full SHA for d3bcddf

• Revert "(fix): use MutableMapping instead of dict due to broken docstring"

  Show description for 84fdc96
  flying-sheep

  committed
  84fdc96

  Copy full SHA for 84fdc96

• (fix): add clarification

  ilan-gold

  committed
  2608bc3

  Copy full SHA for 2608bc3

• Simplify

  flying-sheep

  committed
  e551e18

  Copy full SHA for e551e18

• Merge branch 'ig/protocol_for_callback' into ig/read_dask_elem

  ilan-gold

  authored
  13e3bb1

  Copy full SHA for 13e3bb1

• Merge branch 'main' into ig/read_dask_elem

  ilan-gold

  committed
  2935e45

  Copy full SHA for 2935e45

• Merge branch 'ig/read_dask_elem' of github.com:scverse/anndata into ig/read_dask_elem

  ilan-gold

  committed
  bf0be15

  Copy full SHA for bf0be15

Commits on Jul 12, 2024

• Merge branch 'main' into ig/read_dask_elem

  ilan-gold

  authored
  9d37fc8

  Copy full SHA for 9d37fc8

• (fix): remove double dask intersphinx

  ilan-gold

  authored
  1ffe43e

  Copy full SHA for 1ffe43e

• (fix): remove _types.DaskArray from type checking block

  ilan-gold

  authored
  f9df5bc

  Copy full SHA for f9df5bc

Commits on Jul 15, 2024

• (refactor): use block_info for resolving fetch location

  ilan-gold

  committed
  a85da39

  Copy full SHA for a85da39

• Merge branch 'ig/read_dask_elem' of github.com:scverse/anndata into ig/read_dask_elem

  ilan-gold

  committed
  3bef77c

  Copy full SHA for 3bef77c

• (fix): dtype for reading

  ilan-gold

  committed
  899184f

  Copy full SHA for 899184f

Commits on Jul 16, 2024

• (fix): ignore import cycle problem (why??)

  ilan-gold

  committed
  efb70ec

  Copy full SHA for efb70ec

• (fix): add issue

  ilan-gold

  committed
  118f43c

  Copy full SHA for 118f43c

Commits on Jul 18, 2024

• (fix): subclass Reader to remove datasetkwargs

  ilan-gold

  committed
  f742a0a

  Copy full SHA for f742a0a

• (fix): add message tp errpr

  ilan-gold

  committed
  ae68731

  Copy full SHA for ae68731

• Update tests/test_io_elementwise.py

  Show description for f5e7760
  

  ilan-gold

  and

  ivirshup

  authored
  f5e7760

  Copy full SHA for f5e7760

• (fix): correct self.callback check

  ilan-gold

  committed
  96b13a3

  Copy full SHA for 96b13a3

• (fix): erroneous diffs

  ilan-gold

  committed
  9c68e36

  Copy full SHA for 9c68e36

• (fix): extra read_elem dataset_kwargs

  ilan-gold

  committed
  410aeda

  Copy full SHA for 410aeda

• (fix): remove more dataset_kwargs nonsense

  ilan-gold

  committed
  31a30c4

  Copy full SHA for 31a30c4

• (chore): add docs

  ilan-gold

  committed
  80fe8cb

  Copy full SHA for 80fe8cb

• (fix): use block_info for dense

  ilan-gold

  committed
  b314248

  Copy full SHA for b314248

• (fix): more erroneous diffs

  ilan-gold

  committed
  02d4735

  Copy full SHA for 02d4735

• (fix): use context again

  ilan-gold

  committed
  6e5534a

  Copy full SHA for 6e5534a

Commits on Jul 22, 2024

• (fix): change size by dimension in tests

  ilan-gold

  committed
  d26cfe8

  Copy full SHA for d26cfe8

• (refactor): clean up get_elem_name

  ilan-gold

  committed
  94e43a3

  Copy full SHA for 94e43a3

• (fix): try new sphinx for error

  ilan-gold

  committed
  5160016

  Copy full SHA for 5160016

• (fix): return type

  ilan-gold

  committed
  43da9a3

  Copy full SHA for 43da9a3

• (fix): protocol for reading

  ilan-gold

  committed
  9735ced

  Copy full SHA for 9735ced

• (fix): bring back ignored warning

  ilan-gold

  committed
  f1730c3

  Copy full SHA for f1730c3

• Fix docs

  flying-sheep

  committed
  9861b56

  Copy full SHA for 9861b56

• almost fix typing

  flying-sheep

  committed
  235096a

  Copy full SHA for 235096a

• add wrapper

  flying-sheep

  committed
  dce9f07

  Copy full SHA for dce9f07

• move into type checking

  flying-sheep

  committed
  2725ef2

  Copy full SHA for 2725ef2

• (fix): small type fxes

  ilan-gold

  committed
  ffe89f0

  Copy full SHA for ffe89f0

• Merge branch 'main' into ig/read_dask_elem

  ilan-gold

  authored
  6cb231e

  Copy full SHA for 6cb231e

• block info types

  flying-sheep

  committed
  75a64fc

  Copy full SHA for 75a64fc

• simplify

  flying-sheep

  committed
  3f734fe

  Copy full SHA for 3f734fe

• rename

  flying-sheep

  committed
  c4c2356

  Copy full SHA for c4c2356

• simplify more

  flying-sheep

  committed
  cc67a9b

  Copy full SHA for cc67a9b