feat: add the zarr-indexing package (TensorStore-style index transfor…

…ms, ndsel wire format) (#4196)

* fix: byte-order handling for structured dtypes in the bytes codec (#220)

* fix: byte-order handling for structured dtypes in the bytes codec

The bytes codec neither byte-swapped structured-dtype fields to its
configured endian on encode (numpy reports byteorder '|' for void
dtypes, so the top-level byteorder comparison never detected a
mismatch) nor honored its endian when decoding, silently corrupting
any structured data whose field byte order differed from the stored
one (e.g. virtual references to external big-endian data).

Encode now detects byte-order mismatches by comparing full dtypes via
newbyteorder, and decode reinterprets raw bytes in the stored byte
order before converting to the data type's declared byte order, so the
stored layout (codec state) and the in-memory layout (array data type)
are independent.

Closes #4141

Assisted-by: ClaudeCode:claude-fable-5

* test: fold structured byte-order cases into existing bytes codec tests

Extend test_endian's parametrization with structured dtypes and
test_bytes_codec_sync_roundtrip with endian/dtype parametrization plus
stored-layout and decoded-dtype assertions, instead of adding parallel
test functions for the same properties.

Assisted-by: ClaudeCode:claude-fable-5

* refactor: rename stored_dtype to view_dtype in BytesCodec decode

The variable is the dtype used to view the raw chunk bytes (byte order
from the codec's endian configuration), not a property of the stored
data or of the returned buffer, which always carries the array's
declared dtype.

Assisted-by: ClaudeCode:claude-fable-5

* docs: note that the decode-side byte-order conversion copies the chunk

Assisted-by: ClaudeCode:claude-fable-5

* feat: add the zarr-indexing package (TensorStore-style index transforms)

Standalone workspace package extracted from the lazy-indexing branch
(#3906): composable, lazy coordinate transforms
(IndexTransform / IndexDomain / output maps), dependency-aware chunk
resolution against a DimensionGridLike protocol, and an ndsel-conformant
JSON wire format validated against the vendored conformance corpus.

zarr itself does not depend on zarr-indexing yet — the runtime wiring
lands separately once 0.1.0 is published. The package is numpy-only;
its tests exercise chunk resolution against zarr's concrete ChunkGrid,
so they run from the workspace root (uv sync --all-packages).

Assisted-by: ClaudeCode:claude-fable-5

* style: conventional submodule import in the chunk-resolution tests

Assisted-by: ClaudeCode:claude-fable-5

* perf(zarr-indexing): joint chunk enumeration for correlated vindex maps

Candidate-chunk enumeration took the cartesian product of each correlated
ArrayMap's per-dimension distinct chunk ids and relied on intersect() to
filter untouched combinations. For a diagonal selection of P scattered
points that is P**2 intersect calls — quadratic in the number of selected
points, the same workload shape as #4174 (400 points:
~2.6s; 10k points: ~30min).

Group correlated maps jointly instead: broadcast their per-point chunk
ids, take the distinct rows (np.unique(axis=0), O(P log P)), and
enumerate exactly the touched combinations. Candidate slots now carry
chunk-coordinate tuples covering one or more output dimensions;
orthogonal/constant/slice dimensions keep their existing per-dimension
candidates. 400-point diagonal resolution drops from 2628ms to 14ms and
scales linearly.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): standalone documentation site; add package justfile

Mirror the treatment zarr-metadata received in #4208/#4210 onto
zarr-indexing: a self-contained mkdocs site under the package (own
mkdocs.yml, landing page, ndsel wire-format guide, mkdocstrings page per
module, and .readthedocs.yaml for a dedicated RTD project), so the
package presents as a separate project with docs versioned by its own
zarr_indexing-v* release tags rather than zarr-python's. The zarr-python
site's API Reference nav links out to it, and each RTD project now skips
PR builds that do not touch its half of the repo.

The package gains a pinned docs dependency group, a docs build job in its
CI workflow, and a justfile with package-scoped dev recipes. Two recipes
deviate from the zarr-metadata original by design:

- `test` runs against the workspace-root environment (`uv run --project
  ../.. --all-packages --group test`), because the chunk-resolution tests
  exercise this package against zarr's chunk grids and `zarr` is
  deliberately not a dependency of this package.
- `typecheck` uses plain `pyright`, unpinned and on the default
  interpreter, mirroring this package's own CI invocation. The
  zarr-metadata pin exists for a PEP 661 sentinel regression that
  zarr-indexing's sources do not hit.

composition.py gains the module docstring the other modules already have,
since mkdocstrings renders it as the page introduction.

Assisted-by: ClaudeCode:claude-fable-5

* chore: drop the already-released 4141 changelog fragment

The bytes-codec byte-order fix this fragment describes shipped upstream
and its entry is already in docs/release-notes.md; the fragment survived
on this branch only as a rebase remnant, and would emit a duplicate entry
in the next release.

Assisted-by: ClaudeCode:claude-fable-5

* docs(zarr-indexing): canonicalize ndsel references to zarr-developers/ndsel

Also aligns the zarr-indexing workflow's setup-uv pin (v8.3.2) with the
rest of the repo. The vendored-corpus sha is present upstream; the
historical d-v-b/ndsel#1 PR reference stays as provenance.

Assisted-by: ClaudeCode:claude-fable-5

* chore: drop the root uv-workspace wiring for zarr-indexing

Per review: the root pyproject.toml should not change in this PR. The
package now operates fully standalone (like zarr-metadata); the test
invocations layer the package into the repo-root environment as an
editable overlay instead (python -m pytest, since a base-env console
script would not see the overlay).

Assisted-by: ClaudeCode:claude-fable-5

docs: dev blog, performance examples, and compiled 3.3.0 release notes (

#4191)

Adds a development blog to the documentation site with a 3.3.0 release
post covering the FusedCodecPipeline and sharded partial-read
coalescing, plus two runnable examples referenced by the post
(examples/codec_pipeline_performance, examples/sharding_coalescing).

Compiles every pending changelog fragment into a single 3.3.0 release
notes section dated 2026-07-30, merging it with the section compiled
earlier in #4148, and empties changes/ — making this commit suitable to
tag as v3.3.0.

Assisted-by: ClaudeCode:claude-fable-5

chore(zarr-metadata): build 0.4.0 changelog (#4211)

Consume the pending news fragments (the #4119 model layer's feature and
removal notes, plus a new doc fragment for the standalone documentation
site and justfile from #4208/#4210) into CHANGELOG.md via towncrier for
the zarr_metadata-v0.4.0 release.

Assisted-by: ClaudeCode:claude-fable-5

Release zarr-metadata 0.3.0

Widen ChunksLike type alias (#3990)

* Widen ChunksLike type alias

Fixes #3869

* Add changelog file.

* Prefer if/else statement to if/else expression

release : 3.2.1 release notes (#3942)

* release:3.2.1 release notes

* docs: add changelog entry

* docs: remove changelog

* Fix date and add link

---------

Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com>

docs: build 3.2.0 release notes (#3932)

* docs: build release notes

* docs: fix release notes

* fix: whitespace

docs: build 3.2.0 release notes (#3932)

* docs: build release notes

* docs: fix release notes

* fix: whitespace

chore/3.1.6 release notes (#3790)

* have towncrier use h2 header for each release note

* remove h1 headers

* release notes

* rebuild release notes docs with #3769

* add more release notes

* style

* Update release-notes.md

Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com>

---------

Co-authored-by: Max Jones <14077947+maxrjones@users.noreply.github.com>

update release notes (#3598)