Skip to content

Releases: dolthub/dolt

1.86.2

16 Apr 01:08

Choose a tag to compare

Merged PRs

dolt

  • 10887: fixed failing bats test
  • 10876: go/store/nbs: Fix a rare race condition whould could cause GC and some concurrent readers or writers to deadlock.
    If a GC was started immediately after another GC finished, a session which was waiting on the first GC completion could get stuck waiting for the second GC to complete instead. But the second GC itself was going to block on that session finishing whatever work it was doing.
    Added tracking for the gcCycleCounter which reflects the actual GC operation which any individual call into the NomsBlockStore actually needs to block on.
  • 10875: Add swappable identifier validation function
    Adds a function variable that Doltgres can override to change identifier validation logic.
    Related to dolthub/doltgresql#2580
  • 10873: Fix panic when branch tracking has no merge ref
    Fixes a nil pointer panic in DOLT_PULL when a branch has a remote recorded in its tracking configuration but no merge ref set.
    Fix #10839
  • 10820: go: derive internal cache sizes from GOMEMLIMIT

    Summary

    • Solves #10821
    • Adds a memlimit package that reads standard Go GOMEMLIMIT and proportionally sizes Dolt's three largest caches (node cache, memtable, decoded chunks)
    • When GOMEMLIMIT is unset, all defaults preserved exactly — zero behavior change
    • Replaces the per-cache env var approach from #10815/#10816 with a single standard knob

    Usage

    GOMEMLIMIT=512MiB dolt sql-server

    Design

    Cache Default At GOMEMLIMIT=512MiB
    Prolly node cache 256 MiB ~120 MiB
    Memtable write buffer 256 MiB ~120 MiB
    Decoded chunks cache 32 MiB ~15 MiB
    Budget partitioning: 50% usable (50% GC reserve), then 47/47/6 across caches — matching the ratio of the original hardcoded defaults (256:256:32 MiB). Minimums prevent pathologically small caches. Maximum caps ensure caches never exceed current defaults, even with high GOMEMLIMIT — we can loosen these later if needed.

    Changes

    • New: go/libraries/doltcore/memlimit/ — budget calculator with 8 unit tests
    • Modified: go/store/prolly/tree/node_store.go — lazy sync.Once init via memlimit
    • Modified: go/store/types/value_store.go — decoded chunks from memlimit
    • Modified: go/libraries/doltcore/dbfactory/ — all 8 factory impls call memlimit.MemtableSize() directly
    • Modified: go/cmd/dolt/dolt.gomemlimit.Init() in runMain()

    What this doesn't do

    • No custom DOLT_* env vars
    • No YAML config additions
    • No automemlimit dep yet (follow-up for container cgroup auto-detection)
    • Doesn't touch commit cache (already has DOLT_COMMIT_CACHE_SIZE)

    Test plan

    • 8 unit tests: defaults, zero, 512 MiB, 128 MiB, 32 MiB (minimums), 4 GiB (capped at defaults), accessors
    • store/prolly/tree, store/types, libraries/doltcore/dbfactory, store/prolly test suites pass
    • Full binary build + live RSS measurement (needs CI with ICU headers)
      🤖 Generated with Claude Code

go-mysql-server

  • 3510: Ignore unique key errors when inserting into fulltext position table
    fixes #10882
    Ignore unique key errors when inserting into fulltext index.Position table (similar to how unique key errors are ignored for index.DocCount).
    We should consider using a non-unique index for index.Position and index.DocCount or restructuring how we're writing to these tables if we're always going to be ignoring unique key errors.
  • 3508: go.mod: Bump go-icu-regex.
  • 3505: trim analyzer rules for simple selects
    This PR adds a fast analysis pass for queries of the form SELECT ... FROM ... WHERE ....
    The idea is that simple select queries (no subqueries) can avoid many (recursive) analyzer rules like applying triggers and foreign keys.
    Benchmarks: #10824 (comment)
  • 3501: Several new spatial funcs
    Also revived our MySQL engine test harness to verify these new tests match the current MySQL behavior

Closed Issues

  • 10882: REPLACE INTO panics on FULLTEXT-indexed table when old and new content share a word at the same position
  • 10839: DOLT_PULL panics with nil pointer when branch tracking not configured in repo_state.json
  • 10841: (Dolt shell) Do not throw error when parsing delimiter statements that follow a line comment
  • 10573: Support jsonl format in dolt table export
  • 10849: dolt server consumes significant CPU while idle
  • 3506: ICU regex dependency causes portability problems for downstream consumers
  • 3507: CGO compilation fails because icu.go includes C++ ICU header unicode/regex.h via C compiler

1.86.1

10 Apr 23:34

Choose a tag to compare

Merged PRs

dolt

  • 10844: Truncate line comments when parsing sql statements
    Fixes #10828
    We were not able to detect delimiter statements that were preceded with a line comment since we look for the delimiter keyword at the beginning of a statement, and the delimiter statement would have a line comment in front of it.
    This PR strips out any line comments that either take up a whole line on its own or follow a delimited statement on the same line (line comments in the middle of a statement are preserved and parsed out later on). This allows us to detect the delimiter keyword even if there's a line comment that precedes it, since the line comment gets parsed out as its own empty statement.
    Also rearranged struct fields via fieldalignment and removed unused statementScanner struct
  • 10838: go/store/nbs: Fix races between GC PruneTableFiles and Conjoin/fetch/pull.
    The mechanisms that nbs.(*fileTablePersister) used to try to avoid cleaning up in-flight files were not safe in all cases. Under some conditions files which were in use could be removed.
    Rework the interfaces for landing new files into the store so that those files are never removed until the unit of work associated with them is completed.
  • 10830: go/store/nbs: archive_writer.go: Make an archive file's name (more) dependent on its contents, more closely matching the behavior of table file names.
    Table files are named based on a hash of their suffixes array. And their suffixes array is stored in insertion order. And their suffixes arrays reflect every "byte span" in the file. So the table files are named based on their chunks and their chunks' order in the file.
    Archive files used to be named based on a hash of their suffixes array as well. But their suffixes are stored sorted by chunk address. And they have dictionaries and stuff which aren't reflected in the suffixes array at all. So their names were dependent on the chunks they stored, but not fully dependent on their data section, and not even dependent on the order of their chunks within their data section.
    All existing TableFilePersisters are happy to overwrite files. The assumption is that the files themselves are named based on their contents and any version of it is as good as any other. For archives, that meant that two archives containing the same chunks but in a different order or with different dictionaries would get the same name and potentially cause misbehavior when one version changed out for another from underneath a Dolt.
    This change makes archive names much more data dependent. Their entire contents go into the file name, including the metadata payload. We changed the metadata payload to record source that contributed to the conjoin, when the archive is the result of a conjoin. The metadata payload also always includes the Dolt version. So this makes archive names much less deterministic across versions and provenance, but with the exact some contents, than their table file counterparts. However, deterministically building table files as part of pull/push or GC is not really a thing, and things are already written in a non-deterministic order on basically all write paths. So this is not deemed to be a concern.
  • 10818: error early when rebase cli operating against a remote server
    We overlooked the rebase cli command when talking to a remote server should error out quickly due to the inability to checkout the branch on the client.
  • 10714: feat(pull): add --rebase flag to dolt pull
    Running tests for: #10618
  • 10618: #10058 feat(pull): add --rebase flag to dolt pull

    Summary

    Changes

    • Add --rebase/-r flag to dolt pull (CLI) and CALL dolt_pull (SQL) so that after fetching, local commits are rebased onto the upstream branch instead of merging
    • Handles three cases: up-to-date, fast-forward, and diverged history (calls startRebase + continueRebase)
    • Validates flag conflicts (--rebase vs --squash, --no-ff, --ff-only, --no-commit)
    • CLI conflict resolution sets @@dolt_allow_commit_conflicts and syncs to the temp rebase branch so users can resolve and dolt rebase --continue

    Tests:

    • 21 CLI BATS tests in pull.bats
    • 19 SQL BATS tests in sql-pull.bats
    • covers: divergent history, fast-forward, up-to-date, flag conflicts, data/schema conflict resolution (--theirs/--ours), multi-commit rebase, empty commit drop,
      uncommitted changes guard, abort and retry, branch collision, explicit remote+branch args, conflict state inspection during paused rebase
      Closes #10058

go-mysql-server

  • 3505: trim analyzer rules for simple selects
    This PR adds a fast analysis pass for queries of the form SELECT ... FROM ... WHERE ....
    The idea is that simple select queries (no subqueries) can avoid many (recursive) analyzer rules like applying triggers and foreign keys.
    Benchmarks: #10824 (comment)
  • 3503: Preserve whether left join excludes null when building best plan
    fixes #10699
    Building all left joins using NewLeftOuterJoin was causing all left joins to be LeftOuterJoin[IncludingNulls], but for left joins created from an AntiJoin (which are used for NOT IN queries), we want to be using LeftOuterJoinExcludingNulls (left joins created from an AntiJoinIncludingNulls, which are used for NOT EXISTS queries, are default LeftOuterJoin since LeftOuterJoin does include null by default). Instead, we should be using NewJoin and specifying the join op type.
  • 3499: Support for unwrapping geometry values, which may be stored inline or out of band

Closed Issues

  • 10828: Parse error when comment precedes delimiter in a .sql file import
  • 10058: [Feature Request] dolt pull --rebase
  • 10699: Bug: NOT IN with subquery that returns NULL
  • 8572: Potential TUI tool like Lazygit

1.86.0

06 Apr 23:49

Choose a tag to compare

Backwards Incompatible Changes

This version adds support for conflict resolution with the dolt_revert() stored procedure and the dolt revert CLI. To support that, the response schema from dolt_revert() has changed to the same response schema as dolt_merge() and dolt_cherry_pick() so that a count of data conflicts, schema conflicts, and constraint violations can be returned.

Per Dolt’s versioning policy, this is a minor version bump because these changes may impact existing applications. Please reach out to us on GitHub or Discord if you have questions or need help with any of these changes.

Merged PRs

dolt

  • 10810: Fix remotesapi push fail with "target has uncommitted changes" after any successful push
    • Fixed regression where pushing to a SQL server left the server's staged and working state pointing at the previous commit instead of the new one.
    • Pushes to servers with ignored tables correctly advances all three branch pointers together.
      Fix #10807
  • 10806: validate noop in secondary index Update()
    Updates to non-indexed columns do not need Delete and Put or materialize tuples.
    Turns out it is worth it to compare the sql.Row fields involved for secondary index Update statements.
  • 10799: Support bare format with ssh transport
    Altered the dolt transfer command to determine if we are pushing to a bare repository or a full fledge generational repository. Also added the ability to push to an empty directory - which results in a bare repository.
  • 10797: go: store/nbs: Use pointer receivers for some large structs related to working with archives.
  • 10794: enable adaptive encoding for geometry types
    Relies on dolthub/go-mysql-server#3499
  • 10790: go/store/nbs: gcFinalizer: Have Finalize open and carry a chunkSourceSet for the specs.
    This can be used to avoid opening the files in the rebase within swapTables and addTableFiles.
    It can also be used to directly implement the correct HasManyFunc result of AddChunksToStore. This is more robust than requiring the table file stay in the destination chunk store exactly as is, for example.
    Combined with #10788 opening this file immediately and keeping it open until the rebase makes concurrent destructive operations like PruneTableFiles safer.
  • 10788: go/store/nbs: fileTablePersister keeps track of open files and tracks its own keepers as well.
  • 10786: Add support for manually resolving conflicts with dolt_revert() and dolt revert
    Previously any conflict or constraint violation would cause dolt_revert() to error out. With this change, it returns information on the conflicts or constraint violations so that customers can manually resolve them and then continue the revert action.
    Note: in order to return information about conflicts and constraint violations, the return schema of dolt_revert() has changed. It now matches the return schema for dolt_cherry_pick() and dolt_merge(). Revert's behavior also changes to create one revert commit per input commit (instead of a single revert commit) to match Git's behavior. Because of that, this change should be released as a minor version bump.
  • 10785: Fix diff and show hiding committed tables matching dolt_ignore
    Ignore patterns apply to staging untracked tables, not committed history. dolt diff, dolt show, and dolt_diff_summary() were incorrectly filtering tables from commit-to-commit diffs when those tables matched a dolt_ignore pattern after the commit.
    • Add IsWorkingSetRef to doltdb and ShouldIgnoreDelta to IgnorePatterns as shared helpers.
    • Fix diffUserTables to skip ignore pattern evaluation when both refs are commits, using hasWorkingSet on diffDatasets
    • Fix dolt_diff_summary() to apply the same gate, correcting commit-to-commit queries.
    • Restrict ignore filtering to newly added or dropped tables in both layers; modifications to tracked tables are always shown.
    • Add enginetests for dolt_diff(), dolt_patch() and dolt_diff_summary() to lock in correct commit-to-commit behavior across independent implementations.
      Fix #10782
      Related dolthub/docs#2813
  • 10781: go/nbs/store: store.go,conjoiner.go: Small improvements to the efficiency and robustness of conjoin.
    Keep the conjoined chunk source open as part of the conjoinOp after the table file persister makes and opens it. Make it available as a pre-opened chunkSource to the rebase call, assuming the manifest update lands. This keeps Dolt from having to re-open the table file immediately after the manifest update.
    Fix a dropped error in finalizeConjoin after the updateManifest call.
    Fix a bug where ConjoinTableFiles didn't close oldTables before the cleanup.
    Fix a bug where ConjoinTableFiles didn't manifestMgr.LockForUpdate before the manifest updates. This does not have an impact in Dolt, but for hygiene's sake we should be consistent for now.

go-mysql-server

  • 3499: Support for unwrapping geometry values, which may be stored inline or out of band
  • 3498: eventscheduler: Block event scheduler's Close() on finishing its spawned goroutines.
    Otherwise the background thread is still running and potentially accessing the database.

Closed Issues

  • 10807: RemotesAPI push: FastForward skips working set update, causing 'target has uncommitted changes' on next push
  • 10808: Non-local tables do not persist across branches (dolt sql-server)
  • 10782: dolt show / dolt diff retroactively hides committed tables added to dolt_ignore

1.85.0

01 Apr 21:29

Choose a tag to compare

There are no known backwards incompatibilities in this release. It is a minor version release in order to mark the oldest version of Dolt 1.x that will be binary compatible with 2.x releases, which will introduce storage options that will not be able to be read by clients older than 1.85.0.

Merged PRs

dolt

  • 10793: go: sqle: auto_gc.go: Rework naive cpu scheduler so that we do not have to poll it if we have no work to dispatch.
    Helps with #10563.
  • 10792: go: sqle,commands/engine: Integrate with the new GMS mechanism that allows eventscheduler to quiesce when there are no events to dispatch.
    Amends the changes made in #10784, since QuiescableEventDatabase is not longer a marker interface. QuiesceEvents returns a bool, and databases from sqle/database_provider return true now.
  • 10791: enable adaptive encoding with an env var
    Added new CI configurations to exercise this code path.
    Also added a compatibility-breaking changes to prevent older dolt clients from misinterpreting adaptive encoded values.
  • 10784: Revert "go: sqle,commands/engine: Integrate with the new GMS mechanism that allows eventscheduler to quiesce when there are no events to dispatch."
    Reverts #10777
    QuiescableEvents interface will change to better accommodate mysql_db.PrivilegedDatabase.
  • 10777: go: sqle,commands/engine: Integrate with the new GMS mechanism that allows eventscheduler to quiesce when there are no events to dispatch.
    Part of the feature request #10563.
  • 10774: Fix panic merging a changed unique index
    When a unique index's definition changed on one branch (e.g. a column was added to it), merging would panic with byte slice is not of expected size. This happened because the uniqueness checker loaded the index's old binary data and tried to read it under the new schema's layout.
    • Unique constraint violations produced by the changed index are now surfaced as inspectable conflict artifacts.
      Fix #10753
  • 10768: go: doltdb,sqle/cluster: Update database hooks so that some do not fire on replication writes.
    Since #10722, sql-server running a remotesapi endpoint has run commit hooks. This is the correct behavior for a standalone remotesapi endpoint, but remotesapi is also used for cluster replication. When a standby receives a replicated write over remotesapi, it does not want to fire its own replication hooks. It should only fire some hooks --- those involved in maintenance of the server itself.
    This PR updates hooks infrastructure to make the distinction between the two types of hooks. It updates cluster replication's remotesrv to only fire the appropriate hooks.
  • 10766: Bump golang.org/x/image from 0.18.0 to 0.38.0 in /go
    Bumps golang.org/x/image from 0.18.0 to 0.38.0.
    Commits
    • 23ae9ed tiff: cap buffer growth to prevent OOM from malicious IFD offset
    • e589e60 webp: allow VP8L + VP8X(with alpha)
    • fe7d73d go.mod: update golang.org/x dependencies
    • e3d762b all: upgrade go directive to at least 1.25.0 [generated]
    • 833c6ed go.mod: update golang.org/x dependencies
    • bc7fe0b go.mod: update golang.org/x dependencies
    • c53c97f go.mod: update golang.org/x dependencies
    • 9032ff7 all: eliminate vet diagnostics
    • 9c9d08c go.mod: update golang.org/x dependencies
    • 742b1b7 all: fix some comments
    • Additional commits viewable in compare view

    [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/image&package-manager=go_modules&previous-version=0.18.0&new-version=0.38.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
    Dependabot commands and options
    You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/dolthub/dolt/network/alerts).
  • 10749: bidirectional compatibility testing
    This PR introduces compatibility testing that exercises alternating operations of two different releases on the same database, simulating an upgrade / downgrade scenario. It also adopts the same bats test naming convention that dolt uses for its main test suite.

go-mysql-server

  • 3498: eventscheduler: Block event scheduler's Close() on finishing its spawned goroutines.
    Otherwise the background thread is still running and potentially accessing the database.
  • 3497: sql: Change from a marker method on QuiescableEventDatabase to something that returns a bool.
    This interacts better with PrivilegedDatabase because type assertions on that thing are everywhere.
  • 3496: Revert "Make QuiescableEvents discoverable on the PrivilegedDatabase wrapper."
    Reverts dolthub/go-mysql-server#3493
    Tons of call sites in go-mysql-server and some in Dolt and Doltgres have concrete type assertions on mysql_db.PrivilegedDatabase
  • 3493: Make QuiescableEvents discoverable on the PrivilegedDatabase wrapper.
  • 3492: eventscheduler: Add the ability for eventscheduler to quiesce if there are no registered events and all registered databases promise to notify the scheduler anytime events change.
  • 3484: sql/plan: cached_results.go: Remove the global CachedResultsManager. The Analyzer now owns and threads it.
    Removing global state and making it per-Engine allows GMS and Dolt to better support multiple isolated Engines running concurrently in the same process.

Closed Issues

  • 10563: Server twiddles its thumbs every 10ms when "idle"
  • 10131: [feature request] dolt stash apply
  • 10488: Add --overwrite-ignore and --no-overwrite-ignore to dolt checkout
  • 10753: Panic attempting to merge: byte slice is not of expected size
  • 10757: SELECT v.1 FROM v fails with syntax error when view has integer column name

Performance

Read Tests MySQL Dolt Multiple
covering_index_scan 1.93 0.55 0.28
groupby_scan 13.46 10.09 0.75
index_join 1.5 1.86 1.24
index_join_scan 1.47 1.34 0.91
index_scan 34.33 22.28 0.65
oltp_point_select 0.2 0.27 1.35
oltp_read_only 3.82 5.28 1.38
select_random_points 0.35 0.53 1.51
select_random_ranges 0.39 0.55 1.41
table_scan 34.95 22.69 0.65
types_table_sc...
Read more

1.84.1

30 Mar 23:01

Choose a tag to compare

Merged PRs

dolt

  • 10773: go: cmd/dolt: sqlserver: metrics_listener.go: Only run the background goroutine for updating system metrics and cluster state if we actually have a metrics port configured.
    Related to feature request #10563.
  • 10772: Relax PK column mapping to match by name if tags don't match
  • 10771: go: sqle: commit_hooks.go: Quiesce async_push_sync_replica thread when there is no work.
    Signal over a channel when new work is potentially pending.
    Related to feature request #10563.
  • 10770: go: sqle: auto_gc.go: Remove periodic call to checkForGC in autoGCCommitHook thread.
    Previously we wanted standby replicas to run auto gc, but standby replicas did not fire commit hooks when receiving writes over remotesapi. To get autogc working on standby replicas, we had the thread wake up every second and check the database sizes.
    The hooks behavior was changed in #10722. As a consequence, we no longer need to wake up every second and check database sizes. It is more efficient to not do so.
    One behavior change to note: this periodic check would also apply to long running transactions which generate lots of writes/garbage but which never commit and so never generate database writes. Now that pattern will have to wait for an actual database write to trigger an auto gc. A sequence of transactions which generate a lot of garbage but which always rollback will also fail to trigger auto gc. To fix this, auto gc should be triggering on a table file persister callback, not on a commit callback.
    Related to the user request in #10563.
  • 10761: go: store/nbs: Disable conjoin when the store is being garbage collected.
    A conjoin running during GC could violate the assumptions made by the garbage collector about which files it has created and were safe to reference in the future. A separate issue on a specialized write path which GC uses to interact with the store means that the store can be updated to reference files which no longer exist. The end result was that the GC would fail and the database would enter a state that would require expert manual invervention to recover.
    Trigger this path requires a dolt_gc('--full') to be called when the oldgen is already in a very particular state.
    This path does not effect auto_gc.
  • 10752: Bump fast-xml-parser from 5.5.6 to 5.5.9 in /.github/actions/ses-email-action
    Bumps fast-xml-parser from 5.5.6 to 5.5.9.
    Commits

    [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=fast-xml-parser&package-manager=npm_and_yarn&previous-version=5.5.6&new-version=5.5.9)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
    Dependabot commands and options
    You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/dolthub/dolt/network/alerts).
  • 10747: /go/go.{mod,sum} bump google.golang.org/grpc to 1.79.3
  • 10744: go: sqle: txLocks: Move txLocks to be on the DatabaseProvider.
    This has no real impact on dolt itself. Removing global state is part of improving dolthub/driver support.
  • 10740: Prevent panic when first branch control load fails
  • 10739: Delay autogc during heavy CPU load
    autogc and archiving causes spikes during TPCC benchmarks, specifically the delivery transaction. benchmark.
    This PR implements a very naive scheduler that delays garbage collection when the CPU is busy.
    There is an environment variable and logic that allows us to select different gc schedulers (if more are added in the future), and it currently defaults to NAIVE.
  • 10736: Fix getting gen sys tables
    Related to my last change, decided on this design.
  • 10735: go: sqle: replication.go: Use unique thread names in BackgroundThreads for each async replication background thread.
    While this does not impact behavior today, because BackgroundThreads is happy to overwrite existing map entries and leave the old threads still running, if BackgroundThreads adds more requirements or instrospection in the future, having unique names will be nice.
  • 10733: go: cmd/dolt: cli/stdio.go: Redirect Stdout and Stderr to DevNull for now.
    Yeeting it into random UUID files in $TMPDIR is not the right approach.
  • 10731: Fix dolt push failing when remote has ignored tables in working set
    dolt push returned "target has uncommitted changes. --force required to overwrite" when the remote's working set contained tables listed in dolt_ignore, even though dolt status showed clean. Push now succeeds in this case, consistent with how other commands (rebase, pull, stash) already handle ignored tables.
    Fix #10727
  • 10725: restore forward-compatibility testing
    We can only go back to around v1.50.0, because beyond that clients can't read the archive format created by a remote push (test setup involved pushing to a remote and then cloning from it). But this will at least alert us when we're about to make a change that can't accommodate older clients going forward.
  • 10718: Get generated sys tables correctly
    Some current issues where when looking a specific scheme, we call getGeneratedSytemTables, which gets system tables from all schemas, resulting in trying to check for tables that don't exist in scope.
  • 10638: go: dolt_backup.go: More efficiently restore backups.
    When dolt backup restore was migrated to SQL, it lost the ability to restore from table files. It always did the merkle dag pull instead.
    This restores the ability of dolt backup restore to do a table file pull when that is appropriate.
  • 10616: Codeaucafe/10488
    Companion #10549
  • 10549: #10488: Add --overwrite-ignore and --no-overwrite-ignore
    This PR adds --overwrite-ignore and --no-overwrite-ignore to dolt checkout (for both CLI and SQL procedure). This attempts to mirror Git's behavior. By default, --overwrite-ignore is implied, and checkout silently overwrites ignored tables during a branch switch. When --no-overwrite-ignore is specified, checkout aborts if the target branch would overwrite ignored tables in the working set. Per Git docs, the check only triggers "when the new branch contains ignored files," so tables that only exist locally are not flagged.
    • Register flag constants and argparser entries
    • Add overwriteIgnore bool through all checkout paths: doDoltCheckout -> checkoutNewBranch / checkoutExistingBranch / checkoutRemoteBranch -> doGlobalCheckout -> MoveWorkingSetToBranch -> transferWorkingChanges -> RootsForBranch
    • Add FindOverwrittenIgnoredTables() using dolt_ignore to detect ignored tables that differ between working set and target branch
    • Add ErrCheckoutWouldOverwriteIgnoredTables error type with clear user-facing message
      Closes #10488
      Closes #10616

go-mysql-server

  • 3490: bug fix: add check for unmatched column
    conjCollector maintains a map of column ordinals, but the code in addEq() and addIneq() wasn't normalizing the column name before doing a lookup in the column ordinal map. These functions also weren't checking if there wasn't a matching column in the ordinal map.
    This resulted in one of our query plan tests choosing the wrong index, because the zero value of the missed ordinal lookup incorrectly pointed to an entry in the ordinal map that contained the wrong value.
  • 3488: Fix WHERE filter on view columns backed by string literals fails with index out of bounds
    When using CREATE VIEW v(c1, c2) AS SELECT ..., 'literal' from t, filtering the view with WHERE col2 = 'something'...
Read more

1.84.0

20 Mar 23:22

Choose a tag to compare

A minor version bump due to some small backwards incompatibility changes:

  1. respecting schema serialized encoding instructions on decoding Geometry values, instead of falling back to the other encoding if the default one failed
  2. call dolt_stats_info() JSON payload gets a new field lastUpdate
  3. Pushes to a sql-server runnings a remotesapi endpoint have some new behavior:
    3a) A server at 1.84.0+ receiving such a push will trigger database hooks, which control things like push-on-write replication
    3b) A client at 1.84.0+ making such a push, when pushing a new branch, will cause a working set for that branch to also be created. This might change first-transaction-against-that-branch semantics in some cases, since previously the working set would have been created automatically as part of the first transaction running against the branch.

Merged PRs

dolt

  • 10724: allow both Geometry and GeometryAddr encoding for existing Geometry columns
  • 10722: go: sqle/remotesrv.go: remotesapi writes against sql-server now trigger DoltDB commit hooks.
    When dolt sql-server runs with --remotesapi-port, incoming dolt push operations write directly to the underlying ChunkStore, bypassing the hooksDatabase wrapper on DoltDB. This meant commit hooks — including push-on-write replication, stats, cluster sync, and auto-GC — were silently skipped for any write that arrived via the remotesapi endpoint rather than through the SQL engine.
    Changes
    sqle/remotesrv.go — the DBCache implementation used by sql-server's remotesrv — previously discarded the *doltdb.DoltDB (with its registered hooks) after extracting the raw ChunkStore. It now returns a hooksFiringRemoteSrvStore wrapper instead of the bare store.
    After a successful Commit(), the wrapper:
    1. Diffs the dataset map at last vs current noms root hashes
    2. Fires ddb.ExecuteCommitHooks() for every ref-typed dataset (refs/heads/, refs/tags/, etc.) whose head address changed or was deleted — matching the behaviour of hooksDatabase
      No changes to remotesrv, doltdb, or the gRPC layer were required.
      Tests
    • sqle/remotesrv_hook_test.go — unit tests covering the basic behavior of firing hooks on an incoming commit; hooks fire for successful commit, hooks do not fire on a rejected (CAS-failed) commit; hooks fire for deleted datasets (matching hooksDatabase.Delete); hooks only fire for datasets that actually changed.
    • integration-tests/bats/sql-server-remotesrv.bats — end-to-end test: starts a sql-server with push-on-write replication to a file remote and --remotesapi-port, performs a dolt push from a client, and asserts the replication remote is updated.
  • 10721: couple random splunk improvements
    This fixes the ability of noms show to print index data (panicking before this change).
    Also patches the splunk tool to allow more than one hash on a single output line to be examined.
    Sample output:
    Table -  {
    1)  	Schema: #q3kfuk6atrj3b6e46ultp97f86gh9crk
    2)  	Violations: #00000000000000000000000000000000
    3)  	Artifacts: #00000000000000000000000000000000
    Autoinc: 0
    4)  	Primary Index (rows 1, depth 1) #ep59g4n947egeqpqfe7ou4ikjvi81rov {
    5) 6)      { key: 7669657700, 6d797600 value:  #jcch47j2ueiiuq31iuuvvbg7ve6inccc,  #53drvdrsgbvfhtnd03tlgbm1ngm72dp4, 4e4f5f454e47494e455f535542535449545554494f4e2c4f4e4c595f46554c4c5f47524f55505f42592c5354524943545f5452414e535f5441424c455300 }
    }
    Secondary Indexes (indexes 0, depth 1) o0te4bsh {
    }
    }
    > 5
    SerialMessage - {
    Blob - create view myv as select * from abc
    }
    > ..
    Table -  {
    1)  	Schema: #q3kfuk6atrj3b6e46ultp97f86gh9crk
    2)  	Violations: #00000000000000000000000000000000
    3)  	Artifacts: #00000000000000000000000000000000
    Autoinc: 0
    4)  	Primary Index (rows 1, depth 1) #ep59g4n947egeqpqfe7ou4ikjvi81rov {
    5) 6)      { key: 7669657700, 6d797600 value:  #jcch47j2ueiiuq31iuuvvbg7ve6inccc,  #53drvdrsgbvfhtnd03tlgbm1ngm72dp4, 4e4f5f454e47494e455f535542535449545554494f4e2c4f4e4c595f46554c4c5f47524f55505f42592c5354524943545f5452414e535f5441424c455300 }
    }
    Secondary Indexes (indexes 0, depth 1) o0te4bsh {
    }
    }
    > 6
    SerialMessage - {
    Blob - {"CreatedAt":0}
    }
    
  • 10719: go: statspro: Add a quiesced state.
    Stats should not continuously consume CPU resources scanning database roots when it knows that stats are up to date and no intervening writes have occurred.
  • 10715: go: store/datas: database_common: On FastForward with working set checks, create the working set with the correct root hash if it does not already exist.
    Previously, we avoided doing this because there is logic in the SQL server which automatically create the working set if it doesn't already exist. However, that logic cannot work in all cases, such as multi-branch access within a SQL transaction. It's more appropriate and more aligned with Dolt's expectations going forward to always create the working set on the branch write if it does not already exist.
    Because of remotesapi.PushConcurrencyControl, this change does not change the behavior of Dolt when writing to a pure remotesapi endpoint, like dolthub.com. In that case, working sets are not typically pushed to the remote and they are not written on a branch write.
  • 10708: Store encoding in typeinfo, and use the serialized encoding as canonical when reading values from disk
    The goal of this change is to honor the encoding of a schema column, as used by values for that column written to disk, when we deserialize it. Currently we compute the encoding for a column from its SQL type, serialize it to disk as part of the schema, then on deserialization throw it away and compute it from the SQL type again. Now, we derive it from the SQL type on first write, and honor the value serialized to disk when reading it back.
    This makes it possible for different versions of Dolt, or the same version with different configuration / settings, to change how the same SQL type is serialized to disk. For 2.0, we will begin serializing BLOB and TEXT types with adaptive encoding by default. Existing columns will continue using the encoding they designated at the time of column creation.
    For compatibility testing, I've expanded our existing integration testing to consider every possible SQL type and all possible DML operations on it.
  • 10707: Remove ValidateTagUniqueness check

go-mysql-server

  • 3473: fix missing check expressions in child expressions of update
  • 3471: Changed some reflection checks to use type equality
    reflect.DeepEquals has issues with some types represented by integrators, so we're using type.Equals in such scenarios.

Closed Issues

  • 10089: Dolt does not show the correct precision for datetime
  • 10706: JSON_OBJECT on datetime field doesn't represent fractional seconds
  • 10698: Interactive rebase fails with 'changes in branch' when ignored tables exist in working set

Performance

Read Tests MySQL Dolt Multiple
covering_index_scan 1.93 0.55 0.28
groupby_scan 13.46 9.91 0.74
index_join 1.5 1.86 1.24
index_join_scan 1.44 1.39 0.97
index_scan 34.33 21.89 0.64
oltp_point_select 0.2 0.27 1.35
oltp_read_only 3.82 5.18 1.36
select_random_points 0.35 0.53 1.51
select_random_ranges 0.39 0.56 1.44
table_scan 34.95 22.28 0.64
types_table_scan 77.19 66.84 0.87
reads_mean_multiplier 1.0
Write Tests MySQL Dolt Multiple
oltp_delete_insert 8.28 6.43 0.78
oltp_insert 4.1 3.13 0.76
oltp_read_write 9.06 11.45 1.26
oltp_update_index 4.18 3.19 0.76
oltp_update_non_index 4.18 3.13 0.75
oltp_write_only 5.28 6.09 1.15
types_delete_insert 8.43 6.79 0.81
writes_mean_multiplier 0.9
TPC-C TPS Tests MySQL Dolt Multiple
tpcc-scale-factor-1 94.04 38.04 2.47
tpcc_tps_multiplier 2.47
Overall Mean Multiple 1.46

1.83.8

18 Mar 20:20

Choose a tag to compare

Merged PRs

dolt

  • 10705: bug fix: don't error on ignored tables when continuing a rebase
    When continuing an active rebase, Dolt checks to make sure that the branch being rebased hasn't had other changes applied before it update that branch pointer to the new rebased commits. This logic wasn't taking account of ignored tables before, which preventing users from being able to complete an interactive rebase.
    Fixes: #10698
  • 10704: Set timezone in Doltgres CI tests
  • 10703: go: store/nbs,doltdb: When Clone into a destination database, disable Conjoin running as part of landing files into the destination.
    When we land files into a destination database as part of a Clone, by default we used to be able to trigger a conjoin on the destination database. After the change in #10652, we now also block on any of those conjoin operations completing as we closed the DestDB that was created as part of a backup sync/sync-url operation.
    In the case of backup, this conjoin behavior is a little contrary to what we are trying to accomplish.
    This PR changes it so that we dynamically disable conjoin on the destination while we clone into it. After the clone is finished, we reenable conjoin. At that point, if the database is used as an actual database, conjoin will kick in as expected. But if it is immediately closed, the destination files stay as they were.
  • 10681: bug fix: binlog schema change replication to a MySQL replica
    Fixes bugs with schema changes when binary logging is enabled, such as changing a column's type (e.g. VARCHAR to TEXT), adding and dropping columns. Existing type serialization logic that converts diff data into binlog messages was split into two separate phases: deserializing the data from the Dolt tuple store (using the original schema) and serializing the data into the MySQL binlog protocol (using the target schema).
  • 10665: cache node hash

go-mysql-server

  • 3468: Separate update expressions into explicit and derived update expressions
    fixes #10657
    Update expressions are separated into explicit and derived. Derived update expressions are only evaluated if the explicit update expressions actually made any changes to the row.
    Also removes some dead code and includes some minor fixes/refactors

Closed Issues

  • 10698: Interactive rebase fails with 'changes in branch' when ignored tables exist in working set
  • 10692: Server permanently unresponsive after client crash during CREATE/DROP DATABASE
  • 8572: Potential TUI tool like Lazygit

Performance

Read Tests MySQL Dolt Multiple
covering_index_scan 1.96 0.55 0.28
groupby_scan 14.46 9.91 0.69
index_join 1.52 1.82 1.2
index_join_scan 1.47 1.42 0.97
index_scan 34.33 22.28 0.65
oltp_point_select 0.2 0.27 1.35
oltp_read_only 3.82 5.28 1.38
select_random_points 0.35 0.53 1.51
select_random_ranges 0.39 0.55 1.41
table_scan 34.95 22.28 0.64
types_table_scan 75.82 66.84 0.88
reads_mean_multiplier 1.0
Write Tests MySQL Dolt Multiple
oltp_delete_insert 8.28 6.43 0.78
oltp_insert 4.18 3.13 0.75
oltp_read_write 9.22 11.45 1.24
oltp_update_index 4.25 3.19 0.75
oltp_update_non_index 4.18 3.13 0.75
oltp_write_only 5.28 6.09 1.15
types_delete_insert 8.43 6.79 0.81
writes_mean_multiplier 0.89
TPC-C TPS Tests MySQL Dolt Multiple
tpcc-scale-factor-1 93.91 37.53 2.5
tpcc_tps_multiplier 2.5
Overall Mean Multiple 1.46

1.83.7

18 Mar 02:55

Choose a tag to compare

Merged PRs

dolt

  • 10702: Special handling of comment delimiters in StreamScanner
    fixes #10694
  • 10700: Fix Dolt table function privilege checks when current database is revision-qualified
    MySQL privilege grants are stored against the base database name, so when a user called a Dolt table function with mydb/revision as the current database, the privilege check looked up the revision-qualfied name instead of mydb.
    • Update dolt_schema_diff, dolt_diff, dolt_diff_stat, dolt_diff_summary, dolt_log, dolt_patch, dolt_preview_merge_conflicts, dolt_preview_merge_conflicts_summary, and dolt_query_diff to strip revision qualifier before performing privilege checks.
      Fix #10695
      Companion dolthub/docs#2804
  • 10696: go: doltcore/sqle: database_provider.go: Fix a deadlock in DROP DATABASE.
    A lock-acquisition order error could cause DROP DATABASE to conflict on lock acquisition with a DoltSession's own Mutex. The result was that all sessions would then block indefinitely and irrecoverably on trying to access any database loaded in the server.
    This changes the order of operations and the lock acquisition sequencing to not dead lock.
    Note: finalization of DROP DATABASE with regards to sessions and ongoing queries which may be accessing the database is still somewhat lacking. To be improved in the future.
    Fixes: #10692.
  • 10691: Have fsTablePersister implement movingTableFilePersister
    The movingTableFilePersister interface is for tablePersister implementations that build a temporary table file and have the ability to move the temporary file into the final location instead of copying the contents into a new one.
    #9118 added support for persisting archive files. As part of this change, the signature of a method on movingTableFilePersister was changed to accept the more generic GenericTableWriter interface instead of the more specific CmpChunkTableWriter.
    Unfortunately, the signature was not also updated on the fsTablePersister implementation, resulting in it no longer implementing the interface. As a result, we lost the ability move table files during garbage collection. This had no impact on correctness, but potentially had a performance impact.
    This PR fixes this issue.
  • 10690: /go/{cmd,libraries} check for valid dbs, close opened dbs on error
  • 10689: /{dist,package-lock.json}: rebuild dist
  • 10683: Bump undici from 6.23.0 to 6.24.0 in /.github/actions/ses-email-action
    Bumps undici from 6.23.0 to 6.24.0.
    Release notes

    Sourced from undici's releases.

    v6.24.0

    Undici v6.24.0 Security Release Notes (LTS)

    This release backports fixes for security vulnerabilities affecting the v6 line.

    Upgrade guidance

    All users on v6 should upgrade to v6.24.0 or later.

    Fixed advisories

    • GHSA-2mjp-6q6p-2qxm / CVE-2026-1525 (Medium)
      Inconsistent interpretation of HTTP requests (request/response smuggling class issue).

    • GHSA-f269-vfmq-vjvj / CVE-2026-1528 (High)
      Malicious WebSocket 64-bit frame length handling could crash the client.

    • GHSA-4992-7rv2-5pvq / CVE-2026-1527 (Medium)
      CRLF injection via the upgrade option.

    • GHSA-v9p9-hfj2-hcw8 / CVE-2026-2229 (High)
      Unhandled exception from invalid server_max_window_bits in WebSocket permessage-deflate negotiation.

    • GHSA-vrm6-8vpv-qv8q / CVE-2026-1526 (High)
      Unbounded memory consumption in WebSocket permessage-deflate decompression.

    Not applicable to v6

    Affected and patched ranges (v6)

    • CVE-2026-1525: affected < 6.24.0, patched 6.24.0
    • CVE-2026-1528: affected >= 6.0.0 < 6.24.0, patched 6.24.0
    • CVE-2026-1527: affected < 6.24.0, patched 6.24.0
    • CVE-2026-2229: affected < 6.24.0, patched 6.24.0
    • CVE-2026-1526: affected < 6.24.0, patched 6.24.0

    References

    Commits
    • 8873c94 Bumped v6.24.0
    • 411bd01 test(websocket): use node:assert for Node 18 compatibility
    • 844bf59 test: fix http2 lint regressions in backport
    • a444e4f test: stabilize h2 and tls-cert-leak under current test runner
    • dc032a1 fix: h2 CI (#4395)
    • 4cd3f4b test: increase bitness in test/fixtures/*.pem (#3659)
    • 7df6442 fix: adapt websocket frame-limit handling for v6 parser
    • 4e0179a fix: reject duplicate content-length and host headers
    • 5a97f08 Fix websocket 64-bit length overflow
    • e43e898 fix: validate upgrade header to prevent CRLF injection
    • Additional commits viewable in compare view

    [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=undici&package-manager=npm_and_yarn&previous-version=6.23.0&new-version=6.24.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
    Dependabot commands and options
    You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/dolthub/dolt/network/alerts)...
Read more

1.83.6

14 Mar 01:30

Choose a tag to compare

Merged PRs

dolt

  • 10679: remove archive sanity check
    For sanity, we recalculate the hash for chunks and compare it to the one we expect from the archive.
    If we are confident that it works, we can speed up AutoGC (at least according to pprofs).
  • 10673: go: remotesrv: getUploadUrl: Respect X-Forwarded-Proto so returned URLs work appropriately behind tls-terminating reverse proxies.
    Adopts the same behavior as download URLs already have. Upload URLs were missed in the original fix.
    Fixes #10672.
  • 10551: ssh transport support
    This change adds a new command, dolt transfer which is enables ssh remotes. It's similar to git upload-pack, in that it is intended to be executed by SSH with a few parameters in order to perform clone,push, and fetch operations.
    Dolt's native replication is over a combination of gRPC and HTTP operations. dolt transfer uses SMUX to perform those transports over a bi-directional byte stream. We use the stdin / stdout of the SSH connection as that stream.
    Fixes: #1027

go-mysql-server

  • 3468: Separate update expressions into explicit and derived update expressions
    fixes #10657
    Update expressions are separated into explicit and derived. Derived update expressions are only evaluated if the explicit update expressions actually made any changes to the row.
    Also removes some dead code and includes some minor fixes/refactors
  • 3467: add sql.Databaser to alter default and alter index
  • 3458: Move prepared queries to sql.Session and add parsed query cache
    This PR moves prepared statements into the sql.Session instead of the engine. This avoids the need for a mutex and cleans up the code. It's also just a more appropriate place for prepare and execute queries.
    Additionally, this PR also adds a cache for triggers. This avoids reparsing triggers that have already been parsed.

Closed Issues

  • 10657: ON_UPDATE columns should only update if other column values have actually changed
  • 10648: Panic during binlog replication
  • 10655: Mobile View
  • 10677: mysql_native_password plugin not found when connecting with MySQL 9.x client
  • 10672: getUploadUrl ignores X-Forwarded-Proto — push fails behind TLS reverse proxy
  • 1027: SSH Support for connecting to remotes

Performance

Read Tests MySQL Dolt Multiple
covering_index_scan 1.96 0.55 0.28
groupby_scan 13.22 9.91 0.75
index_join 1.52 1.82 1.2
index_join_scan 1.47 1.39 0.95
index_scan 34.33 22.28 0.65
oltp_point_select 0.2 0.27 1.35
oltp_read_only 3.82 5.18 1.36
select_random_points 0.35 0.53 1.51
select_random_ranges 0.39 0.55 1.41
table_scan 34.95 22.28 0.64
types_table_scan 77.19 66.84 0.87
reads_mean_multiplier 1.0
Write Tests MySQL Dolt Multiple
oltp_delete_insert 8.43 6.43 0.76
oltp_insert 4.18 3.13 0.75
oltp_read_write 9.06 11.24 1.24
oltp_update_index 4.25 3.19 0.75
oltp_update_non_index 4.18 3.13 0.75
oltp_write_only 5.28 6.09 1.15
types_delete_insert 8.58 6.79 0.79
writes_mean_multiplier 0.88
TPC-C TPS Tests MySQL Dolt Multiple
tpcc-scale-factor-1 93.4 37.76 2.47
tpcc_tps_multiplier 2.47
Overall Mean Multiple 1.45

1.83.5

11 Mar 22:47

Choose a tag to compare

Merged PRs

dolt

  • 10669: fix concurrent create database if exist atomicity
  • 10668: /.github/workflows/cd-push-docker-image.yaml: option for disabling la…
    …test push

go-mysql-server

  • 3466: fix create db if not exists and drop db if exists to not return errors under concurrency
  • 3465: support IF EXISTS in ALTER TABLE
  • 3463: remove hacky handling of trigger parsing
    Companion PR: dolthub/vitess#459
  • 3461: Update ON UPDATE columns during INSERT...ON DUPLICATE KEY UPDATE
    fixes #10627
  • 3458: Move prepared queries to sql.Session and add parsed query cache
    This PR moves prepared statements into the sql.Session instead of the engine. This avoids the need for a mutex and cleans up the code. It's also just a more appropriate place for prepare and execute queries.
    Additionally, this PR also adds a cache for triggers. This avoids reparsing triggers that have already been parsed.
  • 3455: Do not convert AntiJoins to LeftOuterJoin when inside an Update node
    fixes #10600

vitess

  • 459: fix trigger parsing
    Should use old_lexer_position instead of lexer_position to avoid token getting swallowed.

Closed Issues

  • 10667: CREATE DATABASE IF NOT EXISTS returns error under concurrent execution
  • 9077: ERRO[6058] stats executor errorserialQueue panicked running work: assertion failed: empty chunk returned from ChunkStore