Releases: dolthub/dolt
1.86.2
Merged PRs
dolt
- 10887: fixed failing bats test
- 10876: go/store/nbs: Fix a rare race condition whould could cause GC and some concurrent readers or writers to deadlock.
If a GC was started immediately after another GC finished, a session which was waiting on the first GC completion could get stuck waiting for the second GC to complete instead. But the second GC itself was going to block on that session finishing whatever work it was doing.
Added tracking for the gcCycleCounter which reflects the actual GC operation which any individual call into the NomsBlockStore actually needs to block on. - 10875: Add swappable identifier validation function
Adds a function variable that Doltgres can override to change identifier validation logic.
Related to dolthub/doltgresql#2580 - 10873: Fix panic when branch tracking has no merge ref
Fixes anilpointer panic inDOLT_PULLwhen a branch has a remote recorded in its tracking configuration but no merge ref set.
Fix #10839 - 10820: go: derive internal cache sizes from GOMEMLIMIT
Summary
- Solves #10821
- Adds a
memlimitpackage that reads standard GoGOMEMLIMITand proportionally sizes Dolt's three largest caches (node cache, memtable, decoded chunks) - When
GOMEMLIMITis unset, all defaults preserved exactly — zero behavior change - Replaces the per-cache env var approach from #10815/#10816 with a single standard knob
Usage
GOMEMLIMIT=512MiB dolt sql-server
Design
Cache Default At GOMEMLIMIT=512MiB Prolly node cache 256 MiB ~120 MiB Memtable write buffer 256 MiB ~120 MiB Decoded chunks cache 32 MiB ~15 MiB Budget partitioning: 50% usable (50% GC reserve), then 47/47/6 across caches — matching the ratio of the original hardcoded defaults (256:256:32 MiB). Minimums prevent pathologically small caches. Maximum caps ensure caches never exceed current defaults, even with high GOMEMLIMIT — we can loosen these later if needed. Changes
- New:
go/libraries/doltcore/memlimit/— budget calculator with 8 unit tests - Modified:
go/store/prolly/tree/node_store.go— lazysync.Onceinit via memlimit - Modified:
go/store/types/value_store.go— decoded chunks from memlimit - Modified:
go/libraries/doltcore/dbfactory/— all 8 factory impls callmemlimit.MemtableSize()directly - Modified:
go/cmd/dolt/dolt.go—memlimit.Init()inrunMain()
What this doesn't do
- No custom
DOLT_*env vars - No YAML config additions
- No
automemlimitdep yet (follow-up for container cgroup auto-detection) - Doesn't touch commit cache (already has
DOLT_COMMIT_CACHE_SIZE)
Test plan
- 8 unit tests: defaults, zero, 512 MiB, 128 MiB, 32 MiB (minimums), 4 GiB (capped at defaults), accessors
-
store/prolly/tree,store/types,libraries/doltcore/dbfactory,store/prollytest suites pass - Full binary build + live RSS measurement (needs CI with ICU headers)
🤖 Generated with Claude Code
go-mysql-server
- 3510: Ignore unique key errors when inserting into fulltext position table
fixes #10882
Ignore unique key errors when inserting into fulltextindex.Positiontable (similar to how unique key errors are ignored forindex.DocCount).
We should consider using a non-unique index forindex.Positionandindex.DocCountor restructuring how we're writing to these tables if we're always going to be ignoring unique key errors. - 3508: go.mod: Bump go-icu-regex.
- 3505: trim analyzer rules for simple selects
This PR adds a fast analysis pass for queries of the formSELECT ... FROM ... WHERE ....
The idea is that simple select queries (no subqueries) can avoid many (recursive) analyzer rules like applying triggers and foreign keys.
Benchmarks: #10824 (comment) - 3501: Several new spatial funcs
Also revived our MySQL engine test harness to verify these new tests match the current MySQL behavior
Closed Issues
- 10882: REPLACE INTO panics on FULLTEXT-indexed table when old and new content share a word at the same position
- 10839: DOLT_PULL panics with nil pointer when branch tracking not configured in repo_state.json
- 10841: (Dolt shell) Do not throw error when parsing delimiter statements that follow a line comment
- 10573: Support jsonl format in
dolt table export - 10849: dolt server consumes significant CPU while idle
- 3506: ICU regex dependency causes portability problems for downstream consumers
- 3507: CGO compilation fails because icu.go includes C++ ICU header unicode/regex.h via C compiler
1.86.1
Merged PRs
dolt
- 10844: Truncate line comments when parsing sql statements
Fixes #10828
We were not able to detectdelimiterstatements that were preceded with a line comment since we look for thedelimiterkeyword at the beginning of a statement, and thedelimiterstatement would have a line comment in front of it.
This PR strips out any line comments that either take up a whole line on its own or follow a delimited statement on the same line (line comments in the middle of a statement are preserved and parsed out later on). This allows us to detect thedelimiterkeyword even if there's a line comment that precedes it, since the line comment gets parsed out as its own empty statement.
Also rearranged struct fields via fieldalignment and removed unusedstatementScannerstruct - 10838: go/store/nbs: Fix races between GC PruneTableFiles and Conjoin/fetch/pull.
The mechanisms thatnbs.(*fileTablePersister)used to try to avoid cleaning up in-flight files were not safe in all cases. Under some conditions files which were in use could be removed.
Rework the interfaces for landing new files into the store so that those files are never removed until the unit of work associated with them is completed. - 10830: go/store/nbs: archive_writer.go: Make an archive file's name (more) dependent on its contents, more closely matching the behavior of table file names.
Table files are named based on a hash of their suffixes array. And their suffixes array is stored in insertion order. And their suffixes arrays reflect every "byte span" in the file. So the table files are named based on their chunks and their chunks' order in the file.
Archive files used to be named based on a hash of their suffixes array as well. But their suffixes are stored sorted by chunk address. And they have dictionaries and stuff which aren't reflected in the suffixes array at all. So their names were dependent on the chunks they stored, but not fully dependent on their data section, and not even dependent on the order of their chunks within their data section.
All existing TableFilePersisters are happy to overwrite files. The assumption is that the files themselves are named based on their contents and any version of it is as good as any other. For archives, that meant that two archives containing the same chunks but in a different order or with different dictionaries would get the same name and potentially cause misbehavior when one version changed out for another from underneath a Dolt.
This change makes archive names much more data dependent. Their entire contents go into the file name, including the metadata payload. We changed the metadata payload to record source that contributed to the conjoin, when the archive is the result of a conjoin. The metadata payload also always includes the Dolt version. So this makes archive names much less deterministic across versions and provenance, but with the exact some contents, than their table file counterparts. However, deterministically building table files as part of pull/push or GC is not really a thing, and things are already written in a non-deterministic order on basically all write paths. So this is not deemed to be a concern. - 10818: error early when rebase cli operating against a remote server
We overlooked the rebase cli command when talking to a remote server should error out quickly due to the inability to checkout the branch on the client. - 10714: feat(pull): add --rebase flag to dolt pull
Running tests for: #10618 - 10618: #10058 feat(pull): add --rebase flag to dolt pull
Summary
Changes
- Add
--rebase/-rflag todolt pull(CLI) andCALL dolt_pull(SQL) so that after fetching, local commits are rebased onto the upstream branch instead of merging - Handles three cases: up-to-date, fast-forward, and diverged history (calls
startRebase+continueRebase) - Validates flag conflicts (
--rebasevs--squash,--no-ff,--ff-only,--no-commit) - CLI conflict resolution sets
@@dolt_allow_commit_conflictsand syncs to the temp rebase branch so users can resolve anddolt rebase --continue
Tests:
- 21 CLI BATS tests in
pull.bats - 19 SQL BATS tests in
sql-pull.bats - covers: divergent history, fast-forward, up-to-date, flag conflicts, data/schema conflict resolution (--theirs/--ours), multi-commit rebase, empty commit drop,
uncommitted changes guard, abort and retry, branch collision, explicit remote+branch args, conflict state inspection during paused rebase
Closes #10058
- Add
go-mysql-server
- 3505: trim analyzer rules for simple selects
This PR adds a fast analysis pass for queries of the formSELECT ... FROM ... WHERE ....
The idea is that simple select queries (no subqueries) can avoid many (recursive) analyzer rules like applying triggers and foreign keys.
Benchmarks: #10824 (comment) - 3503: Preserve whether left join excludes null when building best plan
fixes #10699
Building all left joins usingNewLeftOuterJoinwas causing all left joins to beLeftOuterJoin[IncludingNulls], but for left joins created from anAntiJoin(which are used forNOT INqueries), we want to be usingLeftOuterJoinExcludingNulls(left joins created from anAntiJoinIncludingNulls, which are used forNOT EXISTSqueries, are defaultLeftOuterJoinsinceLeftOuterJoindoes include null by default). Instead, we should be usingNewJoinand specifying the join op type. - 3499: Support for unwrapping geometry values, which may be stored inline or out of band
Closed Issues
1.86.0
Backwards Incompatible Changes
This version adds support for conflict resolution with the dolt_revert() stored procedure and the dolt revert CLI. To support that, the response schema from dolt_revert() has changed to the same response schema as dolt_merge() and dolt_cherry_pick() so that a count of data conflicts, schema conflicts, and constraint violations can be returned.
Per Dolt’s versioning policy, this is a minor version bump because these changes may impact existing applications. Please reach out to us on GitHub or Discord if you have questions or need help with any of these changes.
Merged PRs
dolt
- 10810: Fix remotesapi push fail with "target has uncommitted changes" after any successful push
- Fixed regression where pushing to a SQL server left the server's staged and working state pointing at the previous commit instead of the new one.
- Pushes to servers with ignored tables correctly advances all three branch pointers together.
Fix #10807
- 10806: validate noop in secondary index
Update()
Updates to non-indexed columns do not need Delete and Put or materialize tuples.
Turns out it is worth it to compare thesql.Rowfields involved for secondary index Update statements. - 10799: Support bare format with ssh transport
Altered thedolt transfercommand to determine if we are pushing to a bare repository or a full fledge generational repository. Also added the ability to push to an empty directory - which results in a bare repository. - 10797: go: store/nbs: Use pointer receivers for some large structs related to working with archives.
- 10794: enable adaptive encoding for geometry types
Relies on dolthub/go-mysql-server#3499 - 10790: go/store/nbs: gcFinalizer: Have Finalize open and carry a chunkSourceSet for the specs.
This can be used to avoid opening the files in therebasewithin swapTables and addTableFiles.
It can also be used to directly implement the correct HasManyFunc result of AddChunksToStore. This is more robust than requiring the table file stay in the destination chunk store exactly as is, for example.
Combined with #10788 opening this file immediately and keeping it open until the rebase makes concurrent destructive operations like PruneTableFiles safer. - 10788: go/store/nbs: fileTablePersister keeps track of open files and tracks its own keepers as well.
- 10786: Add support for manually resolving conflicts with
dolt_revert()anddolt revert
Previously any conflict or constraint violation would causedolt_revert()to error out. With this change, it returns information on the conflicts or constraint violations so that customers can manually resolve them and then continue the revert action.
Note: in order to return information about conflicts and constraint violations, the return schema ofdolt_revert()has changed. It now matches the return schema fordolt_cherry_pick()anddolt_merge(). Revert's behavior also changes to create one revert commit per input commit (instead of a single revert commit) to match Git's behavior. Because of that, this change should be released as a minor version bump. - 10785: Fix
diffandshowhiding committed tables matchingdolt_ignore
Ignore patterns apply to staging untracked tables, not committed history.dolt diff,dolt show, anddolt_diff_summary()were incorrectly filtering tables from commit-to-commit diffs when those tables matched adolt_ignorepattern after the commit.- Add
IsWorkingSetReftodoltdbandShouldIgnoreDeltatoIgnorePatternsas shared helpers. - Fix
diffUserTablesto skip ignore pattern evaluation when both refs are commits, usinghasWorkingSetondiffDatasets - Fix
dolt_diff_summary()to apply the same gate, correcting commit-to-commit queries. - Restrict ignore filtering to newly added or dropped tables in both layers; modifications to tracked tables are always shown.
- Add enginetests for
dolt_diff(),dolt_patch()anddolt_diff_summary()to lock in correct commit-to-commit behavior across independent implementations.
Fix #10782
Related dolthub/docs#2813
- Add
- 10781: go/nbs/store: store.go,conjoiner.go: Small improvements to the efficiency and robustness of conjoin.
Keep the conjoined chunk source open as part of the conjoinOp after the table file persister makes and opens it. Make it available as a pre-opened chunkSource to the rebase call, assuming the manifest update lands. This keeps Dolt from having to re-open the table file immediately after the manifest update.
Fix a dropped error in finalizeConjoin after the updateManifest call.
Fix a bug where ConjoinTableFiles didn't close oldTables before the cleanup.
Fix a bug where ConjoinTableFiles didn't manifestMgr.LockForUpdate before the manifest updates. This does not have an impact in Dolt, but for hygiene's sake we should be consistent for now.
go-mysql-server
- 3499: Support for unwrapping geometry values, which may be stored inline or out of band
- 3498: eventscheduler: Block event scheduler's Close() on finishing its spawned goroutines.
Otherwise the background thread is still running and potentially accessing the database.
Closed Issues
1.85.0
There are no known backwards incompatibilities in this release. It is a minor version release in order to mark the oldest version of Dolt 1.x that will be binary compatible with 2.x releases, which will introduce storage options that will not be able to be read by clients older than 1.85.0.
Merged PRs
dolt
- 10793: go: sqle: auto_gc.go: Rework naive cpu scheduler so that we do not have to poll it if we have no work to dispatch.
Helps with #10563. - 10792: go: sqle,commands/engine: Integrate with the new GMS mechanism that allows eventscheduler to quiesce when there are no events to dispatch.
Amends the changes made in #10784, since QuiescableEventDatabase is not longer a marker interface. QuiesceEvents returns a bool, and databases from sqle/database_provider returntruenow. - 10791: enable adaptive encoding with an env var
Added new CI configurations to exercise this code path.
Also added a compatibility-breaking changes to prevent older dolt clients from misinterpreting adaptive encoded values. - 10784: Revert "go: sqle,commands/engine: Integrate with the new GMS mechanism that allows eventscheduler to quiesce when there are no events to dispatch."
Reverts #10777
QuiescableEvents interface will change to better accommodate mysql_db.PrivilegedDatabase. - 10777: go: sqle,commands/engine: Integrate with the new GMS mechanism that allows eventscheduler to quiesce when there are no events to dispatch.
Part of the feature request #10563. - 10774: Fix panic merging a changed unique index
When a unique index's definition changed on one branch (e.g. a column was added to it), merging would panic withbyte slice is not of expected size. This happened because the uniqueness checker loaded the index's old binary data and tried to read it under the new schema's layout.- Unique constraint violations produced by the changed index are now surfaced as inspectable conflict artifacts.
Fix #10753
- Unique constraint violations produced by the changed index are now surfaced as inspectable conflict artifacts.
- 10768: go: doltdb,sqle/cluster: Update database hooks so that some do not fire on replication writes.
Since #10722, sql-server running a remotesapi endpoint has run commit hooks. This is the correct behavior for a standalone remotesapi endpoint, but remotesapi is also used for cluster replication. When a standby receives a replicated write over remotesapi, it does not want to fire its own replication hooks. It should only fire some hooks --- those involved in maintenance of the server itself.
This PR updates hooks infrastructure to make the distinction between the two types of hooks. It updates cluster replication's remotesrv to only fire the appropriate hooks. - 10766: Bump golang.org/x/image from 0.18.0 to 0.38.0 in /go
Bumps golang.org/x/image from 0.18.0 to 0.38.0.Commits
23ae9edtiff: cap buffer growth to prevent OOM from malicious IFD offsete589e60webp: allow VP8L + VP8X(with alpha)fe7d73dgo.mod: update golang.org/x dependenciese3d762ball: upgrade go directive to at least 1.25.0 [generated]833c6edgo.mod: update golang.org/x dependenciesbc7fe0bgo.mod: update golang.org/x dependenciesc53c97fgo.mod: update golang.org/x dependencies9032ff7all: eliminate vet diagnostics9c9d08cgo.mod: update golang.org/x dependencies742b1b7all: fix some comments- Additional commits viewable in compare view
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/dolthub/dolt/network/alerts). - 10749: bidirectional compatibility testing
This PR introduces compatibility testing that exercises alternating operations of two different releases on the same database, simulating an upgrade / downgrade scenario. It also adopts the same bats test naming convention that dolt uses for its main test suite.
go-mysql-server
- 3498: eventscheduler: Block event scheduler's Close() on finishing its spawned goroutines.
Otherwise the background thread is still running and potentially accessing the database. - 3497: sql: Change from a marker method on QuiescableEventDatabase to something that returns a bool.
This interacts better with PrivilegedDatabase because type assertions on that thing are everywhere. - 3496: Revert "Make QuiescableEvents discoverable on the PrivilegedDatabase wrapper."
Reverts dolthub/go-mysql-server#3493
Tons of call sites in go-mysql-server and some in Dolt and Doltgres have concrete type assertions onmysql_db.PrivilegedDatabase - 3493: Make QuiescableEvents discoverable on the PrivilegedDatabase wrapper.
- 3492: eventscheduler: Add the ability for eventscheduler to quiesce if there are no registered events and all registered databases promise to notify the scheduler anytime events change.
- 3484: sql/plan: cached_results.go: Remove the global CachedResultsManager. The Analyzer now owns and threads it.
Removing global state and making it per-Engine allows GMS and Dolt to better support multiple isolated Engines running concurrently in the same process.
Closed Issues
- 10563: Server twiddles its thumbs every 10ms when "idle"
- 10131: [feature request] dolt stash apply
- 10488: Add
--overwrite-ignoreand--no-overwrite-ignoretodolt checkout - 10753: Panic attempting to merge: byte slice is not of expected size
- 10757:
SELECT v.1 FROM vfails with syntax error when view has integer column name
Performance
| Read Tests | MySQL | Dolt | Multiple |
|---|---|---|---|
| covering_index_scan | 1.93 | 0.55 | 0.28 |
| groupby_scan | 13.46 | 10.09 | 0.75 |
| index_join | 1.5 | 1.86 | 1.24 |
| index_join_scan | 1.47 | 1.34 | 0.91 |
| index_scan | 34.33 | 22.28 | 0.65 |
| oltp_point_select | 0.2 | 0.27 | 1.35 |
| oltp_read_only | 3.82 | 5.28 | 1.38 |
| select_random_points | 0.35 | 0.53 | 1.51 |
| select_random_ranges | 0.39 | 0.55 | 1.41 |
| table_scan | 34.95 | 22.69 | 0.65 |
| types_table_sc... |
1.84.1
Merged PRs
dolt
- 10773: go: cmd/dolt: sqlserver: metrics_listener.go: Only run the background goroutine for updating system metrics and cluster state if we actually have a metrics port configured.
Related to feature request #10563. - 10772: Relax PK column mapping to match by name if tags don't match
- 10771: go: sqle: commit_hooks.go: Quiesce async_push_sync_replica thread when there is no work.
Signal over a channel when new work is potentially pending.
Related to feature request #10563. - 10770: go: sqle: auto_gc.go: Remove periodic call to checkForGC in autoGCCommitHook thread.
Previously we wanted standby replicas to run auto gc, but standby replicas did not fire commit hooks when receiving writes over remotesapi. To get autogc working on standby replicas, we had the thread wake up every second and check the database sizes.
The hooks behavior was changed in #10722. As a consequence, we no longer need to wake up every second and check database sizes. It is more efficient to not do so.
One behavior change to note: this periodic check would also apply to long running transactions which generate lots of writes/garbage but which never commit and so never generate database writes. Now that pattern will have to wait for an actual database write to trigger an auto gc. A sequence of transactions which generate a lot of garbage but which always rollback will also fail to trigger auto gc. To fix this, auto gc should be triggering on a table file persister callback, not on a commit callback.
Related to the user request in #10563. - 10761: go: store/nbs: Disable conjoin when the store is being garbage collected.
A conjoin running during GC could violate the assumptions made by the garbage collector about which files it has created and were safe to reference in the future. A separate issue on a specialized write path which GC uses to interact with the store means that the store can be updated to reference files which no longer exist. The end result was that the GC would fail and the database would enter a state that would require expert manual invervention to recover.
Trigger this path requires a dolt_gc('--full') to be called when the oldgen is already in a very particular state.
This path does not effect auto_gc. - 10752: Bump fast-xml-parser from 5.5.6 to 5.5.9 in /.github/actions/ses-email-action
Bumps fast-xml-parser from 5.5.6 to 5.5.9.Commits
- See full diff in compare view
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/dolthub/dolt/network/alerts). - 10747: /go/go.{mod,sum} bump google.golang.org/grpc to 1.79.3
- 10744: go: sqle: txLocks: Move txLocks to be on the DatabaseProvider.
This has no real impact ondoltitself. Removing global state is part of improving dolthub/driver support. - 10740: Prevent panic when first branch control load fails
- 10739: Delay autogc during heavy CPU load
autogc and archiving causes spikes during TPCC benchmarks, specifically thedeliverytransaction. benchmark.
This PR implements a very naive scheduler that delays garbage collection when the CPU is busy.
There is an environment variable and logic that allows us to select different gc schedulers (if more are added in the future), and it currently defaults toNAIVE. - 10736: Fix getting gen sys tables
Related to my last change, decided on this design. - 10735: go: sqle: replication.go: Use unique thread names in BackgroundThreads for each async replication background thread.
While this does not impact behavior today, because BackgroundThreads is happy to overwrite existing map entries and leave the old threads still running, if BackgroundThreads adds more requirements or instrospection in the future, having unique names will be nice. - 10733: go: cmd/dolt: cli/stdio.go: Redirect Stdout and Stderr to DevNull for now.
Yeeting it into random UUID files in $TMPDIR is not the right approach. - 10731: Fix dolt push failing when remote has ignored tables in working set
dolt pushreturned "target has uncommitted changes. --force required to overwrite" when the remote's working set contained tables listed indolt_ignore, even thoughdolt statusshowed clean. Push now succeeds in this case, consistent with how other commands (rebase, pull, stash) already handle ignored tables.
Fix #10727 - 10725: restore forward-compatibility testing
We can only go back to around v1.50.0, because beyond that clients can't read the archive format created by a remote push (test setup involved pushing to a remote and then cloning from it). But this will at least alert us when we're about to make a change that can't accommodate older clients going forward. - 10718: Get generated sys tables correctly
Some current issues where when looking a specific scheme, we callgetGeneratedSytemTables, which gets system tables from all schemas, resulting in trying to check for tables that don't exist in scope. - 10638: go: dolt_backup.go: More efficiently restore backups.
When dolt backup restore was migrated to SQL, it lost the ability to restore from table files. It always did the merkle dag pull instead.
This restores the ability of dolt backup restore to do a table file pull when that is appropriate. - 10616: Codeaucafe/10488
Companion #10549 - 10549: #10488: Add --overwrite-ignore and --no-overwrite-ignore
This PR adds--overwrite-ignoreand--no-overwrite-ignoretodolt checkout(for both CLI and SQL procedure). This attempts to mirror Git's behavior. By default,--overwrite-ignoreis implied, and checkout silently overwrites ignored tables during a branch switch. When--no-overwrite-ignoreis specified, checkout aborts if the target branch would overwrite ignored tables in the working set. Per Git docs, the check only triggers "when the new branch contains ignored files," so tables that only exist locally are not flagged.- Register flag constants and argparser entries
- Add
overwriteIgnore boolthrough all checkout paths:doDoltCheckout->checkoutNewBranch/checkoutExistingBranch/checkoutRemoteBranch->doGlobalCheckout->MoveWorkingSetToBranch->transferWorkingChanges->RootsForBranch - Add
FindOverwrittenIgnoredTables()usingdolt_ignoreto detect ignored tables that differ between working set and target branch - Add
ErrCheckoutWouldOverwriteIgnoredTableserror type with clear user-facing message
Closes #10488
Closes #10616
go-mysql-server
- 3490: bug fix: add check for unmatched column
conjCollectormaintains a map of column ordinals, but the code inaddEq()andaddIneq()wasn't normalizing the column name before doing a lookup in the column ordinal map. These functions also weren't checking if there wasn't a matching column in the ordinal map.
This resulted in one of our query plan tests choosing the wrong index, because the zero value of the missed ordinal lookup incorrectly pointed to an entry in the ordinal map that contained the wrong value. - 3488: Fix
WHEREfilter on view columns backed by string literals fails with index out of bounds
When usingCREATE VIEW v(c1, c2) AS SELECT ..., 'literal' from t, filtering the view withWHERE col2 = 'something'...
1.84.0
A minor version bump due to some small backwards incompatibility changes:
- respecting schema serialized encoding instructions on decoding Geometry values, instead of falling back to the other encoding if the default one failed
call dolt_stats_info()JSON payload gets a new fieldlastUpdate- Pushes to a
sql-serverrunnings aremotesapiendpoint have some new behavior:
3a) A server at1.84.0+ receiving such a push will trigger database hooks, which control things like push-on-write replication
3b) A client at1.84.0+ making such a push, when pushing a new branch, will cause a working set for that branch to also be created. This might change first-transaction-against-that-branch semantics in some cases, since previously the working set would have been created automatically as part of the first transaction running against the branch.
Merged PRs
dolt
- 10724: allow both Geometry and GeometryAddr encoding for existing Geometry columns
- 10722: go: sqle/remotesrv.go: remotesapi writes against sql-server now trigger DoltDB commit hooks.
When dolt sql-server runs with --remotesapi-port, incoming dolt push operations write directly to the underlying ChunkStore, bypassing the hooksDatabase wrapper on DoltDB. This meant commit hooks — including push-on-write replication, stats, cluster sync, and auto-GC — were silently skipped for any write that arrived via the remotesapi endpoint rather than through the SQL engine.
Changes
sqle/remotesrv.go — the DBCache implementation used by sql-server's remotesrv — previously discarded the *doltdb.DoltDB (with its registered hooks) after extracting the raw ChunkStore. It now returns a hooksFiringRemoteSrvStore wrapper instead of the bare store.
After a successful Commit(), the wrapper:- Diffs the dataset map at last vs current noms root hashes
- Fires ddb.ExecuteCommitHooks() for every ref-typed dataset (refs/heads/, refs/tags/, etc.) whose head address changed or was deleted — matching the behaviour of hooksDatabase
No changes to remotesrv, doltdb, or the gRPC layer were required.
Tests
- sqle/remotesrv_hook_test.go — unit tests covering the basic behavior of firing hooks on an incoming commit; hooks fire for successful commit, hooks do not fire on a rejected (CAS-failed) commit; hooks fire for deleted datasets (matching hooksDatabase.Delete); hooks only fire for datasets that actually changed.
- integration-tests/bats/sql-server-remotesrv.bats — end-to-end test: starts a sql-server with push-on-write replication to a file remote and --remotesapi-port, performs a dolt push from a client, and asserts the replication remote is updated.
- 10721: couple random splunk improvements
This fixes the ability ofnoms showto print index data (panicking before this change).
Also patches the splunk tool to allow more than one hash on a single output line to be examined.
Sample output:Table - { 1) Schema: #q3kfuk6atrj3b6e46ultp97f86gh9crk 2) Violations: #00000000000000000000000000000000 3) Artifacts: #00000000000000000000000000000000 Autoinc: 0 4) Primary Index (rows 1, depth 1) #ep59g4n947egeqpqfe7ou4ikjvi81rov { 5) 6) { key: 7669657700, 6d797600 value: #jcch47j2ueiiuq31iuuvvbg7ve6inccc, #53drvdrsgbvfhtnd03tlgbm1ngm72dp4, 4e4f5f454e47494e455f535542535449545554494f4e2c4f4e4c595f46554c4c5f47524f55505f42592c5354524943545f5452414e535f5441424c455300 } } Secondary Indexes (indexes 0, depth 1) o0te4bsh { } } > 5 SerialMessage - { Blob - create view myv as select * from abc } > .. Table - { 1) Schema: #q3kfuk6atrj3b6e46ultp97f86gh9crk 2) Violations: #00000000000000000000000000000000 3) Artifacts: #00000000000000000000000000000000 Autoinc: 0 4) Primary Index (rows 1, depth 1) #ep59g4n947egeqpqfe7ou4ikjvi81rov { 5) 6) { key: 7669657700, 6d797600 value: #jcch47j2ueiiuq31iuuvvbg7ve6inccc, #53drvdrsgbvfhtnd03tlgbm1ngm72dp4, 4e4f5f454e47494e455f535542535449545554494f4e2c4f4e4c595f46554c4c5f47524f55505f42592c5354524943545f5452414e535f5441424c455300 } } Secondary Indexes (indexes 0, depth 1) o0te4bsh { } } > 6 SerialMessage - { Blob - {"CreatedAt":0} } - 10719: go: statspro: Add a quiesced state.
Stats should not continuously consume CPU resources scanning database roots when it knows that stats are up to date and no intervening writes have occurred. - 10715: go: store/datas: database_common: On FastForward with working set checks, create the working set with the correct root hash if it does not already exist.
Previously, we avoided doing this because there is logic in the SQL server which automatically create the working set if it doesn't already exist. However, that logic cannot work in all cases, such as multi-branch access within a SQL transaction. It's more appropriate and more aligned with Dolt's expectations going forward to always create the working set on the branch write if it does not already exist.
Because of remotesapi.PushConcurrencyControl, this change does not change the behavior of Dolt when writing to a pure remotesapi endpoint, like dolthub.com. In that case, working sets are not typically pushed to the remote and they are not written on a branch write. - 10708: Store encoding in typeinfo, and use the serialized encoding as canonical when reading values from disk
The goal of this change is to honor the encoding of a schema column, as used by values for that column written to disk, when we deserialize it. Currently we compute the encoding for a column from its SQL type, serialize it to disk as part of the schema, then on deserialization throw it away and compute it from the SQL type again. Now, we derive it from the SQL type on first write, and honor the value serialized to disk when reading it back.
This makes it possible for different versions of Dolt, or the same version with different configuration / settings, to change how the same SQL type is serialized to disk. For 2.0, we will begin serializing BLOB and TEXT types with adaptive encoding by default. Existing columns will continue using the encoding they designated at the time of column creation.
For compatibility testing, I've expanded our existing integration testing to consider every possible SQL type and all possible DML operations on it. - 10707: Remove
ValidateTagUniquenesscheck
go-mysql-server
- 3473: fix missing check expressions in child expressions of update
- 3471: Changed some reflection checks to use type equality
reflect.DeepEqualshas issues with some types represented by integrators, so we're usingtype.Equalsin such scenarios.
Closed Issues
- 10089: Dolt does not show the correct precision for datetime
- 10706: JSON_OBJECT on datetime field doesn't represent fractional seconds
- 10698: Interactive rebase fails with 'changes in branch' when ignored tables exist in working set
Performance
| Read Tests | MySQL | Dolt | Multiple |
|---|---|---|---|
| covering_index_scan | 1.93 | 0.55 | 0.28 |
| groupby_scan | 13.46 | 9.91 | 0.74 |
| index_join | 1.5 | 1.86 | 1.24 |
| index_join_scan | 1.44 | 1.39 | 0.97 |
| index_scan | 34.33 | 21.89 | 0.64 |
| oltp_point_select | 0.2 | 0.27 | 1.35 |
| oltp_read_only | 3.82 | 5.18 | 1.36 |
| select_random_points | 0.35 | 0.53 | 1.51 |
| select_random_ranges | 0.39 | 0.56 | 1.44 |
| table_scan | 34.95 | 22.28 | 0.64 |
| types_table_scan | 77.19 | 66.84 | 0.87 |
| reads_mean_multiplier | 1.0 |
| Write Tests | MySQL | Dolt | Multiple |
|---|---|---|---|
| oltp_delete_insert | 8.28 | 6.43 | 0.78 |
| oltp_insert | 4.1 | 3.13 | 0.76 |
| oltp_read_write | 9.06 | 11.45 | 1.26 |
| oltp_update_index | 4.18 | 3.19 | 0.76 |
| oltp_update_non_index | 4.18 | 3.13 | 0.75 |
| oltp_write_only | 5.28 | 6.09 | 1.15 |
| types_delete_insert | 8.43 | 6.79 | 0.81 |
| writes_mean_multiplier | 0.9 |
| TPC-C TPS Tests | MySQL | Dolt | Multiple |
|---|---|---|---|
| tpcc-scale-factor-1 | 94.04 | 38.04 | 2.47 |
| tpcc_tps_multiplier | 2.47 |
| Overall Mean Multiple | 1.46 |
|---|
1.83.8
Merged PRs
dolt
- 10705: bug fix: don't error on ignored tables when continuing a rebase
When continuing an active rebase, Dolt checks to make sure that the branch being rebased hasn't had other changes applied before it update that branch pointer to the new rebased commits. This logic wasn't taking account of ignored tables before, which preventing users from being able to complete an interactive rebase.
Fixes: #10698 - 10704: Set timezone in Doltgres CI tests
- 10703: go: store/nbs,doltdb: When Clone into a destination database, disable Conjoin running as part of landing files into the destination.
When we land files into a destination database as part of a Clone, by default we used to be able to trigger a conjoin on the destination database. After the change in #10652, we now also block on any of those conjoin operations completing as we closed the DestDB that was created as part of a backup sync/sync-url operation.
In the case of backup, this conjoin behavior is a little contrary to what we are trying to accomplish.
This PR changes it so that we dynamically disable conjoin on the destination while we clone into it. After the clone is finished, we reenable conjoin. At that point, if the database is used as an actual database, conjoin will kick in as expected. But if it is immediately closed, the destination files stay as they were. - 10681: bug fix: binlog schema change replication to a MySQL replica
Fixes bugs with schema changes when binary logging is enabled, such as changing a column's type (e.g.VARCHARtoTEXT), adding and dropping columns. Existing type serialization logic that converts diff data into binlog messages was split into two separate phases: deserializing the data from the Dolt tuple store (using the original schema) and serializing the data into the MySQL binlog protocol (using the target schema). - 10665: cache node hash
go-mysql-server
- 3468: Separate update expressions into explicit and derived update expressions
fixes #10657
Update expressions are separated into explicit and derived. Derived update expressions are only evaluated if the explicit update expressions actually made any changes to the row.
Also removes some dead code and includes some minor fixes/refactors
Closed Issues
- 10698: Interactive rebase fails with 'changes in branch' when ignored tables exist in working set
- 10692: Server permanently unresponsive after client crash during CREATE/DROP DATABASE
- 8572: Potential TUI tool like Lazygit
Performance
| Read Tests | MySQL | Dolt | Multiple |
|---|---|---|---|
| covering_index_scan | 1.96 | 0.55 | 0.28 |
| groupby_scan | 14.46 | 9.91 | 0.69 |
| index_join | 1.52 | 1.82 | 1.2 |
| index_join_scan | 1.47 | 1.42 | 0.97 |
| index_scan | 34.33 | 22.28 | 0.65 |
| oltp_point_select | 0.2 | 0.27 | 1.35 |
| oltp_read_only | 3.82 | 5.28 | 1.38 |
| select_random_points | 0.35 | 0.53 | 1.51 |
| select_random_ranges | 0.39 | 0.55 | 1.41 |
| table_scan | 34.95 | 22.28 | 0.64 |
| types_table_scan | 75.82 | 66.84 | 0.88 |
| reads_mean_multiplier | 1.0 |
| Write Tests | MySQL | Dolt | Multiple |
|---|---|---|---|
| oltp_delete_insert | 8.28 | 6.43 | 0.78 |
| oltp_insert | 4.18 | 3.13 | 0.75 |
| oltp_read_write | 9.22 | 11.45 | 1.24 |
| oltp_update_index | 4.25 | 3.19 | 0.75 |
| oltp_update_non_index | 4.18 | 3.13 | 0.75 |
| oltp_write_only | 5.28 | 6.09 | 1.15 |
| types_delete_insert | 8.43 | 6.79 | 0.81 |
| writes_mean_multiplier | 0.89 |
| TPC-C TPS Tests | MySQL | Dolt | Multiple |
|---|---|---|---|
| tpcc-scale-factor-1 | 93.91 | 37.53 | 2.5 |
| tpcc_tps_multiplier | 2.5 |
| Overall Mean Multiple | 1.46 |
|---|
1.83.7
Merged PRs
dolt
- 10702: Special handling of comment delimiters in StreamScanner
fixes #10694 - 10700: Fix Dolt table function privilege checks when current database is revision-qualified
MySQL privilege grants are stored against the base database name, so when a user called a Dolt table function withmydb/revisionas the current database, the privilege check looked up the revision-qualfied name instead ofmydb.- Update
dolt_schema_diff,dolt_diff,dolt_diff_stat,dolt_diff_summary,dolt_log,dolt_patch,dolt_preview_merge_conflicts,dolt_preview_merge_conflicts_summary, anddolt_query_diffto strip revision qualifier before performing privilege checks.
Fix #10695
Companion dolthub/docs#2804
- Update
- 10696: go: doltcore/sqle: database_provider.go: Fix a deadlock in DROP DATABASE.
A lock-acquisition order error could cause DROP DATABASE to conflict on lock acquisition with a DoltSession's own Mutex. The result was that all sessions would then block indefinitely and irrecoverably on trying to access any database loaded in the server.
This changes the order of operations and the lock acquisition sequencing to not dead lock.
Note: finalization of DROP DATABASE with regards to sessions and ongoing queries which may be accessing the database is still somewhat lacking. To be improved in the future.
Fixes: #10692. - 10691: Have fsTablePersister implement movingTableFilePersister
ThemovingTableFilePersisterinterface is for tablePersister implementations that build a temporary table file and have the ability to move the temporary file into the final location instead of copying the contents into a new one.
#9118 added support for persisting archive files. As part of this change, the signature of a method onmovingTableFilePersisterwas changed to accept the more genericGenericTableWriterinterface instead of the more specificCmpChunkTableWriter.
Unfortunately, the signature was not also updated on thefsTablePersisterimplementation, resulting in it no longer implementing the interface. As a result, we lost the ability move table files during garbage collection. This had no impact on correctness, but potentially had a performance impact.
This PR fixes this issue. - 10690: /go/{cmd,libraries} check for valid dbs, close opened dbs on error
- 10689: /{dist,package-lock.json}: rebuild dist
- 10683: Bump undici from 6.23.0 to 6.24.0 in /.github/actions/ses-email-action
Bumps undici from 6.23.0 to 6.24.0.Release notes
Sourced from undici's releases.
v6.24.0
Undici v6.24.0 Security Release Notes (LTS)
This release backports fixes for security vulnerabilities affecting the v6 line.
Upgrade guidance
All users on v6 should upgrade to v6.24.0 or later.
Fixed advisories
-
GHSA-2mjp-6q6p-2qxm / CVE-2026-1525 (Medium)
Inconsistent interpretation of HTTP requests (request/response smuggling class issue). -
GHSA-f269-vfmq-vjvj / CVE-2026-1528 (High)
Malicious WebSocket 64-bit frame length handling could crash the client. -
GHSA-4992-7rv2-5pvq / CVE-2026-1527 (Medium)
CRLF injection via theupgradeoption. -
GHSA-v9p9-hfj2-hcw8 / CVE-2026-2229 (High)
Unhandled exception from invalidserver_max_window_bitsin WebSocket permessage-deflate negotiation. -
GHSA-vrm6-8vpv-qv8q / CVE-2026-1526 (High)
Unbounded memory consumption in WebSocket permessage-deflate decompression.
Not applicable to v6
- GHSA-phc3-fgpg-7m6h / CVE-2026-2581 affects
>= 7.17.0 < 7.24.0only.
Affected and patched ranges (v6)
- CVE-2026-1525: affected
< 6.24.0, patched6.24.0 - CVE-2026-1528: affected
>= 6.0.0 < 6.24.0, patched6.24.0 - CVE-2026-1527: affected
< 6.24.0, patched6.24.0 - CVE-2026-2229: affected
< 6.24.0, patched6.24.0 - CVE-2026-1526: affected
< 6.24.0, patched6.24.0
References
- GitHub Security Advisories: https://github.com/nodejs/undici/security/advisories
- NVD CVE-2026-1525: https://nvd.nist.gov/vuln/detail/CVE-2026-1525
- NVD CVE-2026-1528: https://nvd.nist.gov/vuln/detail/CVE-2026-1528
- NVD CVE-2026-1527: https://nvd.nist.gov/vuln/detail/CVE-2026-1527
- NVD CVE-2026-2229: https://nvd.nist.gov/vuln/detail/CVE-2026-2229
- NVD CVE-2026-1526: https://nvd.nist.gov/vuln/detail/CVE-2026-1526
Commits
8873c94Bumped v6.24.0411bd01test(websocket): use node:assert for Node 18 compatibility844bf59test: fix http2 lint regressions in backporta444e4ftest: stabilize h2 and tls-cert-leak under current test runnerdc032a1fix: h2 CI (#4395)4cd3f4btest: increase bitness intest/fixtures/*.pem(#3659)7df6442fix: adapt websocket frame-limit handling for v6 parser4e0179afix: reject duplicate content-length and host headers5a97f08Fix websocket 64-bit length overflowe43e898fix: validate upgrade header to prevent CRLF injection- Additional commits viewable in compare view
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/dolthub/dolt/network/alerts)... -
1.83.6
Merged PRs
dolt
- 10679: remove archive sanity check
For sanity, we recalculate the hash for chunks and compare it to the one we expect from the archive.
If we are confident that it works, we can speed up AutoGC (at least according to pprofs). - 10673: go: remotesrv: getUploadUrl: Respect X-Forwarded-Proto so returned URLs work appropriately behind tls-terminating reverse proxies.
Adopts the same behavior as download URLs already have. Upload URLs were missed in the original fix.
Fixes #10672. - 10551: ssh transport support
This change adds a new command,dolt transferwhich is enables ssh remotes. It's similar to git upload-pack, in that it is intended to be executed by SSH with a few parameters in order to perform clone,push, and fetch operations.
Dolt's native replication is over a combination of gRPC and HTTP operations.dolt transferuses SMUX to perform those transports over a bi-directional byte stream. We use the stdin / stdout of the SSH connection as that stream.
Fixes: #1027
go-mysql-server
- 3468: Separate update expressions into explicit and derived update expressions
fixes #10657
Update expressions are separated into explicit and derived. Derived update expressions are only evaluated if the explicit update expressions actually made any changes to the row.
Also removes some dead code and includes some minor fixes/refactors - 3467: add sql.Databaser to alter default and alter index
- 3458: Move prepared queries to
sql.Sessionand add parsed query cache
This PR moves prepared statements into thesql.Sessioninstead of the engine. This avoids the need for a mutex and cleans up the code. It's also just a more appropriate place forprepareandexecutequeries.
Additionally, this PR also adds a cache for triggers. This avoids reparsing triggers that have already been parsed.
Closed Issues
- 10657:
ON_UPDATEcolumns should only update if other column values have actually changed - 10648: Panic during binlog replication
- 10655: Mobile View
- 10677: mysql_native_password plugin not found when connecting with MySQL 9.x client
- 10672: getUploadUrl ignores X-Forwarded-Proto — push fails behind TLS reverse proxy
- 1027: SSH Support for connecting to remotes
Performance
| Read Tests | MySQL | Dolt | Multiple |
|---|---|---|---|
| covering_index_scan | 1.96 | 0.55 | 0.28 |
| groupby_scan | 13.22 | 9.91 | 0.75 |
| index_join | 1.52 | 1.82 | 1.2 |
| index_join_scan | 1.47 | 1.39 | 0.95 |
| index_scan | 34.33 | 22.28 | 0.65 |
| oltp_point_select | 0.2 | 0.27 | 1.35 |
| oltp_read_only | 3.82 | 5.18 | 1.36 |
| select_random_points | 0.35 | 0.53 | 1.51 |
| select_random_ranges | 0.39 | 0.55 | 1.41 |
| table_scan | 34.95 | 22.28 | 0.64 |
| types_table_scan | 77.19 | 66.84 | 0.87 |
| reads_mean_multiplier | 1.0 |
| Write Tests | MySQL | Dolt | Multiple |
|---|---|---|---|
| oltp_delete_insert | 8.43 | 6.43 | 0.76 |
| oltp_insert | 4.18 | 3.13 | 0.75 |
| oltp_read_write | 9.06 | 11.24 | 1.24 |
| oltp_update_index | 4.25 | 3.19 | 0.75 |
| oltp_update_non_index | 4.18 | 3.13 | 0.75 |
| oltp_write_only | 5.28 | 6.09 | 1.15 |
| types_delete_insert | 8.58 | 6.79 | 0.79 |
| writes_mean_multiplier | 0.88 |
| TPC-C TPS Tests | MySQL | Dolt | Multiple |
|---|---|---|---|
| tpcc-scale-factor-1 | 93.4 | 37.76 | 2.47 |
| tpcc_tps_multiplier | 2.47 |
| Overall Mean Multiple | 1.45 |
|---|
1.83.5
Merged PRs
dolt
- 10669: fix concurrent create database if exist atomicity
- 10668: /.github/workflows/cd-push-docker-image.yaml: option for disabling la…
…test push
go-mysql-server
- 3466: fix create db if not exists and drop db if exists to not return errors under concurrency
- 3465: support IF EXISTS in ALTER TABLE
- 3463: remove hacky handling of trigger parsing
Companion PR: dolthub/vitess#459 - 3461: Update
ON UPDATEcolumns duringINSERT...ON DUPLICATE KEY UPDATE
fixes #10627 - 3458: Move prepared queries to
sql.Sessionand add parsed query cache
This PR moves prepared statements into thesql.Sessioninstead of the engine. This avoids the need for a mutex and cleans up the code. It's also just a more appropriate place forprepareandexecutequeries.
Additionally, this PR also adds a cache for triggers. This avoids reparsing triggers that have already been parsed. - 3455: Do not convert AntiJoins to LeftOuterJoin when inside an Update node
fixes #10600
vitess
- 459: fix trigger parsing
Should useold_lexer_positioninstead oflexer_positionto avoid token getting swallowed.