Skip to content

Commit 7d8b950

Browse files
committed
Feat(editor): a Markdown table converts into a database, linked from the note (#832)
A note often starts with a small pipe table that outgrows inline editing: the moment you want a sort, a filter or a board, the only route was New Database, then retyping or exporting the rows into it, and the note lost track of where its data went. #832 asked for a conversion that keeps you in the keyboard-first flow. Put the cursor in a table and run Convert Table to Database… from the command palette (:table_to_database on the ex line, Space l d in Vim mode, or the bottom of the table cell menu, right-click or m). A prompt takes the name, prefilled from the heading above the table or the note's title. ZenNotes creates a <Name>.base database where new databases go and swaps the table for a [[Name]] wikilink, the same link the [[ picker inserts for a database, so the note still points at its data and the grid is a gd or a click away. The swap is an ordinary edit: autosaved by the pane, one u away, and the database stays behind when you undo. A toast confirms with an Open button. What goes in the database is decided once, in shared-domain. createDatabase takes an optional seed (headers, rows, column widths) across the bridge, and initialDatabaseContents builds the sidecar and rows from it through the same inferFields that adopts a plain CSV, so a converted table gets exactly the schema its data would get if it had been saved as a CSV and opened: number, checkbox and date columns typed, blank and duplicate headers renamed, an existing unique id column adopted and a hidden one synthesized otherwise. Both IO twins (desktop main's databases.ts and the IO-abstracted database-ops the web bridge, the CLI and remote workspaces use) build from it, so the files stay byte-compatible across transports. Cells travel verbatim, inline markdown included; the CSV layer does the RFC 4180 quoting. Widths from a zen:cols marker become field widths. Alignment is dropped, a grid has none. Names that a wikilink cannot carry ([ ] # ^ | and the slashes) are refused up front rather than sanitized into a link that resolves to nothing. Finding "the table under the cursor" needed one new piece. When tables render as widgets the caret cannot be inside one (the range is atomic), and by the time a palette command runs, focus has already left the cell for the palette input. cm-table now remembers the widget that last took focus, per editor, and forgets it as soon as focus lands back in the note text, where the caret is the better authority again. Raw tables (Split mode, live tables off) resolve from the caret through the syntax tree with the #485 line-scan fallback. After the prompt and the write, the note is touched only if this editor still shows the same note and the table's source is byte-for-byte where it was; otherwise the database exists and the table stays, with a toast saying so. Also fixed on the way, in Editor.tsx: registerCommandPaletteEx ran at Editor mount and used buildCommands(), which drops every command whose `when` says no at that instant. No pane had a view or a note yet, so the whole editor-scoped family (:editor_reflow_paragraph, :task_forward and the rest) was never an ex command; the built app answered "Not an editor command". It now registers with includeUnavailable, and runCommand keeps re-checking `when` when the name is typed. Tests: initialDatabaseContents (empty grid, typed seed keyed by field id and round-tripped through the CSV, adopted id column, header renames and short rows, widths), the seeded path in both IO twins on disk and in the memory vault, parseTableBlock with a zen:cols marker, tableUnderCursor for the caret and for a focused widget across a focus change, the heading default (a # inside a code fence is not a heading), the name validation, and the whole conversion in a real EditorView: prompt prefill, the seed handed to the store, the deduped title in the link, undo, the Open toast, cancel, no table under the cursor, a table that moved while the prompt was up, and a refused creation. npm run typecheck and npm run test:run clean. Verified over CDP in the built app with both stores isolated, 51 checks in three configurations: Vim with rendered tables (the m cell menu, Space l d with a colliding name landing as [[Roadmap 2]], u and Ctrl+R, gd opening the grid), Vim with raw tables (:table_to_database, :cmd, the outside-a-table toast) and Vim off (right-click). data.csv and schema.json asserted on disk each time.
1 parent 9184b48 commit 7d8b950

26 files changed

Lines changed: 1082 additions & 59 deletions

‎apps/desktop/src/main/database-domain.test.ts‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ import {
55
parseRows,
66
serializeRows,
77
inferFields,
8+
initialDatabaseContents,
89
buildDefaultViews,
910
type GenId
1011
} from '@shared/database-csv'
@@ -142,6 +143,98 @@ describe('inferFields', () => {
142143
})
143144
})
144145

146+
describe('initialDatabaseContents', () => {
147+
it('without a seed is the empty id + Name grid', () => {
148+
const { sidecar, rows } = initialDatabaseContents(undefined, counterGenId())
149+
expect(rows).toEqual([])
150+
expect(sidecar.fields.map((f) => [f.name, f.type, f.hidden ?? false])).toEqual([
151+
['id', 'text', true],
152+
['Name', 'text', false]
153+
])
154+
expect(sidecar.idFieldId).toBe(sidecar.fields[0].id)
155+
expect(sidecar.views[0].hiddenFieldIds).toEqual([sidecar.idFieldId])
156+
expect(serializeRows(rows, sidecar.fields)).toBe('id,Name\n')
157+
})
158+
159+
it('types a seeded table like an adopted CSV and keys rows by field id (#832)', () => {
160+
const { sidecar, rows } = initialDatabaseContents(
161+
{
162+
headers: ['Task', 'Done', 'Due', 'Hours'],
163+
rows: [
164+
['Write **draft**', 'x', '2026-03-01', '2'],
165+
['Ship, then rest', '', '2026-03-09', '0.5']
166+
]
167+
},
168+
counterGenId()
169+
)
170+
const byName = new Map(sidecar.fields.map((f) => [f.name, f]))
171+
// A hidden id field is synthesized in front of the table's own columns.
172+
expect(sidecar.fields[0].name).toBe('id')
173+
expect(sidecar.fields[0].id).toBe(sidecar.idFieldId)
174+
expect(byName.get('Task')!.type).toBe('text')
175+
expect(byName.get('Done')!.type).toBe('checkbox')
176+
expect(byName.get('Due')!.type).toBe('date')
177+
expect(byName.get('Hours')!.type).toBe('number')
178+
expect(rows).toHaveLength(2)
179+
for (const row of rows) {
180+
expect(row.id).toBeTruthy()
181+
expect(row.cells[sidecar.idFieldId]).toBe(row.id)
182+
}
183+
expect(rows[0].cells[byName.get('Task')!.id]).toBe('Write **draft**')
184+
expect(rows[1].cells[byName.get('Done')!.id]).toBe('')
185+
// What lands in data.csv: RFC 4180 quoting for the comma, markdown kept verbatim.
186+
const csv = serializeRows(rows, sidecar.fields)
187+
expect(csv.split('\n')[0]).toBe('id,Task,Done,Due,Hours')
188+
expect(csv).toContain(',"Ship, then rest",,2026-03-09,0.5\n')
189+
expect(parseRows(csv, sidecar.fields, sidecar.idFieldId)).toEqual(rows)
190+
})
191+
192+
it('adopts an all-unique id column from the seed instead of synthesizing one', () => {
193+
const { sidecar, rows } = initialDatabaseContents(
194+
{ headers: ['id', 'Name'], rows: [['a1', 'Alpha'], ['b2', 'Beta']] },
195+
counterGenId()
196+
)
197+
expect(sidecar.fields.map((f) => f.name)).toEqual(['id', 'Name'])
198+
expect(sidecar.fields[0].hidden).toBe(true)
199+
expect(rows.map((r) => r.id)).toEqual(['a1', 'b2'])
200+
})
201+
202+
it('renames blank and duplicate headers and pads short rows', () => {
203+
const { sidecar, rows } = initialDatabaseContents(
204+
{ headers: ['', 'Name', 'Name'], rows: [['only one cell']] },
205+
counterGenId()
206+
)
207+
expect(sidecar.fields.map((f) => f.name)).toEqual(['id', 'Column 1', 'Name', 'Name (2)'])
208+
const [, c1, n1, n2] = sidecar.fields
209+
expect(rows[0].cells[c1.id]).toBe('only one cell')
210+
expect(rows[0].cells[n1.id]).toBe('')
211+
expect(rows[0].cells[n2.id]).toBe('')
212+
})
213+
214+
it('carries positive column widths onto the matching fields and drops the rest', () => {
215+
const { sidecar } = initialDatabaseContents(
216+
{
217+
headers: ['A', 'B', 'C'],
218+
rows: [],
219+
columnWidths: [120, null, -4]
220+
},
221+
counterGenId()
222+
)
223+
const widths = sidecar.fields.map((f) => f.width)
224+
expect(widths).toEqual([undefined, 120, undefined, undefined])
225+
})
226+
227+
it('a header-only seed makes an empty database with those fields', () => {
228+
const { sidecar, rows } = initialDatabaseContents(
229+
{ headers: ['Title', 'Notes'], rows: [] },
230+
counterGenId()
231+
)
232+
expect(rows).toEqual([])
233+
expect(sidecar.fields.map((f) => f.name)).toEqual(['id', 'Title', 'Notes'])
234+
expect(sidecar.fields.slice(1).every((f) => f.type === 'text')).toBe(true)
235+
})
236+
})
237+
145238
describe('transforms', () => {
146239
const fields: DbField[] = [
147240
{ id: 'name', name: 'Name', type: 'text' },

‎apps/desktop/src/main/databases.test.ts‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,6 +53,38 @@ describe('createDatabase + readDatabase', () => {
5353
const b = await createDatabase(root, 'inbox', '', 'Notes')
5454
expect(a.path).not.toBe(b.path)
5555
})
56+
57+
it('writes a seeded database with the seed rows and inferred types (#832)', async () => {
58+
const root = await makeVault()
59+
const doc = await createDatabase(root, 'inbox', 'Work', 'Roadmap', {
60+
headers: ['Item', 'Done', 'Estimate'],
61+
rows: [
62+
['Write "spec", v1', 'yes', '3'],
63+
['Ship', 'no', '1']
64+
],
65+
columnWidths: [240, null, null]
66+
})
67+
expect(doc.path).toBe('inbox/Work/Roadmap.base/data.csv')
68+
expect(doc.fields.map((f) => [f.name, f.type])).toEqual([
69+
['id', 'text'],
70+
['Item', 'text'],
71+
['Done', 'checkbox'],
72+
['Estimate', 'number']
73+
])
74+
expect(doc.fields[1].width).toBe(240)
75+
expect(doc.rows).toHaveLength(2)
76+
77+
const csv = await readFile(path.join(root, doc.path), 'utf8')
78+
const lines = csv.trimEnd().split('\n')
79+
expect(lines[0]).toBe('id,Item,Done,Estimate')
80+
expect(lines[1]).toBe(`${doc.rows[0].id},"Write ""spec"", v1",yes,3`)
81+
expect(lines[2]).toBe(`${doc.rows[1].id},Ship,no,1`)
82+
83+
// Reopening through the sidecar yields the same rows and ids.
84+
const reopened = await readDatabase(root, doc.path)
85+
expect(reopened.rows).toEqual(doc.rows)
86+
expect(reopened.fields).toEqual(doc.fields)
87+
})
5688
})
5789

5890
describe('writeDatabaseRows round-trip', () => {

‎apps/desktop/src/main/databases.ts‎

Lines changed: 12 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import path from 'node:path'
1010
import { randomUUID } from 'node:crypto'
1111
import {
1212
inferFields,
13+
initialDatabaseContents,
1314
buildDefaultViews,
1415
parseCsv,
1516
parseRows,
@@ -23,6 +24,7 @@ import {
2324
formTitleFromCsvPath,
2425
isFormDirName,
2526
type DatabaseDoc,
27+
type DatabaseSeed,
2628
type DatabaseSidecar,
2729
type DatabaseSummary,
2830
type DbField,
@@ -250,15 +252,18 @@ export async function writeDatabaseSchema(
250252
}
251253

252254
/**
253-
* Create a new empty database (`id` + `Name` fields) under `folder`/`subpath`
254-
* and return it hydrated. Uses `folderRoot` so a root-mode vault creates at the
255-
* vault root rather than inventing an `inbox/` directory.
255+
* Create a new database under `folder`/`subpath` and return it hydrated: empty
256+
* (`id` + `Name` fields) by default, or holding the `seed`'s columns and rows
257+
* when one is given (a Markdown table converted in place, #832). Uses
258+
* `folderRoot` so a root-mode vault creates at the vault root rather than
259+
* inventing an `inbox/` directory.
256260
*/
257261
export async function createDatabase(
258262
root: string,
259263
folder: NoteFolder,
260264
subpath: string,
261-
title?: string
265+
title?: string,
266+
seed?: DatabaseSeed
262267
): Promise<DatabaseDoc> {
263268
const safeTitle = (title ?? 'Untitled Database').trim() || 'Untitled Database'
264269
const baseName = safeTitle.replace(/[\\/:*?"<>|]/g, '-')
@@ -281,22 +286,12 @@ export async function createDatabase(
281286
}
282287
const rel = csvPathForFormDir(formDirRel)
283288

284-
const idField: DbField = { id: randomUUID(), name: 'id', type: 'text', hidden: true }
285-
const nameField: DbField = { id: randomUUID(), name: 'Name', type: 'text' }
286-
const fields = [idField, nameField]
287-
const { views, activeViewId } = buildDefaultViews(fields)
288-
const sidecar: DatabaseSidecar = {
289-
version: 1,
290-
idFieldId: idField.id,
291-
fields,
292-
views,
293-
activeViewId
294-
}
289+
const { sidecar, rows } = initialDatabaseContents(seed, randomUUID)
295290
// Create the folder, then the two data files.
296291
await fs.mkdir(databaseDataPath(root, formDirRel), { recursive: true })
297292
await persistSidecar(root, rel, sidecar)
298-
await writeFileAtomic(databaseDataPath(root, rel), serializeRows([], fields))
299-
return hydrate(rel, sidecar, [])
293+
await writeFileAtomic(databaseDataPath(root, rel), serializeRows(rows, sidecar.fields))
294+
return hydrate(rel, sidecar, rows)
300295
}
301296

302297
/**

‎apps/desktop/src/main/index.ts‎

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -210,7 +210,11 @@ import {
210210
type DatabaseOps as SharedDatabaseOps,
211211
} from "@shared/database-ops";
212212
import { createAbsenceAwareReader } from "@shared/remote-absence";
213-
import type { DatabaseSidecar, DbRow } from "@shared/databases";
213+
import type {
214+
DatabaseSeed,
215+
DatabaseSidecar,
216+
DbRow,
217+
} from "@shared/databases";
214218
import { VaultWatcher } from "./watcher";
215219
import { WindowVaultRegistry } from "./window-vaults";
216220
import { registerEphemeralRoot, isEphemeralRoot } from "./ephemeral-vaults";
@@ -3809,15 +3813,28 @@ function registerIpc(): void {
38093813

38103814
handle(
38113815
IPC.VAULT_CREATE_DATABASE,
3812-
async (_e, folder: NoteFolder, subpath: string, title?: string) => {
3816+
async (
3817+
_e,
3818+
folder: NoteFolder,
3819+
subpath: string,
3820+
title?: string,
3821+
seed?: DatabaseSeed,
3822+
) => {
38133823
if (isRemoteWorkspaceActive()) {
38143824
return await databaseOpsForRemote().createDatabase(
38153825
folder,
38163826
subpath,
38173827
title,
3828+
seed,
38183829
);
38193830
}
3820-
return await createDatabase(requireVault().root, folder, subpath, title);
3831+
return await createDatabase(
3832+
requireVault().root,
3833+
folder,
3834+
subpath,
3835+
title,
3836+
seed,
3837+
);
38213838
},
38223839
);
38233840

‎apps/desktop/src/preload/index.ts‎

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,13 @@ import type {
9393
VaultTextSearchToolPaths
9494
} from '@shared/ipc'
9595
import type { VaultTask } from '@shared/tasks'
96-
import type { DatabaseDoc, DatabaseSidecar, DatabaseSummary, DbRow } from '@shared/databases'
96+
import type {
97+
DatabaseDoc,
98+
DatabaseSeed,
99+
DatabaseSidecar,
100+
DatabaseSummary,
101+
DbRow
102+
} from '@shared/databases'
97103
import type {
98104
McpClientId,
99105
McpClientStatus,
@@ -520,8 +526,13 @@ const api: ZenBridge = {
520526
rows: DbRow[]
521527
): Promise<DatabaseDoc> =>
522528
ipcRenderer.invoke(IPC.VAULT_WRITE_DATABASE_SCHEMA, relPath, sidecar, rows),
523-
createDatabase: (folder: NoteFolder, subpath: string, title?: string): Promise<DatabaseDoc> =>
524-
ipcRenderer.invoke(IPC.VAULT_CREATE_DATABASE, folder, subpath, title),
529+
createDatabase: (
530+
folder: NoteFolder,
531+
subpath: string,
532+
title?: string,
533+
seed?: DatabaseSeed
534+
): Promise<DatabaseDoc> =>
535+
ipcRenderer.invoke(IPC.VAULT_CREATE_DATABASE, folder, subpath, title, seed),
525536
renameDatabase: (csvPath: string, newTitle: string): Promise<string> =>
526537
ipcRenderer.invoke(IPC.VAULT_RENAME_DATABASE, csvPath, newTitle),
527538
createRecordPage: (csvPath: string, title: string, body: string): Promise<string> =>

‎packages/app-core/src/browse-actions.test.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ async function setup() {
1919
const create = vi.fn(async () => {})
2020
const rename = vi.fn(async () => {})
2121
const remove = vi.fn(async () => {})
22-
const createDatabase = vi.fn(async () => {})
22+
const createDatabase = vi.fn(async () => undefined)
2323
const renameDatabase = vi.fn(async () => {})
2424
useStore.setState({
2525
vault: { root: '/test', name: 'Test' },

‎packages/app-core/src/components/Editor.tsx‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1420,7 +1420,13 @@ function registerCommandPaletteEx(): void {
14201420
};
14211421

14221422
const names = new Set<string>(MANUAL_EX_NAMES);
1423-
for (const cmd of buildCommands()) {
1423+
// Register every command, gated or not. This runs when the Editor mounts,
1424+
// before any pane has a view or a note, and buildCommands() drops every
1425+
// command whose `when` says no at that instant, which is the whole
1426+
// editor-scoped family (:editor_reflow_paragraph, :task_forward, and the
1427+
// rest) for the life of the window. `runCommand` re-checks `when` when the
1428+
// name is actually typed, so nothing runs out of context.
1429+
for (const cmd of buildCommands({ includeUnavailable: true })) {
14241430
const name = commandIdToExName(cmd.id);
14251431
if (names.has(name)) continue;
14261432
names.add(name);

‎packages/app-core/src/components/VimNav.tsx‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ import {
3939
sequenceTokenFromEvent
4040
} from '../lib/keymaps'
4141
import { toggleWrap, wrapLink } from '../lib/cm-format'
42+
import { convertTableToDatabase } from '../lib/table-to-database'
4243
import {
4344
ZEN_OPEN_EDITOR_CONTEXT_MENU_EVENT,
4445
dispatchKeyboardContextMenu,
@@ -257,6 +258,11 @@ export function VimNav(): JSX.Element | null {
257258
keyLabel: getKeymapDisplay(keymapOverrides, 'vim.leaderToggleFavorite'),
258259
label: 'Toggle favorite',
259260
detail: 'Add or remove the active note from Favorites.'
261+
},
262+
{
263+
keyLabel: getKeymapDisplay(keymapOverrides, 'vim.leaderTableToDatabase'),
264+
label: 'Table to database',
265+
detail: 'Turn the table under the cursor into a database, linked from the note.'
260266
}
261267
]
262268
}
@@ -1070,6 +1076,13 @@ export function VimNav(): JSX.Element | null {
10701076
void state.toggleFavoriteActiveNote()
10711077
return
10721078
}
1079+
if (matchesSequenceToken(e, overrides, 'vim.leaderTableToDatabase') && editorNormalMode) {
1080+
e.preventDefault()
1081+
e.stopImmediatePropagation()
1082+
resetLeader()
1083+
if (state.editorViewRef) void convertTableToDatabase(state.editorViewRef)
1084+
return
1085+
}
10731086
resetLeader()
10741087
}
10751088

‎packages/app-core/src/lib/cm-table-menu.ts‎

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,10 @@ export interface TableMenuRequest {
2828
col: number
2929
model: MarkdownTable
3030
apply: (next: MarkdownTable, focus?: { row: number; col: number }) => void
31+
/** Turn the whole table into a `.base` database and leave a link in its
32+
* place (#832). Unlike `apply`, this leaves the table model alone: the
33+
* converter reads the committed document itself. Omitted = no such item. */
34+
convertToDatabase?: () => void
3135
}
3236

3337
type MenuItem =
@@ -43,7 +47,7 @@ export function closeTableContextMenu(): void {
4347

4448
export function openTableContextMenu(req: TableMenuRequest): void {
4549
closeTableContextMenu()
46-
const { row, col, model, apply } = req
50+
const { row, col, model, apply, convertToDatabase } = req
4751
// Restore focus to whatever opened the menu (e.g. a table cell) on close,
4852
// unless an action ran — that focuses its own target cell.
4953
const previouslyFocused = document.activeElement as HTMLElement | null
@@ -142,7 +146,13 @@ export function openTableContextMenu(req: TableMenuRequest): void {
142146
kind: 'item',
143147
label: 'Sort column (Z → A)',
144148
run: () => apply(sortByColumn(model, col, 'desc'))
145-
}
149+
},
150+
...(convertToDatabase
151+
? [
152+
{ kind: 'sep' } as const,
153+
{ kind: 'item', label: 'Convert to database…', run: convertToDatabase } as const
154+
]
155+
: [])
146156
]
147157

148158
const menu = document.createElement('div')

0 commit comments

Comments
 (0)