Elgato_dark/data-peek
1
0
Fork 0
mirror of https://github.com/Rohithgilla12/data-peek synced 2026-04-21 12:57:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

content(blog): add posts for v0.21.0 features

Browse source 
Covers Smart Sort Bar (multi-column chip sorting), multi-statement
step-through debugger, and Schema Intel diagnostics surface.
This commit is contained in:
Rohith Gilla 2026-04-20 11:26:11 +05:30
parent 4d12860414
commit 40e7b446a8
No known key found for this signature in database
4 changed files with 527 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
notes/README.md
View file

@ -14,6 +14,9 @@ This folder is the single source of truth for technical notes and blog posts. Fi
| [listen-notify-without-tears.mdx](./listen-notify-without-tears.mdx) | Postgres LISTEN/NOTIFY debugger with SQLite history |
| [benchmark-mode-p50-p90-p99.mdx](./benchmark-mode-p50-p90-p99.mdx) | Benchmark mode with p90/p95/p99 percentiles |
| [fk-aware-fake-data-generator.mdx](./fk-aware-fake-data-generator.mdx) | FK-aware fake data generator with Faker.js |
| [smart-sort-bar.mdx](./smart-sort-bar.mdx) | Multi-column chip-based sorting with type-aware modes |
| [multi-statement-step-through.mdx](./multi-statement-step-through.mdx) | Step-through debugging for multi-statement SQL scripts |
| [schema-intel-diagnostics.mdx](./schema-intel-diagnostics.mdx) | One-click schema diagnostics for Postgres / MySQL / MSSQL |
## Creating a New Post

184
notes/multi-statement-step-through.mdx Normal file
View file

@ -0,0 +1,184 @@
---
title: "Step-Through Debugging for SQL, Because `BEGIN; -- nope` Is Not a Workflow"
description: "Running a multi-statement SQL script is an all-or-nothing gamble. data-peek's step-through executor pauses between statements, keeps the connection open across pauses, and lets you inspect, skip, or abort — without manually pasting statements one by one."
date: "2026-04-20"
author: "Rohith Gilla"
tags: ["sql", "postgres", "database", "productivity"]
published: true
---
Here is how I used to run a migration script in a SQL client:
1. Paste the whole thing
2. Wrap it in `BEGIN;` … `ROLLBACK;`
3. Run it
4. It errors on statement 7
5. Panic briefly
6. Delete statements 8 through 20
7. Re-run to see statement 7's error in context
8. Fix statement 7
9. Restore statements 8 through 20
10. Hope nothing depended on statement 7 actually completing
This is not a workflow. It is a sequence of regrets. And it is what every
SQL client forces you into, because the only two modes on offer are
"run everything" and "highlight the one line I want."
v0.21 adds a third mode: **step-through**.
## What it does
Highlight a script. Hit `Cmd+Shift+Enter` (or click the **Step** button).
data-peek parses it into individual statements, opens a database
connection, and runs the first one. Then it stops.
A **ribbon** appears above the editor:
- Current statement index, e.g. `3/12`
- Elapsed time
- Controls: **Next** · **Skip** · **Continue to end** · **Stop**
- Breakpoint gutter in the editor margin
You inspect the result. You look at a side table to confirm the row
count is what you expected. You check a temp table you created in
statement 2. Then you hit `Shift+Enter` to advance. Or `Escape` to stop
and roll back the whole thing.
The connection stays open across pauses. That is the important detail —
everything you did in statement 2 (temp tables, session variables, an
open transaction) is still there when statement 3 runs.
## The state machine that makes it work
Every step-through session is an instance of `StepSessionRegistry`,
which holds an open DB client and a state:
```ts
type StepState =
| { kind: 'paused'; cursorIndex: number; lastResult: ... }
| { kind: 'running'; cursorIndex: number }
| { kind: 'errored'; cursorIndex: number; error: ... }
| { kind: 'done'; cursorIndex: number }
```
User actions (`next`, `skip`, `continue`, `retry`, `stop`) are
transitions between these states. The registry lives in the Electron
main process, so the React side is just sending IPC messages and
rendering whatever state the server reports.
This matters because the alternative — parsing statements in the
renderer, sending each one as a separate `query()` call — would open a
new connection for every statement. Temp tables would evaporate.
`SET LOCAL` would not persist. The whole point of step-through would
collapse.
### Why a state machine and not a loop
I started with a loop. Something like:
```ts
for (const stmt of statements) {
const result = await runOne(stmt)
await waitForUserToHitNext()
}
```
This is clean until you realise that "wait for user to hit Next" means
the loop is blocked for possibly hours while the user goes to lunch.
During that time the client is holding a connection, the event loop is
fine but the React side cannot introspect what is happening, and if the
user crashes the renderer the main process has no idea.
Explicit state on the main side, IPC events for transitions, and a
Zustand store in the renderer that mirrors the server's state. The
renderer never holds the canonical state. If the renderer crashes and
reopens, it asks the main process what the state is and resumes.
## Breakpoints
The editor margin has a gutter for breakpoints, same as any IDE. Click
a line to set one. **Continue** runs until the next breakpoint or the
end of the script.
Breakpoints are how you scale this from "I want to inspect every step"
to "I want to inspect step 7 and step 12, the rest is fine." Which is
most of what I actually want when running a 30-statement migration.
## Monaco keybindings that don't fight you
There is a subtle bug you run into when you try to give `Shift+Enter`
meaning in Monaco: Monaco already has a `Shift+Enter` handler (insert
a newline without autocomplete), and a window-level keydown listener
will not win.
The fix is registering the shortcut as a Monaco action:
```ts
editor.addAction({
id: 'data-peek.step.next',
label: 'Step: Next statement',
keybindings: [monaco.KeyMod.Shift | monaco.KeyCode.Enter],
precondition: 'data-peek.step.active',
run: () => stepNext(),
})
```
The `precondition` context is set only while a step session is running,
so Monaco's built-in `Shift+Enter` behaves normally the rest of the
time. Same pattern for `Escape` (stop) and `Cmd+Shift+Enter` (start).
## The counter drift bug
Early versions had a fun bug where the ribbon would show `0/3` after
running the first statement, then `2/3`, then `3/3`. Off-by-one on
the first step only.
The cause was the renderer computing `cursorIndex = response.index + 1`
to display "1-indexed" position — and getting confused about whether
the response represented "the statement I just ran" or "the statement
I am about to run." Different code paths disagreed.
The fix was making the server authoritative. Every step response
(`NextStepResponse`, `SkipStepResponse`, `ContinueStepResponse`,
`RetryStepResponse`) now includes a `cursorIndex` field, and the
renderer displays whatever the server sent. No client-side math. No
drift.
## Pinned results
If statement 3 returns 12 rows you want to remember while you inspect
statement 7's output, click **Pin** on that result tab. It sticks
around in the tab strip for the rest of the session.
This is the step-through equivalent of having five terminal tabs open
while debugging — one for the diagnostic query, one for the current
state, one for the rollback-safety check. All in the same window, all
tied to one open connection.
## Why not just use `psql` in single-step mode
`psql` has `\set ON_ERROR_STOP on` and you can paste a script and it
will halt on the first error. That is fine for scripted use. It is not
fine for iterative debugging because you cannot *continue past* the
error without rerunning everything, and you cannot inspect between
statements that succeed — `psql` races to the next prompt.
The step-through model is closer to a debugger than to a script
runner. The unit of execution is one statement; the default action
between statements is "stop and wait."
## What you use this for
The obvious use: running a migration by hand on staging before you
commit it. Catch the one statement that assumes a column exists that
you dropped in step 2.
Less obvious: walking a junior engineer through a data repair. You
highlight a block, hit Step, they watch each query run and see the
intermediate state before the next one modifies it. It is the SQL
equivalent of screen-sharing a debugger session.
Least obvious but most useful: **dry-running stored procedures** that
got written as "just a script" and never wrapped in a function. These
are the queries nobody trusts to run unattended. Now you do not have
to.

176
notes/schema-intel-diagnostics.mdx Normal file
View file

@ -0,0 +1,176 @@
---
title: "One-Click Diagnostics for the Schema Problems You Already Have"
description: "Every Postgres database accumulates schema debt — tables without primary keys, foreign keys without indexes, duplicate indexes, bloat. data-peek's Schema Intel runs the diagnostic queries you keep meaning to write, surfaces the findings, and gives you copyable SQL fixes."
date: "2026-04-20"
author: "Rohith Gilla"
tags: ["postgres", "database", "performance", "devops"]
published: true
---
Every Postgres database older than six months has some version of the
same problems:
- A table someone created without a primary key
- A foreign key column nobody indexed
- Two indexes that do the same thing because a migration forgot to drop
the old one
- A table that has not been vacuumed since the bicentennial
- A FK that is nullable when nobody remembers why
You know the diagnostic queries exist. They live in a Notion page
somewhere. You copy them once a quarter when the database is slow. You
mean to automate this. You do not automate this.
v0.21 ships **Schema Intel** — a one-click surface that runs the
queries, groups the findings, and hands you the fix.
## What it checks
Click **Schema Intel** in the sidebar's Automation & Monitoring group.
It runs a configurable set of read-only diagnostics against the active
connection and returns a findings list. Each finding has:
- A **title** (`Table has no primary key`)
- A **detail** (`public.audit_log`)
- A **severity** (info, warn, critical)
- Where possible, a **suggested SQL fix** you can copy or open in a new
query tab
The check set on Postgres covers:
1. **Tables without a primary key** — anything in `pg_class` where
`relkind = 'r'` and no `indisprimary` row exists in `pg_index`
2. **Missing foreign key indexes** — FK columns with no index that
starts with that column, which turns every parent DELETE into a
sequential scan
3. **Duplicate indexes** — indexes with identical column lists and
expressions
4. **Unused indexes** — `pg_stat_user_indexes.idx_scan = 0` over a
threshold period (skip if the stats were recently reset)
5. **Invalid indexes** — `indisvalid = false`, usually a failed
`CREATE INDEX CONCURRENTLY`
6. **Bloated tables** — estimated bloat ratio above a threshold, using
the [pgstattuple][bloat] heuristic
7. **Never-vacuumed tables** — `pg_stat_user_tables.last_vacuum` is
null and `last_autovacuum` is older than 30 days
8. **Nullable foreign keys** — FK column with `NOT NULL` missing,
often unintentional
On MySQL the checks run over `information_schema` (core set). On MSSQL
it is currently just tables-without-PK. SQLite returns skipped entries.
[bloat]: https://www.postgresql.org/docs/current/pgstattuple.html
## Why "read-only" is load-bearing
Every query in Schema Intel is a pure `SELECT` against system catalogs
or `information_schema`. It never touches user data. It never runs
`ANALYZE`. It never acquires a lock stronger than `AccessShareLock`.
This matters because the natural audience for this feature is "person
who is nervous about their database right now." If running the
diagnostic has any chance of making things worse, nobody will click
the button. So the button is safe by construction.
## Permission failures do not hide
If your role cannot read `pg_stat_user_indexes`, the "unused indexes"
check cannot run. The question is what to do about that.
Early versions wrapped each check in `try/catch`, logged the error,
and showed a green checkmark. This was terrible — you would look at
the Schema Intel tab, see everything passing, and conclude your
database was healthy when in fact half the checks had silently
skipped.
The current behaviour: each check reports a status of `ok`,
`findings`, `skipped`, or `error`. Skipped and errored checks show up
in the results list with a grey badge and the reason:
```
Unused indexes — skipped
Permission denied on pg_stat_user_indexes.
Grant pg_read_all_stats to this role, or run as superuser.
```
Absence of evidence is not evidence of absence. The UI is now explicit
about that.
## Copyable fixes
When a finding has a mechanical fix, Schema Intel ships the SQL with
it. "Table has no primary key" on a table with a clear candidate
column suggests:
```sql
ALTER TABLE public.audit_log
ADD CONSTRAINT audit_log_pkey PRIMARY KEY (id);
```
"Missing foreign key index" suggests:
```sql
CREATE INDEX CONCURRENTLY idx_orders_customer_id
ON public.orders (customer_id);
```
Note the `CONCURRENTLY` — any suggestion that creates an index in a
table scan does it concurrently by default, because the user hitting
this button on production should not pay an AccessExclusiveLock for
the privilege.
You can copy the SQL to the clipboard, or click **Open in new tab** to
drop it into a query editor where you can edit the name, add `IF NOT
EXISTS`, and run it when you are ready.
## Why not `pgtune` / `pgbadger` / insert-tool-here
Those tools exist and they are good. `pgtune` tunes configuration.
`pgbadger` analyses logs. `pganalyze` is a whole commercial product.
Schema Intel is explicitly not that. It is the lightweight version
that lives inside a SQL client you already have open, runs in under a
second against the connection you already have, and tells you the
thing you already suspected. "Yes, `audit_log` has no primary key."
"Yes, that FK is unindexed." "Yes, that index has not been used in
six weeks."
No agent to install. No log shipping. No separate dashboard. No
monthly subscription. One button, a list, and copyable SQL.
## The checks are pluggable
Internally, each check is an object:
```ts
interface SchemaIntelCheck {
id: string
title: string
description: string
severity: 'info' | 'warn' | 'critical'
dialects: Array<'postgres' | 'mysql' | 'mssql' | 'sqlite'>
run(adapter: DatabaseAdapter): Promise<SchemaIntelResult>
}
```
Adding a new check is a file in `src/main/schema-intel/checks/`. It
registers itself, declares which dialects it supports, and returns
findings. The orchestrator runs the checks in parallel, collects the
results, and ships them over IPC.
This is the bit I am most excited about — the set of things people
want to know about their database is open-ended, and most of the
useful checks are 20 lines of SQL plus a couple of lines of
scaffolding. If you have a check you run often, a PR adds it for
everyone.
## What I use it for
Every new connection, first thing: click Schema Intel. See what the
database thinks of itself. Usually there are a few findings — a
missing index on an FK that nobody noticed, a duplicate index from a
migration, an `audit_log` table that never got a PK.
Twenty minutes with Schema Intel is the cheapest database
performance audit you will ever do. Most of the findings were already
true yesterday. You just did not know them.

164
notes/smart-sort-bar.mdx Normal file
View file

@ -0,0 +1,164 @@
---
title: "Sorting by Three Columns Shouldn't Require a CASE Statement"
description: "Every SQL client lets you click a column header to sort. Almost none let you sort by three columns at once without dropping into SQL. Here's how data-peek's Smart Sort Bar adds type-aware, chip-based multi-sort on top of any result set — without touching the query."
date: "2026-04-20"
author: "Rohith Gilla"
tags: ["sql", "database", "webdev", "ux"]
published: true
---
I have a results grid open. 40,000 rows. I want to see the most recent
orders per customer, grouped by region, with the largest amounts on top.
In almost every SQL client I have used, the path to that view is:
1. Click the column header — sorts by that one column
2. Realise I actually need three columns in the sort
3. Rewrite the query with a multi-column `ORDER BY`
4. Re-run it
5. Discover one of the columns needs `NULLS LAST`
6. Rewrite again
The client had the data the whole time. It just did not give me a way to
reshape the view without round-tripping through the database.
## The Smart Sort Bar
data-peek v0.21 ships a **Smart Sort Bar** above every result set. It is
parallel to the Smart Filter Bar and works the same way: each active sort
is a chip, chips have settings, and the bar is the truth.
Click a column header: sort by that column. Shift-click another: append
it. Shift-click a third: append that too. The chips render in the bar in
priority order. Drag to reorder, or use `Option/Cmd + ←/→` from the
keyboard.
`Cmd+Shift+S` focuses the bar.
That is the entire interaction. No SQL rewrite, no round-trip, no
`ORDER BY customer_id, region, amount DESC` typed out by hand.
## Every chip is a little state machine
Clicking a header once gives you ascending. Click again for descending.
Click again to remove. This is the standard TanStack cycle and it works
fine — but the chip also has a gear that opens a few more knobs:
- **Mode** — how the values should be compared
- **Nulls** — first or last
- **Direction** — ascending or descending
- **Drag handle** — move the chip in the priority order
The modes are where it gets interesting.
### Type-aware modes
Most grids sort by coercing everything to a string, which is why you end
up with `10` appearing between `1` and `2`. We fixed that, then went
further: each chip's **mode** decides how its column gets compared.
```ts
type SortMode =
| 'natural' // default, dispatches on declared column type
| 'length' // sort strings by character length
| 'absolute' // sort numbers by magnitude, ignoring sign
| 'byMonth' // pull the month out of a date and sort 1..12
| 'byDayOfWeek'// pull the weekday out of a date
| 'byTime' // pull the time-of-day out of a timestamp
| 'random' // seeded shuffle
```
`natural` is the important one. It dispatches **strictly by declared
column type** — a `TEXT` column is compared as a string, a `NUMERIC`
column as a number, a `TIMESTAMP` as a date. No cross-type sniffing that
silently reorders string columns as numbers because the first few
happened to look numeric. That class of "why is `email` sorted weird"
bug is gone.
The other modes are for when `ORDER BY` is not expressive enough without
a `CASE` statement. Sorting logs by time-of-day across different dates.
Finding the biggest absolute variances in a balance column. Grouping
birthday records by month. Things I used to do with ad-hoc SQL, now a
dropdown on the chip.
### Random, done properly
There is also a `random` mode. It uses a seeded [mulberry32][mulberry]
stream per chip, so the shuffle is **reproducible** — you see the same
order when you come back. A dice icon rerolls the seed.
`SortChip` is a discriminated union in TypeScript, so random chips
require a seed at compile time:
```ts
type SortChip =
| { id: string; mode: 'random'; seed: number; columnId: string; ... }
| { id: string; mode: Exclude<SortMode, 'random'>; columnId: string; ... }
```
If the seed is missing, the code does not compile. A small thing, but it
is the difference between "why did my random order change on reload" and
"this just works."
[mulberry]: https://github.com/bryc/code/blob/master/jshash/PRNGs.md
## Null handling that actually asks
Every chip has a `nulls: 'first' | 'last'` toggle. Postgres lets you
specify this in `ORDER BY`; most grid libraries silently pick one. We
expose it on the chip because the right answer depends on what you are
looking for — nulls-first if you are hunting down records that are
missing a value, nulls-last if you are looking at the real data.
Invalid dates in `byMonth`, `byDayOfWeek`, and `byTime` route through
the same nulls-position path. No phantom bucket `-1` quietly appearing
at the top.
## Pagination resets on chip changes
Small thing, but: if you are on page 40 of a 200-row-per-page table and
you add a sort that shrinks the visible set to 12 rows, you used to
stare at an empty page and wonder what broke. Sort-chip and
filter-chip changes now reset pagination to page 1.
Same for filter chips. It is a two-line fix in
`data-table.tsx` that removes about ten "is this a bug" support
questions.
## Why a bar and not a menu
The obvious alternative is a "Sort by..." menu that opens a panel, you
pick columns, hit apply. Every SQL client has one. Nobody uses it,
because the first click already gave them what they wanted and the
menu is the thing you open when you need the second sort — by which
time you have forgotten the first was even there.
Keeping every active sort visible as a chip means the state is never
hidden. You can see what is active, you can tweak it in place, and
`Escape` (or clicking the X on a chip) removes it. It is the same
reason the filter bar works the way it does.
## Testing the invariants
The sort layer has 35 unit tests covering:
- Each comparator path (natural per type, length, absolute, byMonth,
byDayOfWeek, byTime)
- Null handling in both directions
- Stable-sort invariant when two rows compare equal
- `toggleColumnSort` cycle semantics (asc → desc → removed)
Sort is one of those "obvious" features that breaks in subtle ways on
real data. An empty string sorting before or after `null`. A `BIGINT`
column with a value that loses precision as a JS number. Dates stored
as strings. The tests are the only reason I trust this code.
## What's next
The chip model is going to be the thing data-peek uses for any "filter
or shape the displayed rows" action. Filters already use it. Sort now
uses it. Grouping is the next candidate. Eventually the bar above a
result set should be a single row of chips that fully describes the
view — copyable, shareable, persistable to a saved query.
For now: `Cmd+Shift+S`, add three chips, done.