data-peek/docs/features.md
Rohith Gilla a168975867
feat: add 6 new features — stats, masking, CSV import, data gen, notifications, health monitor (#119)
* docs: add design spec for 6 new features

Column Statistics, Data Masking, CSV Import, Data Generator,
DB Notifications, and Connection Health Monitor.

* docs: update spec with review fixes

Resolve all 15 review issues: naming conflicts, preload layer,
batch insert interface, guard clauses, confirmation dialogs, etc.

* docs: add implementation plan for 6 features

Fix critical issues from review: adapter connection pattern,
preload event subscription pattern, tab interface definitions,
ColumnStatsRequest type, and implementation notes.

* feat: add column statistics panel (Tasks 1.1-1.8)

Implement the Column Statistics feature allowing users to right-click a
column header and view min/max/nulls/distinct/histogram in a right sidebar
panel. Supports PostgreSQL (full implementation with median + histogram),
MySQL (CASE WHEN histogram, no median), and MSSQL (NTILE histogram +
PERCENTILE_CONT median). Includes shared types, IPC handler, Zustand
store, stats panel UI, and DataTable integration via onColumnStatsClick.

* feat: add Data Masking feature (renderer-only)

Adds blur-based data masking so users can hide sensitive columns during
demos, screenshots, and pair programming. Entirely renderer-only — no
IPC or main process changes.

- masking-store: Zustand store with manual per-tab masking, auto-mask
  rules (email/password/ssn/token etc.), hover-to-peek, and persist
  middleware for rules/settings only (runtime maskedColumns not persisted)
- masking-toolbar: dropdown UI for toggling auto-mask, managing rules,
  adding custom regex patterns, and unmasking all columns for a tab
- data-table: accepts optional tabId, shows lock icon on masked headers,
  adds Mask/Unmask to column dropdown, applies blur(5px) + userSelect:none
  to masked cells with alt-hover peek support
- editable-data-table: same masking UX; masked cells block editing
- tab-query-editor: wires tabId into DataTable, renders MaskingToolbar
  near Export, intercepts exports when masked columns are present and
  shows an AlertDialog to confirm exporting with [MASKED] values

* feat(csv-import): implement CSV import feature (Tasks 3.1-3.5)

- Add shared types: CsvColumnMapping, CsvImportOptions, CsvImportRequest,
  CsvImportProgress, CsvImportResult, BatchInsertOptions, BatchInsertResult
- Install papaparse for CSV parsing in renderer
- Create batch-insert.ts: parameterized INSERT with per-dialect placeholder
  syntax (pg: $1/$2, mysql: ?, mssql: @p1/@p2), ON CONFLICT handling, and
  cancellation support
- Add import-handlers.ts IPC handler for db:import-csv and db:import-cancel;
  progress events pushed via event.sender.send('db:import-progress')
- Add import-store.ts Zustand store with 5-step wizard state, type inference
  for new tables (integer/numeric/timestamp/boolean/text), and auto-mapping
- Create csv-import-dialog.tsx Sheet dialog: file drop zone, target selection,
  column mapping, options (batch size/conflict/truncate/transaction), and
  progress/result step
- Add "Import CSV" to schema explorer table context menu (both virtualized
  and non-virtualized paths)

* feat: add Data Generator feature (Tasks 4.1-4.5)

Adds schema-aware fake data generation using @faker-js/faker. Users can
open a data-generator tab from the schema explorer context menu on any
table, configure per-column generators (faker, random-int/float/bool/date,
uuid, auto-increment, fk-reference, fixed, null), preview 5 sample rows,
and insert up to 10 000 rows in configurable batches with live progress.

- packages/shared: GeneratorType, ColumnGenerator, DataGenConfig,
  DataGenProgress, DataGenResult types
- main: data-generator.ts engine with heuristics + faker dispatch
- main: data-gen-handlers.ts IPC for generate, cancel, preview
- preload + preload d.ts: generateData, cancelGenerate, generatePreview,
  onGenerateProgress on window.api.db
- stores: DataGeneratorTab type added to tab-store; data-gen-store for
  per-tab generator state and actions
- renderer: data-generator.tsx UI; schema-explorer "Generate Data" menu
  item; tab-query-editor dispatch for data-generator tab type

* feat: add PostgreSQL LISTEN/NOTIFY dashboard (Chunk 5)

Implements a full LISTEN/NOTIFY panel for PostgreSQL connections:

- Shared types: PgNotificationEvent, PgNotificationChannel, PgNotificationStats
- Main process: dedicated pg.Client listener per connection with SSH tunnel
  support, auto-reconnect with exponential backoff, and SQLite event history
  (capped at 10k events per connection)
- IPC handlers under pg-notify: prefix; cleanup on app quit
- Preload: pgNotify namespace exposing subscribe/unsubscribe/send/getChannels/
  getHistory/clearHistory/onEvent
- Store: usePgNotificationStore with rolling 60s stats and in-memory event cap
- Tab type: pg-notifications added to TabType union with deduplication
- UI: channel subscription bar, scrollable event log with JSON highlighting,
  collapsible send panel and stats; PostgreSQL-only guard
- Sidebar: Notifications entry visible only for PostgreSQL connections

* feat(shared): add health monitor types (Task 6.1)

Add ActiveQuery, TableSizeInfo, CacheStats, LockInfo, and
DatabaseSizeInfo types to the shared package for IPC contract.

* feat(adapters): implement health monitor queries (Task 6.2)

Add getActiveQueries, getTableSizes, getCacheStats, getLocks, and
killQuery to DatabaseAdapter interface with implementations for
PostgreSQL, MySQL, MSSQL, and SQLite stub.

* feat(ipc): add health monitor IPC handlers (Task 6.3)

Register IPC handlers for db:active-queries, db:table-sizes,
db:cache-stats, db:locks, and db:kill-query channels.

* feat(preload): expose health monitor API to renderer (Task 6.4)

Add health namespace to preload bridge with activeQueries, tableSizes,
cacheStats, locks, and killQuery methods plus type declarations.

* feat(store): add health monitor Zustand store (Task 6.5)

Create useHealthStore with polling, fetch actions for all health
metrics, and kill query support.

* feat(ui): add health monitor dashboard tab (Task 6.6)

Add HealthMonitor component with 4-panel dashboard: active queries
with kill support, table sizes with sorting, cache hit ratios with
color coding, and lock/blocking detection. Integrate into tab system
and sidebar navigation.

* chore: apply prettier formatting

* fix: resolve empty block statement lint errors

* fix: add column stats to editable table and fix masking infinite loop

Column statistics dropdown now available in table preview (editable)
mode. Fixed MaskingToolbar infinite re-render caused by creating a new
empty array reference on every store snapshot.

* docs: update README, features, roadmap, and marketing site for 6 new features

Add Column Statistics, Data Masking, CSV Import, Data Generator,
Connection Health Monitor, and PostgreSQL Notifications to all
documentation and the marketing website features grid.

* Apply suggestions from code review

Co-authored-by: tembo[bot] <208362400+tembo[bot]@users.noreply.github.com>

---------

Co-authored-by: tembo[bot] <208362400+tembo[bot]@users.noreply.github.com>
2026-03-18 08:18:55 +05:30

434 lines
18 KiB
Markdown

# data-peek Feature Overview
> A minimal, fast, beautiful PostgreSQL client for developers who want to quickly peek at their data — now with AI-powered database exploration.
---
## Product Summary
**data-peek** is a lightweight desktop database client designed for developers who need quick, frictionless access to their databases. Unlike bloated alternatives like pgAdmin or DBeaver, data-peek focuses on speed, simplicity, and a keyboard-first experience — now with AI-powered assistance.
**Target Audience:** Developers, data engineers, backend engineers, and anyone who needs to quickly query and explore databases without the overhead of enterprise tools.
**Supported Databases:** PostgreSQL, MySQL (SQLite coming soon)
**Platforms:** macOS (Apple Silicon + Intel), Windows, Linux
---
## Key Value Propositions
| Benefit | Description |
|---------|-------------|
| **Lightning Fast** | Opens in under 2 seconds. No splash screens, no waiting. |
| **AI-Powered** | Ask questions in natural language, get SQL queries and charts. |
| **Zero Configuration** | Connect and query immediately. No complex setup required. |
| **Keyboard-First** | Power users can do everything without touching the mouse. |
| **Beautiful & Modern** | Dark and light themes with a clean, distraction-free UI. |
| **Privacy-First** | No telemetry, no tracking. Your data stays on your machine. |
| **Secure** | Connection credentials are encrypted locally. |
| **Pay Once, Own Forever** | No subscriptions. One-time purchase with 1 year of updates. |
---
## Pricing
### Free Tier
Get started at no cost:
- 2 database connections
- 50 query history items
- 3 editor tabs
- 1 schema for ER diagrams
- CSV/JSON export
### Pro License — ~~$99~~ $29 (Early Bird)
Unlock everything:
- **Unlimited** connections
- **Unlimited** query history
- **Unlimited** tabs
- **Unlimited** ER diagrams
- Inline data editing (INSERT/UPDATE/DELETE)
- Query execution plans (EXPLAIN/ANALYZE)
- 3 device activations
- 1 year of updates
- **Pay once, use forever**
### Cloud (Coming Soon)
For power users and teams:
- Everything in Pro
- Sync connections across devices
- Cloud-saved queries
- Team sharing
- ~$5-8/month
---
## Feature List
### Connection Management
| Feature | Description |
|---------|-------------|
| **Quick Connection Setup** | Add connections with host, port, database, user, and password — or paste a connection string |
| **Connection String Parsing** | Paste any PostgreSQL connection URL and auto-fill all fields |
| **Test Before Save** | Verify connections work before adding them |
| **Encrypted Storage** | Credentials stored securely with encryption |
| **SSL Support** | Connect to SSL-enabled databases |
| **Connection Switcher** | Quickly switch between multiple database connections |
| **Edit & Delete** | Manage saved connections with ease |
### Query Editor
| Feature | Description |
|---------|-------------|
| **Monaco Editor** | Same editor engine that powers VS Code |
| **SQL Syntax Highlighting** | Full SQL syntax highlighting with PostgreSQL support |
| **Smart Autocomplete** | Schema-aware suggestions for tables, columns, and SQL keywords |
| **Multi-Tab Support** | Work on multiple queries simultaneously with independent tabs |
| **Query Formatting** | Auto-format SQL with `Cmd/Ctrl + Shift + F` |
| **Run Query** | Execute with `Cmd/Ctrl + Enter` |
| **Collapsible Editor** | Minimize the editor to focus on results |
### Results Viewer
| Feature | Description |
|---------|-------------|
| **Data Table View** | View results in a clean, sortable table |
| **Data Type Indicators** | Color-coded badges showing column types |
| **Query Metrics** | See row count and query execution time |
| **Pagination** | Navigate large result sets with customizable page sizes |
| **Copy Cell** | Click any cell to copy its value |
| **Copy Row as JSON** | Export individual rows as JSON objects |
| **Export to CSV** | Download results as CSV files |
| **Export to JSON** | Download results as JSON files |
| **NULL Styling** | Clear visual distinction for NULL values |
| **Foreign Key Navigation** | Click FK cells to view related records |
| **JSON Tree Viewer** | Expand and inspect JSON with collapsible tree view |
| **JSON Syntax Highlighting** | Color-coded values by type (strings, numbers, booleans) |
| **JSON Preview** | Partial content preview in cell with tooltip |
### Schema Explorer
| Feature | Description |
|---------|-------------|
| **Tree View Navigation** | Browse schemas, tables, and views hierarchically |
| **Column Details** | See column names, data types, and constraints |
| **Primary Key Indicators** | Visual markers for primary key columns |
| **Nullable Indicators** | See which columns allow NULL values |
| **Foreign Key Display** | View foreign key relationships |
| **Table Search** | Filter tables by name with instant search |
| **Click to Query** | Click any table to generate a SELECT query |
| **Schema Refresh** | Reload schema after database changes |
### Query History
| Feature | Description |
|---------|-------------|
| **Auto-Save** | Every executed query is automatically saved |
| **Persistent Storage** | History survives app restarts |
| **Query Metadata** | See execution time, row count, and status for each query |
| **Quick Load** | Click any history item to load it into the editor |
| **Copy to Clipboard** | Copy previous queries without loading |
| **Clear History** | Remove all or individual history items |
| **Query Type Badges** | Visual indicators for SELECT, INSERT, UPDATE, DELETE |
| **Relative Timestamps** | "5 minutes ago", "yesterday", etc. |
### AI Assistant (New in v0.2.0)
| Feature | Description |
|---------|-------------|
| **Natural Language Queries** | Ask questions in plain English, get SQL queries |
| **Multi-Provider Support** | Choose OpenAI, Anthropic, Google, Groq, or local Ollama |
| **Schema Awareness** | AI understands your tables, columns, and relationships |
| **Inline Execution** | Run AI-generated queries and see results in chat |
| **Chart Generation** | Create bar, line, pie, and area charts from data |
| **Metric Cards** | Display formatted single-value insights |
| **Chat Persistence** | Conversations saved per connection |
| **Session Management** | Create, rename, and delete chat sessions |
| **Privacy Options** | Use local Ollama models for complete data privacy |
**Supported AI Providers:**
| Provider | Best For | Key Features |
|----------|----------|--------------|
| OpenAI | General use | GPT-5, GPT-4o models |
| Anthropic | SQL generation | Claude Opus 4.5, Sonnet 4 |
| Google | Balanced | Gemini 3 Pro, 2.5 |
| Groq | Speed | Ultra-fast inference |
| Ollama | Privacy | Local models, no API key needed |
### Column Statistics
| Feature | Description |
|---------|-------------|
| **One-Click Profiling** | Click the chart icon on any column header to open statistics |
| **Base Stats** | Row count, null count, null percentage, distinct count for every column |
| **Numeric Stats** | Min, max, average, standard deviation, median, histogram distribution |
| **Text Stats** | Min/max/average length, top 5 most common values |
| **DateTime Stats** | Earliest, latest, and date range |
| **Boolean Stats** | True/false counts and percentages |
### Data Masking
| Feature | Description |
|---------|-------------|
| **Blur Masking** | Blur sensitive column values with CSS filter |
| **Auto-Mask Rules** | Regex-based rules to automatically mask columns matching patterns (email, password, SSN, etc.) |
| **Hover to Peek** | Hold Alt and hover to temporarily reveal masked values |
| **Per-Tab State** | Each tab maintains its own set of masked columns |
| **Export Safety** | Warning dialog when exporting data with masked columns |
| **Persistent Rules** | Auto-mask rules persist across sessions |
### CSV Import
| Feature | Description |
|---------|-------------|
| **File Picker** | Drag-and-drop or file picker for CSV files |
| **Auto Column Mapping** | Automatically maps CSV columns to table columns by name |
| **Type Inference** | Detects data types from CSV content |
| **Conflict Handling** | Skip, update, or error on duplicate key conflicts |
| **Batch Insert** | Configurable batch size for large imports |
| **Progress Tracking** | Real-time progress bar with row count |
| **Cancel Support** | Cancel long-running imports mid-operation |
### Data Generator
| Feature | Description |
|---------|-------------|
| **Faker.js Integration** | Generate realistic fake data (names, emails, addresses, etc.) |
| **Heuristic Detection** | Auto-detects appropriate generator type from column names |
| **FK-Aware** | Samples existing foreign key values to maintain referential integrity |
| **Preview Mode** | Preview generated rows before inserting |
| **Configurable Row Count** | Generate 1 to 100,000+ rows |
| **Dedicated Tab** | Full-featured tab UI with per-column generator configuration |
### Inline Data Editing
| Feature | Description |
|---------|-------------|
| **Click-to-Edit Cells** | Single click to start editing (fast workflow) |
| **Enum Dropdowns** | Dropdown selectors for PostgreSQL enum columns |
| **JSON Cell Editor** | Sheet-based editor with live preview |
| **Add Rows** | Insert new records with a visual form |
| **Delete Rows** | Remove records with confirmation |
| **SQL Preview** | Review generated SQL before executing changes |
| **Batch Operations** | Queue multiple changes before committing |
| **Discard Changes** | Undo pending edits before saving |
| **Type-Safe Editing** | Input validation based on column data types |
### ER Diagram Visualization
| Feature | Description |
|---------|-------------|
| **Visual Schema Map** | See your database structure as an interactive diagram |
| **Table Nodes** | Each table displays all columns with types |
| **Relationship Lines** | Foreign key connections visualized as links |
| **Primary Key Highlights** | Yellow indicators for PK columns |
| **Foreign Key Highlights** | Blue indicators for FK columns |
| **Smart Auto-Layout** | Automatic positioning prevents overlapping tables |
| **Pan & Zoom** | Navigate large schemas with ease |
| **Mini Map** | Overview navigation for complex databases |
### Query Execution Plans
| Feature | Description |
|---------|-------------|
| **EXPLAIN Visualization** | See query execution plans in a visual tree |
| **Node Type Coloring** | Color-coded operations (scans, joins, sorts) |
| **Cost Analysis** | View estimated vs actual costs |
| **Performance Metrics** | Execution time breakdown by operation |
| **Buffer Statistics** | I/O and memory usage details |
| **Expandable Nodes** | Drill into plan details |
### Query Performance Indicator (PostgreSQL)
| Feature | Description |
|---------|-------------|
| **Missing Index Detection** | Identifies sequential scans on large tables that would benefit from indexes |
| **N+1 Pattern Detection** | Analyzes query history to flag repeated similar queries within time windows |
| **Slow Query Analysis** | Flags queries exceeding configurable threshold (default: 1 second) |
| **High Filter Ratio Detection** | Identifies queries scanning many rows to return few |
| **Row Estimate Mismatch** | Highlights planner vs actual row count variance |
| **Auto-Generated Index Suggestions** | Copy-ready `CREATE INDEX CONCURRENTLY` statements |
| **Severity Filtering** | Filter issues by critical, warning, or info severity |
| **Query Fingerprinting** | Normalizes queries to detect patterns across history |
### Database Monitoring
| Feature | Description |
|---------|-------------|
| **Connection Health Monitor** | Dashboard with 4 cards: active queries, table sizes, cache stats, locks |
| **Active Query Viewer** | See all running queries with PID, user, duration, and state |
| **Kill Queries** | Terminate long-running or blocking queries with one click |
| **Table Size Analysis** | View row counts, data size, index size, and total size per table |
| **Cache Hit Ratios** | Buffer cache and index hit ratios with color-coded thresholds |
| **Lock Detection** | See blocking locks with blocked/blocking PID, lock type, and wait time |
| **Auto-Refresh** | Configurable refresh interval (2s/5s/10s/30s/off) |
| **PostgreSQL Notifications** | Subscribe to LISTEN/NOTIFY channels |
| **Real-Time Events** | See notification events arrive in real-time |
| **Send Notifications** | Publish NOTIFY messages to channels from the UI |
| **Event History** | Persistent event log with SQLite storage |
### User Interface
| Feature | Description |
|---------|-------------|
| **Dark Mode** | Easy on the eyes for long coding sessions |
| **Light Mode** | Clean, bright interface when you prefer it |
| **System Preference** | Automatically match your OS theme |
| **Resizable Panels** | Drag to resize sidebar and editor |
| **Collapsible Sidebar** | Maximize workspace when needed |
| **Collapsible Query Editor** | Hide/show query panel to focus on results |
| **Settings Modal** | Configure preferences via UI (persisted) |
| **Loading States** | Clear feedback during operations |
| **Error Handling** | Helpful error messages with details |
| **Empty States** | Guided prompts when there's no data |
### Settings
| Setting | Description |
|---------|-------------|
| **Hide Query Editor by Default** | Start table previews with editor collapsed |
| **Expand JSON by Default** | Auto-expand JSON objects in viewer |
### Saved Queries Library (New in v0.2.0)
| Feature | Description |
|---------|-------------|
| **Save Queries** | Bookmark any query with name and description |
| **Folders/Tags** | Organize queries into categories |
| **Quick Access** | Search and filter saved queries |
| **Usage Tracking** | See how often each query is used |
| **Run Directly** | Execute saved queries with one click |
| **Copy to Editor** | Load queries into current tab |
### Command Palette (New in v0.2.0)
| Feature | Description |
|---------|-------------|
| **Quick Access** | `Cmd/Ctrl + K` to open |
| **Fuzzy Search** | Find any command by typing |
| **Grouped Commands** | Organized by category |
| **Keyboard Navigation** | Arrow keys + Enter to execute |
### Keyboard Shortcuts
| Shortcut | Action |
|----------|--------|
| `Cmd/Ctrl + Enter` | Execute query |
| `Cmd/Ctrl + K` | Open command palette |
| `Cmd/Ctrl + B` | Toggle sidebar |
| `Cmd/Ctrl + Shift + F` | Format SQL |
| `Cmd/Ctrl + P` | Open connection picker |
| `Cmd/Ctrl + T` | New tab |
| `Cmd/Ctrl + W` | Close tab |
| `Cmd/Ctrl + 1-9` | Switch to tab 1-9 |
| `Cmd/Ctrl + Shift + 1-9` | Switch between connections |
| `Cmd/Ctrl + I` | Toggle AI assistant |
#### Data Editing Shortcuts
| Shortcut | Action |
|----------|--------|
| `Cmd/Ctrl + S` | Save pending changes |
| `Cmd/Ctrl + Shift + Z` | Discard all changes |
| `Cmd/Ctrl + Shift + I` | Add new row |
| `Escape` | Exit edit mode |
| `Cmd/Ctrl + Click` on FK | Open foreign key in new tab |
See the full [Keyboard Shortcuts Guide](/docs/keyboard-shortcuts.md) for all available shortcuts.
---
## Technical Highlights
| Aspect | Details |
|--------|---------|
| **Framework** | Electron with React 19 |
| **Editor** | Monaco (VS Code engine) |
| **AI Integration** | Vercel AI SDK with structured output |
| **Database Driver** | Native PostgreSQL (pg), MySQL (mysql2) |
| **Charts** | Recharts for AI-generated visualizations |
| **Local Storage** | SQLite for history and settings |
| **Security** | Encrypted credential storage |
| **Build Targets** | macOS DMG, Windows exe/msi, Linux AppImage/tar.gz |
---
## What data-peek is NOT
To set clear expectations:
- **Not a database admin tool** — Focus is on querying and exploring, not server management
- **Not a data migration tool** — CSV import is supported, but not full database migration
- **Limited multi-database** — PostgreSQL and MySQL supported (SQLite coming soon)
- **Not enterprise software** — Built for individual developers (team features coming with Cloud tier)
---
## Comparison with Alternatives
| Feature | data-peek | pgAdmin | DBeaver | TablePlus |
|---------|-----------|---------|---------|-----------|
| Startup Time | < 2s | 5-10s | 10-15s | 2-3s |
| Memory Usage | Low | High | Very High | Low |
| Learning Curve | Minimal | Steep | Steep | Minimal |
| Price | Free + $29 Pro | Free | Free/Paid | $69 |
| PostgreSQL Focus | Yes | Yes | No | No |
| **AI Assistant** | **Yes** | No | No | No |
| ER Diagrams | Yes | Yes | Yes | Yes |
| Inline Editing | Yes | Yes | Yes | Yes |
| Query Plans | Yes | Yes | Yes | Limited |
| Modern UI | Yes | No | No | Yes |
---
## Coming Soon
Features planned for future releases:
- SQLite support
- Connection groups/folders
- More AI visualization types (scatter, heatmap)
- AI query explanations and optimization suggestions
- **Cloud Sync** Sync connections and saved queries across devices
- **Team Features** Share queries and connections with your team
---
## Screenshots
*[Add screenshots here]*
- Connection dialog
- Query editor with results
- Schema explorer tree
- ER diagram view
- Query execution plan
- Inline data editing
- AI Assistant chat panel
- AI-generated charts
- Settings modal
- Dark/Light theme comparison
---
## One-Liner Descriptions
For various marketing contexts:
**Tagline:**
> Peek at your data. Fast. With AI.
**Short (10 words):**
> A fast, AI-powered database client for developers who value simplicity.
**Medium (25 words):**
> data-peek is a lightweight database client with AI assistance, modern UI, and features like ER diagrams and query plans — without the bloat.
**Long (50 words):**
> data-peek is the database client developers actually want to use. AI-powered natural language queries, lightning-fast startup, Monaco-powered SQL editor, visual ER diagrams, query execution plans, inline data editing, and a beautiful dark/light UI. No telemetry, no subscriptions, no bloat. Pay once, own forever. Available for macOS, Windows, and Linux.
---
*Document updated: March 2026 (v0.15.0)*