Elgato_dark/waveterm
1
0
Fork 0
mirror of https://github.com/wavetermdev/waveterm synced 2026-05-23 16:58:30 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

feat(terminal): improve input handling for interactive CLI tools (#2523)

Browse source 
## Summary

This PR includes three terminal input improvements that enhance the
experience when using interactive CLI tools and IME input:

1. **Shift+Enter newline support**: Enable Shift+Enter to insert
newlines by default
2. **Image paste support**: Allow pasting images by saving them as
temporary files and pasting the file path
3. **IME duplicate input fix**: Fix duplicate text when switching input
methods during composition

## Motivation

### Shift+Enter for newlines
Currently, pressing Shift+Enter in the terminal behaves the same as
Enter, making it difficult to input multi-line commands or text in
interactive CLI tools. This change enables Shift+Enter to insert
newlines by default, matching common terminal emulator behavior.

### Image paste support
Interactive AI tools like Claude Code support receiving images through
file paths, but Wave Terminal currently doesn't support pasting images.
This change implements image paste functionality similar to iTerm2's
behavior: when an image is pasted, it's saved to a temporary file and
the path is pasted into the terminal.

### IME duplicate input fix
When using Chinese/Japanese/Korean IME in the terminal, switching input
methods with Capslock during composition causes the composed text to be
sent twice, resulting in duplicate output (e.g., "你好" becomes "你好你好").
This issue severely impacts users who frequently switch between
languages.

## Changes

### Shift+Enter newline (`frontend/app/view/term/term-model.ts`)
- Change default `shiftenternewline` config from `false` to `true`
- Send standard newline character (`\n`) instead of escape sequence
(`\^[\n`)

### Image paste (`frontend/app/view/term/term-model.ts`,
`frontend/app/view/term/termwrap.ts`)
- Add `handlePaste()` method to intercept Cmd+Shift+V paste events
- Add `handleImagePasteBlob()` to save images to `/tmp` and paste the
file path
- Detect image data in clipboard using both
`ClipboardEvent.clipboardData` and Clipboard API
- Support both screenshot paste and file copy scenarios
- Add 5MB size limit for pasted images
- Temporary files are created with format:
`waveterm_paste_[timestamp].[ext]`

### IME duplicate input fix (`frontend/app/view/term/termwrap.ts`,
`frontend/app/view/term/term-model.ts`)

**IME Composition Handling:**
- Track composition state (isComposing, composingData, etc.) in TermWrap
- Register compositionstart/update/end event listeners on xterm.js
textarea
- Block all data sends during composition, only allow after
compositionend
- Prevents xterm.js from sending intermediate data during
compositionupdate phase

**Deduplication Logic:**
- Implement 50ms time window deduplication for both IME and paste
operations
- Track first send after composition, block duplicate sends from
Capslock switching
- Ensure Ctrl+Space and Fn switching work correctly (single send only)

**Edge Case Handling:**
- Add blur event handler to reset composition state on focus loss
- Add Escape key handling to cancel composition in progress

## Testing

### Shift+Enter
1. Open a terminal in Wave
2. Press Shift+Enter
3. Verify that a newline is inserted instead of executing the command

### Image paste
1. Take a screenshot and copy it to clipboard (or copy an image file in
Finder)
2. In a terminal running Claude Code, paste the image (Cmd+V or
Cmd+Shift+V)
3. Verify that the image path appears and Claude Code recognizes it

### IME Input Testing

**IME Input:**
- [x] macOS Zhuyin IME + Capslock switching - no duplicate output ✅
- [x] macOS Zhuyin IME + Ctrl+Space switching - normal single input ✅
- [x] macOS Zhuyin IME + Fn switching - normal single input ✅

**Regression Testing:**
- [x] English keyboard input - normal operation ✅
- [x] Shift+Enter multiline input - works correctly ✅
- [x] Text paste (Cmd+Shift+V) - no duplicates ✅
- [x] Image paste - works correctly ✅
- [x] Basic command execution (ls, echo, etc.) - normal ✅
- [x] Cmd+K clear terminal - works correctly ✅
- [x] Copy selected text (Cmd+Shift+C) - works correctly ✅

## Demo


https://github.com/user-attachments/assets/8341cdf9-6c57-413e-b940-89e50cc79ff0


https://github.com/user-attachments/assets/d3a6e72a-f488-45c1-ab58-88391639455a


https://github.com/user-attachments/assets/ac178abd-caf3-40bf-9ef7-7cc0567a32c3




All features have been tested successfully on macOS with Claude Code in
Wave Terminal.
This commit is contained in:
togo01 2025-11-08 05:56:06 +08:00 committed by GitHub
parent a2f982509e
commit 8435958e32
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 396 additions and 10 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
frontend/app/store/wshclientapi.ts
View file

@ -327,6 +327,11 @@ class RpcApiType {
return client.wshRpcCall("gettab", data, opts);
}
// command "gettempdir" [call]
GetTempDirCommand(client: WshClient, data: CommandGetTempDirData, opts?: RpcOpts): Promise<string> {
return client.wshRpcCall("gettempdir", data, opts);
}
// command "getupdatechannel" [call]
GetUpdateChannelCommand(client: WshClient, opts?: RpcOpts): Promise<string> {
return client.wshRpcCall("getupdatechannel", null, opts);

70
frontend/app/view/term/term-model.ts
View file

@ -29,11 +29,18 @@ import * as keyutil from "@/util/keyutil";
import { boundNumber, stringToBase64 } from "@/util/util";
import * as jotai from "jotai";
import * as React from "react";
import { computeTheme, DefaultTermTheme } from "./termutil";
import {
computeTheme,
createTempFileFromBlob,
DefaultTermTheme,
handleImagePasteBlob as handleImagePasteBlobUtil,
supportsImageInput as supportsImageInputUtil,
} from "./termutil";
import { TermWrap } from "./termwrap";
import { getBlockingCommand } from "./shellblocking";
export class TermViewModel implements ViewModel {
viewType: string;
nodeModel: BlockNodeModel;
connected: boolean;
@ -391,6 +398,51 @@ export class TermViewModel implements ViewModel {
RpcApi.ControllerInputCommand(TabRpcClient, { blockid: this.blockId, inputdata64: b64data });
}
async handlePaste() {
try {
const clipboardItems = await navigator.clipboard.read();
for (const item of clipboardItems) {
// Check for images first
const imageTypes = item.types.filter((type) => type.startsWith("image/"));
if (imageTypes.length > 0 && this.supportsImageInput()) {
const blob = await item.getType(imageTypes[0]);
await this.handleImagePasteBlob(blob);
return;
}
// Handle text
if (item.types.includes("text/plain")) {
const blob = await item.getType("text/plain");
const text = await blob.text();
this.termRef.current?.terminal.paste(text);
return;
}
}
} catch (err) {
console.error("Paste error:", err);
// Fallback to text-only paste
try {
const text = await navigator.clipboard.readText();
if (text) {
this.termRef.current?.terminal.paste(text);
}
} catch (fallbackErr) {
console.error("Fallback paste error:", fallbackErr);
}
}
}
supportsImageInput(): boolean {
return supportsImageInputUtil();
}
async handleImagePasteBlob(blob: Blob): Promise<void> {
await handleImagePasteBlobUtil(blob, TabRpcClient, (text) => {
this.termRef.current?.terminal.paste(text);
});
}
setTermMode(mode: "term" | "vdom") {
if (mode == "term") {
mode = null;
@ -489,6 +541,15 @@ export class TermViewModel implements ViewModel {
if (waveEvent.type != "keydown") {
return true;
}
// Handle Escape key during IME composition
if (keyutil.checkKeyPressed(waveEvent, "Escape")) {
if (this.termRef.current?.isComposing) {
// Reset composition state when Escape is pressed during composition
this.termRef.current.resetCompositionState();
}
}
if (this.keyDownHandler(waveEvent)) {
event.preventDefault();
event.stopPropagation();
@ -496,7 +557,7 @@ export class TermViewModel implements ViewModel {
}
if (keyutil.checkKeyPressed(waveEvent, "Shift:Enter")) {
const shiftEnterNewlineAtom = getOverrideConfigAtom(this.blockId, "term:shiftenternewline");
const shiftEnterNewlineEnabled = globalStore.get(shiftEnterNewlineAtom) ?? false;
const shiftEnterNewlineEnabled = globalStore.get(shiftEnterNewlineAtom) ?? true;
if (shiftEnterNewlineEnabled) {
this.sendDataToController("\u001b\n");
event.preventDefault();
@ -505,10 +566,7 @@ export class TermViewModel implements ViewModel {
}
}
if (keyutil.checkKeyPressed(waveEvent, "Ctrl:Shift:v")) {
const p = navigator.clipboard.readText();
p.then((text) => {
this.termRef.current?.terminal.paste(text);
});
this.handlePaste();
event.preventDefault();
event.stopPropagation();
return false;

98
frontend/app/view/term/termutil.ts
View file

@ -34,3 +34,101 @@ function computeTheme(
}
export { computeTheme };
import { RpcApi } from "@/app/store/wshclientapi";
import { WshClient } from "@/app/store/wshclient";
export const MIME_TO_EXT: Record<string, string> = {
"image/png": "png",
"image/jpeg": "jpg",
"image/jpg": "jpg",
"image/gif": "gif",
"image/webp": "webp",
"image/bmp": "bmp",
"image/svg+xml": "svg",
"image/tiff": "tiff",
};
/**
* Creates a temporary file from a Blob (typically an image).
* Validates size, generates a unique filename, saves to temp directory,
* and returns the file path.
*
* @param blob - The Blob to save
* @param client - The WshClient for RPC calls
* @returns The path to the created temporary file
* @throws Error if blob is too large (>5MB) or data URL is invalid
*/
export async function createTempFileFromBlob(blob: Blob, client: WshClient): Promise<string> {
// Check size limit (5MB)
if (blob.size > 5 * 1024 * 1024) {
throw new Error("Image too large (>5MB)");
}
// Get file extension from MIME type
const ext = MIME_TO_EXT[blob.type] || "png";
// Generate unique filename with timestamp and random component
const timestamp = Date.now();
const random = Math.random().toString(36).substring(2, 8);
const filename = `waveterm_paste_${timestamp}_${random}.${ext}`;
// Get platform-appropriate temp file path from backend
const tempPath = await RpcApi.GetTempDirCommand(client, { filename });
// Convert blob to base64 using FileReader
const dataUrl = await new Promise<string>((resolve, reject) => {
const reader = new FileReader();
reader.onload = () => resolve(reader.result as string);
reader.onerror = reject;
reader.readAsDataURL(blob);
});
// Extract base64 data from data URL (remove "data:image/png;base64," prefix)
const parts = dataUrl.split(",");
if (parts.length < 2) {
throw new Error("Invalid data URL format");
}
const base64Data = parts[1];
// Write image to temp file
await RpcApi.FileWriteCommand(client, {
info: { path: tempPath },
data64: base64Data,
});
return tempPath;
}
/**
* Checks if image input is supported.
* Images will be saved as temp files and the path will be pasted.
* Claude Code and other AI tools can then read the file.
*
* @returns true if image input is supported
*/
export function supportsImageInput(): boolean {
return true;
}
/**
* Handles pasting an image blob by creating a temp file and pasting its path.
*
* @param blob - The image blob to paste
* @param client - The WshClient for RPC calls
* @param pasteFn - Function to paste the file path into the terminal
*/
export async function handleImagePasteBlob(
blob: Blob,
client: WshClient,
pasteFn: (text: string) => void
): Promise<void> {
try {
const tempPath = await createTempFileFromBlob(blob, client);
// Paste the file path (like iTerm2 does when you copy a file)
// Claude Code will read the file and display it as [Image #N]
pasteFn(tempPath + " ");
} catch (err) {
console.error("Error pasting image:", err);
}
}

201
frontend/app/view/term/termwrap.ts
View file

@ -18,6 +18,11 @@ import { Terminal } from "@xterm/xterm";
import debug from "debug";
import * as jotai from "jotai";
import { debounce } from "throttle-debounce";
import {
createTempFileFromBlob,
handleImagePasteBlob as handleImagePasteBlobUtil,
supportsImageInput as supportsImageInputUtil,
} from "./termutil";
import { FitAddon } from "./fitaddon";
const dlog = debug("wave:termwrap");
@ -351,6 +356,20 @@ export class TermWrap {
shellIntegrationStatusAtom: jotai.PrimitiveAtom<"ready" | "running-command" | null>;
lastCommandAtom: jotai.PrimitiveAtom<string | null>;
// IME composition state tracking
// Prevents duplicate input when switching input methods during composition (e.g., using Capslock)
// xterm.js sends data during compositionupdate AND after compositionend, causing duplicates
isComposing: boolean = false;
composingData: string = "";
lastCompositionEnd: number = 0;
lastComposedText: string = "";
firstDataAfterCompositionSent: boolean = false;
// Paste deduplication
// xterm.js paste() method triggers onData event, which can cause duplicate sends
lastPasteData: string = "";
lastPasteTime: number = 0;
constructor(
blockId: string,
connectElem: HTMLDivElement,
@ -422,11 +441,68 @@ export class TermWrap {
this.handleResize_debounced = debounce(50, this.handleResize.bind(this));
this.terminal.open(this.connectElem);
this.handleResize();
let pasteEventHandler = () => {
let pasteEventHandler = async (e: ClipboardEvent) => {
this.pasteActive = true;
setTimeout(() => {
this.pasteActive = false;
}, 30);
try {
// First try using ClipboardEvent.clipboardData (works in Electron)
if (e.clipboardData && e.clipboardData.items) {
const items = e.clipboardData.items;
// Check for images first
for (let i = 0; i < items.length; i++) {
const item = items[i];
if (item.type.startsWith("image/")) {
if (this.supportsImageInput()) {
e.preventDefault();
const blob = item.getAsFile();
if (blob) {
await this.handleImagePasteBlob(blob);
return;
}
}
}
}
// Handle text
const text = e.clipboardData.getData("text/plain");
if (text) {
this.terminal.paste(text);
return;
}
}
// Fallback: Try Clipboard API for newer browsers
const clipboardItems = await navigator.clipboard.read();
for (const item of clipboardItems) {
const imageTypes = item.types.filter((type) => type.startsWith("image/"));
if (imageTypes.length > 0 && this.supportsImageInput()) {
await this.handleImagePaste(item, imageTypes[0]);
return;
}
if (item.types.includes("text/plain")) {
const blob = await item.getType("text/plain");
const text = await blob.text();
this.terminal.paste(text);
return;
}
}
} catch (err) {
console.error("Paste error:", err);
// Final fallback to simple text paste
if (e.clipboardData) {
const text = e.clipboardData.getData("text/plain");
if (text) {
this.terminal.paste(text);
}
}
} finally {
setTimeout(() => {
this.pasteActive = false;
}, 30);
}
};
pasteEventHandler = pasteEventHandler.bind(this);
this.connectElem.addEventListener("paste", pasteEventHandler, true);
@ -437,6 +513,30 @@ export class TermWrap {
});
}
resetCompositionState() {
this.isComposing = false;
this.composingData = "";
}
private handleCompositionStart = (e: CompositionEvent) => {
dlog("compositionstart", e.data);
this.isComposing = true;
this.composingData = "";
};
private handleCompositionUpdate = (e: CompositionEvent) => {
dlog("compositionupdate", e.data);
this.composingData = e.data || "";
};
private handleCompositionEnd = (e: CompositionEvent) => {
dlog("compositionend", e.data);
this.isComposing = false;
this.lastComposedText = e.data || "";
this.lastCompositionEnd = Date.now();
this.firstDataAfterCompositionSent = false;
};
async initTerminal() {
const copyOnSelectAtom = getSettingsKeyAtom("term:copyonselect");
this.toDispose.push(this.terminal.onData(this.handleTermData.bind(this)));
@ -457,6 +557,33 @@ export class TermWrap {
if (this.onSearchResultsDidChange != null) {
this.toDispose.push(this.searchAddon.onDidChangeResults(this.onSearchResultsDidChange.bind(this)));
}
// Register IME composition event listeners on the xterm.js textarea
const textareaElem = this.connectElem.querySelector("textarea");
if (textareaElem) {
textareaElem.addEventListener("compositionstart", this.handleCompositionStart);
textareaElem.addEventListener("compositionupdate", this.handleCompositionUpdate);
textareaElem.addEventListener("compositionend", this.handleCompositionEnd);
// Handle blur during composition - reset state to avoid stale data
const blurHandler = () => {
if (this.isComposing) {
dlog("Terminal lost focus during composition, resetting IME state");
this.resetCompositionState();
}
};
textareaElem.addEventListener("blur", blurHandler);
this.toDispose.push({
dispose: () => {
textareaElem.removeEventListener("compositionstart", this.handleCompositionStart);
textareaElem.removeEventListener("compositionupdate", this.handleCompositionUpdate);
textareaElem.removeEventListener("compositionend", this.handleCompositionEnd);
textareaElem.removeEventListener("blur", blurHandler);
},
});
}
this.mainFileSubject = getFileSubject(this.blockId, TermFileName);
this.mainFileSubject.subscribe(this.handleNewFileSubjectData.bind(this));
@ -506,12 +633,58 @@ export class TermWrap {
if (!this.loaded) {
return;
}
// IME Composition Handling
// Block all data during composition - only send the final text after compositionend
// This prevents xterm.js from sending intermediate composition data (e.g., during compositionupdate)
if (this.isComposing) {
dlog("Blocked data during composition:", data);
return;
}
// Paste Deduplication
// xterm.js paste() method triggers onData event, causing handleTermData to be called twice:
// 1. From our paste handler (pasteActive=true)
// 2. From xterm.js onData (pasteActive=false)
// We allow the first call and block the second duplicate
const DEDUP_WINDOW_MS = 50;
const now = Date.now();
const timeSinceLastPaste = now - this.lastPasteTime;
if (this.pasteActive) {
// First paste event - record it and allow through
this.pasteActive = false;
this.lastPasteData = data;
this.lastPasteTime = now;
if (this.multiInputCallback) {
this.multiInputCallback(data);
}
} else if (timeSinceLastPaste < DEDUP_WINDOW_MS && data === this.lastPasteData && this.lastPasteData) {
// Second paste event with same data within time window - this is a duplicate, block it
dlog("Blocked duplicate paste data:", data);
this.lastPasteData = ""; // Clear to allow same data to be pasted later
return;
}
// IME Deduplication (for Capslock input method switching)
// When switching input methods with Capslock during composition, some systems send the
// composed text twice. We allow the first send and block subsequent duplicates.
const timeSinceCompositionEnd = now - this.lastCompositionEnd;
if (timeSinceCompositionEnd < DEDUP_WINDOW_MS && data === this.lastComposedText && this.lastComposedText) {
if (!this.firstDataAfterCompositionSent) {
// First send after composition - allow it but mark as sent
this.firstDataAfterCompositionSent = true;
dlog("First data after composition, allowing:", data);
} else {
// Second send of the same data - this is a duplicate from Capslock switching, block it
dlog("Blocked duplicate IME data:", data);
this.lastComposedText = ""; // Clear to allow same text to be typed again later
this.firstDataAfterCompositionSent = false;
return;
}
}
this.sendDataHandler?.(data);
}
@ -608,6 +781,26 @@ export class TermWrap {
}
}
supportsImageInput(): boolean {
return supportsImageInputUtil();
}
async handleImagePasteBlob(blob: Blob): Promise<void> {
await handleImagePasteBlobUtil(blob, TabRpcClient, (text) => {
this.terminal.paste(text);
});
}
async handleImagePaste(item: ClipboardItem, mimeType: string): Promise<void> {
try {
const blob = await item.getType(mimeType);
// Reuse the existing handleImagePasteBlob logic
await this.handleImagePasteBlob(blob);
} catch (err) {
console.error("Error processing image:", err);
}
}
handleResize() {
const oldRows = this.terminal.rows;
const oldCols = this.terminal.cols;

5
frontend/types/gotypes.d.ts vendored
View file

@ -279,6 +279,11 @@ declare global {
oref: ORef;
};
// wshrpc.CommandGetTempDirData
type CommandGetTempDirData = {
filename?: string;
};
// wshrpc.CommandGetWaveAIChatData
type CommandGetWaveAIChatData = {
chatid: string;

6
pkg/wshrpc/wshclient/wshclient.go
View file

@ -398,6 +398,12 @@ func GetTabCommand(w *wshutil.WshRpc, data string, opts *wshrpc.RpcOpts) (*waveo
return resp, err
}
// command "gettempdir", wshserver.GetTempDirCommand
func GetTempDirCommand(w *wshutil.WshRpc, data wshrpc.CommandGetTempDirData, opts *wshrpc.RpcOpts) (string, error) {
resp, err := sendRpcRequestCallHelper[string](w, "gettempdir", data, opts)
return resp, err
}
// command "getupdatechannel", wshserver.GetUpdateChannelCommand
func GetUpdateChannelCommand(w *wshutil.WshRpc, opts *wshrpc.RpcOpts) (string, error) {
resp, err := sendRpcRequestCallHelper[string](w, "getupdatechannel", nil, opts)

6
pkg/wshrpc/wshrpctypes.go
View file

@ -83,6 +83,7 @@ const (
Command_FileJoin = "filejoin"
Command_FileShareCapability = "filesharecapability"
Command_FileRestoreBackup = "filerestorebackup"
Command_GetTempDir = "gettempdir"
Command_EventPublish = "eventpublish"
Command_EventRecv = "eventrecv"
@ -222,6 +223,7 @@ type WshRpcInterface interface {
FileShareCapabilityCommand(ctx context.Context, path string) (FileShareCapability, error)
FileRestoreBackupCommand(ctx context.Context, data CommandFileRestoreBackupData) error
GetTempDirCommand(ctx context.Context, data CommandGetTempDirData) (string, error)
EventPublishCommand(ctx context.Context, data wps.WaveEvent) error
EventSubCommand(ctx context.Context, data wps.SubscriptionRequest) error
EventUnsubCommand(ctx context.Context, data string) error
@ -622,6 +624,10 @@ type CommandFileRestoreBackupData struct {
RestoreToFileName string `json:"restoretofilename"`
}
type CommandGetTempDirData struct {
FileName string `json:"filename,omitempty"`
}
type CommandRemoteStreamTarData struct {
Path string `json:"path"`
Opts *FileCopyOpts `json:"opts,omitempty"`

15
pkg/wshrpc/wshserver/wshserver.go
View file

@ -453,6 +453,21 @@ func (ws *WshServer) FileRestoreBackupCommand(ctx context.Context, data wshrpc.C
return filebackup.RestoreBackup(expandedBackupPath, expandedRestorePath)
}
func (ws *WshServer) GetTempDirCommand(ctx context.Context, data wshrpc.CommandGetTempDirData) (string, error) {
tempDir := os.TempDir()
if data.FileName != "" {
// Reduce to a simple file name to avoid absolute paths or traversal
name := filepath.Base(data.FileName)
// Normalize/trim any stray separators and whitespace
name = strings.Trim(name, `/\`+" ")
if name == "" || name == "." {
return tempDir, nil
}
return filepath.Join(tempDir, name), nil
}
return tempDir, nil
}
func (ws *WshServer) DeleteSubBlockCommand(ctx context.Context, data wshrpc.CommandDeleteBlockData) error {
err := wcore.DeleteBlock(ctx, data.BlockId, false)
if err != nil {