Big simplification. Remove the FileShare interface that abstracted
wsh://, s3://, and wavefile:// files.
It produced a lot of complexity for very little usage. We're just going
to focus on the wsh:// implementation since that's core to our remote
workflows.
* remove s3 implementation (and connections, and picker items for
preview)
* remove capabilities for FE
* remove wavefile backend impl as well
* simplify wsh file remote backend
* remove ability to copy/move/ls recursively
* limit file transfers to 32m
the longer term fix here is to use the new streaming RPC primitives.
they have full end-to-end flow-control built in and will not create
pipeline stalls, blocking other requests, and OOM issues.
these other impls had to be removed (or fixed) because transferring
large files could cause stalls or crashes with the new router
infrastructure.
Lots of updates across all parts of the system to get this working. Big
changes to routing, streaming, connection management, etc.
* Persistent sessions behind a metadata flag for now
* New backlog queue in the router to prevent hanging
* Fix connection Close() issues that caused hangs when network was down
* Fix issue with random routeids (need to be generated fresh each time
the JWT is used and not fixed) so you can run multiple-wsh commands at
once
* Fix issue with domain sockets changing names across wave restarts
(added a symlink mechanism to resolve new names)
* ClientId caching in main server
* Quick reorder queue for input to prevent out of order delivery across
multiple hops
* Fix out-of-order event delivery in router (remove unnecessary go
routine creation)
* Environment testing and fix environment variables for remote jobs (get
from connserver, add to remote job starts)
* Add new ConnServerInit() remote method to call before marking
connection up
* TODO -- remote file transfer needs to be fixed to not create OOM
issues when transferring large files or directories
lots of stuff here.
introduces a streaming framework for the RPC system with flow control.
new authentication primitives for the RPC system. this is used to create
a persistent "job manager" process (via wsh) that can survive
disconnects. and then a jobcontroller in the main server that can
create, reconnect, and manage these new persistent jobs.
code is currently not actively hooked up to anything minus some new
debugging wsh commands, and a switch in the term block that lets me test
viewing the output.
after PRing this change the next steps are more testing and then
integrating this functionality into the product.
the PR spiraled and ended up being much larger than anticipated.
it is a refactor of wshrouter to have it track "links" as opposed to
just routes. this lets us simplify a lot of things when it comes to
multi-level routing.
* now the router can handle unauthenticated links directly, instead of a
weird limbo in wshproxy
* no more wshmultiproxy
* no more "authtoken" weirdness
* more straightforward handling in connserver (when using router option)
also adds more debugging, more logging, some windows fixes, other wsl
fixes
* announce and unannounce should propagate
* route the sendmessage command
* add timeouts for sending to OutputCh (appropriately)
* add ctx for SendCancel function
tie the tool approvals to the lifecycle of the SSE connection, NOT FE
keepalive packets. this prevents timeouts when the FE hasn't rendered
the tools (or FE tool rendering lifecycle issues). also allows immediate
timeouts on refresh.
other small fixes: convert to use wshclient in wshcmd-*.go files,
cleanup in SSE code
Removes the embedded Docusaurus build from the application. The help
view now always loads from https://docs.waveterm.dev/ instead of serving
a locally bundled copy. This eliminates package size constraints on
documentation content (e.g., videos) and removes unnecessary
build/packaging overhead.
**Backend changes:**
- Deleted `pkg/docsite` package entirely
- Removed docsite handler from `pkg/web/web.go`
- Simplified `emain/docsite.ts` to return constant URL instead of
probing for embedded docs
**Frontend changes:**
- Removed embedded URL translation logic from `helpview.tsx`
- Homepage URL now uses constant instead of IPC call
**Build configuration:**
- Removed `docsite:build:embedded` task and all references
- Removed `dist/docsite/**/*` from electron-builder asarUnpack
- Removed `build-embedded` script from docs package.json
Net result: -141 lines, +9 lines across 8 files.
<!-- START COPILOT CODING AGENT SUFFIX -->
<details>
<summary>Original prompt</summary>
> right now we build a docusaurus project to run both the online docs at
docs.waveterm.dev AND to embed into the application. but this limits
what we can show in the docs (especially large things like videos etc.)
>
> the internal docs already fall back to the website when it cannot find
the embedded docs.
>
> i'd like to update the build and packaging etc to no longer build the
embedded doc site. i'd also like to have the "help" view always render
the live doc site at https://docs.waveterm.dev.
</details>
<!-- START COPILOT CODING AGENT TIPS -->
---
💡 You can make Copilot smarter by setting up custom instructions,
customizing its development environment and configuring Model Context
Protocol (MCP) servers. Learn more [Copilot coding agent
tips](https://gh.io/copilot-coding-agent-tips) in the docs.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: sawka <2722291+sawka@users.noreply.github.com>
Massive PR, over 13k LOC updated, 128 commits to implement the first pass at the new Wave AI panel. Two backend adapters (OpenAI and Anthropic), layout changes to support the panel, keyboard shortcuts, and a huge focus/layout change to integrate the panel seamlessly into the UI.
Also fixes some small issues found during the Wave AI journey (zoom fixes, documentation, more scss removal, circular dependency issues, settings, etc)
Working on AI SDK compatible backends for OpenAI and Anthropic. Thinking + ToolUse etc. For use with AI SDK useChat on frontend. Still needs more testing, WIP, but this is a good start. Want to get this committed to so I can work on more integrations.
Adds the S3 `fileshare` implementation
This also updates `wsh file cp` so it behaves more like `cp` for things
like copying directories and directory entries. It's not meant to align
with `cp` on everything, though. Our `wsh cp` will be recursive and will
create intermediate directories by default.
This also adds new aliases for `wsh view`: `wsh preview` and `wsh open`
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: sawka <mike@commandline.dev>
Co-authored-by: Sylvia Crowe <software@oneirocosm.com>
Adds schema.json support to the settings file to provide type hints and
other eventual details. This also adds a system to easily add more
schema files for other type of configurations.
This adds the ability to stream `tar` archives over channels between
`wsh` instances. The main use cases for this are remote copy and move
operations.
It also completes the `wavefs` implementation of the FileShare interface
to allow copy/move interoperability between wavefiles and other storage
types.
The tar streaming functionality has been broken out into the new
`tarcopy` package for easy reuse.
New `fileshare` functions are added for `CopyInternal`, which allows
copying files internal to a filesystem to bypass the expensive interop
layer, and `MoveInternal`, which does the same for moving a file within
a filesystem. Copying between remotes is now handled by `CopyRemote`,
which accepts the source `FileShareClient` as a parameter. `wsh`
connections use the same implementation for `CopyInternal` and
`CopyRemote` as they need to request the channel on the remote
destination, since we don't offer a way to pass channels as a parameter
to a remote call.
This also adds a recursive `-r` flag to `wsh file rm` to allow for
deleting a directory and all its contents.
S3 support will be addressed in a future PR.
---------
Co-authored-by: sawka <mike@commandline.dev>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Fix VDom url caching -- use regular requests
* new boilerplate to make writing apps easier
* render-blocking global styles (to prevent render flash)
* bug fixes and new functionality etc.
Going forward for new installations, config and data files will be
stored at the platform default paths, as defined by
[env-paths](https://www.npmjs.com/package/env-paths).
For backwards compatibility, if the `~/.waveterm` or `WAVETERM_HOME`
directory exists and contains valid data, it will be used. If this check
fails, then `WAVETERM_DATA_HOME` and `WAVETERM_CONFIG_HOME` will be
used. If these are not defined, then `XDG_DATA_HOME` and
`XDG_CONFIG_HOME` will be used. Finally, if none of these are defined,
the [env-paths](https://www.npmjs.com/package/env-paths) defaults will
be used.
As with the existing app, dev instances will write to `waveterm-dev`
directories, while all others will write to `waveterm`.
This will take the latest artifact from the waveterm-docs repo and embed
it in the app binary. When the help view is launched, it will be served
from our backend. If the embedded copy doesn't exist, such as in
unpackaged versions of the app or in locally packaged versions, it will
use the hosted site instead.
There is a sibling PR in the docs repository to build the embedded
version of the app (strips out some external links, removes Algolia
DocSearch, updates the baseUrl)
https://github.com/wavetermdev/waveterm-docs/pull/46
With this PR, Electron will generate a new authorization key that the Go
backend will look for in any incoming requests. The Electron backend
will inject this header with all requests to the backend to ensure no
additional work is required on the frontend.
This also adds a `fetchutil` abstraction that will use the Electron
`net` module when calls are made from the Electron backend to the Go
backend. When using the `node:fetch` module, Electron can't inject
headers to requests. The Electron `net` module is also faster than the
Node module.
This also breaks out platform functions in emain into their own file so
other emain modules can import them.
This makes it possible to send wsh commands from wsh on a remote session
to wavesrv running locally. The exact behavior of running those commands
isn't implemented, but the underlying interface is added here.
