* ConnMonitor to track stalled connections
* New Stalled Overlay to show feedback when we think a connection is
stalled
* New Icon in ConnButton to show stalled connections
* Callbacks in domain socket and PTYs to track activity
Working on bug fixes and UX. Streams restarting, fixed lots of bugs,
timing issues, concurrency bugs. Get status shipped to the FE to drive
"shield" state display. Deal with stale streams.
Also big UX changes to the block headers. Specialize the terminal
headers to prioritize the connection (sense of place), remove old
terminal icon and word "Terminal" from the header. Also drop "Web" and
"Preview" labels on web/preview blocks.
Added `wsh focusblock` command.
Big simplification. Remove the FileShare interface that abstracted
wsh://, s3://, and wavefile:// files.
It produced a lot of complexity for very little usage. We're just going
to focus on the wsh:// implementation since that's core to our remote
workflows.
* remove s3 implementation (and connections, and picker items for
preview)
* remove capabilities for FE
* remove wavefile backend impl as well
* simplify wsh file remote backend
* remove ability to copy/move/ls recursively
* limit file transfers to 32m
the longer term fix here is to use the new streaming RPC primitives.
they have full end-to-end flow-control built in and will not create
pipeline stalls, blocking other requests, and OOM issues.
these other impls had to be removed (or fixed) because transferring
large files could cause stalls or crashes with the new router
infrastructure.
Lots of updates across all parts of the system to get this working. Big
changes to routing, streaming, connection management, etc.
* Persistent sessions behind a metadata flag for now
* New backlog queue in the router to prevent hanging
* Fix connection Close() issues that caused hangs when network was down
* Fix issue with random routeids (need to be generated fresh each time
the JWT is used and not fixed) so you can run multiple-wsh commands at
once
* Fix issue with domain sockets changing names across wave restarts
(added a symlink mechanism to resolve new names)
* ClientId caching in main server
* Quick reorder queue for input to prevent out of order delivery across
multiple hops
* Fix out-of-order event delivery in router (remove unnecessary go
routine creation)
* Environment testing and fix environment variables for remote jobs (get
from connserver, add to remote job starts)
* Add new ConnServerInit() remote method to call before marking
connection up
* TODO -- remote file transfer needs to be fixed to not create OOM
issues when transferring large files or directories
lots of stuff here.
introduces a streaming framework for the RPC system with flow control.
new authentication primitives for the RPC system. this is used to create
a persistent "job manager" process (via wsh) that can survive
disconnects. and then a jobcontroller in the main server that can
create, reconnect, and manage these new persistent jobs.
code is currently not actively hooked up to anything minus some new
debugging wsh commands, and a switch in the term block that lets me test
viewing the output.
after PRing this change the next steps are more testing and then
integrating this functionality into the product.
the PR spiraled and ended up being much larger than anticipated.
it is a refactor of wshrouter to have it track "links" as opposed to
just routes. this lets us simplify a lot of things when it comes to
multi-level routing.
* now the router can handle unauthenticated links directly, instead of a
weird limbo in wshproxy
* no more wshmultiproxy
* no more "authtoken" weirdness
* more straightforward handling in connserver (when using router option)
also adds more debugging, more logging, some windows fixes, other wsl
fixes
Gracefully handle prefix paths that don't exist, representing them as
directories so they can be escaped from.
Also removes the ".." file info from the backend, instead only creating
it on the frontend
Adds the S3 `fileshare` implementation
This also updates `wsh file cp` so it behaves more like `cp` for things
like copying directories and directory entries. It's not meant to align
with `cp` on everything, though. Our `wsh cp` will be recursive and will
create intermediate directories by default.
This also adds new aliases for `wsh view`: `wsh preview` and `wsh open`
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: sawka <mike@commandline.dev>
Co-authored-by: Sylvia Crowe <software@oneirocosm.com>
I was dumb and used `os.Rename` for copy on the same WSH remote. This
change makes it a proper copy, with recursion if needed.
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
This adds the ability to stream `tar` archives over channels between
`wsh` instances. The main use cases for this are remote copy and move
operations.
It also completes the `wavefs` implementation of the FileShare interface
to allow copy/move interoperability between wavefiles and other storage
types.
The tar streaming functionality has been broken out into the new
`tarcopy` package for easy reuse.
New `fileshare` functions are added for `CopyInternal`, which allows
copying files internal to a filesystem to bypass the expensive interop
layer, and `MoveInternal`, which does the same for moving a file within
a filesystem. Copying between remotes is now handled by `CopyRemote`,
which accepts the source `FileShareClient` as a parameter. `wsh`
connections use the same implementation for `CopyInternal` and
`CopyRemote` as they need to request the channel on the remote
destination, since we don't offer a way to pass channels as a parameter
to a remote call.
This also adds a recursive `-r` flag to `wsh file rm` to allow for
deleting a directory and all its contents.
S3 support will be addressed in a future PR.
---------
Co-authored-by: sawka <mike@commandline.dev>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
This provides a new configuration option that will turn off the ssh
config parsing. It also removes the flag required to override the ssh
config values with internal json values
Skip SSH Agent client initialization if the IdentitiesOnly config is set
for a given host
For now, we only need to support this for the SSH Agent, but if we ever
add support for PKCS11Provider or SecurityKeyProvider, we'll need to
update this check to also ignore those.
See https://man.openbsd.org/ssh_config#IdentitiesOnly
This adds an RPC command for updating wsh on a remote machine without
starting a new session. It is not being used yet, but will be used for
connections using a single server in the future.
lots of misc connection refactoring / fixes:
* adds blocklogger as a way to writing logging information from the backend directly to the a terminal block
* use blocklogger in conncontroller
* use blocklogger in sshclient
* fix remote name in password prompt
* use sh -c to get around shell weirdness
* remove cmd.exe special cases
* use GetWatcher().GetFullConfig() rather than re-reading the config file
* change order of things we do when establishing a connection. ask for wsh up front. then do domain socket, then connserver
* reduce number of sessions required in the common case when wsh is already installed. running the connserver is now a "multi-command" which checks if it is installed, then asks for the version
* send jwt token over stdin instead of in initial command string
* fix focus bug for frontend conn modal
* track more information in connstatus
* simplify wshinstall function
* add nowshreason
* other misc cleanup
* use pwsh over powershell if installed (on windows) for default shell
* refactor blockcontroller.DoRunShellCommand into a "setup" and "manage" phase
* fix wshcmd-conn to also disconnect wsl connections
* new genconn interfaces to make a standardized environment to run SSH/WSL commands via `sh -c`. also create better quoting functions that are composable
* replace html/template with text/template for shell command templating (avoids special chars getting turned into HTML entities, breaking the commands)
* do not reinstall wsh if the installed version has a higher version (prevents flip-flopping on shared systems)
* simplify clientOs/clientArch detection. use `uname -sm`. also validate the os/arch combo as compatible with our builds.
* replace CpHostToRemote with CpWshToRemote. hard codes wsh paths inside of the function instead of having them passed in (quoting restrictions)
* new SyncBuffer class to use with commands that properly synchronizes Writes/String output
* fix setTermSize to actually update DB with terminal size
