mirror of
https://github.com/wavetermdev/waveterm
synced 2026-05-24 09:18:27 +00:00
f36187f619
This pull request introduces a new integration test tool for the StreamManager streaming system, adding a standalone test binary with supporting modules for simulating and verifying high-throughput data transfer. The changes include a test driver, a configurable in-memory delivery pipe for simulating network conditions, a data generator, a verifier for end-to-end integrity, and a metrics tracker. Additionally, several improvements are made to the circular buffer and StreamManager for better handling of blocking writes and out-of-order acknowledgments. **New StreamManager Integration Test Tool** * Added a new test binary `cmd/test-streammanager` with a main driver (`main-test-streammanager.go`) that orchestrates end-to-end streaming tests, including configuration for data size, delivery delay/skew, window size, slow reader simulation, and verbose logging. * Implemented a configurable `DeliveryPipe` (`deliverypipe.go`) for simulating network delivery with delay and skew, supporting separate data and ack channels, out-of-order delivery, and high water mark tracking. * Added `WriterBridge` and `ReaderBridge` modules for interfacing between brokers and the delivery pipe, enforcing correct directionality of data and acks. * Created a sequential test data generator (`generator.go`) and a verifier (`verifier.go`) for checking data integrity and reporting mismatches. [[1]](diffhunk://#diff-3f2d6e0349089e3748c001791a383687b33a2c2391fd3baccfceb83e76e6ee0dR1-R40) [[2]](diffhunk://#diff-cb3aab0bae9bec15ef0c06fe5d9e0e96094affcf4720680605a92054ab717575R1-R61) * Introduced a metrics module (`metrics.go`) for tracking throughput, packet counts, out-of-order events, and pipe usage, with a summary report at test completion. **StreamManager and CirBuf Improvements** * Refactored circular buffer (`pkg/jobmanager/cirbuf.go`) to replace blocking writes with a non-blocking `WriteAvailable` method, returning a wait channel for buffer-full scenarios, and removed context-based cancellation logic. * Updated StreamManager (`pkg/jobmanager/streammanager.go`) to track the maximum acknowledged sequence/rwnd tuple, ignoring stale or out-of-order ACKs, and resetting this state on disconnect. * Modified StreamManager's data handling to use the new non-blocking buffer write logic, ensuring correct signaling and waiting for space when needed. **Minor Cleanup** * Removed unused context import from `cirbuf.go`. * Minor whitespace cleanup in `streambroker.go`.
40 lines
1 KiB
Go
40 lines
1 KiB
Go
// Copyright 2026, Command Line Inc.
|
|
// SPDX-License-Identifier: Apache-2.0
|
|
|
|
package main
|
|
|
|
import (
|
|
"fmt"
|
|
|
|
"github.com/wavetermdev/waveterm/pkg/wshrpc"
|
|
)
|
|
|
|
// WriterBridge - used by the writer broker
|
|
// Sends data to the pipe, receives acks from the pipe
|
|
type WriterBridge struct {
|
|
pipe *DeliveryPipe
|
|
}
|
|
|
|
func (b *WriterBridge) StreamDataCommand(data wshrpc.CommandStreamData, opts *wshrpc.RpcOpts) error {
|
|
b.pipe.EnqueueData(data)
|
|
return nil
|
|
}
|
|
|
|
func (b *WriterBridge) StreamDataAckCommand(ack wshrpc.CommandStreamAckData, opts *wshrpc.RpcOpts) error {
|
|
return fmt.Errorf("writer bridge should not send acks")
|
|
}
|
|
|
|
// ReaderBridge - used by the reader broker
|
|
// Sends acks to the pipe, receives data from the pipe
|
|
type ReaderBridge struct {
|
|
pipe *DeliveryPipe
|
|
}
|
|
|
|
func (b *ReaderBridge) StreamDataCommand(data wshrpc.CommandStreamData, opts *wshrpc.RpcOpts) error {
|
|
return fmt.Errorf("reader bridge should not send data")
|
|
}
|
|
|
|
func (b *ReaderBridge) StreamDataAckCommand(ack wshrpc.CommandStreamAckData, opts *wshrpc.RpcOpts) error {
|
|
b.pipe.EnqueueAck(ack)
|
|
return nil
|
|
}
|