mirror of
https://github.com/wavetermdev/waveterm
synced 2026-05-24 09:18:27 +00:00
f36187f619
This pull request introduces a new integration test tool for the StreamManager streaming system, adding a standalone test binary with supporting modules for simulating and verifying high-throughput data transfer. The changes include a test driver, a configurable in-memory delivery pipe for simulating network conditions, a data generator, a verifier for end-to-end integrity, and a metrics tracker. Additionally, several improvements are made to the circular buffer and StreamManager for better handling of blocking writes and out-of-order acknowledgments. **New StreamManager Integration Test Tool** * Added a new test binary `cmd/test-streammanager` with a main driver (`main-test-streammanager.go`) that orchestrates end-to-end streaming tests, including configuration for data size, delivery delay/skew, window size, slow reader simulation, and verbose logging. * Implemented a configurable `DeliveryPipe` (`deliverypipe.go`) for simulating network delivery with delay and skew, supporting separate data and ack channels, out-of-order delivery, and high water mark tracking. * Added `WriterBridge` and `ReaderBridge` modules for interfacing between brokers and the delivery pipe, enforcing correct directionality of data and acks. * Created a sequential test data generator (`generator.go`) and a verifier (`verifier.go`) for checking data integrity and reporting mismatches. [[1]](diffhunk://#diff-3f2d6e0349089e3748c001791a383687b33a2c2391fd3baccfceb83e76e6ee0dR1-R40) [[2]](diffhunk://#diff-cb3aab0bae9bec15ef0c06fe5d9e0e96094affcf4720680605a92054ab717575R1-R61) * Introduced a metrics module (`metrics.go`) for tracking throughput, packet counts, out-of-order events, and pipe usage, with a summary report at test completion. **StreamManager and CirBuf Improvements** * Refactored circular buffer (`pkg/jobmanager/cirbuf.go`) to replace blocking writes with a non-blocking `WriteAvailable` method, returning a wait channel for buffer-full scenarios, and removed context-based cancellation logic. * Updated StreamManager (`pkg/jobmanager/streammanager.go`) to track the maximum acknowledged sequence/rwnd tuple, ignoring stale or out-of-order ACKs, and resetting this state on disconnect. * Modified StreamManager's data handling to use the new non-blocking buffer write logic, ensuring correct signaling and waiting for space when needed. **Minor Cleanup** * Removed unused context import from `cirbuf.go`. * Minor whitespace cleanup in `streambroker.go`.
40 lines
873 B
Go
40 lines
873 B
Go
// Copyright 2026, Command Line Inc.
|
|
// SPDX-License-Identifier: Apache-2.0
|
|
|
|
package main
|
|
|
|
import (
|
|
"io"
|
|
)
|
|
|
|
// Base64 charset: all printable, easy to inspect manually
|
|
const Base64Chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"
|
|
|
|
type TestDataGenerator struct {
|
|
totalBytes int64
|
|
generated int64
|
|
}
|
|
|
|
func NewTestDataGenerator(totalBytes int64) *TestDataGenerator {
|
|
return &TestDataGenerator{totalBytes: totalBytes}
|
|
}
|
|
|
|
func (g *TestDataGenerator) Read(p []byte) (n int, err error) {
|
|
if g.generated >= g.totalBytes {
|
|
return 0, io.EOF
|
|
}
|
|
|
|
remaining := g.totalBytes - g.generated
|
|
toRead := int64(len(p))
|
|
if toRead > remaining {
|
|
toRead = remaining
|
|
}
|
|
|
|
// Sequential pattern using base64 chars (0-63 cycling)
|
|
for i := int64(0); i < toRead; i++ {
|
|
p[i] = Base64Chars[(g.generated+i)%64]
|
|
}
|
|
|
|
g.generated += toRead
|
|
return int(toRead), nil
|
|
}
|