* Added controller and service for fetching logs and nodes seperately
* Add inifinte scroll pagination for workflow logs
* Fix "All Nodes" not renderd on Nodes column
* Fix reducer to append logs on Logs column
* Fix: incorrect code placement
* Fix reducer to append logs on Nodes column
* Prepend execution log on workflow run
* Fix node ordering
* Decouple logs panel from workflow editor
* Update execution nodes when log panel is opened
* Reset log selection on workflow run
* Added 'updatePreviewState' function
* Update ee-server submodule reference
Migrate hotfix for workflow blockers including:
- Feature: Ability to call workflow within a workflow
- Feature: Preview query node with other previewed node results
- Fix: startTrigger input accepts string values instead of code/JSON
Frontend changes:
- Add workflow-in-workflow support in FlowBuilder and QueryNodeConfiguration
- Enhance query node preview with upstream node state management
- Implement preview state cleanup with deleteNodePreviewState calls
- Replace TestParameter input with CodeHinter for better JSON editing
- Add workflow node type support and error styling
- Update WorkflowEditor components for nested workflow handling
Backend changes:
- Update workflow execution controller to handle state parameter
- Enhance PreviewWorkflowNodeDto with optional state field
- Modify workflow services for state-aware query execution
- Update triggers listener and webhooks service
Migration follows composition pattern for frontend EE modules and
inheritance pattern for backend services.
Co-authored-by: Akshay Sasidharan <akshaysasidrn@tooljet.com>
Co-authored-by: Shah <shah21@tooljet.com>
Co-authored-by: Devanshu <devanshu052000@tooljet.com>
Overview
This PR implements real-time workflow execution monitoring using Server-Sent Events (SSE) as part of our LTS feature support. The implementation allows for tracking long-running workflows without requiring deployment changes.
Technical Changes
1. Server Components
Added WorkflowStreamService to manage persistent SSE streams with automatic cleanup
Implemented @sse endpoint in WorkflowExecutionsController for streaming status updates
Created WorkflowTriggersListener to emit workflow execution events with EventEmitter2
Added workflow execution status constants to identify different states in the execution lifecycle
2. Client-side Components
Implemented AsyncQueryHandler to manage SSE connections and parse event streams
Enhanced queryPanelSlice with methods to create async handlers and trigger workflows
Added support for non-blocking workflow execution with real-time status updates
3. Workflow Integration
Modified workflow triggering to use the SSE-based monitoring approach
Maintained same-server architecture to avoid deployment changes
Added automatic reconnection handling and error recovery for client connections
Architecture Decisions
Selected Same Server Approach: Chose to implement workflows within the same HTTP server to maintain the existing deployment setup for LTS users
Real-time Updates with SSE: Leveraged Server-Sent Events for their simplicity, efficiency, and compatibility with existing infrastructure
Future Extension Path: Implementation can be extended to Worker Threads or Microservice architecture later if needed
