ToolJet/docker/LTS/ee/redis.conf

94 lines
2.7 KiB
Text
Raw Normal View History

Refactor: Workflows with BullMQ (#14275) * refactor(workflows): migrate core modules from Temporal to BullMQ Update main application modules to support BullMQ-based workflow scheduling: - Remove Temporal worker bootstrap code from main.ts - Migrate from @nestjs/bull to @nestjs/bullmq - Add Bull Board dashboard at /jobs with basic auth - Register BullMQ queues in WorkflowsModule - Add IWorkflowScheduler interface for scheduler abstraction - Create CE stubs for WorkflowSchedulerService and ScheduleBootstrapService - Remove workflow dependencies from AppsModule (moved to WorkflowsModule) - Add proper route exclusion for /jobs dashboard - Support WORKER env var for conditional processor registration This commit completes the migration from Temporal to BullMQ for workflow scheduling, enabling simpler deployment and better horizontal scaling. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: remove unused Temporal imports and commented code Clean up migration artifacts: - Remove unused imports from main.ts (TOOLJET_EDITIONS, getImportPath, ITemporalService, getTooljetEdition) - Remove commented TemporalService references from WorkflowsModule - Remove temporal.service from getProviders path array - Add missing newlines at EOF for IWorkflowScheduler.ts and schedule-bootstrap.service.ts This cleanup prepares the codebase for complete Temporal code removal in a future commit. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * build: add BullMQ and Bull Board dependencies Add required packages for BullMQ-based workflow scheduling: - @nestjs/bullmq: NestJS integration for BullMQ - @bull-board/api, @bull-board/express, @bull-board/nestjs: Queue dashboard - bullmq: Core BullMQ library - express-basic-auth: Authentication for Bull Board dashboard Note: @nestjs/bull is kept for backward compatibility during migration. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: remove Temporal service files from CE Remove deprecated Temporal-based workflow implementation files: - server/src/modules/workflows/interfaces/ITemporalService.ts - server/src/modules/workflows/services/temporal.service.ts These files are replaced by IWorkflowScheduler interface and BullMQ-based WorkflowSchedulerService implementation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * feat: add comprehensive Redis configuration support for BullMQ * refactor: remove Temporal setup and configuration from entrypoint scripts and Dockerfiles (#14294) * refactor: remove Temporal setup and configuration from entrypoint scripts and Dockerfiles * feat: integrate Redis support for BullMQ in preview environment * remove worker execution logic from setup script * Refactor: Centralise workflow execution through BullMQ (#14321) * refactor: run all workflows through bullmq * refactor: update imports * chore: update subproject commit reference in server/ee * feat: ablity to cancel workflow * feat: implement workflow cancellation functionality with Redis support * feat: add optional timeout parameter to requestCancellation method * refactor: clean up formatting and add maintenance toggle event emission in AppsService * feat: ability to cancel multiple inprogress executions * feat: implement execution state management and display logic * chore: update submodule commit reference --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Adish M <44204658+adishM98@users.noreply.github.com>
2025-11-05 12:24:38 +00:00
# Redis Configuration for BullMQ in ToolJet PR Preview Environments
# Optimized for Render.com Standard plan (1GB RAM)
################################## NETWORK #####################################
# Bind to localhost only for security (same container access only)
bind 127.0.0.1
# Accept connections on default Redis port
port 6379
# Enable protected mode (requires password or bind to localhost)
protected-mode yes
################################# PERSISTENCE ##################################
# Enable AOF (Append Only File) persistence - CRITICAL for BullMQ
# Ensures job data survives Redis restarts
appendonly yes
# Sync strategy: everysec balances performance and durability
# - always: Safest but slowest
# - everysec: Good balance (recommended)
# - no: Fastest but risky
appendfsync everysec
# Enable RDB snapshots as backup to AOF
# Format: save <seconds> <changes>
# After 900 sec (15 min) if at least 1 key changed
save 900 1
# After 300 sec (5 min) if at least 10 keys changed
save 300 10
# After 60 sec if at least 10000 keys changed
save 60 10000
Refactor: Workflows with BullMQ (#14275) * refactor(workflows): migrate core modules from Temporal to BullMQ Update main application modules to support BullMQ-based workflow scheduling: - Remove Temporal worker bootstrap code from main.ts - Migrate from @nestjs/bull to @nestjs/bullmq - Add Bull Board dashboard at /jobs with basic auth - Register BullMQ queues in WorkflowsModule - Add IWorkflowScheduler interface for scheduler abstraction - Create CE stubs for WorkflowSchedulerService and ScheduleBootstrapService - Remove workflow dependencies from AppsModule (moved to WorkflowsModule) - Add proper route exclusion for /jobs dashboard - Support WORKER env var for conditional processor registration This commit completes the migration from Temporal to BullMQ for workflow scheduling, enabling simpler deployment and better horizontal scaling. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: remove unused Temporal imports and commented code Clean up migration artifacts: - Remove unused imports from main.ts (TOOLJET_EDITIONS, getImportPath, ITemporalService, getTooljetEdition) - Remove commented TemporalService references from WorkflowsModule - Remove temporal.service from getProviders path array - Add missing newlines at EOF for IWorkflowScheduler.ts and schedule-bootstrap.service.ts This cleanup prepares the codebase for complete Temporal code removal in a future commit. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * build: add BullMQ and Bull Board dependencies Add required packages for BullMQ-based workflow scheduling: - @nestjs/bullmq: NestJS integration for BullMQ - @bull-board/api, @bull-board/express, @bull-board/nestjs: Queue dashboard - bullmq: Core BullMQ library - express-basic-auth: Authentication for Bull Board dashboard Note: @nestjs/bull is kept for backward compatibility during migration. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: remove Temporal service files from CE Remove deprecated Temporal-based workflow implementation files: - server/src/modules/workflows/interfaces/ITemporalService.ts - server/src/modules/workflows/services/temporal.service.ts These files are replaced by IWorkflowScheduler interface and BullMQ-based WorkflowSchedulerService implementation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * feat: add comprehensive Redis configuration support for BullMQ * refactor: remove Temporal setup and configuration from entrypoint scripts and Dockerfiles (#14294) * refactor: remove Temporal setup and configuration from entrypoint scripts and Dockerfiles * feat: integrate Redis support for BullMQ in preview environment * remove worker execution logic from setup script * Refactor: Centralise workflow execution through BullMQ (#14321) * refactor: run all workflows through bullmq * refactor: update imports * chore: update subproject commit reference in server/ee * feat: ablity to cancel workflow * feat: implement workflow cancellation functionality with Redis support * feat: add optional timeout parameter to requestCancellation method * refactor: clean up formatting and add maintenance toggle event emission in AppsService * feat: ability to cancel multiple inprogress executions * feat: implement execution state management and display logic * chore: update submodule commit reference --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Adish M <44204658+adishM98@users.noreply.github.com>
2025-11-05 12:24:38 +00:00
################################## MEMORY ######################################
# CRITICAL for BullMQ: Set max memory to 256MB (25% of 1GB container)
# Leave room for PostgreSQL (256MB), ToolJet (400MB), and system (112MB)
maxmemory 256mb
# CRITICAL for BullMQ: NEVER evict keys - throw error instead
# BullMQ requires all job data to remain in Redis
# Without this, jobs will be lost when memory is full
maxmemory-policy noeviction
################################### GENERAL ####################################
# Set the number of databases (default is 16)
databases 16
# Close connection after client is idle for N seconds (0 = disable)
timeout 0
# TCP keepalive
# Send TCP ACKs to clients every N seconds to detect dead connections
tcp-keepalive 300
# TCP listen() backlog
tcp-backlog 511
################################### LOGGING ####################################
# Logging level
# Options: debug, verbose, notice, warning
loglevel notice
# Log to stdout (captured by Docker/Render)
logfile ""
################################### CLIENTS ####################################
# Max number of connected clients
maxclients 10000
################################## SLOW LOG ####################################
# Log queries slower than N microseconds
slowlog-log-slower-than 10000
# Max length of slow log
slowlog-max-len 128
################################ LATENCY MONITOR ###############################
# Enable latency monitoring to track latency spikes
latency-monitor-threshold 100
################################## DIRECTORIES #################################
# Working directory for RDB and AOF files
# This will be set via command-line or Docker volume
dir /var/lib/redis