mirror of
https://github.com/documenso/documenso
synced 2026-04-21 13:27:18 +00:00
feat: add BullMQ background job provider with Bull Board dashboard (#2657)
Add a new BullMQ/Redis-backed job provider as an alternative to the existing Inngest and Local providers. Includes Bull Board UI for job monitoring at /api/jobs/board (admin-only in production, open in dev).
This commit is contained in:
Lucas Smith
GitHub
parent
025a27d385
commit
ad559f72dd
18 changed files with 1576 additions and 321 deletions
239
.agents/plans/bright-emerald-flower-bullmq-background-jobs.md
Normal file
239
.agents/plans/bright-emerald-flower-bullmq-background-jobs.md
Normal file
|
|
@ -0,0 +1,239 @@
|
|||
---
|
||||
date: 2026-03-26
|
||||
title: Bullmq Background Jobs
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
The codebase has a well-designed background job provider abstraction (`BaseJobProvider`) with two existing implementations:
|
||||
|
||||
- **InngestJobProvider** — cloud/SaaS provider, externally hosted
|
||||
- **LocalJobProvider** — database-backed (Postgres via Prisma), uses HTTP self-calls to dispatch
|
||||
|
||||
The goal is to add a third provider backed by a proper job queue library for self-hosted deployments that need more reliability than the Local provider offers.
|
||||
|
||||
### Current Architecture
|
||||
|
||||
All code lives in `packages/lib/jobs/`:
|
||||
|
||||
- `client/base.ts` — Abstract `BaseJobProvider` with 4 methods: `defineJob()`, `triggerJob()`, `getApiHandler()`, `startCron()`
|
||||
- `client/client.ts` — `JobClient` facade, selects provider via `NEXT_PRIVATE_JOBS_PROVIDER` env var
|
||||
- `client/inngest.ts` — Inngest implementation
|
||||
- `client/local.ts` — Local/Postgres implementation
|
||||
- `client/_internal/job.ts` — Core types: `JobDefinition`, `JobRunIO`, `SimpleTriggerJobOptions`
|
||||
- `definitions/` — 19 job definitions (15 event-triggered, 4 cron)
|
||||
|
||||
The `JobRunIO` interface provided to handlers includes:
|
||||
|
||||
- `runTask(cacheKey, callback)` — idempotent task execution (cached via `BackgroundJobTask` table)
|
||||
- `triggerJob(cacheKey, options)` — chain jobs from within handlers
|
||||
- `wait(cacheKey, ms)` — delay/sleep (not implemented in Local provider)
|
||||
- `logger` — structured logging
|
||||
|
||||
### Local Provider Limitations
|
||||
|
||||
The current Local provider has several issues that motivate this work:
|
||||
|
||||
1. `io.wait()` throws "Not implemented"
|
||||
2. HTTP self-call with 150ms fire-and-forget `Promise.race` is fragile
|
||||
3. No concurrency control — jobs run in the web server process
|
||||
4. No real retry backoff (immediate re-dispatch)
|
||||
5. No monitoring/visibility into job status
|
||||
6. Jobs compete for resources with HTTP request handling
|
||||
|
||||
---
|
||||
|
||||
## Provider Evaluation
|
||||
|
||||
Three alternatives were evaluated against the existing provider interface and project requirements.
|
||||
|
||||
### BullMQ (Redis-backed) — Recommended
|
||||
|
||||
| Attribute | Detail |
|
||||
| ------------------- | -------------------------- |
|
||||
| Backend | Redis 7.x |
|
||||
| npm downloads/month | ~15M |
|
||||
| TypeScript | Native |
|
||||
| Delayed jobs | Yes (ms precision) |
|
||||
| Cron/repeatable | Yes (`upsertJobScheduler`) |
|
||||
| Retries + backoff | Yes (exponential, custom) |
|
||||
| Concurrency control | Yes (per-worker) |
|
||||
| Rate limiting | Yes (per-queue, per-group) |
|
||||
| Dashboard | Bull Board (mature) |
|
||||
| New infrastructure | Yes — Redis required |
|
||||
|
||||
**Why BullMQ**: Most mature and widely-adopted Node.js queue. Native delayed jobs solve the `io.wait()` gap. Redis is purpose-built for queue workloads and keeps Postgres clean for application data. Bull Board gives immediate operational visibility. The provider abstraction already exists so wrapping BullMQ is straightforward.
|
||||
|
||||
**Trade-off**: Requires Redis, which is additional infrastructure. However, Redis is a single Docker Compose service or a free Upstash tier, and the operational benefit is significant.
|
||||
|
||||
### pg-boss (PostgreSQL-backed) — Strong Alternative
|
||||
|
||||
| Attribute | Detail |
|
||||
| ------------------- | ----------------------------- |
|
||||
| Backend | PostgreSQL (existing) |
|
||||
| npm downloads/month | ~1.4M |
|
||||
| TypeScript | Native |
|
||||
| Delayed jobs | Yes (`startAfter`) |
|
||||
| Cron/repeatable | Yes (`schedule()`) |
|
||||
| New infrastructure | No — reuses existing Postgres |
|
||||
|
||||
**Why it could work**: Zero new infrastructure since the project already uses Postgres. API maps well to existing patterns.
|
||||
|
||||
**Why it's second choice**: Polling-based (no LISTEN/NOTIFY), adds write amplification to the primary database, smaller ecosystem, no dashboard. At scale, queue operations on the primary database become a concern.
|
||||
|
||||
### Graphile Worker (PostgreSQL-backed) — Less Suitable
|
||||
|
||||
Uses LISTEN/NOTIFY for instant pickup but has a file-based task convention and separate schema that don't mesh well with the existing Prisma-centric architecture. Would require more adapter work.
|
||||
|
||||
### Improving the Local Provider — Not Recommended
|
||||
|
||||
Fixing the Local provider's issues (wait support, replacing HTTP self-calls, adding concurrency control, backoff) essentially means rebuilding a queue library from scratch with less robustness and no community maintenance.
|
||||
|
||||
---
|
||||
|
||||
## Recommendation
|
||||
|
||||
**Proceed with BullMQ.** It's the most capable option, maps cleanly to the existing provider interface, and is the standard choice for production Node.js applications. Redis is lightweight infrastructure with managed options available at every cloud provider.
|
||||
|
||||
**If Redis is a hard blocker**, pg-boss is the clear fallback — but the plan below assumes BullMQ.
|
||||
|
||||
---
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: BullMQ Provider Core
|
||||
|
||||
**File: `packages/lib/jobs/client/bullmq.ts`**
|
||||
|
||||
Create `BullMQJobProvider extends BaseJobProvider` with singleton pattern matching the existing providers.
|
||||
|
||||
Key implementation details:
|
||||
|
||||
1. **Constructor / `getInstance()`**
|
||||
- Initialize a Redis `IORedis` connection using new env var: `NEXT_PRIVATE_REDIS_URL`
|
||||
- Create a single `Queue` instance for dispatching jobs, using `NEXT_PRIVATE_REDIS_PREFIX` as the BullMQ `prefix` option (defaults to `documenso` if unset). This namespaces all Redis keys so multiple environments (worktrees, branches, developers) sharing the same Redis instance don't collide.
|
||||
- Create a single `Worker` instance for processing jobs (in-process, same prefix)
|
||||
- Store job definitions in a `_jobDefinitions` record (same pattern as Local provider)
|
||||
|
||||
2. **`defineJob()`**
|
||||
- Store definition in `_jobDefinitions` keyed by ID
|
||||
- If the definition has a `trigger.cron`, register it via `queue.upsertJobScheduler()` with the cron expression
|
||||
|
||||
3. **`triggerJob(options)`**
|
||||
- Find eligible definitions by `trigger.name` (same lookup as Local provider)
|
||||
- For each, call `queue.add(jobDefinitionId, payload)` with appropriate options
|
||||
- Support `options.id` for deduplication via BullMQ's `jobId` option
|
||||
|
||||
4. **`getApiHandler()`**
|
||||
- Return a minimal health-check / queue-status handler. Unlike the Local provider, BullMQ workers don't need an HTTP endpoint to receive jobs — they pull from Redis directly. The API handler can return queue metrics for monitoring.
|
||||
|
||||
5. **`startCron()`**
|
||||
- No-op — cron is handled by BullMQ's `upsertJobScheduler` registered during `defineJob()`
|
||||
|
||||
6. **Worker setup**
|
||||
- Single worker processes all job types by dispatching to the correct handler from `_jobDefinitions`
|
||||
- Configure concurrency with a default of 10 (overridable via `NEXT_PRIVATE_BULLMQ_CONCURRENCY` env var for those who need to tune it)
|
||||
- Configure retry with exponential backoff: `backoff: { type: 'exponential', delay: 1000 }`
|
||||
- Default 3 retries (matching current Local provider behavior)
|
||||
|
||||
7. **`createJobRunIO(jobId)`** — Implement `JobRunIO`:
|
||||
- `runTask()`: Reuse the existing `BackgroundJobTask` Prisma table for idempotent task tracking (same pattern as Local provider)
|
||||
- `triggerJob()`: Delegate to `this.triggerJob()`
|
||||
- `wait()`: Throw "Not implemented" (same as Local provider). No handler uses `io.wait()` so this has zero impact
|
||||
- `logger`: Same console-based logger pattern as Local provider
|
||||
|
||||
### Phase 2: Provider Registration
|
||||
|
||||
**File: `packages/lib/jobs/client/client.ts`**
|
||||
|
||||
Add `'bullmq'` case to the provider match:
|
||||
|
||||
```typescript
|
||||
this._provider = match(env('NEXT_PRIVATE_JOBS_PROVIDER'))
|
||||
.with('inngest', () => InngestJobProvider.getInstance())
|
||||
.with('bullmq', () => BullMQJobProvider.getInstance())
|
||||
.otherwise(() => LocalJobProvider.getInstance());
|
||||
```
|
||||
|
||||
**File: `packages/tsconfig/process-env.d.ts`**
|
||||
|
||||
Add `'bullmq'` to the `NEXT_PRIVATE_JOBS_PROVIDER` type union and add Redis env var types:
|
||||
|
||||
```typescript
|
||||
NEXT_PRIVATE_JOBS_PROVIDER?: 'inngest' | 'local' | 'bullmq';
|
||||
NEXT_PRIVATE_REDIS_URL?: string;
|
||||
NEXT_PRIVATE_REDIS_PREFIX?: string;
|
||||
NEXT_PRIVATE_BULLMQ_CONCURRENCY?: string;
|
||||
```
|
||||
|
||||
**File: `.env.example`**
|
||||
|
||||
Add Redis configuration examples:
|
||||
|
||||
```env
|
||||
NEXT_PRIVATE_JOBS_PROVIDER="local" # Options: local, inngest, bullmq
|
||||
NEXT_PRIVATE_REDIS_URL="redis://localhost:63790"
|
||||
NEXT_PRIVATE_REDIS_PREFIX="documenso" # Namespace for Redis keys (useful when sharing a Redis instance)
|
||||
```
|
||||
|
||||
**File: `turbo.json`**
|
||||
|
||||
Add `NEXT_PRIVATE_REDIS_URL`, `NEXT_PRIVATE_REDIS_PREFIX`, and `NEXT_PRIVATE_BULLMQ_CONCURRENCY` to the env vars list for cache invalidation.
|
||||
|
||||
### Phase 3: Infrastructure & Dependencies
|
||||
|
||||
**File: `packages/lib/package.json`**
|
||||
|
||||
Add dependencies:
|
||||
|
||||
- `bullmq` — the queue library
|
||||
- `ioredis` — Redis client (peer dependency of BullMQ, but explicit is better)
|
||||
|
||||
**File: `docker-compose.yml` (or equivalent)**
|
||||
|
||||
Add Redis service for local development:
|
||||
|
||||
```yaml
|
||||
redis:
|
||||
image: redis:7-alpine
|
||||
ports:
|
||||
- '6379:6379'
|
||||
```
|
||||
|
||||
### Phase 4: Optional Enhancements
|
||||
|
||||
These are not required for the initial implementation but worth considering for follow-up:
|
||||
|
||||
1. **Bull Board integration** — Add a `/api/jobs/dashboard` route that serves Bull Board UI for monitoring. Gate behind an admin auth check.
|
||||
|
||||
2. **Separate worker process** — Add an `apps/worker` entry point that runs BullMQ workers without the web server, for deployments that want to isolate job processing from request handling.
|
||||
|
||||
3. **Graceful shutdown** — Register `SIGTERM`/`SIGINT` handlers to call `worker.close()` and `queue.close()` for clean shutdown.
|
||||
|
||||
4. **BackgroundJob table integration** — Optionally continue writing to the `BackgroundJob` Prisma table for audit/history, using BullMQ events (`completed`, `failed`) to update status. This preserves the existing database-level visibility.
|
||||
|
||||
---
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
| File | Action | Description |
|
||||
| ------------------------------------ | ---------- | ---------------------------------------- |
|
||||
| `packages/lib/jobs/client/bullmq.ts` | **Create** | BullMQ provider implementation |
|
||||
| `packages/lib/jobs/client/client.ts` | Modify | Add `'bullmq'` provider case |
|
||||
| `packages/tsconfig/process-env.d.ts` | Modify | Add type for `'bullmq'` + Redis env vars |
|
||||
| `.env.example` | Modify | Add Redis config example |
|
||||
| `turbo.json` | Modify | Add Redis env var to cache keys |
|
||||
| `packages/lib/package.json` | Modify | Add `bullmq` + `ioredis` dependencies |
|
||||
| `docker-compose.yml` | Modify | Add Redis service |
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Should the BullMQ provider also write to the `BackgroundJob` Prisma table?** This would maintain audit history and allow existing admin tooling to query job status. Trade-off is dual-write complexity.
|
||||
|
||||
2. **Redis connection resilience**: Should the provider gracefully degrade if Redis is unavailable (e.g., fall back to Local provider), or fail hard? Failing hard is simpler and more predictable.
|
||||
|
||||
## Resolved Questions
|
||||
|
||||
- **`io.wait()`**: Not a concern. Only Inngest implements it (via `step.sleep`), the Local provider throws "Not implemented", and no job handler calls `io.wait()`. The BullMQ provider can throw "Not implemented" identically to the Local provider.
|
||||
|
|
@ -143,8 +143,15 @@ NEXT_PRIVATE_STRIPE_API_KEY=
|
|||
NEXT_PRIVATE_STRIPE_WEBHOOK_SECRET=
|
||||
|
||||
# [[BACKGROUND JOBS]]
|
||||
# Available options: local (default) | inngest | bullmq
|
||||
NEXT_PRIVATE_JOBS_PROVIDER="local"
|
||||
NEXT_PRIVATE_INNGEST_EVENT_KEY=
|
||||
# OPTIONAL: Redis URL for the BullMQ jobs provider.
|
||||
NEXT_PRIVATE_REDIS_URL="redis://localhost:63790"
|
||||
# OPTIONAL: Key prefix for Redis to namespace queues (useful when sharing a Redis instance).
|
||||
NEXT_PRIVATE_REDIS_PREFIX="documenso"
|
||||
# OPTIONAL: Number of concurrent jobs to process. Defaults to 10.
|
||||
# NEXT_PRIVATE_BULLMQ_CONCURRENCY=10
|
||||
|
||||
# [[FEATURES]]
|
||||
# OPTIONAL: Leave blank to disable PostHog and feature flags.
|
||||
|
|
|
|||
|
|
@ -235,6 +235,7 @@ Processes async jobs.
|
|||
| Provider | Description | Env Value |
|
||||
| -------- | --------------------- | ----------------- |
|
||||
| Local | Database-backed queue | `local` (default) |
|
||||
| BullMQ | Redis-backed queue | `bullmq` |
|
||||
| Inngest | Managed cloud service | `inngest` |
|
||||
|
||||
**Config**: `NEXT_PRIVATE_JOBS_PROVIDER`
|
||||
|
|
|
|||
|
|
@ -0,0 +1,187 @@
|
|||
---
|
||||
title: Background Jobs
|
||||
description: Configure how Documenso processes background tasks like email delivery, document processing, and webhook dispatch.
|
||||
---
|
||||
|
||||
import { Callout } from 'fumadocs-ui/components/callout';
|
||||
import { Tab, Tabs } from 'fumadocs-ui/components/tabs';
|
||||
|
||||
## Overview
|
||||
|
||||
Documenso processes background jobs for email delivery, document sealing, webhook dispatch, and scheduled maintenance tasks. Three providers are available:
|
||||
|
||||
| Provider | Backend | Best For | Infrastructure |
|
||||
| -------- | ---------- | ----------------------------------------------- | -------------- |
|
||||
| Inngest | Managed | Production with zero ops overhead | None |
|
||||
| BullMQ | Redis | Self-hosted production with full control | Redis |
|
||||
| Local | PostgreSQL | Development and small self-hosted deployments | None |
|
||||
|
||||
Select a provider with the `NEXT_PRIVATE_JOBS_PROVIDER` environment variable:
|
||||
|
||||
```bash
|
||||
NEXT_PRIVATE_JOBS_PROVIDER=inngest # or bullmq, local
|
||||
```
|
||||
|
||||
<Callout type="info">
|
||||
The default provider is `local`. It requires no additional infrastructure and works well for development and small deployments, but is not recommended for production workloads.
|
||||
</Callout>
|
||||
|
||||
---
|
||||
|
||||
## Inngest (Recommended)
|
||||
|
||||
[Inngest](https://www.inngest.com/) is a managed background job service. It handles scheduling, retries, concurrency, and observability without any infrastructure to manage. This is the recommended provider for production deployments.
|
||||
|
||||
### Setup
|
||||
|
||||
{/* prettier-ignore */}
|
||||
1. Create an account at [inngest.com](https://www.inngest.com/)
|
||||
2. Create an app and obtain your event key and signing key
|
||||
3. Configure the environment variables:
|
||||
|
||||
```bash
|
||||
NEXT_PRIVATE_JOBS_PROVIDER=inngest
|
||||
NEXT_PRIVATE_INNGEST_EVENT_KEY=your-event-key
|
||||
INNGEST_SIGNING_KEY=your-signing-key
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description | Required |
|
||||
| -------------------------------- | -------------------------------------------- | -------- |
|
||||
| `NEXT_PRIVATE_INNGEST_EVENT_KEY` | Inngest event key | Yes |
|
||||
| `INNGEST_EVENT_KEY` | Alternative Inngest event key | No |
|
||||
| `INNGEST_SIGNING_KEY` | Inngest signing key for webhook verification | Yes |
|
||||
| `NEXT_PRIVATE_INNGEST_APP_ID` | Custom Inngest app ID | No |
|
||||
|
||||
### Advantages
|
||||
|
||||
- No infrastructure to manage
|
||||
- Built-in monitoring dashboard
|
||||
- Automatic retries with backoff
|
||||
- Cron scheduling handled externally
|
||||
- Scales automatically
|
||||
|
||||
---
|
||||
|
||||
## BullMQ
|
||||
|
||||
[BullMQ](https://docs.bullmq.io/) is a Redis-backed job queue that runs inside the Documenso process. It provides higher throughput than the local provider, configurable concurrency, and a built-in dashboard for monitoring jobs.
|
||||
|
||||
### Requirements
|
||||
|
||||
- **Redis 6.2+** - any Redis-compatible service works (Redis, KeyDB, Dragonfly, AWS ElastiCache, Upstash, etc.)
|
||||
|
||||
### Setup
|
||||
|
||||
```bash
|
||||
NEXT_PRIVATE_JOBS_PROVIDER=bullmq
|
||||
NEXT_PRIVATE_REDIS_URL=redis://localhost:6379
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description | Default |
|
||||
| ---------------------------------- | -------------------------------------------------------------------------- | ----------- |
|
||||
| `NEXT_PRIVATE_REDIS_URL` | Redis connection URL | _(required)_ |
|
||||
| `NEXT_PRIVATE_REDIS_PREFIX` | Key prefix for Redis queues (useful when sharing an instance) | `documenso` |
|
||||
| `NEXT_PRIVATE_BULLMQ_CONCURRENCY` | Number of concurrent jobs to process | `10` |
|
||||
|
||||
### Dashboard
|
||||
|
||||
BullMQ includes a job monitoring dashboard at `/api/jobs/board`. In production, only admin users can access the dashboard. In development, it is open to all users.
|
||||
|
||||
The dashboard provides visibility into queued, active, completed, and failed jobs.
|
||||
|
||||
### Docker Compose with Redis
|
||||
|
||||
If you're using Docker Compose, add a Redis service:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
redis:
|
||||
image: redis:8-alpine
|
||||
ports:
|
||||
- '6379:6379'
|
||||
volumes:
|
||||
- redis_data:/data
|
||||
|
||||
volumes:
|
||||
redis_data:
|
||||
```
|
||||
|
||||
Then set `NEXT_PRIVATE_REDIS_URL=redis://redis:6379` in your Documenso environment.
|
||||
|
||||
### Advantages
|
||||
|
||||
- Self-hosted with no external service dependencies beyond Redis
|
||||
- Configurable concurrency
|
||||
- Built-in job monitoring dashboard
|
||||
- Reliable retries with exponential backoff
|
||||
- Queue namespacing for shared Redis instances
|
||||
|
||||
---
|
||||
|
||||
## Local
|
||||
|
||||
The local provider uses your PostgreSQL database as a job queue. Jobs are stored in the `BackgroundJob` table and processed via internal HTTP requests that Documenso sends to itself.
|
||||
|
||||
### Setup
|
||||
|
||||
No configuration required. The local provider is the default when `NEXT_PRIVATE_JOBS_PROVIDER` is unset or set to `local`.
|
||||
|
||||
```bash
|
||||
# Optional - this is the default
|
||||
NEXT_PRIVATE_JOBS_PROVIDER=local
|
||||
```
|
||||
|
||||
### Internal URL
|
||||
|
||||
Background jobs in the local provider work by Documenso sending HTTP requests to itself. If your reverse proxy or network setup causes issues with the app reaching its own public URL, set the internal URL:
|
||||
|
||||
```bash
|
||||
NEXT_PRIVATE_INTERNAL_WEBAPP_URL=http://localhost:3000
|
||||
```
|
||||
|
||||
This tells the job system to use the internal address instead of `NEXT_PUBLIC_WEBAPP_URL` for self-requests.
|
||||
|
||||
<Callout type="warn">
|
||||
The local provider is suitable for development and small deployments. For production workloads, use Inngest or BullMQ.
|
||||
</Callout>
|
||||
|
||||
### Limitations
|
||||
|
||||
- No concurrency control - jobs are processed one at a time per request cycle
|
||||
- No built-in monitoring
|
||||
- Depends on the application being able to reach itself over HTTP
|
||||
- Not suitable for high-throughput workloads
|
||||
|
||||
---
|
||||
|
||||
## Choosing a Provider
|
||||
|
||||
<Tabs items={['Managed hosting', 'Self-hosted production', 'Development']}>
|
||||
<Tab value="Managed hosting">
|
||||
|
||||
Use **Inngest**. Zero infrastructure, automatic scaling, and built-in observability. The simplest path to reliable background jobs in production.
|
||||
|
||||
</Tab>
|
||||
<Tab value="Self-hosted production">
|
||||
|
||||
Use **BullMQ**. Add a Redis instance to your infrastructure and get reliable job processing with a monitoring dashboard. Good fit if you already run Redis or want to keep everything self-hosted.
|
||||
|
||||
</Tab>
|
||||
<Tab value="Development">
|
||||
|
||||
Use **Local** (the default). No additional setup required. Works out of the box with just PostgreSQL.
|
||||
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
---
|
||||
|
||||
## See Also
|
||||
|
||||
- [Environment Variables](/docs/self-hosting/configuration/environment) - Complete configuration reference
|
||||
- [Requirements](/docs/self-hosting/getting-started/requirements) - Infrastructure requirements
|
||||
- [Docker Compose](/docs/self-hosting/deployment/docker-compose) - Deploy with Docker Compose
|
||||
|
|
@ -268,20 +268,40 @@ AI features must also be enabled in organisation/team settings after configurati
|
|||
|
||||
## Background Jobs
|
||||
|
||||
Documenso uses a PostgreSQL-based job queue by default. Jobs (email delivery, document processing, webhook dispatch) are stored in the `BackgroundJob` table and processed via internal HTTP requests. No external queue service like Redis is required.
|
||||
Documenso supports multiple background job providers for processing emails, documents, webhooks, and scheduled tasks.
|
||||
|
||||
### Provider Selection
|
||||
|
||||
| Variable | Description | Default |
|
||||
| ---------------------------- | ------------------------------------------------------------------------------ | ------- |
|
||||
| `NEXT_PRIVATE_JOBS_PROVIDER` | Jobs provider: `local` (PostgreSQL-based queue) or `inngest` (managed service) | `local` |
|
||||
| ---------------------------- | -------------------------------------------------------------------------------------- | ------- |
|
||||
| `NEXT_PRIVATE_JOBS_PROVIDER` | Jobs provider: `local` (PostgreSQL), `bullmq` (Redis), or `inngest` (managed service) | `local` |
|
||||
|
||||
### Inngest Configuration
|
||||
### Local (local)
|
||||
|
||||
| Variable | Description |
|
||||
| -------------------------------- | -------------------------------------------- |
|
||||
| `NEXT_PRIVATE_INNGEST_EVENT_KEY` | Inngest event key |
|
||||
| `INNGEST_EVENT_KEY` | Alternative Inngest event key |
|
||||
| `INNGEST_SIGNING_KEY` | Inngest signing key for webhook verification |
|
||||
| `NEXT_PRIVATE_INNGEST_APP_ID` | Custom Inngest app ID |
|
||||
No additional configuration required. Jobs are stored in PostgreSQL and processed via internal HTTP requests.
|
||||
|
||||
| Variable | Description | Default |
|
||||
| ---------------------------------- | ------------------------------------------------------------ | -------------------------------- |
|
||||
| `NEXT_PRIVATE_INTERNAL_WEBAPP_URL` | Internal URL for the app to send job requests to itself | Same as `NEXT_PUBLIC_WEBAPP_URL` |
|
||||
|
||||
### BullMQ (bullmq)
|
||||
|
||||
| Variable | Required | Description | Default |
|
||||
| ---------------------------------- | -------- | ------------------------------------------------------------- | ----------- |
|
||||
| `NEXT_PRIVATE_REDIS_URL` | Yes | Redis connection URL (e.g., `redis://localhost:6379`) | |
|
||||
| `NEXT_PRIVATE_REDIS_PREFIX` | No | Key prefix for Redis queues (useful when sharing an instance) | `documenso` |
|
||||
| `NEXT_PRIVATE_BULLMQ_CONCURRENCY` | No | Number of concurrent jobs to process | `10` |
|
||||
|
||||
### Inngest (inngest)
|
||||
|
||||
| Variable | Required | Description |
|
||||
| -------------------------------- | -------- | -------------------------------------------- |
|
||||
| `NEXT_PRIVATE_INNGEST_EVENT_KEY` | Yes | Inngest event key |
|
||||
| `INNGEST_EVENT_KEY` | No | Alternative Inngest event key |
|
||||
| `INNGEST_SIGNING_KEY` | Yes | Inngest signing key for webhook verification |
|
||||
| `NEXT_PRIVATE_INNGEST_APP_ID` | No | Custom Inngest app ID |
|
||||
|
||||
For setup guides and provider recommendations, see [Background Jobs](/docs/self-hosting/configuration/background-jobs).
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -5,6 +5,7 @@
|
|||
"database",
|
||||
"email",
|
||||
"storage",
|
||||
"background-jobs",
|
||||
"signing-certificate",
|
||||
"telemetry",
|
||||
"advanced"
|
||||
|
|
|
|||
|
|
@ -85,9 +85,13 @@ See [Storage Configuration](/docs/self-hosting/configuration/storage) for setup
|
|||
|
||||
### Background Jobs
|
||||
|
||||
Documenso processes background jobs (email delivery, document processing) using a PostgreSQL-based queue. No additional services like Redis are required: the job queue is built into the application and uses your existing database.
|
||||
Documenso processes background jobs (email delivery, document processing) using a PostgreSQL-based queue by default. No additional services are required: the job queue is built into the application and uses your existing database.
|
||||
|
||||
For high-throughput deployments, Documenso optionally supports [Inngest](https://www.inngest.com/) as an alternative job provider. Set `NEXT_PRIVATE_JOBS_PROVIDER=inngest` and configure `INNGEST_EVENT_KEY` and `INNGEST_SIGNING_KEY`. Most self-hosted instances do not need this.
|
||||
For production deployments that need higher throughput or more reliable job processing, Documenso supports [BullMQ](https://docs.bullmq.io/) as an alternative provider. BullMQ requires a **Redis** instance (v6.2+). Set `NEXT_PRIVATE_JOBS_PROVIDER=bullmq` and configure `NEXT_PRIVATE_REDIS_URL`.
|
||||
|
||||
For managed/cloud deployments, [Inngest](https://www.inngest.com/) is also supported as a job provider. Set `NEXT_PRIVATE_JOBS_PROVIDER=inngest` and configure `INNGEST_EVENT_KEY` and `INNGEST_SIGNING_KEY`.
|
||||
|
||||
See [Background Jobs Configuration](/docs/self-hosting/configuration/background-jobs) for full details.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -144,19 +144,13 @@ See [Storage Configuration](/docs/self-hosting/configuration/storage) for full s
|
|||
|
||||
---
|
||||
|
||||
## Background Jobs Don't Need Redis
|
||||
## Background Jobs
|
||||
|
||||
Documenso uses a PostgreSQL-based job queue by default. No Redis, no external message broker. The job system uses your existing database to store and process background tasks like email delivery and document processing.
|
||||
Documenso uses a PostgreSQL-based job queue by default (`local` provider). No Redis or external message broker is required for basic deployments.
|
||||
|
||||
For high-throughput deployments, Documenso optionally supports [Inngest](https://www.inngest.com/) as an alternative job provider:
|
||||
For production workloads, consider switching to **Inngest** (managed) or **BullMQ** (self-hosted with Redis) for better reliability and throughput.
|
||||
|
||||
```bash
|
||||
NEXT_PRIVATE_JOBS_PROVIDER=inngest
|
||||
INNGEST_EVENT_KEY=your-event-key
|
||||
INNGEST_SIGNING_KEY=your-signing-key
|
||||
```
|
||||
|
||||
Most self-hosted instances do not need Inngest.
|
||||
See [Background Jobs Configuration](/docs/self-hosting/configuration/background-jobs) for setup instructions and provider comparison.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
|
|
@ -99,6 +99,7 @@ app.route('/api/ai', aiRoute);
|
|||
// API servers.
|
||||
app.route('/api/v1', tsRestHonoApp);
|
||||
app.use('/api/jobs/*', jobsClient.getApiHandler());
|
||||
|
||||
app.use('/api/trpc/*', trpcRateLimitMiddleware);
|
||||
app.use('/api/trpc/*', reactRouterTrpcServer);
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import { defaultOptions as devServerDefaults } from '@hono/vite-dev-server';
|
||||
import { lingui } from '@lingui/vite-plugin';
|
||||
import { reactRouter } from '@react-router/dev/vite';
|
||||
import autoprefixer from 'autoprefixer';
|
||||
|
|
@ -46,6 +47,18 @@ export default defineConfig({
|
|||
tsconfigPaths(),
|
||||
serverAdapter({
|
||||
entry: 'server/router.ts',
|
||||
exclude: [
|
||||
// Spread the defaults but replace the /.css$/ rule so that Bull
|
||||
// Board's static CSS at /api/jobs/board/static/** passes through to Hono.
|
||||
...devServerDefaults.exclude.map((pattern) =>
|
||||
pattern instanceof RegExp && pattern.source === '.*\\.css$'
|
||||
? /^(?!\/api\/jobs\/board\/).*\.css$/
|
||||
: pattern,
|
||||
),
|
||||
'/assets/**',
|
||||
'/src/app/**',
|
||||
/\?(?:inline|url|no-inline|raw|import(?:&(?:inline|url|no-inline|raw)?)?)$/,
|
||||
],
|
||||
}),
|
||||
],
|
||||
ssr: {
|
||||
|
|
|
|||
|
|
@ -26,6 +26,14 @@ services:
|
|||
- 2500:2500
|
||||
- 1100:1100
|
||||
|
||||
redis:
|
||||
image: redis:8-alpine
|
||||
container_name: redis
|
||||
ports:
|
||||
- 63790:6379
|
||||
volumes:
|
||||
- redis:/data
|
||||
|
||||
minio:
|
||||
image: minio/minio
|
||||
container_name: minio
|
||||
|
|
@ -42,4 +50,5 @@ services:
|
|||
|
||||
volumes:
|
||||
minio:
|
||||
redis:
|
||||
documenso_database:
|
||||
|
|
|
|||
960
package-lock.json
generated
960
package-lock.json
generated
File diff suppressed because it is too large
Load diff
|
|
@ -5,9 +5,9 @@
|
|||
"description": "",
|
||||
"main": "index.js",
|
||||
"scripts": {
|
||||
"test:dev": "NODE_OPTIONS=--experimental-require-module playwright test",
|
||||
"test-ui:dev": "NODE_OPTIONS=--experimental-require-module playwright test --ui",
|
||||
"test:e2e": "NODE_OPTIONS=--experimental-require-module NODE_ENV=test NEXT_PRIVATE_LOGGER_FILE_PATH=./logs.json start-server-and-test \"npm run start -w @documenso/remix\" http://localhost:3000 \"playwright test $E2E_TEST_PATH\""
|
||||
"test:dev": "NODE_OPTIONS='--import tsx' playwright test",
|
||||
"test-ui:dev": "NODE_OPTIONS='--import tsx' playwright test --ui",
|
||||
"test:e2e": "NODE_OPTIONS='--import tsx' NODE_ENV=test NEXT_PRIVATE_LOGGER_FILE_PATH=./logs.json start-server-and-test \"npm run start -w @documenso/remix\" http://localhost:3000 \"playwright test $E2E_TEST_PATH\""
|
||||
},
|
||||
"keywords": [],
|
||||
"author": "",
|
||||
|
|
@ -18,6 +18,7 @@
|
|||
"@playwright/test": "1.56.1",
|
||||
"@types/node": "^20",
|
||||
"@types/pngjs": "^6.0.5",
|
||||
"tsx": "^4.20.6",
|
||||
"pixelmatch": "^7.1.0",
|
||||
"pngjs": "^7.0.0"
|
||||
},
|
||||
|
|
|
|||
387
packages/lib/jobs/client/bullmq.ts
Normal file
387
packages/lib/jobs/client/bullmq.ts
Normal file
|
|
@ -0,0 +1,387 @@
|
|||
import { createBullBoard } from '@bull-board/api';
|
||||
import { BullMQAdapter } from '@bull-board/api/bullMQAdapter';
|
||||
import { HonoAdapter } from '@bull-board/hono';
|
||||
import { serveStatic } from '@hono/node-server/serve-static';
|
||||
import { sha256 } from '@noble/hashes/sha2';
|
||||
import { BackgroundJobStatus, Prisma } from '@prisma/client';
|
||||
import { Queue, Worker } from 'bullmq';
|
||||
import type { Job } from 'bullmq';
|
||||
import { Hono } from 'hono';
|
||||
import type { Context as HonoContext } from 'hono';
|
||||
import IORedis from 'ioredis';
|
||||
import { createRequire } from 'node:module';
|
||||
import path from 'node:path';
|
||||
|
||||
import { prisma } from '@documenso/prisma';
|
||||
|
||||
import { env } from '../../utils/env';
|
||||
import type { JobDefinition, JobRunIO, SimpleTriggerJobOptions } from './_internal/job';
|
||||
import type { Json } from './_internal/json';
|
||||
import { BaseJobProvider } from './base';
|
||||
|
||||
const QUEUE_NAME = 'documenso-jobs';
|
||||
|
||||
const DEFAULT_CONCURRENCY = 10;
|
||||
const DEFAULT_MAX_RETRIES = 3;
|
||||
const DEFAULT_BACKOFF_DELAY = 1000;
|
||||
|
||||
declare global {
|
||||
// eslint-disable-next-line no-var
|
||||
var __documenso_bullmq_provider__: BullMQJobProvider | undefined;
|
||||
}
|
||||
|
||||
export class BullMQJobProvider extends BaseJobProvider {
|
||||
private _queue: Queue;
|
||||
private _worker: Worker;
|
||||
private _connection: IORedis;
|
||||
private _jobDefinitions: Record<string, JobDefinition> = {};
|
||||
|
||||
private constructor() {
|
||||
super();
|
||||
|
||||
const redisUrl = env('NEXT_PRIVATE_REDIS_URL');
|
||||
|
||||
if (!redisUrl) {
|
||||
throw new Error(
|
||||
'[JOBS]: NEXT_PRIVATE_REDIS_URL is required when using the BullMQ jobs provider',
|
||||
);
|
||||
}
|
||||
|
||||
const prefix = env('NEXT_PRIVATE_REDIS_PREFIX') || 'documenso';
|
||||
|
||||
this._connection = new IORedis(redisUrl, {
|
||||
maxRetriesPerRequest: null,
|
||||
});
|
||||
|
||||
this._queue = new Queue(QUEUE_NAME, {
|
||||
connection: this._connection,
|
||||
prefix,
|
||||
});
|
||||
|
||||
const concurrency = Number(env('NEXT_PRIVATE_BULLMQ_CONCURRENCY')) || DEFAULT_CONCURRENCY;
|
||||
|
||||
this._worker = new Worker(
|
||||
QUEUE_NAME,
|
||||
async (job: Job) => {
|
||||
await this.processJob(job);
|
||||
},
|
||||
{
|
||||
connection: this._connection,
|
||||
prefix,
|
||||
concurrency,
|
||||
},
|
||||
);
|
||||
|
||||
this._worker.on('failed', (job, error) => {
|
||||
console.error(`[JOBS]: Job ${job?.name ?? 'unknown'} failed`, error);
|
||||
});
|
||||
|
||||
this._worker.on('error', (error) => {
|
||||
console.error('[JOBS]: Worker error', error);
|
||||
});
|
||||
|
||||
console.log(`[JOBS]: BullMQ provider initialized (concurrency: ${concurrency})`);
|
||||
}
|
||||
|
||||
/**
|
||||
* Uses globalThis to store the singleton instance so that it's shared across
|
||||
* different bundles (e.g. Hono and Vite/React Router) at runtime.
|
||||
*/
|
||||
static getInstance() {
|
||||
if (globalThis.__documenso_bullmq_provider__) {
|
||||
return globalThis.__documenso_bullmq_provider__;
|
||||
}
|
||||
|
||||
const instance = new BullMQJobProvider();
|
||||
|
||||
globalThis.__documenso_bullmq_provider__ = instance;
|
||||
|
||||
return instance;
|
||||
}
|
||||
|
||||
public defineJob<N extends string, T>(definition: JobDefinition<N, T>) {
|
||||
this._jobDefinitions[definition.id] = {
|
||||
...definition,
|
||||
enabled: definition.enabled ?? true,
|
||||
};
|
||||
|
||||
if (definition.trigger.cron && definition.enabled !== false) {
|
||||
void this._queue
|
||||
.upsertJobScheduler(
|
||||
definition.id,
|
||||
{ pattern: definition.trigger.cron },
|
||||
{
|
||||
name: definition.id,
|
||||
data: {
|
||||
name: definition.trigger.name,
|
||||
payload: {},
|
||||
},
|
||||
opts: {
|
||||
attempts: DEFAULT_MAX_RETRIES,
|
||||
backoff: {
|
||||
type: 'exponential',
|
||||
delay: DEFAULT_BACKOFF_DELAY,
|
||||
},
|
||||
},
|
||||
},
|
||||
)
|
||||
.then(() => {
|
||||
console.log(`[JOBS]: Registered cron job ${definition.id} (${definition.trigger.cron})`);
|
||||
})
|
||||
.catch((error) => {
|
||||
console.error(`[JOBS]: Failed to register cron job ${definition.id}`, error);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
public async triggerJob(options: SimpleTriggerJobOptions) {
|
||||
const eligibleJobs = Object.values(this._jobDefinitions).filter(
|
||||
(job) => job.trigger.name === options.name,
|
||||
);
|
||||
|
||||
await Promise.all(
|
||||
eligibleJobs.map(async (job) => {
|
||||
const backgroundJob = await prisma.backgroundJob.create({
|
||||
data: {
|
||||
jobId: job.id,
|
||||
name: job.name,
|
||||
version: job.version,
|
||||
// eslint-disable-next-line @typescript-eslint/consistent-type-assertions
|
||||
payload: options.payload as Prisma.InputJsonValue,
|
||||
},
|
||||
});
|
||||
|
||||
await this._queue.add(
|
||||
job.id,
|
||||
{
|
||||
name: options.name,
|
||||
payload: options.payload,
|
||||
backgroundJobId: backgroundJob.id,
|
||||
},
|
||||
{
|
||||
jobId: options.id,
|
||||
attempts: DEFAULT_MAX_RETRIES,
|
||||
backoff: {
|
||||
type: 'exponential',
|
||||
delay: DEFAULT_BACKOFF_DELAY,
|
||||
},
|
||||
},
|
||||
);
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
public override getApiHandler(): (c: HonoContext) => Promise<Response | void> {
|
||||
const boardApp = this.createBoardApp();
|
||||
|
||||
return async (c: HonoContext) => {
|
||||
const reqPath = new URL(c.req.url).pathname;
|
||||
|
||||
if (!reqPath.startsWith('/api/jobs/board')) {
|
||||
return c.text('OK', 200);
|
||||
}
|
||||
|
||||
// Auth check — open in dev, admin-only in production.
|
||||
if (env('NODE_ENV') !== 'development') {
|
||||
const { getOptionalSession } = await import('@documenso/auth/server/lib/utils/get-session');
|
||||
const { isAdmin } = await import('../../utils/is-admin');
|
||||
|
||||
const { user } = await getOptionalSession(c);
|
||||
|
||||
if (!user || !isAdmin(user)) {
|
||||
return c.text('Unauthorized', 401);
|
||||
}
|
||||
}
|
||||
|
||||
return boardApp.fetch(c.req.raw);
|
||||
};
|
||||
}
|
||||
|
||||
private createBoardApp(): Hono {
|
||||
const _require = createRequire(import.meta.url);
|
||||
const uiPackagePath = path.dirname(_require.resolve('@bull-board/ui/package.json'));
|
||||
|
||||
const serverAdapter = new HonoAdapter(serveStatic);
|
||||
|
||||
createBullBoard({
|
||||
queues: [new BullMQAdapter(this._queue)],
|
||||
serverAdapter,
|
||||
options: { uiBasePath: uiPackagePath },
|
||||
});
|
||||
|
||||
serverAdapter.setBasePath('/api/jobs/board');
|
||||
|
||||
const app = new Hono();
|
||||
|
||||
app.route('/api/jobs/board', serverAdapter.registerPlugin());
|
||||
|
||||
return app;
|
||||
}
|
||||
|
||||
private async processJob(job: Job) {
|
||||
const definitionId = job.name;
|
||||
const definition = this._jobDefinitions[definitionId];
|
||||
|
||||
if (!definition) {
|
||||
console.error(`[JOBS]: No definition found for job ${definitionId}`);
|
||||
throw new Error(`No definition found for job ${definitionId}`);
|
||||
}
|
||||
|
||||
if (!definition.enabled) {
|
||||
console.log(`[JOBS]: Skipping disabled job ${definitionId}`);
|
||||
return;
|
||||
}
|
||||
|
||||
const jobData = job.data as {
|
||||
name: string;
|
||||
payload: unknown;
|
||||
backgroundJobId?: string;
|
||||
};
|
||||
|
||||
if (definition.trigger.schema) {
|
||||
const result = definition.trigger.schema.safeParse(jobData.payload);
|
||||
|
||||
if (!result.success) {
|
||||
console.error(`[JOBS]: Payload validation failed for ${definitionId}`, result.error);
|
||||
throw new Error(`Payload validation failed for ${definitionId}`);
|
||||
}
|
||||
}
|
||||
|
||||
const backgroundJobId = jobData.backgroundJobId;
|
||||
|
||||
if (backgroundJobId) {
|
||||
await prisma.backgroundJob
|
||||
.update({
|
||||
where: {
|
||||
id: backgroundJobId,
|
||||
status: BackgroundJobStatus.PENDING,
|
||||
},
|
||||
data: {
|
||||
status: BackgroundJobStatus.PROCESSING,
|
||||
retried: job.attemptsMade > 0 ? job.attemptsMade : 0,
|
||||
lastRetriedAt: job.attemptsMade > 0 ? new Date() : undefined,
|
||||
},
|
||||
})
|
||||
.catch(() => null);
|
||||
}
|
||||
|
||||
console.log(`[JOBS]: Processing job ${definitionId} with payload`, jobData.payload);
|
||||
|
||||
try {
|
||||
await definition.handler({
|
||||
payload: jobData.payload,
|
||||
io: this.createJobRunIO(backgroundJobId ?? job.id ?? definitionId),
|
||||
});
|
||||
|
||||
if (backgroundJobId) {
|
||||
await prisma.backgroundJob
|
||||
.update({
|
||||
where: { id: backgroundJobId },
|
||||
data: {
|
||||
status: BackgroundJobStatus.COMPLETED,
|
||||
completedAt: new Date(),
|
||||
},
|
||||
})
|
||||
.catch(() => null);
|
||||
}
|
||||
} catch (error) {
|
||||
if (backgroundJobId) {
|
||||
const isFinalAttempt = job.attemptsMade >= (job.opts.attempts ?? DEFAULT_MAX_RETRIES) - 1;
|
||||
|
||||
await prisma.backgroundJob
|
||||
.update({
|
||||
where: { id: backgroundJobId },
|
||||
data: {
|
||||
status: isFinalAttempt ? BackgroundJobStatus.FAILED : BackgroundJobStatus.PENDING,
|
||||
completedAt: isFinalAttempt ? new Date() : undefined,
|
||||
},
|
||||
})
|
||||
.catch(() => null);
|
||||
}
|
||||
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
private createJobRunIO(jobId: string): JobRunIO {
|
||||
return {
|
||||
runTask: async <T extends void | Json>(cacheKey: string, callback: () => Promise<T>) => {
|
||||
const hashedKey = Buffer.from(sha256(cacheKey)).toString('hex');
|
||||
|
||||
let task = await prisma.backgroundJobTask.findFirst({
|
||||
where: {
|
||||
id: `task-${hashedKey}--${jobId}`,
|
||||
jobId,
|
||||
},
|
||||
});
|
||||
|
||||
if (!task) {
|
||||
task = await prisma.backgroundJobTask.create({
|
||||
data: {
|
||||
id: `task-${hashedKey}--${jobId}`,
|
||||
name: cacheKey,
|
||||
jobId,
|
||||
status: BackgroundJobStatus.PENDING,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
if (task.status === BackgroundJobStatus.COMPLETED) {
|
||||
// eslint-disable-next-line @typescript-eslint/consistent-type-assertions
|
||||
return task.result as T;
|
||||
}
|
||||
|
||||
if (task.retried >= DEFAULT_MAX_RETRIES) {
|
||||
throw new Error('Task exceeded retries');
|
||||
}
|
||||
|
||||
try {
|
||||
const result = await callback();
|
||||
|
||||
await prisma.backgroundJobTask.update({
|
||||
where: {
|
||||
id: task.id,
|
||||
jobId,
|
||||
},
|
||||
data: {
|
||||
status: BackgroundJobStatus.COMPLETED,
|
||||
result: result === null ? Prisma.JsonNull : result,
|
||||
completedAt: new Date(),
|
||||
},
|
||||
});
|
||||
|
||||
return result;
|
||||
} catch (err) {
|
||||
await prisma.backgroundJobTask.update({
|
||||
where: {
|
||||
id: task.id,
|
||||
jobId,
|
||||
},
|
||||
data: {
|
||||
status: BackgroundJobStatus.PENDING,
|
||||
retried: {
|
||||
increment: 1,
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
console.log(`[JOBS:${task.id}] Task failed`, err);
|
||||
|
||||
throw err;
|
||||
}
|
||||
},
|
||||
triggerJob: async (_cacheKey, payload) => await this.triggerJob(payload),
|
||||
logger: {
|
||||
debug: (...args) => console.debug(`[${jobId}]`, ...args),
|
||||
error: (...args) => console.error(`[${jobId}]`, ...args),
|
||||
info: (...args) => console.info(`[${jobId}]`, ...args),
|
||||
log: (...args) => console.log(`[${jobId}]`, ...args),
|
||||
warn: (...args) => console.warn(`[${jobId}]`, ...args),
|
||||
},
|
||||
// eslint-disable-next-line @typescript-eslint/require-await
|
||||
wait: async () => {
|
||||
throw new Error('Not implemented');
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
|
|
@ -3,6 +3,7 @@ import { match } from 'ts-pattern';
|
|||
import { env } from '../../utils/env';
|
||||
import type { JobDefinition, TriggerJobOptions } from './_internal/job';
|
||||
import type { BaseJobProvider as JobClientProvider } from './base';
|
||||
import { BullMQJobProvider } from './bullmq';
|
||||
import { InngestJobProvider } from './inngest';
|
||||
import { LocalJobProvider } from './local';
|
||||
|
||||
|
|
@ -12,6 +13,7 @@ export class JobClient<T extends ReadonlyArray<JobDefinition> = []> {
|
|||
public constructor(definitions: T) {
|
||||
this._provider = match(env('NEXT_PRIVATE_JOBS_PROVIDER'))
|
||||
.with('inngest', () => InngestJobProvider.getInstance())
|
||||
.with('bullmq', () => BullMQJobProvider.getInstance())
|
||||
.otherwise(() => LocalJobProvider.getInstance());
|
||||
|
||||
definitions.forEach((definition) => {
|
||||
|
|
|
|||
|
|
@ -23,6 +23,9 @@
|
|||
"@aws-sdk/cloudfront-signer": "^3.998.0",
|
||||
"@aws-sdk/s3-request-presigner": "^3.998.0",
|
||||
"@aws-sdk/signature-v4-crt": "^3.998.0",
|
||||
"@bull-board/api": "^6.20.6",
|
||||
"@bull-board/hono": "^6.20.6",
|
||||
"@bull-board/ui": "^6.20.6",
|
||||
"@documenso/assets": "*",
|
||||
"@documenso/email": "*",
|
||||
"@documenso/prisma": "*",
|
||||
|
|
@ -41,8 +44,10 @@
|
|||
"@team-plain/typescript-sdk": "^5.11.0",
|
||||
"@vvo/tzdb": "^6.196.0",
|
||||
"ai": "^5.0.104",
|
||||
"bullmq": "^5.71.1",
|
||||
"csv-parse": "^6.1.0",
|
||||
"inngest": "^3.45.1",
|
||||
"ioredis": "^5.10.1",
|
||||
"jose": "^6.1.2",
|
||||
"konva": "^10.0.9",
|
||||
"kysely": "0.28.8",
|
||||
|
|
|
|||
9
packages/tsconfig/process-env.d.ts
vendored
9
packages/tsconfig/process-env.d.ts
vendored
|
|
@ -78,10 +78,17 @@ declare namespace NodeJS {
|
|||
|
||||
NEXT_PRIVATE_BROWSERLESS_URL?: string;
|
||||
|
||||
NEXT_PRIVATE_JOBS_PROVIDER?: 'inngest' | 'local';
|
||||
NEXT_PRIVATE_JOBS_PROVIDER?: 'inngest' | 'local' | 'bullmq';
|
||||
|
||||
NEXT_PUBLIC_USE_INTERNAL_URL_BROWSERLESS?: string;
|
||||
|
||||
/**
|
||||
* Redis / BullMQ environment variables
|
||||
*/
|
||||
NEXT_PRIVATE_REDIS_URL?: string;
|
||||
NEXT_PRIVATE_REDIS_PREFIX?: string;
|
||||
NEXT_PRIVATE_BULLMQ_CONCURRENCY?: string;
|
||||
|
||||
/**
|
||||
* Inngest environment variables
|
||||
*/
|
||||
|
|
|
|||
|
|
@ -115,6 +115,9 @@
|
|||
"NEXT_PRIVATE_GITHUB_TOKEN",
|
||||
"NEXT_PRIVATE_BROWSERLESS_URL",
|
||||
"NEXT_PRIVATE_JOBS_PROVIDER",
|
||||
"NEXT_PRIVATE_REDIS_URL",
|
||||
"NEXT_PRIVATE_REDIS_PREFIX",
|
||||
"NEXT_PRIVATE_BULLMQ_CONCURRENCY",
|
||||
"NEXT_PRIVATE_INNGEST_APP_ID",
|
||||
"INNGEST_EVENT_KEY",
|
||||
"NEXT_PRIVATE_INNGEST_EVENT_KEY",
|
||||
|
|
|
|||
Loading…
Reference in a new issue