Grafana is an open-source observability and visualization platform for exploring metrics, traces, and logs. ToolJet integrates with Grafana using OpenTelemetry by exporting telemetry data to an OpenTelemetry Collector, which then forwards the data to Grafana-supported storage backends. This guide walks through setting up observability using Grafana.
ToolJet does not send telemetry data directly to Grafana. Instead, it uses OpenTelemetry to export traces and metrics to an OpenTelemetry Collector running in your environment. The collector then forwards this data to Grafana-supported storage backends.
The observability flow consists of the following components:
- ToolJet: Generates traces and metrics using OpenTelemetry.
- OpenTelemetry Collector: Receives and processes telemetry locally.
- Tempo: Stores distributed traces.
- Prometheus: Stores metrics.
- Grafana: Visualizes traces and metrics via dashboards and Explore views.
## Setting Up Observability Using Grafana
### Step 1: Set Up an OpenTelemetry Collector
Use the following configuration file to run an OpenTelemetry Collector that forwards telemetry data to Tempo and Prometheus.
```yaml
# otel-collector-config.yml
receivers:
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
include_metadata: true
processors:
batch:
exporters:
# Export traces to Tempo
otlp/tempo:
endpoint: tempo:4317
tls:
insecure: true
# Export metrics for Prometheus to scrape
prometheus:
endpoint: "0.0.0.0:8889"
extensions:
health_check:
pprof:
endpoint: :1888
zpages:
endpoint: :55679
service:
extensions: [health_check, pprof, zpages]
pipelines:
traces:
receivers: [otlp]
processors: [batch]
exporters: [otlp/tempo]
metrics:
receivers: [otlp]
processors: [batch]
exporters: [prometheus]
```
### Step 2: Run the Observability Stack
Run the OpenTelemetry Collector along with Tempo, Prometheus, and Grafana by adding the following services to your `docker-compose.yml`.
These configuration files are mounted into their respective containers by Docker Compose. If you are running the services outside Docker, place these files according to your service manager or deployment setup.
#### Configure Tempo (Trace Storage)
Create a `tempo.yml` file with the following configuration:
```yaml
server:
http_listen_port: 3200
distributor:
receivers:
otlp:
protocols:
grpc:
http:
storage:
trace:
backend: local
local:
path: /tmp/tempo
wal:
path: /tmp/tempo/wal
```
#### Configure Prometheus (Metrics Storage)
Create a `prometheus.yml` file with the following configuration:
```yaml
global:
scrape_interval: 15s
scrape_configs:
- job_name: "otel-collector"
static_configs:
- targets: ["otel-collector:8889"]
```
### Step 4: Configure Environment Variables in ToolJet
Set the following environment variables in the .env file used by ToolJet:
Navigate to Connections > Data Sources, then click Add data source. You’ll create two data sources: one for metrics (Prometheus) and one for traces (Tempo).
