console/packages/services/usage-ingestor
2025-09-26 20:21:17 +09:00
..
__tests__ Make organization non-optional in usage-ingestor (#5064) 2024-07-10 13:51:35 +02:00
src feat: hash operation helper (#6876) 2025-07-10 10:21:02 +02:00
.env.template run integration tests locally against local running services (#5096) 2024-07-15 11:21:23 +00:00
.gitignore Hello 2022-05-18 09:26:57 +02:00
package.json fix(deps): update dependency zod to v3.25.76 (#6991) 2025-09-14 16:16:15 +03:00
README.md fix: prometheus port documentation and expose (#3710) 2024-02-01 12:53:21 +00:00
tsconfig.json TS skipLibCheck everywhere, starting from root tsconfig (#588) 2022-11-03 15:00:58 +00:00
turbo.json chore: add dev:hive script using turbo (#7053) 2025-09-26 20:21:17 +09:00

@hive/usage-ingestor

This service takes care of feeding usage data into the ClickHouse instance.

Configuration

Name Required Description Example Value
ENVIRONMENT No The environment of your Hive app. (Note: This will be used for Sentry reporting.) staging
KAFKA_TOPIC Yes The kafka topic. usage_reports_v2
KAFKA_BROKER Yes The address of the Kafka broker. 127.0.0.1:29092
KAFKA_CONCURRENCY Yes The concurrency of the Kafka connection. 3
KAFKA_SSL No Whether an SSL connection should be established to the kafka service. 1 (enabled) or 0 (disabled)
KAFKA_SSL_CA_PATH No Refer to TLS create secure context for more information ./path_to_ca
KAFKA_SSL_CERT_PATH No Refer to TLS create secure context for more information ./path_to_cert
KAFKA_SSL_KEY_PATH No Refer to TLS create secure context for more information ./path_to_key
KAFKA_SASL_MECHANISM No The mechanism used for doing SASL authentication plain or scram-sha-256 or scram-sha-512
KAFKA_SASL_USERNAME No (Yes, if KAFKA_SASL_MECHANISM is defined) The username for the SASL authentication letmein
KAFKA_SASL_PASSWORD No (Yes, if KAFKA_SASL_MECHANISM is defined) Whether an SSL connection should be established to the kafka service. letmein
CLICKHOUSE_PROTOCOL Yes The ClickHouse protocol. http or https
CLICKHOUSE_HOST Yes The ClickHouse host. 127.0.0.1
CLICKHOUSE_PORT Yes The ClickHouse port. 8443
CLICKHOUSE_USERNAME Yes The username for accessing ClickHouse. letmein
CLICKHOUSE_PASSWORD Yes The password for accessing ClickHouse. letmein
HEARTBEAT_ENDPOINT No The endpoint for a heartbeat. http://127.0.0.1:6969/heartbeat
SENTRY No Whether Sentry error reporting should be enabled. 1 (enabled) or 0 (disabled)
SENTRY_DSN No (Yes if SENTRY is defined) The DSN for reporting errors to Sentry. https://dooobars@o557896.ingest.sentry.io/12121212
PROMETHEUS_METRICS No Whether Prometheus metrics should be enabled 1 (enabled) or 0 (disabled)
PROMETHEUS_METRICS_LABEL_INSTANCE No The instance label added for the prometheus metrics. usage-ingestor
PROMETHEUS_METRICS_PORT No Port on which prometheus metrics are exposed Defaults to 10254
REQUEST_LOGGING No Log http requests 1 (enabled) or 0 (disabled)
LOG_LEVEL No The verbosity of the service logs. One of trace, debug, info, warn ,error, fatal or silent info (default)