console/packages/services/usage/README.md

# `@hive/usage`

This service takes care of handling HTTP requests for usage reporting.

The data is written to a Kafka broker, form Kafka the data is feed into clickhouse via the
`usage-ingestor` service.

## Configuration

| Name                                | Required | Description                                                                                                                                          | Example Value                                        |
| ----------------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- |
| `PORT`                              | No       | The port this service is running on.                                                                                                                 | `4001`                                               |
| `TOKENS_ENDPOINT`                   | **Yes**  | The endpoint of the tokens service.                                                                                                                  | `http://127.0.0.1:6001`                              |
| `COMMERCE_ENDPOINT`                 | No       | The endpoint of the commerce service.                                                                                                                | `http://127.0.0.1:4012`                              |
| `KAFKA_TOPIC`                       | **Yes**  | The kafka topic.                                                                                                                                     | `usage_reports_v2`                                   |
| `KAFKA_CONSUMER_GROUP`              | **Yes**  | The kafka consumer group.                                                                                                                            | `usage_reports_v2`                                   |
| `KAFKA_BROKER`                      | **Yes**  | The address of the Kafka broker.                                                                                                                     | `127.0.0.1:29092`                                    |
| `KAFKA_SSL`                         | No       | Whether an SSL connection should be established to the kafka service.                                                                                | `1` (enabled) or `0` (disabled)                      |
| `KAFKA_SSL_CA_PATH`                 | No       | Refer to [TLS create secure context](https://nodejs.org/dist/latest-v8.x/docs/api/tls.html#tls_tls_createsecurecontext_options) for more information | `./path_to_ca`                                       |
| `KAFKA_SSL_CERT_PATH`               | No       | Refer to [TLS create secure context](https://nodejs.org/dist/latest-v8.x/docs/api/tls.html#tls_tls_createsecurecontext_options) for more information | `./path_to_cert`                                     |
| `KAFKA_SSL_KEY_PATH`                | No       | Refer to [TLS create secure context](https://nodejs.org/dist/latest-v8.x/docs/api/tls.html#tls_tls_createsecurecontext_options) for more information | `./path_to_key`                                      |
| `KAFKA_SASL_MECHANISM`              | No       | The mechanism used for doing SASL authentication                                                                                                     | `plain` or `scram-sha-256` or `scram-sha-512`        |
| `KAFKA_SASL_USERNAME`               | No       | The username for the SASL authentication                                                                                                             | `letmein`                                            |
| `KAFKA_SASL_PASSWORD`               | No       | Whether an SSL connection should be established to the kafka service.                                                                                | `letmein`                                            |
| `KAFKA_BUFFER_SIZE`                 | No       | The buffer size ???                                                                                                                                  | `12`                                                 |
| `KAFKA_BUFFER_INTERVAL`             | No       | The buffer interval ???                                                                                                                              | `1`                                                  |
| `KAFKA_BUFFER_DYNAMIC`              | No       | The buffer interval ???                                                                                                                              | `1`                                                  |
| `POSTGRES_SSL`                      | No       | Whether the postgres connection should be established via SSL.                                                                                       | `1` (enabled) or `0` (disabled)                      |
| `POSTGRES_HOST`                     | **Yes**  | Host of the postgres database                                                                                                                        | `127.0.0.1`                                          |
| `POSTGRES_PORT`                     | **Yes**  | Port of the postgres database                                                                                                                        | `5432`                                               |
| `POSTGRES_DB`                       | **Yes**  | Name of the postgres database.                                                                                                                       | `registry`                                           |
| `POSTGRES_USER`                     | **Yes**  | User name for accessing the postgres database.                                                                                                       | `postgres`                                           |
| `POSTGRES_PASSWORD`                 | No       | Password for accessing the postgres database.                                                                                                        | `postgres`                                           |
| `REDIS_HOST`                        | **Yes**  | The host of your redis instance.                                                                                                                     | `"127.0.0.1"`                                        |
| `REDIS_PORT`                        | **Yes**  | The port of your redis instance.                                                                                                                     | `6379`                                               |
| `REDIS_PASSWORD`                    | **Yes**  | The password of your redis instance.                                                                                                                 | `"apollorocks"`                                      |
| `REDIS_TLS_ENABLED`                 | **No**   | Enable TLS for redis connection (rediss://).                                                                                                         | `"0"`                                                |
| `ENVIRONMENT`                       | No       | The environment of your Hive app. (**Note:** This will be used for Sentry reporting.)                                                                | `staging`                                            |
| `SENTRY_DSN`                        | No       | The DSN for reporting errors to Sentry.                                                                                                              | `https://dooobars@o557896.ingest.sentry.io/12121212` |
| `SENTRY`                            | No       | Whether Sentry error reporting should be enabled.                                                                                                    | `1` (enabled) or `0` (disabled)                      |
| `PROMETHEUS_METRICS`                | No       | Whether Prometheus metrics should be enabled                                                                                                         | `1` (enabled) or `0` (disabled)                      |
| `PROMETHEUS_METRICS_LABEL_INSTANCE` | No       | The instance label added for the prometheus metrics.                                                                                                 | `usage-service`                                      |
| `PROMETHEUS_METRICS_PORT`           | No       | Port on which prometheus metrics are exposed                                                                                                         | Defaults to `10254`                                  |
| `REQUEST_LOGGING`                   | No       | Log http requests                                                                                                                                    | `1` (enabled) or `0` (disabled)                      |
| `LOG_LEVEL`                         | No       | The verbosity of the service logs. One of `trace`, `debug`, `info`, `warn` ,`error`, `fatal` or `silent`                                             | `info` (default)                                     |
| `OPENTELEMETRY_COLLECTOR_ENDPOINT`  | No       | OpenTelemetry Collector endpoint. The expected traces transport is HTTP (port `4318`).                                                               | `http://localhost:4318/v1/traces`                    |
feat: clean app env (#422) * feat: document environment variables * more env documentation * wip * add noop for backend env * typo * feat: embed environment validation/parsing in built app * fix the sentry integration 😌 * feat: use env * feat: decode the usage service environment * feat: decode the webhooks service environment * feat: disallow process.env * feat: decode the tokens service environment * feat: decode the stripe-billing service environment * feat: decode server service environment * feat: decode schema service environment * feat: decode rate-limit service environment * feat: decode usage-estimator service environment * feat: decode emails service environment * adjust env * remove commented out code * adjust env variable name * use separate env variables * env fixes * more environmental stuff :) * ... * replace POSTGRES_CONNECTION_STRING with specific environment variables * handle optional clickhouse (for now :) * add missing POSTGRES_DB environment variable * make ENVIRONMENT optional * the other matters lol * feat: support delivering mails via SMTP (#412) * feat: optional CDN (#410) * feat: optional CDN * enable CDN in deployment * enable the CDN in integration tests * add sendmail provider * remove unused env variables from the deployment * only show login alert when legacy auth0 migration is enabled * feat: make TOKENS_ENDPOINT mandatory and RATE_LIMIT_ENDPOINT optional for usage service * feat: upgrade supertokens and enable server side email confirmation (#423) * feat: upgrade supertokens and enable server side email confirmation * feat: opt into required email verification * docs: self hosting (#428) * docs: self-hosting quick start * Update packages/web/docs/pages/self-hosting/get-started.mdx Co-authored-by: Kamil Kisiela <kamil.kisiela@gmail.com> Co-authored-by: Kamil Kisiela <kamil.kisiela@gmail.com> Co-authored-by: Kamil Kisiela <kamil.kisiela@gmail.com> * feat: optional GitHub and Slack environment variables (#440) * feat: optional github integration environment variables * feat: optional slack integration (#441) * use latest stable supertokens Co-authored-by: Kamil Kisiela <kamil.kisiela@gmail.com> 2022-10-04 12:30:21 +00:00			# `@hive/usage`

			`This service takes care of handling HTTP requests for usage reporting.`

Update dependency @theguild/prettier-config to v1 (#676) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Kamil Kisiela <kamil.kisiela@gmail.com> 2022-11-24 10:00:41 +00:00			`The data is written to a Kafka broker, form Kafka the data is feed into clickhouse via the`
			`usage-ingestor` service.
feat: clean app env (#422) * feat: document environment variables * more env documentation * wip * add noop for backend env * typo * feat: embed environment validation/parsing in built app * fix the sentry integration 😌 * feat: use env * feat: decode the usage service environment * feat: decode the webhooks service environment * feat: disallow process.env * feat: decode the tokens service environment * feat: decode the stripe-billing service environment * feat: decode server service environment * feat: decode schema service environment * feat: decode rate-limit service environment * feat: decode usage-estimator service environment * feat: decode emails service environment * adjust env * remove commented out code * adjust env variable name * use separate env variables * env fixes * more environmental stuff :) * ... * replace POSTGRES_CONNECTION_STRING with specific environment variables * handle optional clickhouse (for now :) * add missing POSTGRES_DB environment variable * make ENVIRONMENT optional * the other matters lol * feat: support delivering mails via SMTP (#412) * feat: optional CDN (#410) * feat: optional CDN * enable CDN in deployment * enable the CDN in integration tests * add sendmail provider * remove unused env variables from the deployment * only show login alert when legacy auth0 migration is enabled * feat: make TOKENS_ENDPOINT mandatory and RATE_LIMIT_ENDPOINT optional for usage service * feat: upgrade supertokens and enable server side email confirmation (#423) * feat: upgrade supertokens and enable server side email confirmation * feat: opt into required email verification * docs: self hosting (#428) * docs: self-hosting quick start * Update packages/web/docs/pages/self-hosting/get-started.mdx Co-authored-by: Kamil Kisiela <kamil.kisiela@gmail.com> Co-authored-by: Kamil Kisiela <kamil.kisiela@gmail.com> Co-authored-by: Kamil Kisiela <kamil.kisiela@gmail.com> * feat: optional GitHub and Slack environment variables (#440) * feat: optional github integration environment variables * feat: optional slack integration (#441) * use latest stable supertokens Co-authored-by: Kamil Kisiela <kamil.kisiela@gmail.com> 2022-10-04 12:30:21 +00:00
			`## Configuration`

add optional ssl options for kafka client (#2757) (#2758) Co-authored-by: Saurav Tapader <stapader@expediagroup.com> 2023-08-22 14:38:47 +00:00			`\| Name \| Required \| Description \| Example Value \|`
			`\| ----------------------------------- \| -------- \| ---------------------------------------------------------------------------------------------------------------------------------------------------- \| ---------------------------------------------------- \|`
			\| `PORT` \| No \| The port this service is running on. \| `4001` \|
			\| `TOKENS_ENDPOINT` \| Yes \| The endpoint of the tokens service. \| `http://127.0.0.1:6001` \|
Replaces rate-limit, stripe-billing and usage-estimator with commerce (#6540) 2025-02-24 10:41:01 +00:00			\| `COMMERCE_ENDPOINT` \| No \| The endpoint of the commerce service. \| `http://127.0.0.1:4012` \|
add optional ssl options for kafka client (#2757) (#2758) Co-authored-by: Saurav Tapader <stapader@expediagroup.com> 2023-08-22 14:38:47 +00:00			\| `KAFKA_TOPIC` \| Yes \| The kafka topic. \| `usage_reports_v2` \|
			\| `KAFKA_CONSUMER_GROUP` \| Yes \| The kafka consumer group. \| `usage_reports_v2` \|
			\| `KAFKA_BROKER` \| Yes \| The address of the Kafka broker. \| `127.0.0.1:29092` \|
			\| `KAFKA_SSL` \| No \| Whether an SSL connection should be established to the kafka service. \| `1` (enabled) or `0` (disabled) \|
			\| `KAFKA_SSL_CA_PATH` \| No \| Refer to [TLS create secure context](https://nodejs.org/dist/latest-v8.x/docs/api/tls.html#tls_tls_createsecurecontext_options) for more information \| `./path_to_ca` \|
			\| `KAFKA_SSL_CERT_PATH` \| No \| Refer to [TLS create secure context](https://nodejs.org/dist/latest-v8.x/docs/api/tls.html#tls_tls_createsecurecontext_options) for more information \| `./path_to_cert` \|
			\| `KAFKA_SSL_KEY_PATH` \| No \| Refer to [TLS create secure context](https://nodejs.org/dist/latest-v8.x/docs/api/tls.html#tls_tls_createsecurecontext_options) for more information \| `./path_to_key` \|
			\| `KAFKA_SASL_MECHANISM` \| No \| The mechanism used for doing SASL authentication \| `plain` or `scram-sha-256` or `scram-sha-512` \|
			\| `KAFKA_SASL_USERNAME` \| No \| The username for the SASL authentication \| `letmein` \|
			\| `KAFKA_SASL_PASSWORD` \| No \| Whether an SSL connection should be established to the kafka service. \| `letmein` \|
			\| `KAFKA_BUFFER_SIZE` \| No \| The buffer size ??? \| `12` \|
			\| `KAFKA_BUFFER_INTERVAL` \| No \| The buffer interval ??? \| `1` \|
			\| `KAFKA_BUFFER_DYNAMIC` \| No \| The buffer interval ??? \| `1` \|
feat(usage): report with organization access tokens (#6538) Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> 2025-03-06 01:19:37 +00:00			\| `POSTGRES_SSL` \| No \| Whether the postgres connection should be established via SSL. \| `1` (enabled) or `0` (disabled) \|
			\| `POSTGRES_HOST` \| Yes \| Host of the postgres database \| `127.0.0.1` \|
			\| `POSTGRES_PORT` \| Yes \| Port of the postgres database \| `5432` \|
			\| `POSTGRES_DB` \| Yes \| Name of the postgres database. \| `registry` \|
			\| `POSTGRES_USER` \| Yes \| User name for accessing the postgres database. \| `postgres` \|
			\| `POSTGRES_PASSWORD` \| No \| Password for accessing the postgres database. \| `postgres` \|
			\| `REDIS_HOST` \| Yes \| The host of your redis instance. \| `"127.0.0.1"` \|
			\| `REDIS_PORT` \| Yes \| The port of your redis instance. \| `6379` \|
			\| `REDIS_PASSWORD` \| Yes \| The password of your redis instance. \| `"apollorocks"` \|
			\| `REDIS_TLS_ENABLED` \| No \| Enable TLS for redis connection (rediss://). \| `"0"` \|
add optional ssl options for kafka client (#2757) (#2758) Co-authored-by: Saurav Tapader <stapader@expediagroup.com> 2023-08-22 14:38:47 +00:00			\| `ENVIRONMENT` \| No \| The environment of your Hive app. (Note: This will be used for Sentry reporting.) \| `staging` \|
			\| `SENTRY_DSN` \| No \| The DSN for reporting errors to Sentry. \| `https://dooobars@o557896.ingest.sentry.io/12121212` \|
			\| `SENTRY` \| No \| Whether Sentry error reporting should be enabled. \| `1` (enabled) or `0` (disabled) \|
			\| `PROMETHEUS_METRICS` \| No \| Whether Prometheus metrics should be enabled \| `1` (enabled) or `0` (disabled) \|
			\| `PROMETHEUS_METRICS_LABEL_INSTANCE` \| No \| The instance label added for the prometheus metrics. \| `usage-service` \|
fix: prometheus port documentation and expose (#3710) 2024-02-01 12:53:21 +00:00			\| `PROMETHEUS_METRICS_PORT` \| No \| Port on which prometheus metrics are exposed \| Defaults to `10254` \|
add optional ssl options for kafka client (#2757) (#2758) Co-authored-by: Saurav Tapader <stapader@expediagroup.com> 2023-08-22 14:38:47 +00:00			\| `REQUEST_LOGGING` \| No \| Log http requests \| `1` (enabled) or `0` (disabled) \|
docs: missing environment variables and troubleshooting page for self-hosting (#3480) 2023-11-28 12:04:34 +00:00			\| `LOG_LEVEL` \| No \| The verbosity of the service logs. One of `trace`, `debug`, `info`, `warn` ,`error`, `fatal` or `silent` \| `info` (default) \|
feat: open telemetry tracing for `usage` service (#6351) 2025-01-17 09:40:18 +00:00			\| `OPENTELEMETRY_COLLECTOR_ENDPOINT` \| No \| OpenTelemetry Collector endpoint. The expected traces transport is HTTP (port `4318`). \| `http://localhost:4318/v1/traces` \|