From 22f8477446ea96fe2aa4841c889fa4a9e9ab674e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Th=C3=A9ophile=20Diot?= Date: Thu, 8 Aug 2024 18:54:42 +0100 Subject: [PATCH] Start adding documentation about 1.6 specifications --- docs/concepts.md | 85 +++++++++++++++++++++++++++++++++++++++++++++++ docs/settings.md | 50 +++++++++++++++------------- docs/upgrading.md | 32 +++++++++++++++++- 3 files changed, 142 insertions(+), 25 deletions(-) diff --git a/docs/concepts.md b/docs/concepts.md index 313ce7743..510879a6f 100644 --- a/docs/concepts.md +++ b/docs/concepts.md @@ -146,3 +146,88 @@ In essence, the scheduler serves as the brain of BunkerWeb, orchestrating variou Depending on the integration approach, the execution environment of the scheduler may differ. In container-based integrations, the scheduler is executed within its dedicated container, providing isolation and flexibility. On the other hand, for Linux-based integrations, the scheduler is self-contained within the bunkerweb service, simplifying the deployment and management process. By employing the scheduler, BunkerWeb streamlines the automation and coordination of essential tasks, enabling efficient and reliable operation of the entire system. + +## Templates + +BunkerWeb leverages the power of templates to simplify the configuration process and enhance flexibility. Templates provide a structured and standardized approach to defining settings and custom configurations, ensuring consistency and ease of use. + +- **Predefined templates**: The community version offers three predefined templates that encapsulate common custom configurations and settings. These templates serve as a starting point for configuring BunkerWeb, enabling quick setup and deployment. The predefined templates are the following: + - **low**: A basic template that provides essential settings for web application protection. + - **medium**: A balanced template that offers a mix of security features and performance optimizations. + - **high**: An advanced template that focuses on robust security measures and comprehensive protection. + +- **Custom templates**: In addition to predefined templates, BunkerWeb allows users to create custom templates tailored to their specific requirements. Custom templates enable fine-tuning of settings and custom configurations, ensuring that BunkerWeb aligns perfectly with the user's needs. + +### Creating custom templates + +Creating a custom template is a straightforward process that involves defining the desired settings, custom configurations and steps in a structured format. + +* **Template structure**: A custom template consists of a name, series of settings, custom configurations and optional steps. The template structure is defined in a JSON file that adheres to the specified format. The key components of a custom template include: + * **Settings**: A setting is defined with a name and corresponding value. This value will override the default value of the setting. **Only multisite settings are supported.** + * **Configs**: A custom configuration is a path to an NGINX configuration file that will be included as a custom configuration. To know where to place the custom configuration file, refer to the example of a plugin's tree below. **Only multisite configuration types are supported.** + * **Steps**: A step contains a title, subtitle, settings and custom configurations. Each step represents a configuration step that the user can follow to set up BunkerWeb according to the custom template in the web UI. + +!!! info "Specifications about steps" + + If steps are declared, **it is not mandatory to include all the settings and custom configurations in the settings and configs sections**. keep in mind that when a setting or a custom configuration is declared in a step, the user will be allowed to make edits to it in the web UI. + +* **Template file**: The custom template is defined in a json file in a `templates` folder inside the plugin directory that adheres to the specified structure. The template file contains a name, the settings, custom configurations, and steps required to configure BunkerWeb according to the user's preferences. + +* **Selecting a template**: Once the custom template is defined, users can select it during the easy-mode configuration process of a service in the web UI. A template can also be selected with the `USE_TEMPLATE` setting in the configuration. The name of the template file (without the `.json` extension) should be specified as the value of the `USE_TEMPLATE` setting. + +Example of a custom template file: +```json +{ + "name": "template name", + // optional + "settings": { + "SETTING_1": "value", + "SETTING_2": "value" + }, + // optional + "configs": [ + "modsec/false_positives.conf", + "modsec/non_editable.conf", + "modsec-crs/custom_rules.conf" + ], + // optional + "steps": [ + { + "title": "Title 1", + "subtitle": "subtitle 1", + "settings": [ + "SETTING_1" + ], + "configs": [ + "modsec-crs/custom_rules.conf" + ] + }, + { + "title": "Title 2", + "subtitle": "subtitle 2", + "settings": [ + "SETTING_2" + ], + "configs": [ + "modsec/false_positives.conf" + ] + } + ] +} +``` + +Example of a plugin's tree including custom templates: +```tree +. +├── plugin.json +└── templates + ├── my_other_template.json + ├── my_template + │   └── configs + │   ├── modsec + │   │   ├── false_positives.conf + │   │   └── non_editable.conf + │   └── modsec-crs + │   └── custom_rules.conf + └── my_template.json +``` diff --git a/docs/settings.md b/docs/settings.md index cdbf3207e..75bea22a4 100644 --- a/docs/settings.md +++ b/docs/settings.md @@ -51,6 +51,7 @@ STREAM support :warning: |`IS_DRAFT` |`no` |multisite|no |Internal use : set to yes when the service is in draft mode. | |`TIMERS_LOG_LEVEL` |`debug` |global |no |Log level for timers. | |`BUNKERWEB_INSTANCES` |`127.0.0.1` |global |no |List of BunkerWeb instances separated with spaces (format : fqdn-or-ip:5000 http://fqdn-or-ip:5000)| +|`USE_TEMPLATE` | |multisite|no |Config template to use that will override the default values of specific settings. | ## Antibot @@ -253,11 +254,12 @@ STREAM support :white_check_mark: Integrate easily the Database. -| Setting | Default |Context|Multiple| Description | -|-----------------------|-----------------------------------------|-------|--------|-----------------------------------------------------------------------------------------------------------------------------------------| -|`DATABASE_URI` |`sqlite:////var/lib/bunkerweb/db.sqlite3`|global |no |The database URI, following the sqlalchemy format. | -|`DATABASE_URI_READONLY`| |global |no |The database URI for read-only operations, it can also serve as a fallback if the main database is down. Following the sqlalchemy format.| -|`DATABASE_LOG_LEVEL` |`warning` |global |no |The level to use for database logs. | +| Setting | Default |Context|Multiple| Description | +|------------------------|-----------------------------------------|-------|--------|-----------------------------------------------------------------------------------------------------------------------------------------| +|`DATABASE_URI` |`sqlite:////var/lib/bunkerweb/db.sqlite3`|global |no |The database URI, following the sqlalchemy format. | +|`DATABASE_URI_READONLY` | |global |no |The database URI for read-only operations, it can also serve as a fallback if the main database is down. Following the sqlalchemy format.| +|`DATABASE_LOG_LEVEL` |`warning` |global |no |The level to use for database logs. | +|`DATABASE_MAX_JOBS_RUNS`|`10000` |global |no |The maximum number of jobs runs to keep in the database. | ## DNSBL @@ -332,23 +334,22 @@ STREAM support :x: Manage HTTP headers sent to clients. -| Setting | Default | Context |Multiple| Description | -|-------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|--------|------------------------------------------------------------------------------------------------------------------------------------------------------| -|`CUSTOM_HEADER` | |multisite|yes |Custom header to add (HeaderName: HeaderValue). | -|`REMOVE_HEADERS` |`Server Expect-CT X-Powered-By X-AspNet-Version X-AspNetMvc-Version Public-Key-Pins` |multisite|no |Headers to remove (Header1 Header2 Header3 ...) | -|`KEEP_UPSTREAM_HEADERS` |`Content-Security-Policy Permissions-Policy X-Frame-Options` |multisite|no |Headers to keep from upstream (Header1 Header2 Header3 ... or * for all). | -|`STRICT_TRANSPORT_SECURITY` |`max-age=31536000; includeSubDomains; preload` |multisite|no |Value for the Strict-Transport-Security header. | -|`COOKIE_FLAGS` |`* HttpOnly SameSite=Lax` |multisite|yes |Cookie flags automatically added to all cookies (value accepted for nginx_cookie_flag_module). | -|`COOKIE_AUTO_SECURE_FLAG` |`yes` |multisite|no |Automatically add the Secure flag to all cookies. | -|`CONTENT_SECURITY_POLICY` |`object-src 'none'; form-action 'self'; frame-ancestors 'self';` |multisite|no |Value for the Content-Security-Policy header. | -|`CONTENT_SECURITY_POLICY_REPORT_ONLY`|`no` |multisite|no |Send reports for violations of the Content-Security-Policy header instead of blocking them. | -|`REFERRER_POLICY` |`strict-origin-when-cross-origin` |multisite|no |Value for the Referrer-Policy header. | -|`PERMISSIONS_POLICY` |`accelerometer=(), ambient-light-sensor=(), attribution-reporting=(), autoplay=(), battery=(), bluetooth=(), browsing-topics=(), camera=(), compute-pressure=(), display-capture=(), document-domain=(), encrypted-media=(), execution-while-not-rendered=(), execution-while-out-of-viewport=(), fullscreen=(), gamepad=(), geolocation=(), gyroscope=(), hid=(), identity-credentials-get=(), idle-detection=(), local-fonts=(), magnetometer=(), microphone=(), midi=(), otp-credentials=(), payment=(), picture-in-picture=(), publickey-credentials-create=(), publickey-credentials-get=(), screen-wake-lock=(), serial=(), speaker-selection=(), storage-access=(), usb=(), web-share=(), window-management=(), xr-spatial-tracking=()`|multisite|no |Value for the Permissions-Policy header. | -|`DISABLE_FLOC` |`yes` |multisite|no |Disable FLoC (Federated Learning of Cohorts) by adding the interest-cohort=() directive to the Permissions-Policy header if it is not already present.| -|`X_FRAME_OPTIONS` |`SAMEORIGIN` |multisite|no |Value for the X-Frame-Options header. | -|`X_CONTENT_TYPE_OPTIONS` |`nosniff` |multisite|no |Value for the X-Content-Type-Options header. | -|`X_XSS_PROTECTION` |`1; mode=block` |multisite|no |Value for the X-XSS-Protection header. | -|`X_DNS_PREFETCH_CONTROL` |`off` |multisite|no |Value for the X-DNS-Prefetch-Control header. | +| Setting | Default | Context |Multiple| Description | +|-------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|--------|----------------------------------------------------------------------------------------------| +|`CUSTOM_HEADER` | |multisite|yes |Custom header to add (HeaderName: HeaderValue). | +|`REMOVE_HEADERS` |`Server Expect-CT X-Powered-By X-AspNet-Version X-AspNetMvc-Version Public-Key-Pins` |multisite|no |Headers to remove (Header1 Header2 Header3 ...) | +|`KEEP_UPSTREAM_HEADERS` |`Content-Security-Policy Permissions-Policy X-Frame-Options` |multisite|no |Headers to keep from upstream (Header1 Header2 Header3 ... or * for all). | +|`STRICT_TRANSPORT_SECURITY` |`max-age=31536000; includeSubDomains; preload` |multisite|no |Value for the Strict-Transport-Security header. | +|`COOKIE_FLAGS` |`* HttpOnly SameSite=Lax` |multisite|yes |Cookie flags automatically added to all cookies (value accepted for nginx_cookie_flag_module).| +|`COOKIE_AUTO_SECURE_FLAG` |`yes` |multisite|no |Automatically add the Secure flag to all cookies. | +|`CONTENT_SECURITY_POLICY` |`object-src 'none'; form-action 'self'; frame-ancestors 'self';` |multisite|no |Value for the Content-Security-Policy header. | +|`CONTENT_SECURITY_POLICY_REPORT_ONLY`|`no` |multisite|no |Send reports for violations of the Content-Security-Policy header instead of blocking them. | +|`REFERRER_POLICY` |`strict-origin-when-cross-origin` |multisite|no |Value for the Referrer-Policy header. | +|`PERMISSIONS_POLICY` |`accelerometer=(), ambient-light-sensor=(), attribution-reporting=(), autoplay=(), battery=(), bluetooth=(), browsing-topics=(), camera=(), compute-pressure=(), display-capture=(), document-domain=(), encrypted-media=(), execution-while-not-rendered=(), execution-while-out-of-viewport=(), fullscreen=(), gamepad=(), geolocation=(), gyroscope=(), hid=(), identity-credentials-get=(), idle-detection=(), local-fonts=(), magnetometer=(), microphone=(), midi=(), otp-credentials=(), payment=(), picture-in-picture=(), publickey-credentials-create=(), publickey-credentials-get=(), screen-wake-lock=(), serial=(), speaker-selection=(), storage-access=(), usb=(), web-share=(), window-management=(), xr-spatial-tracking=(), interest-cohort=()`|multisite|no |Value for the Permissions-Policy header. | +|`X_FRAME_OPTIONS` |`SAMEORIGIN` |multisite|no |Value for the X-Frame-Options header. | +|`X_CONTENT_TYPE_OPTIONS` |`nosniff` |multisite|no |Value for the X-Content-Type-Options header. | +|`X_XSS_PROTECTION` |`1; mode=block` |multisite|no |Value for the X-XSS-Protection header. | +|`X_DNS_PREFETCH_CONTROL` |`off` |multisite|no |Value for the X-DNS-Prefetch-Control header. | ## Let's Encrypt @@ -395,6 +396,7 @@ Limit maximum number of requests and connections. |`USE_LIMIT_CONN` |`yes` |multisite|no |Activate limit connections feature. | |`LIMIT_CONN_MAX_HTTP1` |`10` |multisite|no |Maximum number of connections per IP when using HTTP/1.X protocol. | |`LIMIT_CONN_MAX_HTTP2` |`100` |multisite|no |Maximum number of streams per IP when using HTTP/2 protocol. | +|`LIMIT_CONN_MAX_HTTP3` |`100` |multisite|no |Maximum number of streams per IP when using HTTP/3 protocol. | |`LIMIT_CONN_MAX_STREAM`|`10` |multisite|no |Maximum number of connections per IP when using stream. | ## Metrics @@ -434,7 +436,7 @@ Miscellaneous settings. |`ROOT_FOLDER` | |multisite|no |Root folder containing files to serve (/var/www/html/{server_name} if unset). | |`SSL_PROTOCOLS` |`TLSv1.2 TLSv1.3` |multisite|no |The supported version of TLS. We recommend the default value TLSv1.2 TLSv1.3 for compatibility reasons. | |`HTTP2` |`yes` |multisite|no |Support HTTP2 protocol when HTTPS is enabled. | -|`HTTP3` |`no` |multisite|no |Support HTTP3 protocol when HTTPS is enabled. | +|`HTTP3` |`yes` |multisite|no |Support HTTP3 protocol when HTTPS is enabled. | |`HTTP3_ALT_SVC_PORT` |`443` |multisite|no |HTTP3 alternate service port. This value will be used as part of the Alt-Svc header. | |`LISTEN_HTTP` |`yes` |multisite|no |Respond to (insecure) HTTP requests. | |`USE_OPEN_FILE_CACHE` |`no` |multisite|no |Enable open file cache feature | @@ -523,7 +525,7 @@ Get real IP of clients when BunkerWeb is behind a reverse proxy / load balancer. | Setting | Default | Context |Multiple| Description | |--------------------|-----------------------------------------|---------|--------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |`USE_REAL_IP` |`no` |multisite|no |Retrieve the real IP of client. | -|`USE_PROXY_PROTOCOL`|`no` |multisite|no |Enable PROXY protocol communication. | +|`USE_PROXY_PROTOCOL`|`no` |global |no |Enable PROXY protocol communication. | |`REAL_IP_FROM` |`192.168.0.0/16 172.16.0.0/12 10.0.0.0/8`|multisite|no |List of trusted IPs / networks, separated with spaces, where proxied requests come from. | |`REAL_IP_HEADER` |`X-Forwarded-For` |multisite|no |HTTP header containing the real IP or special value proxy_protocol for PROXY protocol. | |`REAL_IP_RECURSIVE` |`yes` |multisite|no |Perform a recursive search in the header container IP address. | diff --git a/docs/upgrading.md b/docs/upgrading.md index 68035ee55..222f0f7ce 100644 --- a/docs/upgrading.md +++ b/docs/upgrading.md @@ -1,6 +1,6 @@ # Upgrading -## Upgrade from 1.5.X +## Upgrade from 1.6.X !!! warning "Read me first" @@ -376,6 +376,36 @@ 6. **Downgrade BunkerWeb**. - Downgrade BunkerWeb to the previous version by following the same steps as when upgrading BunkerWeb in the [integration Linux page](integrations.md#linux) +## Upgrade from 1.5.X + +### Scheduler + +Unlike the 1.5.X releases, the Scheduler service **no longer uses the *docker socket proxy* to fetch BunkerWeb's instances**. Instead, it uses the new `BUNKERWEB_INSTANCES` environment variable. + +!!! info "About the `BUNKERWEB_INSTANCES` environment variable" + + This new variable is a list of BunkerWeb instances separated by spaces in this format: `http://bunkerweb:5000 bunkerweb1:5000 bunkerweb2 ...`. The scheduler will then use this list to fetch the instances' configuration and to send the configuration to them. + + * The `http://` prefix is optional. + * The port is optional and defaults to the value of the `API_HTTP_PORT` environment variable. + * The default value of the `BUNKERWEB_INSTANCES` environment variable is `127.0.0.1`. + +!!! tip "Autoconf/Swarm/Kubernetes integrations" + + If you are using the `Autoconf`, `Swarm`, or `Kubernetes` integrations, you can set the `BUNKERWEB_INSTANCES` environment variable to an empty string (so that it doesn't try to send the configuration to the default one which is `127.0.0.1`). + + **The instances will be automatically fetched by the controller**. You can also add custom instances to the list that may not be picked up by the controller. + +### BunkerWeb container + +Another important change is that the **settings** that were previously declared on the BunkerWeb container **are now declared on the scheduler**. This means that you'll have to move your settings from the BunkerWeb container to the Scheduler container. + +While the settings are now declared on the Scheduler container, **you'll still need to declare api related mandatory settings on the BunkerWeb container** like the `API_WHITELIST_IP` setting which is used to whitelist the Scheduler's IP address, so that it can send the configuration to the instance. + +!!! warning "BunkerWeb's container settings" + + Every api related setting that you declare on the BunkerWeb container **have to be mirrored on the Scheduler container** so that it keeps working, as the configuration will be overwritten by the Scheduler's generated configuration. + ## Upgrade from 1.4.X !!! warning "Read this if you were a 1.4.X user"