](https://internews.org/)_Today we wanted to feature [Josh](https://defensivedepth.com/), a member of our community. His work was sponsored by [Internews](https://internews.org/). If you are interested in contributing to the Fleet blog, feel free to [contact us](https://fleetdm.com/company/contact) or reach out to [@jdstrong](https://osquery.slack.com/team/U04MTPBAHQS) on the osquery slack._
+
+This guide provides a detailed walkthrough for setting up a small production environment of Fleet alongside Elastic components (Elasticsearch, Kibana, Filebeat). The setup integrates Filebeat to collect scheduled query results from Fleet and feed them into Elasticsearch, while Kibana will be utilized for data visualization and the creation of detections. Additionally, Nginx will serve as a reverse proxy for the Kibana and Fleet web interfaces and will segregate the web administration and agent data+control planes of Fleet for more fine-grained access control.
+
+The installation and configuration will begin with the Elastic stack components, followed by Fleet and its dependencies. For this guide, they will all be installed on a single server; however, for larger deployments or requirements of higher availability and scalability, a more distributed approach across multiple servers and geographical regions is recommended.
+
+### Network, server & DNS setup
+
+This guide is based on Ubuntu 22.04 LTS, although the installation procedures for the components remain consistent across newer versions of the operating system.
+
+For this guide, subdomain `fleet.localhost.invalid` is pointed to the server's public IP. Replace this subdomain with a valid one configured as such.
+
+Ports needed, inbound to server:
+- `TCP/80` (Only used for the initial Let's Encrypt setup)
+- `TCP/443` (Used initially for the Let's Encrypt setup, and then longterm for Fleet distributed agents to checking for data and control)
+- `TCP/8443` (Used for Kibana web interface)
+- `TCP/9443` (Used for Fleet web interface)
+
+Set up access control where it makes sense - perimeter firewall or on the server itself. Set the ports for the Kibana (`TCP/8443`) and Fleet (`TCP/9443`) web interfaces to only be accessible from a known-trusted IP space. Also set rules for `TCP/443`, which is used for the deployed osquery agents to check in with Fleet. A common configuration is for the web interface ports to be accessible to a single IP or small set of IPs, and for the osquery check in port to be accessible anywhere.
+
+Be aware that if you are using a proxy like Cloudflare, you will need to confirm that the ports in this guide will work as expected.
+
+### Update OS
+
+Let's start by updating the system's packages and creating a workspace directory:
+
+```sh
+sudo apt-get update && sudo apt-get dist-upgrade -y
+mkdir workspace && cd workspace
+```
+
+### Install & configure Certbot
+
+Next up is to install Certbot to create and manage our free Let's Encrypt SSL certificate. This certificate will be used by for all components.
+
+```sh
+sudo apt-get install certbot -y
+sudo certbot certonly --standalone
+```
+
+Select option 1 to spin up a temporary web server. Enter the domain that you have pointed to your public IP. You will need TCP/80 & TCP/443 open to the server.
+
+By default, the certificate and key are saved at:
+
+- Certificate: `/etc/letsencrypt/live/fleet.localhost.invalid/fullchain.pem`
+- Key: `/etc/letsencrypt/live/fleet.localhost.invalid/privkey.pem`
+
+### Install & configure Nginx
+
+Let's install Nginx and configure it as a reverse proxy for Fleet and Kibana.
+
+```sh
+sudo apt-get install nginx
+nano /etc/nginx/sites-available/fleet # use the below config, remember to update the path to the certificate files
+sudo ln -s /etc/nginx/sites-available/fleet /etc/nginx/sites-enabled/ # symlink the config file to enable it
+nginx -t # Test the config to make sure there are no syntax errors
+sudo systemctl reload nginx # Reload nginx to make the config active
+sudo systemctl status nginx # Check the reload to confirm that there are no errors
+```
+Nginx Config file:
+```sh
+# Define SSL configuration
+ssl_certificate /etc/letsencrypt/live/fleet.localhost.invalid/fullchain.pem;
+ssl_certificate_key /etc/letsencrypt/live/fleet.localhost.invalid/privkey.pem;
+
+# Common proxy settings
+proxy_set_header Host $host;
+proxy_set_header X-Real-IP $remote_addr;
+proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+proxy_set_header X-Forwarded-Proto $scheme;
+
+# Server block for Kibana on port 8443
+server {
+ listen 8443 ssl default_server;
+
+ location / {
+ proxy_pass http://localhost:5601;
+ }
+}
+
+# Server block for Fleet on port 9443 with WebSocket support
+server {
+ listen 9443 ssl;
+ add_header Content-Security-Policy "default-src 'self' 'unsafe-inline' 'unsafe-eval' https: data: blob: wss:; frame-ancestors 'self'";
+
+ location / {
+ proxy_pass https://localhost:4443/;
+ proxy_read_timeout 300;
+ proxy_connect_timeout 300;
+ proxy_set_header Upgrade $http_upgrade;
+ proxy_set_header Connection "Upgrade";
+ }
+}
+
+# Server block for specific Orbit osquery paths on port 443
+server {
+ listen 443 ssl;
+
+ location ~* ^/api/(osquery|fleet/orbit/(config|ping)|v1/osquery) {
+ proxy_pass https://localhost:4443;
+ }
+}
+```
+
+
+### Install & configure Elasticsearch
+
+
+In case the below does not work, consult Debian package installation instructions at https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html
+
+Let's download and install Elasticsearch via an Ubuntu package.
+
+One-time prep needed to add the Elastic APT repository:
+```sh
+wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
+echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-8.x.list
+sudo apt-get update
+```
+
+Install the Elasticsearch package (this will install the latest stable version):
+
+```sh
+sudo apt-get install elasticsearch
+```
+The post-install message will contain a password generated for the Elasticsearch built-in superuser (`elastic`). Make note of it as we will need it later.
+
+Enable and start the Elasticsearch service:
+
+```sh
+sudo systemctl daemon-reload
+sudo systemctl enable --now elasticsearch.service
+```
+
+## Install & configure Kibana
+
+Onto Kibana. Let's download, install and do the initial configuration.
+
+```sh
+sudo apt-get install kibana
+```
+Before we start Kibana, we need to edit the configuration file:
+
+```sh
+nano /etc/kibana/kibana.yml
+```
+
+Set the server host and public base URL by uncommenting and editing the below lines:
+
+```yaml
+server.host: "0.0.0.0" # Sets Kibana to listen on all interfaces
+server.publicBaseUrl: "https://fleetmd.localhost.invalid:8443" # This should be set to your custom subdomain/port
+```
+
+Enable and start the Kibana service:
+
+```sh
+sudo /bin/systemctl daemon-reload
+sudo /bin/systemctl enable --now kibana.service
+```
+
+### Initial configuration
+
+Access Kibana at `https://fleet.localhost.invalid:8443`. If you get stuck at this step, you may not have opened ports 8443 and 9443, as needed in this walkthrough. Generate and enter the initial setup token and the verification code:
+
+```sh
+/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana
+/usr/share/kibana/bin/kibana-verification-code
+```
+
+From there, log in with the username `elastic` and the password that was generated previously, and choose `Explore on my own`. Navigate to `Management` -> `Stack Monitoring` and set up self-monitoring with `set up with self monitoring` and `Turn on monitoring`. This will give you a nice overview of Elasticsearch, Kibana and eventually Filebeat.
+
+## Install & configure Filebeat
+
+The final Elastic component to install is Filebeat. Let's download and configure it to pick up our osquery logs.
+
+```sh
+sudo apt-get install filebeat
+```
+
+Edit the Filebeat configuration to set up where to send its logs (Elasticsearch). We disable ssl.verification because the connection from Filebeat to Elasticsearch is local (from Filebeat on the server to Elasticsearch on the same system).
+Filebeat has built-in support for osquery logs. Let's configure and then enable that filebeat module and then start the Filebeat service:
+
+
+```sh
+sudo nano /etc/filebeat/modules.d/osquery.yml.disabled # Use the following config
+```
+
+```yaml
+# Module: osquery
+
+- module: osquery
+ result:
+ enabled: true
+
+ # Set custom paths for the log files. If left empty,
+ # Filebeat will choose the paths depending on your OS.
+ var.paths: ["/tmp/osquery_result"]
+
+ # If true, all fields created by this module are prefixed with
+ # `osquery.result`. Set to false to copy the fields in the root
+ # of the document. The default is true.
+ #var.use_namespace: true
+```
+
+
+```sh
+sudo filebeat modules enable osquery # Enable the Filebeat osquery module
+sudo /bin/systemctl daemon-reload
+sudo /bin/systemctl enable --now filebeat.service
+```
+
+## Install & configure MySQL
+
+With the Elastic components installed, we can move on to Fleet. First up is installing MySQL and creating the Fleet user and database.
+
+```sh
+sudo apt-get install mysql-server -y
+mysql -uroot
+create database fleet; # This is the database that will be used by Fleet
+create user fleet@'localhost' identified by 'FleetDMPW!'; # Create the mysql user for the Fleet database and set a strong password.
+grant all privileges on fleet.* to fleet@'localhost'; # Grant the new user the necessary privileges to the Fleet database.
+exit
+```
+
+## Install & configure Redis
+
+Redis is used for the Live Query functionality. Let's get it installed.
+
+```sh
+sudo apt-get install redis-server -y
+```
+
+## Install & configure Fleet
+
+Finally, the linchpin - Fleet. Let's download the latest version. You can find the latest version here: https://github.com/fleetdm/fleet/releases/latest - make sure you download the main Fleet package and not `fleetctl` at this time.
+
+```sh
+wget https://github.com/fleetdm/fleet/releases/download/fleet-$VERSION/fleet_$VERSION_linux.tar.gz
+tar -xf fleet_v*_linux.tar.gz # Extract the Fleet binary
+sudo cp fleet_v*_linux/fleet /usr/bin/ # Copy the the Fleet binary to /usr/bin
+fleet version # Sanity check to make sure it runs as expected
+```
+
+Next we will create the directory that will contain the config and installers, and create the config itself.
+
+```sh
+mkdir /etc/fleet
+nano /etc/fleet/fleet.config
+```
+
+Use the following as a baseline for your Fleet config:
+
+```yaml
+mysql:
+ address: 127.0.0.1:3306
+ database: fleet
+ username: fleet
+ password: FleetPW!
+redis:
+ address: 127.0.0.1:6379
+server:
+ address: 0.0.0.0:4443
+ cert: /etc/letsencrypt/live/fleet.localhost.invalid/fullchain.pem
+ key: /etc/letsencrypt/live/fleet.localhost.invalid/privkey.pem
+ websockets_allow_unsafe_origin: true # This is needed for Live Query functionality to work with the nginx reverse proxy we are using
+```
+
+Next, let's run the `prepare db` command to complete the necessary database prep.
+
+```sh
+fleet prepare db --config /etc/fleet/fleet.config
+```
+
+### Setup systemd unit file
+
+Now that we are ready to run Fleet, let's create a `systemd` unit file to manage Fleet as a service, and then go ahead and start the service:
+
+```sh
+sudo nano /etc/systemd/system/fleet.service # Use the example unit file below
+sudo systemctl enable --now fleet.service
+sudo systemctl status fleet.service
+```
+
+```sh
+[Unit]
+Description=fleet
+After=network.target
+
+[Service]
+ExecStart=/usr/bin/fleet serve -c /etc/fleet/fleet.config
+
+[Install]
+WantedBy=multi-user.target
+```
+
+
+Finally, complete the Fleet setup via the web interface at https://fleet.localhost.invalid:9443
+
+## fleetctl
+
+fleetctl is a utility from Fleet that is used to manage Fleet from the command line. Let's download it and get it logged into our instance of Fleet. You can find the latest version here: https://github.com/fleetdm/fleet/releases/latest
+
+```sh
+wget https://github.com/fleetdm/fleet/releases/download/fleet-$VERSION/fleetctl_$VERSION_linux.tar.gz
+tar -xf fleetctl_*_linux.tar.gz# Extract the fleetct binary
+sudo cp fleetctl_v*_linux/fleetctl /usr/bin/ # Copy the the fleetctl binary to /usr/bin
+/usr/bin/fleetctl --version # Sanity check to make sure it runs as expected
+```
+
+Next, we need to configure it to work with our local instance of Fleet and login to it.
+
+```sh
+fleetctl config set --address https://fleet.localhost.invalid::4443
+fleetctl login
+```
+
+## Generate agents
+
+Fleet ships with support for Orbit, a wrapper around osquery. Orbit makes configuration of osquery much simpler, offers auto-update functionality of osquery as well as additional tables developed by Fleet. In order to install an Orbit/osquery agent, you will need to generate an installer.
+
+You can start the process of generating Orbit agent packages from the Fleet interface - click on the `Add Hosts` button. You can generate the packages anywhere that you have `fleetctl`, including on the server itself. Be sure to install the Docker engine if you need to generate installers for Windows.
+
+## Load Fleet standard query library
+
+Fleet has a library of queries that are useful in many different situations - https://fleetdm.com/docs/using-fleet/standard-query-library
+
+Let's go ahead and load them - once this is complete, you can find them in the web interface under Queries.
+
+```sh
+git clone https://github.com/fleetdm/fleet.git
+cd fleet
+fleetctl apply -f docs/01-Using-Fleet/standard-query-library/standard-query-library.yml
+```
+
+
+
+
+
+
+
+
+
diff --git a/articles/fleet-4.51.0.md b/articles/fleet-4.51.0.md
new file mode 100644
index 0000000000..7f43b9f66b
--- /dev/null
+++ b/articles/fleet-4.51.0.md
@@ -0,0 +1,114 @@
+# Fleet 4.51.0 | Global activity webhook, macOS TCC table, and software self-service.
+
+
+
+Fleet 4.51.0 is live. Check out the full [changelog](https://github.com/fleetdm/fleet/releases/tag/fleet-v4.51.0) or continue reading to get the highlights.
+For upgrade instructions, see our [upgrade guide](https://fleetdm.com/docs/deploying/upgrading-fleet) in the Fleet docs.
+
+## Highlights
+
+* Global activity webhook
+* macOS TCC table
+* Software self-service
+* Simplified APNs and ABM token uploads
+
+
+## Global activity webhook
+
+Fleet adds webhook support for global activities, broadening automation and real-time notification capabilities. This feature allows IT administrators to set up webhooks triggered by specific events within Fleet, such as changes in MDM features or re-enrollment activities. This also supports reporting mechanisms, enabling administrators to monitor the alignment between the number of devices enrolled and employees onboarded.
+
+This update enhances operational efficiency by automating workflows and providing timely data, helping administrators manage device configurations and compliance more effectively. By leveraging webhooks for these critical events, Fleet ensures that administrators can maintain continuous oversight and respond swiftly to changes, ultimately bolstering the organization's device management and security frameworks.
+
+
+## macOS TCC table
+
+Fleet adds to its monitoring capabilities for macOS devices with support for querying the macOS TCC (Transparency, Consent, and Control) databases. This gives administrators valuable insights into applications' permissions on individual devices, particularly concerning accessing sensitive user data. The TCC framework is a critical component of macOS, designed to safeguard user privacy by managing app permissions across the system. With this update, Fleet enables IT teams to audit and verify that applications comply with organizational policies and privacy standards by accessing detailed, granular permission settings. This capability is essential for maintaining stringent security and privacy protocols, ensuring that only authorized applications can access sensitive information, and enhancing organizations' overall security posture by utilizing macOS within their fleets.
+
+
+## Software self-service
+
+Fleet aims to streamline the software installation process across organizations through software self-service. IT administrators can easily add software packages to Fleet and make them available for end-users to install via Fleet Desktop. Administrators can offer a curated list of pre-approved and organizationally vetted software directly to users, simplifying the installation process and ensuring compliance with organizational software standards. This addition not only empowers users by providing them with the autonomy to install necessary applications as needed but also ensures that all software deployed across the organization is secure and authorized, thereby maintaining high standards of IT security and operational efficiency.
+
+
+## Simplified APNs and ABM token uploads
+
+Fleet has simplified the integration of Apple Push Notification service (APNs) certificates and Apple Business Manager (ABM) tokens directly through its user interface. This update marks a significant shift from the previous requirement of using `fleetctl` commands and environmental variables for these tasks. IT administrators can effortlessly upload APNs certificates and ABM tokens via the Fleet UI, enhancing the setup process for managing Apple devices within their networks. This streamlined approach reduces the complexity of configuring necessary services for device management. It accelerates the deployment process, allowing administrators to focus more on strategic tasks than manual configurations. \
+
+
+For self-managed users, the integration of these certificates requires a server private key, which is essential for activating macOS MDM features within Fleet. See Fleet's documentation for guidance on [configuring a private key](https://fleetdm.com/learn-more-about/fleet-server-private-key), which provides detailed instructions and best practices.
+
+
+
+## Changes
+
+### Endpoint Operations
+- Added support for environment variables in configuration profiles for GitOps.
+- `fleetctl gitops --dry-run` now errors on duplicate (or conflicting) global/team enroll secrets.
+- Added `activities_webhook` configuration option to allow for a webhook to be called when an activity is recorded. This can be used to send activity data to external services. If the webhook response is a 429 error code, the webhook retries for up to 30 minutes.
+- Added Tuxedo OS to the Linux distribution platform list.
+
+### Device Management (MDM)
+- **NOTE:** Added new required Fleet server config environment variable when MDM is enabled,
+ `FLEET_SERVER_PRIVATE_KEY`. This variable contains the private key used to encrypt the MDM
+ certificates and keys stored in Fleet. Learm more at
+ https://fleetdm.com/learn-more-about/fleet-server-private-key.
+- Added MDM support for iPhone/iPad.
+- Added software self-service support.
+- Added query parameter `self_service` to filter the list of software titles and the list of a host's software so that only those available to install via self-service are returned.
+- Added the device-authenticated endpoint `POST /device/{token}/software/install/{software_title_id}` to self-install software.
+- Added new endpoints to configure ABM keypairs and tokens.
+- Added `GET /fleet/mdm/apple/request_csr` endpoint, which returns the signed APNS CSR needed to activate Apple MDM.
+- Added the ability to automatically log off and lock out `Administrator` users on Windows hosts.
+- Added clearer error messages when attempting to set up Apple MDM without a server private key configured.
+- Added UI for the global and host activities for self-service software installation.
+- Updated UI to support new workflows for macOS MDM setup and credentials.
+- Updated UI to support software self-service features.
+- Updated UI controls page language and hid CTA button for users without access to turn on MDM.
+
+### Vulnerability Management
+- Updated the CIS policies for Windows 11 Enterprise from v2.0.0 (03-07-2023) to v3.0.0 (02-22-2024).
+- Fleet now detects Ubuntu kernel vulnerabilities from the Canonical OVAL feed.
+- Fleet now detects and reports vulnerabilities on Firefox ESR editions on macOS.
+
+### Bug fixes and improvements
+- Fixed a bug that might prevent enqueuing commands to renew SCEP certificates if the host was enrolled more than once.
+- Prevented the `host_id`s field from being returned from the list labels endpoint.
+- Improved software ingestion performance by deduplicating incoming software.
+- Placed all form field label tooltips on top.
+- Fixed a number of related issues with the filtering and sorting of the queries table.
+- Added various optimizations to the rendering of the queries table.
+- Fixed host query page styling bugs.
+- Fixed a UI bug where "Wipe" action was not being hidden from observers.
+- Fixed UI bug for builtin label names for selecting targets.
+- Removed references to Administrator accounts in the comments of the Windows lock script.
+
+## Fleet 4.50.2 (May 31, 2024)
+
+### Bug fixes
+
+* Fixed a critical bug where S3 operation were not possible on a different AWS account.
+
+## Fleet 4.50.1 (May 29, 2024)
+
+### Bug fixes
+
+* Fixed a bug that might prevent enqueing commands to renew SCEP certificates if the host was enrolled more than once.
+* Fixed a bug by preventing the `host_id`s field from being returned from the list labels endpoint.
+* Fixed a number of related issues with the filtering and sorting of the queries table.
+* Added various optimizations to the rendering of the queries table.
+* Fixed a bug where Bulk Host Delete and Transfer now support status and labelID filters together.
+* Added the ability to automatically log off and lock out `Administrator` users on Windows hosts.
+* Removed references to Administrator accounts in the comments of the Windows lock script.
+
+
+
+## Ready to upgrade?
+
+Visit our [Upgrade guide](https://fleetdm.com/docs/deploying/upgrading-fleet) in the Fleet docs for instructions on updating to Fleet 4.51.0.
+
+
+
+
+
+
+
diff --git a/articles/what-api-endpoints-to-expose-to-the-public-internet.md b/articles/what-api-endpoints-to-expose-to-the-public-internet.md
index d022f0f1ff..0c24d923ae 100644
--- a/articles/what-api-endpoints-to-expose-to-the-public-internet.md
+++ b/articles/what-api-endpoints-to-expose-to-the-public-internet.md
@@ -61,7 +61,7 @@ If you would like to use Fleet's Windows MDM features, the following endpoints n
The `/api/*/fleet/*` endpoints accessed by the fleetd agent can use mTLS with the certificate provided via the `--fleet-tls-client-certificate` flag in the `fleetctl package` command.
-The `/mdm/apple/mdm` and `/api/mdm/apple/enroll` endpoints can use mTLS with the [SCEP certificate issued by the Fleet server](https://fleetdm.com/docs/configuration/fleet-server-configuration#mdm-apple-scep-cert-bytes).
+The `/mdm/apple/mdm` and `/api/mdm/apple/enroll` endpoints can use mTLS with the SCEP certificate issued by the Fleet server.
These endpoints don't use mTLS:
- `/mdm/apple/scep`
diff --git a/articles/windows-mdm-setup.md b/articles/windows-mdm-setup.md
index 89398b5df1..1fb36f3bf6 100644
--- a/articles/windows-mdm-setup.md
+++ b/articles/windows-mdm-setup.md
@@ -12,15 +12,14 @@ To use automatic enrollment (aka zero-touch) features on Windows, follow instruc
Fleet uses a certificate and key pair to authenticate and manage interactions between Fleet and Windows host.
-> If you're already using Fleet's macOS MDM features, you already have a SCEP certificate and key. Skip to step 2 and reuse the SCEP certificate and key as your WSTEP certificate and key.
+How to generate a certificate and key:
-If you're not using macOS MDM features, run the following command to download three files and send an email to you with an attached CSR file.
+1. With [OpenSSL](https://www.openssl.org/) installed, open your Terminal (macOS) or PowerShell (Windows) and run the following command to create a key: `openssl genrsa --traditional -out fleet-mdm-win-wstep.key 4096`.
-```
-fleetctl generate mdm-apple --email
- <>{val}} />
+
);
},
@@ -106,9 +106,7 @@ const defaultTableHeaders: IDataColumn[] = [
disableSortBy: true,
accessor: "windowsHosts",
Cell: ({ cell: { value: aggregateCount } }: ICellProps) => {
- return (
-
{name}
@@ -125,6 +125,7 @@ const PackageStatusCount = ({
position="top"
tipContent={displayData.tooltip}
underline={false}
+ showArrow
>
>>,
- potentialKey?: string
-) => {
- if (!potentialKey) {
- return null;
- }
+const SOFTWARE_ICON_SIZES: Record = {
+ medium: "24",
+ meduim_large: "64", // TODO: rename this to large and update large to xlarge
+ large: "96",
+} as const;
- const sanitizedKey = potentialKey.trim().toLowerCase();
- const match = Object.entries(map).find(([namePrefix, icon]) => {
- if (sanitizedKey.startsWith(namePrefix)) {
- return icon;
- }
- return null;
- });
-
- return match ? match[1] : null;
-};
+type SoftwareIconSizes = keyof typeof SOFTWARE_ICON_SIZES;
const SoftwareIcon = ({
- name,
- source,
+ name = "",
+ source = "",
size = "medium",
}: ISoftwareIconProps) => {
- // try to find a match for name
- let MatchedIcon = matchInMap(SOFTWARE_NAME_TO_ICON_MAP, name);
-
- // otherwise, try to find a match for source
- if (!MatchedIcon) {
- MatchedIcon = matchInMap(SOFTWARE_SOURCE_TO_ICON_MAP, source);
- }
-
- // default to 'package'
- if (!MatchedIcon) {
- MatchedIcon = SOFTWARE_SOURCE_TO_ICON_MAP.package;
- }
-
+ const MatchedIcon = getMatchedSoftwareIcon({ name, source });
return (
= {
- medium: "24",
- meduim_large: "64", // TODO: rename this to large and update large to xlarge
- large: "96",
-} as const;
+/**
+ * This attempts to loosely match the provided string to a key in a provided dictionary, returning the key if the
+ * provided string starts with the key or undefined otherwise.
+ */
+const matchLoosePrefixToKey = >(
+ dict: T,
+ s: string
+) => {
+ s = s.trim().toLowerCase();
+ if (!s) {
+ return undefined;
+ }
+ const match = Object.keys(dict).find((k) =>
+ s.startsWith(k.trim().toLowerCase())
+ );
-export type SoftwareIconSizes = keyof typeof SOFTWARE_ICON_SIZES;
+ return match ? (match as keyof T) : undefined;
+};
+
+/**
+ * This strictly matches the provided name and source to a software icon, returning the icon if a match is found or
+ * null otherwise. It is intended to be used for special cases where a strict match is required
+ * (e.g. Zoom). The caller should handle null cases by falling back to loose matching on name prefixes.
+ */
+const matchStrictNameSourceToIcon = ({
+ name = "",
+ source = "",
+}: Pick) => {
+ name = name.trim().toLowerCase();
+ source = source.trim().toLowerCase();
+ switch (true) {
+ case name === "zoom.us.app" && source === "apps":
+ return Zoom;
+ case name === "zoom":
+ return Zoom;
+ case name === "google chrome":
+ return ChromeApp;
+ default:
+ return null;
+ }
+};
+
+/**
+ * This returns the icon component for a given software name and source. If a strict match is found,
+ * it will be returned, otherwise it will fall back to loose matching on name and source prefixes.
+ * If no match is found, the default package icon will be returned.
+ */
+const getMatchedSoftwareIcon = ({
+ name = "",
+ source = "",
+}: Pick) => {
+ // first, try strict matching on name and source
+ let Icon = matchStrictNameSourceToIcon({
+ name,
+ source,
+ });
+
+ // if no match, try loose matching on name prefixes
+ if (!Icon) {
+ const matchedName = matchLoosePrefixToKey(SOFTWARE_NAME_TO_ICON_MAP, name);
+ if (matchedName) {
+ Icon = SOFTWARE_NAME_TO_ICON_MAP[matchedName];
+ }
+ }
+
+ // if still no match, try loose matching on source prefixes
+ if (!Icon) {
+ const matchedSource = matchLoosePrefixToKey(
+ SOFTWARE_SOURCE_TO_ICON_MAP,
+ source
+ );
+ if (matchedSource) {
+ Icon = SOFTWARE_SOURCE_TO_ICON_MAP[matchedSource];
+ }
+ }
+
+ // if still no match, default to 'package'
+ if (!Icon) {
+ Icon = SOFTWARE_SOURCE_TO_ICON_MAP.package;
+ }
+
+ return Icon;
+};
+
+export default getMatchedSoftwareIcon;
diff --git a/frontend/pages/admin/IntegrationsPage/cards/AutomaticEnrollment/AppleAutomaticEnrollmentPage/AppleAutomaticEnrollmentPage.tsx b/frontend/pages/admin/IntegrationsPage/cards/AutomaticEnrollment/AppleAutomaticEnrollmentPage/AppleAutomaticEnrollmentPage.tsx
index a91acb0771..89760f8cc3 100644
--- a/frontend/pages/admin/IntegrationsPage/cards/AutomaticEnrollment/AppleAutomaticEnrollmentPage/AppleAutomaticEnrollmentPage.tsx
+++ b/frontend/pages/admin/IntegrationsPage/cards/AutomaticEnrollment/AppleAutomaticEnrollmentPage/AppleAutomaticEnrollmentPage.tsx
@@ -189,7 +189,7 @@ const AppleAutomaticEnrollmentPage = ({
/>
If your organization doesn’t have an account, select{" "} - Enroll now. + Sign up now. diff --git a/frontend/pages/admin/IntegrationsPage/cards/Calendars/Calendars.tsx b/frontend/pages/admin/IntegrationsPage/cards/Calendars/Calendars.tsx index 019d5f192e..63573337d7 100644 --- a/frontend/pages/admin/IntegrationsPage/cards/Calendars/Calendars.tsx +++ b/frontend/pages/admin/IntegrationsPage/cards/Calendars/Calendars.tsx @@ -375,7 +375,7 @@ const Calendars = (): JSX.Element => {
For the OAuth scopes, paste the following value:
{
page.
}
- helpText="Only policy queries (SQL) are sent to the LLM. Fleet doesn’t use this data to train models."
+ helpText="If enabled, only policy queries (SQL) are sent to the LLM. Fleet doesn’t use this data to train models."
>
Disable generative AI features
diff --git a/frontend/pages/admin/UserManagementPage/components/UserForm/UserForm.tsx b/frontend/pages/admin/UserManagementPage/components/UserForm/UserForm.tsx
index eae662cee6..cf87b9f6b9 100644
--- a/frontend/pages/admin/UserManagementPage/components/UserForm/UserForm.tsx
+++ b/frontend/pages/admin/UserManagementPage/components/UserForm/UserForm.tsx
@@ -412,7 +412,7 @@ const UserForm = ({
onChange={onInputChange("email")}
placeholder="Email"
value={formData.email || ""}
- disabled={!isNewUser && !(smtpConfigured || sesConfigured)}
+ readOnly={!isNewUser && !(smtpConfigured || sesConfigured)}
tooltip={
<>
Editing an email address requires that SMTP or SES is configured in
@@ -450,7 +450,7 @@ const UserForm = ({
name="sso_enabled"
onChange={onCheckboxChange("sso_enabled")}
value={formData.sso_enabled}
- disabled={!canUseSso}
+ readOnly={!canUseSso}
wrapperClassName={`${baseClass}__invite-admin`}
helpText={
canUseSso ? (
@@ -473,11 +473,7 @@ const UserForm = ({
)
}
>
-
- Enable single sign-on
-
+ Enable single sign-on
)}
diff --git a/frontend/pages/admin/UserManagementPage/components/UserForm/_styles.scss b/frontend/pages/admin/UserManagementPage/components/UserForm/_styles.scss
index a2a2a41541..386ea703ca 100644
--- a/frontend/pages/admin/UserManagementPage/components/UserForm/_styles.scss
+++ b/frontend/pages/admin/UserManagementPage/components/UserForm/_styles.scss
@@ -3,20 +3,12 @@
.fleet-checkbox {
margin-top: 5px;
- &__tick--disabled {
- @include disabled;
- }
-
&__label {
font-size: $x-small;
font-weight: $bold;
color: $core-fleet-black;
}
}
-
- &--disabled {
- @include disabled;
- }
}
.sso-disabled {
diff --git a/frontend/pages/hosts/details/DeviceUserPage/DeviceUserPage.tsx b/frontend/pages/hosts/details/DeviceUserPage/DeviceUserPage.tsx
index b052371d50..d950153f48 100644
--- a/frontend/pages/hosts/details/DeviceUserPage/DeviceUserPage.tsx
+++ b/frontend/pages/hosts/details/DeviceUserPage/DeviceUserPage.tsx
@@ -452,6 +452,7 @@ const DeviceUserPage = ({
)}
diff --git a/frontend/pages/hosts/details/HostDetailsPage/HostDetailsPage.tsx b/frontend/pages/hosts/details/HostDetailsPage/HostDetailsPage.tsx
index df0405009c..741249f5cc 100644
--- a/frontend/pages/hosts/details/HostDetailsPage/HostDetailsPage.tsx
+++ b/frontend/pages/hosts/details/HostDetailsPage/HostDetailsPage.tsx
@@ -866,6 +866,7 @@ const HostDetailsPage = ({
diff --git a/frontend/pages/hosts/details/HostDetailsPage/modals/RunScriptModal/ScriptsTableConfig.tsx b/frontend/pages/hosts/details/HostDetailsPage/modals/RunScriptModal/ScriptsTableConfig.tsx
index 344c0c8b36..c870ab55e4 100644
--- a/frontend/pages/hosts/details/HostDetailsPage/modals/RunScriptModal/ScriptsTableConfig.tsx
+++ b/frontend/pages/hosts/details/HostDetailsPage/modals/RunScriptModal/ScriptsTableConfig.tsx
@@ -37,40 +37,10 @@ interface IDropdownCellProps {
};
}
-const ScriptRunActionDropdownLabel = ({
- scriptId,
- disabled,
-}: {
- scriptId: number;
- disabled: boolean;
-}) => {
- const tipId = `run-script-${scriptId}`;
- return disabled ? (
- <>
-
- Run
-
-
- Script is already running.
-
-
- ) : (
- <>Run
- );
-};
-
const generateActionDropdownOptions = (
currentUser: IUser | null,
teamId: number | null,
- { script_id, last_execution }: IHostScript
+ { last_execution }: IHostScript
): IDropdownOption[] => {
const hasRunPermission =
!!currentUser &&
@@ -89,14 +59,10 @@ const generateActionDropdownOptions = (
value: "showDetails",
},
{
- label: (
- = {
pass: {
@@ -65,12 +69,17 @@ const generatePolicyTableHeaders = (
disableSortBy: true,
Cell: (cellProps) => {
const { name } = cellProps.row.original;
+
+ const onClickPolicyName = (e: React.MouseEvent) => {
+ // Allows for button to be clickable in a clickable row
+ e.stopPropagation();
+ togglePolicyDetails(cellProps.row.original);
+ };
+
return (
If your organization doesn’t have an account, select{" "} - Enroll now. + Sign up now. diff --git a/frontend/pages/admin/IntegrationsPage/cards/Calendars/Calendars.tsx b/frontend/pages/admin/IntegrationsPage/cards/Calendars/Calendars.tsx index 019d5f192e..63573337d7 100644 --- a/frontend/pages/admin/IntegrationsPage/cards/Calendars/Calendars.tsx +++ b/frontend/pages/admin/IntegrationsPage/cards/Calendars/Calendars.tsx @@ -375,7 +375,7 @@ const Calendars = (): JSX.Element => {
{data.software.map((s) => {
- const key = `${s.id}${s.last_install?.install_uuid}`; // concatenating install_uuid so item updates with fresh data on refetch
+ // concatenating install_uuid so item updates with fresh data on refetch
+ const key = `${s.id}${s.last_install?.install_uuid}`;
return (
{
- return `${PATHS.MANAGE_HOSTS}?${buildQueryStringFromParams({
- policy_id: policyId,
- policy_response: policyResponse,
- team_id: teamId,
- })}`;
-};
-
const getPolicyRefreshTime = (ms: number): string => {
const seconds = ms / 1000;
if (seconds < 60) {
diff --git a/frontend/pages/queries/details/components/QueryReport/QueryReportTableConfig.tsx b/frontend/pages/queries/details/components/QueryReport/QueryReportTableConfig.tsx
index b46383ecf6..0294dffe5c 100644
--- a/frontend/pages/queries/details/components/QueryReport/QueryReportTableConfig.tsx
+++ b/frontend/pages/queries/details/components/QueryReport/QueryReportTableConfig.tsx
@@ -65,7 +65,7 @@ const generateReportColumnConfigsFromResults = (
isSortedDesc={headerProps.column.isSortedDesc}
/>
),
- accessor: key,
+ accessor: (data) => data[key],
Cell: (cellProps: ITableCellProps) => {
if (typeof cellProps.cell.value !== "string") return null;
diff --git a/frontend/pages/queries/edit/components/QueryResults/QueryResultsTableConfig.tsx b/frontend/pages/queries/edit/components/QueryResults/QueryResultsTableConfig.tsx
index fd29edceb4..3f45c702d6 100644
--- a/frontend/pages/queries/edit/components/QueryResults/QueryResultsTableConfig.tsx
+++ b/frontend/pages/queries/edit/components/QueryResults/QueryResultsTableConfig.tsx
@@ -50,7 +50,7 @@ const generateColumnConfigsFromRows = >(
isSortedDesc={headerProps.column.isSortedDesc}
/>
),
- accessor: colName,
+ accessor: (data) => data[colName],
Cell: (cellProps: CellProps) => {
const val = cellProps?.cell?.value;
return !!val?.length && val.length > 300
diff --git a/frontend/services/entities/device_user.ts b/frontend/services/entities/device_user.ts
index 33955a419d..11aead8495 100644
--- a/frontend/services/entities/device_user.ts
+++ b/frontend/services/entities/device_user.ts
@@ -10,6 +10,7 @@ export type ILoadHostDetailsExtension = "device_mapping" | "macadmins";
export interface IDeviceSoftwareQueryKey extends IHostSoftwareQueryParams {
scope: "device_software";
id: string;
+ softwareUpdatedAt?: string;
}
export interface IGetDeviceSoftwareResponse {
diff --git a/frontend/services/entities/hosts.ts b/frontend/services/entities/hosts.ts
index ac1f71673f..259d9ae9a2 100644
--- a/frontend/services/entities/hosts.ts
+++ b/frontend/services/entities/hosts.ts
@@ -169,6 +169,7 @@ export interface IHostSoftwareQueryParams extends QueryParams {
export interface IHostSoftwareQueryKey extends IHostSoftwareQueryParams {
scope: "host_software";
id: number;
+ softwareUpdatedAt?: string;
}
export type ILoadHostDetailsExtension = "device_mapping" | "macadmins";
@@ -439,7 +440,7 @@ export default {
},
loadHostDetails: (hostID: number) => {
const { HOSTS } = endpoints;
- const path = `${HOSTS}/${hostID}`;
+ const path = `${HOSTS}/${hostID}?exclude_software=true`;
return sendRequest("GET", path);
},
diff --git a/frontend/test/default-handlers.ts b/frontend/test/default-handlers.ts
index 1941481a8c..e9696a7f9b 100644
--- a/frontend/test/default-handlers.ts
+++ b/frontend/test/default-handlers.ts
@@ -9,6 +9,11 @@ export const baseUrl = (path: string) => {
return `/api/latest/fleet${path}`;
};
+// These are the default handlers that are used when testing the frontend. They
+// are used to mock the responses from the Fleet API when running tests.
+// These can be overridden in individual tests using the .use() method on the
+// mock server within the desired test.
+// More info on .use() here: https://mswjs.io/docs/api/setup-worker/use/
const handlers = [
defaultDeviceHandler,
defaultDeviceMappingHandler,
diff --git a/frontend/test/handlers/device-handler.ts b/frontend/test/handlers/device-handler.ts
index 306c67a01d..8b92a57c78 100644
--- a/frontend/test/handlers/device-handler.ts
+++ b/frontend/test/handlers/device-handler.ts
@@ -1,11 +1,14 @@
import { rest } from "msw";
-import createMockDeviceUser from "__mocks__/deviceUserMock";
+import createMockDeviceUser, {
+ createMockDeviceSoftwareResponse,
+} from "__mocks__/deviceUserMock";
import createMockHost from "__mocks__/hostMock";
import createMockLicense from "__mocks__/licenseMock";
import createMockMacAdmins from "__mocks__/macAdminsMock";
import { baseUrl } from "test/test-utils";
import { IDeviceUserResponse } from "interfaces/host";
+import { IGetDeviceSoftwareResponse } from "services/entities/device_user";
export const defaultDeviceHandler = rest.get(
baseUrl("/device/:token"),
@@ -64,3 +67,10 @@ export const defaultMacAdminsHandler = rest.get(
);
}
);
+
+export const customDeviceSoftwareHandler = (
+ overrides?: Partial
+) =>
+ rest.get(baseUrl("/device/:token/software"), (req, res, context) => {
+ return res(context.json(createMockDeviceSoftwareResponse(overrides)));
+ });
diff --git a/frontend/test/test-utils.tsx b/frontend/test/test-utils.tsx
index 4111300afc..ce7bde4298 100644
--- a/frontend/test/test-utils.tsx
+++ b/frontend/test/test-utils.tsx
@@ -1,4 +1,5 @@
import React from "react";
+import { InjectedRouter } from "react-router";
import { render, RenderOptions, RenderResult } from "@testing-library/react";
import type { UserEvent } from "@testing-library/user-event/dist/types/setup/setup";
import userEvent from "@testing-library/user-event";
@@ -151,3 +152,22 @@ export const renderWithSetup = (component: JSX.Element) => {
...render(component),
};
};
+
+const DEFAULT_MOCK_ROUTER: InjectedRouter = {
+ push: jest.fn(),
+ replace: jest.fn(),
+ goBack: jest.fn(),
+ goForward: jest.fn(),
+ go: jest.fn(),
+ setRouteLeaveHook: jest.fn(),
+ isActive: jest.fn(),
+ createHref: jest.fn(),
+ createPath: jest.fn(),
+};
+
+export const createMockRouter = (overrides?: Partial) => {
+ return {
+ ...DEFAULT_MOCK_ROUTER,
+ ...overrides,
+ };
+};
diff --git a/frontend/utilities/endpoints.ts b/frontend/utilities/endpoints.ts
index c1dc7057d9..d096fd4a0a 100644
--- a/frontend/utilities/endpoints.ts
+++ b/frontend/utilities/endpoints.ts
@@ -1,5 +1,3 @@
-import software from "interfaces/software";
-
const API_VERSION = "latest";
export default {
diff --git a/frontend/utilities/helpers.tsx b/frontend/utilities/helpers.tsx
index b84f8c3fa7..f050ef02c4 100644
--- a/frontend/utilities/helpers.tsx
+++ b/frontend/utilities/helpers.tsx
@@ -19,7 +19,6 @@ import {
intlFormat,
intervalToDuration,
isAfter,
- isBefore,
addDays,
} from "date-fns";
import yaml from "js-yaml";
@@ -41,6 +40,7 @@ import {
import { ITeam } from "interfaces/team";
import { UserRole } from "interfaces/user";
+import PATHS from "router/paths";
import stringUtils from "utilities/strings";
import sortUtils from "utilities/sort";
import { checkTable } from "utilities/sql_tools";
@@ -54,6 +54,7 @@ import {
INITIAL_FLEET_DATE,
PLATFORM_LABEL_DISPLAY_TYPES,
isPlatformLabelNameFromAPI,
+ PolicyResponse,
} from "utilities/constants";
import { ISchedulableQueryStats } from "interfaces/schedulable_query";
import { IDropdownOption } from "interfaces/dropdownOption";
@@ -91,6 +92,18 @@ export const addGravatarUrlToResource = (resource: any): any => {
};
};
+export const createHostsByPolicyPath = (
+ policyId: number,
+ policyResponse: PolicyResponse,
+ teamId?: number | null
+) => {
+ return `${PATHS.MANAGE_HOSTS}?${buildQueryStringFromParams({
+ policy_id: policyId,
+ policy_response: policyResponse,
+ team_id: teamId,
+ })}`;
+};
+
const labelSlug = (label: ILabel): string => {
const { id, name } = label;
@@ -965,6 +978,7 @@ export function getCustomDropdownOptions(
export default {
addGravatarUrlToResource,
+ createHostsByPolicyPath,
formatConfigDataForServer,
formatLabelResponse,
formatFloatAsPercentage,
diff --git a/handbook/business-operations/README.md b/handbook/business-operations/README.md
index 597c10f149..5fb48ff42d 100644
--- a/handbook/business-operations/README.md
+++ b/handbook/business-operations/README.md
@@ -37,7 +37,7 @@ Recurring monthly or annual expenses, such as the tools we use throughout Fleet,
### Access a background check
-Fleet team members with access to Fleet's infrastructure undergo a background check provided through [Vetty](https://vetty.co/). Only the most recent background checks appear on the home page of Vetty's dashboard. To access a complete list of background checks run in Vetty, scroll down to the bottom of the candidates page and click "View Historical".
+All Fleet team members undergo a background check provided through [Vetty](https://vetty.co/). Only the most recent background checks appear on the home page of Vetty's dashboard. To access a complete list of background checks run in Vetty, scroll down to the bottom of the candidates page and click "View Historical".
### Process an email from a state agency
@@ -252,7 +252,36 @@ When BizOps receives notification of a Fleetie's manager changing, follow these
- For a team member moving from "classified" to "confidential" access, check Gusto, Plane, and other systems to remove their access.
> **Note:** The Fleeties spreadsheet is the source of truth for who everyone's manager is and their job titles.
-
+
+### Recognize employee workiversaries
+
+At Fleet, everyone is recognized on their [workiversary](https://fleetdm.com/handbook/company/communications#workiversaries). To ensure this happens, take the following steps:
+
+1. Bimonthly, use [Fleeties (private google doc)](https://docs.google.com/spreadsheets/d/1OSLn-ZCbGSjPusHPiR5dwQhheH1K8-xqyZdsOe9y7qc/edit#gid=0) to determine who is celebrating their workiversary in the following two months.
+2. Post in the #help-classifed Slack channel and cc the Head of Business Operations. Use the following template:
+
+
+ ```
+ [Month]
+ [workiversary date (DD-MMM)] - [teammate name] - [number of years at Fleet]
+ ```
+
+
+ The Apprentice to the CEO will also use this post to update the [All hands](https://fleetdm.com/handbook/company/communications#all-hands) deck.
+3. On the day prior to a workiversary, send the teammate’s manager a DM on Slack:
+
+
+ ```
+ Hey! Just a heads up, tomorrow is [teammate’s name] [number of years at Fleet] workiversary at Fleet.
+ BizOps were planning on posting something in the #random channel to recognize them, but I was wondering if you would like to instead?
+ ```
+
+
+ > If a manager elects to post and hasn't done so by 2pm ET on the day of the workiversary, send them a friendly reminder and offer to post instead.
+
+4. If the manager has deferred to BizOps, schedule a Slack post for the following day to recognize the teammate's contributions at Fleet. If you’re unsure about what to post, take a look at what’s been [posted previously](https://docs.google.com/document/d/1Va4TYAs9Tb0soDQPeoeMr-qHxk0Xrlf-DUlBe4jn29Q/edit).
+
+
### Prepare salary benchmarking information
1. Use the relevant template text in the README section of the [¶¶ 💌 Compensation decisions document](https://docs.google.com/document/d/1NQ-IjcOTbyFluCWqsFLMfP4SvnopoXDcX0civ-STS5c/edit?usp=sharing) for a current Fleetie, a new role, a prospective hire, or other benchmarking use case.
@@ -283,6 +312,15 @@ Create a [new montly accounting issue](https://github.com/fleetdm/confidential/i
- **When is the issue created?** We create and close the monthly accounting issue for the previous month within the first 7 days of the following month. For example, the monthly accounting issue to close out the month of January is created promptly in February and closed before the end of the day, Feb 7th. A convenient trick is to create the issue on the first Friday of the month and close it ASAP.
+### Respond to low credit alert
+Fleet admins will receive an email alert when the usage of company cards for the month is aproaching the company credit limit. To avoid the limit being exceeded, a Brex admin will follow these steps:
+1. Sign in to Fleet's Brex account.
+2. On the landing page, use the "Move money" button to "Add funds to your Brex business accounts".
+3. Select "Transfer from a connected account" and select the primary business account.
+4. Choose the "One time" transfer option and process the transfer.
+
+No further action needs to be taken, the amount available for use will increase without disruption to regular processes.
+
### Check franchise tax status
No later than the second month of every quarter, we check [Delaware divison of corporations](https://icis.corp.delaware.gov) to ensure that Fleet has paid the quarterly franchise tax amounts to remain in good standing with the state of Delaware.
- Go to the [DCIS - eCorp website](https://icis.corp.delaware.gov/ecorp/logintax.aspx?FilingType=FranchiseTax) and use the details in 1Password to look up Fleet's status.
diff --git a/handbook/business-operations/security-policies.md b/handbook/business-operations/security-policies.md
index 514754eac3..501dd7d881 100644
--- a/handbook/business-operations/security-policies.md
+++ b/handbook/business-operations/security-policies.md
@@ -378,7 +378,7 @@ Fleet policy requires all workforce members to comply with the HR Security Polic
Fleet policy requires that:
-- Background verification checks on candidates for employees and contractors with production access to the Fleet infrastructure resources must be carried out in accordance with relevant laws, regulations, and ethics. These checks should be proportional to the business requirements, the classification of the information to be accessed, and the perceived risk.
+- Background verification checks on candidates for all employees and contractors must be carried out in accordance with relevant laws, regulations, and ethics. These checks should be proportional to the business requirements, the classification of the information to be accessed, and the perceived risk.
- Employees, contractors, and third-party users must agree to and sign the terms and conditions of their employment contract and comply with acceptable use.
- Employees will perform an onboarding process that familiarizes them with the environments, systems, security requirements, and procedures that Fleet already has in place. Employees will also have ongoing security awareness training that is audited.
- Employee offboarding will include reiterating any duties and responsibilities still valid after terminations, verifying that access to any Fleet systems has been removed, and ensuring that all company-owned assets are returned.
diff --git a/handbook/business-operations/vendor-questionnaires.md b/handbook/business-operations/vendor-questionnaires.md
index b171346023..8af1763870 100644
--- a/handbook/business-operations/vendor-questionnaires.md
+++ b/handbook/business-operations/vendor-questionnaires.md
@@ -67,7 +67,7 @@ Please also see [privacy](https://fleetdm.com/legal/privacy)
## Sub-processors
| Question | Answer |
| ----------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Does Fleet possess an APEC PRP certification issued by a certification body (or Accountability Agent)? If not, is Fleet able to provide any evidence that the PRP requirements are being met as it relates to the Scoped Services provided to its customers? | Fleet has not undergone APEC PRP certification but has undergone an external security audit that included pen testing. For a complete list of subprocessors, please refer to https://trust.fleetdm.com/subprocessors |
+| Does Fleet possess an APEC PRP certification issued by a certification body (or Accountability Agent)? If not, is Fleet able to provide any evidence that the PRP requirements are being met as it relates to the Scoped Services provided to its customers? | Fleet has not undergone APEC PRP certification but has undergone an external security audit that included pen testing. For a complete list of subprocessors, please refer to our [trust page](https://trust.fleetdm.com/subprocessors). |
diff --git a/handbook/company/pricing-features-table.yml b/handbook/company/pricing-features-table.yml
index d98d4f1038..12d67f61f4 100644
--- a/handbook/company/pricing-features-table.yml
+++ b/handbook/company/pricing-features-table.yml
@@ -558,10 +558,10 @@
waysToUse:
- description: Deploy configuration profiles on macOS and Windows and verify that they're installed.
moreInfoUrl: https://github.com/fleetdm/fleet/issues/13281
- - description: Deploy custom declaration (DDM) profiles on macOS. Coming soon (2024-03-31).
+ - description: Deploy custom declaration (DDM) profiles on macOS.
moreInfoUrl: https://github.com/fleetdm/fleet/issues/14550
- - description: Target profiles to specific hosts using SQL.
- moreInfoUrl: https://github.com/fleetdm/fleet/issues/14715
+ - description: Target profiles to specific hosts using SQL. Exclusions coming soon (2024-07-15) #customer-rosner
+ moreInfoUrl: https://github.com/fleetdm/fleet/issues/17315
- description: Automatically re-deploy configuration profiles when they're not installed.
- description: Deploy configuration profiles on iOS/iPadOS. Coming soon (2024-07-15).
productCategories: [Device management]
@@ -581,6 +581,9 @@
usualDepartment: IT
productCategories: [Device management]
pricingTableCategories: [Device management]
+ waysToUse:
+ - description: Support ACME as a protocol for MDM certificate generation. Coming soon (2024-12-31) #customer-rosner
+ moreInfoUrl: https://github.com/fleetdm/fleet/issues/15611
- industryName: Low-level MDM commands for macOS, iOS/iPadOS*, and Windows (e.g. remote restart)
documentationUrl: https://fleetdm.com/docs/using-fleet/mdm-commands
tier: Free
@@ -618,7 +621,7 @@
pricingTableCategories: [Device management]
waysToUse:
- description: Enforce macOS updates via Nudge.
- - description: Progressively enhance from Nudge to DDM-based OS updates. Coming soon (2024-03-31).
+ - description: Progressively enhance from Nudge to DDM-based OS updates.
moreInfoUrl: https://github.com/fleetdm/fleet/issues/17295
- description: Automatically update Windows after the end user reaches a deadline.
- industryName: Cross-platform remote lock and wipe
@@ -637,12 +640,11 @@
description: Easily configure and install SentinelOne, Crowdstrike, and other security tools.
moreInfoUrl: https://github.com/fleetdm/fleet/issues/14921
tier: Premium
- comingSoonOn: 2024-05-13 #customer-flacourtia
usualDepartment: IT
productCategories: [Device management]
pricingTableCategories: [Device management]
- industryName: Update apps on macOS, Windows, and Linux computers.
- description: Install Zoom, Slack, Chrome, and other apps without interrupting your end users.
+ description: Update Zoom, Slack, Chrome, and other apps without interrupting your end users.
tier: Premium
comingSoonOn: 2024-07-15
usualDepartment: IT
@@ -660,7 +662,7 @@
- industryName: License apps on macOS and iOS/iPadOS through Volume Purchasing Program (VPP).
description: Offer licenses for Photoshop and other App Sore apps for your end users.
tier: Premium
- comingSoonOn: 2024-06-30
+ comingSoonOn: 2024-07-15 #customer-rosner
usualDepartment: IT
productCategories: [Device management]
pricingTableCategories: [Device management]
@@ -757,6 +759,15 @@
pricingTableCategories: [Endpoint operations]
usualDepartment: IT
tier: Free
+- industryName: Two-factor authentication
+ moreInfoUrl: https://github.com/fleetdm/fleet/issues/5478
+ productCategories: [Endpoint operations,Device management,Vulnerability management]
+ pricingTableCategories: [Endpoint operations]
+ usualDepartment: IT
+ tier: Free
+ waysToUse:
+ - description: Enforce two-factor authentication when logging in to Fleet for added security.
+ comingSoonOn: 2024-12-31 #customer-rosner
- industryName: Disk encryption
documentationURL: https://fleetdm.com/docs/using-fleet/mdm-disk-encryption
friendlyName: Ensure hard disks are encrypted
@@ -789,6 +800,13 @@
pricingTableCategories: [Endpoint operations]
usualDepartment: IT
tier: Premium
+- industryName: System for Cross-domain Identity Management (SCIM) provisioning
+ moreInfoUrl: https://github.com/fleetdm/fleet/issues/15671
+ productCategories: [Endpoint operations,Device management,Vulnerability management]
+ pricingTableCategories: [Endpoint operations]
+ usualDepartment: IT
+ tier: Premium
+ comingSoonOn: 2024-12-31 #customer-rosner
- industryName: Automated user role sync via Okta, AD, or any IDP
documentationUrl:
productCategories: [Endpoint operations,Device management,Vulnerability management]
@@ -921,9 +939,6 @@
productCategories: [Endpoint operations]
pricingTableCategories: [Endpoint operations]
buzzwords: [Real-time export,Ship logs]
- waysToUse:
- - description: Choose different file sizes for automated query results and agent logs. Coming soon (2024-04-22) #Customer-blanco
- moreInfoUrl: https://github.com/fleetdm/fleet/issues/11999
- industryName: File carving (AWS S3)
documentationUrl: https://fleetdm.com/docs/configuration/fleet-server-configuration#s-3-file-carving-backend
tier: Free
diff --git a/handbook/digital-experience/README.md b/handbook/digital-experience/README.md
index 91d51db08d..4084572a0c 100644
--- a/handbook/digital-experience/README.md
+++ b/handbook/digital-experience/README.md
@@ -165,28 +165,23 @@ If the action fails, please complete the following steps:
### Communicate Fleet's potential energy to stakeholders
-On the first business day of every month, the Apprentice will send an update to the stakeholders of Fleet using the following steps:
+On the first business day of every month, the Head of Digital Experience will send an update to the stakeholders of Fleet using the following steps:
1. Copy the following template into an outgoing email with the subject line: "[Investor update] Fleet, YYYY-MM".
```
Hi investors and friends,
-Here’s a quick update on the numbers from last month:
-• Gross new ∆ARR (QTD): + TODO
-• Social media mentions (LinkedIn): 3.8 per day (Goal: 5) (Want to help?)
-• Current version: 4.48.0 (See what's new)
-• Next in-person event: Kansas City, (April 20) BSides KC
-• Next press release: 2024-04-30: "Stop nudging"
-"Stop installing updates and forcing restarts when your users are busy using their computers. Fleet finds time in the calendar for a reboot and uses AI to explain why."
+FYI we just updated the self-service investor update portal with the numbers from last month: https://docs.google.com/spreadsheets/d/10T7Q9iuHA4vpfV7qZCm6oMd5U1bLftBSobYD0RR8RkM/edit#gid=0
Thanks for your support,
Mike and the Fleet team
+
```
2. Address the email to the executive team's Gmail.
-3. Using the [🌧️🦉 Investors + advisors](https://docs.google.com/spreadsheets/d/15knBE2-PrQ1Ad-QcIk0mxCN-xFsATKK9hcifqrm0qFQ/edit#gid=1068113636) spreadsheet, collect all of the investor emails from previous funding rounds and add them to bcc of the email and send.
+3. Using the [🌧️🦉 Investors + advisors](https://docs.google.com/spreadsheets/d/15knBE2-PrQ1Ad-QcIk0mxCN-xFsATKK9hcifqrm0qFQ/edit#gid=1068113636) spreadsheet, bcc the correct individuals and send the email.
### Refresh event calendar
diff --git a/handbook/engineering/README.md b/handbook/engineering/README.md
index 494f9351d5..47fec5cb51 100644
--- a/handbook/engineering/README.md
+++ b/handbook/engineering/README.md
@@ -414,7 +414,7 @@ Steps to renew the certificate:
* Update `sails_custom__mdmVendorKeyPassphrase` with the passphrase used in step 4
* Update `sails_custom__mdmVendorKeyPem` with `VendorPrivateKey.key` from step 4
9. Store updated values in [Confidential 1Password Vault](https://start.1password.com/open/i?a=N3F7LHAKQ5G3JPFPX234EC4ZDQ&v=lcvkjobeheaqdgnz33ontpuhxq&i=byyfn2knejwh42a2cbc5war5sa&h=fleetdevicemanagement.1password.com)
-10. Verify by logging into a normal apple account (not billing@...) and Generate a new Push Certificate following our [setup MDM](https://fleetdm.com/docs/using-fleet/mdm-setup#step-2-generate-an-apns-certificate) steps and verify the Expiration date is 1 year from today.
+10. Verify by logging into a normal apple account (not billing@...) and Generate a new Push Certificate following our [setup MDM](https://fleetdm.com/docs/using-fleet/mdm-setup) steps and verify the Expiration date is 1 year from today.
11. Adjust calendar event to be between 2-4 weeks before the next expiration.
### Perform an incident postmortem
diff --git a/handbook/sales/sales.rituals.yml b/handbook/sales/sales.rituals.yml
index aa01598dfb..639c896715 100644
--- a/handbook/sales/sales.rituals.yml
+++ b/handbook/sales/sales.rituals.yml
@@ -2,7 +2,7 @@
--
+ -
task: "Prioritize for next sprint" # Title that will actually show in rituals table
startedOn: "2023-09-04" # Needs to align with frequency e.g. if frequency is every thrid Thursday startedOn === any third thursday
frequency: "Triweekly" # must be supported by
@@ -12,7 +12,27 @@
autoIssue: # Enables automation of GitHub issues
labels: [ "#g-sales" ] # label to be applied to issue
repo: "confidential"
--
+ -
+ task: "g-sales standup" # Title that will actually show in rituals table
+ startedOn: "2023-09-04" # Needs to align with frequency e.g. if frequency is every thrid Thursday startedOn === any third thursday
+ frequency: "Daily" # must be supported by
+ description: "Review progress on priorities for Sprint. Discuss previous day accomplishments, goals for today and any blockers." # example of a longer thing: description: "[Prioritizing next sprint](https://fleetdm.com/handbook/company/communication)"
+ moreInfoUrl: "https://fleetdm.com/handbook/company/why-this-way#why-make-work-visible" #URL used to highlight "description:" test in table
+ dri: "alexmitchelliii" # DRI for ritual (assignee if autoIssue) (TODO display GitHub proflie pic instead of name or title)
+ autoIssue: # Enables automation of GitHub issues
+ labels: [ "#g-sales" ] # label to be applied to issue
+ repo: "confidential"
+ -
+ task: "Opportunity pipeline review" # Title that will actually show in rituals table
+ startedOn: "2023-09-04" # Needs to align with frequency e.g. if frequency is every thrid Thursday startedOn === any third thursday
+ frequency: "Weekly" # must be supported by
+ description: "Review status of sales opportunities and discuss next steps." # example of a longer thing: description: "[Prioritizing next sprint](https://fleetdm.com/handbook/company/communication)"
+ moreInfoUrl: "https://fleetdm.com/handbook/customers#review-rep-activity" #URL used to highlight "description:" test in table
+ dri: "alexmitchelliii" # DRI for ritual (assignee if autoIssue) (TODO display GitHub proflie pic instead of name or title)
+ autoIssue: # Enables automation of GitHub issues
+ labels: [ "#g-sales" ] # label to be applied to issue
+ repo: "confidential"
+ -
task: "Review rep activity"
startedOn: "2023-09-18"
frequency: "Monthly"
diff --git a/infrastructure/dogfood/terraform/aws-tf-module/docker/.gitignore b/infrastructure/dogfood/terraform/aws-tf-module/docker/.gitignore
new file mode 100644
index 0000000000..b0bcff9fe7
--- /dev/null
+++ b/infrastructure/dogfood/terraform/aws-tf-module/docker/.gitignore
@@ -0,0 +1,2 @@
+osquery
+osquery-docker.patch
diff --git a/infrastructure/dogfood/terraform/aws-tf-module/docker/main.tf b/infrastructure/dogfood/terraform/aws-tf-module/docker/main.tf
index 46e5038957..a2d1655ed2 100644
--- a/infrastructure/dogfood/terraform/aws-tf-module/docker/main.tf
+++ b/infrastructure/dogfood/terraform/aws-tf-module/docker/main.tf
@@ -11,31 +11,65 @@ terraform {
}
}
-variable "osquery_tag" {
- description = "The osquery tag to take from dockerhub to your ecr repo."
+variable "osquery_version" {
+ description = "The osquery version to push to your ecr repo."
type = string
}
+variable "osquery_tags" {
+ description = "The tags that you wish to push among the built images"
+ type = list(string)
+}
+
variable "ecr_repo" {
description = "The ecr repo to push to"
type = string
}
-resource "docker_image" "dockerhub" {
- name = "osquery/osquery:${var.osquery_tag}"
+resource "local_file" "osquery_patch" {
+ content = templatefile("${path.module}/osquery-docker.patch.tmpl", { osquery_version = var.osquery_version })
+ filename = "${path.module}/osquery-docker.patch"
+ file_permission = "0644"
+}
+
+resource "null_resource" "build_osquery" {
+ depends_on = [local_file.osquery_patch]
+ triggers = {
+ osquery_version_changed = var.osquery_version
+ osquery_tags_changed = sha256(jsonencode(var.osquery_tags))
+ }
+ provisioner "local-exec" {
+ working_dir = "${path.module}"
+ command = <<-EOT
+ mkdir -p osquery
+ cd osquery
+ if [ "$(git remote -vvv | head -n1 | awk '{ print $2 }')" = "https://github.com/osquery/osquery.git" ]; then
+ git reset --hard
+ git pull
+ else
+ git clone https://github.com/osquery/osquery.git .
+ fi
+ git apply ../osquery-docker.patch
+ cd tools/docker
+ ./build.sh
+ EOT
+ }
}
resource "docker_tag" "osquery" {
- source_image = docker_image.dockerhub.name
+ depends_on = [null_resource.build_osquery]
+ for_each = toset(var.osquery_tags)
+ source_image = "osquery/osquery:${each.key}"
# We can't include the sha256 when pushing even if they match
- target_image = "${var.ecr_repo}:${split("@sha256", var.osquery_tag)[0]}"
+ target_image = "${var.ecr_repo}:${each.key}"
}
resource "docker_registry_image" "osquery" {
- name = docker_tag.osquery.target_image
+ for_each = toset(var.osquery_tags)
+ name = docker_tag.osquery[each.key].target_image
keep_remotely = true
}
-output "ecr_image" {
- value = docker_tag.osquery.target_image
+output "ecr_images" {
+ value = { for docker_tag in docker_tag.osquery : split(":", docker_tag.target_image)[1] => docker_tag.target_image }
}
diff --git a/infrastructure/dogfood/terraform/aws-tf-module/docker/osquery-docker.patch.tmpl b/infrastructure/dogfood/terraform/aws-tf-module/docker/osquery-docker.patch.tmpl
new file mode 100644
index 0000000000..2ba4208e47
--- /dev/null
+++ b/infrastructure/dogfood/terraform/aws-tf-module/docker/osquery-docker.patch.tmpl
@@ -0,0 +1,28 @@
+diff --git a/tools/docker/build.sh b/tools/docker/build.sh
+index 9efba34f6..34ecd8a4e 100755
+--- a/tools/docker/build.sh
++++ b/tools/docker/build.sh
+@@ -6,7 +6,7 @@ build_deb() {
+
+ TAG=$(echo $OS | sed 's/://g')
+
+- docker build -f deb-dockerfile . --build-arg OSQUERY_URL=https://pkg.osquery.io/deb/osquery_$${VERSION}-1.linux_amd64.deb --build-arg OS_IMAGE=$OS -t osquery/osquery:$${VERSION}-$${TAG}
++ docker build --platform=linux/amd64 -f deb-dockerfile . --build-arg OSQUERY_URL=https://pkg.osquery.io/deb/osquery_$${VERSION}-1.linux_amd64.deb --build-arg OS_IMAGE=$OS -t osquery/osquery:$${VERSION}-$${TAG}
+ }
+
+ build_rpm() {
+@@ -15,11 +15,11 @@ build_rpm() {
+
+ TAG=$(echo $OS | sed 's/://g')
+
+- docker build -f rpm-dockerfile . --build-arg OSQUERY_URL=https://pkg.osquery.io/rpm/osquery-$${VERSION}-1.linux.x86_64.rpm --build-arg OS_IMAGE=$OS -t osquery/osquery:$${VERSION}-$${TAG}
++ docker build --platform=linux/amd64 -f rpm-dockerfile . --build-arg OSQUERY_URL=https://pkg.osquery.io/rpm/osquery-$${VERSION}-1.linux.x86_64.rpm --build-arg OS_IMAGE=$OS -t osquery/osquery:$${VERSION}-$${TAG}
+ }
+
+-versions='5.2.3'
+-deb_platforms='ubuntu:16.04 ubuntu:18.04 ubuntu:20.04 ubuntu:22.04 debian:10 debian:9 debian:8 debian:7'
++versions='${osquery_version}'
++deb_platforms='ubuntu:16.04 ubuntu:18.04 ubuntu:20.04 ubuntu:22.04 ubuntu:24.04 debian:10 debian:9 debian:8 debian:7'
+ rpm_platforms='centos:6 centos:7 centos:8'
+
+ for v in $versions
diff --git a/infrastructure/dogfood/terraform/aws-tf-module/free-ecs-hosts.tf b/infrastructure/dogfood/terraform/aws-tf-module/free-ecs-hosts.tf
index 8021f5892c..0e3ebac81d 100644
--- a/infrastructure/dogfood/terraform/aws-tf-module/free-ecs-hosts.tf
+++ b/infrastructure/dogfood/terraform/aws-tf-module/free-ecs-hosts.tf
@@ -1,16 +1,18 @@
## Linux hosts in ECS
locals {
+ osquery_version = "5.12.2"
osquery_hosts = {
- "5.8.2-ubuntu22.04@sha256:b77c7b06c4d7f2a3c58cc3a34e51fffc480e97795fb3c75cb1dc1cf3709e3dc6" = "Skys-laptop"
- "5.8.2-ubuntu20.04@sha256:3496ffd0ad570c88a9f405e6ef517079cfeed6ce405b9d22db4dc5ef6ed3faac" = "Cloud-City-server"
- "5.8.2-ubuntu18.04@sha256:372575e876c218dde3c5c0e24fd240d193800fca9b314e94b4ad4e6e22006c9b" = "Mists-laptop"
- "5.8.2-ubuntu16.04@sha256:112655c42951960d8858c116529fb4c64951e4cf2e34cb7c08cd599a009025bb" = "Ethers-laptop"
- "5.8.2-debian10@sha256:de29337896aac89b2b03c7642805859d3fb6d52e5dc08230f987bbab4eeba9c5" = "Breezes-laptop"
- "5.8.2-debian9@sha256:47e46c19cebdf0dc704dd0061328856bda7e1e86b8c0fefdd6f78bd092c6200e" = "Aero-server"
- "5.8.2-centos8@sha256:88a8adde80bd3b1b257e098bc6e41b6afea840f60033653dcb9fe984f36b0f97" = "Stratuss-laptop"
- "5.8.2-centos7@sha256:ff251de4935b80a91c5fc1ac352aebdab9a6bbbf5bda1aaada8e26d22b50202d" = "Zephyrs-Laptop"
- "5.8.2-centos6@sha256:b56736be8436288d3fbd2549ec6165e0588cd7197e91600de4a2f00f1df28617" = "Halo-server"
+ "${local.osquery_version}-ubuntu24.04" = "Atmosphere-database"
+ "${local.osquery_version}-ubuntu22.04" = "Skys-laptop"
+ "${local.osquery_version}-ubuntu20.04" = "Cloud-City-server"
+ "${local.osquery_version}-ubuntu18.04" = "Mists-laptop"
+ "${local.osquery_version}-ubuntu16.04" = "Ethers-laptop"
+ "${local.osquery_version}-debian10" = "Breezes-laptop"
+ "${local.osquery_version}-debian9" = "Aero-server"
+ "${local.osquery_version}-centos8" = "Stratuss-laptop"
+ "${local.osquery_version}-centos7" = "Zephyrs-Laptop"
+ "${local.osquery_version}-centos6" = "Halo-server"
}
}
@@ -123,10 +125,10 @@ provider "docker" {
}
module "osquery_docker" {
- for_each = local.osquery_hosts
- source = "./docker"
- ecr_repo = aws_ecr_repository.osquery.repository_url
- osquery_tag = each.key
+ source = "./docker"
+ ecr_repo = aws_ecr_repository.osquery.repository_url
+ osquery_version = local.osquery_version
+ osquery_tags = keys(local.osquery_hosts)
}
resource "random_uuid" "osquery" {
@@ -135,7 +137,7 @@ resource "random_uuid" "osquery" {
resource "aws_ecs_task_definition" "osquery" {
for_each = local.osquery_hosts
- // e.g. 5-8-2-ubuntu22-04 to match naming requirements
+ // e.g. ${osquery_version}-ubuntu22-04 to match naming requirements
family = "osquery-${replace(split("@sha256", each.key)[0], ".", "-")}"
network_mode = "awsvpc"
requires_compatibilities = ["FARGATE"]
@@ -147,7 +149,7 @@ resource "aws_ecs_task_definition" "osquery" {
[
{
name = "osquery"
- image = module.osquery_docker[each.key].ecr_image
+ image = module.osquery_docker.ecr_images[each.key]
cpu = 256
memory = 512
mountPoints = []
@@ -215,8 +217,8 @@ resource "aws_ecs_task_definition" "osquery" {
resource "aws_ecs_service" "osquery" {
for_each = local.osquery_hosts
- # Name must match ^[A-Za-z-_]+$ e.g. 5-8-2-ubuntu22-04
- name = "osquery_${replace(split("@sha256", each.key)[0], ".", "-")}"
+ # Name must match ^[A-Za-z-_]+$ e.g. 5.12.2-ubuntu22-04
+ name = "osquery_${replace(each.key, ".", "-")}"
launch_type = "FARGATE"
cluster = module.free.byo-db.byo-ecs.service.cluster
task_definition = aws_ecs_task_definition.osquery[each.key].arn
diff --git a/infrastructure/dogfood/terraform/aws-tf-module/main.tf b/infrastructure/dogfood/terraform/aws-tf-module/main.tf
index a08aafc158..1f3f8fd282 100644
--- a/infrastructure/dogfood/terraform/aws-tf-module/main.tf
+++ b/infrastructure/dogfood/terraform/aws-tf-module/main.tf
@@ -36,6 +36,11 @@ variable "geolite2_license" {}
variable "fleet_sentry_dsn" {}
variable "elastic_url" {}
variable "elastic_token" {}
+variable "fleet_calendar_periodicity" {
+ default = "30s"
+ description = "The refresh period for the calendar integration."
+}
+variable "dogfood_sidecar_enroll_secret" {}
data "aws_caller_identity" "current" {}
@@ -55,6 +60,7 @@ locals {
ELASTIC_APM_SERVER_URL = var.elastic_url
ELASTIC_APM_SECRET_TOKEN = var.elastic_token
ELASTIC_APM_SERVICE_NAME = "dogfood"
+ FLEET_CALENDAR_PERIODICITY = var.fleet_calendar_periodicity
}
sentry_secrets = {
FLEET_SENTRY_DSN = "${aws_secretsmanager_secret.sentry.arn}:FLEET_SENTRY_DSN::"
@@ -63,7 +69,7 @@ locals {
}
module "main" {
- source = "github.com/fleetdm/fleet//terraform?ref=tf-mod-root-v1.8.0"
+ source = "github.com/fleetdm/fleet//terraform?ref=tf-mod-root-v1.9.1"
certificate_arn = module.acm.acm_certificate_arn
vpc = {
name = local.customer
@@ -92,10 +98,13 @@ module "main" {
cluster_name = local.customer
}
fleet_config = {
- image = local.geolite2_image
- family = local.customer
- cpu = 1024
- mem = 4096
+ image = local.geolite2_image
+ family = local.customer
+ task_cpu = 2048
+ task_mem = 5120
+ cpu = 1024
+ mem = 4096
+ pid_mode = "task"
autoscaling = {
min_capacity = 2
max_capacity = 5
@@ -115,7 +124,7 @@ module "main" {
}
}
extra_iam_policies = concat(module.firehose-logging.fleet_extra_iam_policies, module.osquery-carve.fleet_extra_iam_policies, module.ses.fleet_extra_iam_policies)
- extra_execution_iam_policies = concat(module.mdm.extra_execution_iam_policies, [aws_iam_policy.sentry.arn]) #, module.saml_auth_proxy.fleet_extra_execution_policies)
+ extra_execution_iam_policies = concat(module.mdm.extra_execution_iam_policies, [aws_iam_policy.sentry.arn, aws_iam_policy.osquery_sidecar.arn]) #, module.saml_auth_proxy.fleet_extra_execution_policies)
extra_environment_variables = merge(
module.mdm.extra_environment_variables,
module.firehose-logging.fleet_extra_environment_variables,
@@ -132,6 +141,71 @@ module "main" {
# container_name = "fleet"
# container_port = 8080
# }]
+ software_installers = {
+ bucket_prefix = "${local.customer}-software-installers-"
+ }
+ sidecars = [
+ {
+ name = "osquery"
+ image = module.osquery_docker.ecr_images["${local.osquery_version}-ubuntu24.04"]
+ cpu = 1024
+ memory = 1024
+ mountPoints = []
+ volumesFrom = []
+ essential = true
+ ulimits = [
+ {
+ softLimit = 999999,
+ hardLimit = 999999,
+ name = "nofile"
+ }
+ ]
+ networkMode = "awsvpc"
+ logConfiguration = {
+ logDriver = "awslogs"
+ options = {
+ awslogs-group = local.customer
+ awslogs-region = "us-east-2"
+ awslogs-stream-prefix = "osquery"
+ }
+ }
+ secrets = [
+ {
+ name = "ENROLL_SECRET"
+ valueFrom = aws_secretsmanager_secret.dogfood_sidecar_enroll_secret.arn
+ }
+ ]
+ workingDirectory = "/",
+ command = [
+ "osqueryd",
+ "--tls_hostname=dogfood.fleetdm.com",
+ "--force=true",
+ # Ensure that the host identifier remains the same between invocations
+ # "--host_identifier=specified",
+ # "--specified_identifier=${random_uuid.osquery[each.key].result}",
+ "--verbose=true",
+ "--tls_dump=true",
+ "--enroll_secret_env=ENROLL_SECRET",
+ "--enroll_tls_endpoint=/api/osquery/enroll",
+ "--config_plugin=tls",
+ "--config_tls_endpoint=/api/osquery/config",
+ "--config_refresh=10",
+ "--disable_distributed=false",
+ "--distributed_plugin=tls",
+ "--distributed_interval=10",
+ "--distributed_tls_max_attempts=3",
+ "--distributed_tls_read_endpoint=/api/osquery/distributed/read",
+ "--distributed_tls_write_endpoint=/api/osquery/distributed/write",
+ "--logger_plugin=tls",
+ "--logger_tls_endpoint=/api/osquery/log",
+ "--logger_tls_period=10",
+ "--disable_carver=false",
+ "--carver_start_endpoint=/api/osquery/carve/begin",
+ "--carver_continue_endpoint=/api/osquery/carve/block",
+ "--carver_block_size=8000000",
+ ]
+ }
+ ]
}
alb_config = {
name = local.customer
@@ -288,10 +362,9 @@ module "firehose-logging" {
}
module "osquery-carve" {
- source = "github.com/fleetdm/fleet//terraform/addons/osquery-carve?ref=tf-mod-addon-osquery-carve-v1.0.1"
+ source = "github.com/fleetdm/fleet//terraform/addons/osquery-carve?ref=tf-mod-addon-osquery-carve-v1.1.0"
osquery_carve_s3_bucket = {
name = "fleet-${local.customer}-osquery-carve"
- expires_days = 3650
}
}
@@ -450,7 +523,7 @@ module "geolite2" {
}
module "vuln-processing" {
- source = "github.com/fleetdm/fleet//terraform/addons/external-vuln-scans?ref=tf-mod-addon-external-vuln-scans-v2.1.0"
+ source = "github.com/fleetdm/fleet//terraform/addons/external-vuln-scans?ref=tf-mod-addon-external-vuln-scans-v2.2.0"
ecs_cluster = module.main.byo-vpc.byo-db.byo-ecs.service.cluster
execution_iam_role_arn = module.main.byo-vpc.byo-db.byo-ecs.execution_iam_role_arn
subnets = module.main.byo-vpc.byo-db.byo-ecs.service.network_configuration[0].subnets
@@ -458,9 +531,56 @@ module "vuln-processing" {
fleet_config = module.main.byo-vpc.byo-db.byo-ecs.fleet_config
task_role_arn = module.main.byo-vpc.byo-db.byo-ecs.iam_role_arn
fleet_server_private_key_secret_arn = module.main.byo-vpc.byo-db.byo-ecs.fleet_server_private_key_secret_arn
+ vuln_processing_task_memory = 5120
+ vuln_processing_task_cpu = 2048
awslogs_config = {
group = module.main.byo-vpc.byo-db.byo-ecs.fleet_config.awslogs.name
region = module.main.byo-vpc.byo-db.byo-ecs.fleet_config.awslogs.region
prefix = module.main.byo-vpc.byo-db.byo-ecs.fleet_config.awslogs.prefix
}
+ fleet_s3_software_installers_config = module.main.byo-vpc.byo-db.byo-ecs.fleet_s3_software_installers_config
+}
+
+resource "aws_secretsmanager_secret" "dogfood_sidecar_enroll_secret" {
+ name = "dogfood-sidecar-enroll-secret"
+}
+
+resource "aws_secretsmanager_secret_version" "dogfood_sidecar_enroll_secret" {
+ secret_id = aws_secretsmanager_secret.dogfood_sidecar_enroll_secret.id
+ secret_string = var.dogfood_sidecar_enroll_secret
+}
+
+data "aws_iam_policy_document" "osquery_sidecar" {
+ statement {
+ actions = [
+ "ecr:BatchCheckLayerAvailability",
+ "ecr:BatchGetImage",
+ "ecr:GetDownloadUrlForLayer",
+ "ecr:GetAuthorizationToken"
+ ]
+ resources = ["*"]
+ }
+ statement {
+ actions = [ #tfsec:ignore:aws-iam-no-policy-wildcards
+ "kms:Encrypt*",
+ "kms:Decrypt*",
+ "kms:ReEncrypt*",
+ "kms:GenerateDataKey*",
+ "kms:Describe*"
+ ]
+ resources = [aws_kms_key.osquery.arn]
+ }
+ statement {
+ actions = [ #tfsec:ignore:aws-iam-no-policy-wildcards
+ "secretsmanager:GetSecretValue"
+ ]
+ resources = [aws_secretsmanager_secret.dogfood_sidecar_enroll_secret.arn]
+
+ }
+}
+
+resource "aws_iam_policy" "osquery_sidecar" {
+ name = "osquery-sidecar-policy"
+ description = "IAM policy that Osquery sidecar containers use to define access to AWS resources"
+ policy = data.aws_iam_policy_document.osquery_sidecar.json
}
diff --git a/infrastructure/dogfood/terraform/aws/variables.tf b/infrastructure/dogfood/terraform/aws/variables.tf
index 5fd2ad6def..f02ec39fa9 100644
--- a/infrastructure/dogfood/terraform/aws/variables.tf
+++ b/infrastructure/dogfood/terraform/aws/variables.tf
@@ -56,7 +56,7 @@ variable "database_name" {
variable "fleet_image" {
description = "the name of the container image to run"
- default = "fleetdm/fleet:v4.50.2"
+ default = "fleetdm/fleet:v4.51.1"
}
variable "software_inventory" {
diff --git a/infrastructure/dogfood/terraform/gcp/variables.tf b/infrastructure/dogfood/terraform/gcp/variables.tf
index c8912229ea..ab2e8ee55e 100644
--- a/infrastructure/dogfood/terraform/gcp/variables.tf
+++ b/infrastructure/dogfood/terraform/gcp/variables.tf
@@ -68,5 +68,5 @@ variable "redis_mem" {
}
variable "image" {
- default = "fleet:v4.50.2"
+ default = "fleet:v4.51.1"
}
diff --git a/infrastructure/render/README.md b/infrastructure/render/README.md
index 322231d5bb..23bd1618d0 100644
--- a/infrastructure/render/README.md
+++ b/infrastructure/render/README.md
@@ -39,4 +39,5 @@ Click the deploy on render button or import the blueprint from the Render servic
### Post-Deployment
-Navigate to the generated URL and run through the initial setup.
+Navigate to the generated URL and run through the initial setup. If you have a license key you can add it post-deploy as
+an environment variable `FLEET_LICENSE_KEY=value` in the Fleet service configuration.
diff --git a/it-and-security/lib/configuration-profiles/macos-disable-update-notifications.mobileconfig b/it-and-security/lib/configuration-profiles/macos-disable-update-notifications.mobileconfig
new file mode 100644
index 0000000000..cfda6f3d03
--- /dev/null
+++ b/it-and-security/lib/configuration-profiles/macos-disable-update-notifications.mobileconfig
@@ -0,0 +1,84 @@
+
+
+
+
+ PayloadUUID
+ CDB0BC64-F3EB-4B1A-AA5E-9A5D994CA592
+ PayloadType
+ Configuration
+ PayloadOrganization
+ Macjutsu
+ PayloadIdentifier
+ CDB0BC64-F3EB-4B1A-AA5E-9A5D994CA592
+ PayloadDisplayName
+ Apple Software Update Disable Notifications
+ PayloadDescription
+ These settings disable the Apple software update notifications and banners.
+ PayloadVersion
+ 1
+ PayloadEnabled
+ PayloadRemovalDisallowed
+ PayloadScope
+ System
+ PayloadContent
+
+
+ PayloadDisplayName
+ Notifications Payload
+ PayloadIdentifier
+ 84DB38D0-8A4B-4382-B1D2-11235122FF6D
+ PayloadOrganization
+ Macjutsu
+ PayloadType
+ com.apple.notificationsettings
+ PayloadUUID
+ 84DB38D0-8A4B-4382-B1D2-11235122FF6D
+ PayloadVersion
+ 1
+ NotificationSettings
+
+
+ BundleIdentifier
+ _system_center_:com.apple.softwareupdatenotification
+ CriticalAlertEnabled
+ NotificationsEnabled
+
+
+
+
+ PayloadDisplayName
+ Custom Settings
+ PayloadIdentifier
+ 87E2F5E4-1C8A-4D43-AA52-676364A3326E
+ PayloadOrganization
+ Macjutsu
+ PayloadType
+ com.apple.ManagedClient.preferences
+ PayloadUUID
+ 87E2F5E4-1C8A-4D43-AA52-676364A3326E
+ PayloadVersion
+ 1
+ PayloadContent
+
+ com.apple.systempreferences
+
+ Forced
+
+
+ mcx_preference_settings
+
+ AttentionPrefBundleIDs
+ 0
+
+
+
+
+
+
+
+
+
diff --git a/it-and-security/lib/macos-cis.policies.yml b/it-and-security/lib/macos-cis.policies.yml
new file mode 100644
index 0000000000..71cd37efb8
--- /dev/null
+++ b/it-and-security/lib/macos-cis.policies.yml
@@ -0,0 +1,105 @@
+- name: CIS - Ensure Auto Update Is Enabled (MDM Required)
+ critical: false
+ platform: darwin
+ description: Checks that the system is configured via MDM to automatically install updates.
+ resolution: "Ask your system administrator to deploy an MDM profile that enables automatic updates."
+ query: |
+ SELECT 1 WHERE
+ EXISTS (
+ SELECT 1 FROM managed_policies WHERE
+ domain='com.apple.SoftwareUpdate' AND
+ name='AutomaticCheckEnabled' AND
+ (value = 1 OR value = 'true') AND
+ username = ''
+ )
+ AND NOT EXISTS (
+ SELECT 1 FROM managed_policies WHERE
+ domain='com.apple.SoftwareUpdate' AND
+ name='AutomaticCheckEnabled' AND
+ (value != 1 AND value != 'true')
+ );
+- name: CIS - Ensure Download New Updates When Available Is Enabled (MDM Required)
+ critical: false
+ platform: darwin
+ description: Checks that the system is configured via MDM to automatically download updates.
+ resolution: "Ask your system administrator to deploy an MDM profile that enables automatic update downloads."
+ query: |
+ SELECT 1 WHERE
+ EXISTS (
+ SELECT 1 FROM managed_policies WHERE
+ domain='com.apple.SoftwareUpdate' AND
+ name='AutomaticDownload' AND
+ (value = 1 OR value = 'true') AND
+ username = ''
+ )
+ AND NOT EXISTS (
+ SELECT 1 FROM managed_policies WHERE
+ domain='com.apple.SoftwareUpdate' AND
+ name='AutomaticDownload' AND
+ (value != 1 AND value != 'true')
+ );
+- name: CIS - Ensure Install of macOS Updates Is Enabled (MDM Required)
+ critical: false
+ platform: darwin
+ description: Ensure that macOS updates are installed after they are available from Apple.
+ resolution: "Ask your system administrator to deploy an MDM profile that enables automatic install of macOS updates."
+ query: |
+ SELECT 1 WHERE
+ EXISTS (
+ SELECT 1 FROM managed_policies WHERE
+ domain='com.apple.SoftwareUpdate' AND
+ name='AutomaticallyInstallMacOSUpdates' AND
+ (value = 1 OR value = 'true') AND
+ username = ''
+ )
+ AND NOT EXISTS (
+ SELECT 1 FROM managed_policies WHERE
+ domain='com.apple.SoftwareUpdate' AND
+ name='AutomaticallyInstallMacOSUpdates' AND
+ (value != 1 AND value != 'true')
+ );
+- name: CIS - Ensure Install Application Updates from the App Store Is Enabled (MDM Required)
+ critical: false
+ platform: darwin
+ description: Ensure that application updates are installed after they are available from Apple.
+ resolution: Ask your system administrator to deploy an MDM profile that enables automatic updates of Apple apps.
+ query: |
+ SELECT 1 WHERE
+ EXISTS (
+ SELECT 1 FROM managed_policies WHERE
+ domain='com.apple.SoftwareUpdate' AND
+ name='AutomaticallyInstallAppUpdates' AND
+ (value = 1 OR value = 'true') AND
+ username = ''
+ )
+ AND NOT EXISTS (
+ SELECT 1 FROM managed_policies WHERE
+ domain='com.apple.SoftwareUpdate' AND
+ name='AutomaticallyInstallAppUpdates' AND
+ (value != 1 AND value != 'true')
+ );
+
+- name: CIS - Ensure Install Security Responses and System Files Is Enabled (MDM Required)
+ critical: false
+ platform: darwin
+ description: |
+ Ensure that system and security updates are installed after they are available from
+ Apple. This setting enables definition updates for XProtect and Gatekeeper. With this
+ setting in place, new malware and adware that Apple has added to the list of malware or
+ untrusted software will not execute.
+ resolution: "Ask your system administrator to deploy an MDM profile that enables automatic critical system and security updates."
+ query: |
+ SELECT 1 WHERE
+ EXISTS (
+ SELECT 1 FROM managed_policies WHERE
+ domain='com.apple.SoftwareUpdate' AND
+ name='CriticalUpdateInstall' AND
+ (value = 1 OR value = 'true') AND
+ username = ''
+ )
+ AND NOT EXISTS (
+ SELECT 1 FROM managed_policies WHERE
+ domain='com.apple.SoftwareUpdate' AND
+ name='CriticalUpdateInstall' AND
+ (value != 1 AND value != 'true')
+ );
diff --git a/it-and-security/lib/macos-device-health.policies.yml b/it-and-security/lib/macos-device-health.policies.yml
index c9b7879157..75bfaf4b0b 100644
--- a/it-and-security/lib/macos-device-health.policies.yml
+++ b/it-and-security/lib/macos-device-health.policies.yml
@@ -64,11 +64,4 @@
description: Looks for PDF files with file names typically used by 1Password for emergency recovery kits. To protect the performance of your devices, the search is one level deep and limited to the Desktop, Documents, Downloads, and Shared folders.
resolution: Delete 1Password emergency kits from your computer, and empty the trash. 1Password emergency kits should only be printed and stored in a physically secure location.
platform: darwin
-- name: macOS - Check if latest version
- query: SELECT 1 FROM os_version WHERE major = '14' AND minor = '5';
- # patch query: SELECT 1 FROM os_version WHERE major = "14" AND minor = "5" AND patch >= "1";
- critical: false
- description: Using an outdated macOS version risks exposure to security vulnerabilities and potential system instability.
- resolution: We will update your macOS to the latest version.
- platform: darwin
- calendar_events_enabled: true
\ No newline at end of file
+
\ No newline at end of file
diff --git a/it-and-security/lib/macos-mdm-migration.sh b/it-and-security/lib/macos-mdm-migration.sh
new file mode 100755
index 0000000000..3ddab5f5eb
--- /dev/null
+++ b/it-and-security/lib/macos-mdm-migration.sh
@@ -0,0 +1,102 @@
+#!/bin/zsh
+
+# Function to start System Events if it isn't running
+start_system_events() {
+ osascript -e '
+ tell application "System Events"
+ if not running then
+ launch
+ delay 2
+ end if
+ end tell'
+}
+
+# Define the URL of the new wallpaper
+new_wallpaper_url="https://fleetdm.com/images/demo/fleet-desktop-migration.png"
+
+# Define the path where the new wallpaper will be saved
+new_wallpaper_path="/tmp/fleet-desktop-migration.png"
+
+current_user=$(ls -l /dev/console | awk '{print $3}')
+
+# Download the new wallpaper
+curl -o "$new_wallpaper_path" "$new_wallpaper_url"
+
+# Check if the download was successful
+if [[ ! -f "$new_wallpaper_path" ]] || [[ ! -s "$new_wallpaper_path" ]]; then
+ echo "Failed to download the new wallpaper."
+ exit 1
+fi
+
+# Start System Events if it isn't running
+start_system_events
+
+# Get the current wallpaper
+current_wallpaper=$(osascript -e '
+tell application "System Events"
+ set currentDesktop to a reference to current desktop
+ set desktopPicture to picture of currentDesktop
+ try
+ return POSIX path of desktopPicture
+ on error
+ return desktopPicture
+ end try
+end tell
+')
+
+# Check if the current wallpaper path is valid
+if [[ -z "$current_wallpaper" ]]; then
+ echo "Failed to get the current wallpaper path."
+ exit 1
+fi
+
+echo "Current wallpaper: $current_wallpaper"
+echo "Fleet wallpaper: $new_wallpaper_path"
+
+# Function to change wallpaper using Finder
+change_wallpaper() {
+ local wallpaper_path=$1
+ osascript -e "
+ tell application \"Finder\"
+ set desktop picture to POSIX file \"$wallpaper_path\"
+ end tell"
+}
+
+# Function to check the result of the previous command
+check_result() {
+ if [[ $? -ne 0 ]]; then
+ echo "Failed to change to the new wallpaper."
+ exit 1
+ fi
+}
+
+# Set the new wallpaper
+change_wallpaper "$new_wallpaper_path"
+check_result
+
+# Wait for 5 seconds
+sleep 5
+
+# Open Chrome with a specific URL and maximize the window and set full screen and unmute
+chrome_url="https://www.loom.com/share/e5f733b92773476690b8d4f38592b35d?t=254&sid=f993c904-bf49-40e4-b55c-26c81a91c60&autoplay=1"
+sudo -u "$current_user" osascript -e "
+tell application \"Google Chrome\"
+ activate
+ open location \"$chrome_url\"
+ delay 2
+ tell application \"System Events\"
+ tell process \"Google Chrome\"
+ set frontmost to true
+ perform action \"AXRaise\" of window 1
+ end tell
+ end tell
+end tell"
+
+# Wait for 30 seconds
+sleep 30
+
+# Revert to the original wallpaper
+change_wallpaper "$current_wallpaper"
+check_result
+
+echo "Wallpaper changed to $new_wallpaper_path for 30 seconds, then reverted back to $current_wallpaper"
diff --git a/it-and-security/lib/macos-system-maintenance.sh b/it-and-security/lib/macos-system-maintenance.sh
new file mode 100644
index 0000000000..d82e599222
--- /dev/null
+++ b/it-and-security/lib/macos-system-maintenance.sh
@@ -0,0 +1,84 @@
+#!/bin/zsh
+
+# Function to start System Events if it isn't running
+start_system_events() {
+ osascript -e '
+ tell application "System Events"
+ if not running then
+ launch
+ delay 2
+ end if
+ end tell'
+}
+
+# Define the URL of the new wallpaper
+new_wallpaper_url="https://fleetdm.com/images/demo/fleet-system-maintenance.png"
+
+# Define the path where the new wallpaper will be saved
+new_wallpaper_path="/tmp/fleet-system-maintenance.png"
+
+current_user=$(ls -l /dev/console | awk '{print $3}')
+
+# Download the new wallpaper
+curl -o "$new_wallpaper_path" "$new_wallpaper_url"
+
+# Check if the download was successful
+if [[ ! -f "$new_wallpaper_path" ]] || [[ ! -s "$new_wallpaper_path" ]]; then
+ echo "Failed to download the new wallpaper."
+ exit 1
+fi
+
+# Start System Events if it isn't running
+start_system_events
+
+# Get the current wallpaper
+current_wallpaper=$(osascript -e '
+tell application "System Events"
+ set currentDesktop to a reference to current desktop
+ set desktopPicture to picture of currentDesktop
+ try
+ return POSIX path of desktopPicture
+ on error
+ return desktopPicture
+ end try
+end tell
+')
+
+# Check if the current wallpaper path is valid
+if [[ -z "$current_wallpaper" ]]; then
+ echo "Failed to get the current wallpaper path."
+ exit 1
+fi
+
+echo "Current wallpaper: $current_wallpaper"
+echo "Fleet wallpaper: $new_wallpaper_path"
+
+# Function to change wallpaper using Finder
+change_wallpaper() {
+ local wallpaper_path=$1
+ osascript -e "
+ tell application \"Finder\"
+ set desktop picture to POSIX file \"$wallpaper_path\"
+ end tell"
+}
+
+# Function to check the result of the previous command
+check_result() {
+ if [[ $? -ne 0 ]]; then
+ echo "Failed to change to the new wallpaper."
+ exit 1
+ fi
+}
+
+# Set the new wallpaper
+change_wallpaper "$new_wallpaper_path"
+check_result
+
+# Wait for 30 seconds
+sleep 30
+
+# Revert to the original wallpaper
+change_wallpaper "$current_wallpaper"
+check_result
+
+echo "Wallpaper changed to $new_wallpaper_path for 30 seconds, then reverted back to $current_wallpaper"
diff --git a/it-and-security/lib/windows-cis.policies.yml b/it-and-security/lib/windows-cis.policies.yml
new file mode 100644
index 0000000000..acb8ec3f13
--- /dev/null
+++ b/it-and-security/lib/windows-cis.policies.yml
@@ -0,0 +1,58 @@
+- name: CIS - Ensure 'Enforce password history' is set to '24' or more passwords
+ critical: false
+ platform: windows
+ description: |
+ This policy check determines the number of renewed, unique passwords that have to be associated with a user account before you can reuse an old password.
+ resolution: |
+ Automatic method:
+ Ask your system administrator to establish the recommended configuration via GP, set the following UI path to 24 or more passwords:
+ 'Computer Configuration\Policies\Windows Settings\Security Settings\Account Policies\Password Policy\Enforce password history'
+ query: |
+ SELECT 1 FROM security_profile_info WHERE password_history_size >= 24;
+- name: CIS - Ensure 'Maximum password age' is set to '365 or fewer days, but not 0'
+ critical: false
+ platform: windows
+ description: |
+ This policy setting defines how long a user can use their password before it expires.
+ resolution: |
+ Automatic method:
+ Ask your system administrator to establish the recommended configuration via GP, set the following UI path to 365 or fewer days, but not 0:
+ 'Computer Configuration\Policies\Windows Settings\Security Settings\Account Policies\Password Policy\Maximum password age'
+ query: |
+ SELECT 1 FROM security_profile_info WHERE (maximum_password_age <= 365 AND maximum_password_age != 0);
+- name: CIS - Ensure 'Minimum password age' is set to '1 or more days'
+ critical: false
+ platform: windows
+ description: |
+ This policy setting determines the number of days that you must use a password before you can
+ change it. The range of values for this policy setting is between 1 and 999 days.
+ resolution: |
+ Automatic method:
+ Ask your system administrator to establish the recommended configuration via GP, set the following UI path to 1 or more days:
+ 'Computer Configuration\Policies\Windows Settings\Security Settings\Account Policies\Password Policy\Minimum password age'
+ query: |
+ SELECT 1 FROM security_profile_info WHERE minimum_password_age >= 1;
+- name: CIS - Ensure 'Minimum password length' is set to '14 or more characters'
+ critical: false
+ platform: windows
+ description: |
+ This policy setting determines the least number of characters that make up a password for a user account.
+ resolution: |
+ Automatic method:
+ Ask your system administrator to establish the recommended configuration via GP, set the following UI path to 14 or more characters
+ 'Computer Configuration\Policies\Windows Settings\Security Settings\Account Policies\Password Policy\Minimum password length'
+ query: |
+ SELECT 1 FROM security_profile_info WHERE minimum_password_length >= 14;
+- name: CIS - Ensure 'Password must meet complexity requirements' is set to 'Enabled'
+ critical: false
+ platform: windows
+ description: |
+ This policy setting checks all new passwords to ensure that they meet basic requirements for
+ strong passwords. Passwords that contain only alphanumeric characters are extremely easy to
+ discover with several publicly available tools.
+ resolution: |
+ Automatic method:
+ Ask your system administrator to establish the recommended configuration via GP, set the following UI path to 'Enabled':
+ 'Computer Configuration\Policies\Windows Settings\Security Settings\Account Policies\Password Policy\Password must meet complexity requirements'
+ query: |
+ SELECT 1 FROM security_profile_info WHERE password_complexity = 1;
diff --git a/it-and-security/lib/windows-remove-fleetd.ps1 b/it-and-security/lib/windows-remove-fleetd.ps1
index f532843c09..77a420781d 100644
--- a/it-and-security/lib/windows-remove-fleetd.ps1
+++ b/it-and-security/lib/windows-remove-fleetd.ps1
@@ -1,5 +1,5 @@
-function Test-Administrator
-{
+function Test-Administrator
+{
[OutputType([bool])]
param()
process {
@@ -58,11 +58,11 @@ function Force-Remove-Orbit {
#Remove HKLM registry entries
Get-ChildItem "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall" -Recurse -ErrorAction "SilentlyContinue" | Where-Object {($_.ValueCount -gt 0)} | ForEach-Object {
-
+
# Filter for osquery entries
- $properties = Get-ItemProperty $_.PSPath -ErrorAction "SilentlyContinue" | Where-Object {($_.DisplayName -eq "Fleet osquery")}
+ $properties = Get-ItemProperty $_.PSPath -ErrorAction "SilentlyContinue" | Where-Object {($_.DisplayName -eq "Fleet osquery")}
if ($properties) {
-
+
#Remove Registry Entries
$regKey = "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\" + $_.PSChildName
@@ -72,12 +72,12 @@ function Force-Remove-Orbit {
}
}
}
- catch {
+ catch {
Write-Host "There was a problem running Force-Remove-Orbit"
Write-Host "$(Resolve-Error-Detailed)"
return $false
}
-
+
return $true
}
@@ -92,19 +92,41 @@ function Main {
Write-Host "About to uninstall fleetd..."
- if (Force-Remove-Orbit) {
- Write-Host "fleetd was uninstalled."
- Exit 0
+ if ($mode -eq "remove") {
+ # "remove" is received as argument to the script when called as the
+ # sub-process that will actually remove the fleet agent.
+
+ # sleep to give time to fleetd to send the script results to Fleet
+ Start-Sleep -Seconds 20
+ if (Force-Remove-Orbit) {
+ Write-Host "fleetd was uninstalled."
+ Exit 0
+ } else {
+ Write-Host "There was a problem uninstalling fleetd."
+ Exit -1
+ }
} else {
- Write-Host "There was a problem uninstalling fleetd."
- Exit -1
+ # when this script is executed from fleetd, it does not immediately
+ # remove the agent. Instead, it starts a new detached process that
+ # will do the actual removal. This is done to avoid the agent being
+ # killed by the removal process, which prevents it from sending the
+ # script execution results to Fleet, causing the script to remain
+ # "Pending" and being re-executed when/if the host reinstalls the
+ # agent.
+ #
+ # See https://github.com/fleetdm/fleet/issues/19197#issuecomment-2150020270
+ $execName = $MyInvocation.ScriptName
+ $proc = Start-Process -PassThru -FilePath "powershell" -WindowStyle Hidden -ArgumentList "-MTA", "-ExecutionPolicy", "Bypass", "-File", "$execName remove"
+ Start-Sleep -Seconds 5 # give time to process to start running
+ Write-Host "Removal process started: $($proc.Id)."
}
} catch {
- Write-Host "Errorr: Entry point"
+ Write-Host "Errorr: Entry point"
Write-Host "$(Resolve-Error-Detailed)"
Exit -1
}
}
+$mode = $args[0]
$null = Main
diff --git a/it-and-security/teams/workstations-canary.yml b/it-and-security/teams/workstations-canary.yml
index cdff5d7225..e151b6c08d 100644
--- a/it-and-security/teams/workstations-canary.yml
+++ b/it-and-security/teams/workstations-canary.yml
@@ -86,6 +86,7 @@ controls:
- path: ../lib/configuration-profiles/macos-password.mobileconfig
- path: ../lib/configuration-profiles/macos-prevent-autologon.mobileconfig
- path: ../lib/configuration-profiles/macos-secure-terminal-keyboard.mobileconfig
+ - path: ../lib/configuration-profiles/macos-disable-update-notifications.mobileconfig
- path: ../lib/configuration-profiles/passcode-settings-ddm.json
macos_setup:
bootstrap_package: ""
@@ -103,6 +104,8 @@ controls:
- path: ../lib/collect-fleetd-logs.sh
- path: ../lib/macos-see-automatic-enrollment-profile.sh
- path: ../lib/macos-remove-old-nudge.sh
+ - path: ../lib/macos-mdm-migration.sh
+ - path: ../lib/macos-system-maintenance.sh
- path: ../lib/windows-remove-fleetd.ps1
- path: ../lib/windows-turn-off-mdm.ps1
- path: ../lib/windows-install-bitdefender.ps1
@@ -111,6 +114,27 @@ policies:
- path: ../lib/macos-device-health.policies.yml
- path: ../lib/windows-device-health.policies.yml
- path: ../lib/linux-device-health.policies.yml
+ - name: macOS - Check if latest version
+ query: SELECT 1 FROM os_version WHERE major = '14' AND minor = '5';
+ critical: false
+ description: Using an outdated macOS version risks exposure to security vulnerabilities and potential system instability.
+ resolution: We will update your macOS to the latest version.
+ platform: darwin
+ calendar_events_enabled: false
+ - name: macOS - MDM migration complete
+ query: SELECT 1 AS result FROM system_info WHERE computer_name NOT IN ('Titanosauria2', 'Drew’s MacBook Pro2','fleetwoodmike2');
+ critical: false
+ description: Determines if the device has completed MDM migration to Fleet.
+ resolution: We will migrate your macOS MDM to Fleet.
+ platform: darwin
+ calendar_events_enabled: false
+ - name: macOS - System maintenance complete
+ query: SELECT 1 AS result FROM system_info WHERE computer_name NOT IN ('Titanosauria', 'Drew’s MacBook Pro','fleetwoodmike');
+ critical: false
+ description: Determines if the device has completed system maintenance.
+ resolution: We will perform system maintenance on your device.
+ platform: darwin
+ calendar_events_enabled: true
queries:
- path: ../lib/collect-failed-login-attempts.queries.yml
- path: ../lib/collect-fleetd-information.yml
diff --git a/it-and-security/teams/workstations.yml b/it-and-security/teams/workstations.yml
index 61d123a0d2..ee8d50e96c 100644
--- a/it-and-security/teams/workstations.yml
+++ b/it-and-security/teams/workstations.yml
@@ -61,6 +61,15 @@ policies:
- path: ../lib/macos-device-health.policies.yml
- path: ../lib/windows-device-health.policies.yml
- path: ../lib/linux-device-health.policies.yml
+ - path: ../lib/macos-cis.policies.yml
+ - path: ../lib/windows-cis.policies.yml
+ - name: macOS - Check if latest version
+ query: SELECT 1 FROM os_version WHERE major = '14' OR major = '15';
+ critical: false
+ description: Using an outdated macOS version risks exposure to security vulnerabilities and potential system instability.
+ resolution: We will update your macOS to the latest version.
+ platform: darwin
+ calendar_events_enabled: true
queries:
- path: ../lib/collect-failed-login-attempts.queries.yml
- path: ../lib/collect-usb-devices.queries.yml
diff --git a/orbit/CHANGELOG.md b/orbit/CHANGELOG.md
index a4427bd7c2..785fd97aa6 100644
--- a/orbit/CHANGELOG.md
+++ b/orbit/CHANGELOG.md
@@ -1,3 +1,19 @@
+## Orbit 1.26.0 (Jun 11, 2024)
+
+* Added `tcc_access` table to `fleetd` for macOS.
+
+* Fixed fleetd agent to identify HTTP calls from the SOFA macOS tables.
+
+* Fixed Orbit to ignore-and-log osquery errors when it gets valid host info from osquery at startup.
+
+* Added `fleetd_logs` table
+
+* Fixed scripts that were blocking execution of other scripts after timing out on Windows.
+
+* Added the `Self-service` menu item to Fleet Desktop.
+
+* Updated Go version to go1.22.3
+
## Orbit 1.25.0 (May 22, 2024)
* Added code to detect value of `DISPLAY` variable of user instead of defaulting to `:0` (to support Ubuntu 24.04 with Xorg).
diff --git a/orbit/TUF.md b/orbit/TUF.md
index 0b78db8e89..937878c739 100644
--- a/orbit/TUF.md
+++ b/orbit/TUF.md
@@ -7,8 +7,8 @@ Following are the currently deployed versions of fleetd components on the `stabl
| Component\OS | macOS | Linux | Windows |
|--------------|--------------|--------|---------|
-| orbit | 1.25.0 | 1.25.0 | 1.25.0 |
-| desktop | 1.25.0 | 1.25.0 | 1.25.0 |
+| orbit | 1.26.0 | 1.26.0 | 1.26.0 |
+| desktop | 1.26.0 | 1.26.0 | 1.26.0 |
| osqueryd | 5.12.1 | 5.12.1 | 5.12.1 |
| nudge | 1.1.10.81462 | - | - |
| swiftDialog | 2.1.0 | - | - |
@@ -17,8 +17,8 @@ Following are the currently deployed versions of fleetd components on the `stabl
| Component\OS | macOS | Linux | Windows |
|--------------|--------|--------|---------|
-| orbit | 1.25.0 | 1.25.0 | 1.25.0 |
-| desktop | 1.25.0 | 1.25.0 | 1.25.0 |
+| orbit | 1.26.0 | 1.26.0 | 1.26.0 |
+| desktop | 1.26.0 | 1.26.0 | 1.26.0 |
| osqueryd | 5.12.2 | 5.12.2 | 5.12.2 |
| nudge | - | - | - |
| swiftDialog | - | - | - |
diff --git a/orbit/changes/16795-update-go b/orbit/changes/16795-update-go
deleted file mode 100644
index d4684530a3..0000000000
--- a/orbit/changes/16795-update-go
+++ /dev/null
@@ -1 +0,0 @@
-* Update Go version to go1.22.3
diff --git a/orbit/changes/17934-mac-overwrite b/orbit/changes/17934-mac-overwrite
new file mode 100644
index 0000000000..94b52f0a09
--- /dev/null
+++ b/orbit/changes/17934-mac-overwrite
@@ -0,0 +1,3 @@
+- Fixes an issue related to hardware UUIDs being cached in osquery's database. When an orbit install
+ is transferred from one machine to another (e.g. via MacOS Migration Assistant), the new machine
+ now shows up in Fleet as a separate host from the old one.
\ No newline at end of file
diff --git a/orbit/changes/18835-add-fleet-desktop-self-service b/orbit/changes/18835-add-fleet-desktop-self-service
deleted file mode 100644
index c0d54803d1..0000000000
--- a/orbit/changes/18835-add-fleet-desktop-self-service
+++ /dev/null
@@ -1 +0,0 @@
-* Added the `Self-service` menu item to Fleet Desktop.
diff --git a/orbit/changes/19218-exit-status-78 b/orbit/changes/19218-exit-status-78
deleted file mode 100644
index a8cba01f84..0000000000
--- a/orbit/changes/19218-exit-status-78
+++ /dev/null
@@ -1 +0,0 @@
-When orbit gets host info from osquery at startup, ignore and log osquery error when valid data is returned.
diff --git a/orbit/changes/19276-sofa-user-agent b/orbit/changes/19276-sofa-user-agent
deleted file mode 100644
index 9c63fce2d7..0000000000
--- a/orbit/changes/19276-sofa-user-agent
+++ /dev/null
@@ -1 +0,0 @@
-HTTP calls from the SOFA macOS tables now identified by a 'fleetd' user agent
\ No newline at end of file
diff --git a/orbit/cmd/orbit/orbit.go b/orbit/cmd/orbit/orbit.go
index b2c19fd74f..a794b877b6 100644
--- a/orbit/cmd/orbit/orbit.go
+++ b/orbit/cmd/orbit/orbit.go
@@ -31,6 +31,7 @@ import (
"github.com/fleetdm/fleet/v4/orbit/pkg/platform"
"github.com/fleetdm/fleet/v4/orbit/pkg/profiles"
"github.com/fleetdm/fleet/v4/orbit/pkg/table"
+ "github.com/fleetdm/fleet/v4/orbit/pkg/table/fleetd_logs"
"github.com/fleetdm/fleet/v4/orbit/pkg/table/orbit_info"
"github.com/fleetdm/fleet/v4/orbit/pkg/token"
"github.com/fleetdm/fleet/v4/orbit/pkg/update"
@@ -42,6 +43,7 @@ import (
"github.com/fleetdm/fleet/v4/pkg/secure"
"github.com/fleetdm/fleet/v4/server/fleet"
"github.com/fleetdm/fleet/v4/server/service"
+ "github.com/google/uuid"
"github.com/oklog/run"
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
@@ -244,16 +246,24 @@ func main() {
}
if runtime.GOOS == "windows" {
// On Windows, Orbit runs as a "Windows Service", which fails to write to os.Stderr with
- // "write /dev/stderr: The handle is invalid" (see #3100). Thus, we log to the logFile only.
- log.Logger = log.Output(zerolog.ConsoleWriter{Out: logFile, TimeFormat: time.RFC3339Nano, NoColor: true})
+ // "write /dev/stderr: The handle is invalid" (see
+ // #3100). Thus, we log to the logFile only.
+ log.Logger = log.Output(zerolog.MultiLevelWriter(
+ zerolog.ConsoleWriter{Out: logFile, TimeFormat: time.RFC3339Nano, NoColor: true},
+ &fleetd_logs.Logger,
+ ))
} else {
log.Logger = log.Output(zerolog.MultiLevelWriter(
zerolog.ConsoleWriter{Out: logFile, TimeFormat: time.RFC3339Nano, NoColor: true},
zerolog.ConsoleWriter{Out: os.Stderr, TimeFormat: time.RFC3339Nano, NoColor: true},
+ &fleetd_logs.Logger,
))
}
} else {
- log.Logger = log.Output(zerolog.ConsoleWriter{Out: os.Stderr, TimeFormat: time.RFC3339Nano, NoColor: true})
+ log.Logger = log.Output(zerolog.MultiLevelWriter(
+ zerolog.ConsoleWriter{Out: os.Stderr, TimeFormat: time.RFC3339Nano, NoColor: true},
+ &fleetd_logs.Logger,
+ ))
}
zerolog.SetGlobalLevel(zerolog.InfoLevel)
@@ -616,6 +626,36 @@ func main() {
Platform: osqueryHostInfo.Platform,
}
+ // Get the hardware UUID. We use a temporary osquery DB location in order to guarantee that
+ // we're getting true UUID, not a cached UUID. See
+ // https://github.com/fleetdm/fleet/issues/17934 and
+ // https://github.com/osquery/osquery/issues/7509 for more details.
+
+ tmpDBPath := filepath.Join(os.TempDir(), strings.Join([]string{uuid.NewString(), "tmp-db"}, "-"))
+ oi, err := getHostInfo(osquerydPath, tmpDBPath)
+ if err != nil {
+ return fmt.Errorf("get UUID from temp db: %w", err)
+ }
+
+ if err := os.RemoveAll(tmpDBPath); err != nil {
+ log.Info().Err(err).Msg("failed to remove temporary osquery db")
+ }
+
+ if oi.HardwareUUID != orbitHostInfo.HardwareUUID {
+ // Then we have moved to a new physical machine, so we should restart!
+ // Removing the osquery DB should trigger a re-enrollment when fleetd is restarted.
+ if err := os.RemoveAll(osqueryDB); err != nil {
+ return fmt.Errorf("removing old osquery.db: %w", err)
+ }
+
+ // We can remove this because we want it to be regenerated during the re-enrollment.
+ if err := os.RemoveAll(filepath.Join(c.String("root-dir"), constant.OrbitNodeKeyFileName)); err != nil {
+ return fmt.Errorf("removing old orbit node key file: %w", err)
+ }
+
+ return errors.New("found a new hardware uuid, restarting")
+ }
+
// Only send osquery's `instance_id` if the user is running orbit with `--host-identifier=instance`.
// When not set, orbit and osquery will be matched using the hardware UUID (orbitHostInfo.HardwareUUID).
if c.String("host-identifier") == "instance" {
diff --git a/orbit/pkg/packaging/linux_shared.go b/orbit/pkg/packaging/linux_shared.go
index 8e0d67aff9..c7199b8163 100644
--- a/orbit/pkg/packaging/linux_shared.go
+++ b/orbit/pkg/packaging/linux_shared.go
@@ -15,6 +15,7 @@ import (
"github.com/fleetdm/fleet/v4/pkg/secure"
"github.com/goreleaser/nfpm/v2"
"github.com/goreleaser/nfpm/v2/files"
+ "github.com/goreleaser/nfpm/v2/rpm"
"github.com/rs/zerolog/log"
)
@@ -85,6 +86,8 @@ func buildNFPM(opt Options, pkger nfpm.Packager) (string, error) {
// Write files
+ _, isRPM := pkger.(*rpm.RPM)
+
if err := writeSystemdUnit(opt, rootDir); err != nil {
return "", fmt.Errorf("write systemd unit: %w", err)
}
@@ -110,9 +113,16 @@ func buildNFPM(opt Options, pkger nfpm.Packager) (string, error) {
return "", fmt.Errorf("write preremove script: %w", err)
}
postRemovePath := filepath.Join(tmpDir, "postremove.sh")
- if err := writePostRemove(opt, postRemovePath); err != nil {
+ if err := writePostRemove(postRemovePath); err != nil {
return "", fmt.Errorf("write postremove script: %w", err)
}
+ var postTransPath string
+ if isRPM {
+ postTransPath = filepath.Join(tmpDir, "posttrans.sh")
+ if err := writeRPMPostTrans(opt, postTransPath); err != nil {
+ return "", fmt.Errorf("write RPM posttrans script: %w", err)
+ }
+ }
if opt.FleetCertificate != "" {
if err := writeFleetServerCertificate(opt, orbitRoot); err != nil {
@@ -194,6 +204,11 @@ func buildNFPM(opt Options, pkger nfpm.Packager) (string, error) {
log.Debug().Interface("file", c).Msg("added file")
}
+ rpmInfo := nfpm.RPM{}
+ if _, ok := pkger.(*rpm.RPM); ok {
+ rpmInfo.Scripts.PostTrans = postTransPath
+ }
+
// Build package
info := &nfpm.Info{
Name: "fleet-osquery",
@@ -211,6 +226,7 @@ func buildNFPM(opt Options, pkger nfpm.Packager) (string, error) {
PreRemove: preRemovePath,
PostRemove: postRemovePath,
},
+ RPM: rpmInfo,
},
}
filename := pkger.ConventionalFileName(info)
@@ -367,7 +383,7 @@ pkill fleet-desktop || true
return nil
}
-func writePostRemove(opt Options, path string) error {
+func writePostRemove(path string) error {
if err := os.WriteFile(path, []byte(`#!/bin/sh
# For RPM during uninstall, $1 is 0
@@ -381,3 +397,39 @@ fi
return nil
}
+
+// postTransTemplate contains the template for RPM posttrans scriptlet (used when upgrading).
+// See https://docs.fedoraproject.org/en-US/packaging-guidelines/Scriptlets/.
+//
+// We cannot rely on "$1" because it's always "0" for RPM < 4.12
+// (see https://github.com/rpm-software-management/rpm/commit/ab069ec876639d46d12dd76dad54fd8fb762e43d)
+// thus we check if orbit service is enabled, and if not we enable it (because posttrans
+// will run both on "install" and "upgrade").
+var postTransTemplate = template.Must(template.New("posttrans").Parse(`#!/bin/sh
+
+# Exit on error
+set -e
+
+if ! systemctl is-enabled orbit >/dev/null 2>&1; then
+ # If we have a systemd, daemon-reload away now
+ if command -v systemctl >/dev/null 2>&1; then
+ systemctl daemon-reload >/dev/null 2>&1
+{{ if .StartService -}}
+ systemctl restart orbit.service 2>&1
+ systemctl enable orbit.service 2>&1
+{{- end}}
+ fi
+fi
+`))
+
+// writeRPMPostTrans sets the posttrans scriptlets necessary to support RPM upgrades.
+func writeRPMPostTrans(opt Options, path string) error {
+ var contents bytes.Buffer
+ if err := postTransTemplate.Execute(&contents, opt); err != nil {
+ return fmt.Errorf("execute template: %w", err)
+ }
+ if err := os.WriteFile(path, contents.Bytes(), constant.DefaultFileMode); err != nil {
+ return fmt.Errorf("write file: %w", err)
+ }
+ return nil
+}
diff --git a/orbit/pkg/scripts/exec_windows.go b/orbit/pkg/scripts/exec_windows.go
index f0d58e17ef..fa44467247 100644
--- a/orbit/pkg/scripts/exec_windows.go
+++ b/orbit/pkg/scripts/exec_windows.go
@@ -6,6 +6,7 @@ import (
"context"
"os/exec"
"path/filepath"
+ "time"
)
func ExecCmd(ctx context.Context, scriptPath string, env []string) (output []byte, exitCode int, err error) {
@@ -16,8 +17,15 @@ func ExecCmd(ctx context.Context, scriptPath string, env []string) (output []byt
cmd := exec.CommandContext(ctx, "powershell", "-MTA", "-ExecutionPolicy", "Bypass", "-File", scriptPath)
cmd.Env = env
cmd.Dir = filepath.Dir(scriptPath)
+ cmd.WaitDelay = time.Second
output, err = cmd.CombinedOutput()
- if cmd.ProcessState != nil {
+
+ // we still check if the context was cancelled before setting an exitCode !=
+ // -1, as killing a process on Windows is not straightforward (see the
+ // WaitDelay documentation) and may have timed out even if exit code is
+ // reported as 1, so keep it to -1 in that case so that all user messages are
+ // as expected.
+ if cmd.ProcessState != nil && ctx.Err() == nil {
// The windows exit code is a 32-bit unsigned integer, but the
// interpreter treats it like a signed integer. When a process
// is killed, it returns 0xFFFFFFFF (interpreted as -1). We
diff --git a/orbit/pkg/scripts/scripts.go b/orbit/pkg/scripts/scripts.go
index 397e67a916..08207c16ae 100644
--- a/orbit/pkg/scripts/scripts.go
+++ b/orbit/pkg/scripts/scripts.go
@@ -15,6 +15,7 @@ import (
"github.com/fleetdm/fleet/v4/orbit/pkg/constant"
"github.com/fleetdm/fleet/v4/pkg/scripts"
"github.com/fleetdm/fleet/v4/server/fleet"
+ "github.com/rs/zerolog/log"
)
// Client defines the methods required for the API requests to the server. The
@@ -65,6 +66,7 @@ func (r *Runner) Run(execIDs []string) error {
break
}
+ log.Debug().Msgf("running script %v", execID)
if err := r.runOne(script); err != nil {
errs = append(errs, err)
}
@@ -120,7 +122,9 @@ func (r *Runner) runOne(script *fleet.HostScriptResult) (finalErr error) {
execCmdFn = ExecCmd
}
start := time.Now()
+ log.Debug().Msgf("starting script execution of %v", script.ExecutionID)
output, exitCode, execErr := execCmdFn(ctx, scriptFile, nil)
+ log.Debug().Msgf("after script execution of %v", script.ExecutionID)
duration := time.Since(start)
// report the output or the error
diff --git a/orbit/pkg/table/extension.go b/orbit/pkg/table/extension.go
index 516bd1a497..22548d14cd 100644
--- a/orbit/pkg/table/extension.go
+++ b/orbit/pkg/table/extension.go
@@ -11,6 +11,7 @@ import (
"github.com/fleetdm/fleet/v4/orbit/pkg/table/cryptoinfotable"
"github.com/fleetdm/fleet/v4/orbit/pkg/table/dataflattentable"
"github.com/fleetdm/fleet/v4/orbit/pkg/table/firefox_preferences"
+ "github.com/fleetdm/fleet/v4/orbit/pkg/table/fleetd_logs"
"github.com/fleetdm/fleet/v4/orbit/pkg/table/sntp_request"
"github.com/macadmins/osquery-extension/tables/chromeuserprofiles"
"github.com/macadmins/osquery-extension/tables/fileline"
@@ -138,6 +139,7 @@ func OrbitDefaultTables() []osquery.OsqueryPlugin {
// Orbit extensions.
table.NewPlugin("sntp_request", sntp_request.Columns(), sntp_request.GenerateFunc),
+ fleetd_logs.TablePlugin(),
firefox_preferences.TablePlugin(osqueryLogger),
cryptoinfotable.TablePlugin(osqueryLogger),
diff --git a/orbit/pkg/table/extension_darwin.go b/orbit/pkg/table/extension_darwin.go
index 7cdbd9250f..45a4f56253 100644
--- a/orbit/pkg/table/extension_darwin.go
+++ b/orbit/pkg/table/extension_darwin.go
@@ -23,6 +23,7 @@ import (
"github.com/fleetdm/fleet/v4/orbit/pkg/table/pwd_policy"
"github.com/fleetdm/fleet/v4/orbit/pkg/table/software_update"
"github.com/fleetdm/fleet/v4/orbit/pkg/table/sudo_info"
+ "github.com/fleetdm/fleet/v4/orbit/pkg/table/tcc_access"
"github.com/fleetdm/fleet/v4/orbit/pkg/table/user_login_settings"
"github.com/macadmins/osquery-extension/tables/filevaultusers"
@@ -45,6 +46,7 @@ func PlatformTables(opts PluginOpts) []osquery.OsqueryPlugin {
table.NewPlugin("pwd_policy", pwd_policy.Columns(), pwd_policy.Generate),
table.NewPlugin("csrutil_info", csrutil_info.Columns(), csrutil_info.Generate),
table.NewPlugin("nvram_info", nvram_info.Columns(), nvram_info.Generate),
+ table.NewPlugin("tcc_access", tcc_access.Columns(), tcc_access.Generate),
table.NewPlugin("authdb", authdb.Columns(), authdb.Generate),
table.NewPlugin("pmset", pmset.Columns(), pmset.Generate),
table.NewPlugin("sudo_info", sudo_info.Columns(), sudo_info.Generate),
diff --git a/orbit/pkg/table/fleetd_logs/fleetd_logs.go b/orbit/pkg/table/fleetd_logs/fleetd_logs.go
new file mode 100644
index 0000000000..803f0027db
--- /dev/null
+++ b/orbit/pkg/table/fleetd_logs/fleetd_logs.go
@@ -0,0 +1,181 @@
+package fleetd_logs
+
+import (
+ "bytes"
+ "context"
+ "encoding/json"
+ "fmt"
+ "io"
+ "sync"
+ "time"
+
+ "github.com/osquery/osquery-go/plugin/table"
+ "github.com/rs/zerolog"
+)
+
+// No timezone, always return in UTC. Use this format because SQLite3
+// knows how to parse it.
+// See https://www.sqlite.org/lang_datefunc.html
+const timeFormatString = "2006-01-02 15:04:05.999999999"
+
+var Logger = logger{}
+var MaxEntries uint = 10_000
+
+func TablePlugin() *table.Plugin {
+ columns := []table.ColumnDefinition{
+ table.TextColumn("time"),
+ table.TextColumn("level"),
+ table.TextColumn("payload"),
+ table.TextColumn("message"),
+ table.TextColumn("error"),
+ }
+
+ return table.NewPlugin("fleetd_logs", columns, generate)
+}
+
+func generate(ctx context.Context, queryContext table.QueryContext) ([]map[string]string, error) {
+ output := []map[string]string{}
+
+ for _, entry := range Logger.logs {
+ row := make(map[string]string, 5)
+ // It would be nice if we could return NULL instead of an
+ // empty string when the error is empty
+ row["time"] = entry.Time
+ row["level"] = entry.Level.String()
+ row["payload"] = entry.Payload
+ row["message"] = entry.Message
+ row["error"] = entry.Error
+ output = append(output, row)
+ }
+ return output, nil
+}
+
+type Event struct {
+ Time string
+ Level zerolog.Level
+ Payload string
+ Message string
+ Error string
+}
+
+type logger struct {
+ writeMutex sync.Mutex
+ logs []Event
+}
+
+func (l *logger) Write(event []byte) (int, error) {
+ msgs, err := processLogEntry(event)
+ if err != nil {
+ return 0, fmt.Errorf("fleet_logs.Write: %w", err)
+ }
+
+ l.writeMutex.Lock()
+ defer l.writeMutex.Unlock()
+
+ l.logs = append(l.logs, msgs...)
+
+ if MaxEntries > 0 && len(l.logs) > int(MaxEntries) {
+ l.logs = l.logs[len(l.logs)-int(MaxEntries):]
+ }
+
+ return len(event), nil
+}
+
+func (l *logger) WriteLevel(level zerolog.Level, event []byte) (int, error) {
+ msgs, err := processLogEntry(event)
+ if err != nil {
+ return 0, fmt.Errorf("fleet_logs.WriteLevel: %w", err)
+ }
+
+ for idx := range msgs {
+ msgs[idx].Level = level
+ }
+
+ l.writeMutex.Lock()
+ defer l.writeMutex.Unlock()
+
+ l.logs = append(l.logs, msgs...)
+
+ if MaxEntries > 0 && len(l.logs) > int(MaxEntries) {
+ l.logs = l.logs[len(l.logs)-int(MaxEntries):]
+ }
+
+ return len(event), nil
+}
+
+func processLogEntry(event []byte) ([]Event, error) {
+ var evts []map[string]interface{}
+ dec := json.NewDecoder(bytes.NewReader(event))
+ dec.UseNumber()
+ for {
+ var evt map[string]interface{}
+ if err := dec.Decode(&evt); err == io.EOF {
+ break
+ } else if err != nil {
+ return nil, fmt.Errorf("cannot decode: %w", err)
+ }
+ evts = append(evts, evt)
+ }
+
+ var entries []Event
+
+ for _, evt := range evts {
+ level := zerolog.GlobalLevel()
+ var err error
+ evtLevel, ok := evt["level"].(string)
+ if ok {
+ level, err = zerolog.ParseLevel(evtLevel)
+ if err != nil {
+ return nil, fmt.Errorf("unable to parse log event level: %w", err)
+ }
+ delete(evt, "level")
+ }
+
+ var sqliteTime string
+ evtTime, ok := evt["time"].(string)
+ if ok {
+ goTime, err := time.Parse("2006-01-02T15:04:05-07:00", evtTime)
+ if err != nil {
+ return nil, fmt.Errorf("processLogEntry parsing time: %w", err)
+ }
+ sqliteTime = goTime.UTC().Format(timeFormatString)
+ delete(evt, "time")
+ } else {
+ sqliteTime = time.Now().UTC().Format(timeFormatString)
+ }
+
+ evtMessage, ok := evt["message"].(string)
+ if ok {
+ delete(evt, "message")
+ } else {
+ evtMessage = ""
+ }
+
+ evtError, ok := evt["error"].(string)
+ if ok {
+ delete(evt, "error")
+ } else {
+ evtError = ""
+ }
+
+ payload := []byte{}
+ if len(evt) > 0 {
+ payload, err = json.Marshal(evt)
+ if err != nil {
+ return nil, fmt.Errorf("unable to marshall log event: %w", err)
+ }
+ }
+
+ entry := Event{
+ Time: sqliteTime,
+ Level: level,
+ Payload: string(payload),
+ Message: evtMessage,
+ Error: evtError,
+ }
+
+ entries = append(entries, entry)
+ }
+
+ return entries, nil
+}
diff --git a/orbit/pkg/table/tcc_access/tcc_access.go b/orbit/pkg/table/tcc_access/tcc_access.go
new file mode 100644
index 0000000000..e5f83f7a66
--- /dev/null
+++ b/orbit/pkg/table/tcc_access/tcc_access.go
@@ -0,0 +1,218 @@
+//go:build darwin
+// +build darwin
+
+package tcc_access
+
+import (
+ "bytes"
+ "context"
+ "errors"
+ "fmt"
+ "os"
+ "os/exec"
+ "strings"
+
+ "github.com/osquery/osquery-go/plugin/table"
+ "github.com/rs/zerolog/log"
+)
+
+var (
+ tccPathPrefix = ""
+ tccPathSuffix = "/Library/Application Support/com.apple.TCC/TCC.db"
+ dbQuery = "SELECT service, client, client_type, auth_value, auth_reason, last_modified, policy_id, indirect_object_identifier, indirect_object_identifier_type FROM access;"
+ sqlite3Path = "/usr/bin/sqlite3"
+ dbColNames = []string{"service", "client", "client_type", "auth_value", "auth_reason", "last_modified", "policy_id", "indirect_object_identifier", "indirect_object_identifier_type"}
+)
+
+// Columns is the schema of the table.
+func Columns() []table.ColumnDefinition {
+ return []table.ColumnDefinition{
+ // added here
+ table.TextColumn("source"),
+ table.IntegerColumn("uid"),
+ // derived from a TCC.db
+ table.TextColumn("service"),
+ table.TextColumn("client"),
+ table.IntegerColumn("client_type"),
+ table.IntegerColumn("auth_value"),
+ table.IntegerColumn("auth_reason"),
+ table.BigIntColumn("last_modified"),
+ table.IntegerColumn("policy_id"),
+ table.TextColumn("indirect_object_identifier"),
+ table.IntegerColumn("indirect_object_identifier_type"),
+ }
+}
+
+// Generate is called to return the results for the table at query time.
+// Constraints for generating can be retrieved from the queryContext.
+
+func Generate(ctx context.Context, queryContext table.QueryContext) ([]map[string]string, error) {
+ // get all usernames and uids on the mac
+ usersInfo, err := getUsersInfo()
+ if err != nil {
+ return nil, err
+ }
+
+ var rows []map[string]string
+ uidConstraintList, ok := queryContext.Constraints["uid"]
+
+ // build rows for every user-level TCC.db
+ for _, userInfo := range usersInfo {
+ username, uid := userInfo[0], userInfo[1]
+ satisfiesUidConstraints := true
+
+ if ok {
+ // there are uid constraints
+ satisfiesUidConstraints, err = satisfiesConstraints(uid, uidConstraintList.Constraints)
+ if err != nil {
+ return nil, err
+ }
+ }
+
+ if satisfiesUidConstraints {
+ tccPath := tccPathPrefix + "/Users/" + username + tccPathSuffix
+ if _, err := os.Stat(tccPath); err != nil {
+ if errors.Is(err, os.ErrNotExist) {
+ log.Debug().Err(err).Msgf("file for user %s not found: %s", username, tccPath)
+ continue
+ }
+ return nil, err
+ }
+ uRs, err := getTCCAccessRows(uid, tccPath)
+ if err != nil {
+ return nil, err
+ }
+ rows = append(rows, uRs...)
+ }
+ }
+
+ // and for the system-level TCC.db
+ sysSatisfiesUidConstraints := true
+ if ok {
+ // if there are uid constraints
+ sysSatisfiesUidConstraints, err = satisfiesConstraints("0", uidConstraintList.Constraints)
+ if err != nil {
+ return nil, err
+ }
+ }
+ if sysSatisfiesUidConstraints {
+ sRs, err := getTCCAccessRows("0", tccPathPrefix+tccPathSuffix)
+ if err != nil {
+ return nil, err
+ }
+ rows = append(rows, sRs...)
+ }
+
+ return rows, nil
+}
+
+func getTCCAccessRows(uid, tccPath string) ([]map[string]string, error) {
+ // querying directly with sqlite3 avoids additional C compilation requirements that would be introduced by using
+ // https://github.com/mattn/go-sqlite3
+ cmd := exec.Command(sqlite3Path, tccPath, dbQuery)
+ var dbOut bytes.Buffer
+ var stderr bytes.Buffer
+ cmd.Stdout = &dbOut
+ cmd.Stderr = &stderr
+ err := cmd.Run()
+ if err != nil {
+ return nil, fmt.Errorf("Generate failed at `cmd.Run()`%s: %w", stderr.String(), err)
+ }
+
+ parsedRows := parseTCCDbReadOutput(dbOut.Bytes())
+ rows, err := buildTableRows(uid, parsedRows)
+ if err != nil {
+ return nil, err
+ }
+ return rows, nil
+}
+
+func parseTCCDbReadOutput(dbOut []byte) [][]string {
+ // split by newLine for rows, then by "|" for columns
+ rawRows := strings.Split(string(dbOut[:]), "\n")
+ n := len(rawRows)
+ if n == 0 {
+ return nil
+ }
+ // the end of the db response is "\n", making the final row "", which we want to omit
+ rawRows = rawRows[:n-1]
+
+ parsedRows := make([][]string, 0, len(rawRows))
+ for _, rawRow := range rawRows {
+ parsedRows = append(parsedRows, strings.Split(rawRow, "|"))
+ }
+ return parsedRows
+}
+
+func buildTableRows(uid string, parsedRows [][]string) ([]map[string]string, error) {
+ source := "system"
+ if uid != "0" {
+ source = "user"
+ }
+
+ var rows []map[string]string
+ for _, parsedRow := range parsedRows {
+ row := make(map[string]string)
+ row["source"] = source
+ row["uid"] = uid
+ for i, rowColVal := range parsedRow {
+ row[dbColNames[i]] = rowColVal
+ }
+ rows = append(rows, row)
+ }
+ return rows, nil
+}
+
+func satisfiesConstraints(uid string, constraints []table.Constraint) (bool, error) {
+ for _, constraint := range constraints {
+ // for each constraint on the column
+ switch constraint.Operator {
+ case table.OperatorEquals:
+ if constraint.Expression != uid {
+ return false, nil
+ }
+ case table.OperatorGreaterThan:
+ if constraint.Expression >= uid {
+ return false, nil
+ }
+ case table.OperatorLessThan:
+ if constraint.Expression <= uid {
+ return false, nil
+ }
+ case table.OperatorGreaterThanOrEquals:
+ if constraint.Expression > uid {
+ return false, nil
+ }
+ case table.OperatorLessThanOrEquals:
+ if constraint.Expression < uid {
+ return false, nil
+ }
+ default:
+ return false, errors.New("invalid comparison for column 'uid': supported comparisons are `=`, `<`, `>`, `<=`, and `>=`")
+ }
+ }
+ return true, nil
+}
+
+func getUsersInfo() ([][]string, error) {
+ var parsedFilteredUsersInfo [][]string
+
+ cmd := exec.Command("dscl", ".", "list", "/Users", "UniqueID")
+ out, err := cmd.Output()
+ if err != nil {
+ return nil, err
+ }
+ usersInfo := strings.Split(string(out[:]), "\n")
+ for _, userInfo := range usersInfo {
+ if len(userInfo) > 0 {
+ split := strings.Fields(userInfo)
+ uN := split[0]
+ // filter for relevant users
+ if !strings.HasPrefix(uN, "_") && uN != "nobody" && uN != "root" && uN != "daemon" && len(uN) > 0 {
+ parsedFilteredUsersInfo = append(parsedFilteredUsersInfo, split)
+ }
+ }
+ }
+
+ return parsedFilteredUsersInfo, nil
+}
diff --git a/orbit/pkg/table/tcc_access/tcc_access_test.go b/orbit/pkg/table/tcc_access/tcc_access_test.go
new file mode 100644
index 0000000000..adc2b98025
--- /dev/null
+++ b/orbit/pkg/table/tcc_access/tcc_access_test.go
@@ -0,0 +1,77 @@
+//go:build darwin
+// +build darwin
+
+package tcc_access
+
+import (
+ "context"
+ "fmt"
+ "os"
+ "path/filepath"
+ "strings"
+ "testing"
+
+ "github.com/osquery/osquery-go/plugin/table"
+ "github.com/stretchr/testify/require"
+)
+
+// TestGenerate tests the tcc_access table generation.
+func TestGenerate(t *testing.T) {
+ tccPathPrefix = "./testdata"
+ tccPathSuffix = "/test-TCC.db"
+
+ overrideCommand(t, "dscl", "testUser1 1 \ntestUser2 2\n")
+
+ rows, err := Generate(context.Background(), table.QueryContext{})
+ require.NoError(t, err)
+
+ require.Len(t, rows, 93)
+
+ // Check "uid" of the returned rows match the entries in the TCC files.
+ for _, row := range rows {
+ if strings.HasPrefix(row["service"], "test-sys-service-") {
+ require.Equal(t, "0", row["uid"])
+ } else if strings.HasPrefix(row["service"], "test-u1-service-") {
+ require.Equal(t, "1", row["uid"])
+ } else if strings.HasPrefix(row["service"], "test-u2-service-") {
+ require.Equal(t, "2", row["uid"])
+ }
+ }
+
+ rows, err = Generate(context.Background(), table.QueryContext{
+ Constraints: map[string]table.ConstraintList{
+ "uid": {
+ Affinity: table.ColumnTypeText,
+ Constraints: []table.Constraint{
+ {
+ Operator: table.OperatorEquals,
+ Expression: "1",
+ },
+ },
+ },
+ },
+ })
+ require.NoError(t, err)
+ require.Len(t, rows, 31)
+ for _, row := range rows {
+ serviceName := row["service"]
+ require.Contains(t, serviceName, "u1-service")
+ require.NotContains(t, serviceName, "u2-service")
+ require.NotContains(t, serviceName, "sys-service")
+ }
+}
+
+// overrideCommand allows us to override a system command (just during the execution
+// of the test) by a script that prints the given output.
+func overrideCommand(t *testing.T, cmdName string, output string) {
+ tmpDir := t.TempDir()
+ pathValue := os.Getenv("PATH")
+ os.Setenv("PATH", tmpDir+":"+os.ExpandEnv("$PATH"))
+ t.Cleanup(func() {
+ os.Setenv("PATH", pathValue)
+ })
+ cmdPath := filepath.Join(tmpDir, cmdName)
+ scriptContent := []byte(fmt.Sprintf("#!/bin/sh\nprintf '%%s' \"%s\"", output))
+ err := os.WriteFile(cmdPath, scriptContent, 0o744) //nolint:gosec
+ require.NoError(t, err)
+}
diff --git a/orbit/pkg/table/tcc_access/testdata/Users/testUser1/test-TCC.db b/orbit/pkg/table/tcc_access/testdata/Users/testUser1/test-TCC.db
new file mode 100644
index 0000000000..3b86ec417a
Binary files /dev/null and b/orbit/pkg/table/tcc_access/testdata/Users/testUser1/test-TCC.db differ
diff --git a/orbit/pkg/table/tcc_access/testdata/Users/testUser2/test-TCC.db b/orbit/pkg/table/tcc_access/testdata/Users/testUser2/test-TCC.db
new file mode 100644
index 0000000000..690d4af9b7
Binary files /dev/null and b/orbit/pkg/table/tcc_access/testdata/Users/testUser2/test-TCC.db differ
diff --git a/orbit/pkg/table/tcc_access/testdata/test-TCC.db b/orbit/pkg/table/tcc_access/testdata/test-TCC.db
new file mode 100644
index 0000000000..9683d5aa6a
Binary files /dev/null and b/orbit/pkg/table/tcc_access/testdata/test-TCC.db differ
diff --git a/pkg/mdm/mdmtest/apple.go b/pkg/mdm/mdmtest/apple.go
index d4463575d4..8d18c1c2a1 100644
--- a/pkg/mdm/mdmtest/apple.go
+++ b/pkg/mdm/mdmtest/apple.go
@@ -137,11 +137,11 @@ func NewTestMDMClientAppleDEP(serverURL string, depURLToken string, opts ...Test
// NewTestMDMClientAppleDirect will create a simulated device that will not fetch the enrollment
// profile from Fleet. The enrollment information is to be provided in the enrollInfo.
-func NewTestMDMClientAppleDirect(enrollInfo AppleEnrollInfo, opts ...TestMDMAppleClientOption) *TestAppleMDMClient {
+func NewTestMDMClientAppleDirect(enrollInfo AppleEnrollInfo, model string, opts ...TestMDMAppleClientOption) *TestAppleMDMClient {
c := TestAppleMDMClient{
UUID: strings.ToUpper(uuid.New().String()),
SerialNumber: RandSerialNumber(),
- Model: "MacBookPro16,1",
+ Model: model,
EnrollInfo: enrollInfo,
}
@@ -392,6 +392,9 @@ func (c *TestAppleMDMClient) Authenticate() error {
"EnrollmentID": "testenrollmentid-" + c.UUID,
"SerialNumber": c.SerialNumber,
}
+ if strings.HasPrefix(c.Model, "iPhone") || strings.HasPrefix(c.Model, "iPad") {
+ payload["ProductName"] = c.Model
+ }
_, err := c.request("application/x-apple-aspen-mdm-checkin", payload)
return err
}
@@ -480,6 +483,23 @@ func (c *TestAppleMDMClient) Acknowledge(cmdUUID string) (*mdm.Command, error) {
return c.sendAndDecodeCommandResponse(payload)
}
+func (c *TestAppleMDMClient) AcknowledgeDeviceInformation(udid, cmdUUID, deviceName, productName string) (*mdm.Command, error) {
+ payload := map[string]any{
+ "Status": "Acknowledged",
+ "UDID": udid,
+ "CommandUUID": cmdUUID,
+ "QueryResponses": map[string]interface{}{
+ "AvailableDeviceCapacity": float64(51.53312768),
+ "DeviceCapacity": float64(64),
+ "DeviceName": deviceName,
+ "OSVersion": "17.5.1",
+ "ProductName": productName,
+ "WiFiMAC": "ff:ff:ff:ff:ff:ff",
+ },
+ }
+ return c.sendAndDecodeCommandResponse(payload)
+}
+
func (c *TestAppleMDMClient) GetBootstrapToken() ([]byte, error) {
payload := map[string]any{
"MessageType": "GetBootstrapToken",
@@ -648,7 +668,11 @@ const serialLetters = "0123456789ABCDEFGHJKMNPQRSTUVWXYZ"
// RandSerialNumber returns a fake random serial number.
func RandSerialNumber() string {
- b := make([]byte, 12)
+ return randStr(12)
+}
+
+func randStr(n int) string {
+ b := make([]byte, n)
for i := range b {
//nolint:gosec // not used for crypto, only to generate random serial for testing
b[i] = serialLetters[mrand.Intn(len(serialLetters))]
@@ -656,6 +680,11 @@ func RandSerialNumber() string {
return string(b)
}
+// RandUDID returns a fake random iOS/iPadOS 17+ UDID.
+func RandUDID() string {
+ return fmt.Sprintf("%s-%s", randStr(8), randStr(16))
+}
+
type scepClient interface {
scepserver.Service
Supports(cap string) bool
diff --git a/pkg/spec/gitops.go b/pkg/spec/gitops.go
index 0fbebe70b5..a416c33787 100644
--- a/pkg/spec/gitops.go
+++ b/pkg/spec/gitops.go
@@ -244,7 +244,8 @@ func parseSecrets(result *GitOps, multiError *multierror.Error) *multierror.Erro
return multierror.Append(multiError, errors.New("'team_settings.secrets' is required"))
}
}
- var enrollSecrets []*fleet.EnrollSecret
+ // When secrets slice is empty, all secrets are removed.
+ enrollSecrets := make([]*fleet.EnrollSecret, 0)
if rawSecrets != nil {
secrets, ok := rawSecrets.([]interface{})
if !ok {
diff --git a/render.yaml b/render.yaml
index 4916488818..709a1bdc95 100644
--- a/render.yaml
+++ b/render.yaml
@@ -7,7 +7,15 @@ services:
url: 'fleetdm/fleet:latest'
preDeployCommand: "fleet prepare --no-prompt=true db"
healthCheckPath: /healthz
+ disk:
+ name: installers
+ mountPath: /opt/fleet/installers
+ sizeGB: 10
envVars:
+ - key: FLEET_SOFTWARE_INSTALLER_STORE_DIR
+ value: '/opt/fleet/installers'
+ - key: FLEET_SERVER_PRIVATE_KEY
+ generateValue: true
- key: FLEET_MYSQL_ADDRESS
fromService:
name: fleet-mysql
diff --git a/schema/osquery_fleet_schema.json b/schema/osquery_fleet_schema.json
index 87ce3656c3..fc665ac993 100644
--- a/schema/osquery_fleet_schema.json
+++ b/schema/osquery_fleet_schema.json
@@ -9730,7 +9730,7 @@
},
{
"name": "etc_hosts",
- "description": "Line-parsed /etc/hosts.",
+ "description": "The `hosts` file comprises a local, plain-text configuration for mapping IP addresses to host names. It does not necessarily rely on an external Domain Name System (DNS) for routing. The `etc_hosts` osquery table expresses the data in the `hosts` file.",
"url": "https://fleetdm.com/tables/etc_hosts",
"platforms": [
"darwin",
@@ -9739,8 +9739,8 @@
],
"evented": false,
"cacheable": true,
- "notes": "",
- "examples": "Identify host\"name\"s pointed to IP addresses using the hosts file. This\ntechnique is often abused by malware, but can also indicate services that do\nnot have proper DNS configuration to be reached from workstations.\n\n```\nSELECT * FROM etc_hosts WHERE address!='127.0.0.1' AND address!='::1' AND address!='255.255.255.255';\n```",
+ "notes": "The `hosts` file is customized by many organizations. As part of a defense-in-depth security posture it's important to track `hosts` modifications. Endpoints with a modified `hosts` configuration connected to enterprise networks can potentially bypass network rules, proxies and firewalls or be routed to malicious sites.\n\nFile paths to `hosts`:\n- Linux: `/etc/hosts`\n- macOS: `/private/etc/hosts`\n- Windows: `C:\\Windows\\system32\\drivers\\etc`\n\n**More info**:\n- [DNS](https://en.wikipedia.org/wiki/Domain_Name_System)\n- The `/etc/hosts` [Guide For Linux](https://thelinuxcode.com/etc-hosts-file-complete-guide-for-linux/)\n- [How to edit the hosts file on Windows](https://www.howtogeek.com/784196/how-to-edit-the-hosts-file-on-windows-10-or-11)",
+ "examples": "This query detects if the macOS `/private/etc/hosts` file has been modified from its default state:\n\n```\nSELECT * FROM etc_hosts WHERE address != '127.0.0.1' AND address != '::1' AND address != '255.255.255.255';\n```",
"columns": [
{
"name": "address",
@@ -11047,6 +11047,51 @@
"url": "https://fleetdm.com/tables/firmwarepasswd",
"fleetRepoUrl": "https://github.com/fleetdm/fleet/blob/main/schema/tables/firmwarepasswd.yml"
},
+ {
+ "name": "fleetd_logs",
+ "evented": false,
+ "platforms": [
+ "darwin",
+ "windows",
+ "linux"
+ ],
+ "description": "Returns the logs from fleetd's current session. Logs are stored in memory, so they are erased when it restarts.",
+ "examples": "```\nSELECT * FROM fleetd_logs\n```\n\nReturn only log entries with errors attached\n\n```\nSELECT * FROM fleetd_logs WHERE error != \"\"\n```",
+ "columns": [
+ {
+ "name": "time",
+ "description": "The time the event was captured, UTC.",
+ "type": "text",
+ "required": false
+ },
+ {
+ "name": "level",
+ "description": "The log-level of the event. Info, Debug, etc.",
+ "type": "text",
+ "required": false
+ },
+ {
+ "name": "error",
+ "description": "The error attached to the event",
+ "type": "text",
+ "required": false
+ },
+ {
+ "name": "message",
+ "description": "The message attached to the event",
+ "type": "text",
+ "required": false
+ },
+ {
+ "name": "payload",
+ "description": "Any extra data attached to the event, JSON",
+ "type": "text",
+ "required": false
+ }
+ ],
+ "url": "https://fleetdm.com/tables/fleetd_logs",
+ "fleetRepoUrl": "https://github.com/fleetdm/fleet/blob/main/schema/tables/fleetd_logs.yml"
+ },
{
"name": "gatekeeper",
"description": "macOS Gatekeeper Details.",
@@ -18625,15 +18670,15 @@
},
{
"name": "package_bom",
- "description": "macOS package bill of materials (BOM) file list.",
+ "description": "The \"bill of materials\" (`.bom`) file in a macOS installer package (`.pkg`) lists all files installed by the package. The `package_bom` osquery table collects the data from the `.bom` files created in `/private/var/db/receipts` by macOS when a `.pkg` file is executed.",
"url": "https://fleetdm.com/tables/package_bom",
"platforms": [
"darwin"
],
"evented": false,
"cacheable": false,
- "notes": "",
- "examples": "List the bill of materials of a package. The receipts directory contains\npackages to installed applications.\n\n```\nSELECT * FROM package_bom WHERE path='/private/var/db/receipts/com.yubico.ykman.bom';\n```",
+ "notes": "Keeping track of files installed by applications is critical for upholding software management best security practices.\n\nApple’s [installer package documentation](https://developer.apple.com/documentation/xcode/packaging-mac-software-for-distribution)",
+ "examples": "This query collects the filepath and time of installation for the libVFXCore.dylib (Dynamic Library) file installed as part of Xcode.app:\n\n```\nSELECT filepath,modified_time FROM package_bom WHERE path='/private/var/db/receipts/com.apple.pkg.Xcode.bom' AND filepath LIKE '%libVFXCore.dylib';\n```",
"columns": [
{
"name": "filepath",
@@ -18703,15 +18748,15 @@
},
{
"name": "package_install_history",
- "description": "macOS package install history.",
+ "description": "The `package_install_history` table provides a detailed log of all packages installled on macOS.",
"url": "https://fleetdm.com/tables/package_install_history",
"platforms": [
"darwin"
],
"evented": false,
"cacheable": false,
- "notes": "",
- "examples": "See a list of packages installed in the last week.\n\n```\nSELECT name, version, source, datetime(time,'unixepoch') AS install_time from package_install_history WHERE install_time |-= datetime('now','-7 days');\n```",
+ "notes": "\nMonitoring the macOS package install history is useful for:\n- Regularly checking for newly installed packages and identifying suspicious software\n- Verifying that only approved packages are installed\n- Creating a Fleet policy to receive alerts for any unauthorized or vulnerable installations\n\nApple’s [installer package documentation](https://developer.apple.com/documentation/xcode/packaging-mac-software-for-distribution)",
+ "examples": "Basic query:\n\n```\nSELECT name,package_id,version,source,datetime(time,'unixepoch') AS install_time FROM package_install_history WHERE install_time >= datetime('now','-7 days');\n```\n\nThis query fetches the following data for a macOS package:\n- Name\n- Package ID\n- Version\n- Source\n- Install time\n\nThe `WHERE` clause filters the results to show only packages installed in the past 7 days.",
"columns": [
{
"name": "package_id",
@@ -19455,15 +19500,15 @@
},
{
"name": "pipes",
- "description": "Named and Anonymous pipes.",
+ "description": "Named pipes in Windows can be used to provide communication between processes on a computer or between processes on different computers across a network. The `pipes` osquery table lists the named pipes currently running on a Windows computer.",
"url": "https://fleetdm.com/tables/pipes",
"platforms": [
"windows"
],
"evented": false,
"cacheable": false,
- "notes": "",
- "examples": "```\nselect * from pipes\n```",
+ "notes": "Running the following command at a prompt in PowerShell lists the named pipes currently open on a Windows computer:\n\n```\nget-childitem \\\\.\\pipe\\\n```\n\nLinks:\n- Microsoft documentation on [named pipes](https://learn.microsoft.com/en-us/windows/win32/ipc/named-pipes)\n- Discover files linked to processes with Windows [Process Explorer](https://learn.microsoft.com/en-us/sysinternals/downloads/process-explorer)",
+ "examples": "This query displays all attributes (columns) for the named pipe enabled by opening PowerShell:\n\n```\nSELECT * FROM pipes WHERE name LIKE '%powershell';\n```",
"columns": [
{
"name": "pid",
@@ -19511,12 +19556,11 @@
"index": false
}
],
- "osqueryRepoUrl": "https://github.com/osquery/osquery/blob/master/specs/windows/pipes.table",
- "fleetRepoUrl": "https://github.com/fleetdm/fleet/new/main/schema?filename=tables%2Fpipes.yml&value=name%3A%20pipes%0Adescription%3A%20%7C-%20%23%20(required)%20string%20-%20The%20description%20for%20this%20table.%20Note%3A%20this%20field%20supports%20Markdown%0A%09%23%20Add%20description%20here%0Aexamples%3A%20%7C-%20%23%20(optional)%20string%20-%20An%20example%20query%20for%20this%20table.%20Note%3A%20This%20field%20supports%20Markdown%0A%09%23%20Add%20examples%20here%0Anotes%3A%20%7C-%20%23%20(optional)%20string%20-%20Notes%20about%20this%20table.%20Note%3A%20This%20field%20supports%20Markdown.%0A%09%23%20Add%20notes%20here%0Acolumns%3A%20%23%20(required)%0A%09-%20name%3A%20%23%20(required)%20string%20-%20The%20name%20of%20the%20column%0A%09%20%20description%3A%20%23%20(required)%20string%20-%20The%20column's%20description.%20Note%3A%20this%20field%20supports%20Markdown%0A%09%20%20type%3A%20%23%20(required)%20string%20-%20the%20column's%20data%20type%0A%09%20%20required%3A%20%23%20(required)%20boolean%20-%20whether%20or%20not%20this%20column%20is%20required%20to%20query%20this%20table."
+ "fleetRepoUrl": "https://github.com/fleetdm/fleet/blob/main/schema/tables/pipes.yml"
},
{
"name": "platform_info",
- "description": "Information about EFI/UEFI/ROM and platform/boot.",
+ "description": "The `platform_info` osquery table collects boot platform information from a computer. The `platform_info` table works on Linux, macOS and Windows.",
"url": "https://fleetdm.com/tables/platform_info",
"platforms": [
"darwin",
@@ -19525,8 +19569,8 @@
],
"evented": false,
"cacheable": false,
- "notes": "",
- "examples": "See version information about the boot system, such as iBoot on Apple Silicon\n\n```\nSELECT version FROM platform_info;\n```",
+ "notes": "Links:\n- EFI: https://en.wikipedia.org/wiki/EFI_system_partition \n- iboot: https://en.wikipedia.org/wiki/IBoot \n- UEFI: https://en.wikipedia.org/wiki/UEFI#Classes \n- System booting: https://en.wikipedia.org/wiki/Booting ",
+ "examples": "Basic query:\n\n```\nSELECT extra,firmware_type,vendor FROM platform_info;\n```\n\nThis query results in a listing of the following attributes on a macOS host running a Windows 11 virtual machine in the Parallels.app:\n\nMac -\n- extra = \"Darwin Kernel Version 23.5.0: Wed May 1 20:14:38 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6020\"\n- firmware_type = \"iboot\"\n- vendor = \"Apple Inc.\"\n\nWindows -\n- extra = \"\"\n- firmware_type = \"uefi\"\n- vendor = \"Parallels International GmbH.\"",
"columns": [
{
"name": "vendor",
diff --git a/schema/tables/etc_hosts.yml b/schema/tables/etc_hosts.yml
index d04c712705..63adce5acb 100644
--- a/schema/tables/etc_hosts.yml
+++ b/schema/tables/etc_hosts.yml
@@ -9,11 +9,12 @@ examples: |-
notes: |-
The `hosts` file is customized by many organizations. As part of a defense-in-depth security posture it's important to track `hosts` modifications. Endpoints with a modified `hosts` configuration connected to enterprise networks can potentially bypass network rules, proxies and firewalls or be routed to malicious sites.
- File paths:
- Linux: /etc/hosts
- macOS: /private/etc/hosts
- Windows: C:\Windows\system32\drivers\etc
+ File paths to `hosts`:
+ - Linux: `/etc/hosts`
+ - macOS: `/private/etc/hosts`
+ - Windows: `C:\Windows\system32\drivers\etc`
+ **More info**:
- [DNS](https://en.wikipedia.org/wiki/Domain_Name_System)
- The `/etc/hosts` [Guide For Linux](https://thelinuxcode.com/etc-hosts-file-complete-guide-for-linux/)
- [How to edit the hosts file on Windows](https://www.howtogeek.com/784196/how-to-edit-the-hosts-file-on-windows-10-or-11)
@@ -21,3 +22,4 @@ columns:
- name: pid_with_namespace
platforms:
- linux
+
diff --git a/schema/tables/fleed_logs.yml b/schema/tables/fleed_logs.yml
new file mode 100644
index 0000000000..bed48c9b3e
--- /dev/null
+++ b/schema/tables/fleed_logs.yml
@@ -0,0 +1,39 @@
+name: fleetd_logs
+evented: false
+platforms:
+ - darwin
+ - windows
+ - linux
+description: |-
+ Returns the logs from fleetd's current session. Logs are stored in memory, so they are erased when it restarts.
+examples: |-
+ ```
+ SELECT * FROM fleetd_logs
+ ```
+
+ Return only log entries with errors attached
+
+ ```
+ SELECT * FROM fleetd_logs WHERE error != ""
+ ```
+columns:
+ - name: time
+ description: The time the event was captured, UTC.
+ type: text
+ required: false
+ - name: level
+ description: The log-level of the event. Info, Debug, etc.
+ type: text
+ required: false
+ - name: error
+ description: The error attached to the event
+ type: text
+ required: false
+ - name: message
+ description: The message attached to the event
+ type: text
+ required: false
+ - name: payload
+ description: Any extra data attached to the event, JSON
+ type: text
+ required: false
diff --git a/schema/tables/package_bom.yml b/schema/tables/package_bom.yml
index 999e583e80..0c3974c98f 100644
--- a/schema/tables/package_bom.yml
+++ b/schema/tables/package_bom.yml
@@ -1,8 +1,12 @@
name: package_bom
+description: The "bill of materials" (`.bom`) file in a macOS installer package (`.pkg`) lists all files installed by the package. The `package_bom` osquery table collects the data from the `.bom` files created in `/private/var/db/receipts` by macOS when a `.pkg` file is executed.
examples: |-
- List the bill of materials of a package. The receipts directory contains
- packages to installed applications.
+ This query collects the filepath and time of installation for the libVFXCore.dylib (Dynamic Library) file installed as part of Xcode.app:
```
- SELECT * FROM package_bom WHERE path='/private/var/db/receipts/com.yubico.ykman.bom';
+ SELECT filepath,modified_time FROM package_bom WHERE path='/private/var/db/receipts/com.apple.pkg.Xcode.bom' AND filepath LIKE '%libVFXCore.dylib';
```
+notes: |-
+ Keeping track of files installed by applications is critical for upholding software management best security practices.
+
+ Apple’s [installer package documentation](https://developer.apple.com/documentation/xcode/packaging-mac-software-for-distribution)
diff --git a/schema/tables/package_install_history.yml b/schema/tables/package_install_history.yml
index 0c34d2b1a9..7073521346 100644
--- a/schema/tables/package_install_history.yml
+++ b/schema/tables/package_install_history.yml
@@ -1,7 +1,25 @@
name: package_install_history
+description: The `package_install_history` table provides a detailed log of all packages installled on macOS.
examples: |-
- See a list of packages installed in the last week.
+ Basic query:
```
- SELECT name, version, source, datetime(time,'unixepoch') AS install_time from package_install_history WHERE install_time |-= datetime('now','-7 days');
+ SELECT name,package_id,version,source,datetime(time,'unixepoch') AS install_time FROM package_install_history WHERE install_time >= datetime('now','-7 days');
```
+
+ This query fetches the following data for a macOS package:
+ - Name
+ - Package ID
+ - Version
+ - Source
+ - Install time
+
+ The `WHERE` clause filters the results to show only packages installed in the past 7 days.
+notes: |-
+
+ Monitoring the macOS package install history is useful for:
+ - Regularly checking for newly installed packages and identifying suspicious software
+ - Verifying that only approved packages are installed
+ - Creating a Fleet policy to receive alerts for any unauthorized or vulnerable installations
+
+ Apple’s [installer package documentation](https://developer.apple.com/documentation/xcode/packaging-mac-software-for-distribution)
diff --git a/schema/tables/pipes.yml b/schema/tables/pipes.yml
new file mode 100644
index 0000000000..b6865ef92e
--- /dev/null
+++ b/schema/tables/pipes.yml
@@ -0,0 +1,20 @@
+name: pipes
+description: |- # (required) string - The description for this table. Note: this field supports Markdown
+ Named pipes in Windows can be used to provide communication between processes on a computer or between processes on different computers across a network. The `pipes` osquery table lists the named pipes currently running on a Windows computer.
+examples: |- # (optional) string - An example query for this table. Note: This field supports Markdown
+ This query displays all attributes (columns) for the named pipe enabled by opening PowerShell:
+
+ ```
+ SELECT * FROM pipes WHERE name LIKE '%powershell';
+ ```
+notes: |- # (optional) string - Notes about this table. Note: This field supports Markdown.
+ Running the following command at a prompt in PowerShell lists the named pipes currently open on a Windows computer:
+
+ ```
+ PS C:\Windows\System32> get-childitem \\.\pipe\
+ ```
+
+ Links:
+ - Microsoft documentation on [named pipes](https://learn.microsoft.com/en-us/windows/win32/ipc/named-pipes)
+ - Discover files linked to processes with Windows [Process Explorer](https://learn.microsoft.com/en-us/sysinternals/downloads/process-explorer)
+ - Windows [PowerShell](https://learn.microsoft.com/en-us/powershell/scripting/overview?view=powershell-7.4)
diff --git a/schema/tables/platform_info.yml b/schema/tables/platform_info.yml
index 6075a2344d..dad1cb11d8 100644
--- a/schema/tables/platform_info.yml
+++ b/schema/tables/platform_info.yml
@@ -1,7 +1,26 @@
name: platform_info
+description: The `platform_info` osquery table collects boot platform information from a computer. The `platform_info` table works on Linux, macOS and Windows.
examples: |-
- See version information about the boot system, such as iBoot on Apple Silicon
+ Basic query:
```
- SELECT version FROM platform_info;
+ SELECT extra,firmware_type,vendor FROM platform_info;
```
+
+ This query results in a listing of the following attributes on a macOS host running a Windows 11 virtual machine in the Parallels.app:
+
+ Mac -
+ - extra = "Darwin Kernel Version 23.5.0: Wed May 1 20:14:38 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6020"
+ - firmware_type = "iboot"
+ - vendor = "Apple Inc."
+
+ Windows -
+ - extra = ""
+ - firmware_type = "uefi"
+ - vendor = "Parallels International GmbH."
+notes: |-
+ Links:
+ - EFI: https://en.wikipedia.org/wiki/EFI_system_partition
+ - iboot: https://en.wikipedia.org/wiki/IBoot
+ - UEFI: https://en.wikipedia.org/wiki/UEFI#Classes
+ - System booting: https://en.wikipedia.org/wiki/Booting
diff --git a/schema/tables/programs.yml b/schema/tables/programs.yml
new file mode 100644
index 0000000000..a3a3e2bccd
--- /dev/null
+++ b/schema/tables/programs.yml
@@ -0,0 +1,32 @@
+name: programs
+description: |- # (required) string - The description for this table. Note: this field supports Markdown
+ The `programs` table lists applications installed via Windows Installer from a package.
+examples: |- # (optional) string - An example query for this table. Note: This field supports Markdown
+ Basic query:
+
+ ```
+ SELECT * FROM programs;
+ ```
+
+ This query determines if a specific version of Google Chrome.exe is installed:
+
+ ```
+ SELECT name,version FROM programs WHERE name='Google Chrome' AND version='125.0.6422.142';
+ ```
+notes: |- # (optional) string - Notes about this table. Note: This field supports Markdown.
+ This table includes references for applications:
+
+ - downloaded directly from websites and installed by an end user like Google Chrome or Notepad++
+ - installed via automation frameworks like winget or Chocolatey
+ - installed via command line in cmd or PowerShell
+
+ Links:
+
+ - [Windows Installer](https://learn.microsoft.com/en-us/windows/win32/msi/windows-installer-portal)
+ - [Chocolatey](https://chocolatey.org/)
+ - [winget](https://learn.microsoft.com/en-us/windows/package-manager/winget/)
+ - [winget.run](https://winget.run/)
+ - Windows [cmd](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/cmd)
+ - Windows [PowerShell](https://learn.microsoft.com/en-us/powershell/scripting/overview?view=powershell-7.4)
+ - [PowerShell primer](https://www.howtogeek.com/devops/how-to-get-started-with-learning-powershell/)
+ - [Notepad++](https://notepad-plus-plus.org/)
diff --git a/schema/tables/registry.yml b/schema/tables/registry.yml
new file mode 100644
index 0000000000..1397a5ded0
--- /dev/null
+++ b/schema/tables/registry.yml
@@ -0,0 +1,40 @@
+name: registry
+description: The Windows Registry is a database that stores Windows application data and low-level Windows settings like driver, security, service, system and user information. The `registry` osquery table expresses the data in the Windows Registry.
+examples: |- # (optional) string - An example query for this table. Note: This field supports Markdown
+ This query returns the date a Windows Host was enrolled in Fleet:
+
+ ```
+ SELECT strftime('%Y-%m-%d %H:%M:%S', mtime, 'unixepoch') AS enroll_time FROM registry WHERE path LIKE 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Enrollments\%%\DeviceEnroller';
+ ```
+
+ This query returns the state of the configurable profiles (i.e., domain, public, standard) in the Windows firewall settings (a value of 1 means the firewall is enabled for the profile):
+
+ ```
+ WITH profiles AS (
+ SELECT SPLIT(KEY, '\', 7) AS enabled,name,data,'profile' AS grpkey
+ FROM registry r
+ WHERE r.path IN (
+ '\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SharedAccess\Parameters\FirewallPolicy\DomainProfile\EnableFirewall',
+ '\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SharedAccess\Parameters\FirewallPolicy\PublicProfile\EnableFirewall',
+ '\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\SharedAccess\Parameters\FirewallPolicy\StandardProfile\EnableFirewall'
+ )
+ ),
+ firewall AS (
+ SELECT
+ MAX(CASE WHEN enabled='DomainProfile' THEN DATA END) AS domain_enabled,
+ MAX(CASE WHEN enabled='PublicProfile' THEN DATA END) AS public_enabled,
+ MAX(CASE WHEN enabled='StandardProfile' THEN DATA END) AS standard_enabled
+ FROM profiles
+ GROUP BY grpkey
+ )
+ SELECT *
+ FROM firewall;
+ ```
+notes: |- # (optional) string - Notes about this table. Note: This field supports Markdown.
+ The `registry` table is ideal for use in Fleet policies and queries because of the critical operating system and application data stored in the Windows Registry.
+
+ Links:
+
+ - [Windows Registry](https://learn.microsoft.com/en-us/windows/win32/sysinfo/registry)
+ - [Fleet Windows MDM Setup](https://fleetdm.com/guides/windows-mdm-setup)
+ - [Windows Firewall](https://learn.microsoft.com/en-us/windows/security/operating-system-security/network-security/windows-firewall/)
diff --git a/scripts/mdm/linux/linux-unlock.sh b/scripts/mdm/linux/linux-unlock.sh
index 2122fb837b..e981d4ce44 100644
--- a/scripts/mdm/linux/linux-unlock.sh
+++ b/scripts/mdm/linux/linux-unlock.sh
@@ -6,6 +6,16 @@ do
echo "$user"
if [ "$user" != "root" ]; then
echo "Unlocking password for $user"
- passwd -u $user
+ STDERR=$(passwd -u "$user" 2>&1 >/dev/null)
+ if [ $? -eq 3 ]; then
+ # possibly due to the user not having a password
+ # use this convoluted case approach to avoid bashisms (POSIX portable)
+ case "$STDERR" in
+ *"unlocking the password would result in a passwordless account"* )
+ # unlock and delete password to set it back to empty
+ passwd -ud "$user"
+ ;;
+ esac
+ fi
fi
done
diff --git a/scripts/mdm/windows/windows-remove-fleetd.ps1 b/scripts/mdm/windows/windows-remove-fleetd.ps1
index f532843c09..77a420781d 100644
--- a/scripts/mdm/windows/windows-remove-fleetd.ps1
+++ b/scripts/mdm/windows/windows-remove-fleetd.ps1
@@ -1,5 +1,5 @@
-function Test-Administrator
-{
+function Test-Administrator
+{
[OutputType([bool])]
param()
process {
@@ -58,11 +58,11 @@ function Force-Remove-Orbit {
#Remove HKLM registry entries
Get-ChildItem "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall" -Recurse -ErrorAction "SilentlyContinue" | Where-Object {($_.ValueCount -gt 0)} | ForEach-Object {
-
+
# Filter for osquery entries
- $properties = Get-ItemProperty $_.PSPath -ErrorAction "SilentlyContinue" | Where-Object {($_.DisplayName -eq "Fleet osquery")}
+ $properties = Get-ItemProperty $_.PSPath -ErrorAction "SilentlyContinue" | Where-Object {($_.DisplayName -eq "Fleet osquery")}
if ($properties) {
-
+
#Remove Registry Entries
$regKey = "HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\" + $_.PSChildName
@@ -72,12 +72,12 @@ function Force-Remove-Orbit {
}
}
}
- catch {
+ catch {
Write-Host "There was a problem running Force-Remove-Orbit"
Write-Host "$(Resolve-Error-Detailed)"
return $false
}
-
+
return $true
}
@@ -92,19 +92,41 @@ function Main {
Write-Host "About to uninstall fleetd..."
- if (Force-Remove-Orbit) {
- Write-Host "fleetd was uninstalled."
- Exit 0
+ if ($mode -eq "remove") {
+ # "remove" is received as argument to the script when called as the
+ # sub-process that will actually remove the fleet agent.
+
+ # sleep to give time to fleetd to send the script results to Fleet
+ Start-Sleep -Seconds 20
+ if (Force-Remove-Orbit) {
+ Write-Host "fleetd was uninstalled."
+ Exit 0
+ } else {
+ Write-Host "There was a problem uninstalling fleetd."
+ Exit -1
+ }
} else {
- Write-Host "There was a problem uninstalling fleetd."
- Exit -1
+ # when this script is executed from fleetd, it does not immediately
+ # remove the agent. Instead, it starts a new detached process that
+ # will do the actual removal. This is done to avoid the agent being
+ # killed by the removal process, which prevents it from sending the
+ # script execution results to Fleet, causing the script to remain
+ # "Pending" and being re-executed when/if the host reinstalls the
+ # agent.
+ #
+ # See https://github.com/fleetdm/fleet/issues/19197#issuecomment-2150020270
+ $execName = $MyInvocation.ScriptName
+ $proc = Start-Process -PassThru -FilePath "powershell" -WindowStyle Hidden -ArgumentList "-MTA", "-ExecutionPolicy", "Bypass", "-File", "$execName remove"
+ Start-Sleep -Seconds 5 # give time to process to start running
+ Write-Host "Removal process started: $($proc.Id)."
}
} catch {
- Write-Host "Errorr: Entry point"
+ Write-Host "Errorr: Entry point"
Write-Host "$(Resolve-Error-Detailed)"
Exit -1
}
}
+$mode = $args[0]
$null = Main
diff --git a/server/config/config.go b/server/config/config.go
index 5cb401752e..9e4cb6003a 100644
--- a/server/config/config.go
+++ b/server/config/config.go
@@ -301,6 +301,120 @@ type S3Config struct {
StsExternalID string `yaml:"sts_external_id"`
DisableSSL bool `yaml:"disable_ssl"`
ForceS3PathStyle bool `yaml:"force_s3_path_style"`
+
+ CarvesBucket string `yaml:"carves_bucket"`
+ CarvesPrefix string `yaml:"carves_prefix"`
+ CarvesRegion string `yaml:"carves_region"`
+ CarvesEndpointURL string `yaml:"carves_endpoint_url"`
+ CarvesAccessKeyID string `yaml:"carves_access_key_id"`
+ CarvesSecretAccessKey string `yaml:"carves_secret_access_key"`
+ CarvesStsAssumeRoleArn string `yaml:"carves_sts_assume_role_arn"`
+ CarvesStsExternalID string `yaml:"carves_sts_external_id"`
+ CarvesDisableSSL bool `yaml:"carves_disable_ssl"`
+ CarvesForceS3PathStyle bool `yaml:"carves_force_s3_path_style"`
+
+ SoftwareInstallersBucket string `yaml:"software_installers_bucket"`
+ SoftwareInstallersPrefix string `yaml:"software_installers_prefix"`
+ SoftwareInstallersRegion string `yaml:"software_installers_region"`
+ SoftwareInstallersEndpointURL string `yaml:"software_installers_endpoint_url"`
+ SoftwareInstallersAccessKeyID string `yaml:"software_installers_access_key_id"`
+ SoftwareInstallersSecretAccessKey string `yaml:"software_installers_secret_access_key"`
+ SoftwareInstallersStsAssumeRoleArn string `yaml:"software_installers_sts_assume_role_arn"`
+ SoftwareInstallersStsExternalID string `yaml:"software_installers_sts_external_id"`
+ SoftwareInstallersDisableSSL bool `yaml:"software_installers_disable_ssl"`
+ SoftwareInstallersForceS3PathStyle bool `yaml:"software_installers_force_s3_path_style"`
+}
+
+func (s S3Config) BucketsAndPrefixesMatch() bool {
+ cb := s.CarvesBucket
+ if cb == "" {
+ cb = s.Bucket
+ }
+
+ cp := s.CarvesPrefix
+ if cp == "" {
+ cp = s.Prefix
+ }
+
+ return s.SoftwareInstallersBucket == cb && s.SoftwareInstallersPrefix == cp
+}
+
+func (s S3Config) SoftwareInstallersToInternalCfg() S3ConfigInternal {
+ return S3ConfigInternal{
+ Bucket: s.SoftwareInstallersBucket,
+ Prefix: s.SoftwareInstallersPrefix,
+ Region: s.SoftwareInstallersRegion,
+ EndpointURL: s.SoftwareInstallersEndpointURL,
+ AccessKeyID: s.SoftwareInstallersAccessKeyID,
+ SecretAccessKey: s.SoftwareInstallersSecretAccessKey,
+ StsAssumeRoleArn: s.SoftwareInstallersStsAssumeRoleArn,
+ StsExternalID: s.SoftwareInstallersStsExternalID,
+ DisableSSL: s.SoftwareInstallersDisableSSL,
+ ForceS3PathStyle: s.SoftwareInstallersForceS3PathStyle,
+ }
+}
+
+// CarvesToInternalCfg creates an internal S3 config struct from the ingested S3 config. Note: we
+// fall back to the deprecated fields without the `carves_` prefix for backwards compatibility.
+func (s S3Config) CarvesToInternalCfg() S3ConfigInternal {
+ var internal S3ConfigInternal
+
+ internal.Bucket = s.CarvesBucket
+ if s.CarvesBucket == "" {
+ internal.Bucket = s.Bucket
+ }
+ internal.Prefix = s.CarvesPrefix
+ if s.CarvesPrefix == "" {
+ internal.Prefix = s.Prefix
+ }
+ internal.Region = s.CarvesRegion
+ if s.CarvesRegion == "" {
+ internal.Region = s.Region
+ }
+ internal.EndpointURL = s.CarvesEndpointURL
+ if s.CarvesEndpointURL == "" {
+ internal.EndpointURL = s.EndpointURL
+ }
+ internal.AccessKeyID = s.CarvesAccessKeyID
+ if s.CarvesAccessKeyID == "" {
+ internal.AccessKeyID = s.AccessKeyID
+ }
+ internal.SecretAccessKey = s.CarvesSecretAccessKey
+ if s.CarvesSecretAccessKey == "" {
+ internal.SecretAccessKey = s.SecretAccessKey
+ }
+ internal.StsAssumeRoleArn = s.CarvesStsAssumeRoleArn
+ if s.CarvesStsAssumeRoleArn == "" {
+ internal.StsAssumeRoleArn = s.StsAssumeRoleArn
+ }
+ internal.StsExternalID = s.CarvesStsExternalID
+ if s.CarvesStsExternalID == "" {
+ internal.StsExternalID = s.StsExternalID
+ }
+ internal.DisableSSL = s.CarvesDisableSSL
+ if s.CarvesDisableSSL == false {
+ internal.DisableSSL = s.DisableSSL
+ }
+ internal.ForceS3PathStyle = s.CarvesForceS3PathStyle
+ if s.CarvesForceS3PathStyle == false {
+ internal.ForceS3PathStyle = s.ForceS3PathStyle
+ }
+
+ return internal
+}
+
+// S3ConfigInternal is used internally
+type S3ConfigInternal struct {
+ Bucket string
+ Prefix string
+ Region string
+ EndpointURL string
+ AccessKeyID string
+ SecretAccessKey string
+ StsAssumeRoleArn string
+ StsExternalID string
+ DisableSSL bool
+ ForceS3PathStyle bool
}
// PubSubConfig defines configs the for Google PubSub logging plugin
@@ -424,6 +538,7 @@ type FleetConfig struct {
Prometheus PrometheusConfig
Packaging PackagingConfig
MDM MDMConfig
+ Calendar CalendarConfig
}
type MDMConfig struct {
@@ -498,6 +613,19 @@ type MDMConfig struct {
microsoftWSTEPKeyPEM []byte
}
+type CalendarConfig struct {
+ Periodicity time.Duration
+ // Hide alwaysReloadEvent from YAML config
+ alwaysReloadEvent bool
+}
+
+func (c *CalendarConfig) AlwaysReloadEvent() bool {
+ return c.alwaysReloadEvent
+}
+func (c *CalendarConfig) SetAlwaysReloadEvent(value bool) {
+ c.alwaysReloadEvent = value
+}
+
type x509KeyPairConfig struct {
certPath string
certBytes []byte
@@ -867,10 +995,7 @@ func (man Manager) addConfigs() {
man.addConfigString("server.private_key", "", "Used for encrypting sensitive data, such as MDM certificates.")
// Hide the sandbox flag as we don't want it to be discoverable for users for now
- sandboxFlag := man.command.PersistentFlags().Lookup(flagNameFromConfigKey("server.sandbox_enabled"))
- if sandboxFlag != nil {
- sandboxFlag.Hidden = true
- }
+ man.hideConfig("server.sandbox_enabled")
// Auth
man.addConfigInt("auth.bcrypt_cost", 12,
@@ -1022,17 +1147,57 @@ func (man Manager) addConfigs() {
man.addConfigString("lambda.audit_function", "",
"Lambda function name for audit logs")
+ // S3 for file carving: Deprecated
+ man.addConfigString("s3.bucket", "", "Deprecated: Bucket where to store file carves")
+ man.addConfigString("s3.prefix", "", "Deprecated: Prefix under which carves are stored")
+ man.addConfigString("s3.region", "", "Deprecated: AWS Region (if blank region is derived)")
+ man.addConfigString("s3.endpoint_url", "", "Deprecated: AWS Service Endpoint to use (leave blank for default service endpoints)")
+ man.addConfigString("s3.access_key_id", "", "Deprecated: Access Key ID for AWS authentication")
+ man.addConfigString("s3.secret_access_key", "", "Deprecated: Secret Access Key for AWS authentication")
+ man.addConfigString("s3.sts_assume_role_arn", "", "Deprecated: ARN of role to assume for AWS")
+ man.addConfigString("s3.sts_external_id", "", "Deprecated: Optional unique identifier that can be used by the principal assuming the role to assert its identity.")
+ man.addConfigBool("s3.disable_ssl", false, "Deprecated: Disable SSL (typically for local testing)")
+ man.addConfigBool("s3.force_s3_path_style", false, "Deprecated: Set this to true to force path-style addressing, i.e., `http://s3.amazonaws.com/BUCKET/KEY`")
+
+ // Hide deprecated S3 config options
+ for _, c := range []string{
+ "s3.bucket",
+ "s3.prefix",
+ "s3.region",
+ "s3.endpoint_url",
+ "s3.access_key_id",
+ "s3.secret_access_key",
+ "s3.sts_assume_role_arn",
+ "s3.sts_external_id",
+ "s3.disable_ssl",
+ "s3.force_s3_path_style",
+ } {
+ man.hideConfig(c)
+ }
+
// S3 for file carving
- man.addConfigString("s3.bucket", "", "Bucket where to store file carves")
- man.addConfigString("s3.prefix", "", "Prefix under which carves are stored")
- man.addConfigString("s3.region", "", "AWS Region (if blank region is derived)")
- man.addConfigString("s3.endpoint_url", "", "AWS Service Endpoint to use (leave blank for default service endpoints)")
- man.addConfigString("s3.access_key_id", "", "Access Key ID for AWS authentication")
- man.addConfigString("s3.secret_access_key", "", "Secret Access Key for AWS authentication")
- man.addConfigString("s3.sts_assume_role_arn", "", "ARN of role to assume for AWS")
- man.addConfigString("s3.sts_external_id", "", "Optional unique identifier that can be used by the principal assuming the role to assert its identity.")
- man.addConfigBool("s3.disable_ssl", false, "Disable SSL (typically for local testing)")
- man.addConfigBool("s3.force_s3_path_style", false, "Set this to true to force path-style addressing, i.e., `http://s3.amazonaws.com/BUCKET/KEY`")
+ man.addConfigString("s3.carves_bucket", "", "Bucket where to store file carves")
+ man.addConfigString("s3.carves_prefix", "", "Prefix under which carves are stored")
+ man.addConfigString("s3.carves_region", "", "AWS Region (if blank region is derived)")
+ man.addConfigString("s3.carves_endpoint_url", "", "AWS Service Endpoint to use (leave blank for default service endpoints)")
+ man.addConfigString("s3.carves_access_key_id", "", "Access Key ID for AWS authentication")
+ man.addConfigString("s3.carves_secret_access_key", "", "Secret Access Key for AWS authentication")
+ man.addConfigString("s3.carves_sts_assume_role_arn", "", "ARN of role to assume for AWS")
+ man.addConfigString("s3.carves_sts_external_id", "", "Optional unique identifier that can be used by the principal assuming the role to assert its identity.")
+ man.addConfigBool("s3.carves_disable_ssl", false, "Disable SSL (typically for local testing)")
+ man.addConfigBool("s3.carves_force_s3_path_style", false, "Set this to true to force path-style addressing, i.e., `http://s3.amazonaws.com/BUCKET/KEY`")
+
+ // S3 for software installers
+ man.addConfigString("s3.software_installers_bucket", "", "Bucket where to store uploaded software installers")
+ man.addConfigString("s3.software_installers_prefix", "", "Prefix under which software installers are stored")
+ man.addConfigString("s3.software_installers_region", "", "AWS Region (if blank region is derived)")
+ man.addConfigString("s3.software_installers_endpoint_url", "", "AWS Service Endpoint to use (leave blank for default service endpoints)")
+ man.addConfigString("s3.software_installers_access_key_id", "", "Access Key ID for AWS authentication")
+ man.addConfigString("s3.software_installers_secret_access_key", "", "Secret Access Key for AWS authentication")
+ man.addConfigString("s3.software_installers_sts_assume_role_arn", "", "ARN of role to assume for AWS")
+ man.addConfigString("s3.software_installers_sts_external_id", "", "Optional unique identifier that can be used by the principal assuming the role to assert its identity.")
+ man.addConfigBool("s3.software_installers_disable_ssl", false, "Disable SSL (typically for local testing)")
+ man.addConfigBool("s3.software_installers_force_s3_path_style", false, "Set this to true to force path-style addressing, i.e., `http://s3.amazonaws.com/BUCKET/KEY`")
// PubSub
man.addConfigString("pubsub.project", "", "Google Cloud Project to use")
@@ -1147,6 +1312,12 @@ func (man Manager) addConfigs() {
man.addConfigString("mdm.windows_wstep_identity_cert_bytes", "", "Microsoft WSTEP PEM-encoded certificate bytes")
man.addConfigString("mdm.windows_wstep_identity_key_bytes", "", "Microsoft WSTEP PEM-encoded private key bytes")
+ // Calendar integration
+ man.addConfigDuration(
+ "calendar.periodicity", 0,
+ "How much time to wait between processing calendar integration.",
+ )
+
// Hide Microsoft/Windows MDM flags as we don't want it to be discoverable for users for now
betaMDMFlags := []string{
"mdm.windows_wstep_identity_cert",
@@ -1161,6 +1332,13 @@ func (man Manager) addConfigs() {
}
}
+func (man Manager) hideConfig(name string) {
+ flag := man.command.PersistentFlags().Lookup(flagNameFromConfigKey(name))
+ if flag != nil {
+ flag.Hidden = true
+ }
+}
+
// LoadConfig will load the config variables into a fully initialized
// FleetConfig struct
func (man Manager) LoadConfig() FleetConfig {
@@ -1311,18 +1489,7 @@ func (man Manager) LoadConfig() FleetConfig {
StsAssumeRoleArn: man.getConfigString("lambda.sts_assume_role_arn"),
StsExternalID: man.getConfigString("lambda.sts_external_id"),
},
- S3: S3Config{
- Bucket: man.getConfigString("s3.bucket"),
- Prefix: man.getConfigString("s3.prefix"),
- Region: man.getConfigString("s3.region"),
- EndpointURL: man.getConfigString("s3.endpoint_url"),
- AccessKeyID: man.getConfigString("s3.access_key_id"),
- SecretAccessKey: man.getConfigString("s3.secret_access_key"),
- StsAssumeRoleArn: man.getConfigString("s3.sts_assume_role_arn"),
- StsExternalID: man.getConfigString("s3.sts_external_id"),
- DisableSSL: man.getConfigBool("s3.disable_ssl"),
- ForceS3PathStyle: man.getConfigBool("s3.force_s3_path_style"),
- },
+ S3: man.loadS3Config(),
Email: EmailConfig{
EmailBackend: man.getConfigString("email.backend"),
},
@@ -1432,6 +1599,9 @@ func (man Manager) LoadConfig() FleetConfig {
WindowsWSTEPIdentityCertBytes: man.getConfigString("mdm.windows_wstep_identity_cert_bytes"),
WindowsWSTEPIdentityKeyBytes: man.getConfigString("mdm.windows_wstep_identity_key_bytes"),
},
+ Calendar: CalendarConfig{
+ Periodicity: man.getConfigDuration("calendar.periodicity"),
+ },
}
// ensure immediately that the async config is valid for all known tasks
@@ -1442,6 +1612,43 @@ func (man Manager) LoadConfig() FleetConfig {
return cfg
}
+func (man Manager) loadS3Config() S3Config {
+ return S3Config{
+ CarvesBucket: man.getConfigString("s3.carves_bucket"),
+ CarvesPrefix: man.getConfigString("s3.carves_prefix"),
+ CarvesRegion: man.getConfigString("s3.carves_region"),
+ CarvesEndpointURL: man.getConfigString("s3.carves_endpoint_url"),
+ CarvesAccessKeyID: man.getConfigString("s3.carves_access_key_id"),
+ CarvesSecretAccessKey: man.getConfigString("s3.carves_secret_access_key"),
+ CarvesStsAssumeRoleArn: man.getConfigString("s3.carves_sts_assume_role_arn"),
+ CarvesStsExternalID: man.getConfigString("s3.carves_sts_external_id"),
+ CarvesDisableSSL: man.getConfigBool("s3.carves_disable_ssl"),
+ CarvesForceS3PathStyle: man.getConfigBool("s3.carves_force_s3_path_style"),
+
+ Bucket: man.getConfigString("s3.bucket"),
+ Prefix: man.getConfigString("s3.prefix"),
+ Region: man.getConfigString("s3.region"),
+ EndpointURL: man.getConfigString("s3.endpoint_url"),
+ AccessKeyID: man.getConfigString("s3.access_key_id"),
+ SecretAccessKey: man.getConfigString("s3.secret_access_key"),
+ StsAssumeRoleArn: man.getConfigString("s3.sts_assume_role_arn"),
+ StsExternalID: man.getConfigString("s3.sts_external_id"),
+ DisableSSL: man.getConfigBool("s3.disable_ssl"),
+ ForceS3PathStyle: man.getConfigBool("s3.force_s3_path_style"),
+
+ SoftwareInstallersBucket: man.getConfigString("s3.software_installers_bucket"),
+ SoftwareInstallersPrefix: man.getConfigString("s3.software_installers_prefix"),
+ SoftwareInstallersRegion: man.getConfigString("s3.software_installers_region"),
+ SoftwareInstallersEndpointURL: man.getConfigString("s3.software_installers_endpoint_url"),
+ SoftwareInstallersAccessKeyID: man.getConfigString("s3.software_installers_access_key_id"),
+ SoftwareInstallersSecretAccessKey: man.getConfigString("s3.software_installers_secret_access_key"),
+ SoftwareInstallersStsAssumeRoleArn: man.getConfigString("s3.software_installers_sts_assume_role_arn"),
+ SoftwareInstallersStsExternalID: man.getConfigString("s3.software_installers_sts_external_id"),
+ SoftwareInstallersDisableSSL: man.getConfigBool("s3.software_installers_disable_ssl"),
+ SoftwareInstallersForceS3PathStyle: man.getConfigBool("s3.software_installers_force_s3_path_style"),
+ }
+}
+
// IsSet determines whether a given config key has been explicitly set by any
// of the configuration sources. If false, the default value is being used.
func (man Manager) IsSet(key string) bool {
diff --git a/server/config/config_test.go b/server/config/config_test.go
index b7baa4bb93..c2df315840 100644
--- a/server/config/config_test.go
+++ b/server/config/config_test.go
@@ -60,13 +60,22 @@ func TestConfigRoundtrip(t *testing.T) {
case "AsyncHostCollectInterval", "AsyncHostCollectLockTimeout":
// supports a duration or per-task config
key_v.SetString("30s")
+ // These are deprecated field names in the S3 config. Set them to zero value, which leads to the new fields being populated instead.
+ case "Bucket", "Prefix", "Region", "EndpointURL", "AccessKeyID", "SecretAccessKey", "StsAssumeRoleArn", "StsExternalID":
+ key_v.SetString("")
default:
key_v.SetString(v.Elem().Type().Field(conf_index).Name + "_" + conf_v.Type().Field(key_index).Name)
}
case int:
key_v.SetInt(int64(conf_index*100 + key_index))
case bool:
- key_v.SetBool(true)
+ switch conf_v.Type().Field(key_index).Name {
+ // These are deprecated field names in the S3 config. Set them to zero value, which leads to the new fields being populated instead.
+ case "DisableSSL", "ForceS3PathStyle":
+ key_v.SetBool(false)
+ default:
+ key_v.SetBool(true)
+ }
case time.Duration:
d := time.Duration(conf_index*100 + key_index)
key_v.Set(reflect.ValueOf(d))
diff --git a/server/cron/calendar_cron.go b/server/cron/calendar_cron.go
index becabff31c..d4c855ebcc 100644
--- a/server/cron/calendar_cron.go
+++ b/server/cron/calendar_cron.go
@@ -11,6 +11,7 @@ import (
"time"
"github.com/fleetdm/fleet/v4/ee/server/calendar"
+ "github.com/fleetdm/fleet/v4/server/config"
"github.com/fleetdm/fleet/v4/server/fleet"
"github.com/fleetdm/fleet/v4/server/ptr"
"github.com/fleetdm/fleet/v4/server/service/schedule"
@@ -23,11 +24,16 @@ const calendarConsumers = 18
const defaultDescription = "needs to make sure your device meets the organization's requirements."
const defaultResolution = "During this maintenance window, you can expect updates to be applied automatically. Your device may be unavailable during this time."
+type calendarConfig struct {
+ config.CalendarConfig
+ fleet.GoogleCalendarIntegration
+}
+
func NewCalendarSchedule(
ctx context.Context,
instanceID string,
ds fleet.Datastore,
- interval time.Duration,
+ serverConfig config.CalendarConfig,
logger kitlog.Logger,
) (*schedule.Schedule, error) {
const (
@@ -35,7 +41,7 @@ func NewCalendarSchedule(
)
logger = kitlog.With(logger, "cron", name)
s := schedule.New(
- ctx, name, instanceID, interval, ds, ds,
+ ctx, name, instanceID, serverConfig.Periodicity, ds, ds,
schedule.WithAltLockID("calendar"),
schedule.WithLogger(logger),
schedule.WithJob(
@@ -47,7 +53,7 @@ func NewCalendarSchedule(
schedule.WithJob(
"calendar_events",
func(ctx context.Context) error {
- return cronCalendarEvents(ctx, ds, logger)
+ return cronCalendarEvents(ctx, ds, serverConfig, logger)
},
),
)
@@ -55,7 +61,7 @@ func NewCalendarSchedule(
return s, nil
}
-func cronCalendarEvents(ctx context.Context, ds fleet.Datastore, logger kitlog.Logger) error {
+func cronCalendarEvents(ctx context.Context, ds fleet.Datastore, serverConfig config.CalendarConfig, logger kitlog.Logger) error {
appConfig, err := ds.AppConfig(ctx)
if err != nil {
return fmt.Errorf("load app config: %w", err)
@@ -78,9 +84,13 @@ func cronCalendarEvents(ctx context.Context, ds fleet.Datastore, logger kitlog.L
return fmt.Errorf("list teams: %w", err)
}
+ localConfig := calendarConfig{
+ CalendarConfig: serverConfig,
+ GoogleCalendarIntegration: *googleCalendarIntegrationConfig,
+ }
for _, team := range teams {
if err := cronCalendarEventsForTeam(
- ctx, ds, googleCalendarIntegrationConfig, *team, appConfig.OrgInfo.OrgName, domain, logger,
+ ctx, ds, localConfig, *team, appConfig.OrgInfo.OrgName, domain, logger,
); err != nil {
level.Info(logger).Log("msg", "events calendar cron", "team_id", team.ID, "err", err)
}
@@ -101,7 +111,7 @@ func createUserCalendarFromConfig(ctx context.Context, config *fleet.GoogleCalen
func cronCalendarEventsForTeam(
ctx context.Context,
ds fleet.Datastore,
- calendarConfig *fleet.GoogleCalendarIntegration,
+ calendarConfig calendarConfig,
team fleet.Team,
orgName string,
domain string,
@@ -176,7 +186,7 @@ func cronCalendarEventsForTeam(
// policies on one of its hosts, and possibly create a new calendar event if they have
// another failing host on the same team.
start := time.Now()
- removeCalendarEventsFromPassingHosts(ctx, ds, calendarConfig, passingHosts, logger)
+ removeCalendarEventsFromPassingHosts(ctx, ds, &calendarConfig.GoogleCalendarIntegration, passingHosts, logger)
level.Debug(logger).Log(
"msg", "passing_hosts", "took", time.Since(start),
)
@@ -201,7 +211,7 @@ func cronCalendarEventsForTeam(
func processCalendarFailingHosts(
ctx context.Context,
ds fleet.Datastore,
- calendarConfig *fleet.GoogleCalendarIntegration,
+ calendarConfig calendarConfig,
orgName string,
hosts []fleet.HostPolicyMembershipData,
logger kitlog.Logger,
@@ -248,7 +258,7 @@ func processCalendarFailingHosts(
}
}
- userCalendar := createUserCalendarFromConfig(ctx, calendarConfig, logger)
+ userCalendar := createUserCalendarFromConfig(ctx, &calendarConfig.GoogleCalendarIntegration, logger)
if err := userCalendar.Configure(host.Email); err != nil {
level.Error(logger).Log("msg", "configure user calendar", "err", err)
continue // continue with next host
@@ -257,7 +267,7 @@ func processCalendarFailingHosts(
switch {
case err == nil && !expiredEvent:
if err := processFailingHostExistingCalendarEvent(
- ctx, ds, userCalendar, orgName, hostCalendarEvent, calendarEvent, host, &policyIDtoPolicy, logger,
+ ctx, ds, userCalendar, orgName, hostCalendarEvent, calendarEvent, host, &policyIDtoPolicy, calendarConfig, logger,
); err != nil {
level.Info(logger).Log("msg", "process failing host existing calendar event", "err", err)
continue // continue with next host
@@ -313,13 +323,14 @@ func processFailingHostExistingCalendarEvent(
calendarEvent *fleet.CalendarEvent,
host fleet.HostPolicyMembershipData,
policyIDtoPolicy *sync.Map,
+ calendarConfig calendarConfig,
logger kitlog.Logger,
) error {
updatedEvent := calendarEvent
updated := false
now := time.Now()
- if shouldReloadCalendarEvent(now, calendarEvent, hostCalendarEvent) {
+ if calendarConfig.AlwaysReloadEvent() || shouldReloadCalendarEvent(now, calendarEvent, hostCalendarEvent) {
var err error
updatedEvent, _, err = calendar.GetAndUpdateEvent(
calendarEvent, func(conflict bool) string {
diff --git a/server/cron/calendar_cron_test.go b/server/cron/calendar_cron_test.go
index 94c2576cd1..7a210ed884 100644
--- a/server/cron/calendar_cron_test.go
+++ b/server/cron/calendar_cron_test.go
@@ -4,6 +4,7 @@ import (
"context"
"encoding/json"
"fmt"
+ "github.com/fleetdm/fleet/v4/server/config"
"github.com/fleetdm/fleet/v4/server/ptr"
"github.com/stretchr/testify/assert"
"os"
@@ -20,6 +21,8 @@ import (
"github.com/stretchr/testify/require"
)
+var defaultCalendarConfig = config.CalendarConfig{Periodicity: 5 * time.Minute}
+
func TestGetPreferredCalendarEventDate(t *testing.T) {
t.Parallel()
date := func(year int, month time.Month, day int) time.Time {
@@ -195,7 +198,7 @@ func TestEventForDifferentHost(t *testing.T) {
return hcEvent, calEvent, nil
}
- err := cronCalendarEvents(ctx, ds, logger)
+ err := cronCalendarEvents(ctx, ds, defaultCalendarConfig, logger)
require.NoError(t, err)
}
@@ -365,7 +368,7 @@ func TestCalendarEventsMultipleHosts(t *testing.T) {
return nil, nil
}
- err := cronCalendarEvents(ctx, ds, logger)
+ err := cronCalendarEvents(ctx, ds, defaultCalendarConfig, logger)
require.NoError(t, err)
eventsMu.Lock()
@@ -650,7 +653,7 @@ func TestCalendarEvents1KHosts(t *testing.T) {
return nil, nil
}
- err := cronCalendarEvents(ctx, ds, logger)
+ err := cronCalendarEvents(ctx, ds, defaultCalendarConfig, logger)
require.NoError(t, err)
createdCalendarEvents := calendar.ListGoogleMockEvents()
@@ -687,7 +690,7 @@ func TestCalendarEvents1KHosts(t *testing.T) {
return nil
}
- err = cronCalendarEvents(ctx, ds, logger)
+ err = cronCalendarEvents(ctx, ds, defaultCalendarConfig, logger)
require.NoError(t, err)
createdCalendarEvents = calendar.ListGoogleMockEvents()
@@ -925,7 +928,7 @@ func TestEventDescription(t *testing.T) {
return nil, nil
}
- err := cronCalendarEvents(ctx, ds, logger)
+ err := cronCalendarEvents(ctx, ds, defaultCalendarConfig, logger)
require.NoError(t, err)
numberOfEvents := 7
diff --git a/server/datastore/mysql/activities.go b/server/datastore/mysql/activities.go
index 90764b32a6..f7267c36ba 100644
--- a/server/datastore/mysql/activities.go
+++ b/server/datastore/mysql/activities.go
@@ -231,14 +231,31 @@ func (ds *Datastore) MarkActivitiesAsStreamed(ctx context.Context, activityIDs [
// software to install, etc.) and provides a unified view of those upcoming
// tasks.
func (ds *Datastore) ListHostUpcomingActivities(ctx context.Context, hostID uint, opt fleet.ListOptions) ([]*fleet.Activity, *fleet.PaginationMetadata, error) {
+ // NOTE: Be sure to update both the count (here) and list statements (below)
+ // if the query condition is modified.
countStmts := []string{
- `SELECT COUNT(*) c FROM host_script_results WHERE host_id = :host_id AND exit_code IS NULL`,
- `SELECT COUNT(*) c FROM host_software_installs WHERE host_id = :host_id AND pre_install_query_output IS NULL AND install_script_exit_code IS NULL`,
+ `SELECT
+ COUNT(*) c
+ FROM host_script_results
+ WHERE host_id = :host_id AND
+ exit_code IS NULL AND
+ (sync_request = 0 OR created_at >= DATE_SUB(NOW(), INTERVAL :max_wait_time SECOND))`,
+ `SELECT
+ COUNT(*) c
+ FROM host_software_installs
+ WHERE host_id = :host_id AND
+ pre_install_query_output IS NULL AND
+ install_script_exit_code IS NULL`,
}
var count uint
countStmt := `SELECT SUM(c) FROM ( ` + strings.Join(countStmts, " UNION ALL ") + ` ) AS counts`
- countStmt, args, err := sqlx.Named(countStmt, map[string]any{"host_id": hostID})
+
+ seconds := int(scripts.MaxServerWaitTime.Seconds())
+ countStmt, args, err := sqlx.Named(countStmt, map[string]any{
+ "host_id": hostID,
+ "max_wait_time": seconds,
+ })
if err != nil {
return nil, nil, ctxerr.Wrap(ctx, err, "build count query from named args")
}
@@ -249,7 +266,8 @@ func (ds *Datastore) ListHostUpcomingActivities(ctx context.Context, hostID uint
return []*fleet.Activity{}, &fleet.PaginationMetadata{}, nil
}
- // NOTE: Be sure to update both the count and list statements if the list query is modified
+ // NOTE: Be sure to update both the count (above) and list statements (below)
+ // if the query condition is modified.
listStmts := []string{
// list pending scripts
`SELECT
@@ -318,7 +336,6 @@ func (ds *Datastore) ListHostUpcomingActivities(ctx context.Context, hostID uint
`, softwareInstallerHostStatusNamedQuery("hsi", "")),
}
- seconds := int(scripts.MaxServerWaitTime.Seconds())
listStmt := `
SELECT
uuid,
diff --git a/server/datastore/mysql/activities_test.go b/server/datastore/mysql/activities_test.go
index aad9419485..e91a825227 100644
--- a/server/datastore/mysql/activities_test.go
+++ b/server/datastore/mysql/activities_test.go
@@ -10,6 +10,7 @@ import (
"testing"
"time"
+ "github.com/fleetdm/fleet/v4/pkg/scripts"
"github.com/fleetdm/fleet/v4/server/contexts/viewer"
"github.com/fleetdm/fleet/v4/server/fleet"
"github.com/fleetdm/fleet/v4/server/ptr"
@@ -414,8 +415,17 @@ func testListHostUpcomingActivities(t *testing.T, ds *Datastore) {
sw2Meta, err := ds.GetSoftwareInstallerMetadataByID(ctx, sw2)
require.NoError(t, err)
+ // create a sync script request for h1 that has been pending for > MaxWaitTime, will not show up
+ hsr, err := ds.NewHostScriptExecutionRequest(ctx, &fleet.HostScriptRequestPayload{HostID: h1.ID, ScriptContents: "sync", UserID: &u.ID, SyncRequest: true})
+ require.NoError(t, err)
+ hSyncExpired := hsr.ExecutionID
+ ExecAdhocSQL(t, ds, func(q sqlx.ExtContext) error {
+ _, err := q.ExecContext(ctx, "UPDATE host_script_results SET created_at = ? WHERE execution_id = ?", time.Now().Add(-(scripts.MaxServerWaitTime + time.Minute)), hSyncExpired)
+ return err
+ })
+
// create some script requests for h1
- hsr, err := ds.NewHostScriptExecutionRequest(ctx, &fleet.HostScriptRequestPayload{HostID: h1.ID, ScriptID: &scr1.ID, ScriptContents: scr1.ScriptContents, UserID: &u.ID})
+ hsr, err = ds.NewHostScriptExecutionRequest(ctx, &fleet.HostScriptRequestPayload{HostID: h1.ID, ScriptID: &scr1.ID, ScriptContents: scr1.ScriptContents, UserID: &u.ID})
require.NoError(t, err)
h1A := hsr.ExecutionID
hsr, err = ds.NewHostScriptExecutionRequest(ctx, &fleet.HostScriptRequestPayload{HostID: h1.ID, ScriptID: &scr2.ID, ScriptContents: scr2.ScriptContents, UserID: &u.ID})
diff --git a/server/datastore/mysql/apple_mdm.go b/server/datastore/mysql/apple_mdm.go
index 17f643bd2a..4140977088 100644
--- a/server/datastore/mysql/apple_mdm.go
+++ b/server/datastore/mysql/apple_mdm.go
@@ -734,7 +734,7 @@ func ingestMDMAppleDeviceFromCheckinDB(
// MDM is necessarily enabled if this gets called, always pass true for that
// parameter.
- matchID, _, err := matchHostDuringEnrollment(ctx, tx, mdmEnroll, true, "", mdmHost.UUID, mdmHost.HardwareSerial)
+ enrolledHostInfo, err := matchHostDuringEnrollment(ctx, tx, mdmEnroll, true, "", mdmHost.UUID, mdmHost.HardwareSerial)
switch {
case errors.Is(err, sql.ErrNoRows):
return insertMDMAppleHostDB(ctx, tx, mdmHost, logger, appCfg)
@@ -743,7 +743,7 @@ func ingestMDMAppleDeviceFromCheckinDB(
return ctxerr.Wrap(ctx, err, "get mdm apple host by serial number or udid")
default:
- return updateMDMAppleHostDB(ctx, tx, matchID, mdmHost, appCfg)
+ return updateMDMAppleHostDB(ctx, tx, enrolledHostInfo.ID, mdmHost, appCfg)
}
}
diff --git a/server/datastore/mysql/hosts.go b/server/datastore/mysql/hosts.go
index 190d65ba0c..9c3ae36c3b 100644
--- a/server/datastore/mysql/hosts.go
+++ b/server/datastore/mysql/hosts.go
@@ -500,7 +500,6 @@ var hostRefs = []string{
"host_updates",
"host_disk_encryption_keys",
"host_software_installed_paths",
- "host_script_results",
"query_results",
"host_activities",
"host_mdm_actions",
@@ -526,6 +525,13 @@ var additionalHostRefsByUUID = map[string]string{
"host_mdm_apple_declarations": "host_uuid",
}
+// additionalHostRefsSoftDelete are tables that reference a host but for which
+// the rows are not deleted when the host is deleted, only a soft delete is
+// performed by setting a timestamp column to the current time.
+var additionalHostRefsSoftDelete = map[string]string{
+ "host_script_results": "host_deleted_at",
+}
+
func (ds *Datastore) DeleteHost(ctx context.Context, hid uint) error {
delHostRef := func(tx sqlx.ExtContext, table string) error {
_, err := tx.ExecContext(ctx, fmt.Sprintf(`DELETE FROM %s WHERE host_id=?`, table), hid)
@@ -568,6 +574,13 @@ func (ds *Datastore) DeleteHost(ctx context.Context, hid uint) error {
}
}
+ // perform the soft-deletion of host-referencing tables
+ for table, col := range additionalHostRefsSoftDelete {
+ if _, err := tx.ExecContext(ctx, fmt.Sprintf("UPDATE `%s` SET `%s` = NOW() WHERE host_id=?", table, col), hid); err != nil {
+ return ctxerr.Wrapf(ctx, err, "soft-deleting %s for host id %d", table, hid)
+ }
+ }
+
return nil
})
}
@@ -1697,6 +1710,22 @@ const (
mdmEnroll
)
+// enrolledHostInfo contains information of an enrolled host to
+// be used when enrolling orbit/osquery, MDM or just re-enrolling hosts
+// (e.g. when a osquery.db is deleted from a host).
+//
+// NOTE: orbit and osquery running as part of fleetd on a device are identified
+// with the same entry in the hosts table.
+type enrolledHostInfo struct {
+ // ID is the identifier of the host.
+ ID uint
+ // LastEnrolledAt is the time the host last enrolled to Fleet.
+ LastEnrolledAt time.Time
+ // NodeKeySet indicates whether `node_key` is set (NOT NULL) for a osquery host
+ // or if `orbit_node_key` is set (NOT NULL) for a orbit host.
+ NodeKeySet bool
+}
+
// Attempts to find the matching host ID by osqueryID, host UUID or serial
// number. Any of those fields can be left empty if not available, and it will
// use the best match in this order:
@@ -1711,10 +1740,17 @@ const (
// able to match by serial in this scenario, since this is the only information
// we get when enrolling hosts via Apple DEP) AND if the matched host is on the
// macOS platform (darwin).
-func matchHostDuringEnrollment(ctx context.Context, q sqlx.QueryerContext, enrollType enroll, isMDMEnabled bool, osqueryID, uuid, serial string) (uint, time.Time, error) {
+func matchHostDuringEnrollment(
+ ctx context.Context,
+ q sqlx.QueryerContext,
+ enrollType enroll,
+ isMDMEnabled bool,
+ osqueryID, uuid, serial string,
+) (*enrolledHostInfo, error) {
type hostMatch struct {
ID uint
LastEnrolledAt time.Time `db:"last_enrolled_at"`
+ NodeKeySet bool `db:"node_key_set"`
Priority int
}
@@ -1724,8 +1760,14 @@ func matchHostDuringEnrollment(ctx context.Context, q sqlx.QueryerContext, enrol
rows []hostMatch
)
+ // For enrollType == mdmEnroll, nodeKeyColumn doesn't matter.
+ nodeKeyColumn := "node_key"
+ if enrollType == orbitEnroll {
+ nodeKeyColumn = "orbit_node_key"
+ }
+
if osqueryID != "" || uuid != "" {
- _, _ = query.WriteString(`(SELECT id, last_enrolled_at, 1 priority FROM hosts WHERE osquery_host_id = ?)`)
+ _, _ = query.WriteString(fmt.Sprintf(`(SELECT id, last_enrolled_at, %s IS NOT NULL AS node_key_set, 1 priority FROM hosts WHERE osquery_host_id = ?)`, nodeKeyColumn))
osqueryHostID := osqueryID
if osqueryID == "" {
// special-case, if there's no osquery identifier, use the uuid
@@ -1741,21 +1783,25 @@ func matchHostDuringEnrollment(ctx context.Context, q sqlx.QueryerContext, enrol
if query.Len() > 0 {
_, _ = query.WriteString(" UNION ")
}
- _, _ = query.WriteString(`(SELECT id, last_enrolled_at, 2 priority FROM hosts WHERE hardware_serial = ? AND (platform = 'darwin' OR platform = 'ios' OR platform = 'ipados') ORDER BY id LIMIT 1)`)
+ _, _ = query.WriteString(fmt.Sprintf(`(SELECT id, last_enrolled_at, %s IS NOT NULL AS node_key_set, 2 priority FROM hosts WHERE hardware_serial = ? AND (platform = 'darwin' OR platform = 'ios' OR platform = 'ipados') ORDER BY id LIMIT 1)`, nodeKeyColumn))
args = append(args, serial)
}
if err := sqlx.SelectContext(ctx, q, &rows, query.String(), args...); err != nil {
- return 0, time.Time{}, ctxerr.Wrap(ctx, err, "match host during enrollment")
+ return nil, ctxerr.Wrap(ctx, err, "match host during enrollment")
}
if len(rows) == 0 {
- return 0, time.Time{}, sql.ErrNoRows
+ return nil, sql.ErrNoRows
}
sort.Slice(rows, func(i, j int) bool {
l, r := rows[i], rows[j]
return l.Priority < r.Priority
})
- return rows[0].ID, rows[0].LastEnrolledAt, nil
+ return &enrolledHostInfo{
+ ID: rows[0].ID,
+ LastEnrolledAt: rows[0].LastEnrolledAt,
+ NodeKeySet: rows[0].NodeKeySet,
+ }, nil
}
func (ds *Datastore) EnrollOrbit(ctx context.Context, isMDMEnabled bool, hostInfo fleet.OrbitHostInfo, orbitNodeKey string, teamID *uint) (*fleet.Host, error) {
@@ -1769,7 +1815,7 @@ func (ds *Datastore) EnrollOrbit(ctx context.Context, isMDMEnabled bool, hostInf
var host fleet.Host
err := ds.withRetryTxx(ctx, func(tx sqlx.ExtContext) error {
- hostID, _, err := matchHostDuringEnrollment(ctx, tx, orbitEnroll, isMDMEnabled, hostInfo.OsqueryIdentifier, hostInfo.HardwareUUID, hostInfo.HardwareSerial)
+ enrolledHostInfo, err := matchHostDuringEnrollment(ctx, tx, orbitEnroll, isMDMEnabled, hostInfo.OsqueryIdentifier, hostInfo.HardwareUUID, hostInfo.HardwareSerial)
// If the osquery identifier that osqueryd will use was not sent by Orbit, then use the hardware UUID as identifier
// (using the hardware UUID is Orbit's default behavior).
@@ -1780,6 +1826,17 @@ func (ds *Datastore) EnrollOrbit(ctx context.Context, isMDMEnabled bool, hostInf
switch {
case err == nil:
+ if enrolledHostInfo.NodeKeySet {
+ // This means a orbit host already enrolled at this hosts entry.
+ // This can happen if two devices have duplicate hardware identifiers or
+ // if orbit's node key file was deleted from the device (e.g. uninstall+install).
+ level.Warn(ds.logger).Log(
+ "msg", "orbit host with duplicate identifier has enrolled in Fleet and will overwrite existing host data",
+ "identifier", hostInfo.HardwareUUID,
+ "host_id", enrolledHostInfo.ID,
+ )
+ }
+
sqlUpdate := `
UPDATE
hosts
@@ -1788,6 +1845,7 @@ func (ds *Datastore) EnrollOrbit(ctx context.Context, isMDMEnabled bool, hostInf
uuid = COALESCE(NULLIF(uuid, ''), ?),
osquery_host_id = COALESCE(NULLIF(osquery_host_id, ''), ?),
hardware_serial = COALESCE(NULLIF(hardware_serial, ''), ?),
+ last_enrolled_at = NOW(),
team_id = ?
WHERE id = ?`
_, err := tx.ExecContext(ctx, sqlUpdate,
@@ -1796,15 +1854,15 @@ func (ds *Datastore) EnrollOrbit(ctx context.Context, isMDMEnabled bool, hostInf
osqueryIdentifier,
hostInfo.HardwareSerial,
teamID,
- hostID,
+ enrolledHostInfo.ID,
)
if err != nil {
return ctxerr.Wrap(ctx, err, "orbit enroll error updating host details")
}
- host.ID = hostID
+ host.ID = enrolledHostInfo.ID
// clear any host_mdm_actions following re-enrollment here
- if _, err := tx.ExecContext(ctx, `DELETE FROM host_mdm_actions WHERE host_id = ?`, hostID); err != nil {
+ if _, err := tx.ExecContext(ctx, `DELETE FROM host_mdm_actions WHERE host_id = ?`, enrolledHostInfo.ID); err != nil {
return ctxerr.Wrap(ctx, err, "orbit enroll error clearing host_mdm_actions")
}
@@ -1871,7 +1929,7 @@ func (ds *Datastore) EnrollOrbit(ctx context.Context, isMDMEnabled bool, hostInf
return &host, nil
}
-// EnrollHost enrolls a host
+// EnrollHost enrolls the osquery agent to Fleet.
func (ds *Datastore) EnrollHost(ctx context.Context, isMDMEnabled bool, osqueryHostID, hardwareUUID, hardwareSerial, nodeKey string, teamID *uint, cooldown time.Duration) (*fleet.Host, error) {
if osqueryHostID == "" {
return nil, ctxerr.New(ctx, "missing osquery host identifier")
@@ -1881,11 +1939,11 @@ func (ds *Datastore) EnrollHost(ctx context.Context, isMDMEnabled bool, osqueryH
err := ds.withRetryTxx(ctx, func(tx sqlx.ExtContext) error {
zeroTime := time.Unix(0, 0).Add(24 * time.Hour)
- matchedID, lastEnrolledAt, err := matchHostDuringEnrollment(ctx, tx, osqueryEnroll, isMDMEnabled, osqueryHostID, hardwareUUID, hardwareSerial)
+ var hostID uint
+ enrolledHostInfo, err := matchHostDuringEnrollment(ctx, tx, osqueryEnroll, isMDMEnabled, osqueryHostID, hardwareUUID, hardwareSerial)
switch {
case err != nil && !errors.Is(err, sql.ErrNoRows):
return ctxerr.Wrap(ctx, err, "check existing")
-
case errors.Is(err, sql.ErrNoRows):
// Create new host record. We always create newly enrolled hosts with refetch_requested = true
// so that the frontend automatically starts background checks to update the page whenever
@@ -1908,30 +1966,42 @@ func (ds *Datastore) EnrollHost(ctx context.Context, isMDMEnabled bool, osqueryH
level.Info(ds.logger).Log("hostIDError", err.Error())
return ctxerr.Wrap(ctx, err, "insert host")
}
- hostID, _ := result.LastInsertId()
+ lastInsertID, _ := result.LastInsertId()
const sqlHostDisplayName = `
INSERT INTO host_display_names (host_id, display_name) VALUES (?, '')
`
- _, err = tx.ExecContext(ctx, sqlHostDisplayName, hostID)
+ _, err = tx.ExecContext(ctx, sqlHostDisplayName, lastInsertID)
if err != nil {
return ctxerr.Wrap(ctx, err, "insert host_display_names")
}
- matchedID = uint(hostID)
-
+ hostID = uint(lastInsertID)
default:
+ hostID = enrolledHostInfo.ID
+
// Prevent hosts from enrolling too often with the same identifier.
// Prior to adding this we saw many hosts (probably VMs) with the
// same identifier competing for enrollment and causing perf issues.
- if cooldown > 0 && time.Since(lastEnrolledAt) < cooldown {
+ if cooldown > 0 && time.Since(enrolledHostInfo.LastEnrolledAt) < cooldown {
return backoff.Permanent(ctxerr.Errorf(ctx, "host identified by %s enrolling too often", osqueryHostID))
}
- if err := deleteAllPolicyMemberships(ctx, tx, []uint{matchedID}); err != nil {
+ if enrolledHostInfo.NodeKeySet {
+ // This means a osquery host already enrolled at this hosts entry.
+ // This can happen if two devices have duplicate hardware identifiers or
+ // if osquery.db was deleted from the device (e.g. uninstall+install).
+ level.Warn(ds.logger).Log(
+ "msg", "osquery host with duplicate identifier has enrolled in Fleet and will overwrite existing host data",
+ "identifier", hardwareUUID,
+ "host_id", enrolledHostInfo.ID,
+ )
+ }
+
+ if err := deleteAllPolicyMemberships(ctx, tx, []uint{enrolledHostInfo.ID}); err != nil {
return ctxerr.Wrap(ctx, err, "cleanup policy membership on re-enroll")
}
// clear any host_mdm_actions following re-enrollment here
- if _, err := tx.ExecContext(ctx, `DELETE FROM host_mdm_actions WHERE host_id = ?`, matchedID); err != nil {
+ if _, err := tx.ExecContext(ctx, `DELETE FROM host_mdm_actions WHERE host_id = ?`, enrolledHostInfo.ID); err != nil {
return ctxerr.Wrap(ctx, err, "error clearing host_mdm_actions")
}
@@ -1946,7 +2016,7 @@ func (ds *Datastore) EnrollHost(ctx context.Context, isMDMEnabled bool, osqueryH
hardware_serial = COALESCE(NULLIF(hardware_serial, ''), ?)
WHERE id = ?
`
- _, err := tx.ExecContext(ctx, sqlUpdate, nodeKey, teamID, osqueryHostID, hardwareUUID, hardwareSerial, matchedID)
+ _, err := tx.ExecContext(ctx, sqlUpdate, nodeKey, teamID, osqueryHostID, hardwareUUID, hardwareSerial, enrolledHostInfo.ID)
if err != nil {
return ctxerr.Wrap(ctx, err, "update host")
}
@@ -1955,7 +2025,7 @@ func (ds *Datastore) EnrollHost(ctx context.Context, isMDMEnabled bool, osqueryH
_, err = tx.ExecContext(ctx, `
INSERT INTO host_seen_times (host_id, seen_time) VALUES (?, ?)
ON DUPLICATE KEY UPDATE seen_time = VALUES(seen_time)`,
- matchedID, time.Now().UTC())
+ hostID, time.Now().UTC())
if err != nil {
return ctxerr.Wrap(ctx, err, "new host seen time")
}
@@ -2012,11 +2082,11 @@ func (ds *Datastore) EnrollHost(ctx context.Context, isMDMEnabled bool, osqueryH
WHERE h.id = ?
LIMIT 1
`
- err = sqlx.GetContext(ctx, tx, &host, sqlSelect, matchedID)
+ err = sqlx.GetContext(ctx, tx, &host, sqlSelect, hostID)
if err != nil {
return ctxerr.Wrap(ctx, err, "getting the host to return")
}
- _, err = tx.ExecContext(ctx, `INSERT IGNORE INTO label_membership (host_id, label_id) VALUES (?, (SELECT id FROM labels WHERE name = 'All Hosts' AND label_type = 1))`, matchedID)
+ _, err = tx.ExecContext(ctx, `INSERT IGNORE INTO label_membership (host_id, label_id) VALUES (?, (SELECT id FROM labels WHERE name = 'All Hosts' AND label_type = 1))`, hostID)
if err != nil {
return ctxerr.Wrap(ctx, err, "insert new host into all hosts label")
}
diff --git a/server/datastore/mysql/hosts_test.go b/server/datastore/mysql/hosts_test.go
index 26f8dc41c8..91928dd5f5 100644
--- a/server/datastore/mysql/hosts_test.go
+++ b/server/datastore/mysql/hosts_test.go
@@ -6647,6 +6647,12 @@ func testHostsDeleteHosts(t *testing.T, ds *Datastore) {
require.NoError(t, err, tbl)
require.True(t, ok, "table: %s", tbl)
}
+ for tbl, col := range additionalHostRefsSoftDelete {
+ var ok bool
+ err = ds.writer(context.Background()).Get(&ok, fmt.Sprintf("SELECT 1 FROM %s WHERE host_id = ? AND %s IS NULL", tbl, col), host.ID)
+ require.NoError(t, err, tbl)
+ require.True(t, ok, "table: %s", tbl)
+ }
err = ds.DeleteHosts(context.Background(), []uint{host.ID})
require.NoError(t, err)
@@ -6664,6 +6670,12 @@ func testHostsDeleteHosts(t *testing.T, ds *Datastore) {
require.True(t, err == nil || errors.Is(err, sql.ErrNoRows), "table: %s", tbl)
require.False(t, ok, "table: %s", tbl)
}
+ for tbl, col := range additionalHostRefsSoftDelete {
+ var ok bool
+ err = ds.writer(context.Background()).Get(&ok, fmt.Sprintf("SELECT 1 FROM %s WHERE host_id = ? AND %s IS NULL", tbl, col), host.ID)
+ require.True(t, err == nil || errors.Is(err, sql.ErrNoRows), "table: %s", tbl)
+ require.False(t, ok, "table: %s", tbl) // the soft-delete column is not null anymore, so no row is found
+ }
}
func testHostIDsByOSVersion(t *testing.T, ds *Datastore) {
@@ -8069,6 +8081,62 @@ func testHostsEnrollOrbit(t *testing.T, ds *Datastore) {
scenarioC(platform)
})
}
+
+ // Scenario D:
+ // - Fleet with MDM disabled.
+ // - two linux|darwin|windows hosts with the same hardware identifiers (e.g. two cloned VMs).
+ // - fleetd running with host identifier set to uuid (default).
+ // - orbit enrolls first, then osquery
+ // Expected output: The two fleetd instances should be enrolled as one host.
+ scenarioD := func(platform string) {
+ dupUUID := uuid.New().String()
+ dupHWSerial := uuid.New().String()
+
+ h1Orbit, err := ds.EnrollOrbit(ctx, false, fleet.OrbitHostInfo{
+ HardwareUUID: dupUUID,
+ HardwareSerial: dupHWSerial,
+ Platform: platform,
+ }, uuid.New().String(), nil)
+ require.NoError(t, err)
+ h1OrbitFetched, err := ds.Host(ctx, h1Orbit.ID)
+ require.NoError(t, err)
+ time.Sleep(1 * time.Second) // to test the update of last_enrolled_at
+ h1Osquery, err := ds.EnrollHost(ctx, false, dupUUID, dupUUID, dupHWSerial, uuid.New().String(), nil, 0)
+ require.NoError(t, err)
+ h1OsqueryFetched, err := ds.Host(ctx, h1Osquery.ID)
+ require.NoError(t, err)
+ require.NotEqual(t, h1OrbitFetched.LastEnrolledAt, h1OsqueryFetched.LastEnrolledAt)
+ require.Equal(t, h1Orbit.ID, h1Osquery.ID)
+ time.Sleep(1 * time.Second) // to test the update of last_enrolled_at
+ h2Orbit, err := ds.EnrollOrbit(ctx, false, fleet.OrbitHostInfo{
+ HardwareUUID: dupUUID,
+ HardwareSerial: dupHWSerial,
+ Platform: platform,
+ }, uuid.New().String(), nil)
+ require.NoError(t, err)
+ h2OrbitFetched, err := ds.Host(ctx, h2Orbit.ID)
+ require.NoError(t, err)
+ require.NotEqual(t, h1OsqueryFetched.LastEnrolledAt, h2OrbitFetched.LastEnrolledAt)
+ time.Sleep(1 * time.Second) // to test the update of last_enrolled_at
+ h2Osquery, err := ds.EnrollHost(ctx, false, dupUUID, dupUUID, dupHWSerial, uuid.New().String(), nil, 0)
+ require.NoError(t, err)
+ require.Equal(t, h2Orbit.ID, h2Osquery.ID)
+ h2OsqueryFetched, err := ds.Host(ctx, h2Osquery.ID)
+ require.NoError(t, err)
+ require.NotEqual(t, h2OrbitFetched.LastEnrolledAt, h2OsqueryFetched.LastEnrolledAt)
+
+ // the hosts compete for the host entry (all have same row id)
+ require.Equal(t, h1Orbit.ID, h2Orbit.ID)
+ require.Equal(t, h1Orbit.ID, h1Osquery.ID)
+ require.Equal(t, h2Orbit.ID, h2Osquery.ID)
+ }
+ for _, platform := range []string{"ubuntu", "windows", "darwin"} {
+ platform := platform
+ t.Run("scenarioD_"+platform, func(t *testing.T) {
+ t.Parallel()
+ scenarioD(platform)
+ })
+ }
}
func testHostsEnrollUpdatesMissingInfo(t *testing.T, ds *Datastore) {
diff --git a/server/datastore/mysql/microsoft_mdm.go b/server/datastore/mysql/microsoft_mdm.go
index 15c2b1c142..f8035b1bd2 100644
--- a/server/datastore/mysql/microsoft_mdm.go
+++ b/server/datastore/mysql/microsoft_mdm.go
@@ -416,7 +416,7 @@ func updateMDMWindowsHostProfileStatusFromResponseDB(
WHERE host_uuid = ? AND command_uuid IN (?)`
// grab command UUIDs to find matching entries using `getMatchingHostProfilesStmt`
- commandUUIDs := make([]string, len(payloads))
+ commandUUIDs := make([]string, 0, len(payloads))
// also grab the payloads keyed by the command uuid, so we can easily
// grab the corresponding `Detail` and `Status` from the matching
// command later on.
diff --git a/server/datastore/mysql/migrations/tables/20240430111727_CleanupQueryResults.go b/server/datastore/mysql/migrations/tables/20240430111727_CleanupQueryResults.go
index 063a150617..c0be097472 100644
--- a/server/datastore/mysql/migrations/tables/20240430111727_CleanupQueryResults.go
+++ b/server/datastore/mysql/migrations/tables/20240430111727_CleanupQueryResults.go
@@ -2,7 +2,10 @@ package tables
import (
"database/sql"
+ "encoding/json"
"fmt"
+
+ "github.com/pkg/errors"
)
func init() {
@@ -22,6 +25,67 @@ func Up_20240430111727(tx *sql.Tx) error {
if err != nil {
return fmt.Errorf("failed to delete query_results %w", err)
}
+
+ //
+ // The following "fix" was introduced after this migration was released in 4.50.0.
+ // We are adding it here to disable the AI features (set ai_features_disabled=true)
+ // for non-new installations that are upgrading from < 4.50.0 using the new version
+ // of this migration to be released in 4.51.X.
+ //
+ if err := fixDisableAIForNonNewInstallation(tx); err != nil {
+ return fmt.Errorf("failed to update ai_features_disabled: %w", err)
+ }
+
+ return nil
+}
+
+func fixDisableAIForNonNewInstallation(tx *sql.Tx) error {
+ var usersCount int
+ row := tx.QueryRow(`SELECT COUNT(*) FROM users;`)
+ if err := row.Scan(&usersCount); err != nil {
+ return fmt.Errorf("select count users: %w", err)
+ }
+ if usersCount == 0 {
+ return nil
+ }
+
+ //
+ // At least a "setup" user was configured,
+ // thus we assume this is not a new installation.
+ //
+
+ var raw json.RawMessage
+ row = tx.QueryRow(`SELECT json_value FROM app_config_json LIMIT 1;`)
+ if err := row.Scan(&raw); err != nil {
+ if errors.Is(err, sql.ErrNoRows) {
+ return nil
+ }
+ return fmt.Errorf("select app_config_json: %w", err)
+ }
+
+ var config map[string]interface{}
+ if err := json.Unmarshal(raw, &config); err != nil {
+ return fmt.Errorf("unmarshal appconfig: %w", err)
+ }
+
+ ss, ok := config["server_settings"]
+ if !ok {
+ return errors.New("missing server_settings")
+ }
+ serverSettings, ok := ss.(map[string]interface{})
+ if !ok {
+ return fmt.Errorf("invalid type for server_settings: %T", ss)
+ }
+ serverSettings["ai_features_disabled"] = true
+
+ b, err := json.Marshal(config)
+ if err != nil {
+ return fmt.Errorf("marshal updated appconfig: %w", err)
+ }
+ if _, err := tx.Exec(`UPDATE app_config_json SET json_value = ? WHERE id = 1;`, b); err != nil {
+ return fmt.Errorf("update app_config_json: %w", err)
+ }
+
return nil
}
diff --git a/server/datastore/mysql/migrations/tables/20240607133721_ReconcileSoftwareTitles.go b/server/datastore/mysql/migrations/tables/20240607133721_ReconcileSoftwareTitles.go
new file mode 100644
index 0000000000..fd1668b7f1
--- /dev/null
+++ b/server/datastore/mysql/migrations/tables/20240607133721_ReconcileSoftwareTitles.go
@@ -0,0 +1,53 @@
+package tables
+
+import (
+ "database/sql"
+ "fmt"
+)
+
+func init() {
+ MigrationClient.AddMigration(Up_20240607133721, Down_20240607133721)
+}
+
+func Up_20240607133721(tx *sql.Tx) error {
+
+ // For users that are not running vulnerabilities job, we need to ensure that software_titles are up-to-date
+
+ upsertTitlesStmt := `
+INSERT INTO software_titles (name, source, browser)
+SELECT DISTINCT
+ name,
+ source,
+ browser
+FROM
+ software s
+WHERE
+ NOT EXISTS (SELECT 1 FROM software_titles st WHERE (s.name, s.source, s.browser) = (st.name, st.source, st.browser))
+ON DUPLICATE KEY UPDATE software_titles.id = software_titles.id`
+
+ _, err := tx.Exec(upsertTitlesStmt)
+ if err != nil {
+ return fmt.Errorf("failed to upsert software titles: %w", err)
+ }
+
+ // update title ids for software table entries
+ updateSoftwareStmt := `
+UPDATE
+ software s,
+ software_titles st
+SET
+ s.title_id = st.id
+WHERE
+ (s.name, s.source, s.browser) = (st.name, st.source, st.browser)
+ AND (s.title_id IS NULL OR s.title_id != st.id)`
+
+ _, err = tx.Exec(updateSoftwareStmt)
+ if err != nil {
+ return fmt.Errorf("failed to update software title_id: %w", err)
+ }
+ return nil
+}
+
+func Down_20240607133721(tx *sql.Tx) error {
+ return nil
+}
diff --git a/server/datastore/mysql/migrations/tables/20240607133721_ReconcileSoftwareTitles_test.go b/server/datastore/mysql/migrations/tables/20240607133721_ReconcileSoftwareTitles_test.go
new file mode 100644
index 0000000000..802bfea836
--- /dev/null
+++ b/server/datastore/mysql/migrations/tables/20240607133721_ReconcileSoftwareTitles_test.go
@@ -0,0 +1,53 @@
+package tables
+
+import (
+ "context"
+ "github.com/fleetdm/fleet/v4/server/fleet"
+ "github.com/stretchr/testify/assert"
+ "github.com/stretchr/testify/require"
+ "testing"
+)
+
+func TestUp_20240607133721(t *testing.T) {
+ db := applyUpToPrev(t)
+
+ // Insert data into software_titles
+ title1 := execNoErrLastID(t, db, "INSERT INTO software_titles (name, source, browser) VALUES (?, ?, ?)", "sw1", "src1", "")
+
+ // Insert software
+ const insertStmt = `INSERT INTO software
+ (name, version, source, browser, checksum, title_id)
+ VALUES
+ (?, ?, ?, ?, ?, ?)`
+
+ execNoErr(t, db, insertStmt, "sw1", "1.0", "src1", "", "1", title1)
+ execNoErr(t, db, insertStmt, "sw1", "1.0.1", "src1", "", "1a", nil)
+ execNoErr(t, db, insertStmt, "sw2", "2.0", "src2", "", "2", nil)
+ execNoErr(t, db, insertStmt, "sw3", "3.0", "src3", "browser3", "3", nil)
+
+ applyNext(t, db)
+
+ var softwareTitles []fleet.SoftwareTitle
+ require.NoError(t, db.SelectContext(context.Background(), &softwareTitles, `SELECT * FROM software_titles`))
+ require.Len(t, softwareTitles, 3)
+
+ var software []fleet.Software
+ require.NoError(t, db.SelectContext(context.Background(), &software, `SELECT id, name, source, browser, title_id FROM software`))
+ require.Len(t, software, 4)
+
+ for _, sw := range software {
+ require.NotNil(t, sw.TitleID)
+ var found bool
+ for _, title := range softwareTitles {
+ if *sw.TitleID == title.ID {
+ assert.Equal(t, sw.Name, title.Name)
+ assert.Equal(t, sw.Source, title.Source)
+ assert.Equal(t, sw.Browser, title.Browser)
+ found = true
+ break
+ }
+ }
+ assert.True(t, found)
+ }
+
+}
diff --git a/server/datastore/mysql/migrations/tables/20240612150059_AlterHostScriptsAndInstallsSoftDelete.go b/server/datastore/mysql/migrations/tables/20240612150059_AlterHostScriptsAndInstallsSoftDelete.go
new file mode 100644
index 0000000000..e004fd5c12
--- /dev/null
+++ b/server/datastore/mysql/migrations/tables/20240612150059_AlterHostScriptsAndInstallsSoftDelete.go
@@ -0,0 +1,22 @@
+package tables
+
+import (
+ "database/sql"
+ "fmt"
+)
+
+func init() {
+ MigrationClient.AddMigration(Up_20240612150059, Down_20240612150059)
+}
+
+func Up_20240612150059(tx *sql.Tx) error {
+ _, err := tx.Exec(`ALTER TABLE host_script_results ADD COLUMN host_deleted_at TIMESTAMP NULL`)
+ if err != nil {
+ return fmt.Errorf("failed to add host_deleted_at timestamp to host_script_results: %w", err)
+ }
+ return nil
+}
+
+func Down_20240612150059(tx *sql.Tx) error {
+ return nil
+}
diff --git a/server/datastore/mysql/migrations/tables/20240612150059_AlterHostScriptsAndInstallsSoftDelete_test.go b/server/datastore/mysql/migrations/tables/20240612150059_AlterHostScriptsAndInstallsSoftDelete_test.go
new file mode 100644
index 0000000000..8bbb8658f4
--- /dev/null
+++ b/server/datastore/mysql/migrations/tables/20240612150059_AlterHostScriptsAndInstallsSoftDelete_test.go
@@ -0,0 +1,32 @@
+package tables
+
+import (
+ "testing"
+ "time"
+
+ "github.com/stretchr/testify/require"
+)
+
+func TestUp_20240612150059(t *testing.T) {
+ db := applyUpToPrev(t)
+
+ script1 := execNoErrLastID(t, db, "INSERT INTO script_contents(contents, md5_checksum) VALUES ('echo hello', 'a')")
+
+ host := insertHost(t, db, nil)
+
+ hostScript := execNoErrLastID(t, db, `
+INSERT INTO host_script_results (
+ host_id,
+ execution_id,
+ output,
+ script_content_id
+) VALUES (?, ?, '', ?)`, host, "f", script1)
+
+ // Apply current migration.
+ applyNext(t, db)
+
+ var hostDeletedAt *time.Time
+ err := db.Get(&hostDeletedAt, "SELECT host_deleted_at FROM host_script_results WHERE id = ?", hostScript)
+ require.NoError(t, err)
+ require.Nil(t, hostDeletedAt)
+}
diff --git a/server/datastore/mysql/schema.sql b/server/datastore/mysql/schema.sql
index 0d76506734..5abe3d4d53 100644
--- a/server/datastore/mysql/schema.sql
+++ b/server/datastore/mysql/schema.sql
@@ -442,6 +442,7 @@ CREATE TABLE `host_script_results` (
`user_id` int(10) unsigned DEFAULT NULL,
`sync_request` tinyint(1) NOT NULL DEFAULT '0',
`script_content_id` int(10) unsigned DEFAULT NULL,
+ `host_deleted_at` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_host_script_results_execution_id` (`execution_id`),
KEY `idx_host_script_results_host_exit_created` (`host_id`,`exit_code`,`created_at`),
@@ -925,9 +926,9 @@ CREATE TABLE `migration_status_tables` (
`tstamp` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `id` (`id`)
-) ENGINE=InnoDB AUTO_INCREMENT=268 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
+) ENGINE=InnoDB AUTO_INCREMENT=270 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
-INSERT INTO `migration_status_tables` VALUES (1,0,1,'2020-01-01 01:01:01'),(2,20161118193812,1,'2020-01-01 01:01:01'),(3,20161118211713,1,'2020-01-01 01:01:01'),(4,20161118212436,1,'2020-01-01 01:01:01'),(5,20161118212515,1,'2020-01-01 01:01:01'),(6,20161118212528,1,'2020-01-01 01:01:01'),(7,20161118212538,1,'2020-01-01 01:01:01'),(8,20161118212549,1,'2020-01-01 01:01:01'),(9,20161118212557,1,'2020-01-01 01:01:01'),(10,20161118212604,1,'2020-01-01 01:01:01'),(11,20161118212613,1,'2020-01-01 01:01:01'),(12,20161118212621,1,'2020-01-01 01:01:01'),(13,20161118212630,1,'2020-01-01 01:01:01'),(14,20161118212641,1,'2020-01-01 01:01:01'),(15,20161118212649,1,'2020-01-01 01:01:01'),(16,20161118212656,1,'2020-01-01 01:01:01'),(17,20161118212758,1,'2020-01-01 01:01:01'),(18,20161128234849,1,'2020-01-01 01:01:01'),(19,20161230162221,1,'2020-01-01 01:01:01'),(20,20170104113816,1,'2020-01-01 01:01:01'),(21,20170105151732,1,'2020-01-01 01:01:01'),(22,20170108191242,1,'2020-01-01 01:01:01'),(23,20170109094020,1,'2020-01-01 01:01:01'),(24,20170109130438,1,'2020-01-01 01:01:01'),(25,20170110202752,1,'2020-01-01 01:01:01'),(26,20170111133013,1,'2020-01-01 01:01:01'),(27,20170117025759,1,'2020-01-01 01:01:01'),(28,20170118191001,1,'2020-01-01 01:01:01'),(29,20170119234632,1,'2020-01-01 01:01:01'),(30,20170124230432,1,'2020-01-01 01:01:01'),(31,20170127014618,1,'2020-01-01 01:01:01'),(32,20170131232841,1,'2020-01-01 01:01:01'),(33,20170223094154,1,'2020-01-01 01:01:01'),(34,20170306075207,1,'2020-01-01 01:01:01'),(35,20170309100733,1,'2020-01-01 01:01:01'),(36,20170331111922,1,'2020-01-01 01:01:01'),(37,20170502143928,1,'2020-01-01 01:01:01'),(38,20170504130602,1,'2020-01-01 01:01:01'),(39,20170509132100,1,'2020-01-01 01:01:01'),(40,20170519105647,1,'2020-01-01 01:01:01'),(41,20170519105648,1,'2020-01-01 01:01:01'),(42,20170831234300,1,'2020-01-01 01:01:01'),(43,20170831234301,1,'2020-01-01 01:01:01'),(44,20170831234303,1,'2020-01-01 01:01:01'),(45,20171116163618,1,'2020-01-01 01:01:01'),(46,20171219164727,1,'2020-01-01 01:01:01'),(47,20180620164811,1,'2020-01-01 01:01:01'),(48,20180620175054,1,'2020-01-01 01:01:01'),(49,20180620175055,1,'2020-01-01 01:01:01'),(50,20191010101639,1,'2020-01-01 01:01:01'),(51,20191010155147,1,'2020-01-01 01:01:01'),(52,20191220130734,1,'2020-01-01 01:01:01'),(53,20200311140000,1,'2020-01-01 01:01:01'),(54,20200405120000,1,'2020-01-01 01:01:01'),(55,20200407120000,1,'2020-01-01 01:01:01'),(56,20200420120000,1,'2020-01-01 01:01:01'),(57,20200504120000,1,'2020-01-01 01:01:01'),(58,20200512120000,1,'2020-01-01 01:01:01'),(59,20200707120000,1,'2020-01-01 01:01:01'),(60,20201011162341,1,'2020-01-01 01:01:01'),(61,20201021104586,1,'2020-01-01 01:01:01'),(62,20201102112520,1,'2020-01-01 01:01:01'),(63,20201208121729,1,'2020-01-01 01:01:01'),(64,20201215091637,1,'2020-01-01 01:01:01'),(65,20210119174155,1,'2020-01-01 01:01:01'),(66,20210326182902,1,'2020-01-01 01:01:01'),(67,20210421112652,1,'2020-01-01 01:01:01'),(68,20210506095025,1,'2020-01-01 01:01:01'),(69,20210513115729,1,'2020-01-01 01:01:01'),(70,20210526113559,1,'2020-01-01 01:01:01'),(71,20210601000001,1,'2020-01-01 01:01:01'),(72,20210601000002,1,'2020-01-01 01:01:01'),(73,20210601000003,1,'2020-01-01 01:01:01'),(74,20210601000004,1,'2020-01-01 01:01:01'),(75,20210601000005,1,'2020-01-01 01:01:01'),(76,20210601000006,1,'2020-01-01 01:01:01'),(77,20210601000007,1,'2020-01-01 01:01:01'),(78,20210601000008,1,'2020-01-01 01:01:01'),(79,20210606151329,1,'2020-01-01 01:01:01'),(80,20210616163757,1,'2020-01-01 01:01:01'),(81,20210617174723,1,'2020-01-01 01:01:01'),(82,20210622160235,1,'2020-01-01 01:01:01'),(83,20210623100031,1,'2020-01-01 01:01:01'),(84,20210623133615,1,'2020-01-01 01:01:01'),(85,20210708143152,1,'2020-01-01 01:01:01'),(86,20210709124443,1,'2020-01-01 01:01:01'),(87,20210712155608,1,'2020-01-01 01:01:01'),(88,20210714102108,1,'2020-01-01 01:01:01'),(89,20210719153709,1,'2020-01-01 01:01:01'),(90,20210721171531,1,'2020-01-01 01:01:01'),(91,20210723135713,1,'2020-01-01 01:01:01'),(92,20210802135933,1,'2020-01-01 01:01:01'),(93,20210806112844,1,'2020-01-01 01:01:01'),(94,20210810095603,1,'2020-01-01 01:01:01'),(95,20210811150223,1,'2020-01-01 01:01:01'),(96,20210818151827,1,'2020-01-01 01:01:01'),(97,20210818151828,1,'2020-01-01 01:01:01'),(98,20210818182258,1,'2020-01-01 01:01:01'),(99,20210819131107,1,'2020-01-01 01:01:01'),(100,20210819143446,1,'2020-01-01 01:01:01'),(101,20210903132338,1,'2020-01-01 01:01:01'),(102,20210915144307,1,'2020-01-01 01:01:01'),(103,20210920155130,1,'2020-01-01 01:01:01'),(104,20210927143115,1,'2020-01-01 01:01:01'),(105,20210927143116,1,'2020-01-01 01:01:01'),(106,20211013133706,1,'2020-01-01 01:01:01'),(107,20211013133707,1,'2020-01-01 01:01:01'),(108,20211102135149,1,'2020-01-01 01:01:01'),(109,20211109121546,1,'2020-01-01 01:01:01'),(110,20211110163320,1,'2020-01-01 01:01:01'),(111,20211116184029,1,'2020-01-01 01:01:01'),(112,20211116184030,1,'2020-01-01 01:01:01'),(113,20211202092042,1,'2020-01-01 01:01:01'),(114,20211202181033,1,'2020-01-01 01:01:01'),(115,20211207161856,1,'2020-01-01 01:01:01'),(116,20211216131203,1,'2020-01-01 01:01:01'),(117,20211221110132,1,'2020-01-01 01:01:01'),(118,20220107155700,1,'2020-01-01 01:01:01'),(119,20220125105650,1,'2020-01-01 01:01:01'),(120,20220201084510,1,'2020-01-01 01:01:01'),(121,20220208144830,1,'2020-01-01 01:01:01'),(122,20220208144831,1,'2020-01-01 01:01:01'),(123,20220215152203,1,'2020-01-01 01:01:01'),(124,20220223113157,1,'2020-01-01 01:01:01'),(125,20220307104655,1,'2020-01-01 01:01:01'),(126,20220309133956,1,'2020-01-01 01:01:01'),(127,20220316155700,1,'2020-01-01 01:01:01'),(128,20220323152301,1,'2020-01-01 01:01:01'),(129,20220330100659,1,'2020-01-01 01:01:01'),(130,20220404091216,1,'2020-01-01 01:01:01'),(131,20220419140750,1,'2020-01-01 01:01:01'),(132,20220428140039,1,'2020-01-01 01:01:01'),(133,20220503134048,1,'2020-01-01 01:01:01'),(134,20220524102918,1,'2020-01-01 01:01:01'),(135,20220526123327,1,'2020-01-01 01:01:01'),(136,20220526123328,1,'2020-01-01 01:01:01'),(137,20220526123329,1,'2020-01-01 01:01:01'),(138,20220608113128,1,'2020-01-01 01:01:01'),(139,20220627104817,1,'2020-01-01 01:01:01'),(140,20220704101843,1,'2020-01-01 01:01:01'),(141,20220708095046,1,'2020-01-01 01:01:01'),(142,20220713091130,1,'2020-01-01 01:01:01'),(143,20220802135510,1,'2020-01-01 01:01:01'),(144,20220818101352,1,'2020-01-01 01:01:01'),(145,20220822161445,1,'2020-01-01 01:01:01'),(146,20220831100036,1,'2020-01-01 01:01:01'),(147,20220831100151,1,'2020-01-01 01:01:01'),(148,20220908181826,1,'2020-01-01 01:01:01'),(149,20220914154915,1,'2020-01-01 01:01:01'),(150,20220915165115,1,'2020-01-01 01:01:01'),(151,20220915165116,1,'2020-01-01 01:01:01'),(152,20220928100158,1,'2020-01-01 01:01:01'),(153,20221014084130,1,'2020-01-01 01:01:01'),(154,20221027085019,1,'2020-01-01 01:01:01'),(155,20221101103952,1,'2020-01-01 01:01:01'),(156,20221104144401,1,'2020-01-01 01:01:01'),(157,20221109100749,1,'2020-01-01 01:01:01'),(158,20221115104546,1,'2020-01-01 01:01:01'),(159,20221130114928,1,'2020-01-01 01:01:01'),(160,20221205112142,1,'2020-01-01 01:01:01'),(161,20221216115820,1,'2020-01-01 01:01:01'),(162,20221220195934,1,'2020-01-01 01:01:01'),(163,20221220195935,1,'2020-01-01 01:01:01'),(164,20221223174807,1,'2020-01-01 01:01:01'),(165,20221227163855,1,'2020-01-01 01:01:01'),(166,20221227163856,1,'2020-01-01 01:01:01'),(167,20230202224725,1,'2020-01-01 01:01:01'),(168,20230206163608,1,'2020-01-01 01:01:01'),(169,20230214131519,1,'2020-01-01 01:01:01'),(170,20230303135738,1,'2020-01-01 01:01:01'),(171,20230313135301,1,'2020-01-01 01:01:01'),(172,20230313141819,1,'2020-01-01 01:01:01'),(173,20230315104937,1,'2020-01-01 01:01:01'),(174,20230317173844,1,'2020-01-01 01:01:01'),(175,20230320133602,1,'2020-01-01 01:01:01'),(176,20230330100011,1,'2020-01-01 01:01:01'),(177,20230330134823,1,'2020-01-01 01:01:01'),(178,20230405232025,1,'2020-01-01 01:01:01'),(179,20230408084104,1,'2020-01-01 01:01:01'),(180,20230411102858,1,'2020-01-01 01:01:01'),(181,20230421155932,1,'2020-01-01 01:01:01'),(182,20230425082126,1,'2020-01-01 01:01:01'),(183,20230425105727,1,'2020-01-01 01:01:01'),(184,20230501154913,1,'2020-01-01 01:01:01'),(185,20230503101418,1,'2020-01-01 01:01:01'),(186,20230515144206,1,'2020-01-01 01:01:01'),(187,20230517140952,1,'2020-01-01 01:01:01'),(188,20230517152807,1,'2020-01-01 01:01:01'),(189,20230518114155,1,'2020-01-01 01:01:01'),(190,20230520153236,1,'2020-01-01 01:01:01'),(191,20230525151159,1,'2020-01-01 01:01:01'),(192,20230530122103,1,'2020-01-01 01:01:01'),(193,20230602111827,1,'2020-01-01 01:01:01'),(194,20230608103123,1,'2020-01-01 01:01:01'),(195,20230629140529,1,'2020-01-01 01:01:01'),(196,20230629140530,1,'2020-01-01 01:01:01'),(197,20230711144622,1,'2020-01-01 01:01:01'),(198,20230721135421,1,'2020-01-01 01:01:01'),(199,20230721161508,1,'2020-01-01 01:01:01'),(200,20230726115701,1,'2020-01-01 01:01:01'),(201,20230807100822,1,'2020-01-01 01:01:01'),(202,20230814150442,1,'2020-01-01 01:01:01'),(203,20230823122728,1,'2020-01-01 01:01:01'),(204,20230906152143,1,'2020-01-01 01:01:01'),(205,20230911163618,1,'2020-01-01 01:01:01'),(206,20230912101759,1,'2020-01-01 01:01:01'),(207,20230915101341,1,'2020-01-01 01:01:01'),(208,20230918132351,1,'2020-01-01 01:01:01'),(209,20231004144339,1,'2020-01-01 01:01:01'),(210,20231009094541,1,'2020-01-01 01:01:01'),(211,20231009094542,1,'2020-01-01 01:01:01'),(212,20231009094543,1,'2020-01-01 01:01:01'),(213,20231009094544,1,'2020-01-01 01:01:01'),(214,20231016091915,1,'2020-01-01 01:01:01'),(215,20231024174135,1,'2020-01-01 01:01:01'),(216,20231025120016,1,'2020-01-01 01:01:01'),(217,20231025160156,1,'2020-01-01 01:01:01'),(218,20231031165350,1,'2020-01-01 01:01:01'),(219,20231106144110,1,'2020-01-01 01:01:01'),(220,20231107130934,1,'2020-01-01 01:01:01'),(221,20231109115838,1,'2020-01-01 01:01:01'),(222,20231121054530,1,'2020-01-01 01:01:01'),(223,20231122101320,1,'2020-01-01 01:01:01'),(224,20231130132828,1,'2020-01-01 01:01:01'),(225,20231130132931,1,'2020-01-01 01:01:01'),(226,20231204155427,1,'2020-01-01 01:01:01'),(227,20231206142340,1,'2020-01-01 01:01:01'),(228,20231207102320,1,'2020-01-01 01:01:01'),(229,20231207102321,1,'2020-01-01 01:01:01'),(230,20231207133731,1,'2020-01-01 01:01:01'),(231,20231212094238,1,'2020-01-01 01:01:01'),(232,20231212095734,1,'2020-01-01 01:01:01'),(233,20231212161121,1,'2020-01-01 01:01:01'),(234,20231215122713,1,'2020-01-01 01:01:01'),(235,20231219143041,1,'2020-01-01 01:01:01'),(236,20231224070653,1,'2020-01-01 01:01:01'),(237,20240110134315,1,'2020-01-01 01:01:01'),(238,20240119091637,1,'2020-01-01 01:01:01'),(239,20240126020642,1,'2020-01-01 01:01:01'),(240,20240126020643,1,'2020-01-01 01:01:01'),(241,20240129162819,1,'2020-01-01 01:01:01'),(242,20240130115133,1,'2020-01-01 01:01:01'),(243,20240131083822,1,'2020-01-01 01:01:01'),(244,20240205095928,1,'2020-01-01 01:01:01'),(245,20240205121956,1,'2020-01-01 01:01:01'),(246,20240209110212,1,'2020-01-01 01:01:01'),(247,20240212111533,1,'2020-01-01 01:01:01'),(248,20240221112844,1,'2020-01-01 01:01:01'),(249,20240222073518,1,'2020-01-01 01:01:01'),(250,20240222135115,1,'2020-01-01 01:01:01'),(251,20240226082255,1,'2020-01-01 01:01:01'),(252,20240228082706,1,'2020-01-01 01:01:01'),(253,20240301173035,1,'2020-01-01 01:01:01'),(254,20240302111134,1,'2020-01-01 01:01:01'),(255,20240312103753,1,'2020-01-01 01:01:01'),(256,20240313143416,1,'2020-01-01 01:01:01'),(257,20240314085226,1,'2020-01-01 01:01:01'),(258,20240314151747,1,'2020-01-01 01:01:01'),(259,20240320145650,1,'2020-01-01 01:01:01'),(260,20240327115530,1,'2020-01-01 01:01:01'),(261,20240327115617,1,'2020-01-01 01:01:01'),(262,20240408085837,1,'2020-01-01 01:01:01'),(263,20240415104633,1,'2020-01-01 01:01:01'),(264,20240430111727,1,'2020-01-01 01:01:01'),(265,20240515200020,1,'2020-01-01 01:01:01'),(266,20240521143023,1,'2020-01-01 01:01:01'),(267,20240521143024,1,'2020-01-01 01:01:01');
+INSERT INTO `migration_status_tables` VALUES (1,0,1,'2020-01-01 01:01:01'),(2,20161118193812,1,'2020-01-01 01:01:01'),(3,20161118211713,1,'2020-01-01 01:01:01'),(4,20161118212436,1,'2020-01-01 01:01:01'),(5,20161118212515,1,'2020-01-01 01:01:01'),(6,20161118212528,1,'2020-01-01 01:01:01'),(7,20161118212538,1,'2020-01-01 01:01:01'),(8,20161118212549,1,'2020-01-01 01:01:01'),(9,20161118212557,1,'2020-01-01 01:01:01'),(10,20161118212604,1,'2020-01-01 01:01:01'),(11,20161118212613,1,'2020-01-01 01:01:01'),(12,20161118212621,1,'2020-01-01 01:01:01'),(13,20161118212630,1,'2020-01-01 01:01:01'),(14,20161118212641,1,'2020-01-01 01:01:01'),(15,20161118212649,1,'2020-01-01 01:01:01'),(16,20161118212656,1,'2020-01-01 01:01:01'),(17,20161118212758,1,'2020-01-01 01:01:01'),(18,20161128234849,1,'2020-01-01 01:01:01'),(19,20161230162221,1,'2020-01-01 01:01:01'),(20,20170104113816,1,'2020-01-01 01:01:01'),(21,20170105151732,1,'2020-01-01 01:01:01'),(22,20170108191242,1,'2020-01-01 01:01:01'),(23,20170109094020,1,'2020-01-01 01:01:01'),(24,20170109130438,1,'2020-01-01 01:01:01'),(25,20170110202752,1,'2020-01-01 01:01:01'),(26,20170111133013,1,'2020-01-01 01:01:01'),(27,20170117025759,1,'2020-01-01 01:01:01'),(28,20170118191001,1,'2020-01-01 01:01:01'),(29,20170119234632,1,'2020-01-01 01:01:01'),(30,20170124230432,1,'2020-01-01 01:01:01'),(31,20170127014618,1,'2020-01-01 01:01:01'),(32,20170131232841,1,'2020-01-01 01:01:01'),(33,20170223094154,1,'2020-01-01 01:01:01'),(34,20170306075207,1,'2020-01-01 01:01:01'),(35,20170309100733,1,'2020-01-01 01:01:01'),(36,20170331111922,1,'2020-01-01 01:01:01'),(37,20170502143928,1,'2020-01-01 01:01:01'),(38,20170504130602,1,'2020-01-01 01:01:01'),(39,20170509132100,1,'2020-01-01 01:01:01'),(40,20170519105647,1,'2020-01-01 01:01:01'),(41,20170519105648,1,'2020-01-01 01:01:01'),(42,20170831234300,1,'2020-01-01 01:01:01'),(43,20170831234301,1,'2020-01-01 01:01:01'),(44,20170831234303,1,'2020-01-01 01:01:01'),(45,20171116163618,1,'2020-01-01 01:01:01'),(46,20171219164727,1,'2020-01-01 01:01:01'),(47,20180620164811,1,'2020-01-01 01:01:01'),(48,20180620175054,1,'2020-01-01 01:01:01'),(49,20180620175055,1,'2020-01-01 01:01:01'),(50,20191010101639,1,'2020-01-01 01:01:01'),(51,20191010155147,1,'2020-01-01 01:01:01'),(52,20191220130734,1,'2020-01-01 01:01:01'),(53,20200311140000,1,'2020-01-01 01:01:01'),(54,20200405120000,1,'2020-01-01 01:01:01'),(55,20200407120000,1,'2020-01-01 01:01:01'),(56,20200420120000,1,'2020-01-01 01:01:01'),(57,20200504120000,1,'2020-01-01 01:01:01'),(58,20200512120000,1,'2020-01-01 01:01:01'),(59,20200707120000,1,'2020-01-01 01:01:01'),(60,20201011162341,1,'2020-01-01 01:01:01'),(61,20201021104586,1,'2020-01-01 01:01:01'),(62,20201102112520,1,'2020-01-01 01:01:01'),(63,20201208121729,1,'2020-01-01 01:01:01'),(64,20201215091637,1,'2020-01-01 01:01:01'),(65,20210119174155,1,'2020-01-01 01:01:01'),(66,20210326182902,1,'2020-01-01 01:01:01'),(67,20210421112652,1,'2020-01-01 01:01:01'),(68,20210506095025,1,'2020-01-01 01:01:01'),(69,20210513115729,1,'2020-01-01 01:01:01'),(70,20210526113559,1,'2020-01-01 01:01:01'),(71,20210601000001,1,'2020-01-01 01:01:01'),(72,20210601000002,1,'2020-01-01 01:01:01'),(73,20210601000003,1,'2020-01-01 01:01:01'),(74,20210601000004,1,'2020-01-01 01:01:01'),(75,20210601000005,1,'2020-01-01 01:01:01'),(76,20210601000006,1,'2020-01-01 01:01:01'),(77,20210601000007,1,'2020-01-01 01:01:01'),(78,20210601000008,1,'2020-01-01 01:01:01'),(79,20210606151329,1,'2020-01-01 01:01:01'),(80,20210616163757,1,'2020-01-01 01:01:01'),(81,20210617174723,1,'2020-01-01 01:01:01'),(82,20210622160235,1,'2020-01-01 01:01:01'),(83,20210623100031,1,'2020-01-01 01:01:01'),(84,20210623133615,1,'2020-01-01 01:01:01'),(85,20210708143152,1,'2020-01-01 01:01:01'),(86,20210709124443,1,'2020-01-01 01:01:01'),(87,20210712155608,1,'2020-01-01 01:01:01'),(88,20210714102108,1,'2020-01-01 01:01:01'),(89,20210719153709,1,'2020-01-01 01:01:01'),(90,20210721171531,1,'2020-01-01 01:01:01'),(91,20210723135713,1,'2020-01-01 01:01:01'),(92,20210802135933,1,'2020-01-01 01:01:01'),(93,20210806112844,1,'2020-01-01 01:01:01'),(94,20210810095603,1,'2020-01-01 01:01:01'),(95,20210811150223,1,'2020-01-01 01:01:01'),(96,20210818151827,1,'2020-01-01 01:01:01'),(97,20210818151828,1,'2020-01-01 01:01:01'),(98,20210818182258,1,'2020-01-01 01:01:01'),(99,20210819131107,1,'2020-01-01 01:01:01'),(100,20210819143446,1,'2020-01-01 01:01:01'),(101,20210903132338,1,'2020-01-01 01:01:01'),(102,20210915144307,1,'2020-01-01 01:01:01'),(103,20210920155130,1,'2020-01-01 01:01:01'),(104,20210927143115,1,'2020-01-01 01:01:01'),(105,20210927143116,1,'2020-01-01 01:01:01'),(106,20211013133706,1,'2020-01-01 01:01:01'),(107,20211013133707,1,'2020-01-01 01:01:01'),(108,20211102135149,1,'2020-01-01 01:01:01'),(109,20211109121546,1,'2020-01-01 01:01:01'),(110,20211110163320,1,'2020-01-01 01:01:01'),(111,20211116184029,1,'2020-01-01 01:01:01'),(112,20211116184030,1,'2020-01-01 01:01:01'),(113,20211202092042,1,'2020-01-01 01:01:01'),(114,20211202181033,1,'2020-01-01 01:01:01'),(115,20211207161856,1,'2020-01-01 01:01:01'),(116,20211216131203,1,'2020-01-01 01:01:01'),(117,20211221110132,1,'2020-01-01 01:01:01'),(118,20220107155700,1,'2020-01-01 01:01:01'),(119,20220125105650,1,'2020-01-01 01:01:01'),(120,20220201084510,1,'2020-01-01 01:01:01'),(121,20220208144830,1,'2020-01-01 01:01:01'),(122,20220208144831,1,'2020-01-01 01:01:01'),(123,20220215152203,1,'2020-01-01 01:01:01'),(124,20220223113157,1,'2020-01-01 01:01:01'),(125,20220307104655,1,'2020-01-01 01:01:01'),(126,20220309133956,1,'2020-01-01 01:01:01'),(127,20220316155700,1,'2020-01-01 01:01:01'),(128,20220323152301,1,'2020-01-01 01:01:01'),(129,20220330100659,1,'2020-01-01 01:01:01'),(130,20220404091216,1,'2020-01-01 01:01:01'),(131,20220419140750,1,'2020-01-01 01:01:01'),(132,20220428140039,1,'2020-01-01 01:01:01'),(133,20220503134048,1,'2020-01-01 01:01:01'),(134,20220524102918,1,'2020-01-01 01:01:01'),(135,20220526123327,1,'2020-01-01 01:01:01'),(136,20220526123328,1,'2020-01-01 01:01:01'),(137,20220526123329,1,'2020-01-01 01:01:01'),(138,20220608113128,1,'2020-01-01 01:01:01'),(139,20220627104817,1,'2020-01-01 01:01:01'),(140,20220704101843,1,'2020-01-01 01:01:01'),(141,20220708095046,1,'2020-01-01 01:01:01'),(142,20220713091130,1,'2020-01-01 01:01:01'),(143,20220802135510,1,'2020-01-01 01:01:01'),(144,20220818101352,1,'2020-01-01 01:01:01'),(145,20220822161445,1,'2020-01-01 01:01:01'),(146,20220831100036,1,'2020-01-01 01:01:01'),(147,20220831100151,1,'2020-01-01 01:01:01'),(148,20220908181826,1,'2020-01-01 01:01:01'),(149,20220914154915,1,'2020-01-01 01:01:01'),(150,20220915165115,1,'2020-01-01 01:01:01'),(151,20220915165116,1,'2020-01-01 01:01:01'),(152,20220928100158,1,'2020-01-01 01:01:01'),(153,20221014084130,1,'2020-01-01 01:01:01'),(154,20221027085019,1,'2020-01-01 01:01:01'),(155,20221101103952,1,'2020-01-01 01:01:01'),(156,20221104144401,1,'2020-01-01 01:01:01'),(157,20221109100749,1,'2020-01-01 01:01:01'),(158,20221115104546,1,'2020-01-01 01:01:01'),(159,20221130114928,1,'2020-01-01 01:01:01'),(160,20221205112142,1,'2020-01-01 01:01:01'),(161,20221216115820,1,'2020-01-01 01:01:01'),(162,20221220195934,1,'2020-01-01 01:01:01'),(163,20221220195935,1,'2020-01-01 01:01:01'),(164,20221223174807,1,'2020-01-01 01:01:01'),(165,20221227163855,1,'2020-01-01 01:01:01'),(166,20221227163856,1,'2020-01-01 01:01:01'),(167,20230202224725,1,'2020-01-01 01:01:01'),(168,20230206163608,1,'2020-01-01 01:01:01'),(169,20230214131519,1,'2020-01-01 01:01:01'),(170,20230303135738,1,'2020-01-01 01:01:01'),(171,20230313135301,1,'2020-01-01 01:01:01'),(172,20230313141819,1,'2020-01-01 01:01:01'),(173,20230315104937,1,'2020-01-01 01:01:01'),(174,20230317173844,1,'2020-01-01 01:01:01'),(175,20230320133602,1,'2020-01-01 01:01:01'),(176,20230330100011,1,'2020-01-01 01:01:01'),(177,20230330134823,1,'2020-01-01 01:01:01'),(178,20230405232025,1,'2020-01-01 01:01:01'),(179,20230408084104,1,'2020-01-01 01:01:01'),(180,20230411102858,1,'2020-01-01 01:01:01'),(181,20230421155932,1,'2020-01-01 01:01:01'),(182,20230425082126,1,'2020-01-01 01:01:01'),(183,20230425105727,1,'2020-01-01 01:01:01'),(184,20230501154913,1,'2020-01-01 01:01:01'),(185,20230503101418,1,'2020-01-01 01:01:01'),(186,20230515144206,1,'2020-01-01 01:01:01'),(187,20230517140952,1,'2020-01-01 01:01:01'),(188,20230517152807,1,'2020-01-01 01:01:01'),(189,20230518114155,1,'2020-01-01 01:01:01'),(190,20230520153236,1,'2020-01-01 01:01:01'),(191,20230525151159,1,'2020-01-01 01:01:01'),(192,20230530122103,1,'2020-01-01 01:01:01'),(193,20230602111827,1,'2020-01-01 01:01:01'),(194,20230608103123,1,'2020-01-01 01:01:01'),(195,20230629140529,1,'2020-01-01 01:01:01'),(196,20230629140530,1,'2020-01-01 01:01:01'),(197,20230711144622,1,'2020-01-01 01:01:01'),(198,20230721135421,1,'2020-01-01 01:01:01'),(199,20230721161508,1,'2020-01-01 01:01:01'),(200,20230726115701,1,'2020-01-01 01:01:01'),(201,20230807100822,1,'2020-01-01 01:01:01'),(202,20230814150442,1,'2020-01-01 01:01:01'),(203,20230823122728,1,'2020-01-01 01:01:01'),(204,20230906152143,1,'2020-01-01 01:01:01'),(205,20230911163618,1,'2020-01-01 01:01:01'),(206,20230912101759,1,'2020-01-01 01:01:01'),(207,20230915101341,1,'2020-01-01 01:01:01'),(208,20230918132351,1,'2020-01-01 01:01:01'),(209,20231004144339,1,'2020-01-01 01:01:01'),(210,20231009094541,1,'2020-01-01 01:01:01'),(211,20231009094542,1,'2020-01-01 01:01:01'),(212,20231009094543,1,'2020-01-01 01:01:01'),(213,20231009094544,1,'2020-01-01 01:01:01'),(214,20231016091915,1,'2020-01-01 01:01:01'),(215,20231024174135,1,'2020-01-01 01:01:01'),(216,20231025120016,1,'2020-01-01 01:01:01'),(217,20231025160156,1,'2020-01-01 01:01:01'),(218,20231031165350,1,'2020-01-01 01:01:01'),(219,20231106144110,1,'2020-01-01 01:01:01'),(220,20231107130934,1,'2020-01-01 01:01:01'),(221,20231109115838,1,'2020-01-01 01:01:01'),(222,20231121054530,1,'2020-01-01 01:01:01'),(223,20231122101320,1,'2020-01-01 01:01:01'),(224,20231130132828,1,'2020-01-01 01:01:01'),(225,20231130132931,1,'2020-01-01 01:01:01'),(226,20231204155427,1,'2020-01-01 01:01:01'),(227,20231206142340,1,'2020-01-01 01:01:01'),(228,20231207102320,1,'2020-01-01 01:01:01'),(229,20231207102321,1,'2020-01-01 01:01:01'),(230,20231207133731,1,'2020-01-01 01:01:01'),(231,20231212094238,1,'2020-01-01 01:01:01'),(232,20231212095734,1,'2020-01-01 01:01:01'),(233,20231212161121,1,'2020-01-01 01:01:01'),(234,20231215122713,1,'2020-01-01 01:01:01'),(235,20231219143041,1,'2020-01-01 01:01:01'),(236,20231224070653,1,'2020-01-01 01:01:01'),(237,20240110134315,1,'2020-01-01 01:01:01'),(238,20240119091637,1,'2020-01-01 01:01:01'),(239,20240126020642,1,'2020-01-01 01:01:01'),(240,20240126020643,1,'2020-01-01 01:01:01'),(241,20240129162819,1,'2020-01-01 01:01:01'),(242,20240130115133,1,'2020-01-01 01:01:01'),(243,20240131083822,1,'2020-01-01 01:01:01'),(244,20240205095928,1,'2020-01-01 01:01:01'),(245,20240205121956,1,'2020-01-01 01:01:01'),(246,20240209110212,1,'2020-01-01 01:01:01'),(247,20240212111533,1,'2020-01-01 01:01:01'),(248,20240221112844,1,'2020-01-01 01:01:01'),(249,20240222073518,1,'2020-01-01 01:01:01'),(250,20240222135115,1,'2020-01-01 01:01:01'),(251,20240226082255,1,'2020-01-01 01:01:01'),(252,20240228082706,1,'2020-01-01 01:01:01'),(253,20240301173035,1,'2020-01-01 01:01:01'),(254,20240302111134,1,'2020-01-01 01:01:01'),(255,20240312103753,1,'2020-01-01 01:01:01'),(256,20240313143416,1,'2020-01-01 01:01:01'),(257,20240314085226,1,'2020-01-01 01:01:01'),(258,20240314151747,1,'2020-01-01 01:01:01'),(259,20240320145650,1,'2020-01-01 01:01:01'),(260,20240327115530,1,'2020-01-01 01:01:01'),(261,20240327115617,1,'2020-01-01 01:01:01'),(262,20240408085837,1,'2020-01-01 01:01:01'),(263,20240415104633,1,'2020-01-01 01:01:01'),(264,20240430111727,1,'2020-01-01 01:01:01'),(265,20240515200020,1,'2020-01-01 01:01:01'),(266,20240521143023,1,'2020-01-01 01:01:01'),(267,20240521143024,1,'2020-01-01 01:01:01'),(268,20240607133721,1,'2020-01-01 01:01:01'),(269,20240612150059,1,'2020-01-01 01:01:01');
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `mobile_device_management_solutions` (
diff --git a/server/datastore/mysql/scripts.go b/server/datastore/mysql/scripts.go
index 59e5a9235a..b89155a4f8 100644
--- a/server/datastore/mysql/scripts.go
+++ b/server/datastore/mysql/scripts.go
@@ -238,7 +238,8 @@ func (ds *Datastore) getHostScriptExecutionResultDB(ctx context.Context, q sqlx.
hsr.exit_code,
hsr.created_at,
hsr.user_id,
- hsr.sync_request
+ hsr.sync_request,
+ hsr.host_deleted_at
FROM
host_script_results hsr
JOIN
@@ -1094,6 +1095,10 @@ WHERE
SELECT 1 FROM host_script_results WHERE script_content_id = script_contents.id)
AND NOT EXISTS (
SELECT 1 FROM scripts WHERE script_content_id = script_contents.id)
+ AND NOT EXISTS (
+ SELECT 1 FROM software_installers si
+ WHERE script_contents.id IN (si.install_script_content_id, si.post_install_script_content_id)
+ )
`
_, err := ds.writer(ctx).ExecContext(ctx, deleteStmt)
if err != nil {
diff --git a/server/datastore/mysql/scripts_test.go b/server/datastore/mysql/scripts_test.go
index 13c69f77ae..09736a0dd7 100644
--- a/server/datastore/mysql/scripts_test.go
+++ b/server/datastore/mysql/scripts_test.go
@@ -1,6 +1,7 @@
package mysql
import (
+ "bytes"
"context"
_ "embed"
"fmt"
@@ -1134,6 +1135,20 @@ func testCleanupUnusedScriptContents(t *testing.T, ds *Datastore) {
res, err := ds.NewHostScriptExecutionRequest(ctx, &fleet.HostScriptRequestPayload{ScriptContents: "echo something_else", SyncRequest: true})
require.NoError(t, err)
+ // create a software install that references scripts
+ swi, err := ds.MatchOrCreateSoftwareInstaller(ctx, &fleet.UploadSoftwareInstallerPayload{
+ InstallScript: "install-script",
+ PreInstallQuery: "SELECT 1",
+ PostInstallScript: "post-install-script",
+ InstallerFile: bytes.NewReader([]byte("hello")),
+ StorageID: "storage1",
+ Filename: "file1",
+ Title: "file1",
+ Version: "1.0",
+ Source: "apps",
+ })
+ require.NoError(t, err)
+
// delete our saved script without ever executing it
require.NoError(t, ds.DeleteScript(ctx, s.ID))
@@ -1142,12 +1157,65 @@ func testCleanupUnusedScriptContents(t *testing.T, ds *Datastore) {
stmt := `SELECT id, HEX(md5_checksum) as md5_checksum FROM script_contents`
err = sqlx.SelectContext(ctx, ds.reader(ctx), &sc, stmt)
require.NoError(t, err)
- require.Len(t, sc, 2)
+ require.Len(t, sc, 4)
// this should only remove the script_contents of the saved script, since the sync script is
// still "in use" by the script execution
require.NoError(t, ds.CleanupUnusedScriptContents(ctx))
+ sc = []scriptContents{}
+ err = sqlx.SelectContext(ctx, ds.reader(ctx), &sc, stmt)
+ require.NoError(t, err)
+ require.Len(t, sc, 3)
+ require.ElementsMatch(t, []string{
+ md5ChecksumScriptContent(res.ScriptContents),
+ md5ChecksumScriptContent("install-script"),
+ md5ChecksumScriptContent("post-install-script"),
+ }, []string{
+ sc[0].Checksum,
+ sc[1].Checksum,
+ sc[2].Checksum,
+ })
+
+ // remove the software installer from the DB
+ err = ds.DeleteSoftwareInstaller(ctx, swi)
+ require.NoError(t, err)
+
+ require.NoError(t, ds.CleanupUnusedScriptContents(ctx))
+
+ // validate that script contents still exist
+ sc = []scriptContents{}
+ err = sqlx.SelectContext(ctx, ds.reader(ctx), &sc, stmt)
+ require.NoError(t, err)
+ require.Len(t, sc, 1)
+ require.Equal(t, md5ChecksumScriptContent(res.ScriptContents), sc[0].Checksum)
+
+ // create a software install without a post-install script
+ swi, err = ds.MatchOrCreateSoftwareInstaller(ctx, &fleet.UploadSoftwareInstallerPayload{
+ PreInstallQuery: "SELECT 1",
+ InstallScript: "install-script",
+ InstallerFile: bytes.NewReader([]byte("hello")),
+ StorageID: "storage1",
+ Filename: "file1",
+ Title: "file1",
+ Version: "1.0",
+ Source: "apps",
+ })
+ require.NoError(t, err)
+
+ // run the cleanup function
+ require.NoError(t, ds.CleanupUnusedScriptContents(ctx))
+ sc = []scriptContents{}
+ err = sqlx.SelectContext(ctx, ds.reader(ctx), &sc, stmt)
+ require.NoError(t, err)
+ require.Len(t, sc, 2)
+
+ // remove the software installer from the DB
+ err = ds.DeleteSoftwareInstaller(ctx, swi)
+ require.NoError(t, err)
+ require.NoError(t, ds.CleanupUnusedScriptContents(ctx))
+
+ // validate that script contents still exist
sc = []scriptContents{}
err = sqlx.SelectContext(ctx, ds.reader(ctx), &sc, stmt)
require.NoError(t, err)
diff --git a/server/datastore/mysql/software.go b/server/datastore/mysql/software.go
index 4856e1b831..11983063d4 100644
--- a/server/datastore/mysql/software.go
+++ b/server/datastore/mysql/software.go
@@ -347,6 +347,11 @@ func (ds *Datastore) applyChangesForNewSoftwareDB(
return r, nil
}
+ existingSoftware, incomingByChecksum, existingTitlesForNewSoftware, err := ds.getExistingSoftware(ctx, current, incoming)
+ if err != nil {
+ return r, err
+ }
+
err = ds.withRetryTxx(
ctx, func(tx sqlx.ExtContext) error {
deleted, err := deleteUninstalledHostSoftwareDB(ctx, tx, hostID, current, incoming)
@@ -355,7 +360,9 @@ func (ds *Datastore) applyChangesForNewSoftwareDB(
}
r.Deleted = deleted
- inserted, err := ds.insertNewInstalledHostSoftwareDB(ctx, tx, hostID, current, incoming)
+ inserted, err := ds.insertNewInstalledHostSoftwareDB(
+ ctx, tx, hostID, existingSoftware, incomingByChecksum, existingTitlesForNewSoftware,
+ )
if err != nil {
return err
}
@@ -377,6 +384,91 @@ func (ds *Datastore) applyChangesForNewSoftwareDB(
return r, err
}
+func (ds *Datastore) getExistingSoftware(
+ ctx context.Context, current map[string]fleet.Software, incoming map[string]fleet.Software,
+) (
+ currentSoftware []softwareIDChecksum, incomingChecksumToSoftware map[string]fleet.Software,
+ incomingChecksumToTitle map[string]fleet.SoftwareTitle, err error,
+) {
+ // Compute checksums for all incoming software, which we will use for faster retrieval, since checksum is a unique index
+ incomingChecksumToSoftware = make(map[string]fleet.Software, len(current))
+ newSoftware := make(map[string]struct{})
+ for uniqueName, s := range incoming {
+ if _, ok := current[uniqueName]; !ok {
+ checksum, err := computeRawChecksum(s)
+ if err != nil {
+ return nil, nil, nil, err
+ }
+ incomingChecksumToSoftware[string(checksum)] = s
+ newSoftware[string(checksum)] = struct{}{}
+ }
+ }
+
+ if len(incomingChecksumToSoftware) > 0 {
+ keys := make([]string, 0, len(incomingChecksumToSoftware))
+ for checksum := range incomingChecksumToSoftware {
+ keys = append(keys, checksum)
+ }
+ // We use the replica DB for retrieval to minimize the traffic to the master DB.
+ // It is OK if the software is not found in the replica DB, because we will then attempt to insert it in the master DB.
+ currentSoftware, err = getSoftwareIDsByChecksums(ctx, ds.reader(ctx), keys)
+ if err != nil {
+ return nil, nil, nil, err
+ }
+ for _, s := range currentSoftware {
+ _, ok := incomingChecksumToSoftware[s.Checksum]
+ if !ok {
+ // This should never happen. If it does, we have a bug.
+ return nil, nil, nil, ctxerr.New(
+ ctx, fmt.Sprintf("software not found for checksum %s", hex.EncodeToString([]byte(s.Checksum))),
+ )
+ }
+ delete(newSoftware, s.Checksum)
+ }
+ }
+
+ // Get software titles for new software, if any
+ incomingChecksumToTitle = make(map[string]fleet.SoftwareTitle, len(newSoftware))
+ if len(newSoftware) > 0 {
+ totalToProcess := len(newSoftware)
+ const numberOfArgsPerSoftwareTitle = 3 // number of ? in each WHERE clause
+ whereClause := strings.TrimSuffix(
+ strings.Repeat("(name = ? AND source = ? AND browser = ?) OR", totalToProcess), " OR",
+ )
+ stmt := fmt.Sprintf(
+ "SELECT id, name, source, browser FROM software_titles WHERE %s",
+ whereClause,
+ )
+ args := make([]interface{}, 0, totalToProcess*numberOfArgsPerSoftwareTitle)
+ uniqueTitleStrToChecksum := make(map[string]string, totalToProcess)
+ for checksum := range newSoftware {
+ sw := incomingChecksumToSoftware[checksum]
+ args = append(args, sw.Name, sw.Source, sw.Browser)
+ // Map software title identifier to software checksums so that we can map checksums to actual titles later.
+ uniqueTitleStrToChecksum[UniqueSoftwareTitleStr(sw.Name, sw.Source, sw.Browser)] = checksum
+ }
+ var existingSoftwareTitlesForNewSoftware []fleet.SoftwareTitle
+ if err := sqlx.SelectContext(ctx, ds.reader(ctx), &existingSoftwareTitlesForNewSoftware, stmt, args...); err != nil {
+ return nil, nil, nil, ctxerr.Wrap(ctx, err, "get existing titles")
+ }
+
+ // Map software titles to software checksums.
+ for _, title := range existingSoftwareTitlesForNewSoftware {
+ checksum, ok := uniqueTitleStrToChecksum[UniqueSoftwareTitleStr(title.Name, title.Source, title.Browser)]
+ if ok {
+ incomingChecksumToTitle[checksum] = title
+ }
+ }
+ }
+
+ return currentSoftware, incomingChecksumToSoftware, incomingChecksumToTitle, nil
+}
+
+// UniqueSoftwareTitleStr creates a unique string representation of the software title
+func UniqueSoftwareTitleStr(values ...string) string {
+ return strings.Join(values, fleet.SoftwareFieldSeparator)
+}
+
// delete host_software that is in current map, but not in incoming map.
// returns the deleted software on the host
func deleteUninstalledHostSoftwareDB(
@@ -423,59 +515,22 @@ func computeRawChecksum(sw fleet.Software) ([]byte, error) {
return h.Sum(nil), nil
}
-// insert host_software that is in incoming map, but not in current map.
+// Insert host_software that is in softwareChecksums map, but not in existingSoftware.
+// Also insert any new software titles that are needed.
// returns the inserted software on the host
-//
-//nolint:gocritic // This function uses a read from DB replica inside a transaction. This is an intentional optimization to reduce master DB load.
func (ds *Datastore) insertNewInstalledHostSoftwareDB(
ctx context.Context,
tx sqlx.ExtContext,
hostID uint,
- currentMap map[string]fleet.Software,
- incomingMap map[string]fleet.Software,
+ existingSoftware []softwareIDChecksum,
+ softwareChecksums map[string]fleet.Software,
+ existingTitlesForNewSoftware map[string]fleet.SoftwareTitle,
) ([]fleet.Software, error) {
var insertsHostSoftware []interface{}
var insertedSoftware []fleet.Software
- type softwareWithUniqueName struct {
- uniqueName string
- software fleet.Software
- }
- incomingOrdered := make([]softwareWithUniqueName, 0, len(incomingMap))
- for uniqueName, software := range incomingMap {
- incomingOrdered = append(incomingOrdered, softwareWithUniqueName{
- uniqueName: uniqueName,
- software: software,
- })
- }
- sort.Slice(incomingOrdered, func(i, j int) bool {
- return incomingOrdered[i].uniqueName < incomingOrdered[j].uniqueName
- })
-
- // Compute checksums for all incoming software, which we will use for faster retrieval, since checksum is a unique index
- softwareChecksums := make(map[string]*fleet.Software, 0)
- for i, s := range incomingOrdered {
- if _, ok := currentMap[s.uniqueName]; !ok {
- checksum, err := computeRawChecksum(s.software)
- if err != nil {
- return nil, err
- }
- softwareChecksums[string(checksum)] = &incomingOrdered[i].software // we can't use `s` here because it is a pointer to a loop variable
- }
- }
-
- // First, we check if host's new software already exists in the software table.
+ // First, we remove incoming software that already exists in the software table.
if len(softwareChecksums) > 0 {
- keys := make([]string, 0, len(softwareChecksums))
- for checksum := range softwareChecksums {
- keys = append(keys, checksum)
- }
- // We use the replica DB for retrieval to minimize the traffic to the master DB.
- // It is OK if the software is not found in the replica DB, because we will then attempt to insert it in the master DB.
- existingSoftware, err := getSoftwareIDsByChecksums(ctx, ds.reader(ctx), keys)
- if err != nil {
- return nil, err
- }
for _, s := range existingSoftware {
software, ok := softwareChecksums[s.Checksum]
if !ok {
@@ -483,7 +538,7 @@ func (ds *Datastore) insertNewInstalledHostSoftwareDB(
}
software.ID = s.ID
insertsHostSoftware = append(insertsHostSoftware, hostID, software.ID, software.LastOpenedAt)
- insertedSoftware = append(insertedSoftware, *software)
+ insertedSoftware = append(insertedSoftware, software)
delete(softwareChecksums, s.Checksum)
}
}
@@ -501,42 +556,92 @@ func (ds *Datastore) insertNewInstalledHostSoftwareDB(
end = len(keys)
}
totalToProcess := end - start
- const numberOfArgsPerSoftware = 10 // number of ? in each VALUES clause
+
+ // Insert into software
+ const numberOfArgsPerSoftware = 11 // number of ? in each VALUES clause
values := strings.TrimSuffix(
- strings.Repeat("(?,?,?,?,?,?,?,?,?,?),", totalToProcess), ",",
+ strings.Repeat("(?,?,?,?,?,?,?,?,?,?,?),", totalToProcess), ",",
)
// INSERT IGNORE is used to avoid duplicate key errors, which may occur since our previous read came from the replica.
stmt := fmt.Sprintf(
- "INSERT IGNORE INTO software (name, version, source, `release`, vendor, arch, bundle_identifier, extension_id, browser, checksum) VALUES %s",
+ "INSERT IGNORE INTO software (name, version, source, `release`, vendor, arch, bundle_identifier, extension_id, browser, title_id, checksum) VALUES %s",
values,
)
args := make([]interface{}, 0, totalToProcess*numberOfArgsPerSoftware)
+ newTitlesNeeded := make(map[string]fleet.SoftwareTitle)
for j := start; j < end; j++ {
checksum := keys[j]
sw := softwareChecksums[checksum]
+ var titleID *uint
+ title, ok := existingTitlesForNewSoftware[checksum]
+ if ok {
+ titleID = &title.ID
+ } else if _, ok := newTitlesNeeded[checksum]; !ok {
+ newTitlesNeeded[checksum] = fleet.SoftwareTitle{
+ Name: sw.Name,
+ Source: sw.Source,
+ Browser: sw.Browser,
+ }
+ }
args = append(
args, sw.Name, sw.Version, sw.Source, sw.Release, sw.Vendor, sw.Arch, sw.BundleIdentifier, sw.ExtensionID, sw.Browser,
- checksum,
+ titleID, checksum,
)
}
if _, err := tx.ExecContext(ctx, stmt, args...); err != nil {
return nil, ctxerr.Wrap(ctx, err, "insert software")
}
+
+ // Insert into software_titles
+ totalTitlesToProcess := len(newTitlesNeeded)
+ if totalTitlesToProcess > 0 {
+ const numberOfArgsPerSoftwareTitles = 3 // number of ? in each VALUES clause
+ titlesValues := strings.TrimSuffix(strings.Repeat("(?,?,?),", totalTitlesToProcess), ",")
+ // INSERT IGNORE is used to avoid duplicate key errors, which may occur since our previous read came from the replica.
+ titlesStmt := fmt.Sprintf("INSERT IGNORE INTO software_titles (name, source, browser) VALUES %s", titlesValues)
+ titlesArgs := make([]interface{}, 0, totalTitlesToProcess*numberOfArgsPerSoftwareTitles)
+ titleChecksums := make([]string, totalTitlesToProcess)
+ for checksum, title := range newTitlesNeeded {
+ titlesArgs = append(titlesArgs, title.Name, title.Source, title.Browser)
+ titleChecksums = append(titleChecksums, checksum)
+ }
+ if _, err := tx.ExecContext(ctx, titlesStmt, titlesArgs...); err != nil {
+ return nil, ctxerr.Wrap(ctx, err, "insert software_titles")
+ }
+
+ // update new title ids for new software table entries
+ updateSoftwareStmt := `
+ UPDATE
+ software s,
+ software_titles st
+ SET
+ s.title_id = st.id
+ WHERE
+ (s.name, s.source, s.browser) = (st.name, st.source, st.browser)
+ AND s.checksum IN (?)`
+ updateSoftwareStmt, updateArgs, err := sqlx.In(updateSoftwareStmt, titleChecksums)
+ if err != nil {
+ return nil, ctxerr.Wrap(ctx, err, "build update software title_id")
+ }
+ if _, err = tx.ExecContext(ctx, updateSoftwareStmt, updateArgs...); err != nil {
+ return nil, ctxerr.Wrap(ctx, err, "update software title_id")
+ }
+ }
}
// Here, we use the transaction (tx) for retrieval because we must retrieve the software IDs that we just inserted.
- existingSoftware, err := getSoftwareIDsByChecksums(ctx, tx, keys)
+ updatedExistingSoftware, err := getSoftwareIDsByChecksums(ctx, tx, keys)
if err != nil {
return nil, err
}
- for _, s := range existingSoftware {
+ for _, s := range updatedExistingSoftware {
software, ok := softwareChecksums[s.Checksum]
if !ok {
return nil, ctxerr.New(ctx, fmt.Sprintf("software not found for checksum %s", hex.EncodeToString([]byte(s.Checksum))))
}
software.ID = s.ID
insertsHostSoftware = append(insertsHostSoftware, hostID, software.ID, software.LastOpenedAt)
- insertedSoftware = append(insertedSoftware, *software)
+ insertedSoftware = append(insertedSoftware, software)
delete(softwareChecksums, s.Checksum)
}
}
diff --git a/server/datastore/mysql/software_test.go b/server/datastore/mysql/software_test.go
index 01327ea037..329117cb74 100644
--- a/server/datastore/mysql/software_test.go
+++ b/server/datastore/mysql/software_test.go
@@ -224,9 +224,15 @@ func testSoftwareHostDuplicates(t *testing.T, ds *Datastore) {
soft2Key := sw.ToUniqueStr()
incoming[soft2Key] = *sw
+ incomingByChecksum, existingSoftware, existingTitlesForNewSoftware, err := ds.getExistingSoftware(
+ context.Background(), make(map[string]fleet.Software), incoming,
+ )
+ require.NoError(t, err)
tx, err := ds.writer(context.Background()).Beginx()
require.NoError(t, err)
- _, err = ds.insertNewInstalledHostSoftwareDB(context.Background(), tx, host1.ID, make(map[string]fleet.Software), incoming)
+ _, err = ds.insertNewInstalledHostSoftwareDB(
+ context.Background(), tx, host1.ID, incomingByChecksum, existingSoftware, existingTitlesForNewSoftware,
+ )
require.NoError(t, err)
require.NoError(t, tx.Commit())
@@ -247,9 +253,15 @@ func testSoftwareHostDuplicates(t *testing.T, ds *Datastore) {
soft3Key := sw.ToUniqueStr()
incoming[soft3Key] = *sw
+ incomingByChecksum, existingSoftware, existingTitlesForNewSoftware, err = ds.getExistingSoftware(
+ context.Background(), make(map[string]fleet.Software), incoming,
+ )
+ require.NoError(t, err)
tx, err = ds.writer(context.Background()).Beginx()
require.NoError(t, err)
- _, err = ds.insertNewInstalledHostSoftwareDB(context.Background(), tx, host1.ID, make(map[string]fleet.Software), incoming)
+ _, err = ds.insertNewInstalledHostSoftwareDB(
+ context.Background(), tx, host1.ID, incomingByChecksum, existingSoftware, existingTitlesForNewSoftware,
+ )
require.NoError(t, err)
require.NoError(t, tx.Commit())
@@ -1732,6 +1744,16 @@ func testUpdateHostSoftware(t *testing.T, ds *Datastore) {
want := expect[i]
require.Equal(t, want.name, sw.Name)
+ var titleID uint
+ require.NoError(
+ t, ds.writer(ctx).GetContext(
+ ctx, &titleID,
+ `SELECT s.title_id FROM software s INNER JOIN software_titles st ON (s.name = st.name AND s.source = st.source AND s.browser = st.browser) WHERE st.id = ?`,
+ sw.ID,
+ ),
+ )
+ assert.NotZero(t, titleID)
+
if want.ts.IsZero() {
require.Nil(t, sw.LastOpenedAt)
} else {
@@ -1742,7 +1764,7 @@ func testUpdateHostSoftware(t *testing.T, ds *Datastore) {
// set the initial software list
sw := []fleet.Software{
- {Name: "foo", Version: "0.0.1", Source: "test", GenerateCPE: "cpe_foo"},
+ {Name: "foo", Version: "0.0.1", Source: "test", GenerateCPE: "cpe_foo", Browser: "chrome"},
{Name: "bar", Version: "0.0.2", Source: "test", GenerateCPE: "cpe_bar", LastOpenedAt: &lastYear},
{Name: "baz", Version: "0.0.3", Source: "test", GenerateCPE: "cpe_baz", LastOpenedAt: &now},
}
@@ -2761,6 +2783,14 @@ func TestReconcileSoftwareTitles(t *testing.T) {
{Name: "bar", Version: "0.0.3", Source: "deb_packages"},
{Name: "baz", Version: "0.0.1", Source: "deb_packages"},
}
+ expectedTitlesByNSB := map[string]fleet.SoftwareTitle{}
+ for _, s := range expectedSoftware {
+ expectedTitlesByNSB[s.Name+s.Source+s.Browser] = fleet.SoftwareTitle{
+ Name: s.Name,
+ Source: s.Source,
+ Browser: s.Browser,
+ }
+ }
software1 := []fleet.Software{expectedSoftware[0], expectedSoftware[2]}
software2 := []fleet.Software{expectedSoftware[1], expectedSoftware[2], expectedSoftware[3]}
@@ -2793,8 +2823,7 @@ func TestReconcileSoftwareTitles(t *testing.T) {
return swt, nil
}
- expectedTitlesByNSB := map[string]fleet.SoftwareTitle{}
- assertSoftware := func(t *testing.T, wantSoftware []fleet.Software, wantNilTitleID []fleet.Software) {
+ assertSoftware := func(t *testing.T, wantSoftware []fleet.Software) {
gotSoftware, err := getSoftware()
require.NoError(t, err)
require.Len(t, gotSoftware, len(wantSoftware))
@@ -2808,25 +2837,13 @@ func TestReconcileSoftwareTitles(t *testing.T) {
_, ok := byNSBV[r.Name+r.Source+r.Browser+r.Version]
require.True(t, ok)
- if r.TitleID == nil {
- var found bool
- for _, s := range wantNilTitleID {
- if s.Name == r.Name && s.Source == r.Source && s.Browser == r.Browser && s.Version == r.Version {
- found = true
- break
- }
- }
- require.True(t, found)
- } else {
- require.NotNil(t, r.TitleID)
- swt, ok := expectedTitlesByNSB[r.Name+r.Source+r.Browser]
- require.True(t, ok)
- require.NotNil(t, r.TitleID)
- require.Equal(t, swt.ID, *r.TitleID)
- require.Equal(t, swt.Name, r.Name)
- require.Equal(t, swt.Source, r.Source)
- require.Equal(t, swt.Browser, r.Browser)
- }
+ assert.NotNil(t, r.TitleID)
+ swt, ok := expectedTitlesByNSB[r.Name+r.Source+r.Browser]
+ require.True(t, ok)
+ assert.Equal(t, swt.ID, *r.TitleID)
+ assert.Equal(t, swt.Name, r.Name)
+ assert.Equal(t, swt.Source, r.Source)
+ assert.Equal(t, swt.Browser, r.Browser)
}
}
@@ -2844,8 +2861,15 @@ func TestReconcileSoftwareTitles(t *testing.T) {
}
}
- // title_id is initially nil for all software entries
- assertSoftware(t, expectedSoftware, expectedSoftware)
+ swTitles, err := getTitles()
+ require.NoError(t, err)
+ for _, swt := range swTitles {
+ if _, ok := expectedTitlesByNSB[swt.Name+swt.Source+swt.Browser]; ok {
+ expectedTitlesByNSB[swt.Name+swt.Source+swt.Browser] = swt
+ }
+ }
+
+ assertSoftware(t, expectedSoftware)
// reconcile software titles
require.NoError(t, ds.ReconcileSoftwareTitles(ctx))
@@ -2873,15 +2897,15 @@ func TestReconcileSoftwareTitles(t *testing.T) {
require.Equal(t, swt[3].Browser, "chrome")
expectedTitlesByNSB[swt[3].Name+swt[3].Source+swt[3].Browser] = swt[3]
- // title_id is now populated for all software entries
- assertSoftware(t, expectedSoftware, nil)
+ // Double check software and titles
+ assertSoftware(t, expectedSoftware)
// remove the bar software title from host 2
_, err = ds.UpdateHostSoftware(context.Background(), host2.ID, software2[:2])
require.NoError(t, err)
// SyncHostsSoftware will remove the above software item from the software table
require.NoError(t, ds.SyncHostsSoftware(context.Background(), time.Now()))
- assertSoftware(t, []fleet.Software{expectedSoftware[0], expectedSoftware[1], expectedSoftware[2], expectedSoftware[4]}, nil)
+ assertSoftware(t, []fleet.Software{expectedSoftware[0], expectedSoftware[1], expectedSoftware[2], expectedSoftware[4]})
// bar is no longer associated with any host so the title should be deleted
require.NoError(t, ds.ReconcileSoftwareTitles(context.Background()))
@@ -2894,18 +2918,6 @@ func TestReconcileSoftwareTitles(t *testing.T) {
_, err = ds.UpdateHostSoftware(context.Background(), host3.ID, []fleet.Software{expectedSoftware[3], expectedSoftware[4]})
require.NoError(t, err)
require.NoError(t, ds.SyncHostsSoftware(context.Background(), time.Now()))
-
- // title_id is initially nil for new software entries
- assertSoftware(t, expectedSoftware, []fleet.Software{expectedSoftware[3]})
-
- // bar isn't added back to software titles until we reconcile software titles
- gotTitles, err = getTitles()
- require.NoError(t, err)
- require.Len(t, gotTitles, 3)
- assertTitles(t, gotTitles, []string{"bar"})
-
- // reconcile software titles
- require.NoError(t, ds.ReconcileSoftwareTitles(ctx))
gotTitles, err = getTitles()
require.NoError(t, err)
require.Len(t, gotTitles, 4)
@@ -2916,38 +2928,24 @@ func TestReconcileSoftwareTitles(t *testing.T) {
require.NotEqual(t, expectedTitlesByNSB[gotTitles[0].Name+gotTitles[0].Source], gotTitles[0].ID)
expectedTitlesByNSB[gotTitles[0].Name+gotTitles[0].Source] = gotTitles[0]
assertTitles(t, gotTitles, nil)
-
- // title_id is now populated for bar
- assertSoftware(t, expectedSoftware, nil)
+ assertSoftware(t, expectedSoftware)
// add a new version of foo to host 3
expectedSoftware = append(expectedSoftware, fleet.Software{Name: "foo", Version: "0.0.4", Source: "chrome_extensions"})
_, err = ds.UpdateHostSoftware(ctx, host3.ID, expectedSoftware[3:])
require.NoError(t, err)
-
- // title_id is initially nil for new software entries
- assertSoftware(t, expectedSoftware, []fleet.Software{expectedSoftware[5]})
-
- // new version of foo doesn't result in a new software title entry
- require.NoError(t, ds.ReconcileSoftwareTitles(ctx))
gotTitles, err = getTitles()
require.NoError(t, err)
require.Len(t, gotTitles, 4)
assertTitles(t, gotTitles, nil)
-
- // title_id is now populated for new version of foo
- assertSoftware(t, expectedSoftware, nil)
+ assertSoftware(t, expectedSoftware)
// add a new source of foo to host 3
expectedSoftware = append(expectedSoftware, fleet.Software{Name: "foo", Version: "0.0.4", Source: "rpm_packages"})
_, err = ds.UpdateHostSoftware(ctx, host3.ID, expectedSoftware[3:])
require.NoError(t, err)
- // title_id is initially nil for new software entries
- assertSoftware(t, expectedSoftware, []fleet.Software{expectedSoftware[6]})
-
// new source of foo results in a new software title entry
- require.NoError(t, ds.ReconcileSoftwareTitles(ctx))
gotTitles, err = getTitles()
require.NoError(t, err)
require.Len(t, gotTitles, 5)
@@ -2956,9 +2954,7 @@ func TestReconcileSoftwareTitles(t *testing.T) {
require.Equal(t, "", gotTitles[4].Browser)
expectedTitlesByNSB[gotTitles[4].Name+gotTitles[4].Source+gotTitles[4].Browser] = gotTitles[4]
assertTitles(t, gotTitles, nil)
-
- // title_id is now populated for new source of foo
- assertSoftware(t, expectedSoftware, nil)
+ assertSoftware(t, expectedSoftware)
}
func testUpdateHostSoftwareDeadlock(t *testing.T, ds *Datastore) {
diff --git a/server/datastore/s3/carves.go b/server/datastore/s3/carves.go
index 5494e2c958..7aae963301 100644
--- a/server/datastore/s3/carves.go
+++ b/server/datastore/s3/carves.go
@@ -35,7 +35,7 @@ type CarveStore struct {
// NewCarveStore creates a new store with the given config
func NewCarveStore(config config.S3Config, metadatadb fleet.CarveStore) (*CarveStore, error) {
- s3store, err := newS3store(config)
+ s3store, err := newS3store(config.CarvesToInternalCfg())
if err != nil {
return nil, err
}
@@ -57,7 +57,6 @@ func (c *CarveStore) NewCarve(ctx context.Context, metadata *fleet.CarveMetadata
Bucket: &c.bucket,
Key: &objectKey,
})
-
if err != nil {
// even if we fail to create the multipart upload, we still want to create
// the carve in the database and register an error, this way the user can
diff --git a/server/datastore/s3/installer.go b/server/datastore/s3/installer.go
index 8d25ee4cb9..2df34b3359 100644
--- a/server/datastore/s3/installer.go
+++ b/server/datastore/s3/installer.go
@@ -37,7 +37,7 @@ type InstallerStore struct {
// NewInstallerStore creates a new instance with the given S3 config
func NewInstallerStore(config config.S3Config) (*InstallerStore, error) {
- s3store, err := newS3store(config)
+ s3store, err := newS3store(config.CarvesToInternalCfg())
if err != nil {
return nil, err
}
@@ -48,7 +48,6 @@ func NewInstallerStore(config config.S3Config) (*InstallerStore, error) {
func (i *InstallerStore) Get(ctx context.Context, installer fleet.Installer) (io.ReadCloser, int64, error) {
key := i.keyForInstaller(installer)
req, err := i.s3client.GetObject(&s3.GetObjectInput{Bucket: &i.bucket, Key: &key})
-
if err != nil {
if aerr, ok := err.(awserr.Error); ok {
switch aerr.Code() {
@@ -78,7 +77,6 @@ func (i *InstallerStore) Put(ctx context.Context, installer fleet.Installer) (st
func (i *InstallerStore) Exists(ctx context.Context, installer fleet.Installer) (bool, error) {
key := i.keyForInstaller(installer)
_, err := i.s3client.HeadObject(&s3.HeadObjectInput{Bucket: &i.bucket, Key: &key})
-
if err != nil {
if aerr, ok := err.(awserr.Error); ok {
switch aerr.Code() {
diff --git a/server/datastore/s3/s3.go b/server/datastore/s3/s3.go
index 394da53ffa..0c67a64390 100644
--- a/server/datastore/s3/s3.go
+++ b/server/datastore/s3/s3.go
@@ -23,7 +23,7 @@ type s3store struct {
}
// newS3store initializes an S3 Datastore
-func newS3store(config config.S3Config) (*s3store, error) {
+func newS3store(config config.S3ConfigInternal) (*s3store, error) {
conf := &aws.Config{}
// Use default auth provire if no static credentials were provided
diff --git a/server/datastore/s3/software_installer.go b/server/datastore/s3/software_installer.go
index 1966326266..5b33d10127 100644
--- a/server/datastore/s3/software_installer.go
+++ b/server/datastore/s3/software_installer.go
@@ -21,7 +21,7 @@ type SoftwareInstallerStore struct {
// NewSoftwareInstallerStore creates a new instance with the given S3 config.
func NewSoftwareInstallerStore(config config.S3Config) (*SoftwareInstallerStore, error) {
- s3store, err := newS3store(config)
+ s3store, err := newS3store(config.SoftwareInstallersToInternalCfg())
if err != nil {
return nil, err
}
diff --git a/server/datastore/s3/testing_utils.go b/server/datastore/s3/testing_utils.go
index 50e02f5758..7854d3b34f 100644
--- a/server/datastore/s3/testing_utils.go
+++ b/server/datastore/s3/testing_utils.go
@@ -42,14 +42,23 @@ func setupTestStore[T testBucketCreator](tb testing.TB, bucket, prefix string, n
checkEnv(tb)
store, err := newFn(config.S3Config{
- Bucket: bucket,
- Prefix: prefix,
- Region: "minio",
- EndpointURL: testEndpoint,
- AccessKeyID: accessKeyID,
- SecretAccessKey: secretAccessKey,
- ForceS3PathStyle: true,
- DisableSSL: true,
+ SoftwareInstallersBucket: bucket,
+ SoftwareInstallersPrefix: prefix,
+ SoftwareInstallersRegion: "minio",
+ SoftwareInstallersEndpointURL: testEndpoint,
+ SoftwareInstallersAccessKeyID: accessKeyID,
+ SoftwareInstallersSecretAccessKey: secretAccessKey,
+ SoftwareInstallersForceS3PathStyle: true,
+ SoftwareInstallersDisableSSL: true,
+
+ CarvesBucket: bucket,
+ CarvesPrefix: prefix,
+ CarvesRegion: "minio",
+ CarvesEndpointURL: testEndpoint,
+ CarvesAccessKeyID: accessKeyID,
+ CarvesSecretAccessKey: secretAccessKey,
+ CarvesForceS3PathStyle: true,
+ CarvesDisableSSL: true,
})
require.Nil(tb, err)
diff --git a/server/fleet/agent_options.go b/server/fleet/agent_options.go
index ef38d7d23c..53e2d4674a 100644
--- a/server/fleet/agent_options.go
+++ b/server/fleet/agent_options.go
@@ -9,6 +9,8 @@ import (
"strings"
)
+//go:generate go run ../../tools/osquery-agent-options agent_options_generated.go
+
type AgentOptions struct {
// Config is the base config options.
Config json.RawMessage `json:"config"`
@@ -153,9 +155,6 @@ func validateJSONAgentOptionsExtensions(ctx context.Context, ds Datastore, optsE
// JSON definition of the available configuration options in osquery.
// See https://osquery.readthedocs.io/en/stable/deployment/configuration/#configuration-specification
-//
-// NOTE: Update the following line with the version used for validation.
-// Current version: 5.11.0
type osqueryAgentOptions struct {
Options osqueryOptions `json:"options"`
@@ -212,310 +211,6 @@ type osqueryAgentOptions struct {
} `json:"events"`
}
-// NOTE: generate automatically with `go run ./tools/osquery-agent-options/main.go`
-type osqueryOptions struct {
- AuditAllowConfig bool `json:"audit_allow_config"`
- AuditAllowFimEvents bool `json:"audit_allow_fim_events"`
- AuditAllowProcessEvents bool `json:"audit_allow_process_events"`
- AuditAllowSockets bool `json:"audit_allow_sockets"`
- AuditAllowUserEvents bool `json:"audit_allow_user_events"`
- AugeasLenses string `json:"augeas_lenses"`
- AwsAccessKeyId string `json:"aws_access_key_id"`
- AwsDebug bool `json:"aws_debug"`
- AwsDisableImdsv1Fallback bool `json:"aws_disable_imdsv1_fallback"`
- AwsEnableProxy bool `json:"aws_enable_proxy"`
- AwsFirehoseEndpoint string `json:"aws_firehose_endpoint"`
- AwsFirehosePeriod uint64 `json:"aws_firehose_period"`
- AwsFirehoseRegion string `json:"aws_firehose_region"`
- AwsFirehoseStream string `json:"aws_firehose_stream"`
- AwsImdsv2RequestAttempts uint32 `json:"aws_imdsv2_request_attempts"`
- AwsImdsv2RequestInterval uint32 `json:"aws_imdsv2_request_interval"`
- AwsKinesisDisableLogStatus bool `json:"aws_kinesis_disable_log_status"`
- AwsKinesisEndpoint string `json:"aws_kinesis_endpoint"`
- AwsKinesisPeriod uint64 `json:"aws_kinesis_period"`
- AwsKinesisRandomPartitionKey bool `json:"aws_kinesis_random_partition_key"`
- AwsKinesisRegion string `json:"aws_kinesis_region"`
- AwsKinesisStream string `json:"aws_kinesis_stream"`
- AwsProfileName string `json:"aws_profile_name"`
- AwsProxyHost string `json:"aws_proxy_host"`
- AwsProxyPassword string `json:"aws_proxy_password"`
- AwsProxyPort uint32 `json:"aws_proxy_port"`
- AwsProxyScheme string `json:"aws_proxy_scheme"`
- AwsProxyUsername string `json:"aws_proxy_username"`
- AwsRegion string `json:"aws_region"`
- AwsSecretAccessKey string `json:"aws_secret_access_key"`
- AwsSessionToken string `json:"aws_session_token"`
- AwsStsArnRole string `json:"aws_sts_arn_role"`
- AwsStsRegion string `json:"aws_sts_region"`
- AwsStsSessionName string `json:"aws_sts_session_name"`
- AwsStsTimeout uint64 `json:"aws_sts_timeout"`
- BufferedLogMax uint64 `json:"buffered_log_max"`
- DecorationsTopLevel bool `json:"decorations_top_level"`
- DisableAudit bool `json:"disable_audit"`
- DisableCaching bool `json:"disable_caching"`
- DisableDatabase bool `json:"disable_database"`
- DisableDecorators bool `json:"disable_decorators"`
- DisableDistributed bool `json:"disable_distributed"`
- DisableEvents bool `json:"disable_events"`
- DisableHashCache bool `json:"disable_hash_cache"`
- DisableLogging bool `json:"disable_logging"`
- DistributedDenylistDuration uint64 `json:"distributed_denylist_duration"`
- DistributedInterval uint64 `json:"distributed_interval"`
- DistributedLoginfo bool `json:"distributed_loginfo"`
- DistributedPlugin string `json:"distributed_plugin"`
- DistributedTlsMaxAttempts uint64 `json:"distributed_tls_max_attempts"`
- DistributedTlsReadEndpoint string `json:"distributed_tls_read_endpoint"`
- DistributedTlsWriteEndpoint string `json:"distributed_tls_write_endpoint"`
- DockerSocket string `json:"docker_socket"`
- EnableFileEvents bool `json:"enable_file_events"`
- EnableForeign bool `json:"enable_foreign"`
- EnableNumericMonitoring bool `json:"enable_numeric_monitoring"`
- Ephemeral bool `json:"ephemeral"`
- EsFimEnableOpenEvents bool `json:"es_fim_enable_open_events"`
- EventsExpiry uint64 `json:"events_expiry"`
- EventsMax uint64 `json:"events_max"`
- EventsOptimize bool `json:"events_optimize"`
- ExperimentList string `json:"experiment_list"`
- ExtensionsDefaultIndex bool `json:"extensions_default_index"`
- HashCacheMax uint32 `json:"hash_cache_max"`
- HostIdentifier string `json:"host_identifier"`
- IgnoreTableExceptions bool `json:"ignore_table_exceptions"`
- KeychainAccessCache bool `json:"keychain_access_cache"`
- KeychainAccessInterval uint32 `json:"keychain_access_interval"`
- LoggerEventType bool `json:"logger_event_type"`
- LoggerKafkaAcks string `json:"logger_kafka_acks"`
- LoggerKafkaBrokers string `json:"logger_kafka_brokers"`
- LoggerKafkaCompression string `json:"logger_kafka_compression"`
- LoggerKafkaTopic string `json:"logger_kafka_topic"`
- LoggerMinStatus int32 `json:"logger_min_status"`
- LoggerMinStderr int32 `json:"logger_min_stderr"`
- LoggerNumerics bool `json:"logger_numerics"`
- LoggerPath string `json:"logger_path"`
- LoggerRotate bool `json:"logger_rotate"`
- LoggerRotateMaxFiles uint64 `json:"logger_rotate_max_files"`
- LoggerRotateSize uint64 `json:"logger_rotate_size"`
- LoggerSnapshotEventType bool `json:"logger_snapshot_event_type"`
- LoggerSyslogFacility int32 `json:"logger_syslog_facility"`
- LoggerSyslogPrependCee bool `json:"logger_syslog_prepend_cee"`
- LoggerTlsCompress bool `json:"logger_tls_compress"`
- LoggerTlsEndpoint string `json:"logger_tls_endpoint"`
- LoggerTlsMaxLines uint64 `json:"logger_tls_max_lines"`
- LoggerTlsMaxLinesize uint64 `json:"logger_tls_max_linesize"`
- LoggerTlsPeriod uint64 `json:"logger_tls_period"`
- Nullvalue string `json:"nullvalue"`
- NumericMonitoringFilesystemPath string `json:"numeric_monitoring_filesystem_path"`
- NumericMonitoringPlugins string `json:"numeric_monitoring_plugins"`
- NumericMonitoringPreAggregationTime uint64 `json:"numeric_monitoring_pre_aggregation_time"`
- PackDelimiter string `json:"pack_delimiter"`
- PackRefreshInterval uint64 `json:"pack_refresh_interval"`
- ReadMax uint64 `json:"read_max"`
- ScheduleDefaultInterval uint64 `json:"schedule_default_interval"`
- ScheduleEpoch uint64 `json:"schedule_epoch"`
- ScheduleLognames bool `json:"schedule_lognames"`
- ScheduleMaxDrift uint64 `json:"schedule_max_drift"`
- ScheduleReload uint64 `json:"schedule_reload"`
- ScheduleSplayPercent uint64 `json:"schedule_splay_percent"`
- ScheduleTimeout uint64 `json:"schedule_timeout"`
- SpecifiedIdentifier string `json:"specified_identifier"`
- TableDelay uint64 `json:"table_delay"`
- ThriftStringSizeLimit int32 `json:"thrift_string_size_limit"`
- ThriftTimeout uint32 `json:"thrift_timeout"`
- ThriftVerbose bool `json:"thrift_verbose"`
- TlsDisableStatusLog bool `json:"tls_disable_status_log"`
- Verbose bool `json:"verbose"`
- WorkerThreads int32 `json:"worker_threads"`
- YaraDelay uint32 `json:"yara_delay"`
-
- // embed the os-specific structs
- OsqueryCommandLineFlagsLinux
- OsqueryCommandLineFlagsWindows
- OsqueryCommandLineFlagsMacOS
- OsqueryCommandLineFlagsHidden
-}
-
-// NOTE: generate automatically with `go run ./tools/osquery-agent-options/main.go`
-type osqueryCommandLineFlags struct {
- AlarmTimeout uint64 `json:"alarm_timeout"`
- AuditAllowConfig bool `json:"audit_allow_config"`
- AuditAllowFimEvents bool `json:"audit_allow_fim_events"`
- AuditAllowProcessEvents bool `json:"audit_allow_process_events"`
- AuditAllowSockets bool `json:"audit_allow_sockets"`
- AuditAllowUserEvents bool `json:"audit_allow_user_events"`
- AugeasLenses string `json:"augeas_lenses"`
- AwsAccessKeyId string `json:"aws_access_key_id"`
- AwsDebug bool `json:"aws_debug"`
- AwsDisableImdsv1Fallback bool `json:"aws_disable_imdsv1_fallback"`
- AwsEnableProxy bool `json:"aws_enable_proxy"`
- AwsEnforceFips bool `json:"aws_enforce_fips"`
- AwsFirehoseEndpoint string `json:"aws_firehose_endpoint"`
- AwsFirehosePeriod uint64 `json:"aws_firehose_period"`
- AwsFirehoseRegion string `json:"aws_firehose_region"`
- AwsFirehoseStream string `json:"aws_firehose_stream"`
- AwsImdsv2RequestAttempts uint32 `json:"aws_imdsv2_request_attempts"`
- AwsImdsv2RequestInterval uint32 `json:"aws_imdsv2_request_interval"`
- AwsKinesisDisableLogStatus bool `json:"aws_kinesis_disable_log_status"`
- AwsKinesisEndpoint string `json:"aws_kinesis_endpoint"`
- AwsKinesisPeriod uint64 `json:"aws_kinesis_period"`
- AwsKinesisRandomPartitionKey bool `json:"aws_kinesis_random_partition_key"`
- AwsKinesisRegion string `json:"aws_kinesis_region"`
- AwsKinesisStream string `json:"aws_kinesis_stream"`
- AwsProfileName string `json:"aws_profile_name"`
- AwsProxyHost string `json:"aws_proxy_host"`
- AwsProxyPassword string `json:"aws_proxy_password"`
- AwsProxyPort uint32 `json:"aws_proxy_port"`
- AwsProxyScheme string `json:"aws_proxy_scheme"`
- AwsProxyUsername string `json:"aws_proxy_username"`
- AwsRegion string `json:"aws_region"`
- AwsSecretAccessKey string `json:"aws_secret_access_key"`
- AwsSessionToken string `json:"aws_session_token"`
- AwsStsArnRole string `json:"aws_sts_arn_role"`
- AwsStsRegion string `json:"aws_sts_region"`
- AwsStsSessionName string `json:"aws_sts_session_name"`
- AwsStsTimeout uint64 `json:"aws_sts_timeout"`
- BufferedLogMax uint64 `json:"buffered_log_max"`
- CarverBlockSize uint32 `json:"carver_block_size"`
- CarverCompression bool `json:"carver_compression"`
- CarverContinueEndpoint string `json:"carver_continue_endpoint"`
- CarverDisableFunction bool `json:"carver_disable_function"`
- CarverExpiry uint32 `json:"carver_expiry"`
- CarverStartEndpoint string `json:"carver_start_endpoint"`
- ConfigAcceleratedRefresh uint64 `json:"config_accelerated_refresh"`
- ConfigCheck bool `json:"config_check"`
- ConfigDump bool `json:"config_dump"`
- ConfigEnableBackup bool `json:"config_enable_backup"`
- ConfigPath string `json:"config_path"`
- ConfigPlugin string `json:"config_plugin"`
- ConfigRefresh uint64 `json:"config_refresh"`
- ConfigTlsEndpoint string `json:"config_tls_endpoint"`
- ConfigTlsMaxAttempts uint64 `json:"config_tls_max_attempts"`
- Daemonize bool `json:"daemonize"`
- DatabaseDump bool `json:"database_dump"`
- DatabasePath string `json:"database_path"`
- DecorationsTopLevel bool `json:"decorations_top_level"`
- DisableAudit bool `json:"disable_audit"`
- DisableCaching bool `json:"disable_caching"`
- DisableCarver bool `json:"disable_carver"`
- DisableDatabase bool `json:"disable_database"`
- DisableDecorators bool `json:"disable_decorators"`
- DisableDistributed bool `json:"disable_distributed"`
- DisableEnrollment bool `json:"disable_enrollment"`
- DisableEvents bool `json:"disable_events"`
- DisableExtensions bool `json:"disable_extensions"`
- DisableHashCache bool `json:"disable_hash_cache"`
- DisableLogging bool `json:"disable_logging"`
- DisableReenrollment bool `json:"disable_reenrollment"`
- DisableTables string `json:"disable_tables"`
- DisableWatchdog bool `json:"disable_watchdog"`
- DistributedDenylistDuration uint64 `json:"distributed_denylist_duration"`
- DistributedInterval uint64 `json:"distributed_interval"`
- DistributedLoginfo bool `json:"distributed_loginfo"`
- DistributedPlugin string `json:"distributed_plugin"`
- DistributedTlsMaxAttempts uint64 `json:"distributed_tls_max_attempts"`
- DistributedTlsReadEndpoint string `json:"distributed_tls_read_endpoint"`
- DistributedTlsWriteEndpoint string `json:"distributed_tls_write_endpoint"`
- DockerSocket string `json:"docker_socket"`
- EnableExtensionsWatchdog bool `json:"enable_extensions_watchdog"`
- EnableFileEvents bool `json:"enable_file_events"`
- EnableForeign bool `json:"enable_foreign"`
- EnableNumericMonitoring bool `json:"enable_numeric_monitoring"`
- EnableTables string `json:"enable_tables"`
- EnableWatchdogDebug bool `json:"enable_watchdog_debug"`
- EnrollAlways bool `json:"enroll_always"`
- EnrollSecretEnv string `json:"enroll_secret_env"`
- EnrollSecretPath string `json:"enroll_secret_path"`
- EnrollTlsEndpoint string `json:"enroll_tls_endpoint"`
- Ephemeral bool `json:"ephemeral"`
- EsFimEnableOpenEvents bool `json:"es_fim_enable_open_events"`
- EventsExpiry uint64 `json:"events_expiry"`
- EventsMax uint64 `json:"events_max"`
- EventsOptimize bool `json:"events_optimize"`
- ExperimentList string `json:"experiment_list"`
- ExtensionsAutoload string `json:"extensions_autoload"`
- ExtensionsDefaultIndex bool `json:"extensions_default_index"`
- ExtensionsInterval string `json:"extensions_interval"`
- ExtensionsRequire string `json:"extensions_require"`
- ExtensionsSocket string `json:"extensions_socket"`
- ExtensionsTimeout string `json:"extensions_timeout"`
- Force bool `json:"force"`
- HashCacheMax uint32 `json:"hash_cache_max"`
- HostIdentifier string `json:"host_identifier"`
- IgnoreTableExceptions bool `json:"ignore_table_exceptions"`
- Install bool `json:"install"`
- KeychainAccessCache bool `json:"keychain_access_cache"`
- KeychainAccessInterval uint32 `json:"keychain_access_interval"`
- LoggerEventType bool `json:"logger_event_type"`
- LoggerKafkaAcks string `json:"logger_kafka_acks"`
- LoggerKafkaBrokers string `json:"logger_kafka_brokers"`
- LoggerKafkaCompression string `json:"logger_kafka_compression"`
- LoggerKafkaTopic string `json:"logger_kafka_topic"`
- LoggerMinStatus int32 `json:"logger_min_status"`
- LoggerMinStderr int32 `json:"logger_min_stderr"`
- LoggerMode string `json:"logger_mode"`
- LoggerNumerics bool `json:"logger_numerics"`
- LoggerPath string `json:"logger_path"`
- LoggerPlugin string `json:"logger_plugin"`
- LoggerRotate bool `json:"logger_rotate"`
- LoggerRotateMaxFiles uint64 `json:"logger_rotate_max_files"`
- LoggerRotateSize uint64 `json:"logger_rotate_size"`
- LoggerSnapshotEventType bool `json:"logger_snapshot_event_type"`
- LoggerStderr bool `json:"logger_stderr"`
- LoggerSyslogFacility int32 `json:"logger_syslog_facility"`
- LoggerSyslogPrependCee bool `json:"logger_syslog_prepend_cee"`
- LoggerTlsCompress bool `json:"logger_tls_compress"`
- LoggerTlsEndpoint string `json:"logger_tls_endpoint"`
- LoggerTlsMaxLines uint64 `json:"logger_tls_max_lines"`
- LoggerTlsMaxLinesize uint64 `json:"logger_tls_max_linesize"`
- LoggerTlsPeriod uint64 `json:"logger_tls_period"`
- Logtostderr bool `json:"logtostderr"`
- Nullvalue string `json:"nullvalue"`
- NumericMonitoringFilesystemPath string `json:"numeric_monitoring_filesystem_path"`
- NumericMonitoringPlugins string `json:"numeric_monitoring_plugins"`
- NumericMonitoringPreAggregationTime uint64 `json:"numeric_monitoring_pre_aggregation_time"`
- PackDelimiter string `json:"pack_delimiter"`
- PackRefreshInterval uint64 `json:"pack_refresh_interval"`
- Pidfile string `json:"pidfile"`
- ProxyHostname string `json:"proxy_hostname"`
- ReadMax uint64 `json:"read_max"`
- ScheduleDefaultInterval uint64 `json:"schedule_default_interval"`
- ScheduleEpoch uint64 `json:"schedule_epoch"`
- ScheduleLognames bool `json:"schedule_lognames"`
- ScheduleMaxDrift uint64 `json:"schedule_max_drift"`
- ScheduleReload uint64 `json:"schedule_reload"`
- ScheduleSplayPercent uint64 `json:"schedule_splay_percent"`
- ScheduleTimeout uint64 `json:"schedule_timeout"`
- SpecifiedIdentifier string `json:"specified_identifier"`
- Stderrthreshold int32 `json:"stderrthreshold"`
- TableDelay uint64 `json:"table_delay"`
- ThriftStringSizeLimit int32 `json:"thrift_string_size_limit"`
- ThriftTimeout uint32 `json:"thrift_timeout"`
- ThriftVerbose bool `json:"thrift_verbose"`
- TlsClientCert string `json:"tls_client_cert"`
- TlsClientKey string `json:"tls_client_key"`
- TlsDisableStatusLog bool `json:"tls_disable_status_log"`
- TlsEnrollMaxAttempts uint64 `json:"tls_enroll_max_attempts"`
- TlsEnrollMaxInterval uint64 `json:"tls_enroll_max_interval"`
- TlsHostname string `json:"tls_hostname"`
- TlsServerCerts string `json:"tls_server_certs"`
- TlsSessionReuse bool `json:"tls_session_reuse"`
- TlsSessionTimeout uint32 `json:"tls_session_timeout"`
- Uninstall bool `json:"uninstall"`
- Verbose bool `json:"verbose"`
- WatchdogDelay uint64 `json:"watchdog_delay"`
- WatchdogForcedShutdownDelay uint64 `json:"watchdog_forced_shutdown_delay"`
- WatchdogLatencyLimit uint64 `json:"watchdog_latency_limit"`
- WatchdogLevel int32 `json:"watchdog_level"`
- WatchdogMemoryLimit uint64 `json:"watchdog_memory_limit"`
- WatchdogUtilizationLimit uint64 `json:"watchdog_utilization_limit"`
- WorkerThreads int32 `json:"worker_threads"`
- YaraDelay uint32 `json:"yara_delay"`
-
- // embed the os-specific structs
- OsqueryCommandLineFlagsLinux
- OsqueryCommandLineFlagsWindows
- OsqueryCommandLineFlagsMacOS
- OsqueryCommandLineFlagsHidden
-}
-
// the following structs are for OS-specific command-line flags supported by
// osquery. They are exported so they can be used by the
// tools/osquery-agent-options script.
diff --git a/server/fleet/agent_options_generated.go b/server/fleet/agent_options_generated.go
new file mode 100644
index 0000000000..427c390188
--- /dev/null
+++ b/server/fleet/agent_options_generated.go
@@ -0,0 +1,306 @@
+// Automatically generated by tools/osquery-agent-options for osquery 5.12.2. DO NOT EDIT!
+// To update flags for a new osquery version, update the osqueryVersion variable in
+// "tools/osquery-agent-options/main.go" and run "cd server/fleet/ && go generate".
+package fleet
+
+type osqueryOptions struct {
+ AuditAllowConfig bool `json:"audit_allow_config"`
+ AuditAllowFimEvents bool `json:"audit_allow_fim_events"`
+ AuditAllowProcessEvents bool `json:"audit_allow_process_events"`
+ AuditAllowSockets bool `json:"audit_allow_sockets"`
+ AuditAllowUserEvents bool `json:"audit_allow_user_events"`
+ AugeasLenses string `json:"augeas_lenses"`
+ AwsAccessKeyId string `json:"aws_access_key_id"`
+ AwsDebug bool `json:"aws_debug"`
+ AwsDisableImdsv1Fallback bool `json:"aws_disable_imdsv1_fallback"`
+ AwsEnableProxy bool `json:"aws_enable_proxy"`
+ AwsFirehoseEndpoint string `json:"aws_firehose_endpoint"`
+ AwsFirehosePeriod uint64 `json:"aws_firehose_period"`
+ AwsFirehoseRegion string `json:"aws_firehose_region"`
+ AwsFirehoseStream string `json:"aws_firehose_stream"`
+ AwsImdsv2RequestAttempts uint32 `json:"aws_imdsv2_request_attempts"`
+ AwsImdsv2RequestInterval uint32 `json:"aws_imdsv2_request_interval"`
+ AwsKinesisDisableLogStatus bool `json:"aws_kinesis_disable_log_status"`
+ AwsKinesisEndpoint string `json:"aws_kinesis_endpoint"`
+ AwsKinesisPeriod uint64 `json:"aws_kinesis_period"`
+ AwsKinesisRandomPartitionKey bool `json:"aws_kinesis_random_partition_key"`
+ AwsKinesisRegion string `json:"aws_kinesis_region"`
+ AwsKinesisStream string `json:"aws_kinesis_stream"`
+ AwsProfileName string `json:"aws_profile_name"`
+ AwsProxyHost string `json:"aws_proxy_host"`
+ AwsProxyPassword string `json:"aws_proxy_password"`
+ AwsProxyPort uint32 `json:"aws_proxy_port"`
+ AwsProxyScheme string `json:"aws_proxy_scheme"`
+ AwsProxyUsername string `json:"aws_proxy_username"`
+ AwsRegion string `json:"aws_region"`
+ AwsSecretAccessKey string `json:"aws_secret_access_key"`
+ AwsSessionToken string `json:"aws_session_token"`
+ AwsStsArnRole string `json:"aws_sts_arn_role"`
+ AwsStsRegion string `json:"aws_sts_region"`
+ AwsStsSessionName string `json:"aws_sts_session_name"`
+ AwsStsTimeout uint64 `json:"aws_sts_timeout"`
+ BufferedLogMax uint64 `json:"buffered_log_max"`
+ DecorationsTopLevel bool `json:"decorations_top_level"`
+ DisableAudit bool `json:"disable_audit"`
+ DisableCaching bool `json:"disable_caching"`
+ DisableDatabase bool `json:"disable_database"`
+ DisableDecorators bool `json:"disable_decorators"`
+ DisableDistributed bool `json:"disable_distributed"`
+ DisableEvents bool `json:"disable_events"`
+ DisableHashCache bool `json:"disable_hash_cache"`
+ DisableLogging bool `json:"disable_logging"`
+ DistributedDenylistDuration uint64 `json:"distributed_denylist_duration"`
+ DistributedInterval uint64 `json:"distributed_interval"`
+ DistributedLoginfo bool `json:"distributed_loginfo"`
+ DistributedPlugin string `json:"distributed_plugin"`
+ DistributedTlsMaxAttempts uint64 `json:"distributed_tls_max_attempts"`
+ DistributedTlsReadEndpoint string `json:"distributed_tls_read_endpoint"`
+ DistributedTlsWriteEndpoint string `json:"distributed_tls_write_endpoint"`
+ DockerSocket string `json:"docker_socket"`
+ EnableFileEvents bool `json:"enable_file_events"`
+ EnableForeign bool `json:"enable_foreign"`
+ EnableNumericMonitoring bool `json:"enable_numeric_monitoring"`
+ Ephemeral bool `json:"ephemeral"`
+ EsFimEnableOpenEvents bool `json:"es_fim_enable_open_events"`
+ EventsExpiry uint64 `json:"events_expiry"`
+ EventsMax uint64 `json:"events_max"`
+ EventsOptimize bool `json:"events_optimize"`
+ ExperimentList string `json:"experiment_list"`
+ ExtensionsDefaultIndex bool `json:"extensions_default_index"`
+ HashCacheMax uint32 `json:"hash_cache_max"`
+ HostIdentifier string `json:"host_identifier"`
+ IgnoreTableExceptions bool `json:"ignore_table_exceptions"`
+ KeychainAccessCache bool `json:"keychain_access_cache"`
+ KeychainAccessInterval uint32 `json:"keychain_access_interval"`
+ LoggerEventType bool `json:"logger_event_type"`
+ LoggerKafkaAcks string `json:"logger_kafka_acks"`
+ LoggerKafkaBrokers string `json:"logger_kafka_brokers"`
+ LoggerKafkaCompression string `json:"logger_kafka_compression"`
+ LoggerKafkaTopic string `json:"logger_kafka_topic"`
+ LoggerMinStatus int32 `json:"logger_min_status"`
+ LoggerMinStderr int32 `json:"logger_min_stderr"`
+ LoggerNumerics bool `json:"logger_numerics"`
+ LoggerPath string `json:"logger_path"`
+ LoggerRotate bool `json:"logger_rotate"`
+ LoggerRotateMaxFiles uint64 `json:"logger_rotate_max_files"`
+ LoggerRotateSize uint64 `json:"logger_rotate_size"`
+ LoggerSnapshotEventType bool `json:"logger_snapshot_event_type"`
+ LoggerSyslogFacility int32 `json:"logger_syslog_facility"`
+ LoggerSyslogPrependCee bool `json:"logger_syslog_prepend_cee"`
+ LoggerTlsBackoffMax uint64 `json:"logger_tls_backoff_max"`
+ LoggerTlsCompress bool `json:"logger_tls_compress"`
+ LoggerTlsEndpoint string `json:"logger_tls_endpoint"`
+ LoggerTlsMaxLines uint64 `json:"logger_tls_max_lines"`
+ LoggerTlsMaxLinesize uint64 `json:"logger_tls_max_linesize"`
+ LoggerTlsPeriod uint64 `json:"logger_tls_period"`
+ Nullvalue string `json:"nullvalue"`
+ NumericMonitoringFilesystemPath string `json:"numeric_monitoring_filesystem_path"`
+ NumericMonitoringPlugins string `json:"numeric_monitoring_plugins"`
+ NumericMonitoringPreAggregationTime uint64 `json:"numeric_monitoring_pre_aggregation_time"`
+ PackDelimiter string `json:"pack_delimiter"`
+ PackRefreshInterval uint64 `json:"pack_refresh_interval"`
+ ReadMax uint64 `json:"read_max"`
+ ScheduleDefaultInterval uint64 `json:"schedule_default_interval"`
+ ScheduleEpoch uint64 `json:"schedule_epoch"`
+ ScheduleLognames bool `json:"schedule_lognames"`
+ ScheduleMaxDrift uint64 `json:"schedule_max_drift"`
+ ScheduleReload uint64 `json:"schedule_reload"`
+ ScheduleSplayPercent uint64 `json:"schedule_splay_percent"`
+ ScheduleTimeout uint64 `json:"schedule_timeout"`
+ SpecifiedIdentifier string `json:"specified_identifier"`
+ TableDelay uint64 `json:"table_delay"`
+ ThriftStringSizeLimit int32 `json:"thrift_string_size_limit"`
+ ThriftTimeout uint32 `json:"thrift_timeout"`
+ ThriftVerbose bool `json:"thrift_verbose"`
+ TlsDisableStatusLog bool `json:"tls_disable_status_log"`
+ Verbose bool `json:"verbose"`
+ YaraDelay uint32 `json:"yara_delay"`
+
+ // embed the os-specific structs
+ OsqueryCommandLineFlagsLinux
+ OsqueryCommandLineFlagsWindows
+ OsqueryCommandLineFlagsMacOS
+ OsqueryCommandLineFlagsHidden
+}
+
+type osqueryCommandLineFlags struct {
+ AlarmTimeout uint64 `json:"alarm_timeout"`
+ AuditAllowConfig bool `json:"audit_allow_config"`
+ AuditAllowFimEvents bool `json:"audit_allow_fim_events"`
+ AuditAllowProcessEvents bool `json:"audit_allow_process_events"`
+ AuditAllowSockets bool `json:"audit_allow_sockets"`
+ AuditAllowUserEvents bool `json:"audit_allow_user_events"`
+ AugeasLenses string `json:"augeas_lenses"`
+ AwsAccessKeyId string `json:"aws_access_key_id"`
+ AwsDebug bool `json:"aws_debug"`
+ AwsDisableImdsv1Fallback bool `json:"aws_disable_imdsv1_fallback"`
+ AwsEnableProxy bool `json:"aws_enable_proxy"`
+ AwsEnforceFips bool `json:"aws_enforce_fips"`
+ AwsFirehoseEndpoint string `json:"aws_firehose_endpoint"`
+ AwsFirehosePeriod uint64 `json:"aws_firehose_period"`
+ AwsFirehoseRegion string `json:"aws_firehose_region"`
+ AwsFirehoseStream string `json:"aws_firehose_stream"`
+ AwsImdsv2RequestAttempts uint32 `json:"aws_imdsv2_request_attempts"`
+ AwsImdsv2RequestInterval uint32 `json:"aws_imdsv2_request_interval"`
+ AwsKinesisDisableLogStatus bool `json:"aws_kinesis_disable_log_status"`
+ AwsKinesisEndpoint string `json:"aws_kinesis_endpoint"`
+ AwsKinesisPeriod uint64 `json:"aws_kinesis_period"`
+ AwsKinesisRandomPartitionKey bool `json:"aws_kinesis_random_partition_key"`
+ AwsKinesisRegion string `json:"aws_kinesis_region"`
+ AwsKinesisStream string `json:"aws_kinesis_stream"`
+ AwsProfileName string `json:"aws_profile_name"`
+ AwsProxyHost string `json:"aws_proxy_host"`
+ AwsProxyPassword string `json:"aws_proxy_password"`
+ AwsProxyPort uint32 `json:"aws_proxy_port"`
+ AwsProxyScheme string `json:"aws_proxy_scheme"`
+ AwsProxyUsername string `json:"aws_proxy_username"`
+ AwsRegion string `json:"aws_region"`
+ AwsSecretAccessKey string `json:"aws_secret_access_key"`
+ AwsSessionToken string `json:"aws_session_token"`
+ AwsStsArnRole string `json:"aws_sts_arn_role"`
+ AwsStsRegion string `json:"aws_sts_region"`
+ AwsStsSessionName string `json:"aws_sts_session_name"`
+ AwsStsTimeout uint64 `json:"aws_sts_timeout"`
+ BufferedLogMax uint64 `json:"buffered_log_max"`
+ CarverBlockSize uint32 `json:"carver_block_size"`
+ CarverCompression bool `json:"carver_compression"`
+ CarverContinueEndpoint string `json:"carver_continue_endpoint"`
+ CarverDisableFunction bool `json:"carver_disable_function"`
+ CarverExpiry uint32 `json:"carver_expiry"`
+ CarverStartEndpoint string `json:"carver_start_endpoint"`
+ ConfigAcceleratedRefresh uint64 `json:"config_accelerated_refresh"`
+ ConfigCheck bool `json:"config_check"`
+ ConfigDump bool `json:"config_dump"`
+ ConfigEnableBackup bool `json:"config_enable_backup"`
+ ConfigPath string `json:"config_path"`
+ ConfigPlugin string `json:"config_plugin"`
+ ConfigRefresh uint64 `json:"config_refresh"`
+ ConfigTlsEndpoint string `json:"config_tls_endpoint"`
+ ConfigTlsMaxAttempts uint64 `json:"config_tls_max_attempts"`
+ Daemonize bool `json:"daemonize"`
+ DatabaseDump bool `json:"database_dump"`
+ DatabasePath string `json:"database_path"`
+ DecorationsTopLevel bool `json:"decorations_top_level"`
+ DisableAudit bool `json:"disable_audit"`
+ DisableCaching bool `json:"disable_caching"`
+ DisableCarver bool `json:"disable_carver"`
+ DisableDatabase bool `json:"disable_database"`
+ DisableDecorators bool `json:"disable_decorators"`
+ DisableDistributed bool `json:"disable_distributed"`
+ DisableEnrollment bool `json:"disable_enrollment"`
+ DisableEvents bool `json:"disable_events"`
+ DisableExtensions bool `json:"disable_extensions"`
+ DisableHashCache bool `json:"disable_hash_cache"`
+ DisableLogging bool `json:"disable_logging"`
+ DisableReenrollment bool `json:"disable_reenrollment"`
+ DisableTables string `json:"disable_tables"`
+ DisableWatchdog bool `json:"disable_watchdog"`
+ DistributedDenylistDuration uint64 `json:"distributed_denylist_duration"`
+ DistributedInterval uint64 `json:"distributed_interval"`
+ DistributedLoginfo bool `json:"distributed_loginfo"`
+ DistributedPlugin string `json:"distributed_plugin"`
+ DistributedTlsMaxAttempts uint64 `json:"distributed_tls_max_attempts"`
+ DistributedTlsReadEndpoint string `json:"distributed_tls_read_endpoint"`
+ DistributedTlsWriteEndpoint string `json:"distributed_tls_write_endpoint"`
+ DockerSocket string `json:"docker_socket"`
+ EnableExtensionsWatchdog bool `json:"enable_extensions_watchdog"`
+ EnableFileEvents bool `json:"enable_file_events"`
+ EnableForeign bool `json:"enable_foreign"`
+ EnableNumericMonitoring bool `json:"enable_numeric_monitoring"`
+ EnableTables string `json:"enable_tables"`
+ EnableWatchdogDebug bool `json:"enable_watchdog_debug"`
+ EnrollAlways bool `json:"enroll_always"`
+ EnrollSecretEnv string `json:"enroll_secret_env"`
+ EnrollSecretPath string `json:"enroll_secret_path"`
+ EnrollTlsEndpoint string `json:"enroll_tls_endpoint"`
+ Ephemeral bool `json:"ephemeral"`
+ EsFimEnableOpenEvents bool `json:"es_fim_enable_open_events"`
+ EventsExpiry uint64 `json:"events_expiry"`
+ EventsMax uint64 `json:"events_max"`
+ EventsOptimize bool `json:"events_optimize"`
+ ExperimentList string `json:"experiment_list"`
+ ExtensionsAutoload string `json:"extensions_autoload"`
+ ExtensionsDefaultIndex bool `json:"extensions_default_index"`
+ ExtensionsInterval string `json:"extensions_interval"`
+ ExtensionsRequire string `json:"extensions_require"`
+ ExtensionsSocket string `json:"extensions_socket"`
+ ExtensionsTimeout string `json:"extensions_timeout"`
+ Force bool `json:"force"`
+ HashCacheMax uint32 `json:"hash_cache_max"`
+ HostIdentifier string `json:"host_identifier"`
+ IgnoreTableExceptions bool `json:"ignore_table_exceptions"`
+ Install bool `json:"install"`
+ KeychainAccessCache bool `json:"keychain_access_cache"`
+ KeychainAccessInterval uint32 `json:"keychain_access_interval"`
+ LoggerEventType bool `json:"logger_event_type"`
+ LoggerKafkaAcks string `json:"logger_kafka_acks"`
+ LoggerKafkaBrokers string `json:"logger_kafka_brokers"`
+ LoggerKafkaCompression string `json:"logger_kafka_compression"`
+ LoggerKafkaTopic string `json:"logger_kafka_topic"`
+ LoggerMinStatus int32 `json:"logger_min_status"`
+ LoggerMinStderr int32 `json:"logger_min_stderr"`
+ LoggerMode string `json:"logger_mode"`
+ LoggerNumerics bool `json:"logger_numerics"`
+ LoggerPath string `json:"logger_path"`
+ LoggerPlugin string `json:"logger_plugin"`
+ LoggerRotate bool `json:"logger_rotate"`
+ LoggerRotateMaxFiles uint64 `json:"logger_rotate_max_files"`
+ LoggerRotateSize uint64 `json:"logger_rotate_size"`
+ LoggerSnapshotEventType bool `json:"logger_snapshot_event_type"`
+ LoggerStderr bool `json:"logger_stderr"`
+ LoggerSyslogFacility int32 `json:"logger_syslog_facility"`
+ LoggerSyslogPrependCee bool `json:"logger_syslog_prepend_cee"`
+ LoggerTlsBackoffMax uint64 `json:"logger_tls_backoff_max"`
+ LoggerTlsCompress bool `json:"logger_tls_compress"`
+ LoggerTlsEndpoint string `json:"logger_tls_endpoint"`
+ LoggerTlsMaxLines uint64 `json:"logger_tls_max_lines"`
+ LoggerTlsMaxLinesize uint64 `json:"logger_tls_max_linesize"`
+ LoggerTlsPeriod uint64 `json:"logger_tls_period"`
+ Logtostderr bool `json:"logtostderr"`
+ Nullvalue string `json:"nullvalue"`
+ NumericMonitoringFilesystemPath string `json:"numeric_monitoring_filesystem_path"`
+ NumericMonitoringPlugins string `json:"numeric_monitoring_plugins"`
+ NumericMonitoringPreAggregationTime uint64 `json:"numeric_monitoring_pre_aggregation_time"`
+ PackDelimiter string `json:"pack_delimiter"`
+ PackRefreshInterval uint64 `json:"pack_refresh_interval"`
+ Pidfile string `json:"pidfile"`
+ ProxyHostname string `json:"proxy_hostname"`
+ ReadMax uint64 `json:"read_max"`
+ ScheduleDefaultInterval uint64 `json:"schedule_default_interval"`
+ ScheduleEpoch uint64 `json:"schedule_epoch"`
+ ScheduleLognames bool `json:"schedule_lognames"`
+ ScheduleMaxDrift uint64 `json:"schedule_max_drift"`
+ ScheduleReload uint64 `json:"schedule_reload"`
+ ScheduleSplayPercent uint64 `json:"schedule_splay_percent"`
+ ScheduleTimeout uint64 `json:"schedule_timeout"`
+ SpecifiedIdentifier string `json:"specified_identifier"`
+ Stderrthreshold int32 `json:"stderrthreshold"`
+ TableDelay uint64 `json:"table_delay"`
+ ThriftStringSizeLimit int32 `json:"thrift_string_size_limit"`
+ ThriftTimeout uint32 `json:"thrift_timeout"`
+ ThriftVerbose bool `json:"thrift_verbose"`
+ TlsClientCert string `json:"tls_client_cert"`
+ TlsClientKey string `json:"tls_client_key"`
+ TlsDisableStatusLog bool `json:"tls_disable_status_log"`
+ TlsEnrollMaxAttempts uint64 `json:"tls_enroll_max_attempts"`
+ TlsEnrollMaxInterval uint64 `json:"tls_enroll_max_interval"`
+ TlsHostname string `json:"tls_hostname"`
+ TlsServerCerts string `json:"tls_server_certs"`
+ TlsSessionReuse bool `json:"tls_session_reuse"`
+ TlsSessionTimeout uint32 `json:"tls_session_timeout"`
+ Uninstall bool `json:"uninstall"`
+ Verbose bool `json:"verbose"`
+ WatchdogDelay uint64 `json:"watchdog_delay"`
+ WatchdogForcedShutdownDelay uint64 `json:"watchdog_forced_shutdown_delay"`
+ WatchdogLatencyLimit uint64 `json:"watchdog_latency_limit"`
+ WatchdogLevel int32 `json:"watchdog_level"`
+ WatchdogMemoryLimit uint64 `json:"watchdog_memory_limit"`
+ WatchdogUtilizationLimit uint64 `json:"watchdog_utilization_limit"`
+ YaraDelay uint32 `json:"yara_delay"`
+
+ // embed the os-specific structs
+ OsqueryCommandLineFlagsLinux
+ OsqueryCommandLineFlagsWindows
+ OsqueryCommandLineFlagsMacOS
+ OsqueryCommandLineFlagsHidden
+}
diff --git a/server/fleet/agent_options_test.go b/server/fleet/agent_options_test.go
index 103901d7a1..0e199298b6 100644
--- a/server/fleet/agent_options_test.go
+++ b/server/fleet/agent_options_test.go
@@ -184,6 +184,16 @@ func TestValidateAgentOptions(t *testing.T) {
"orbit": "foobar"
}
}`, true, ``},
+ {"setting osquery 5.12.X flag in config.options and command_line_flags", `{
+ "config": {
+ "options": {
+ "logger_tls_backoff_max": 100
+ }
+ },
+ "command_line_flags": {
+ "logger_tls_backoff_max": 200
+ }
+ }`, true, ``},
}
for _, c := range cases {
diff --git a/server/fleet/hosts.go b/server/fleet/hosts.go
index df445951b6..54b81bc824 100644
--- a/server/fleet/hosts.go
+++ b/server/fleet/hosts.go
@@ -1235,6 +1235,7 @@ type OSVersion struct {
type HostDetailOptions struct {
IncludeCVEScores bool
IncludePolicies bool
+ ExcludeSoftware bool
}
// EnrollHostLimiter defines the methods to support enforcement of enrolled
diff --git a/server/fleet/scripts.go b/server/fleet/scripts.go
index 53b34a1387..eb2d1e52d5 100644
--- a/server/fleet/scripts.go
+++ b/server/fleet/scripts.go
@@ -249,6 +249,12 @@ type HostScriptResult struct {
// execution. It is otherwise not part of the host_script_results table and
// not returned as part of the resulting JSON.
Hostname string `json:"-" db:"-"`
+
+ // HostDeletedAt indicates if the results are associated with a deleted host.
+ // This supports the soft-delete feature for script results so that the
+ // results can still be returned to see activity details after the host got
+ // deleted.
+ HostDeletedAt *time.Time `json:"-" db:"host_deleted_at"`
}
func (hsr HostScriptResult) AuthzType() string {
diff --git a/server/fleet/teams.go b/server/fleet/teams.go
index 984d295343..ebdf8a7eb1 100644
--- a/server/fleet/teams.go
+++ b/server/fleet/teams.go
@@ -419,7 +419,7 @@ type TeamSpec struct {
// set to the agent options JSON object.
AgentOptions json.RawMessage `json:"agent_options,omitempty"` // marshals as "null" if omitempty is not set
HostExpirySettings *HostExpirySettings `json:"host_expiry_settings,omitempty"`
- Secrets []EnrollSecret `json:"secrets,omitempty"`
+ Secrets *[]EnrollSecret `json:"secrets,omitempty"`
Features *json.RawMessage `json:"features"`
MDM TeamSpecMDM `json:"mdm"`
Scripts optjson.Slice[string] `json:"scripts"`
@@ -486,7 +486,7 @@ func TeamSpecFromTeam(t *Team) (*TeamSpec, error) {
Name: t.Name,
AgentOptions: agentOptions,
Features: &featuresJSON,
- Secrets: secrets,
+ Secrets: &secrets,
MDM: mdmSpec,
HostExpirySettings: &t.Config.HostExpirySettings,
WebhookSettings: webhookSettings,
diff --git a/server/mdm/nanomdm/http/api/api.go b/server/mdm/nanomdm/http/api/api.go
index 5649470f85..6cdcda6893 100644
--- a/server/mdm/nanomdm/http/api/api.go
+++ b/server/mdm/nanomdm/http/api/api.go
@@ -145,6 +145,11 @@ func RawCommandEnqueueHandler(enqueuer storage.CommandEnqueuer, pusher push.Push
b, err := mdmhttp.ReadAllAndReplaceBody(r)
if err != nil {
logger.Info("msg", "reading body", "err", err)
+ var toErr interface{ Timeout() bool }
+ if errors.As(err, &toErr) && toErr.Timeout() {
+ http.Error(w, http.StatusText(http.StatusRequestTimeout), http.StatusRequestTimeout)
+ return
+ }
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
return
}
@@ -299,6 +304,11 @@ func StorePushCertHandler(storage storage.PushCertStore, logger log.Logger) http
b, err := mdmhttp.ReadAllAndReplaceBody(r)
if err != nil {
logger.Info("msg", "reading body", "err", err)
+ var toErr interface{ Timeout() bool }
+ if errors.As(err, &toErr) && toErr.Timeout() {
+ http.Error(w, http.StatusText(http.StatusRequestTimeout), http.StatusRequestTimeout)
+ return
+ }
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
return
}
diff --git a/server/mdm/nanomdm/http/http_test.go b/server/mdm/nanomdm/http/http_test.go
new file mode 100644
index 0000000000..26fe47dc4f
--- /dev/null
+++ b/server/mdm/nanomdm/http/http_test.go
@@ -0,0 +1,54 @@
+package http
+
+import (
+ "errors"
+ "io"
+ "net/http"
+ "net/http/httptest"
+ "testing"
+ "time"
+
+ "github.com/stretchr/testify/require"
+)
+
+func TestHTTPServerTimeoutError(t *testing.T) {
+ // ensure that a read timeout error is properly detected
+ srv := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+ code := http.StatusOK
+ if _, err := io.ReadAll(r.Body); err != nil {
+ var toErr interface{ Timeout() bool }
+ if errors.As(err, &toErr) && toErr.Timeout() {
+ code = http.StatusRequestTimeout
+ } else {
+ code = http.StatusInternalServerError
+ }
+ }
+ w.WriteHeader(code)
+ }))
+
+ srv.Config.ReadTimeout = time.Second
+ srv.Start()
+ defer srv.Close()
+
+ req, err := http.NewRequest("POST", srv.URL, slowReader{b: []byte("slowly send this")})
+ require.NoError(t, err)
+ res, err := http.DefaultClient.Do(req)
+ require.NoError(t, err)
+ code := res.StatusCode
+ require.Equal(t, http.StatusRequestTimeout, code)
+}
+
+type slowReader struct {
+ b []byte
+}
+
+func (s slowReader) Read(p []byte) (n int, err error) {
+ if len(s.b) == 0 {
+ return 0, io.EOF
+ }
+
+ time.Sleep(200 * time.Millisecond)
+ n = copy(p, s.b[:len(s.b)/2])
+ s.b = s.b[n:]
+ return n, nil
+}
diff --git a/server/mdm/nanomdm/http/mdm/mdm.go b/server/mdm/nanomdm/http/mdm/mdm.go
index a3bd21796a..bad75bce2b 100644
--- a/server/mdm/nanomdm/http/mdm/mdm.go
+++ b/server/mdm/nanomdm/http/mdm/mdm.go
@@ -32,6 +32,11 @@ func CheckinHandler(svc service.Checkin, logger log.Logger) http.HandlerFunc {
bodyBytes, err := mdmhttp.ReadAllAndReplaceBody(r)
if err != nil {
logger.Info("msg", "reading body", "err", err)
+ var toErr interface{ Timeout() bool }
+ if errors.As(err, &toErr) && toErr.Timeout() {
+ http.Error(w, http.StatusText(http.StatusRequestTimeout), http.StatusRequestTimeout)
+ return
+ }
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
return
}
@@ -56,6 +61,11 @@ func CommandAndReportResultsHandler(svc service.CommandAndReportResults, logger
bodyBytes, err := mdmhttp.ReadAllAndReplaceBody(r)
if err != nil {
logger.Info("msg", "reading body", "err", err)
+ var toErr interface{ Timeout() bool }
+ if errors.As(err, &toErr) && toErr.Timeout() {
+ http.Error(w, http.StatusText(http.StatusRequestTimeout), http.StatusRequestTimeout)
+ return
+ }
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
return
}
diff --git a/server/mdm/nanomdm/http/mdm/mdm_cert.go b/server/mdm/nanomdm/http/mdm/mdm_cert.go
index 69ef246ef6..93308ec841 100644
--- a/server/mdm/nanomdm/http/mdm/mdm_cert.go
+++ b/server/mdm/nanomdm/http/mdm/mdm_cert.go
@@ -3,6 +3,7 @@ package mdm
import (
"context"
"crypto/x509"
+ "errors"
"net/http"
"net/url"
@@ -84,6 +85,11 @@ func CertExtractMdmSignatureMiddleware(next http.Handler, logger log.Logger) htt
b, err := mdmhttp.ReadAllAndReplaceBody(r)
if err != nil {
logger.Info("msg", "reading body", "err", err)
+ var toErr interface{ Timeout() bool }
+ if errors.As(err, &toErr) && toErr.Timeout() {
+ http.Error(w, http.StatusText(http.StatusRequestTimeout), http.StatusRequestTimeout)
+ return
+ }
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
return
}
diff --git a/server/service/hosts.go b/server/service/hosts.go
index 530c7fd829..2841ba8b80 100644
--- a/server/service/hosts.go
+++ b/server/service/hosts.go
@@ -479,7 +479,8 @@ func (svc *Service) SearchHosts(ctx context.Context, matchQuery string, queryID
/////////////////////////////////////////////////////////////////////////////////
type getHostRequest struct {
- ID uint `url:"id"`
+ ID uint `url:"id"`
+ ExcludeSoftware bool `query:"exclude_software,optional"`
}
type getHostResponse struct {
@@ -493,7 +494,8 @@ func getHostEndpoint(ctx context.Context, request interface{}, svc fleet.Service
req := request.(*getHostRequest)
opts := fleet.HostDetailOptions{
IncludeCVEScores: false,
- IncludePolicies: true, // intentionally true to preserve existing behavior
+ IncludePolicies: true, // intentionally true to preserve existing behavior,
+ ExcludeSoftware: req.ExcludeSoftware,
}
host, err := svc.GetHost(ctx, req.ID, opts)
if err != nil {
@@ -1009,8 +1011,10 @@ func (svc *Service) RefetchHost(ctx context.Context, id uint) error {
}
func (svc *Service) getHostDetails(ctx context.Context, host *fleet.Host, opts fleet.HostDetailOptions) (*fleet.HostDetail, error) {
- if err := svc.ds.LoadHostSoftware(ctx, host, opts.IncludeCVEScores); err != nil {
- return nil, ctxerr.Wrap(ctx, err, "load host software")
+ if !opts.ExcludeSoftware {
+ if err := svc.ds.LoadHostSoftware(ctx, host, opts.IncludeCVEScores); err != nil {
+ return nil, ctxerr.Wrap(ctx, err, "load host software")
+ }
}
labels, err := svc.ds.ListLabelsForHost(ctx, host.ID)
diff --git a/server/service/integration_core_test.go b/server/service/integration_core_test.go
index a5af5c5d36..a0c0fa916b 100644
--- a/server/service/integration_core_test.go
+++ b/server/service/integration_core_test.go
@@ -6012,7 +6012,7 @@ func (s *integrationTestSuite) TestPremiumEndpointsWithoutLicense() {
// apply team specs
var specResp applyTeamSpecsResponse
- teamSpecs := applyTeamSpecsRequest{Specs: []*fleet.TeamSpec{{Name: "newteam", Secrets: []fleet.EnrollSecret{{Secret: "ABC"}}}}}
+ teamSpecs := applyTeamSpecsRequest{Specs: []*fleet.TeamSpec{{Name: "newteam", Secrets: &[]fleet.EnrollSecret{{Secret: "ABC"}}}}}
s.DoJSON("POST", "/api/latest/fleet/spec/teams", teamSpecs, http.StatusPaymentRequired, &specResp)
// modify team agent options
@@ -6211,6 +6211,9 @@ func (s *integrationTestSuite) TestTeamPoliciesTeamNotExists() {
teamPoliciesResponse := listTeamPoliciesResponse{}
s.DoJSON("GET", fmt.Sprintf("/api/latest/fleet/teams/%d/policies", 9999999), nil, http.StatusNotFound, &teamPoliciesResponse)
require.Len(t, teamPoliciesResponse.Policies, 0)
+
+ deleteTeamPoliciesResponse := deleteTeamPoliciesResponse{}
+ s.DoJSON("POST", fmt.Sprintf("/api/latest/fleet/teams/%d/policies/delete", 9999999), deleteTeamPoliciesRequest{IDs: []uint{1, 1000}}, http.StatusNotFound, &deleteTeamPoliciesResponse)
}
func (s *integrationTestSuite) TestSessionInfo() {
@@ -7908,6 +7911,12 @@ func (s *integrationTestSuite) TestGetHostSoftwareUpdatedAt() {
require.Equal(t, host.ID, getHostResp.Host.ID)
require.Len(t, getHostResp.Host.Software, len(software))
require.Greater(t, getHostResp.Host.SoftwareUpdatedAt, getHostResp.Host.CreatedAt)
+
+ getHostResp = getHostResponse{}
+ s.DoJSON("GET", fmt.Sprintf("/api/latest/fleet/hosts/%d", host.ID), nil, http.StatusOK, &getHostResp, "exclude_software", "true")
+ require.Equal(t, host.ID, getHostResp.Host.ID)
+ require.Empty(t, getHostResp.Host.Software)
+ require.Greater(t, getHostResp.Host.SoftwareUpdatedAt, getHostResp.Host.CreatedAt)
}
func (s *integrationTestSuite) TestHostsReportDownload() {
@@ -11262,8 +11271,9 @@ func (s *integrationTestSuite) TestListHostUpcomingActivities() {
endTime = mysql.SetOrderedCreatedAtTimestamps(t, s.ds, endTime, "host_software_installs", "execution_id", h1Foo)
mysql.SetOrderedCreatedAtTimestamps(t, s.ds, endTime, "host_script_results", "execution_id", h1C, h1D, h1E)
- // modify the timestamp h1A and h1B to simulate an script that has
- // been pending for a long time
+ // modify the timestamp h1A and h1B to simulate an script that has been
+ // pending for a long time (h1A is a sync request, so it will be ignored for
+ // upcoming activities)
mysql.ExecAdhocSQL(t, s.ds, func(tx sqlx.ExtContext) error {
_, err := tx.ExecContext(ctx, "UPDATE host_script_results SET created_at = ? WHERE execution_id IN (?, ?)", time.Now().Add(-24*time.Hour), h1A, h1B)
return err
@@ -11320,7 +11330,7 @@ func (s *integrationTestSuite) TestListHostUpcomingActivities() {
queryArgs := c.queries
s.DoJSON("GET", fmt.Sprintf("/api/latest/fleet/hosts/%d/activities/upcoming", host1.ID), nil, http.StatusOK, &listResp, queryArgs...)
- require.Equal(t, uint(6), listResp.Count)
+ require.Equal(t, uint(5), listResp.Count)
require.Equal(t, len(c.wantExecs), len(listResp.Activities))
require.Equal(t, c.wantMeta, listResp.Meta)
diff --git a/server/service/integration_enterprise_test.go b/server/service/integration_enterprise_test.go
index b1dba17e40..dff33643a0 100644
--- a/server/service/integration_enterprise_test.go
+++ b/server/service/integration_enterprise_test.go
@@ -25,6 +25,7 @@ import (
"github.com/fleetdm/fleet/v4/ee/server/calendar"
"github.com/fleetdm/fleet/v4/pkg/optjson"
+ "github.com/fleetdm/fleet/v4/server/config"
"github.com/fleetdm/fleet/v4/server/cron"
"github.com/fleetdm/fleet/v4/server/datastore/mysql"
"github.com/fleetdm/fleet/v4/server/datastore/redis/redistest"
@@ -84,7 +85,9 @@ func (s *integrationEnterpriseTestSuite) SetupSuite() {
if os.Getenv("FLEET_INTEGRATION_TESTS_DISABLE_LOG") != "" {
cronLog = kitlog.NewNopLogger()
}
- calendarSchedule, err = cron.NewCalendarSchedule(ctx, s.T().Name(), s.ds, 24*time.Hour, cronLog)
+ calendarSchedule, err = cron.NewCalendarSchedule(
+ ctx, s.T().Name(), s.ds, config.CalendarConfig{Periodicity: 24 * time.Hour}, cronLog,
+ )
return calendarSchedule, err
}
},
@@ -9420,6 +9423,9 @@ func (s *integrationEnterpriseTestSuite) TestSoftwareInstallerUploadDownloadAndD
// check activity
s.lastActivityOfTypeMatches(fleet.ActivityTypeDeletedSoftware{}.ActivityName(), fmt.Sprintf(`{"software_title": "ruby", "software_package": "ruby.deb", "team_name": "%s", "team_id": %d, "self_service": true}`, createTeamResp.Team.Name, createTeamResp.Team.ID), 0)
+
+ // download the installer, not found anymore
+ s.Do("GET", fmt.Sprintf("/api/latest/fleet/software/%d/package?alt=media", titleID), nil, http.StatusNotFound, "team_id", fmt.Sprintf("%d", *payload.TeamID))
})
}
@@ -10139,6 +10145,109 @@ func (s *integrationEnterpriseTestSuite) TestHostSoftwareInstallResult() {
s.lastActivityOfTypeMatches(wantAct.ActivityName(), string(jsonMustMarshal(t, wantAct)), lastActID)
}
+func (s *integrationEnterpriseTestSuite) TestHostScriptSoftDelete() {
+ t := s.T()
+ ctx := context.Background()
+
+ // create a host and request a script execution
+ tm, err := s.ds.NewTeam(ctx, &fleet.Team{Name: "host_soft_delete_team"})
+ require.NoError(t, err)
+ host := createOrbitEnrolledHost(t, "linux", "", s.ds)
+ err = s.ds.AddHostsToTeam(ctx, &tm.ID, []uint{host.ID})
+ require.NoError(t, err)
+
+ // create an anonymous script execution request
+ var runResp runScriptResponse
+ s.DoJSON("POST", "/api/latest/fleet/scripts/run", fleet.HostScriptRequestPayload{HostID: host.ID, ScriptContents: "echo"}, http.StatusAccepted, &runResp)
+ scriptExecID := runResp.ExecutionID
+
+ // post a script result so that the (past) activity is created
+ s.Do("POST", "/api/fleet/orbit/scripts/result",
+ json.RawMessage(fmt.Sprintf(`{"orbit_node_key": %q, "execution_id": %q, "exit_code": 0, "output": "ok"}`, *host.OrbitNodeKey, scriptExecID)),
+ http.StatusOK)
+ s.lastActivityOfTypeMatches(
+ fleet.ActivityTypeRanScript{}.ActivityName(),
+ fmt.Sprintf(
+ `{"host_id": %d, "host_display_name": %q, "script_name": "", "script_execution_id": %q, "async": true}`,
+ host.ID, host.DisplayName(), scriptExecID), 0)
+
+ // create a saved script execution request
+ var newScriptResp createScriptResponse
+ body, headers := generateNewScriptMultipartRequest(t,
+ "script1.sh", []byte(`echo "hello"`), s.token, map[string][]string{"team_id": {strconv.Itoa(int(tm.ID))}})
+ res := s.DoRawWithHeaders("POST", "/api/latest/fleet/scripts", body.Bytes(), http.StatusOK, headers)
+ err = json.NewDecoder(res.Body).Decode(&newScriptResp)
+ require.NoError(t, err)
+ require.NotZero(t, newScriptResp.ScriptID)
+ savedScriptID := newScriptResp.ScriptID
+
+ s.DoJSON("POST", "/api/latest/fleet/scripts/run", fleet.HostScriptRequestPayload{HostID: host.ID, ScriptID: &savedScriptID}, http.StatusAccepted, &runResp)
+ savedScriptExecID := runResp.ExecutionID
+
+ // post a script result so that the (past) activity is created
+ s.Do("POST", "/api/fleet/orbit/scripts/result",
+ json.RawMessage(fmt.Sprintf(`{"orbit_node_key": %q, "execution_id": %q, "exit_code": 0, "output": "saved"}`, *host.OrbitNodeKey, savedScriptExecID)),
+ http.StatusOK)
+ s.lastActivityOfTypeMatches(
+ fleet.ActivityTypeRanScript{}.ActivityName(),
+ fmt.Sprintf(`{"host_id": %d, "host_display_name": %q, "script_name": "script1.sh", "script_execution_id": %q, "async": true}`,
+ host.ID, host.DisplayName(), savedScriptExecID), 0)
+
+ // get the anoymous script result details
+ var scriptRes getScriptResultResponse
+ s.DoJSON("GET", "/api/latest/fleet/scripts/results/"+scriptExecID, nil, http.StatusOK, &scriptRes)
+ require.Equal(t, scriptExecID, scriptRes.ExecutionID)
+ require.Equal(t, host.ID, scriptRes.HostID)
+ require.Equal(t, "ok", scriptRes.Output)
+ require.NotNil(t, scriptRes.ExitCode)
+ require.EqualValues(t, 0, *scriptRes.ExitCode)
+
+ // get the saved script result details
+ scriptRes = getScriptResultResponse{}
+ s.DoJSON("GET", "/api/latest/fleet/scripts/results/"+savedScriptExecID, nil, http.StatusOK, &scriptRes)
+ require.Equal(t, savedScriptExecID, scriptRes.ExecutionID)
+ require.Equal(t, host.ID, scriptRes.HostID)
+ require.Equal(t, "saved", scriptRes.Output)
+ require.NotNil(t, scriptRes.ExitCode)
+ require.EqualValues(t, 0, *scriptRes.ExitCode)
+
+ // delete the host
+ var deleteResp deleteHostResponse
+ s.DoJSON("DELETE", fmt.Sprintf("/api/latest/fleet/hosts/%d", host.ID), nil, http.StatusOK, &deleteResp)
+
+ // get the anonymous script result details, still works
+ scriptRes = getScriptResultResponse{}
+ s.DoJSON("GET", "/api/latest/fleet/scripts/results/"+scriptExecID, nil, http.StatusOK, &scriptRes)
+ require.Equal(t, scriptExecID, scriptRes.ExecutionID)
+ require.Equal(t, host.ID, scriptRes.HostID)
+ require.Equal(t, "ok", scriptRes.Output)
+ require.NotNil(t, scriptRes.ExitCode)
+ require.EqualValues(t, 0, *scriptRes.ExitCode)
+
+ // get the saved script result details, still works
+ scriptRes = getScriptResultResponse{}
+ s.DoJSON("GET", "/api/latest/fleet/scripts/results/"+savedScriptExecID, nil, http.StatusOK, &scriptRes)
+ require.Equal(t, savedScriptExecID, scriptRes.ExecutionID)
+ require.Equal(t, host.ID, scriptRes.HostID)
+ require.Equal(t, "saved", scriptRes.Output)
+ require.NotNil(t, scriptRes.ExitCode)
+ require.EqualValues(t, 0, *scriptRes.ExitCode)
+
+ // delete the named script
+ s.Do("DELETE", fmt.Sprintf("/api/latest/fleet/scripts/%d", savedScriptID), nil, http.StatusNoContent)
+
+ // get the saved script result details, still works because the saved script
+ // is a "soft-reference", when deleted the results become essentially results
+ // for an anonymous script (i.e. the script_id FK is "ON DELETE SET NULL").
+ scriptRes = getScriptResultResponse{}
+ s.DoJSON("GET", "/api/latest/fleet/scripts/results/"+savedScriptExecID, nil, http.StatusOK, &scriptRes)
+ require.Equal(t, savedScriptExecID, scriptRes.ExecutionID)
+ require.Equal(t, host.ID, scriptRes.HostID)
+ require.Equal(t, "saved", scriptRes.Output)
+ require.NotNil(t, scriptRes.ExitCode)
+ require.EqualValues(t, 0, *scriptRes.ExitCode)
+}
+
func (s *integrationEnterpriseTestSuite) uploadSoftwareInstaller(payload *fleet.UploadSoftwareInstallerPayload, expectedStatus int, expectedError string) {
t := s.T()
openFile := func(name string) *os.File {
diff --git a/server/service/integration_mdm_lifecycle_test.go b/server/service/integration_mdm_lifecycle_test.go
index 7bd789f56d..75fad197f0 100644
--- a/server/service/integration_mdm_lifecycle_test.go
+++ b/server/service/integration_mdm_lifecycle_test.go
@@ -134,12 +134,10 @@ func (s *integrationMDMTestSuite) TestTurnOnLifecycleEventsApple() {
dupeClient := mdmtest.NewTestMDMClientAppleDirect(
mdmtest.AppleEnrollInfo{
-
SCEPChallenge: s.scepChallenge,
SCEPURL: s.server.URL + apple_mdm.SCEPPath,
MDMURL: s.server.URL + apple_mdm.MDMPath,
- },
- )
+ }, "MacBookPro16,1")
dupeClient.UUID = device.UUID
dupeClient.SerialNumber = device.SerialNumber
dupeClient.Model = device.Model
@@ -159,12 +157,10 @@ func (s *integrationMDMTestSuite) TestTurnOnLifecycleEventsApple() {
dupeClient := mdmtest.NewTestMDMClientAppleDirect(
mdmtest.AppleEnrollInfo{
-
SCEPChallenge: s.scepChallenge,
SCEPURL: s.server.URL + apple_mdm.SCEPPath,
MDMURL: s.server.URL + apple_mdm.MDMPath,
- },
- )
+ }, "MacBookPro16,1")
dupeClient.UUID = device.UUID
dupeClient.SerialNumber = device.SerialNumber
dupeClient.Model = device.Model
diff --git a/server/service/integration_mdm_profiles_test.go b/server/service/integration_mdm_profiles_test.go
index 8a46f3ec95..6a3a28104a 100644
--- a/server/service/integration_mdm_profiles_test.go
+++ b/server/service/integration_mdm_profiles_test.go
@@ -1898,7 +1898,7 @@ func (s *integrationMDMTestSuite) TestHostMDMAppleProfilesStatus() {
SCEPChallenge: s.scepChallenge,
SCEPURL: s.server.URL + apple_mdm.SCEPPath,
MDMURL: s.server.URL + apple_mdm.MDMPath,
- })
+ }, "MacBookPro16,1")
// enroll the device with orbit
var resp EnrollOrbitResponse
diff --git a/server/service/integration_mdm_test.go b/server/service/integration_mdm_test.go
index 4463ab2cda..e61a5e2ad7 100644
--- a/server/service/integration_mdm_test.go
+++ b/server/service/integration_mdm_test.go
@@ -71,7 +71,7 @@ import (
func TestIntegrationsMDM(t *testing.T) {
testingSuite := new(integrationMDMTestSuite)
- testingSuite.s = &testingSuite.Suite
+ testingSuite.withServer.s = &testingSuite.Suite
suite.Run(t, testingSuite)
}
@@ -425,7 +425,7 @@ func (s *integrationMDMTestSuite) TestGetBootstrapToken() {
SCEPChallenge: s.scepChallenge,
SCEPURL: s.server.URL + apple_mdm.SCEPPath,
MDMURL: s.server.URL + apple_mdm.MDMPath,
- })
+ }, "MacBookPro16,1")
err := mdmDevice.Enroll()
require.NoError(t, err)
@@ -776,13 +776,13 @@ func (s *integrationMDMTestSuite) TestAppleMDMDeviceEnrollment() {
SCEPURL: s.server.URL + apple_mdm.SCEPPath,
MDMURL: s.server.URL + apple_mdm.MDMPath,
}
- mdmDeviceA := mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo)
+ mdmDeviceA := mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo, "MacBookPro16,1")
err := mdmDeviceA.Enroll()
require.NoError(t, err)
s.lastActivityOfTypeMatches(fleet.ActivityTypeMDMEnrolled{}.ActivityName(),
fmt.Sprintf(`{"host_serial": "%s", "host_display_name": "%s (%s)", "installed_from_dep": false, "mdm_platform": "apple"}`, mdmDeviceA.SerialNumber, mdmDeviceA.Model, mdmDeviceA.SerialNumber), 0)
- mdmDeviceB := mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo)
+ mdmDeviceB := mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo, "MacBookPro16,1")
err = mdmDeviceB.Enroll()
require.NoError(t, err)
s.lastActivityOfTypeMatches(fleet.ActivityTypeMDMEnrolled{}.ActivityName(),
@@ -873,7 +873,7 @@ func (s *integrationMDMTestSuite) TestDeviceMultipleAuthMessages() {
SCEPChallenge: s.scepChallenge,
SCEPURL: s.server.URL + apple_mdm.SCEPPath,
MDMURL: s.server.URL + apple_mdm.MDMPath,
- })
+ }, "MacBookPro16,1")
err := mdmDevice.Enroll()
require.NoError(t, err)
@@ -1008,7 +1008,7 @@ func (s *integrationMDMTestSuite) TestMDMAppleUnenroll() {
SCEPChallenge: s.scepChallenge,
SCEPURL: s.server.URL + apple_mdm.SCEPPath,
MDMURL: s.server.URL + apple_mdm.MDMPath,
- })
+ }, "MacBookPro16,1")
err := mdmDevice.Enroll()
require.NoError(t, err)
@@ -2316,7 +2316,7 @@ func (s *integrationMDMTestSuite) TestFleetdConfiguration() {
// create an enroll secret for the team
teamSpecs := applyTeamSpecsRequest{Specs: []*fleet.TeamSpec{{
Name: tm.Name,
- Secrets: []fleet.EnrollSecret{{Secret: t.Name() + "team-secret"}},
+ Secrets: &[]fleet.EnrollSecret{{Secret: t.Name() + "team-secret"}},
}}}
s.Do("POST", "/api/latest/fleet/spec/teams", teamSpecs, http.StatusOK)
@@ -2341,7 +2341,7 @@ func (s *integrationMDMTestSuite) TestEnqueueMDMCommand() {
SCEPChallenge: s.scepChallenge,
SCEPURL: s.server.URL + apple_mdm.SCEPPath,
MDMURL: s.server.URL + apple_mdm.MDMPath,
- })
+ }, "MacBookPro16,1")
err := mdmDevice.Enroll()
require.NoError(t, err)
@@ -2754,24 +2754,24 @@ func (s *integrationMDMTestSuite) TestBootstrapPackageStatus() {
MDMURL: s.server.URL + apple_mdm.MDMPath,
}
noTeamDevices := []deviceWithResponse{
- {"Acknowledge", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo)},
- {"Acknowledge", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo)},
- {"Acknowledge", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo)},
- {"Error", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo)},
- {"Offline", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo)},
- {"Offline", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo)},
- {"Pending", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo)},
- {"Pending", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo)},
+ {"Acknowledge", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo, "MacBookPro16,1")},
+ {"Acknowledge", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo, "MacBookPro16,1")},
+ {"Acknowledge", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo, "MacBookPro16,1")},
+ {"Error", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo, "MacBookPro16,1")},
+ {"Offline", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo, "MacBookPro16,1")},
+ {"Offline", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo, "MacBookPro16,1")},
+ {"Pending", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo, "MacBookPro16,1")},
+ {"Pending", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo, "MacBookPro16,1")},
}
teamDevices := []deviceWithResponse{
- {"Acknowledge", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo)},
- {"Acknowledge", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo)},
- {"Error", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo)},
- {"Error", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo)},
- {"Error", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo)},
- {"Offline", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo)},
- {"Pending", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo)},
+ {"Acknowledge", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo, "MacBookPro16,1")},
+ {"Acknowledge", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo, "MacBookPro16,1")},
+ {"Error", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo, "MacBookPro16,1")},
+ {"Error", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo, "MacBookPro16,1")},
+ {"Error", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo, "MacBookPro16,1")},
+ {"Offline", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo, "MacBookPro16,1")},
+ {"Pending", mdmtest.NewTestMDMClientAppleDirect(mdmEnrollInfo, "MacBookPro16,1")},
}
expectedSerialsByTeamAndStatus := make(map[uint]map[fleet.MDMBootstrapPackageStatus][]string)
@@ -4375,7 +4375,7 @@ func (s *integrationMDMTestSuite) TestSSO() {
mdmDevice := mdmtest.NewTestMDMClientAppleDirect(mdmtest.AppleEnrollInfo{
SCEPChallenge: s.scepChallenge,
- })
+ }, "MacBookPro16,1")
var lastSubmittedProfile *godep.Profile
s.mockDEPResponse(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
@@ -5292,7 +5292,7 @@ func (s *integrationMDMTestSuite) TestOrbitConfigNudgeSettings() {
SCEPChallenge: s.scepChallenge,
SCEPURL: s.server.URL + apple_mdm.SCEPPath,
MDMURL: s.server.URL + apple_mdm.MDMPath,
- })
+ }, "MacBookPro16,1")
mdmDevice.SerialNumber = h.HardwareSerial
mdmDevice.UUID = h.UUID
err = mdmDevice.Enroll()
@@ -5349,7 +5349,7 @@ func (s *integrationMDMTestSuite) TestOrbitConfigNudgeSettings() {
SCEPChallenge: s.scepChallenge,
SCEPURL: s.server.URL + apple_mdm.SCEPPath,
MDMURL: s.server.URL + apple_mdm.MDMPath,
- })
+ }, "MacBookPro16,1")
mdmDevice.SerialNumber = h2.HardwareSerial
mdmDevice.UUID = h2.UUID
err = mdmDevice.Enroll()
@@ -5372,7 +5372,7 @@ func (s *integrationMDMTestSuite) TestOrbitConfigNudgeSettings() {
SCEPChallenge: s.scepChallenge,
SCEPURL: s.server.URL + apple_mdm.SCEPPath,
MDMURL: s.server.URL + apple_mdm.MDMPath,
- })
+ }, "MacBookPro16,1")
mdmDevice.SerialNumber = h3.HardwareSerial
mdmDevice.UUID = h3.UUID
err = mdmDevice.Enroll()
@@ -5393,7 +5393,7 @@ func (s *integrationMDMTestSuite) TestOrbitConfigNudgeSettings() {
SCEPChallenge: s.scepChallenge,
SCEPURL: s.server.URL + apple_mdm.SCEPPath,
MDMURL: s.server.URL + apple_mdm.MDMPath,
- })
+ }, "MacBookPro16,1")
mdmDevice.SerialNumber = h4.HardwareSerial
mdmDevice.UUID = h4.UUID
err = mdmDevice.Enroll()
@@ -7797,7 +7797,7 @@ func (s *integrationMDMTestSuite) TestManualEnrollmentCommands() {
SCEPChallenge: s.scepChallenge,
SCEPURL: s.server.URL + apple_mdm.SCEPPath,
MDMURL: s.server.URL + apple_mdm.MDMPath,
- })
+ }, "MacBookPro16,1")
err := mdmDevice.Enroll()
require.NoError(t, err)
s.runWorker()
diff --git a/server/service/osquery_test.go b/server/service/osquery_test.go
index 9aa9ce246f..878e0ce8e0 100644
--- a/server/service/osquery_test.go
+++ b/server/service/osquery_test.go
@@ -1061,6 +1061,7 @@ func verifyDiscovery(t *testing.T, queries, discovery map[string]string) {
hostDetailQueryPrefix + "kubequery_info": {},
hostDetailQueryPrefix + "orbit_info": {},
hostDetailQueryPrefix + "software_vscode_extensions": {},
+ hostDetailQueryPrefix + "software_macos_firefox": {},
}
for name := range queries {
require.NotEmpty(t, discovery[name])
diff --git a/server/service/osquery_utils/queries.go b/server/service/osquery_utils/queries.go
index 6c8d30585c..889b27c619 100644
--- a/server/service/osquery_utils/queries.go
+++ b/server/service/osquery_utils/queries.go
@@ -755,7 +755,7 @@ var mdmQueries = map[string]DetailQuery{
// discoveryTable returns a query to determine whether a table exists or not.
func discoveryTable(tableName string) string {
- return fmt.Sprintf("SELECT 1 FROM osquery_registry WHERE active = true AND registry = 'table' AND name = '%s';", tableName)
+ return fmt.Sprintf("SELECT 1 FROM osquery_registry WHERE active = true AND registry = 'table' AND name = '%s'", tableName)
}
func macOSBundleIDExistsQuery(appName string) string {
@@ -774,7 +774,11 @@ func generateSQLForAllExists(subqueries ...string) string {
// Generate EXISTS clause for each subquery
var conditions []string
for _, query := range subqueries {
- condition := fmt.Sprintf("EXISTS (%s)", query)
+ // Remove trailing semicolons from the query to ensure subqueries
+ // are not terminated early (Issue #19401)
+ sanitized := strings.TrimRight(strings.TrimSpace(query), ";")
+
+ condition := fmt.Sprintf("EXISTS (%s)", sanitized)
conditions = append(conditions, condition)
}
@@ -2015,13 +2019,9 @@ func GetDetailQueries(
generatedMap["software_chrome"] = softwareChrome
generatedMap["software_vscode_extensions"] = softwareVSCodeExtensions
- //
- // Commenting out until we find root cause of discovery query hanging osquery:
- // https://github.com/fleetdm/fleet/issues/19401
- //
- // for key, query := range SoftwareOverrideQueries {
- // generatedMap["software_"+key] = query
- // }
+ for key, query := range SoftwareOverrideQueries {
+ generatedMap["software_"+key] = query
+ }
}
if features != nil && features.EnableHostUsers {
diff --git a/server/service/osquery_utils/queries_test.go b/server/service/osquery_utils/queries_test.go
index 14639a33a8..6b0fb70e23 100644
--- a/server/service/osquery_utils/queries_test.go
+++ b/server/service/osquery_utils/queries_test.go
@@ -304,7 +304,7 @@ func TestGetDetailQueries(t *testing.T) {
sortedKeysCompare(t, queriesWithUsers, qs)
queriesWithUsersAndSoftware := GetDetailQueries(context.Background(), config.FleetConfig{App: config.AppConfig{EnableScheduledQueryStats: true}}, nil, &fleet.Features{EnableHostUsers: true, EnableSoftwareInventory: true})
- qs = append(baseQueries, "users", "users_chrome", "software_macos", "software_linux", "software_windows", "software_vscode_extensions", "software_chrome", "scheduled_query_stats")
+ qs = append(baseQueries, "users", "users_chrome", "software_macos", "software_linux", "software_windows", "software_vscode_extensions", "software_chrome", "scheduled_query_stats", "software_macos_firefox")
require.Len(t, queriesWithUsersAndSoftware, len(qs))
sortedKeysCompare(t, queriesWithUsersAndSoftware, qs)
@@ -1935,12 +1935,25 @@ func TestIngestNetworkInterface(t *testing.T) {
}
func TestGenerateSQLForAllExists(t *testing.T) {
+ // Combine two queries
query1 := "SELECT 1 WHERE foo = bar"
query2 := "SELECT 1 WHERE baz = qux"
-
sql := generateSQLForAllExists(query1, query2)
assert.Equal(t, "SELECT 1 WHERE EXISTS (SELECT 1 WHERE foo = bar) AND EXISTS (SELECT 1 WHERE baz = qux)", sql)
+ // Default
sql = generateSQLForAllExists()
require.Equal(t, "SELECT 0 LIMIT 0", sql)
+
+ // sanitize semicolons from subqueries
+ query1 = "SELECT 1 WHERE foo = bar;"
+ query2 = "SELECT 1 WHERE baz = qux;"
+ sql = generateSQLForAllExists(query1, query2)
+ assert.Equal(t, "SELECT 1 WHERE EXISTS (SELECT 1 WHERE foo = bar) AND EXISTS (SELECT 1 WHERE baz = qux)", sql)
+
+ // sanitize only trailing semicolons
+ query1 = "SELECT 1 WHERE foo = 'ba;r';"
+ query2 = "SELECT 1 WHERE baz = 'qu;x';;; "
+ sql = generateSQLForAllExists(query1, query2)
+ assert.Equal(t, "SELECT 1 WHERE EXISTS (SELECT 1 WHERE foo = 'ba;r') AND EXISTS (SELECT 1 WHERE baz = 'qu;x')", sql)
}
diff --git a/server/service/scripts.go b/server/service/scripts.go
index bde4991867..4bf2d85297 100644
--- a/server/service/scripts.go
+++ b/server/service/scripts.go
@@ -400,23 +400,30 @@ func (svc *Service) GetScriptResult(ctx context.Context, execID string) (*fleet.
return nil, ctxerr.Wrap(ctx, err, "get script result")
}
- host, err := svc.ds.HostLite(ctx, scriptResult.HostID)
- if err != nil {
- // if error is because the host does not exist, check first if the user
- // had access to run a script (to prevent leaking valid host ids).
- if fleet.IsNotFound(err) {
- if err := svc.authz.Authorize(ctx, &fleet.HostScriptResult{}, fleet.ActionRead); err != nil {
- return nil, err
+ if scriptResult.HostDeletedAt == nil {
+ // host is not deleted, get it and authorize for the host's team
+ host, err := svc.ds.HostLite(ctx, scriptResult.HostID)
+ if err != nil {
+ // if error is because the host does not exist, check first if the user
+ // had access to run a script (to prevent leaking valid host ids).
+ if fleet.IsNotFound(err) {
+ if err := svc.authz.Authorize(ctx, &fleet.HostScriptResult{}, fleet.ActionRead); err != nil {
+ return nil, err
+ }
}
+ svc.authz.SkipAuthorization(ctx)
+ return nil, ctxerr.Wrap(ctx, err, "get host lite")
+ }
+ if err := svc.authz.Authorize(ctx, &fleet.HostScriptResult{TeamID: host.TeamID}, fleet.ActionRead); err != nil {
+ return nil, err
+ }
+ scriptResult.Hostname = host.DisplayName()
+ } else {
+ // host was deleted, authorize for no-team as a fallback
+ if err := svc.authz.Authorize(ctx, &fleet.HostScriptResult{}, fleet.ActionRead); err != nil {
+ return nil, err
}
- svc.authz.SkipAuthorization(ctx)
- return nil, ctxerr.Wrap(ctx, err, "get host lite")
}
- if err := svc.authz.Authorize(ctx, &fleet.HostScriptResult{TeamID: host.TeamID}, fleet.ActionRead); err != nil {
- return nil, err
- }
-
- scriptResult.Hostname = host.DisplayName()
return scriptResult, nil
}
diff --git a/server/service/software_installers.go b/server/service/software_installers.go
index f03f05d0b7..d22100a4ef 100644
--- a/server/service/software_installers.go
+++ b/server/service/software_installers.go
@@ -2,6 +2,7 @@ package service
import (
"context"
+ "errors"
"fmt"
"io"
"mime/multipart"
@@ -29,14 +30,25 @@ type uploadSoftwareInstallerResponse struct {
Err error `json:"error,omitempty"`
}
+// MaxSoftwareInstallerSize is the maximum size allowed for software
+// installers. This is enforced by the endpoint that uploads installers.
+const MaxSoftwareInstallerSize = 500 * units.MiB
+
// TODO: We parse the whole body before running svc.authz.Authorize.
// An authenticated but unauthorized user could abuse this.
func (uploadSoftwareInstallerRequest) DecodeRequest(ctx context.Context, r *http.Request) (interface{}, error) {
decoded := uploadSoftwareInstallerRequest{}
err := r.ParseMultipartForm(512 * units.MiB)
if err != nil {
+ var mbe *http.MaxBytesError
+ if errors.As(err, &mbe) {
+ return nil, &fleet.BadRequestError{
+ Message: "The maximum file size is 500 MB.",
+ InternalErr: err,
+ }
+ }
return nil, &fleet.BadRequestError{
- Message: "failed to parse multipart form",
+ Message: "failed to parse multipart form: " + err.Error(),
InternalErr: err,
}
}
@@ -49,9 +61,9 @@ func (uploadSoftwareInstallerRequest) DecodeRequest(ctx context.Context, r *http
}
decoded.File = r.MultipartForm.File["software"][0]
-
- if decoded.File.Size > 500*units.MiB {
- // TODO: Should we try to assess the size earlier in the request processing (before parsing the form)?
+ if decoded.File.Size > MaxSoftwareInstallerSize {
+ // Should never happen here since the request's body is limited to the
+ // maximum size.
return nil, &fleet.BadRequestError{
Message: "The maximum file size is 500 MB.",
}
diff --git a/server/service/team_policies.go b/server/service/team_policies.go
index 62540aef94..81ebee7d40 100644
--- a/server/service/team_policies.go
+++ b/server/service/team_policies.go
@@ -269,6 +269,18 @@ func deleteTeamPoliciesEndpoint(ctx context.Context, request interface{}, svc fl
}
func (svc Service) DeleteTeamPolicies(ctx context.Context, teamID uint, ids []uint) ([]uint, error) {
+ if err := svc.authz.Authorize(ctx, &fleet.Policy{
+ PolicyData: fleet.PolicyData{
+ TeamID: ptr.Uint(teamID),
+ },
+ }, fleet.ActionWrite); err != nil {
+ return nil, err
+ }
+
+ if _, err := svc.ds.Team(ctx, teamID); err != nil {
+ return nil, ctxerr.Wrapf(ctx, err, "loading team %d", teamID)
+ }
+
if len(ids) == 0 {
return nil, nil
}
@@ -277,13 +289,6 @@ func (svc Service) DeleteTeamPolicies(ctx context.Context, teamID uint, ids []ui
return nil, ctxerr.Wrap(ctx, err, "getting policies by ID")
}
- if err := svc.authz.Authorize(ctx, &fleet.Policy{
- PolicyData: fleet.PolicyData{
- TeamID: ptr.Uint(teamID),
- },
- }, fleet.ActionWrite); err != nil {
- return nil, err
- }
for _, policy := range policiesByID {
if t := policy.PolicyData.TeamID; t == nil || *t != teamID {
return nil, authz.ForbiddenWithInternal(
diff --git a/server/service/teams_test.go b/server/service/teams_test.go
index c145c34714..9011e8bce7 100644
--- a/server/service/teams_test.go
+++ b/server/service/teams_test.go
@@ -431,7 +431,7 @@ func TestApplyTeamSpecEnrollSecretForNewTeams(t *testing.T) {
return false, nil
}
_, err := svc.ApplyTeamSpecs(
- ctx, []*fleet.TeamSpec{{Name: "Foo", Secrets: []fleet.EnrollSecret{enrollSecret}}},
+ ctx, []*fleet.TeamSpec{{Name: "Foo", Secrets: &[]fleet.EnrollSecret{enrollSecret}}},
fleet.ApplyTeamSpecOptions{ApplySpecOptions: fleet.ApplySpecOptions{DryRun: true}},
)
assert.ErrorContains(t, err, "is already being used")
@@ -441,14 +441,14 @@ func TestApplyTeamSpecEnrollSecretForNewTeams(t *testing.T) {
return true, nil
}
_, err = svc.ApplyTeamSpecs(
- ctx, []*fleet.TeamSpec{{Name: "Foo", Secrets: []fleet.EnrollSecret{enrollSecret}}},
+ ctx, []*fleet.TeamSpec{{Name: "Foo", Secrets: &[]fleet.EnrollSecret{enrollSecret}}},
fleet.ApplyTeamSpecOptions{ApplySpecOptions: fleet.ApplySpecOptions{DryRun: true}},
)
assert.NoError(t, err)
assert.False(t, ds.NewTeamFuncInvoked)
_, err = svc.ApplyTeamSpecs(
- ctx, []*fleet.TeamSpec{{Name: "Foo", Secrets: []fleet.EnrollSecret{enrollSecret}}}, fleet.ApplyTeamSpecOptions{},
+ ctx, []*fleet.TeamSpec{{Name: "Foo", Secrets: &[]fleet.EnrollSecret{enrollSecret}}}, fleet.ApplyTeamSpecOptions{},
)
require.NoError(t, err)
require.True(t, ds.TeamByNameFuncInvoked)
diff --git a/server/vulnerabilities/nvd/cpe_matching_rule.go b/server/vulnerabilities/nvd/cpe_matching_rule.go
index 32b331d4f3..9bd8a9b751 100644
--- a/server/vulnerabilities/nvd/cpe_matching_rule.go
+++ b/server/vulnerabilities/nvd/cpe_matching_rule.go
@@ -42,6 +42,7 @@ type CPEMatchingRule struct {
// IgnoreAll will cause all CPEs to not match hence ignoring a CVE.
IgnoreAll bool
// IgnoreIf is a function that can determine if a CPE matching rule should be ignored or not.
+ // If IgnoreIf is set, CPESpecs will not be evaluated.
IgnoreIf func(cpeMeta *wfn.Attributes) bool
}
@@ -55,8 +56,11 @@ func (rule CPEMatchingRule) CPEMatches(cpeMeta *wfn.Attributes) bool {
return false
}
- if rule.IgnoreIf != nil && rule.IgnoreIf(cpeMeta) {
- return false
+ if rule.IgnoreIf != nil {
+ if rule.IgnoreIf(cpeMeta) {
+ return false
+ }
+ return true
}
ver, err := semver.NewVersion(wfn.StripSlashes(cpeMeta.Version))
diff --git a/server/vulnerabilities/nvd/cpe_matching_rules.go b/server/vulnerabilities/nvd/cpe_matching_rules.go
index f745f97ec0..01fa401bb2 100644
--- a/server/vulnerabilities/nvd/cpe_matching_rules.go
+++ b/server/vulnerabilities/nvd/cpe_matching_rules.go
@@ -197,6 +197,18 @@ func GetKnownNVDBugRules() (CPEMatchingRules, error) {
return cpeMeta.TargetSW == "visual_studio_code"
},
},
+ // Issue #18733 incorrect CPEs that should be matching
+ // visual studio code extensions
+ CPEMatchingRule{
+ CVEs: map[string]struct{}{
+ "CVE-2021-28967": {},
+ "CVE-2020-1192": {},
+ "CVE-2020-1171": {},
+ },
+ IgnoreIf: func(cpeMeta *wfn.Attributes) bool {
+ return cpeMeta.Product == "visual_studio_code" && cpeMeta.TargetSW == wfn.Any
+ },
+ },
}
for i, rule := range rules {
diff --git a/server/vulnerabilities/nvd/cve_test.go b/server/vulnerabilities/nvd/cve_test.go
index 4b7f095277..05cdca110d 100644
--- a/server/vulnerabilities/nvd/cve_test.go
+++ b/server/vulnerabilities/nvd/cve_test.go
@@ -307,6 +307,12 @@ func TestTranslateCPEToCVE(t *testing.T) {
},
continuesToUpdate: false,
},
+ "cpe:2.3:a:adobe:animate:*:*:*:*:*:macos:*:*": {
+ includedCVEs: []cve{
+ {ID: "CVE-2023-44325"},
+ },
+ continuesToUpdate: true,
+ },
}
cveOSTests := []struct {
diff --git a/server/vulnerabilities/nvd/tools/cvefeed/matching_json_test.go b/server/vulnerabilities/nvd/tools/cvefeed/matching_json_test.go
index baa73cad43..df6629b88b 100644
--- a/server/vulnerabilities/nvd/tools/cvefeed/matching_json_test.go
+++ b/server/vulnerabilities/nvd/tools/cvefeed/matching_json_test.go
@@ -94,6 +94,15 @@ func TestMatchJSON(t *testing.T) {
{Part: "a", Vendor: "mozilla", Product: "firefox", Version: "64\\.0"},
},
},
+ {
+ Rule: 3,
+ Inventory: []*wfn.Attributes{
+ {Part: "o", Vendor: "apple", Product: "macos", Version: "14\\.1\\.2"},
+ },
+ Matches: []*wfn.Attributes{
+ {Part: "o", Vendor: "apple", Product: "macos", Version: "14\\.1\\.2"},
+ },
+ },
}
items, err := ParseJSON(bytes.NewBufferString(testJSONdict))
if err != nil {
@@ -103,7 +112,7 @@ func TestMatchJSON(t *testing.T) {
t.Run(fmt.Sprintf("%d", i), func(t *testing.T) {
mm := items[c.Rule].Match(c.Inventory, false)
if len(mm) != len(c.Matches) {
- t.Fatalf("expected %d matches, got %d matches", len(mm), len(c.Matches))
+ t.Fatalf("expected %d matches, got %d matches", len(c.Matches), len(mm))
}
if len(mm) > 0 && !matchesAll(mm, c.Matches) {
t.Fatalf("wrong match: expected %v, got %v", c.Matches, mm)
@@ -112,6 +121,45 @@ func TestMatchJSON(t *testing.T) {
}
}
+func TestTargetSWMatching(t *testing.T) {
+ inventoryAcrobat := []*wfn.Attributes{
+ {Part: "a", Vendor: "adobe", Product: "acrobat", Version: "20\\.001\\.3005", TargetSW: "macos"},
+ }
+
+ items, err := ParseJSON(bytes.NewBufferString(targetSWMatchingJSON))
+ if err != nil {
+ t.Fatalf("failed to parse the dictionary: %v", err)
+ }
+ // matches OS on targetSW
+ if mm := items[0].Match(inventoryAcrobat, true); len(mm) == 0 {
+ t.Fatal("expected Match to match, it did not")
+ }
+
+ // does not match OS on targetSW
+ if mm := items[1].Match(inventoryAcrobat, true); len(mm) != 0 {
+ t.Fatal("expected Match to not match, it did")
+ }
+
+ // matches when OS is not present
+ if mm := items[2].Match(inventoryAcrobat, true); len(mm) == 0 {
+ t.Fatal("expected Match to match, it did not")
+ }
+
+ // does not match OS on targetSW with multiple nodes
+ if mm := items[3].Match(inventoryAcrobat, true); len(mm) != 0 {
+ t.Fatal("expected Match to not match, it did")
+ }
+
+ inventoryWrongOS := []*wfn.Attributes{
+ {Part: "a", Vendor: "adobe", Product: "acrobat", Version: "20\\.001\\.3005", TargetSW: "linux"},
+ }
+
+ // does not match OS on targetSW
+ if mm := items[0].Match(inventoryWrongOS, true); len(mm) != 0 {
+ t.Fatal("expected Match to not match, it did")
+ }
+}
+
func TestMatchJSONrequireVersion(t *testing.T) {
inventory := []*wfn.Attributes{
{Part: "a", Vendor: "microsoft", Product: "ie", Version: "6\\.0"},
@@ -279,9 +327,265 @@ var testJSONdict = `{
}
]
}
- }
+ },
+ {
+ "cve": {
+ "affects": null,
+ "CVE_data_meta": {
+ "ASSIGNER": "product-security@apple.com",
+ "ID": "CVE-2023-42919"
+ },
+ "data_format": "MITRE",
+ "data_type": "CVE",
+ "data_version": "4.0"
+ },
+ "configurations": {
+ "CVE_data_version": "4.0",
+ "nodes": [
+ {
+ "cpe_match": [
+ {
+ "cpe23Uri": "cpe:2.3:o:apple:ipados:*:*:*:*:*:*:*:*",
+ "versionEndExcluding": "16.7.3",
+ "vulnerable": true
+ },
+ {
+ "cpe23Uri": "cpe:2.3:o:apple:ipados:*:*:*:*:*:*:*:*",
+ "versionEndExcluding": "17.2",
+ "versionStartIncluding": "17.0",
+ "vulnerable": true
+ },
+ {
+ "cpe23Uri": "cpe:2.3:o:apple:iphone_os:*:*:*:*:*:*:*:*",
+ "versionEndExcluding": "16.7.3",
+ "vulnerable": true
+ },
+ {
+ "cpe23Uri": "cpe:2.3:o:apple:iphone_os:*:*:*:*:*:*:*:*",
+ "versionEndExcluding": "17.2",
+ "versionStartIncluding": "17.0",
+ "vulnerable": true
+ },
+ {
+ "cpe23Uri": "cpe:2.3:o:apple:macos:*:*:*:*:*:*:*:*",
+ "versionEndExcluding": "12.7.2",
+ "versionStartIncluding": "12.0.0",
+ "vulnerable": true
+ },
+ {
+ "cpe23Uri": "cpe:2.3:o:apple:macos:*:*:*:*:*:*:*:*",
+ "versionEndExcluding": "13.6.3",
+ "versionStartIncluding": "13.0",
+ "vulnerable": true
+ },
+ {
+ "cpe23Uri": "cpe:2.3:o:apple:macos:*:*:*:*:*:*:*:*",
+ "versionEndExcluding": "14.2",
+ "versionStartIncluding": "14.0",
+ "vulnerable": true
+ }
+ ],
+ "operator": "OR"
+ }
+ ]
+ }
+}
] }`
+var targetSWMatchingJSON = `{
+ "CVE_data_type" : "CVE",
+ "CVE_data_format" : "MITRE",
+ "CVE_data_version" : "4.0",
+ "CVE_data_numberOfCVEs" : "7083",
+ "CVE_data_timestamp" : "2018-07-31T07:00Z",
+ "CVE_Items" : [ {
+ "cve" : {
+ "data_type" : "CVE",
+ "data_format" : "MITRE",
+ "data_version" : "4.0",
+ "CVE_data_meta" : {
+ "ID" : "CVE-2023-26369",
+ "ASSIGNER" : "psirt@adobe.com"
+ }
+ },
+ "configurations" : {
+ "CVE_data_version" : "4.0",
+ "nodes" : [ {
+ "operator" : "AND",
+ "children" : [ {
+ "operator" : "OR",
+ "children" : [ ],
+ "cpe_match" : [ {
+ "vulnerable" : true,
+ "cpe23Uri" : "cpe:2.3:a:adobe:acrobat:*:*:*:*:classic:*:*:*",
+ "versionStartIncluding" : "20.001.3005",
+ "versionEndExcluding" : "20.005.30524",
+ "cpe_name" : [ ]
+ }, {
+ "vulnerable" : true,
+ "cpe23Uri" : "cpe:2.3:a:adobe:acrobat_dc:*:*:*:*:continuous:*:*:*",
+ "versionStartIncluding" : "15.007.20033",
+ "versionEndExcluding" : "23.006.20320",
+ "cpe_name" : [ ]
+ }, {
+ "vulnerable" : true,
+ "cpe23Uri" : "cpe:2.3:a:adobe:acrobat_reader:*:*:*:*:classic:*:*:*",
+ "versionStartIncluding" : "20.001.3005",
+ "versionEndExcluding" : "20.005.30524",
+ "cpe_name" : [ ]
+ }, {
+ "vulnerable" : true,
+ "cpe23Uri" : "cpe:2.3:a:adobe:acrobat_reader_dc:*:*:*:*:continuous:*:*:*",
+ "versionStartIncluding" : "15.007.20033",
+ "versionEndExcluding" : "23.006.20320",
+ "cpe_name" : [ ]
+ } ]
+ }, {
+ "operator" : "OR",
+ "children" : [ ],
+ "cpe_match" : [ {
+ "vulnerable" : false,
+ "cpe23Uri" : "cpe:2.3:o:apple:macos:-:*:*:*:*:*:*:*",
+ "cpe_name" : [ ]
+ }, {
+ "vulnerable" : false,
+ "cpe23Uri" : "cpe:2.3:o:microsoft:windows:-:*:*:*:*:*:*:*",
+ "cpe_name" : [ ]
+ } ]
+ } ],
+ "cpe_match" : [ ]
+ } ]
+ }
+ }, {
+ "cve" : {
+ "data_type" : "CVE",
+ "data_format" : "MITRE",
+ "data_version" : "4.0",
+ "CVE_data_meta" : {
+ "ID" : "CVE-2023-27928",
+ "ASSIGNER" : "product-security@apple.com"
+ }
+ },
+ "configurations" : {
+ "CVE_data_version" : "4.0",
+ "nodes" : [ {
+ "operator" : "OR",
+ "children" : [ ],
+ "cpe_match" : [ {
+ "vulnerable" : true,
+ "cpe23Uri" : "cpe:2.3:o:apple:macos:*:*:*:*:*:*:*:*",
+ "versionStartIncluding" : "13.0",
+ "versionEndExcluding" : "13.3",
+ "cpe_name" : [ ]
+ }, {
+ "vulnerable" : true,
+ "cpe23Uri" : "cpe:2.3:o:apple:tvos:*:*:*:*:*:*:*:*",
+ "versionEndExcluding" : "16.4",
+ "cpe_name" : [ ]
+ }, {
+ "vulnerable" : true,
+ "cpe23Uri" : "cpe:2.3:o:apple:watchos:*:*:*:*:*:*:*:*",
+ "versionEndExcluding" : "9.4",
+ "cpe_name" : [ ]
+ }, {
+ "vulnerable" : true,
+ "cpe23Uri" : "cpe:2.3:o:apple:macos:*:*:*:*:*:*:*:*",
+ "versionEndExcluding" : "11.7.5",
+ "cpe_name" : [ ]
+ } ]
+ } ]
+ }
+ }, {
+ "cve" : {
+ "data_type" : "CVE",
+ "data_format" : "MITRE",
+ "data_version" : "4.0",
+ "CVE_data_meta" : {
+ "ID" : "CVE-2023-27928",
+ "ASSIGNER" : "product-security@apple.com"
+ }
+ },
+ "configurations" : {
+ "CVE_data_version" : "4.0",
+ "nodes" : [ {
+ "operator" : "OR",
+ "children" : [ ],
+ "cpe_match" : [ {
+ "vulnerable" : true,
+ "cpe23Uri" : "cpe:2.3:a:adobe:acrobat:*:*:*:*:classic:*:*:*",
+ "versionStartIncluding" : "20.001.3005",
+ "versionEndExcluding" : "20.005.30524",
+ "cpe_name" : [ ]
+ }, {
+ "vulnerable" : true,
+ "cpe23Uri" : "cpe:2.3:a:adobe:acrobat_dc:*:*:*:*:continuous:*:*:*",
+ "versionStartIncluding" : "15.007.20033",
+ "versionEndExcluding" : "23.006.20320",
+ "cpe_name" : [ ]
+ }, {
+ "vulnerable" : true,
+ "cpe23Uri" : "cpe:2.3:a:adobe:acrobat_reader:*:*:*:*:classic:*:*:*",
+ "versionStartIncluding" : "20.001.3005",
+ "versionEndExcluding" : "20.005.30524",
+ "cpe_name" : [ ]
+ }, {
+ "vulnerable" : true,
+ "cpe23Uri" : "cpe:2.3:a:adobe:acrobat_reader_dc:*:*:*:*:continuous:*:*:*",
+ "versionStartIncluding" : "15.007.20033",
+ "versionEndExcluding" : "23.006.20320",
+ "cpe_name" : [ ]
+ } ]
+ } ]
+ }
+ }, {
+ "cve" : {
+ "data_type" : "CVE",
+ "data_format" : "MITRE",
+ "data_version" : "4.0",
+ "CVE_data_meta" : {
+ "ID" : "CVE-2023-28321",
+ "ASSIGNER" : "support@hackerone.com"
+ }
+ },
+ "configurations" : {
+ "CVE_data_version" : "4.0",
+ "nodes" : [ {
+ "operator" : "OR",
+ "children" : [ ],
+ "cpe_match" : [ {
+ "vulnerable" : true,
+ "cpe23Uri" : "cpe:2.3:a:haxx:curl:*:*:*:*:*:*:*:*",
+ "versionEndExcluding" : "8.1.0",
+ "cpe_name" : [ ]
+ } ]
+ }, {
+ "operator" : "OR",
+ "children" : [ ],
+ "cpe_match" : [ {
+ "vulnerable" : true,
+ "cpe23Uri" : "cpe:2.3:o:apple:macos:*:*:*:*:*:*:*:*",
+ "versionStartIncluding" : "13.0",
+ "versionEndExcluding" : "13.5",
+ "cpe_name" : [ ]
+ }, {
+ "vulnerable" : true,
+ "cpe23Uri" : "cpe:2.3:o:apple:macos:*:*:*:*:*:*:*:*",
+ "versionStartIncluding" : "12.0",
+ "versionEndExcluding" : "12.6.8",
+ "cpe_name" : [ ]
+ }, {
+ "vulnerable" : true,
+ "cpe23Uri" : "cpe:2.3:o:apple:macos:*:*:*:*:*:*:*:*",
+ "versionStartIncluding" : "11.0",
+ "versionEndExcluding" : "11.7.9",
+ "cpe_name" : [ ]
+ } ]
+ } ]
+ }
+ }
+] }
+`
+
func matchesAll(src, tgt []*wfn.Attributes) bool {
if len(src) != len(tgt) {
return false
diff --git a/server/vulnerabilities/nvd/tools/cvefeed/nvd/match_cpe.go b/server/vulnerabilities/nvd/tools/cvefeed/nvd/match_cpe.go
index 636a78a93a..f82866a3dc 100644
--- a/server/vulnerabilities/nvd/tools/cvefeed/nvd/match_cpe.go
+++ b/server/vulnerabilities/nvd/tools/cvefeed/nvd/match_cpe.go
@@ -69,6 +69,9 @@ func (cm *cpeMatch) Match(attrs []*wfn.Attributes, requireVersion bool) (matches
if cm.match(attr, requireVersion) {
matches = append(matches, attr)
}
+ if osMatch := cm.MatchTargetSW(attr); osMatch != nil {
+ matches = append(matches, osMatch)
+ }
}
return matches
}
diff --git a/server/vulnerabilities/nvd/tools/wfn/matcher.go b/server/vulnerabilities/nvd/tools/wfn/matcher.go
index fca69e6e47..edb0043b4c 100644
--- a/server/vulnerabilities/nvd/tools/wfn/matcher.go
+++ b/server/vulnerabilities/nvd/tools/wfn/matcher.go
@@ -49,14 +49,37 @@ func (a *Attributes) MatchWithoutVersion(attr *Attributes) bool {
matchAttr(a.Other, attr.Other)
}
+func (a *Attributes) MatchTargetSW(attr *Attributes) *Attributes {
+ if a == nil || attr == nil {
+ return nil
+ }
+
+ var osMatch bool
+ var osAttr *Attributes
+ if attr.Part == "a" && attr.TargetSW != "" {
+ osAttr = &Attributes{
+ Part: "o",
+ Product: attr.TargetSW,
+ }
+
+ osMatch = matchAttr(a.Part, osAttr.Part) && matchAttr(a.Product, osAttr.Product)
+ }
+
+ if !osMatch {
+ return nil
+ }
+
+ return osAttr
+}
+
// MatchAll returns a Matcher which matches only if all matchers match
func MatchAll(ms ...Matcher) Matcher {
- return &multiMatcher{ms, true}
+ return &multiMatcher{matchers: ms, allMatch: true}
}
// MatchAll returns a Matcher which matches if any of the matchers match
func MatchAny(ms ...Matcher) Matcher {
- return &multiMatcher{ms, false}
+ return &multiMatcher{matchers: ms, allMatch: false}
}
// DontMatch returns a Matcher which matches if the given matchers doesn't
@@ -68,12 +91,23 @@ type multiMatcher struct {
matchers []Matcher
// if true, match will only return something if all matchers matched at least something
allMatch bool
+ depth int
}
// Match is part of the Matcher interface
func (mm *multiMatcher) Match(attrs []*Attributes, requireVersion bool) []*Attributes {
+ defer func() {
+ if mm.depth > 0 {
+ mm.depth--
+ }
+ }()
+
matched := make(map[*Attributes]bool)
for _, matcher := range mm.matchers {
+ // type check matcher against multiMatcher
+ if _, ok := matcher.(*multiMatcher); !ok {
+ mm.depth++
+ }
matches := matcher.Match(attrs, requireVersion)
if mm.allMatch && len(matches) == 0 {
// all matchers need to match at least one attr
@@ -88,6 +122,11 @@ func (mm *multiMatcher) Match(attrs []*Attributes, requireVersion bool) []*Attri
for m := range matched {
matches = append(matches, m)
}
+
+ if mm.depth == 0 && len(matches) > 1 && !attributesIncludeApp(matches) {
+ return nil
+ }
+
return matches
}
@@ -118,3 +157,12 @@ func (nm notMatcher) Match(attrs []*Attributes, requireVersion bool) (matches []
}
return matches
}
+
+func attributesIncludeApp(attrs []*Attributes) bool {
+ for _, a := range attrs {
+ if a.Part == "a" {
+ return true
+ }
+ }
+ return false
+}
diff --git a/terraform/README.md b/terraform/README.md
index 969ebb469d..8ca2664e3b 100644
--- a/terraform/README.md
+++ b/terraform/README.md
@@ -75,7 +75,7 @@ No resources.
| [alb\_config](#input\_alb\_config) | n/a | '+text+' ';
+ var textWithLineBreaks;
+ if(text.match(/\S(\w+\_\S)+(\w\S)+/g) && !text.match(/\s/g)){
+ textWithLineBreaks = text.replace(/(\_)/g, '_');
+ }
+ return ''+(textWithLineBreaks ? textWithLineBreaks : text)+' ';
};
}
diff --git a/website/assets/images/articles/deploy-fleet-on-ubuntu-with-elastic-1600x900@2x.png b/website/assets/images/articles/deploy-fleet-on-ubuntu-with-elastic-1600x900@2x.png
new file mode 100644
index 0000000000..82a8c3dbc1
Binary files /dev/null and b/website/assets/images/articles/deploy-fleet-on-ubuntu-with-elastic-1600x900@2x.png differ
diff --git a/website/assets/images/articles/deploy-fleet-on-ubuntu-with-elastic-internews_logo-256x237@2x.png b/website/assets/images/articles/deploy-fleet-on-ubuntu-with-elastic-internews_logo-256x237@2x.png
new file mode 100644
index 0000000000..189dd8f6bf
Binary files /dev/null and b/website/assets/images/articles/deploy-fleet-on-ubuntu-with-elastic-internews_logo-256x237@2x.png differ
diff --git a/website/assets/images/articles/fleet-4.51.0-1600x900@2x.png b/website/assets/images/articles/fleet-4.51.0-1600x900@2x.png
new file mode 100644
index 0000000000..573d63cb76
Binary files /dev/null and b/website/assets/images/articles/fleet-4.51.0-1600x900@2x.png differ
diff --git a/website/assets/images/demo/fleet-desktop-migration.png b/website/assets/images/demo/fleet-desktop-migration.png
new file mode 100644
index 0000000000..0601570d27
Binary files /dev/null and b/website/assets/images/demo/fleet-desktop-migration.png differ
diff --git a/website/assets/images/demo/fleet-system-maintenance.png b/website/assets/images/demo/fleet-system-maintenance.png
new file mode 100644
index 0000000000..21d11d1327
Binary files /dev/null and b/website/assets/images/demo/fleet-system-maintenance.png differ
diff --git a/website/assets/js/components/modal.component.js b/website/assets/js/components/modal.component.js
index 8d8e682d3c..ce047ba382 100644
--- a/website/assets/js/components/modal.component.js
+++ b/website/assets/js/components/modal.component.js
@@ -34,7 +34,6 @@ parasails.registerComponent('modal', {
// but still.... better safe than sorry!)
_bsModalIsAnimatingOut: false,
- isMobileSafari: false,//« more on this below
originalScrollPosition: undefined,//« more on this below
};
},
@@ -60,25 +59,6 @@ parasails.registerComponent('modal', {
// ║ ║╠╣ ║╣ ║ ╚╦╝║ ║ ║╣
// ╩═╝╩╚ ╚═╝╚═╝ ╩ ╚═╝╩═╝╚═╝
beforeMount: function() {
- // If this is mobile safari, make note of it.
- this.isMobileSafari = (typeof bowser !== 'undefined') && bowser.mobile && bowser.safari;
- // - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- // ^^So there's a bug in mobile safari that misplaces the caret when the keyboard opening
- // causes the page to scroll, so we need to do some special tricks to keep it from getting ugly.
- // It's only in iOS 11... we think. Hopefully it will be fixed.
- // In the mean time, we have to get wacky.
- //
- // > More info about the bug here:
- // > https://github.com/twbs/bootstrap/issues/24835#issuecomment-345974819
- // > https://stackoverflow.com/questions/46567233/how-to-fix-the-ios-11-input-element-in-fixed-modals-bug?rq=1
- //
- // FUTURE: maybe the bug will be fixed and we can remove this someday?
- // - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-
- if(this.isMobileSafari) {
- // Get our original scroll position before opening the modal and save it for later.
- this.originalScrollPosition = $(window).scrollTop();
- }
},
mounted: function(){
// ^^ Note that this is not an `async function`.
@@ -100,26 +80,6 @@ parasails.registerComponent('modal', {
// the parent logic can use this event to update its scope.)
$(this.$el).on('hide.bs.modal', ()=>{
- // Undo any mobile safari workarounds we may have added.
- // (i.e. shed the wackiness)
- if(this.isMobileSafari) {
- // Remove style overrides on our modal dialog.
- $(this.$el).css({
- 'overflow-y': '',
- 'position': '',
- 'left': '',
- 'top': '',
- });
-
- // Beckon to our siblings so they come out of hiding
- this.$get().parent().children().not(this.$el).css({
- 'display': ''
- });
-
- // Scroll to our original position when the modal was summoned.
- window.scrollTo(0, this.originalScrollPosition);
- }//fi
-
this._bsModalIsAnimatingOut = true;
this.$emit('close');
@@ -131,40 +91,6 @@ parasails.registerComponent('modal', {
// us to do cool things like auto-focus the first input in a form modal.
$(this.$el).on('shown.bs.modal', ()=>{
- // If this is mobile safari, let's get wacky.
- if(this.isMobileSafari) {
- // Scroll to the top of the page.
- window.scrollTo(0, 0);
- // - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- // ^^FUTURE: Don't actually do this -- instead, try setting `top` of the
- // modal to whatever the original scrollTop of our window was. This
- // eliminates the need for auto-scrolling to the top and ripping you out
- // of the context you were in before the modal opens. It would also allow
- // us to keep the nice animation when opening/closing modals on iOS.
- // - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-
- // Hide siblings to lop off any extra space at the bottom.
- this.$get().parent().children().not(this.$el).css({
- 'display': 'none'
- });
- // - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
- // ^^FUTURE: Instead of just hiding siblings, which isn't perfect and won't
- // always work for everyone, try grabbing outerHeight of the modal element
- // and using that to set an explicit height for the body.
- // (but also be sure to handle the case where the body is short!)
- // But for now, this should work as long as we have sticky footer styles.
- // - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-
- // Hard code some style overrides on our modal dialog.
- // Without these, it gets weird.
- $(this.$el).css({
- 'overflow-y': 'auto!important',
- 'position': 'absolute',
- 'left': '0',
- 'top': '0',
- });
- }//fi
-
// Focus our "focus-first" field, if relevant.
// (but not on mobile, because it can get weird)
if(typeof bowser !== 'undefined' && !bowser.mobile && this.$find('[focus-first]').length > 0) {
diff --git a/website/assets/styles/pages/query-library.less b/website/assets/styles/pages/query-library.less
index a6d883789e..6584078dd2 100644
--- a/website/assets/styles/pages/query-library.less
+++ b/website/assets/styles/pages/query-library.less
@@ -67,6 +67,7 @@
padding: 2px 8px;
border-radius: 20px;
background-color: #E2E4EA;
+ color: #515774;
}
[purpose='selected-tag'] {
@@ -199,6 +200,7 @@
.description {
padding: 0px 30px 0px 30px;
p {
+ color: #515774;
font-size: 16px;
line-height: 25px;
}
@@ -263,6 +265,7 @@
box-shadow: none;
border: none;
border-radius: 8px;
+ color: #515774;
&:hover {
.query-card {
background-color: #F8F7FF;
@@ -271,6 +274,7 @@
border-radius: 8px;
cursor: pointer;
}
+ text-decoration: none;
}
}
diff --git a/website/config/routes.js b/website/config/routes.js
index bf6f330297..7814ed055c 100644
--- a/website/config/routes.js
+++ b/website/config/routes.js
@@ -522,9 +522,9 @@ module.exports.routes = {
'GET /learn-more-about/calendar-events': '/announcements/fleet-in-your-calendar-introducing-maintenance-windows',
'GET /learn-more-about/setup-windows-mdm': '/docs/using-fleet/mdm-setup',
'GET /learn-more-about/setup-abm': '/docs/using-fleet/mdm-setup#apple-business-manager-abm',
- 'GET /learn-more-about/renew-apns': '/docs/using-fleet/mdm-setup#renewing-apns',
- 'GET /learn-more-about/renew-abm': '/docs/using-fleet/mdm-macos-setup#renewing-abm',
- 'GET /learn-more-about/fleet-server-private-key': '/docs/using-fleet/fleet-server-configuration#server-private-key',
+ 'GET /learn-more-about/renew-apns': '/docs/using-fleet/mdm-setup#apple-push-notification-service-apns',
+ 'GET /learn-more-about/renew-abm': '/docs/using-fleet/mdm-setup#apple-business-manager-abm',
+ 'GET /learn-more-about/fleet-server-private-key': '/docs/configuration/fleet-server-configuration#server-private-key',
// Sitemap
// =============================================================================================================
diff --git a/website/views/pages/query-library.ejs b/website/views/pages/query-library.ejs
index 59172d0798..14f54ec040 100644
--- a/website/views/pages/query-library.ejs
+++ b/website/views/pages/query-library.ejs
@@ -98,15 +98,15 @@
object({
name = optional(string, "fleet")
security_groups = optional(list(string), [])
access_logs = optional(map(string), {})
allowed_cidrs = optional(list(string), ["0.0.0.0/0"])
allowed_ipv6_cidrs = optional(list(string), ["::/0"])
egress_cidrs = optional(list(string), ["0.0.0.0/0"])
egress_ipv6_cidrs = optional(list(string), ["::/0"])
extra_target_groups = optional(any, [])
https_listener_rules = optional(any, [])
tls_policy = optional(string, "ELBSecurityPolicy-TLS-1-2-2017-01")
idle_timeout = optional(number, 60)
}) | `{}` | no |
| [certificate\_arn](#input\_certificate\_arn) | n/a | `string` | n/a | yes |
| [ecs\_cluster](#input\_ecs\_cluster) | The config for the terraform-aws-modules/ecs/aws module | object({
autoscaling_capacity_providers = optional(any, {})
cluster_configuration = optional(any, {
execute_command_configuration = {
logging = "OVERRIDE"
log_configuration = {
cloud_watch_log_group_name = "/aws/ecs/aws-ec2"
}
}
})
cluster_name = optional(string, "fleet")
cluster_settings = optional(map(string), {
"name" : "containerInsights",
"value" : "enabled",
})
create = optional(bool, true)
default_capacity_provider_use_fargate = optional(bool, true)
fargate_capacity_providers = optional(any, {
FARGATE = {
default_capacity_provider_strategy = {
weight = 100
}
}
FARGATE_SPOT = {
default_capacity_provider_strategy = {
weight = 0
}
}
})
tags = optional(map(string))
}) | {
"autoscaling_capacity_providers": {},
"cluster_configuration": {
"execute_command_configuration": {
"log_configuration": {
"cloud_watch_log_group_name": "/aws/ecs/aws-ec2"
},
"logging": "OVERRIDE"
}
},
"cluster_name": "fleet",
"cluster_settings": {
"name": "containerInsights",
"value": "enabled"
},
"create": true,
"default_capacity_provider_use_fargate": true,
"fargate_capacity_providers": {
"FARGATE": {
"default_capacity_provider_strategy": {
"weight": 100
}
},
"FARGATE_SPOT": {
"default_capacity_provider_strategy": {
"weight": 0
}
}
},
"tags": {}
} | no |
-| [fleet\_config](#input\_fleet\_config) | The configuration object for Fleet itself. Fields that default to null will have their respective resources created if not specified. | object({
mem = optional(number, 4096)
cpu = optional(number, 512)
image = optional(string, "fleetdm/fleet:v4.50.2")
family = optional(string, "fleet")
sidecars = optional(list(any), [])
depends_on = optional(list(any), [])
mount_points = optional(list(any), [])
volumes = optional(list(any), [])
extra_environment_variables = optional(map(string), {})
extra_iam_policies = optional(list(string), [])
extra_execution_iam_policies = optional(list(string), [])
extra_secrets = optional(map(string), {})
security_groups = optional(list(string), null)
security_group_name = optional(string, "fleet")
iam_role_arn = optional(string, null)
repository_credentials = optional(string, "")
private_key_secret_name = optional(string, "fleet-server-private-key")
service = optional(object({
name = optional(string, "fleet")
}), {
name = "fleet"
})
database = optional(object({
password_secret_arn = string
user = string
database = string
address = string
rr_address = optional(string, null)
}), {
password_secret_arn = null
user = null
database = null
address = null
rr_address = null
})
redis = optional(object({
address = string
use_tls = optional(bool, true)
}), {
address = null
use_tls = true
})
awslogs = optional(object({
name = optional(string, null)
region = optional(string, null)
create = optional(bool, true)
prefix = optional(string, "fleet")
retention = optional(number, 5)
}), {
name = null
region = null
prefix = "fleet"
retention = 5
})
loadbalancer = optional(object({
arn = string
}), {
arn = null
})
extra_load_balancers = optional(list(any), [])
networking = optional(object({
subnets = list(string)
security_groups = optional(list(string), null)
}), {
subnets = null
security_groups = null
})
autoscaling = optional(object({
max_capacity = optional(number, 5)
min_capacity = optional(number, 1)
memory_tracking_target_value = optional(number, 80)
cpu_tracking_target_value = optional(number, 80)
}), {
max_capacity = 5
min_capacity = 1
memory_tracking_target_value = 80
cpu_tracking_target_value = 80
})
iam = optional(object({
role = optional(object({
name = optional(string, "fleet-role")
policy_name = optional(string, "fleet-iam-policy")
}), {
name = "fleet-role"
policy_name = "fleet-iam-policy"
})
execution = optional(object({
name = optional(string, "fleet-execution-role")
policy_name = optional(string, "fleet-execution-role")
}), {
name = "fleet-execution-role"
policy_name = "fleet-iam-policy-execution"
})
}), {
name = "fleetdm-execution-role"
})
}) | {
"autoscaling": {
"cpu_tracking_target_value": 80,
"max_capacity": 5,
"memory_tracking_target_value": 80,
"min_capacity": 1
},
"awslogs": {
"create": true,
"name": null,
"prefix": "fleet",
"region": null,
"retention": 5
},
"cpu": 256,
"database": {
"address": null,
"database": null,
"password_secret_arn": null,
"rr_address": null,
"user": null
},
"depends_on": [],
"extra_environment_variables": {},
"extra_execution_iam_policies": [],
"extra_iam_policies": [],
"extra_load_balancers": [],
"extra_secrets": {},
"family": "fleet",
"iam": {
"execution": {
"name": "fleet-execution-role",
"policy_name": "fleet-iam-policy-execution"
},
"role": {
"name": "fleet-role",
"policy_name": "fleet-iam-policy"
}
},
"iam_role_arn": null,
"image": "fleetdm/fleet:v4.31.1",
"loadbalancer": {
"arn": null
},
"mem": 512,
"mount_points": [],
"networking": {
"security_groups": null,
"subnets": null
},
"private_key_secret_name": "fleet-server-private-key",
"redis": {
"address": null,
"use_tls": true
},
"repository_credentials": "",
"security_group_name": "fleet",
"security_groups": null,
"service": {
"name": "fleet"
},
"sidecars": [],
"volumes": []
} | no |
+| [fleet\_config](#input\_fleet\_config) | The configuration object for Fleet itself. Fields that default to null will have their respective resources created if not specified. | object({
task_mem = optional(number, null)
task_cpu = optional(number, null)
mem = optional(number, 4096)
cpu = optional(number, 512)
pid_mode = optional(string, null)
image = optional(string, "fleetdm/fleet:v4.51.1")
family = optional(string, "fleet")
sidecars = optional(list(any), [])
depends_on = optional(list(any), [])
mount_points = optional(list(any), [])
volumes = optional(list(any), [])
extra_environment_variables = optional(map(string), {})
extra_iam_policies = optional(list(string), [])
extra_execution_iam_policies = optional(list(string), [])
extra_secrets = optional(map(string), {})
security_groups = optional(list(string), null)
security_group_name = optional(string, "fleet")
iam_role_arn = optional(string, null)
repository_credentials = optional(string, "")
private_key_secret_name = optional(string, "fleet-server-private-key")
service = optional(object({
name = optional(string, "fleet")
}), {
name = "fleet"
})
database = optional(object({
password_secret_arn = string
user = string
database = string
address = string
rr_address = optional(string, null)
}), {
password_secret_arn = null
user = null
database = null
address = null
rr_address = null
})
redis = optional(object({
address = string
use_tls = optional(bool, true)
}), {
address = null
use_tls = true
})
awslogs = optional(object({
name = optional(string, null)
region = optional(string, null)
create = optional(bool, true)
prefix = optional(string, "fleet")
retention = optional(number, 5)
}), {
name = null
region = null
prefix = "fleet"
retention = 5
})
loadbalancer = optional(object({
arn = string
}), {
arn = null
})
extra_load_balancers = optional(list(any), [])
networking = optional(object({
subnets = list(string)
security_groups = optional(list(string), null)
}), {
subnets = null
security_groups = null
})
autoscaling = optional(object({
max_capacity = optional(number, 5)
min_capacity = optional(number, 1)
memory_tracking_target_value = optional(number, 80)
cpu_tracking_target_value = optional(number, 80)
}), {
max_capacity = 5
min_capacity = 1
memory_tracking_target_value = 80
cpu_tracking_target_value = 80
})
iam = optional(object({
role = optional(object({
name = optional(string, "fleet-role")
policy_name = optional(string, "fleet-iam-policy")
}), {
name = "fleet-role"
policy_name = "fleet-iam-policy"
})
execution = optional(object({
name = optional(string, "fleet-execution-role")
policy_name = optional(string, "fleet-execution-role")
}), {
name = "fleet-execution-role"
policy_name = "fleet-iam-policy-execution"
})
}), {
name = "fleetdm-execution-role"
})
software_installers = optional(object({
create_bucket = optional(bool, true)
bucket_name = optional(string, null)
bucket_prefix = optional(string, "fleet-software-installers-")
s3_object_prefix = optional(string, "")
}), {
create_bucket = true
bucket_name = null
bucket_prefix = "fleet-software-installers-"
s3_object_prefix = ""
})
}) | {
"autoscaling": {
"cpu_tracking_target_value": 80,
"max_capacity": 5,
"memory_tracking_target_value": 80,
"min_capacity": 1
},
"awslogs": {
"create": true,
"name": null,
"prefix": "fleet",
"region": null,
"retention": 5
},
"cpu": 256,
"database": {
"address": null,
"database": null,
"password_secret_arn": null,
"rr_address": null,
"user": null
},
"depends_on": [],
"extra_environment_variables": {},
"extra_execution_iam_policies": [],
"extra_iam_policies": [],
"extra_load_balancers": [],
"extra_secrets": {},
"family": "fleet",
"iam": {
"execution": {
"name": "fleet-execution-role",
"policy_name": "fleet-iam-policy-execution"
},
"role": {
"name": "fleet-role",
"policy_name": "fleet-iam-policy"
}
},
"iam_role_arn": null,
"image": "fleetdm/fleet:v4.51.1",
"loadbalancer": {
"arn": null
},
"mem": 512,
"mount_points": [],
"networking": {
"security_groups": null,
"subnets": null
},
"pid_mode": null,
"private_key_secret_name": "fleet-server-private-key",
"redis": {
"address": null,
"use_tls": true
},
"repository_credentials": "",
"security_group_name": "fleet",
"security_groups": null,
"service": {
"name": "fleet"
},
"sidecars": [],
"software_installers": {
"bucket_name": null,
"bucket_prefix": "fleet-software-installers-",
"create_bucket": true,
"s3_object_prefix": ""
},
"task_cpu": null,
"task_mem": null,
"volumes": []
} | no |
| [migration\_config](#input\_migration\_config) | The configuration object for Fleet's migration task. | object({
mem = number
cpu = number
}) | {
"cpu": 1024,
"mem": 2048
} | no |
| [rds\_config](#input\_rds\_config) | The config for the terraform-aws-modules/rds-aurora/aws module | object({
name = optional(string, "fleet")
engine_version = optional(string, "8.0.mysql_aurora.3.04.2")
instance_class = optional(string, "db.t4g.large")
subnets = optional(list(string), [])
allowed_security_groups = optional(list(string), [])
allowed_cidr_blocks = optional(list(string), [])
apply_immediately = optional(bool, true)
monitoring_interval = optional(number, 10)
db_parameter_group_name = optional(string)
db_parameters = optional(map(string), {})
db_cluster_parameter_group_name = optional(string)
db_cluster_parameters = optional(map(string), {})
enabled_cloudwatch_logs_exports = optional(list(string), [])
master_username = optional(string, "fleet")
snapshot_identifier = optional(string)
cluster_tags = optional(map(string), {})
}) | {
"allowed_cidr_blocks": [],
"allowed_security_groups": [],
"apply_immediately": true,
"cluster_tags": {},
"db_cluster_parameter_group_name": null,
"db_cluster_parameters": {},
"db_parameter_group_name": null,
"db_parameters": {},
"enabled_cloudwatch_logs_exports": [],
"engine_version": "8.0.mysql_aurora.3.04.2",
"instance_class": "db.t4g.large",
"master_username": "fleet",
"monitoring_interval": 10,
"name": "fleet",
"snapshot_identifier": null,
"subnets": []
} | no |
| [redis\_config](#input\_redis\_config) | n/a | object({
name = optional(string, "fleet")
replication_group_id = optional(string)
elasticache_subnet_group_name = optional(string)
allowed_security_group_ids = optional(list(string), [])
subnets = optional(list(string))
availability_zones = optional(list(string))
cluster_size = optional(number, 3)
instance_type = optional(string, "cache.m5.large")
apply_immediately = optional(bool, true)
automatic_failover_enabled = optional(bool, false)
engine_version = optional(string, "6.x")
family = optional(string, "redis6.x")
at_rest_encryption_enabled = optional(bool, true)
transit_encryption_enabled = optional(bool, true)
parameter = optional(list(object({
name = string
value = string
})), [])
log_delivery_configuration = optional(list(map(any)), [])
tags = optional(map(string), {})
}) | {
"allowed_security_group_ids": [],
"apply_immediately": true,
"at_rest_encryption_enabled": true,
"automatic_failover_enabled": false,
"availability_zones": null,
"cluster_size": 3,
"elasticache_subnet_group_name": null,
"engine_version": "6.x",
"family": "redis6.x",
"instance_type": "cache.m5.large",
"log_delivery_configuration": [],
"name": "fleet",
"parameter": [],
"replication_group_id": null,
"subnets": null,
"tags": {},
"transit_encryption_enabled": true
} | no |
diff --git a/terraform/addons/byo-file-carving/carving/outputs.tf b/terraform/addons/byo-file-carving/carving/outputs.tf
index d3b11ff580..3622e7dfd6 100644
--- a/terraform/addons/byo-file-carving/carving/outputs.tf
+++ b/terraform/addons/byo-file-carving/carving/outputs.tf
@@ -1,10 +1,10 @@
output "fleet_extra_environment_variables" {
value = {
- FLEET_S3_STS_ASSUME_ROLE_ARN = var.iam_role_arn
- FLEET_S3_STS_EXTERNAL_ID = var.sts_external_id
- FLEET_S3_BUCKET = var.s3_bucket_name
- FLEET_S3_REGION = var.s3_bucket_region
- FLEET_S3_PREFIX = var.s3_carve_prefix
+ FLEET_S3_CARVES_STS_ASSUME_ROLE_ARN = var.iam_role_arn
+ FLEET_S3_CARVES_STS_EXTERNAL_ID = var.sts_external_id
+ FLEET_S3_CARVES_BUCKET = var.s3_bucket_name
+ FLEET_S3_CARVES_REGION = var.s3_bucket_region
+ FLEET_S3_CARVES_PREFIX = var.s3_carve_prefix
}
}
diff --git a/terraform/addons/external-vuln-scans/.terraform.lock.hcl b/terraform/addons/external-vuln-scans/.terraform.lock.hcl
index 99ef555634..f284c8030c 100644
--- a/terraform/addons/external-vuln-scans/.terraform.lock.hcl
+++ b/terraform/addons/external-vuln-scans/.terraform.lock.hcl
@@ -5,6 +5,7 @@ provider "registry.terraform.io/hashicorp/aws" {
version = "5.11.0"
hashes = [
"h1:OyEBhYcTPChBb0gooSlLIcrxakh72qAN+Sd8Oo12uoc=",
+ "h1:Wo6WCPXNnbyeRp57Jvlp7VBm9acVAAg6jVmFRU2IWjk=",
"zh:2913af44f9b584f756e5548d5ddc5a251c6d68a7fcd7c41d1418a800a94ef113",
"zh:31d2bfa84608b74ff5896f41b09e5927d7c37d18875277a51dcd75a1fea3f909",
"zh:8538ff18e3b4822178e793f06764efdbb84c62227c1051af7d2409ab7be37bfc",
diff --git a/terraform/addons/external-vuln-scans/README.md b/terraform/addons/external-vuln-scans/README.md
index 5c0e755359..b7a030a206 100644
--- a/terraform/addons/external-vuln-scans/README.md
+++ b/terraform/addons/external-vuln-scans/README.md
@@ -39,14 +39,19 @@ No modules.
| [ecs\_cluster](#input\_ecs\_cluster) | The ecs cluster module that is created by the byo-db module | `any` | n/a | yes |
| [execution\_iam\_role\_arn](#input\_execution\_iam\_role\_arn) | The ARN of the fleet execution role, this is necessary to pass role from ecs events | `any` | n/a | yes |
| [fleet\_config](#input\_fleet\_config) | The root Fleet config object | `any` | n/a | yes |
+| [fleet\_s3\_software\_installers\_config](#input\_fleet\_s3\_software\_installers\_config) | use the output of the byo-vpc module with the same name | `map(string)` | n/a | yes |
+| [fleet\_server\_private\_key\_secret\_arn](#input\_fleet\_server\_private\_key\_secret\_arn) | The ARN of the secret that stores the Fleet private key | `string` | n/a | yes |
| [security\_groups](#input\_security\_groups) | n/a | `list(string)` | n/a | yes |
| [subnets](#input\_subnets) | n/a | `list(string)` | n/a | yes |
| [task\_role\_arn](#input\_task\_role\_arn) | The ARN of the fleet task role, this is necessary to pass role from ecs events | `any` | n/a | yes |
| [vuln\_processing\_cpu](#input\_vuln\_processing\_cpu) | The amount of CPU to dedicate to the vuln processing command | `number` | `1024` | no |
| [vuln\_processing\_memory](#input\_vuln\_processing\_memory) | The amount of memory to dedicate to the vuln processing command | `number` | `4096` | no |
+| [vuln\_processing\_task\_cpu](#input\_vuln\_processing\_task\_cpu) | The amount of CPU to dedicate to the vuln processing task including sidecars | `number` | `1024` | no |
+| [vuln\_processing\_task\_memory](#input\_vuln\_processing\_task\_memory) | The amount of memory to dedicate to the vuln processing task including sidecars | `number` | `4096` | no |
## Outputs
| Name | Description |
|------|-------------|
| [extra\_environment\_variables](#output\_extra\_environment\_variables) | n/a |
+| [vuln\_service\_arn](#output\_vuln\_service\_arn) | n/a |
diff --git a/terraform/addons/external-vuln-scans/main.tf b/terraform/addons/external-vuln-scans/main.tf
index 210531823c..e38514e448 100644
--- a/terraform/addons/external-vuln-scans/main.tf
+++ b/terraform/addons/external-vuln-scans/main.tf
@@ -50,17 +50,20 @@ resource "aws_ecs_service" "fleet" {
resource "aws_ecs_task_definition" "vuln-processing" {
family = "${var.fleet_config.family}-vuln-processing"
- cpu = var.vuln_processing_cpu
- memory = var.vuln_processing_memory
+ cpu = var.vuln_processing_task_cpu
+ memory = var.vuln_processing_task_memory
execution_role_arn = var.execution_iam_role_arn
task_role_arn = var.task_role_arn
network_mode = "awsvpc"
+ pid_mode = var.fleet_config.pid_mode
requires_compatibilities = ["FARGATE"]
container_definitions = jsonencode(concat([
{
name = "fleet-vuln-processing"
image = var.fleet_config.image
+ cpu = var.vuln_processing_cpu
+ memory = var.vuln_processing_memory
essential = true
networkMode = "awsvpc"
secrets = local.secrets
@@ -109,6 +112,14 @@ resource "aws_ecs_task_definition" "vuln-processing" {
name = "FLEET_SERVER_TLS"
value = "false"
},
+ {
+ name = "FLEET_S3_SOFTWARE_INSTALLERS_BUCKET"
+ value = var.fleet_s3_software_installers_config.bucket_name
+ },
+ {
+ name = "FLEET_S3_SOFTWARE_INSTALLERS_PREFIX"
+ value = var.fleet_s3_software_installers_config.s3_object_prefix
+ },
], local.environment),
logConfiguration = {
logDriver = "awslogs"
diff --git a/terraform/addons/external-vuln-scans/outputs.tf b/terraform/addons/external-vuln-scans/outputs.tf
index 69e8da7bef..913a55da72 100644
--- a/terraform/addons/external-vuln-scans/outputs.tf
+++ b/terraform/addons/external-vuln-scans/outputs.tf
@@ -1,6 +1,6 @@
output "extra_environment_variables" {
value = {
- FLEET_VULNERABILITIES_DISABLE_SCHEDULE = "true"
+ FLEET_VULNERABILITIES_DISABLE_SCHEDULE = "true"
}
}
diff --git a/terraform/addons/external-vuln-scans/variables.tf b/terraform/addons/external-vuln-scans/variables.tf
index fbb14d86d7..03f2a4d471 100644
--- a/terraform/addons/external-vuln-scans/variables.tf
+++ b/terraform/addons/external-vuln-scans/variables.tf
@@ -31,6 +31,10 @@ variable "customer_prefix" {
default = "fleet"
}
+variable "fleet_s3_software_installers_config" {
+ type = map(string)
+ description = "use the output of the byo-vpc module with the same name"
+}
variable "fleet_server_private_key_secret_arn" {
type = string
@@ -45,6 +49,19 @@ variable "task_role_arn" {
description = "The ARN of the fleet task role, this is necessary to pass role from ecs events"
}
+variable "vuln_processing_task_memory" {
+ // note must conform to FARGATE breakpoints https://docs.aws.amazon.com/AmazonECS/latest/userguide/fargate-task-defs.html
+ default = 4096
+ description = "The amount of memory to dedicate to the vuln processing task including sidecars"
+}
+
+variable "vuln_processing_task_cpu" {
+ // note must conform to FARGETE breakpoints https://docs.aws.amazon.com/AmazonECS/latest/userguide/fargate-task-defs.html
+ default = 1024
+ description = "The amount of CPU to dedicate to the vuln processing task including sidecars"
+}
+
+
variable "vuln_processing_memory" {
// note must conform to FARGATE breakpoints https://docs.aws.amazon.com/AmazonECS/latest/userguide/fargate-task-defs.html
default = 4096
diff --git a/terraform/addons/osquery-carve/outputs.tf b/terraform/addons/osquery-carve/outputs.tf
index 0b488ccc10..c62c013d10 100644
--- a/terraform/addons/osquery-carve/outputs.tf
+++ b/terraform/addons/osquery-carve/outputs.tf
@@ -1,7 +1,7 @@
output "fleet_extra_environment_variables" {
value = {
- FLEET_S3_BUCKET = aws_s3_bucket.main.bucket
- FLEET_S3_PREFIX = "carve_results/"
+ FLEET_S3_CARVES_BUCKET = aws_s3_bucket.main.bucket
+ FLEET_S3_CARVES_PREFIX = "carve_results/"
}
}
diff --git a/terraform/addons/vuln-processing/variables.tf b/terraform/addons/vuln-processing/variables.tf
index 69dd445fdf..93ff50d824 100644
--- a/terraform/addons/vuln-processing/variables.tf
+++ b/terraform/addons/vuln-processing/variables.tf
@@ -24,7 +24,7 @@ variable "fleet_config" {
vuln_processing_cpu = optional(number, 2048)
vuln_data_stream_mem = optional(number, 1024)
vuln_data_stream_cpu = optional(number, 512)
- image = optional(string, "fleetdm/fleet:v4.31.1")
+ image = optional(string, "fleetdm/fleet:v4.51.1")
family = optional(string, "fleet-vuln-processing")
sidecars = optional(list(any), [])
extra_environment_variables = optional(map(string), {})
@@ -82,7 +82,7 @@ variable "fleet_config" {
vuln_processing_cpu = 2048
vuln_data_stream_mem = 1024
vuln_data_stream_cpu = 512
- image = "fleetdm/fleet:v4.31.1"
+ image = "fleetdm/fleet:v4.51.1"
family = "fleet-vuln-processing"
sidecars = []
extra_environment_variables = {}
diff --git a/terraform/byo-vpc/README.md b/terraform/byo-vpc/README.md
index 0b6e6a5c0b..591056fab5 100644
--- a/terraform/byo-vpc/README.md
+++ b/terraform/byo-vpc/README.md
@@ -33,7 +33,7 @@ No requirements.
|------|-------------|------|---------|:--------:|
| [alb\_config](#input\_alb\_config) | n/a | object({
name = optional(string, "fleet")
subnets = list(string)
security_groups = optional(list(string), [])
access_logs = optional(map(string), {})
certificate_arn = string
allowed_cidrs = optional(list(string), ["0.0.0.0/0"])
allowed_ipv6_cidrs = optional(list(string), ["::/0"])
egress_cidrs = optional(list(string), ["0.0.0.0/0"])
egress_ipv6_cidrs = optional(list(string), ["::/0"])
extra_target_groups = optional(any, [])
https_listener_rules = optional(any, [])
tls_policy = optional(string, "ELBSecurityPolicy-TLS-1-2-2017-01")
idle_timeout = optional(number, 60)
}) | n/a | yes |
| [ecs\_cluster](#input\_ecs\_cluster) | The config for the terraform-aws-modules/ecs/aws module | object({
autoscaling_capacity_providers = optional(any, {})
cluster_configuration = optional(any, {
execute_command_configuration = {
logging = "OVERRIDE"
log_configuration = {
cloud_watch_log_group_name = "/aws/ecs/aws-ec2"
}
}
})
cluster_name = optional(string, "fleet")
cluster_settings = optional(map(string), {
"name" : "containerInsights",
"value" : "enabled",
})
create = optional(bool, true)
default_capacity_provider_use_fargate = optional(bool, true)
fargate_capacity_providers = optional(any, {
FARGATE = {
default_capacity_provider_strategy = {
weight = 100
}
}
FARGATE_SPOT = {
default_capacity_provider_strategy = {
weight = 0
}
}
})
tags = optional(map(string))
}) | {
"autoscaling_capacity_providers": {},
"cluster_configuration": {
"execute_command_configuration": {
"log_configuration": {
"cloud_watch_log_group_name": "/aws/ecs/aws-ec2"
},
"logging": "OVERRIDE"
}
},
"cluster_name": "fleet",
"cluster_settings": {
"name": "containerInsights",
"value": "enabled"
},
"create": true,
"default_capacity_provider_use_fargate": true,
"fargate_capacity_providers": {
"FARGATE": {
"default_capacity_provider_strategy": {
"weight": 100
}
},
"FARGATE_SPOT": {
"default_capacity_provider_strategy": {
"weight": 0
}
}
},
"tags": {}
} | no |
-| [fleet\_config](#input\_fleet\_config) | The configuration object for Fleet itself. Fields that default to null will have their respective resources created if not specified. | object({
mem = optional(number, 4096)
cpu = optional(number, 512)
image = optional(string, "fleetdm/fleet:v4.50.2")
family = optional(string, "fleet")
sidecars = optional(list(any), [])
depends_on = optional(list(any), [])
mount_points = optional(list(any), [])
volumes = optional(list(any), [])
extra_environment_variables = optional(map(string), {})
extra_iam_policies = optional(list(string), [])
extra_execution_iam_policies = optional(list(string), [])
extra_secrets = optional(map(string), {})
security_groups = optional(list(string), null)
security_group_name = optional(string, "fleet")
iam_role_arn = optional(string, null)
repository_credentials = optional(string, "")
private_key_secret_name = optional(string, "fleet-server-private-key")
service = optional(object({
name = optional(string, "fleet")
}), {
name = "fleet"
})
database = optional(object({
password_secret_arn = string
user = string
database = string
address = string
rr_address = optional(string, null)
}), {
password_secret_arn = null
user = null
database = null
address = null
rr_address = null
})
redis = optional(object({
address = string
use_tls = optional(bool, true)
}), {
address = null
use_tls = true
})
awslogs = optional(object({
name = optional(string, null)
region = optional(string, null)
create = optional(bool, true)
prefix = optional(string, "fleet")
retention = optional(number, 5)
}), {
name = null
region = null
prefix = "fleet"
retention = 5
})
loadbalancer = optional(object({
arn = string
}), {
arn = null
})
extra_load_balancers = optional(list(any), [])
networking = optional(object({
subnets = list(string)
security_groups = optional(list(string), null)
}), {
subnets = null
security_groups = null
})
autoscaling = optional(object({
max_capacity = optional(number, 5)
min_capacity = optional(number, 1)
memory_tracking_target_value = optional(number, 80)
cpu_tracking_target_value = optional(number, 80)
}), {
max_capacity = 5
min_capacity = 1
memory_tracking_target_value = 80
cpu_tracking_target_value = 80
})
iam = optional(object({
role = optional(object({
name = optional(string, "fleet-role")
policy_name = optional(string, "fleet-iam-policy")
}), {
name = "fleet-role"
policy_name = "fleet-iam-policy"
})
execution = optional(object({
name = optional(string, "fleet-execution-role")
policy_name = optional(string, "fleet-execution-role")
}), {
name = "fleet-execution-role"
policy_name = "fleet-iam-policy-execution"
})
}), {
name = "fleetdm-execution-role"
})
}) | {
"autoscaling": {
"cpu_tracking_target_value": 80,
"max_capacity": 5,
"memory_tracking_target_value": 80,
"min_capacity": 1
},
"awslogs": {
"create": true,
"name": null,
"prefix": "fleet",
"region": null,
"retention": 5
},
"cpu": 256,
"database": {
"address": null,
"database": null,
"password_secret_arn": null,
"rr_address": null,
"user": null
},
"depends_on": [],
"extra_environment_variables": {},
"extra_execution_iam_policies": [],
"extra_iam_policies": [],
"extra_load_balancers": [],
"extra_secrets": {},
"family": "fleet",
"iam": {
"execution": {
"name": "fleet-execution-role",
"policy_name": "fleet-iam-policy-execution"
},
"role": {
"name": "fleet-role",
"policy_name": "fleet-iam-policy"
}
},
"iam_role_arn": null,
"image": "fleetdm/fleet:v4.31.1",
"loadbalancer": {
"arn": null
},
"mem": 512,
"mount_points": [],
"networking": {
"security_groups": null,
"subnets": null
},
"private_key_secret_name": "fleet-server-private-key",
"redis": {
"address": null,
"use_tls": true
},
"repository_credentials": "",
"security_group_name": "fleet",
"security_groups": null,
"service": {
"name": "fleet"
},
"sidecars": [],
"volumes": []
} | no |
+| [fleet\_config](#input\_fleet\_config) | The configuration object for Fleet itself. Fields that default to null will have their respective resources created if not specified. | object({
task_mem = optional(number, null)
task_cpu = optional(number, null)
mem = optional(number, 4096)
cpu = optional(number, 512)
pid_mode = optional(string, null)
image = optional(string, "fleetdm/fleet:v4.51.1")
family = optional(string, "fleet")
sidecars = optional(list(any), [])
depends_on = optional(list(any), [])
mount_points = optional(list(any), [])
volumes = optional(list(any), [])
extra_environment_variables = optional(map(string), {})
extra_iam_policies = optional(list(string), [])
extra_execution_iam_policies = optional(list(string), [])
extra_secrets = optional(map(string), {})
security_groups = optional(list(string), null)
security_group_name = optional(string, "fleet")
iam_role_arn = optional(string, null)
repository_credentials = optional(string, "")
private_key_secret_name = optional(string, "fleet-server-private-key")
service = optional(object({
name = optional(string, "fleet")
}), {
name = "fleet"
})
database = optional(object({
password_secret_arn = string
user = string
database = string
address = string
rr_address = optional(string, null)
}), {
password_secret_arn = null
user = null
database = null
address = null
rr_address = null
})
redis = optional(object({
address = string
use_tls = optional(bool, true)
}), {
address = null
use_tls = true
})
awslogs = optional(object({
name = optional(string, null)
region = optional(string, null)
create = optional(bool, true)
prefix = optional(string, "fleet")
retention = optional(number, 5)
}), {
name = null
region = null
prefix = "fleet"
retention = 5
})
loadbalancer = optional(object({
arn = string
}), {
arn = null
})
extra_load_balancers = optional(list(any), [])
networking = optional(object({
subnets = list(string)
security_groups = optional(list(string), null)
}), {
subnets = null
security_groups = null
})
autoscaling = optional(object({
max_capacity = optional(number, 5)
min_capacity = optional(number, 1)
memory_tracking_target_value = optional(number, 80)
cpu_tracking_target_value = optional(number, 80)
}), {
max_capacity = 5
min_capacity = 1
memory_tracking_target_value = 80
cpu_tracking_target_value = 80
})
iam = optional(object({
role = optional(object({
name = optional(string, "fleet-role")
policy_name = optional(string, "fleet-iam-policy")
}), {
name = "fleet-role"
policy_name = "fleet-iam-policy"
})
execution = optional(object({
name = optional(string, "fleet-execution-role")
policy_name = optional(string, "fleet-execution-role")
}), {
name = "fleet-execution-role"
policy_name = "fleet-iam-policy-execution"
})
}), {
name = "fleetdm-execution-role"
})
software_installers = optional(object({
create_bucket = optional(bool, true)
bucket_name = optional(string, null)
bucket_prefix = optional(string, "fleet-software-installers-")
s3_object_prefix = optional(string, "")
}), {
create_bucket = true
bucket_name = null
bucket_prefix = "fleet-software-installers-"
s3_object_prefix = ""
})
}) | {
"autoscaling": {
"cpu_tracking_target_value": 80,
"max_capacity": 5,
"memory_tracking_target_value": 80,
"min_capacity": 1
},
"awslogs": {
"create": true,
"name": null,
"prefix": "fleet",
"region": null,
"retention": 5
},
"cpu": 256,
"database": {
"address": null,
"database": null,
"password_secret_arn": null,
"rr_address": null,
"user": null
},
"depends_on": [],
"extra_environment_variables": {},
"extra_execution_iam_policies": [],
"extra_iam_policies": [],
"extra_load_balancers": [],
"extra_secrets": {},
"family": "fleet",
"iam": {
"execution": {
"name": "fleet-execution-role",
"policy_name": "fleet-iam-policy-execution"
},
"role": {
"name": "fleet-role",
"policy_name": "fleet-iam-policy"
}
},
"iam_role_arn": null,
"image": "fleetdm/fleet:v4.51.1",
"loadbalancer": {
"arn": null
},
"mem": 512,
"mount_points": [],
"networking": {
"security_groups": null,
"subnets": null
},
"pid_mode": null,
"private_key_secret_name": "fleet-server-private-key",
"redis": {
"address": null,
"use_tls": true
},
"repository_credentials": "",
"security_group_name": "fleet",
"security_groups": null,
"service": {
"name": "fleet"
},
"sidecars": [],
"software_installers": {
"bucket_name": null,
"bucket_prefix": "fleet-software-installers-",
"create_bucket": true,
"s3_object_prefix": ""
},
"task_cpu": null,
"task_mem": null,
"volumes": []
} | no |
| [migration\_config](#input\_migration\_config) | The configuration object for Fleet's migration task. | object({
mem = number
cpu = number
}) | {
"cpu": 1024,
"mem": 2048
} | no |
| [rds\_config](#input\_rds\_config) | The config for the terraform-aws-modules/rds-aurora/aws module | object({
name = optional(string, "fleet")
engine_version = optional(string, "8.0.mysql_aurora.3.04.2")
instance_class = optional(string, "db.t4g.large")
subnets = optional(list(string), [])
allowed_security_groups = optional(list(string), [])
allowed_cidr_blocks = optional(list(string), [])
apply_immediately = optional(bool, true)
monitoring_interval = optional(number, 10)
db_parameter_group_name = optional(string)
db_parameters = optional(map(string), {})
db_cluster_parameter_group_name = optional(string)
db_cluster_parameters = optional(map(string), {})
enabled_cloudwatch_logs_exports = optional(list(string), [])
master_username = optional(string, "fleet")
snapshot_identifier = optional(string)
cluster_tags = optional(map(string), {})
preferred_maintenance_window = optional(string, "thu:23:00-fri:00:00")
}) | {
"allowed_cidr_blocks": [],
"allowed_security_groups": [],
"apply_immediately": true,
"cluster_tags": {},
"db_cluster_parameter_group_name": null,
"db_cluster_parameters": {},
"db_parameter_group_name": null,
"db_parameters": {},
"enabled_cloudwatch_logs_exports": [],
"engine_version": "8.0.mysql_aurora.3.04.2",
"instance_class": "db.t4g.large",
"master_username": "fleet",
"monitoring_interval": 10,
"name": "fleet",
"preferred_maintenance_window": "thu:23:00-fri:00:00",
"snapshot_identifier": null,
"subnets": []
} | no |
| [redis\_config](#input\_redis\_config) | n/a | object({
name = optional(string, "fleet")
replication_group_id = optional(string)
elasticache_subnet_group_name = optional(string, "")
allowed_security_group_ids = optional(list(string), [])
subnets = list(string)
allowed_cidrs = list(string)
availability_zones = optional(list(string), [])
cluster_size = optional(number, 3)
instance_type = optional(string, "cache.m5.large")
apply_immediately = optional(bool, true)
automatic_failover_enabled = optional(bool, false)
engine_version = optional(string, "6.x")
family = optional(string, "redis6.x")
at_rest_encryption_enabled = optional(bool, true)
transit_encryption_enabled = optional(bool, true)
parameter = optional(list(object({
name = string
value = string
})), [])
log_delivery_configuration = optional(list(map(any)), [])
tags = optional(map(string), {})
}) | {
"allowed_cidrs": null,
"allowed_security_group_ids": [],
"apply_immediately": true,
"at_rest_encryption_enabled": true,
"automatic_failover_enabled": false,
"availability_zones": [],
"cluster_size": 3,
"elasticache_subnet_group_name": "",
"engine_version": "6.x",
"family": "redis6.x",
"instance_type": "cache.m5.large",
"log_delivery_configuration": [],
"name": "fleet",
"parameter": [],
"replication_group_id": null,
"subnets": null,
"tags": {},
"transit_encryption_enabled": true
} | no |
diff --git a/terraform/byo-vpc/byo-db/README.md b/terraform/byo-vpc/byo-db/README.md
index c1c0e0f820..b65261fa5d 100644
--- a/terraform/byo-vpc/byo-db/README.md
+++ b/terraform/byo-vpc/byo-db/README.md
@@ -28,7 +28,7 @@ No requirements.
|------|-------------|------|---------|:--------:|
| [alb\_config](#input\_alb\_config) | n/a | object({
name = optional(string, "fleet")
subnets = list(string)
security_groups = optional(list(string), [])
access_logs = optional(map(string), {})
certificate_arn = string
allowed_cidrs = optional(list(string), ["0.0.0.0/0"])
allowed_ipv6_cidrs = optional(list(string), ["::/0"])
egress_cidrs = optional(list(string), ["0.0.0.0/0"])
egress_ipv6_cidrs = optional(list(string), ["::/0"])
extra_target_groups = optional(any, [])
https_listener_rules = optional(any, [])
tls_policy = optional(string, "ELBSecurityPolicy-TLS-1-2-2017-01")
idle_timeout = optional(number, 60)
}) | n/a | yes |
| [ecs\_cluster](#input\_ecs\_cluster) | The config for the terraform-aws-modules/ecs/aws module | object({
autoscaling_capacity_providers = optional(any, {})
cluster_configuration = optional(any, {
execute_command_configuration = {
logging = "OVERRIDE"
log_configuration = {
cloud_watch_log_group_name = "/aws/ecs/aws-ec2"
}
}
})
cluster_name = optional(string, "fleet")
cluster_settings = optional(map(string), {
"name" : "containerInsights",
"value" : "enabled",
})
create = optional(bool, true)
default_capacity_provider_use_fargate = optional(bool, true)
fargate_capacity_providers = optional(any, {
FARGATE = {
default_capacity_provider_strategy = {
weight = 100
}
}
FARGATE_SPOT = {
default_capacity_provider_strategy = {
weight = 0
}
}
})
tags = optional(map(string))
}) | {
"autoscaling_capacity_providers": {},
"cluster_configuration": {
"execute_command_configuration": {
"log_configuration": {
"cloud_watch_log_group_name": "/aws/ecs/aws-ec2"
},
"logging": "OVERRIDE"
}
},
"cluster_name": "fleet",
"cluster_settings": {
"name": "containerInsights",
"value": "enabled"
},
"create": true,
"default_capacity_provider_use_fargate": true,
"fargate_capacity_providers": {
"FARGATE": {
"default_capacity_provider_strategy": {
"weight": 100
}
},
"FARGATE_SPOT": {
"default_capacity_provider_strategy": {
"weight": 0
}
}
},
"tags": {}
} | no |
-| [fleet\_config](#input\_fleet\_config) | The configuration object for Fleet itself. Fields that default to null will have their respective resources created if not specified. | object({
mem = optional(number, 4096)
cpu = optional(number, 512)
image = optional(string, "fleetdm/fleet:v4.50.2")
family = optional(string, "fleet")
sidecars = optional(list(any), [])
depends_on = optional(list(any), [])
mount_points = optional(list(any), [])
volumes = optional(list(any), [])
extra_environment_variables = optional(map(string), {})
extra_iam_policies = optional(list(string), [])
extra_execution_iam_policies = optional(list(string), [])
extra_secrets = optional(map(string), {})
security_groups = optional(list(string), null)
security_group_name = optional(string, "fleet")
iam_role_arn = optional(string, null)
repository_credentials = optional(string, "")
private_key_secret_name = optional(string, "fleet-server-private-key")
service = optional(object({
name = optional(string, "fleet")
}), {
name = "fleet"
})
database = optional(object({
password_secret_arn = string
user = string
database = string
address = string
rr_address = optional(string, null)
}), {
password_secret_arn = null
user = null
database = null
address = null
rr_address = null
})
redis = optional(object({
address = string
use_tls = optional(bool, true)
}), {
address = null
use_tls = true
})
awslogs = optional(object({
name = optional(string, null)
region = optional(string, null)
create = optional(bool, true)
prefix = optional(string, "fleet")
retention = optional(number, 5)
}), {
name = null
region = null
prefix = "fleet"
retention = 5
})
loadbalancer = optional(object({
arn = string
}), {
arn = null
})
extra_load_balancers = optional(list(any), [])
networking = optional(object({
subnets = list(string)
security_groups = optional(list(string), null)
}), {
subnets = null
security_groups = null
})
autoscaling = optional(object({
max_capacity = optional(number, 5)
min_capacity = optional(number, 1)
memory_tracking_target_value = optional(number, 80)
cpu_tracking_target_value = optional(number, 80)
}), {
max_capacity = 5
min_capacity = 1
memory_tracking_target_value = 80
cpu_tracking_target_value = 80
})
iam = optional(object({
role = optional(object({
name = optional(string, "fleet-role")
policy_name = optional(string, "fleet-iam-policy")
}), {
name = "fleet-role"
policy_name = "fleet-iam-policy"
})
execution = optional(object({
name = optional(string, "fleet-execution-role")
policy_name = optional(string, "fleet-execution-role")
}), {
name = "fleet-execution-role"
policy_name = "fleet-iam-policy-execution"
})
}), {
name = "fleetdm-execution-role"
})
}) | {
"autoscaling": {
"cpu_tracking_target_value": 80,
"max_capacity": 5,
"memory_tracking_target_value": 80,
"min_capacity": 1
},
"awslogs": {
"create": true,
"name": null,
"prefix": "fleet",
"region": null,
"retention": 5
},
"cpu": 256,
"database": {
"address": null,
"database": null,
"password_secret_arn": null,
"rr_address": null,
"user": null
},
"depends_on": [],
"extra_environment_variables": {},
"extra_execution_iam_policies": [],
"extra_iam_policies": [],
"extra_load_balancers": [],
"extra_secrets": {},
"family": "fleet",
"iam": {
"execution": {
"name": "fleet-execution-role",
"policy_name": "fleet-iam-policy-execution"
},
"role": {
"name": "fleet-role",
"policy_name": "fleet-iam-policy"
}
},
"iam_role_arn": null,
"image": "fleetdm/fleet:v4.31.1",
"loadbalancer": {
"arn": null
},
"mem": 512,
"mount_points": [],
"networking": {
"security_groups": null,
"subnets": null
},
"private_key_secret_name": "fleet-server-private-key",
"redis": {
"address": null,
"use_tls": true
},
"repository_credentials": "",
"security_group_name": "fleet",
"security_groups": null,
"service": {
"name": "fleet"
},
"sidecars": [],
"volumes": []
} | no |
+| [fleet\_config](#input\_fleet\_config) | The configuration object for Fleet itself. Fields that default to null will have their respective resources created if not specified. | object({
task_mem = optional(number, null)
task_cpu = optional(number, null)
mem = optional(number, 4096)
cpu = optional(number, 512)
pid_mode = optional(string, null)
image = optional(string, "fleetdm/fleet:v4.51.1")
family = optional(string, "fleet")
sidecars = optional(list(any), [])
depends_on = optional(list(any), [])
mount_points = optional(list(any), [])
volumes = optional(list(any), [])
extra_environment_variables = optional(map(string), {})
extra_iam_policies = optional(list(string), [])
extra_execution_iam_policies = optional(list(string), [])
extra_secrets = optional(map(string), {})
security_groups = optional(list(string), null)
security_group_name = optional(string, "fleet")
iam_role_arn = optional(string, null)
repository_credentials = optional(string, "")
private_key_secret_name = optional(string, "fleet-server-private-key")
service = optional(object({
name = optional(string, "fleet")
}), {
name = "fleet"
})
database = optional(object({
password_secret_arn = string
user = string
database = string
address = string
rr_address = optional(string, null)
}), {
password_secret_arn = null
user = null
database = null
address = null
rr_address = null
})
redis = optional(object({
address = string
use_tls = optional(bool, true)
}), {
address = null
use_tls = true
})
awslogs = optional(object({
name = optional(string, null)
region = optional(string, null)
create = optional(bool, true)
prefix = optional(string, "fleet")
retention = optional(number, 5)
}), {
name = null
region = null
prefix = "fleet"
retention = 5
})
loadbalancer = optional(object({
arn = string
}), {
arn = null
})
extra_load_balancers = optional(list(any), [])
networking = optional(object({
subnets = list(string)
security_groups = optional(list(string), null)
}), {
subnets = null
security_groups = null
})
autoscaling = optional(object({
max_capacity = optional(number, 5)
min_capacity = optional(number, 1)
memory_tracking_target_value = optional(number, 80)
cpu_tracking_target_value = optional(number, 80)
}), {
max_capacity = 5
min_capacity = 1
memory_tracking_target_value = 80
cpu_tracking_target_value = 80
})
iam = optional(object({
role = optional(object({
name = optional(string, "fleet-role")
policy_name = optional(string, "fleet-iam-policy")
}), {
name = "fleet-role"
policy_name = "fleet-iam-policy"
})
execution = optional(object({
name = optional(string, "fleet-execution-role")
policy_name = optional(string, "fleet-execution-role")
}), {
name = "fleet-execution-role"
policy_name = "fleet-iam-policy-execution"
})
}), {
name = "fleetdm-execution-role"
})
software_installers = optional(object({
create_bucket = optional(bool, true)
bucket_name = optional(string, null)
bucket_prefix = optional(string, "fleet-software-installers-")
s3_object_prefix = optional(string, "")
}), {
create_bucket = true
bucket_name = null
bucket_prefix = "fleet-software-installers-"
s3_object_prefix = ""
})
}) | {
"autoscaling": {
"cpu_tracking_target_value": 80,
"max_capacity": 5,
"memory_tracking_target_value": 80,
"min_capacity": 1
},
"awslogs": {
"create": true,
"name": null,
"prefix": "fleet",
"region": null,
"retention": 5
},
"cpu": 256,
"database": {
"address": null,
"database": null,
"password_secret_arn": null,
"rr_address": null,
"user": null
},
"depends_on": [],
"extra_environment_variables": {},
"extra_execution_iam_policies": [],
"extra_iam_policies": [],
"extra_load_balancers": [],
"extra_secrets": {},
"family": "fleet",
"iam": {
"execution": {
"name": "fleet-execution-role",
"policy_name": "fleet-iam-policy-execution"
},
"role": {
"name": "fleet-role",
"policy_name": "fleet-iam-policy"
}
},
"iam_role_arn": null,
"image": "fleetdm/fleet:v4.51.1",
"loadbalancer": {
"arn": null
},
"mem": 512,
"mount_points": [],
"networking": {
"security_groups": null,
"subnets": null
},
"pid_mode": null,
"private_key_secret_name": "fleet-server-private-key",
"redis": {
"address": null,
"use_tls": true
},
"repository_credentials": "",
"security_group_name": "fleet",
"security_groups": null,
"service": {
"name": "fleet"
},
"sidecars": [],
"software_installers": {
"bucket_name": null,
"bucket_prefix": "fleet-software-installers-",
"create_bucket": true,
"s3_object_prefix": ""
},
"task_cpu": null,
"task_mem": null,
"volumes": []
} | no |
| [migration\_config](#input\_migration\_config) | The configuration object for Fleet's migration task. | object({
mem = number
cpu = number
}) | {
"cpu": 1024,
"mem": 2048
} | no |
| [vpc\_id](#input\_vpc\_id) | n/a | `string` | n/a | yes |
diff --git a/terraform/byo-vpc/byo-db/byo-ecs/README.md b/terraform/byo-vpc/byo-db/byo-ecs/README.md
index b8bae183ae..2055467628 100644
--- a/terraform/byo-vpc/byo-db/byo-ecs/README.md
+++ b/terraform/byo-vpc/byo-db/byo-ecs/README.md
@@ -25,6 +25,7 @@ No modules.
| [aws_ecs_task_definition.backend](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_task_definition) | resource |
| [aws_iam_policy.execution](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.main](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
+| [aws_iam_policy.software_installers](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_role.execution](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
| [aws_iam_role.main](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
| [aws_iam_role_policy_attachment.execution](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
@@ -32,6 +33,10 @@ No modules.
| [aws_iam_role_policy_attachment.extras](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.main](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.role_attachment](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
+| [aws_iam_role_policy_attachment.software_installers](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
+| [aws_s3_bucket.software_installers](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) | resource |
+| [aws_s3_bucket_public_access_block.software_installers](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_public_access_block) | resource |
+| [aws_s3_bucket_server_side_encryption_configuration.software_installers](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_server_side_encryption_configuration) | resource |
| [aws_secretsmanager_secret.fleet_server_private_key](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/secretsmanager_secret) | resource |
| [aws_secretsmanager_secret_version.fleet_server_private_key](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/secretsmanager_secret_version) | resource |
| [aws_security_group.main](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
@@ -39,6 +44,7 @@ No modules.
| [aws_iam_policy_document.assume_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.fleet](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.fleet-execution](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
+| [aws_iam_policy_document.software_installers](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
## Inputs
@@ -46,7 +52,7 @@ No modules.
| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| [ecs\_cluster](#input\_ecs\_cluster) | The name of the ECS cluster to use | `string` | n/a | yes |
-| [fleet\_config](#input\_fleet\_config) | The configuration object for Fleet itself. Fields that default to null will have their respective resources created if not specified. | object({
mem = optional(number, 4096)
cpu = optional(number, 512)
image = optional(string, "fleetdm/fleet:v4.50.2")
family = optional(string, "fleet")
sidecars = optional(list(any), [])
depends_on = optional(list(any), [])
mount_points = optional(list(any), [])
volumes = optional(list(any), [])
extra_environment_variables = optional(map(string), {})
extra_iam_policies = optional(list(string), [])
extra_execution_iam_policies = optional(list(string), [])
extra_secrets = optional(map(string), {})
security_groups = optional(list(string), null)
security_group_name = optional(string, "fleet")
iam_role_arn = optional(string, null)
repository_credentials = optional(string, "")
private_key_secret_name = optional(string, "fleet-server-private-key")
service = optional(object({
name = optional(string, "fleet")
}), {
name = "fleet"
})
database = object({
password_secret_arn = string
user = string
database = string
address = string
rr_address = optional(string, null)
})
redis = object({
address = string
use_tls = optional(bool, true)
})
awslogs = optional(object({
name = optional(string, null)
region = optional(string, null)
create = optional(bool, true)
prefix = optional(string, "fleet")
retention = optional(number, 5)
}), {
name = null
region = null
prefix = "fleet"
retention = 5
})
loadbalancer = object({
arn = string
})
extra_load_balancers = optional(list(any), [])
networking = object({
subnets = list(string)
security_groups = optional(list(string), null)
})
autoscaling = optional(object({
max_capacity = optional(number, 5)
min_capacity = optional(number, 1)
memory_tracking_target_value = optional(number, 80)
cpu_tracking_target_value = optional(number, 80)
}), {
max_capacity = 5
min_capacity = 1
memory_tracking_target_value = 80
cpu_tracking_target_value = 80
})
iam = optional(object({
role = optional(object({
name = optional(string, "fleet-role")
policy_name = optional(string, "fleet-iam-policy")
}), {
name = "fleet-role"
policy_name = "fleet-iam-policy"
})
execution = optional(object({
name = optional(string, "fleet-execution-role")
policy_name = optional(string, "fleet-execution-role")
}), {
name = "fleet-execution-role"
policy_name = "fleet-iam-policy-execution"
})
}), {
name = "fleetdm-execution-role"
})
}) | {
"autoscaling": {
"cpu_tracking_target_value": 80,
"max_capacity": 5,
"memory_tracking_target_value": 80,
"min_capacity": 1
},
"awslogs": {
"create": true,
"name": null,
"prefix": "fleet",
"region": null,
"retention": 5
},
"cpu": 256,
"database": {
"address": null,
"database": null,
"password_secret_arn": null,
"rr_address": null,
"user": null
},
"depends_on": [],
"extra_environment_variables": {},
"extra_execution_iam_policies": [],
"extra_iam_policies": [],
"extra_load_balacners": [],
"extra_secrets": {},
"family": "fleet",
"iam": {
"execution": {
"name": "fleet-execution-role",
"policy_name": "fleet-iam-policy-execution"
},
"role": {
"name": "fleet-role",
"policy_name": "fleet-iam-policy"
}
},
"iam_role_arn": null,
"image": "fleetdm/fleet:v4.31.1",
"loadbalancer": {
"arn": null
},
"mem": 512,
"mount_points": [],
"networking": {
"security_groups": null,
"subnets": null
},
"private_key_secret_name": "fleet-server-private-key",
"redis": {
"address": null,
"use_tls": true
},
"repository_credentials": "",
"security_group_name": "fleet",
"security_groups": null,
"service": {
"name": "fleet"
},
"sidecars": [],
"volumes": []
} | no |
+| [fleet\_config](#input\_fleet\_config) | The configuration object for Fleet itself. Fields that default to null will have their respective resources created if not specified. | object({
task_mem = optional(number, null)
task_cpu = optional(number, null)
mem = optional(number, 4096)
cpu = optional(number, 512)
pid_mode = optional(string, null)
image = optional(string, "fleetdm/fleet:v4.51.1")
family = optional(string, "fleet")
sidecars = optional(list(any), [])
depends_on = optional(list(any), [])
mount_points = optional(list(any), [])
volumes = optional(list(any), [])
extra_environment_variables = optional(map(string), {})
extra_iam_policies = optional(list(string), [])
extra_execution_iam_policies = optional(list(string), [])
extra_secrets = optional(map(string), {})
security_groups = optional(list(string), null)
security_group_name = optional(string, "fleet")
iam_role_arn = optional(string, null)
repository_credentials = optional(string, "")
private_key_secret_name = optional(string, "fleet-server-private-key")
service = optional(object({
name = optional(string, "fleet")
}), {
name = "fleet"
})
database = object({
password_secret_arn = string
user = string
database = string
address = string
rr_address = optional(string, null)
})
redis = object({
address = string
use_tls = optional(bool, true)
})
awslogs = optional(object({
name = optional(string, null)
region = optional(string, null)
create = optional(bool, true)
prefix = optional(string, "fleet")
retention = optional(number, 5)
}), {
name = null
region = null
prefix = "fleet"
retention = 5
})
loadbalancer = object({
arn = string
})
extra_load_balancers = optional(list(any), [])
networking = object({
subnets = list(string)
security_groups = optional(list(string), null)
})
autoscaling = optional(object({
max_capacity = optional(number, 5)
min_capacity = optional(number, 1)
memory_tracking_target_value = optional(number, 80)
cpu_tracking_target_value = optional(number, 80)
}), {
max_capacity = 5
min_capacity = 1
memory_tracking_target_value = 80
cpu_tracking_target_value = 80
})
iam = optional(object({
role = optional(object({
name = optional(string, "fleet-role")
policy_name = optional(string, "fleet-iam-policy")
}), {
name = "fleet-role"
policy_name = "fleet-iam-policy"
})
execution = optional(object({
name = optional(string, "fleet-execution-role")
policy_name = optional(string, "fleet-execution-role")
}), {
name = "fleet-execution-role"
policy_name = "fleet-iam-policy-execution"
})
}), {
name = "fleetdm-execution-role"
})
software_installers = optional(object({
create_bucket = optional(bool, true)
bucket_name = optional(string, null)
bucket_prefix = optional(string, "fleet-software-installers-")
s3_object_prefix = optional(string, "")
}), {
create_bucket = true
bucket_name = null
bucket_prefix = "fleet-software-installers-"
s3_object_prefix = ""
})
}) | {
"autoscaling": {
"cpu_tracking_target_value": 80,
"max_capacity": 5,
"memory_tracking_target_value": 80,
"min_capacity": 1
},
"awslogs": {
"create": true,
"name": null,
"prefix": "fleet",
"region": null,
"retention": 5
},
"cpu": 256,
"database": {
"address": null,
"database": null,
"password_secret_arn": null,
"rr_address": null,
"user": null
},
"depends_on": [],
"extra_environment_variables": {},
"extra_execution_iam_policies": [],
"extra_iam_policies": [],
"extra_load_balacners": [],
"extra_secrets": {},
"family": "fleet",
"iam": {
"execution": {
"name": "fleet-execution-role",
"policy_name": "fleet-iam-policy-execution"
},
"role": {
"name": "fleet-role",
"policy_name": "fleet-iam-policy"
}
},
"iam_role_arn": null,
"image": "fleetdm/fleet:v4.51.1",
"loadbalancer": {
"arn": null
},
"mem": 512,
"mount_points": [],
"networking": {
"security_groups": null,
"subnets": null
},
"pid_mode": null,
"private_key_secret_name": "fleet-server-private-key",
"redis": {
"address": null,
"use_tls": true
},
"repository_credentials": "",
"security_group_name": "fleet",
"security_groups": null,
"service": {
"name": "fleet"
},
"sidecars": [],
"software_installers": {
"bucket_name": null,
"bucket_prefix": "fleet-software-installers-",
"create_bucket": true,
"s3_object_prefix": ""
},
"task_cpu": null,
"task_mem": null,
"volumes": []
} | no |
| [migration\_config](#input\_migration\_config) | The configuration object for Fleet's migration task. | object({
mem = number
cpu = number
}) | {
"cpu": 1024,
"mem": 2048
} | no |
| [vpc\_id](#input\_vpc\_id) | n/a | `string` | `null` | no |
@@ -57,6 +63,8 @@ No modules.
| [appautoscaling\_target](#output\_appautoscaling\_target) | n/a |
| [execution\_iam\_role\_arn](#output\_execution\_iam\_role\_arn) | n/a |
| [fleet\_config](#output\_fleet\_config) | n/a |
+| [fleet\_s3\_software\_installers\_config](#output\_fleet\_s3\_software\_installers\_config) | n/a |
+| [fleet\_server\_private\_key\_secret\_arn](#output\_fleet\_server\_private\_key\_secret\_arn) | n/a |
| [iam\_role\_arn](#output\_iam\_role\_arn) | n/a |
| [logging\_config](#output\_logging\_config) | n/a |
| [non\_circular](#output\_non\_circular) | n/a |
diff --git a/terraform/byo-vpc/byo-db/byo-ecs/iam.tf b/terraform/byo-vpc/byo-db/byo-ecs/iam.tf
index d1f00302e2..a3cc53e830 100644
--- a/terraform/byo-vpc/byo-db/byo-ecs/iam.tf
+++ b/terraform/byo-vpc/byo-db/byo-ecs/iam.tf
@@ -1,3 +1,31 @@
+data "aws_iam_policy_document" "software_installers" {
+ statement {
+ actions = [
+ "s3:GetObject*",
+ "s3:PutObject*",
+ "s3:ListBucket*",
+ "s3:ListMultipartUploadParts*",
+ "s3:DeleteObject",
+ "s3:CreateMultipartUpload",
+ "s3:AbortMultipartUpload",
+ "s3:ListMultipartUploadParts",
+ "s3:GetBucketLocation"
+ ]
+ resources = [aws_s3_bucket.software_installers[0].arn, "${aws_s3_bucket.software_installers[0].arn}/*"]
+ }
+}
+
+resource "aws_iam_policy" "software_installers" {
+ count = var.fleet_config.software_installers.create_bucket == true ? 1 : 0
+ policy = data.aws_iam_policy_document.software_installers.json
+}
+
+resource "aws_iam_role_policy_attachment" "software_installers" {
+ count = var.fleet_config.iam_role_arn == null && var.fleet_config.software_installers.create_bucket == true ? 1 : 0
+ policy_arn = aws_iam_policy.software_installers[0].arn
+ role = aws_iam_role.main[0].name
+}
+
data "aws_iam_policy_document" "fleet" {
statement {
effect = "Allow"
@@ -21,8 +49,8 @@ data "aws_iam_policy_document" "assume_role" {
data "aws_iam_policy_document" "fleet-execution" {
// allow fleet application to obtain the database password from secrets manager
statement {
- effect = "Allow"
- actions = ["secretsmanager:GetSecretValue"]
+ effect = "Allow"
+ actions = ["secretsmanager:GetSecretValue"]
resources = [
var.fleet_config.database.password_secret_arn,
aws_secretsmanager_secret.fleet_server_private_key.arn
diff --git a/terraform/byo-vpc/byo-db/byo-ecs/main.tf b/terraform/byo-vpc/byo-db/byo-ecs/main.tf
index fabf20b413..1cc311cec6 100644
--- a/terraform/byo-vpc/byo-db/byo-ecs/main.tf
+++ b/terraform/byo-vpc/byo-db/byo-ecs/main.tf
@@ -58,8 +58,9 @@ resource "aws_ecs_task_definition" "backend" {
requires_compatibilities = ["FARGATE"]
task_role_arn = var.fleet_config.iam_role_arn == null ? aws_iam_role.main[0].arn : var.fleet_config.iam_role_arn
execution_role_arn = aws_iam_role.execution.arn
- cpu = var.fleet_config.cpu
- memory = var.fleet_config.mem
+ cpu = var.fleet_config.task_cpu == null ? var.fleet_config.cpu : var.fleet_config.task_cpu
+ memory = var.fleet_config.task_mem == null ? var.fleet_config.mem : var.fleet_config.task_mem
+ pid_mode = var.fleet_config.pid_mode
container_definitions = jsonencode(
concat([
{
@@ -146,6 +147,14 @@ resource "aws_ecs_task_definition" "backend" {
name = "FLEET_SERVER_TLS"
value = "false"
},
+ {
+ name = "FLEET_S3_SOFTWARE_INSTALLERS_BUCKET"
+ value = var.fleet_config.software_installers.create_bucket == true ? aws_s3_bucket.software_installers[0].bucket : var.fleet_config.software_installers.bucket_name
+ },
+ {
+ name = "FLEET_S3_SOFTWARE_INSTALLERS_PREFIX"
+ value = var.fleet_config.software_installers.s3_object_prefix
+ },
], local.environment)
}
], var.fleet_config.sidecars))
@@ -262,3 +271,35 @@ resource "aws_secretsmanager_secret_version" "fleet_server_private_key" {
secret_id = aws_secretsmanager_secret.fleet_server_private_key.id
secret_string = random_password.fleet_server_private_key.result
}
+
+// Customer keys are not supported in our Fleet Terraforms at the moment. We will evaluate the
+// possibility of providing this capability in the future.
+// No versioning on this bucket is by design.
+// Bucket logging is not supported in our Fleet Terraforms at the moment. It can be enabled by the
+// organizations deploying Fleet, and we will evaluate the possibility of providing this capability
+// in the future.
+
+resource "aws_s3_bucket" "software_installers" { #tfsec:ignore:aws-s3-encryption-customer-key:exp:2022-07-01 #tfsec:ignore:aws-s3-enable-versioning #tfsec:ignore:aws-s3-enable-bucket-logging:exp:2022-06-15
+ count = var.fleet_config.software_installers.create_bucket == true ? 1 : 0
+ bucket = var.fleet_config.software_installers.bucket_name
+ bucket_prefix = var.fleet_config.software_installers.bucket_prefix
+}
+
+resource "aws_s3_bucket_server_side_encryption_configuration" "software_installers" {
+ count = var.fleet_config.software_installers.create_bucket == true ? 1 : 0
+ bucket = aws_s3_bucket.software_installers[0].bucket
+ rule {
+ apply_server_side_encryption_by_default {
+ sse_algorithm = "aws:kms"
+ }
+ }
+}
+
+resource "aws_s3_bucket_public_access_block" "software_installers" {
+ count = var.fleet_config.software_installers.create_bucket == true ? 1 : 0
+ bucket = aws_s3_bucket.software_installers[0].id
+ block_public_acls = true
+ block_public_policy = true
+ ignore_public_acls = true
+ restrict_public_buckets = true
+}
diff --git a/terraform/byo-vpc/byo-db/byo-ecs/outputs.tf b/terraform/byo-vpc/byo-db/byo-ecs/outputs.tf
index 020f82db51..4c72c7a0b0 100644
--- a/terraform/byo-vpc/byo-db/byo-ecs/outputs.tf
+++ b/terraform/byo-vpc/byo-db/byo-ecs/outputs.tf
@@ -42,3 +42,10 @@ output "fleet_config" {
output "fleet_server_private_key_secret_arn" {
value = aws_secretsmanager_secret.fleet_server_private_key.arn
}
+
+output "fleet_s3_software_installers_config" {
+ value = {
+ bucket_name = var.fleet_config.software_installers.create_bucket == true ? aws_s3_bucket.software_installers[0].bucket : var.fleet_config.software_installers.bucket_name
+ s3_object_prefix = var.fleet_config.software_installers.s3_object_prefix
+ }
+}
diff --git a/terraform/byo-vpc/byo-db/byo-ecs/variables.tf b/terraform/byo-vpc/byo-db/byo-ecs/variables.tf
index f4301a00db..5f97d5d83e 100644
--- a/terraform/byo-vpc/byo-db/byo-ecs/variables.tf
+++ b/terraform/byo-vpc/byo-db/byo-ecs/variables.tf
@@ -11,9 +11,12 @@ variable "vpc_id" {
variable "fleet_config" {
type = object({
+ task_mem = optional(number, null)
+ task_cpu = optional(number, null)
mem = optional(number, 4096)
cpu = optional(number, 512)
- image = optional(string, "fleetdm/fleet:v4.50.2")
+ pid_mode = optional(string, null)
+ image = optional(string, "fleetdm/fleet:v4.51.1")
family = optional(string, "fleet")
sidecars = optional(list(any), [])
depends_on = optional(list(any), [])
@@ -93,11 +96,25 @@ variable "fleet_config" {
}), {
name = "fleetdm-execution-role"
})
+ software_installers = optional(object({
+ create_bucket = optional(bool, true)
+ bucket_name = optional(string, null)
+ bucket_prefix = optional(string, "fleet-software-installers-")
+ s3_object_prefix = optional(string, "")
+ }), {
+ create_bucket = true
+ bucket_name = null
+ bucket_prefix = "fleet-software-installers-"
+ s3_object_prefix = ""
+ })
})
default = {
+ task_mem = null
+ task_cpu = null
mem = 512
cpu = 256
- image = "fleetdm/fleet:v4.31.1"
+ pid_mode = null
+ image = "fleetdm/fleet:v4.51.1"
family = "fleet"
sidecars = []
depends_on = []
@@ -157,6 +174,12 @@ variable "fleet_config" {
policy_name = "fleet-iam-policy-execution"
}
}
+ software_installers = {
+ create_bucket = true
+ bucket_name = null
+ bucket_prefix = "fleet-software-installers-"
+ s3_object_prefix = ""
+ }
}
description = "The configuration object for Fleet itself. Fields that default to null will have their respective resources created if not specified."
nullable = false
diff --git a/terraform/byo-vpc/byo-db/variables.tf b/terraform/byo-vpc/byo-db/variables.tf
index e4821dba00..47af53f303 100644
--- a/terraform/byo-vpc/byo-db/variables.tf
+++ b/terraform/byo-vpc/byo-db/variables.tf
@@ -72,9 +72,12 @@ variable "ecs_cluster" {
variable "fleet_config" {
type = object({
+ task_mem = optional(number, null)
+ task_cpu = optional(number, null)
mem = optional(number, 4096)
cpu = optional(number, 512)
- image = optional(string, "fleetdm/fleet:v4.50.2")
+ pid_mode = optional(string, null)
+ image = optional(string, "fleetdm/fleet:v4.51.1")
family = optional(string, "fleet")
sidecars = optional(list(any), [])
depends_on = optional(list(any), [])
@@ -168,11 +171,25 @@ variable "fleet_config" {
}), {
name = "fleetdm-execution-role"
})
+ software_installers = optional(object({
+ create_bucket = optional(bool, true)
+ bucket_name = optional(string, null)
+ bucket_prefix = optional(string, "fleet-software-installers-")
+ s3_object_prefix = optional(string, "")
+ }), {
+ create_bucket = true
+ bucket_name = null
+ bucket_prefix = "fleet-software-installers-"
+ s3_object_prefix = ""
+ })
})
default = {
+ task_mem = null
+ task_cpu = null
mem = 512
cpu = 256
- image = "fleetdm/fleet:v4.31.1"
+ pid_mode = null
+ image = "fleetdm/fleet:v4.51.1"
family = "fleet"
sidecars = []
depends_on = []
@@ -232,6 +249,12 @@ variable "fleet_config" {
policy_name = "fleet-iam-policy-execution"
}
}
+ software_installers = {
+ create_bucket = true
+ bucket_name = null
+ bucket_prefix = "fleet-software-installers-"
+ s3_object_prefix = ""
+ }
}
description = "The configuration object for Fleet itself. Fields that default to null will have their respective resources created if not specified."
nullable = false
diff --git a/terraform/byo-vpc/example/main.tf b/terraform/byo-vpc/example/main.tf
index 597c43edb7..f37fcc3a4d 100644
--- a/terraform/byo-vpc/example/main.tf
+++ b/terraform/byo-vpc/example/main.tf
@@ -17,7 +17,7 @@ provider "aws" {
}
locals {
- fleet_image = "fleetdm/fleet:v4.50.2"
+ fleet_image = "fleetdm/fleet:v4.51.1"
domain_name = "example.com"
}
@@ -92,7 +92,7 @@ module "vpc" {
}
module "byo-vpc" {
- source = "github.com/fleetdm/fleet//terraform/byo-vpc?ref=tf-mod-byo-vpc-v1.7.0"
+ source = "github.com/fleetdm/fleet//terraform/byo-vpc?ref=tf-mod-byo-vpc-v1.10.1"
vpc_config = {
vpc_id = module.vpc.vpc_id
networking = {
diff --git a/terraform/byo-vpc/variables.tf b/terraform/byo-vpc/variables.tf
index 1d23bc3057..abad2853d9 100644
--- a/terraform/byo-vpc/variables.tf
+++ b/terraform/byo-vpc/variables.tf
@@ -165,9 +165,12 @@ variable "ecs_cluster" {
variable "fleet_config" {
type = object({
+ task_mem = optional(number, null)
+ task_cpu = optional(number, null)
mem = optional(number, 4096)
cpu = optional(number, 512)
- image = optional(string, "fleetdm/fleet:v4.50.2")
+ pid_mode = optional(string, null)
+ image = optional(string, "fleetdm/fleet:v4.51.1")
family = optional(string, "fleet")
sidecars = optional(list(any), [])
depends_on = optional(list(any), [])
@@ -261,11 +264,25 @@ variable "fleet_config" {
}), {
name = "fleetdm-execution-role"
})
+ software_installers = optional(object({
+ create_bucket = optional(bool, true)
+ bucket_name = optional(string, null)
+ bucket_prefix = optional(string, "fleet-software-installers-")
+ s3_object_prefix = optional(string, "")
+ }), {
+ create_bucket = true
+ bucket_name = null
+ bucket_prefix = "fleet-software-installers-"
+ s3_object_prefix = ""
+ })
})
default = {
+ task_mem = null
+ task_cpu = null
mem = 512
cpu = 256
- image = "fleetdm/fleet:v4.31.1"
+ pid_mode = null
+ image = "fleetdm/fleet:v4.51.1"
family = "fleet"
sidecars = []
depends_on = []
@@ -325,6 +342,12 @@ variable "fleet_config" {
policy_name = "fleet-iam-policy-execution"
}
}
+ software_installers = {
+ create_bucket = true
+ bucket_name = null
+ bucket_prefix = "fleet-software-installers-"
+ s3_object_prefix = ""
+ }
}
description = "The configuration object for Fleet itself. Fields that default to null will have their respective resources created if not specified."
nullable = false
diff --git a/terraform/example/main.tf b/terraform/example/main.tf
index e430fd69fa..35220057e2 100644
--- a/terraform/example/main.tf
+++ b/terraform/example/main.tf
@@ -50,7 +50,7 @@ locals {
}
module "fleet" {
- source = "github.com/fleetdm/fleet//terraform?ref=tf-mod-root-v1.8.0"
+ source = "github.com/fleetdm/fleet//terraform?ref=tf-mod-root-v1.9.1"
certificate_arn = module.acm.acm_certificate_arn
vpc = {
@@ -63,8 +63,8 @@ module "fleet" {
fleet_config = {
# To avoid pull-rate limiting from dockerhub, consider using our quay.io mirror
- # for the Fleet image. e.g. "quay.io/fleetdm/fleet:v4.50.2"
- image = "fleetdm/fleet:v4.50.2" # override default to deploy the image you desire
+ # for the Fleet image. e.g. "quay.io/fleetdm/fleet:v4.51.1"
+ image = "fleetdm/fleet:v4.51.1" # override default to deploy the image you desire
# See https://fleetdm.com/docs/deploy/reference-architectures#aws for appropriate scaling
# memory and cpu.
autoscaling = {
@@ -116,7 +116,7 @@ module "fleet" {
# doesn't directly support all the features required. the aws cli is invoked via a null-resource.
module "migrations" {
- source = "github.com/fleetdm/fleet//terraform/addons/migrations?ref=tf-mod-addon-migrations-v2.0.0"
+ source = "github.com/fleetdm/fleet//terraform/addons/migrations?ref=tf-mod-addon-migrations-v2.0.1"
ecs_cluster = module.fleet.byo-vpc.byo-db.byo-ecs.service.cluster
task_definition = module.fleet.byo-vpc.byo-db.byo-ecs.task_definition.family
task_definition_revision = module.fleet.byo-vpc.byo-db.byo-ecs.task_definition.revision
@@ -129,7 +129,7 @@ module "migrations" {
module "osquery-carve" {
# The carve bucket also stores software.
- source = "github.com/fleetdm/fleet//terraform/addons/osquery-carve?ref=tf-mod-addon-osquery-carve-v1.0.1"
+ source = "github.com/fleetdm/fleet//terraform/addons/osquery-carve?ref=tf-mod-addon-osquery-carve-v1.1.0"
osquery_carve_s3_bucket = {
name = local.osquery_carve_bucket_name
}
diff --git a/terraform/variables.tf b/terraform/variables.tf
index e80d41e6b5..c73a29640f 100644
--- a/terraform/variables.tf
+++ b/terraform/variables.tf
@@ -213,9 +213,12 @@ variable "ecs_cluster" {
variable "fleet_config" {
type = object({
+ task_mem = optional(number, null)
+ task_cpu = optional(number, null)
mem = optional(number, 4096)
cpu = optional(number, 512)
- image = optional(string, "fleetdm/fleet:v4.50.2")
+ pid_mode = optional(string, null)
+ image = optional(string, "fleetdm/fleet:v4.51.1")
family = optional(string, "fleet")
sidecars = optional(list(any), [])
depends_on = optional(list(any), [])
@@ -309,11 +312,25 @@ variable "fleet_config" {
}), {
name = "fleetdm-execution-role"
})
+ software_installers = optional(object({
+ create_bucket = optional(bool, true)
+ bucket_name = optional(string, null)
+ bucket_prefix = optional(string, "fleet-software-installers-")
+ s3_object_prefix = optional(string, "")
+ }), {
+ create_bucket = true
+ bucket_name = null
+ bucket_prefix = "fleet-software-installers-"
+ s3_object_prefix = ""
+ })
})
default = {
+ task_mem = null
+ task_cpu = null
mem = 512
cpu = 256
- image = "fleetdm/fleet:v4.31.1"
+ pid_mode = null
+ image = "fleetdm/fleet:v4.51.1"
family = "fleet"
sidecars = []
depends_on = []
@@ -373,6 +390,12 @@ variable "fleet_config" {
policy_name = "fleet-iam-policy-execution"
}
}
+ software_installers = {
+ create_bucket = true
+ bucket_name = null
+ bucket_prefix = "fleet-software-installers-"
+ s3_object_prefix = ""
+ }
}
description = "The configuration object for Fleet itself. Fields that default to null will have their respective resources created if not specified."
nullable = false
diff --git a/tools/fleetctl-npm/package.json b/tools/fleetctl-npm/package.json
index 4ebd3b3761..49cb2d6119 100644
--- a/tools/fleetctl-npm/package.json
+++ b/tools/fleetctl-npm/package.json
@@ -1,6 +1,6 @@
{
"name": "fleetctl",
- "version": "v4.50.2",
+ "version": "v4.51.1",
"description": "Installer for the fleetctl CLI tool",
"bin": {
"fleetctl": "./run.js"
diff --git a/tools/nvd/nvdvuln/nvdvuln.go b/tools/nvd/nvdvuln/nvdvuln.go
index b2f2f565da..5ba23209b7 100644
--- a/tools/nvd/nvdvuln/nvdvuln.go
+++ b/tools/nvd/nvdvuln/nvdvuln.go
@@ -184,6 +184,14 @@ func main() {
return nil
}
+ ds.ListOperatingSystemsForPlatformFunc = func(ctx context.Context, platform string) ([]fleet.OperatingSystem, error) {
+ return nil, nil
+ }
+
+ ds.DeleteOutOfDateOSVulnerabilitiesFunc = func(ctx context.Context, source fleet.VulnerabilitySource, duration time.Duration) error {
+ return nil
+ }
+
printf("Translating software to CPE...\n")
err := nvd.TranslateSoftwareToCPE(ctx, ds, *dbDir, logger)
if err != nil {
diff --git a/tools/osquery-agent-options/README.md b/tools/osquery-agent-options/README.md
index 62b0e5bb3f..081f2e7f79 100644
--- a/tools/osquery-agent-options/README.md
+++ b/tools/osquery-agent-options/README.md
@@ -1,12 +1,12 @@
# osquery-agent-options
-This directory contains a script (a Go command) that generates the struct needed to unmarshal the Agent Options' `options` values that the current version of osquery supports. It extracts this information from `osqueryd --help` to identify which osquery command-line flags can be set via the options and which are only for the command-line (i.e. require a restart), and running a query in `osqueryi` to get the data type of those options.
+This directory contains a script (a Go command) that generates the struct needed to unmarshal the Agent Options' `options` values that the current version of osquery supports. It extracts this information from `osqueryd --help` to identify which osquery command-line flags can be set via the options and which are only for the command-line (i.e. require a restart), and running a query in `osqueryi` (`osqueryd -S`) to get the data type of those options.
-It prints the resulting Go code to stdout (the `osqueryOptions` and the `osqueryCommandLineFlags` structs), you can just copy it and insert it in the proper location in the source code to replace the existing struct (in `server/fleet/agent_options.go`).
+It writes the resulting Go code to stdout (the `osqueryOptions` and the `osqueryCommandLineFlags` structs) to a file provided as argument.
-Note that the latest version of osquery should be installed for this tool to work properly (`osqueryd` and `osqueryi` must be in your $PATH).
+This command only supports macOS.
-The system that you use to run this on makes a difference. On 5.11.0, this flow was run on macOS.
+Whenever there's a new version of osquery, just update the variable `osqueryVersion`.
## OS-specific flags
diff --git a/tools/osquery-agent-options/main.go b/tools/osquery-agent-options/main.go
index bc644b56ad..1426cceba2 100644
--- a/tools/osquery-agent-options/main.go
+++ b/tools/osquery-agent-options/main.go
@@ -1,26 +1,41 @@
package main
import (
+ "archive/tar"
"bufio"
"bytes"
+ "compress/gzip"
"encoding/json"
+ "errors"
+ "fmt"
+ "io"
"log"
+ "net/http"
+ "net/url"
"os"
"os/exec"
+ "path/filepath"
"regexp"
+ "runtime"
"strings"
"text/template"
+ "github.com/fleetdm/fleet/v4/orbit/pkg/constant"
+ "github.com/fleetdm/fleet/v4/pkg/download"
"github.com/fleetdm/fleet/v4/server/fleet"
)
var (
- rxOption = regexp.MustCompile(`\-\-(\w+)\s`)
+ rxOption = regexp.MustCompile(`\-\-(\w+)\s`)
+ osqueryVersion = "5.12.2"
structTpl = template.Must(template.New("struct").Funcs(template.FuncMap{
"camelCase": camelCaseOptionName,
- }).Parse(`
-// NOTE: generate automatically with ` + "`go run ./tools/osquery-agent-options/main.go`" + `
+ }).Parse(`// Automatically generated by tools/osquery-agent-options for osquery {{ .OsqueryVersion }}. DO NOT EDIT!
+// To update flags for a new osquery version, update the osqueryVersion variable in
+// "tools/osquery-agent-options/main.go" and run "cd server/fleet/ && go generate".
+package fleet
+
type osqueryOptions struct { {{ range $name, $type := .Options }}
{{camelCase $name}} {{$type}} ` + "`json:\"{{$name}}\"`" + `{{end}}
@@ -31,7 +46,6 @@ type osqueryOptions struct { {{ range $name, $type := .Options }}
OsqueryCommandLineFlagsHidden
}
-// NOTE: generate automatically with ` + "`go run ./tools/osquery-agent-options/main.go`" + `
type osqueryCommandLineFlags struct { {{ range $name, $type := .Flags }}
{{camelCase $name}} {{$type}} ` + "`json:\"{{$name}}\"`" + `{{end}}
@@ -45,11 +59,35 @@ type osqueryCommandLineFlags struct { {{ range $name, $type := .Flags }}
)
type templateData struct {
- Options map[string]string
- Flags map[string]string
+ OsqueryVersion string
+ Options map[string]string
+ Flags map[string]string
}
func main() {
+ fmt.Printf("Generating osquery flags for version: %s\n", osqueryVersion)
+ if runtime.GOOS != "darwin" {
+ log.Fatal("Currently only supported on macOS")
+ }
+ urlStr := fmt.Sprintf("https://tuf.fleetctl.com/targets/osqueryd/macos-app/%s/osqueryd.app.tar.gz", osqueryVersion)
+ osqueryTUFURL, err := url.Parse(urlStr)
+ if err != nil {
+ log.Fatalf("parse osquery TUF URL: %q: %s", urlStr, err)
+ }
+ tmpDir, err := os.MkdirTemp("", "")
+ if err != nil {
+ log.Fatalf("create temp dir: %s", err)
+ }
+ defer os.RemoveAll(tmpDir)
+ osquerydAppTarGzPath := filepath.Join(tmpDir, "osqueryd.app.tar.gz")
+ if err := download.Download(http.DefaultClient, osqueryTUFURL, osquerydAppTarGzPath); err != nil {
+ log.Fatalf("download osqueryd.app.tar.gz to %s: %s", osquerydAppTarGzPath, err)
+ }
+ if err := extractTarGz(osquerydAppTarGzPath); err != nil {
+ log.Fatalf("extract tar.gz %q: %s", osquerydAppTarGzPath, err)
+ }
+ osquerydPath := filepath.Join(filepath.Dir(osquerydAppTarGzPath), "osquery.app", "Contents", "MacOS", "osqueryd")
+
// marshal/unmarshal the OS-specific structs into a map so we have all their
// keys and we can ignore them in the auto-generated structs (because we
// can't auto- generate those, we'd only see the ones that exist on the
@@ -71,7 +109,7 @@ func main() {
}
// get the list of flags that are valid as configuration options
- b, err = exec.Command("osqueryd", "--help").Output()
+ b, err = exec.Command(osquerydPath, "--help").Output()
if err != nil {
log.Fatalf("failed to run osqueryd --help: %v", err)
}
@@ -118,7 +156,7 @@ func main() {
Name string
Type string
}
- b, err = exec.Command("osqueryi", "--json", "SELECT name, type FROM osquery_flags").Output()
+ b, err = exec.Command(osquerydPath, "-S", "--json", "SELECT name, type FROM osquery_flags").Output()
if err != nil {
log.Fatalf("failed to run osqueryi query: %v", err)
}
@@ -159,9 +197,24 @@ func main() {
}
}
- if err := structTpl.Execute(os.Stdout, templateData{Options: validOptions, Flags: validFlags}); err != nil {
+ outputFilePath := os.Args[1]
+ outputFile, err := os.OpenFile(outputFilePath, os.O_WRONLY|os.O_TRUNC|os.O_CREATE, 0o644)
+ if err != nil {
+ log.Fatalf("open output file %q: %s", outputFilePath, err)
+ }
+ defer outputFile.Close()
+
+ if err := structTpl.Execute(outputFile, templateData{
+ OsqueryVersion: osqueryVersion,
+ Options: validOptions,
+ Flags: validFlags,
+ }); err != nil {
log.Fatalf("failed to execute template: %v", err)
}
+
+ if err := outputFile.Close(); err != nil {
+ log.Fatalf("close file %q: %s", outputFilePath, err)
+ }
}
func camelCaseOptionName(s string) string {
@@ -171,3 +224,77 @@ func camelCaseOptionName(s string) string {
}
return strings.Join(parts, "")
}
+
+// sanitizeArchivePath sanitizes the archive file pathing from "G305: Zip Slip vulnerability"
+func sanitizeArchivePath(d, t string) (string, error) {
+ v := filepath.Join(d, t)
+ if strings.HasPrefix(v, filepath.Clean(d)) {
+ return v, nil
+ }
+
+ return "", fmt.Errorf("%s: %s", "content filepath is tainted", t)
+}
+
+// extractTagGz extracts the contents of the provided tar.gz file.
+func extractTarGz(path string) error {
+ tarGzFile, err := os.OpenFile(path, os.O_RDONLY, 0o755)
+ if err != nil {
+ return fmt.Errorf("open %q: %w", path, err)
+ }
+ defer tarGzFile.Close()
+
+ gzipReader, err := gzip.NewReader(tarGzFile)
+ if err != nil {
+ return fmt.Errorf("gzip reader %q: %w", path, err)
+ }
+ defer gzipReader.Close()
+
+ tarReader := tar.NewReader(gzipReader)
+ for {
+ header, err := tarReader.Next()
+ switch {
+ case err == nil:
+ // OK
+ case errors.Is(err, io.EOF):
+ return nil
+ default:
+ return fmt.Errorf("tar reader %q: %w", path, err)
+ }
+
+ // Prevent zip-slip attack.
+ if strings.Contains(header.Name, "..") {
+ return fmt.Errorf("invalid path in tar.gz: %q", header.Name)
+ }
+
+ targetPath, err := sanitizeArchivePath(filepath.Dir(path), header.Name)
+ if err != nil {
+ return fmt.Errorf("sanitize failed: %s", err)
+ }
+
+ switch header.Typeflag {
+ case tar.TypeDir:
+ if err := os.MkdirAll(targetPath, constant.DefaultDirMode); err != nil {
+ return fmt.Errorf("mkdir %q: %w", header.Name, err)
+ }
+ case tar.TypeReg:
+ err := func() error {
+ outFile, err := os.OpenFile(targetPath, os.O_CREATE|os.O_WRONLY, header.FileInfo().Mode())
+ if err != nil {
+ return fmt.Errorf("failed to create %q: %w", header.Name, err)
+ }
+ defer outFile.Close()
+
+ // Ignoring G110 because we are using this on tooling.
+ if _, err := io.Copy(outFile, tarReader); err != nil { //nolint:gosec
+ return fmt.Errorf("failed to copy %q: %w", header.Name, err)
+ }
+ return nil
+ }()
+ if err != nil {
+ return err
+ }
+ default:
+ return fmt.Errorf("unknown flag type %q: %d", header.Name, header.Typeflag)
+ }
+ }
+}
diff --git a/tools/osquery-agent-options/osquery_5.11.0_codeflags.txt b/tools/osquery-agent-options/osquery_5.12.2_codeflags.txt
similarity index 99%
rename from tools/osquery-agent-options/osquery_5.11.0_codeflags.txt
rename to tools/osquery-agent-options/osquery_5.12.2_codeflags.txt
index 36f32fcfa5..9b69ddc6c4 100644
--- a/tools/osquery-agent-options/osquery_5.11.0_codeflags.txt
+++ b/tools/osquery-agent-options/osquery_5.12.2_codeflags.txt
@@ -1,4 +1,3 @@
-
alarm_timeout
allow_unsafe
alsologtostderr
diff --git a/tools/release/README.md b/tools/release/README.md
index 0fe38b990a..f753d8a6f9 100644
--- a/tools/release/README.md
+++ b/tools/release/README.md
@@ -34,17 +34,35 @@ example
./tools/release/publish_release.sh -a
# Do QA until ready to release
-# QA is passed on all teams and ready for release
+# - QA is passed on all teams and ready for release
+# - Merge changelog and versions update PR into main
+# - git pull main locally with the changelog as the latest commit
# Tag main
./tools/release/publish_release.sh -ag
+
+# - Wait for build to run
+
# Publish main
./tools/release/publish_release.sh -auq
-# Go update osquery-slack version
+
+# - Wait for publish process to complete.
+# - Merge release article and wait for website to build.
+# - When the release article is published, create a LinkedIn post on Fleet's company page.
+# - Copy te LinkedIn post URL as the value for the linkedin_post_url variable in the general_announce_info() function.
+# - Go update osquery-slack version
+
+# Announce release
+# Change $current_version to the current version that was just released
+# For example, ./tools/release/publish_release.sh -anu -v 4.50.0
+./tools/release/publish_release.sh -anu -v {current_version}
```
...
-TODO example output
+:cloud: :rocket: The latest version of Fleet is 4.50.0.
+More info: https://github.com/fleetdm/fleet/releases/tag/fleet-v4.50.0
+Release article: https://fleetdm.com/releases/fleet-4.50.0
+LinkedIn post: https://www.linkedin.com/feed/update/urn:li:activity:7199509896705232898/
...
diff --git a/tools/release/publish_release.sh b/tools/release/publish_release.sh
index 1461ed4cc4..9416e2b68c 100755
--- a/tools/release/publish_release.sh
+++ b/tools/release/publish_release.sh
@@ -674,24 +674,32 @@ fi
next_tag="fleet-$next_ver"
if [[ "$target_milestone_number" == "" ]]; then
- echo "Missing milestone $target_milestone, Please create one and tie tickets to the milestone to continue"
- exit 1
+ if [ "$announce_only" = "false" ]; then
+ echo "Missing milestone $target_milestone, Please create one and tie tickets to the milestone to continue"
+ exit 1
+ fi
fi
echo "Found milestone $target_milestone with number $target_milestone_number"
if [ "$print_info" = "true" ]; then
- print_announce_info
- exit 0
+ if [ "$announce_only" = "false" ]; then
+ print_announce_info
+ exit 0
+ fi
fi
if [ "$do_tag" = "true" ]; then
- tag
- exit 0
+ if [ "$announce_only" = "false" ]; then
+ tag
+ exit 0
+ fi
fi
if [ "$release_notes" = "true" ]; then
- update_release_notes
- exit 0
+ if [ "$announce_only" = "false" ]; then
+ update_release_notes
+ exit 0
+ fi
fi
if [ "$publish_release" = "true" ]; then
@@ -735,7 +743,7 @@ if [ "$cherry_pick_resolved" = "false" ]; then
prs_for_issue=`gh api repos/fleetdm/fleet/issues/$issue/timeline --paginate | jq -r '.[]' | $GREP_CMD "fleetdm/fleet/" | $GREP_CMD -oP "pulls\/\K(?:\d+)"`
echo -n "https://github.com/fleetdm/fleet/issues/$issue"
if [[ "$prs_for_issue" == "" ]]; then
- echo -n "NO PR's found, please verify they are not missing in the issue, if no PR's were required for this ticket please reconsider adding it to this release."
+ echo -n " NO PR's found, please verify they are not missing in the issue, if no PR's were required for this ticket please reconsider adding it to this release."
fi
for val in $prs_for_issue; do
echo -n " $val"
diff --git a/tools/tuf/test/create_repository.sh b/tools/tuf/test/create_repository.sh
index 203eb791bc..2fabf7aa10 100755
--- a/tools/tuf/test/create_repository.sh
+++ b/tools/tuf/test/create_repository.sh
@@ -106,7 +106,9 @@ for system in $SYSTEMS; do
--platform macos \
--name desktop \
--version 42.0.0 -t 42.0 -t 42 -t stable
- rm desktop.app.tar.gz
+ if [[ -z "$MACOS_USE_PREBUILT_DESKTOP_APP_TAR_GZ" ]]; then
+ rm desktop.app.tar.gz
+ fi
fi
# Add Nudge application on macos (if enabled).
diff --git a/website/api/helpers/salesforce/update-or-create-contact-and-account.js b/website/api/helpers/salesforce/update-or-create-contact-and-account.js
index 3388a161fb..891d69154d 100644
--- a/website/api/helpers/salesforce/update-or-create-contact-and-account.js
+++ b/website/api/helpers/salesforce/update-or-create-contact-and-account.js
@@ -103,6 +103,7 @@ module.exports = {
} else {
// If no existing account record was found, create a new one.
// Create a timestamp to use for the new account's assigned date.
+ salesforceAccountOwnerId = '0054x00000735wDAAQ';// « "Integrations admin" user.
let today = new Date();
let nowOn = today.toISOString().replace('Z', '+0000');
@@ -117,6 +118,7 @@ module.exports = {
Website: enrichmentData.employer.emailDomain,
LinkedIn_company_URL__c: enrichmentData.employer.linkedinCompanyPageUrl,// eslint-disable-line camelcase
NumberOfEmployees: enrichmentData.employer.numberOfEmployees,
+ OwnerId: salesforceAccountOwnerId
});
salesforceAccountId = newAccountRecord.id;
}//fi
diff --git a/website/api/helpers/strings/to-html.js b/website/api/helpers/strings/to-html.js
index 91e29a5273..5182528b98 100644
--- a/website/api/helpers/strings/to-html.js
+++ b/website/api/helpers/strings/to-html.js
@@ -94,7 +94,11 @@ module.exports = {
};
} else {
customRenderer.heading = function (text, level) {
- return '
-
There are no results that match your filters. Everyone can contribute.
diff --git a/yarn.lock b/yarn.lock index d63a6f5f9b..88ad4e8ccb 100644 --- a/yarn.lock +++ b/yarn.lock @@ -6666,11 +6666,11 @@ brace-expansion@^2.0.1: balanced-match "^1.0.0" braces@^3.0.2, braces@~3.0.2: - version "3.0.2" - resolved "https://registry.yarnpkg.com/braces/-/braces-3.0.2.tgz#3454e1a462ee8d599e236df336cd9ea4f8afe107" - integrity sha512-b8um+L1RzM3WDSzvhm6gIz1yfTbBt6YTlcEKAvsmqCZZFw46z626lVj9j1yEPW33H5H+lBQpZMP1k8l+78Ha0A== + version "3.0.3" + resolved "https://registry.yarnpkg.com/braces/-/braces-3.0.3.tgz#490332f40919452272d55a8480adc0c441358789" + integrity sha512-yQbXgO/OSZVD2IsiLlro+7Hf6Q18EJrKSEsdoMzKePKXct3gvD8oLcOQdIzGupr5Fj+EDe8gO/lxc1BzfMpxvA== dependencies: - fill-range "^7.0.1" + fill-range "^7.1.1" broadcast-channel@^3.4.1: version "3.7.0" @@ -9222,10 +9222,10 @@ filelist@^1.0.1: dependencies: minimatch "^5.0.1" -fill-range@^7.0.1: - version "7.0.1" - resolved "https://registry.yarnpkg.com/fill-range/-/fill-range-7.0.1.tgz#1919a6a7c75fe38b2c7c77e5198535da9acdda40" - integrity sha512-qOo9F+dMUmC2Lcb4BbVvnKJxTPjCm+RRpe4gDuGrzkL7mEVl/djYSu2OdQ2Pa302N4oqkSg9ir6jaLWJ2USVpQ== +fill-range@^7.1.1: + version "7.1.1" + resolved "https://registry.yarnpkg.com/fill-range/-/fill-range-7.1.1.tgz#44265d3cac07e3ea7dc247516380643754a05292" + integrity sha512-YsGpe3WHLK8ZYi4tWDg2Jy3ebRz2rXowDxnld4bkQB00cc/1Zw9AWnC0i9ztDJitivtQvaI9KaLyKrc+hBW0yg== dependencies: to-regex-range "^5.0.1"