for #24829
See https://github.com/axios/axios/issues/1045 -- by default Axios
buffers uploaded files into memory fully, to support redirects. For
large file uploads this means we get OOM errors, especially when sending
to multiple teams. There's a few other optimizations we can put in place
here but in the short term we can fix the buffering issue by setting
`maxRedirects: 0` on the requests.
I tested this by adding an `onUploadProgress` handler to the Axios
request that dumps memory usage, and uploading a 209mb software file to
3 teams. Before the update, the readout ticket up continuously (the
first number is the # of bytes uploaded):
```
1540129 {rss: 161652736, heapTotal: 65880064, heapUsed: 55625552, external: 28411157, arrayBuffers: 24338844}
edit-software.js:177
1554313 {rss: 149254144, heapTotal: 65880064, heapUsed: 52445200, external: 25193635, arrayBuffers: 21121327}
edit-software.js:177
2339833 {rss: 151703552, heapTotal: 66404352, heapUsed: 52269280, external: 12664377, arrayBuffers: 8592064}
...a minute later...
192708641 {rss: 619323392, heapTotal: 95240192, heapUsed: 55320960, external: 618952429, arrayBuffers: 614879965}
edit-software.js:177
201523233 {rss: 634613760, heapTotal: 95240192, heapUsed: 58514992, external: 636581613, arrayBuffers: 632509154}
edit-software.js:177
209326677 {rss: 637399040, heapTotal: 95240192, heapUsed: 56800016, external: 639441633, arrayBuffers: 635369173}
```
so we start at ~161mb, and by the time we're done, we're using 637mb of
RAM. Render's free tier has a 250mb limit on apps.
With `maxRedirects: 0`, we see:
```
2669337 {rss: 151846912, heapTotal: 66404352, heapUsed: 53297400, external: 26446868, arrayBuffers: 22374419}
edit-software.js:177
2279929 {rss: 152641536, heapTotal: 66404352, heapUsed: 53453664, external: 27233300, arrayBuffers: 23160851}
edit-software.js:177
2228585 {rss: 153038848, heapTotal: 66404352, heapUsed: 53537096, external: 27626516, arrayBuffers: 23554067}
...a minute later...
209326677 {rss: 146989056, heapTotal: 92094464, heapUsed: 53802856, external: 14617518, arrayBuffers: 10545071}
edit-software.js:177
209326677 {rss: 153051136, heapTotal: 92094464, heapUsed: 55376336, external: 22447478, arrayBuffers: 18375026}
edit-software.js:177
209326677 {rss: 152129536, heapTotal: 92094464, heapUsed: 51857632, external: 22447478, arrayBuffers: 16540013}
```
showing that we start and finish with around the same amount of RAM
used.
Changes:
- Updated the MSP dashboard's readme to include a section about required
config values for software features.
- Added config values required for software features to the MSP
dashboard's docker-compose file.
- Added an uploads configuration file that lists the configuration
values required for software features.
For #24509
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
> Related issue: #24120
## Changes
- Added the `software_title_id` field to the activity details for
`added_software` activities, which get generated when adding a customer
installer or a FMA
- Added a return value (`titleID`) to
`ds.MatchOrCreateSoftwareInstaller`
- Removed `ds.GetSoftwareTitleIDByMaintainedAppID`. Since we're
returning the new value above, this method was no longer needed.
## Testing steps
1. Add a custom installer
2. Add a FMA
3. Check the activity details in the response to `GET /activities`.
Verify that the `software_title_id` field exists and is correct.
4. Add a FMA with automatic install. Make sure the policy is correctly
created, has the correct software title ID associated with it, and that
it installs the app.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
The software versions endpoint cve details can be truncated using the
`without_vulnerability_details` flag.
https://github.com/fleetdm/fleet/issues/23679
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
For #23305.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
All edited YAML files were ran through a YAML syntax check before before
committed.
**macOS-13**
- UPDATED: "3.5 - Ensure Access to Audit Records Is Controlled"
Description and Resolution. Query did not change.
- ADDED: "5.10 - Ensure XProtect Is Running and Updated" Checking for
updated is actually handled via a different query.
**macOS-14**
- UPDATED: "3.5 - Ensure Access to Audit Records Is Controlled"
Description and Resolution. Query did not change.
- ADDED: "5.10 - Ensure XProtect Is Running and Updated" Checking for
updated is actually handled via a different query.
**macOS-15**
Initial version duplicated from macOS-14 queries, then the following
changes were applied:
- REMOVED: "3.6 - Ensure Firewall Logging Is Enabled and Configured"
The following controls were not added, further research on how to check
them with osquery is required:
- 2.6.3.1 - 2.6.3.5 and 2.7.2: I am not sure how we can accomplish this.
- "5.11 - Ensure Logging Is Enabled For Sudo" I believe this one can be
accomplished through the file_lines table
---------
Co-authored-by: Sharon Katz <121527325+sharon-fdm@users.noreply.github.com>
Co-authored-by: Sharon Katz <sharon@fleetdm.com>
> Related issue: #22077
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
Changes:
- (msp dashboard) Updated axios usage in upload-software and
edit-software to use the postForm method.
- Added form-data as a dependency to make sure the app uses
form-data@4.0.1
Changes:
- (msp dashboard) Updated the edit-software and upload-software endpoints to log more
information about errors when requests to upload a software installer
fail.
Changes:
- Updated the software page to display different error messages when
requests to software endpoints fail.
- Updated the edit software endpoint to:
- have three new exits, `softwareAlreadyExistsOnThisTeam`,
`couldNotReadVersion` and `softwareDeletionFailed`
- Create database records for newly undeployed software after it has
been removed from all teams on a Fleet instance.
- Return a `softwareAlreadyExistsOnThisTeam` response if undeployed
software is being transferred to a team it is already deployed to.
- Return a `couldNotReadVersion` response if the Fleet instance cannot
read the version information from an uploaded software installer
- Return a `softwareDeletionFailed` response if an installer cannot be
deleted via an API request (If it is configured to be installed during
the macOS setup experience)
- Updated the upload-software endpoint to return a `couldNotReadVersion`
response if the Fleet instance cannot read the version information from
an uploaded software installer
Changes:
- Updated the edit-software endpoint to:
- log a detailed warning when a request to get a stream of an existing
software installer returns a non-200 repsonse
- Use the modify package API endpoint when a software installer is
replaced.
- include the nested response from the Fleet instance in error messages
- Updated the delete-software endpoint to return a
'softwareDeletionFailed` exit when a specified software installer is
included in the macOS setup experience and cannot be deleted via API
requests.
- Updated the software page to show an error message with a link to the
controls page on the connected Fleet instance if the delete-software
endpoint returns a `softwareDeletionFailed` exit
relates to #23137, #23136
implements to the rest of the UI for automatically creating fleet
policies when adding a fleet maintained app. Also includes the API
changes needed for this which include changing the `GET
/software/titles` and `GET /software/titles/:id` endpoints to include
the `automatic_install_policies` data.
UI added includes:
**Adding tag for automatic install software titles**
**Adding modal to show the policies associated with that software
title**
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com>
The host context never includes Orbit version (only the hosts/{id}
endpoint does) so we need to grab that data when we need it.
# Checklist for submitter
- [x] Added/updated tests
~~- [ ] Manual QA for all new/changed functionality~~ will be part of
E2E QA
## Addresses #22702, #23713, #23756, #23746, #23747, and #23876
_-Note that much of this code as is will render as expected only once
integrated with the backend or if manipulated manually for testing
purposes_
**Frontend**:
- Update banners on my device page, tests
- Build new logic for calling endpoint to trigger linux key escrow on
clicking `Create key`
- Add `CreateLinuxKeyModal` to inform user of next steps after clicking
`Create key`
- Update banners on host details page, tests
- Update the Controls > OS settings section with new logic related to
linux disk encryption
- Expect and include counts of Linux hosts in aggregate disk encryption
stats UI
- Add "Linux" column to the disk encryption table
- Show disk encryption related UI for supported Linux platforms
- TODO: confirm platform string matching functionality in manual e2e
testing
- Expand capabilities of `SectionHeader` component, apply to new UI
- Flash "missing private key" error, with clickable link, when trying to
update disk encryption enabled while no server private key is present.
- TODO: QA this once other endpoints on Controls > Disk encryption are
enabled even when MDM not turned on
- Update Disk encryption key modal copy
-Other TODO:
- Confirm when integrated with API:
- Aggregate disk encryption counts
- Disk encryption table Linux column
- Show disk encryption key action on host details page when expected
- Opens Disk encryption key modal, displays key as expected
**Backend**:
- For "No team" and teams, error when trying to update disk encryption
enabled while no server private key is present.
- Remove requirement of mdm being enabled for use of various endpoints
related to Linux disk encryption
- Update tests
_________
**Host details and my device page banners**
**Create key modal**
<img width="1799" alt="create-key-modal"
src="https://github.com/user-attachments/assets/81a55ccb-b6b9-4eb6-b2ff-a463c60724c0">
**Enabling disk encryption**
**Disk encryption: Fleet free**
<img width="1912" alt="free"
src="https://github.com/user-attachments/assets/9f9cace3-8955-47c2-87d9-24ff9387ac1a">
**Custom settings: turn on MDM**
<img width="1912" alt="turn on mdm"
src="https://github.com/user-attachments/assets/4d3ad47b-4035-4d93-86f0-dc2691b38bb4">
**Device status indicators**
**Encryption key action and modal**
- [x] Changes file added for user-visible changes in `changes/`
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
- [ ] Full e2e testing to do when integrated with backend
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Co-authored-by: Ian Littman <iansltx@gmail.com>
> Related issue: #23787
This adds the script name to both the upcoming and past activities.
Demo video: https://www.youtube.com/watch?v=kLSsUZhyMC4
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
#23583, #23584
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [ ] Manual QA for all new/changed functionality -- should be tested
end-to-end
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
> Related issue: #23247
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
#22437
There is a bug in Ubuntu 24.04's distribution of GDM that prevents it
from starting correctly and displaying a prompt to the user if
`/etc/nologin` is present. This issue is not present on the current
release of Fedora, meaning it is Ubuntu specific.
The way we lock users out is by manually creating the `nologin` file and
then masking the `systemd-user-sessions` systemd unit, which creates the
file on shutdown and deletes it on startup. This will cause a PAM policy
to fail and prevents anyone from logging in. When we unlock the system
we delete the `nologin` file, unmask the `systemd-user-sessions` unit,
and manually run the binary that it should start.
This process removes the cause of the GDM bug, but we need to reboot the
machine to get GDM working again.
While I have not yet been able to determine the exact cause of the bug,
this fix will prevent the user from being stuck with a black screen once
the machine is unlocked.
This fix will not remedy GDM showing a black screen upon being locked,
it only ensures that the user isn't stuck having to manually reboot the
machine once it's unlocked.
We should check back on this soon to see if the bug gets been fixed
upstream.
#22740
Full QA is still a WIP but this is ready for review.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
https://github.com/fleetdm/fleet/issues/21928#issuecomment-2436371970
Changes:
- updated the edit-software endpoint to make sure that text values
related to software installers (pre-install query, install script,
post-install script, uninstall script) are updated for all teams when
software is edited, and to delete software on the fleet instance when
the installer is replaced and all teams are removed.
- updated the ace editor component to emit an input event when text is
pasted inside of it.
- Updated the error messages in the upload software modal
- Fixed an issue where the edit software endpoint would return a 400
response when all teams are removed from a software installer when a new
installer package is provided.
> No issue, just some cleanup/bugfix
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
