Commit graph

980 commits

Author SHA1 Message Date
Dante Catalfamo
167e2e3e28
Scope pending host profile rebuilds (#23772)
#21338
2024-11-15 11:55:30 -05:00
gillespi314
28e4cf6cf7 Merge branch 'main' into feat-include-any-label 2024-11-14 08:33:03 -06:00
Jahziel Villasana-Espinoza
e3618e8335
fix: return error in gitops if using deprecated field and there are more than 1 abm tokens (#23768)
> Related issue: #22359

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-11-13 14:07:59 -05:00
Zach Wasserman
8c21dff636
Add capability to serve YARA rules via authenticated Fleet endpoints (#23343)
Implements the Fleet side of #14899

- Add new endpoints to update and retrieve yara rules
- Add support in fleetctl for applying the rules

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`.
  See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features.
- [x] Added/updated tests
- [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes
- [x] If database migrations are included, checked table schema to confirm autoupdate
- For database migrations:
  - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration.
  - [ ] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects.
  - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2024-11-13 09:01:08 -08:00
Martin Angers
c3ebce9293
Improve memory usage of software installers parsing (#23596) 2024-11-12 09:28:08 -05:00
Ian Littman
cfead6a35f
Fix path resolution for installer queries and scripts to always be relative to where the query file or script is referenced (#23502)
#22187

Similar fix to #22555: resolve paths at spec parsing time rather than
when trying to grab files

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-11-07 11:22:08 -06:00
Jahziel Villasana-Espinoza
aed3c3f775 chore: merge in main 2024-11-06 10:13:47 -05:00
Dante Catalfamo
aa3fd29c13
Profile Labels Include Any CLI (#23434)
#22576


Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
2024-11-05 15:13:44 -05:00
Victor Lyuboslavsky
3cf99f8039
Enable staticcheck Go linter. (#23487)
#23486 

Linter: https://staticcheck.dev/
2024-11-05 11:16:24 -06:00
Ian Littman
a2468fa0b4
Re-enable TestCronVulnerabilitiesCreatesDatabasesPath test with more precise SUT and internal panic recovery (#23468)
#23258 (see
[comment](https://github.com/fleetdm/fleet/issues/23258#issuecomment-2443304838)
for rationale)

Validated by removing the two places that would create the directory
(early in scanVulnerabilities in cron.go, partway through download in
download.go) and ensuring the test failed (timeout after 10s).

Both dir creations happen early in the vulns cron so I was able to
drastically tighten the timing on the periodic check on this test, so
this tests completes way quicker than before as an added benefit
(automatic test parallelism notwithstanding).

The panic recovery here theoretically shouldn't be necessary, as on a
passed test the context will get cancelled while syncing the CPE sqlite,
but is included to ensure the test doesn't flake if the implementation
of the vulnerabilities cron changes such that we _would_ get a panic by
cancelling the context this early.

# Checklist for submitter
- [x] Added/updated tests
2024-11-05 07:34:54 -06:00
Victor Lyuboslavsky
e2d9a9016c
Add gosimple linter (#23250)
#23249

Add gosimple linter to golangci-lint CI job.
2024-10-29 14:17:51 -05:00
Lucas Manuel Rodriguez
2e5bf75b6d
fleetd to start up when TUF signatures are expired (#23102)
#22740

Full QA is still a WIP but this is ready for review.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-10-28 20:40:19 -03:00
Dante Catalfamo
6e9955d7c7
Add clause to exclude VPP apps from host software list (#23207) 2024-10-28 15:48:54 -04:00
Martin Angers
d5c9a165e5
Bugfix and SE: support VPP apps and (install during setup of VPP apps) in gitops for no-team (#23160) 2024-10-28 10:35:57 -04:00
Jahziel Villasana-Espinoza
0bf331786e chore: dump sql schema 2024-10-25 18:01:19 -04:00
Victor Lyuboslavsky
722608318e
Minor fixes from analyzing test results (#23260)
Newlines improve test result parsing.

Extra `t.Parallel()` calls removed in subtest since parent test and
package are already parallelized.
2024-10-25 15:57:51 -05:00
Victor Lyuboslavsky
b67127c357
Skip TestCronVulnerabilitiesCreatesDatabasesPath (#23263)
Seeing very high failure rate for this test. Bug already filed:
https://github.com/fleetdm/fleet/issues/23258
2024-10-25 15:05:38 -05:00
Victor Lyuboslavsky
62fdfb68c1
Ignore teams files for non-premium gitops users. (#23194)
#21715

This fix is needed for https://github.com/fleetdm/fleet-gitops CI to run
successfully against a non-premium Fleet server.
That CI test will cover this change.

# Checklist for submitter
- [x] Manual QA for all new/changed functionality
2024-10-25 09:11:43 -05:00
Lucas Manuel Rodriguez
f16c941d89
Fix PATCH /api/latest/fleet/config to not clear VPP token/team associations when not set (#23175)
#23174

PS: Make sure to render the diff without whitespace changes.
![Screenshot 2024-10-24 at 10 58
47 AM](https://github.com/user-attachments/assets/fb36c995-206c-4a15-8bf2-6b375c5cc565)


- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-10-25 10:01:18 -03:00
Martin Angers
37584adcf3
SE: add CLI automated tests for fleetctl gitops (#23125) 2024-10-23 16:57:56 -04:00
Martin Angers
d1c3b5b28e
SE: CLI setup experience changes (#22956) 2024-10-23 14:51:02 -04:00
Tim Lee
d69100940e
Add config for custom MDM URL (#22878) 2024-10-22 11:05:35 -06:00
Victor Lyuboslavsky
f3c18a14fe
Allow Render connectionString as redis.address (#23058)
#23057 

Render provides
[connectionString](https://docs.render.com/blueprint-spec#connectionstring)
for their Redis service, which Fleet cannot take because it includes
`redis://` prefix.

Once this change is released, we can [update our
render.yaml](https://github.com/fleetdm/fleet/pull/23056/files#diff-a64cf250b418ab8feee6c682a3d8cbd3b72cf24d4a241adeaf35c98b84045f93)

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
2024-10-21 13:49:01 -05:00
Victor Lyuboslavsky
476dd3cc86
Ignore --delete-other-teams for non-premium gitops (#23052)
#23050
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
2024-10-21 11:20:41 -05:00
Sarah Gillespie
65e374c85c
Fix bug in ABM token renewal flow (#22988) 2024-10-18 13:16:04 -05:00
Victor Lyuboslavsky
f85b6f776f
Updating golangci-lint to 1.61.0 (#22973) 2024-10-18 12:38:26 -05:00
Gabriel Hernandez
497a0f036e
fix for the enrollment page to not send back a 404 page (#22981)
makes a fix to the enroll byod handler. The hander use to be set once
when the server started but this lead to a user receiving a 404 page
after the fleet instance setup was complete but was not restarted.

now the `/enroll` handler will check for the setup requirement itself.
There is some error handling when the setup is required. if the fleet
instance has been setup, they will get the enroll byod page.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
2024-10-18 11:10:17 +01:00
Martin Angers
0e8053a5c8
Comment-out tests that can cause OOM issues (#22957) 2024-10-16 12:50:30 -04:00
Victor Lyuboslavsky
383d03cd6d
GitOps support for NDES SCEP proxy config. (#22918)
#22124

# Demo
<div>
<a href="https://www.loom.com/share/ad30efda816e4bf3abca0eab92b6e928">
      <p>[Demo] NDES SCEP proxy gitops #21955 - Watch Video</p>
    </a>
<a href="https://www.loom.com/share/ad30efda816e4bf3abca0eab92b6e928">
<img style="max-width:300px;"
src="https://cdn.loom.com/sessions/thumbnails/ad30efda816e4bf3abca0eab92b6e928-33b1f073da294133-full-play.gif">
    </a>
  </div>

# Checklist for submitter
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-10-16 11:12:48 -05:00
Victor Lyuboslavsky
0f251036b3
Resend profile if NDES SCEP challenge has expired. (#22841)
For #21955 (the story has a video demo of core functionality)

Follow up for PR #22542

# Checklist for submitter
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2024-10-11 09:20:19 -05:00
Sarah Gillespie
6a25d6917a
Update batch software upload API to support increased max file size (#22839) 2024-10-11 08:38:08 -05:00
Tim Lee
c462bde176
osquery-perf battery reporting (#22666) 2024-10-10 10:03:21 -06:00
Ian Littman
0a8a396643
Ensure scripts set in no-team.yml can be used in run-script actions for No Team (#22809)
For #22787

Also revises the spec check to explain that scripts have to be defined
"controls" when used in policies for the same team, with an explicit
call-out for no-team.yml since this fix doesn't support pulling scripts
from the global file. This is because parsing and script-matching
happens early enough that we can't throw an error in the part of the
code where we bail when controls are defined in both no-team and default
files.

To minimize diff size, we're both "passing-by-ref" and returning the
maps-by-team of scripts and installers, though the former would be
sufficient on its own.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- N/A Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests (sorta)
- [x] Manual QA for all new/changed functionality
2024-10-10 06:12:24 -05:00
Tim Lee
1ecdad24ad
Remove panic recovery in CI tests (#22644) 2024-10-09 18:29:14 -06:00
Victor Lyuboslavsky
567803955e
NDES SCEP proxy backend (#22542)
#21955

<div>
<a href="https://www.loom.com/share/ba40b440502845d2861fd3ec7611bade">
<p>[Demo] Deploy SCEP certificates from Network Device Enrollment
Service (NDES) #21955 - Watch Video</p>
    </a>
<a href="https://www.loom.com/share/ba40b440502845d2861fd3ec7611bade">
<img style="max-width:300px;"
src="https://cdn.loom.com/sessions/thumbnails/ba40b440502845d2861fd3ec7611bade-84f2d88c9f5106c2-full-play.gif">
    </a>
  </div>

Note: A few remaining subtasks will be done in a follow-up PR. See
#22123 for a detailed list.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-10-09 13:47:27 -05:00
Sarah Gillespie
9c4671a37e
Increase max size and remove timeout for software installer uploads (#22725) 2024-10-08 14:27:55 -05:00
Roberto Dip
a9bd2fb7c1 Merge remote-tracking branch 'origin/main' into feat-fleet-app-library 2024-10-04 08:46:12 -03:00
Ian Littman
e4df7abb67
Backend build for script automation (#22472)
#22115, #22116

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

No changes file, as FE changes file covers the entire feature

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Co-authored-by: Tim Lee <timlee@fleetdm.com>
2024-10-03 20:03:40 -05:00
Gabriel Hernandez
d8fda9f085 Merge branch 'main' into feat-fleet-app-library 2024-10-02 10:01:25 +01:00
Lucas Manuel Rodriguez
f8f24e0a80
Add support to upload RPM packages (#22502)
#22473

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.

---------

Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com>
Co-authored-by: Ian Littman <iansltx@gmail.com>
2024-10-01 13:02:13 -03:00
Jahziel Villasana-Espinoza
80f0fd8889
fix: reset token team assignments to defaults (#22326)
> Related issue: #22198

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-10-01 10:26:16 -04:00
Gabriel Hernandez
35b4f20396 Merge branch 'main' into feat-fleet-app-library 2024-09-30 12:35:06 +01:00
Ian Littman
e8a6456f13
Don't reuse GitHub HTTP client to pull MSRC feeds (#22493)
See #22492

As of today, MSRC's API will 401 if you pass it a JWT it doesn't expect,
and by reusing the GitHub API HTTP client for MSRC pulls we were passing
the API an unexpected JWT. Wasn't able to reproduce this locally because
I didn't need a GitHub token locally to pull release details, while the
token is populated in Actions. Was able to repro both the issue and this
fix inside Actions on my fork.

This also updates to call v3.0 of the API directly, which v2.0 has been
redirecting to for awhile.

Finally, adds slightly better logging so we know which part of the feed
generation process we're in when we're running this in Actions.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
2024-09-27 21:23:48 -05:00
Lucas Manuel Rodriguez
f52e0a0e8c
Fix to not panic if MSRC for current month is not available (#21749)
See #21745.

---------

Co-authored-by: Ian Littman <iansltx@gmail.com>
2024-09-27 16:59:43 -05:00
Gabriel Hernandez
65668bed61 Merge branch 'main' into feat-fleet-app-library 2024-09-27 13:25:21 +01:00
Ian Littman
b43dab7198
Fix software-with-bundle-ID add when the same title with different/no bundle ID, add missing request timeout special case for edit package endpoint (#22413)
Same as #22412, for #21370, but against `main` rather than 4.57.0.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Roberto Dip <rroperzh@gmail.com>
2024-09-26 15:23:50 -03:00
Gabriel Hernandez
062b138c04 Merge branch 'main' into feat-fleet-app-library 2024-09-23 16:03:13 +01:00
Ian Littman
cbf563fb9b
Use sync.Map for stubbed key-value store to avoid data races in GitOps test (#22292)
This override only happens in testing, so this isn't release-blocking,
but this is the quickest way to clean up a test that will otherwise be
flaky due to data races, at the cost of performance (vs. setting up a
more complex solution with mutexes).

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality (via manually running
test using the KV store)
2024-09-23 06:58:13 -03:00
Lucas Manuel Rodriguez
8d664bd456
Make software batch endpoint asynchronous (#22258)
#22069

API changes: https://github.com/fleetdm/fleet/pull/22259

QAd by applying 10 pieces of software on a team, which took 3+ minutes
in total (which, before these changes was timing out at 100s.)

With this approach, a GitOps CI run timing out might leave the
background process running (which will eventually be applied to the
database). The team discussed and agreed that we can fix this edge case
later.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-09-20 11:55:47 -03:00
Gabriel Hernandez
4d54dc8a67 Merge branch 'main' into feat-fleet-app-library 2024-09-19 10:44:38 +01:00