fleet/pkg/spec/gitops_test.go
Scott Gress 34e7b5c358
Deprecate "team" and "query" API params (#39873)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** For #39344 

# Details 

This PR builds on the previous PR
(https://github.com/fleetdm/fleet/pull/39847) which added `renameto`
tags to certain API parameters to mark them as deprecated. How this is
used:

### In requests

* When decoding requests, log a warning if a `json` or `query` param is
used that has a `renameto` tag, e.g. if a `team_id` param is sent but
the related struct has `renameto:"fleet_id"` in it.
* If the `renamedto` version (e.g. `fleet_id`) is sent in the request,
rewrite it to the deprecated name so that it can be unmarshalled into
the struct
* If both versions are sent (e.g. `team_id` AND `fleet_id`), throw an
error and quit
* URLs with deprecated terms have new aliases using `WithAltPaths` --
warning on using old URLSs a TODO that will be handled in a subsequent
PR.

### In responses

* Output _both_ the deprecated and new names for fields that have
`renameto` tags, so that we don't break existing workflows expecting the
old keys. Uses a shared `DuplicateJSONKeys` to do the duplication.
* Most API responses are handled in `EncodeCommonResponse`. Exceptions
are activities, failing policy webhooks and the streaming "list hosts"
endpoints which call the function directly.

### In fleetctl

* Similar to requests, log warnings when deprecated keys are used and
rewrite the new keys internally so that they can be unmarshalled.
* For `fleetctl get` and `fleetctl generate-gitops`, _only_ output the
new names
* The set of keys to replace is hardcoded in `fleetctl` rather than
being dynamically generated as it is for API endpoints. Given the
mixture of typed and untyped data and the level of nesting, dynamic map
generation was very fragile and error-prone.

### Performance considerations

* The biggest performance hit is the addition of the JSON key rewriter
to the request pipeline. The rewriter buffers the entire request into
memory before eventually passing it to the decoder than unmarshals the
data into structs. I tried implementing this as a true streaming
rewriter but encountered issues where the request would hang if the
downstream reader (the decoder) encountered any errors. It's possible we
could implement this in a streaming fashion if we replace our [current
request
decoder](da43bf8371/server/service/endpoint_utils.go (L108))
with the v2 version, which is a bigger change requiring more thoughtful
discussion in the engineering team. As it stands, memory usage for
requests with deprecated fields will double while the request is being
decoded.
* The "alias rules" used to determine the old and new key names are
cached per struct type and for most endpoints are generated on server
start, so no performance impact is expected.
* Some `fleetctl` commands may have an extra unmarshal/marshal step but
as these are user-initiated and not performed in tight loops, the impact
should be minimal.

### TODO

* Log deprecation warnings when old URLs like "/fleet/teams" are used 
* Update API fields that the front-end uses to avoid deprecation
warnings
* Update `fleetctl apply` to accept/return `kind: fleet` rather than
`kind: team`
* Find/update any fleet server config vars with old language
* Update all error messages that use old language

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [X] QA'd all new/changed functionality manually

* Clicking around the front-end, no broken pages due to request
ingestion errors or bad responses
* Looking in network tab to verify that responses have both the old and
new keys
* Running `fleetctl generate-gitops` and verifying that the output looks
correct and can be ingested by `fleetctl gitops`
* Running `fleetctl get` and `fleetctl apply`

---------

Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
2026-02-19 13:53:32 -06:00

1595 lines
55 KiB
Go

package spec
import (
"fmt"
"os"
"path/filepath"
"slices"
"strings"
"testing"
"github.com/fleetdm/fleet/v4/pkg/file"
"github.com/fleetdm/fleet/v4/server/fleet"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
var topLevelOptions = map[string]string{
"controls": "controls:",
"reports": "reports:",
"policies": "policies:",
"agent_options": "agent_options:",
"org_settings": `
org_settings:
server_settings:
server_url: https://fleet.example.com
org_info:
contact_url: https://example.com/contact
org_logo_url: ""
org_logo_url_light_background: ""
org_name: Test Org
secrets:
`,
}
var teamLevelOptions = map[string]string{
"controls": "controls:",
"reports": "reports:",
"policies": "policies:",
"agent_options": "agent_options:",
"name": "name: TeamName",
"settings": `
settings:
secrets:
`,
}
func createTempFile(t *testing.T, pattern, contents string) (filePath string, baseDir string) {
tmpFile, err := os.CreateTemp(t.TempDir(), pattern)
require.NoError(t, err)
_, err = tmpFile.WriteString(contents)
require.NoError(t, err)
require.NoError(t, tmpFile.Close())
return tmpFile.Name(), filepath.Dir(tmpFile.Name())
}
func createNamedFileOnTempDir(t *testing.T, name string, contents string) (filePath string, baseDir string) {
tmpFilePath := filepath.Join(t.TempDir(), name)
tmpFile, err := os.Create(tmpFilePath)
require.NoError(t, err)
_, err = tmpFile.WriteString(contents)
require.NoError(t, err)
require.NoError(t, tmpFile.Close())
return tmpFile.Name(), filepath.Dir(tmpFile.Name())
}
func gitOpsFromString(t *testing.T, s string) (*GitOps, error) {
path, basePath := createTempFile(t, "", s)
return GitOpsFromFile(path, basePath, nil, nopLogf)
}
func nopLogf(_ string, _ ...interface{}) {
}
func TestValidGitOpsYaml(t *testing.T) {
t.Parallel()
tests := map[string]struct {
environment map[string]string
filePath string
isTeam bool
}{
"global_config_no_paths": {
environment: map[string]string{
"FLEET_SECRET_FLEET_SECRET_": "fleet_secret",
"FLEET_SECRET_NAME": "secret_name",
"FLEET_SECRET_LENGTH": "10",
"FLEET_SECRET_BANANA": "bread",
},
filePath: "testdata/global_config_no_paths.yml",
},
"global_config_with_paths": {
environment: map[string]string{
"LINUX_OS": "linux",
"DISTRIBUTED_DENYLIST_DURATION": "0",
"ORG_NAME": "Fleet Device Management",
"FLEET_SECRET_FLEET_SECRET_": "fleet_secret",
"FLEET_SECRET_NAME": "secret_name",
"FLEET_SECRET_LENGTH": "10",
"FLEET_SECRET_BANANA": "bread",
},
filePath: "testdata/global_config.yml",
},
"team_config_no_paths": {
environment: map[string]string{
"FLEET_SECRET_FLEET_SECRET_": "fleet_secret",
"FLEET_SECRET_NAME": "secret_name",
"FLEET_SECRET_LENGTH": "10",
"FLEET_SECRET_BANANA": "bread",
"FLEET_SECRET_CLEMENTINE": "not-an-orange",
"FLEET_SECRET_DURIAN": "fruity", // not used
"FLEET_SECRET_EGGPLANT": "parmesan",
},
filePath: "testdata/team_config_no_paths.yml",
isTeam: true,
},
"team_config_with_paths": {
environment: map[string]string{
"POLICY": "policy",
"LINUX_OS": "linux",
"DISTRIBUTED_DENYLIST_DURATION": "0",
"ENABLE_FAILING_POLICIES_WEBHOOK": "true",
"FLEET_SECRET_FLEET_SECRET_": "fleet_secret",
"FLEET_SECRET_NAME": "secret_name",
"FLEET_SECRET_LENGTH": "10",
"FLEET_SECRET_BANANA": "bread",
"FLEET_SECRET_CLEMENTINE": "not-an-orange",
"FLEET_SECRET_DURIAN": "fruity", // not used
"FLEET_SECRET_EGGPLANT": "parmesan",
},
filePath: "testdata/team_config.yml",
isTeam: true,
},
"team_config_with_paths_and_only_sha256": {
environment: map[string]string{
"POLICY": "policy",
"LINUX_OS": "linux",
"DISTRIBUTED_DENYLIST_DURATION": "0",
"ENABLE_FAILING_POLICIES_WEBHOOK": "true",
"FLEET_SECRET_FLEET_SECRET_": "fleet_secret",
"FLEET_SECRET_NAME": "secret_name",
"FLEET_SECRET_LENGTH": "10",
"FLEET_SECRET_BANANA": "bread",
"FLEET_SECRET_CLEMENTINE": "not-an-orange",
"FLEET_SECRET_DURIAN": "fruity", // not used
"FLEET_SECRET_EGGPLANT": "parmesan",
},
filePath: "testdata/team_config_only_sha256.yml",
isTeam: true,
},
}
for name, test := range tests {
test := test
name := name
t.Run(
name, func(t *testing.T) {
if len(test.environment) > 0 {
for k, v := range test.environment {
os.Setenv(k, v)
}
t.Cleanup(func() {
for k := range test.environment {
os.Unsetenv(k)
}
})
}
var appConfig *fleet.EnrichedAppConfig
if test.isTeam {
appConfig = &fleet.EnrichedAppConfig{}
appConfig.License = &fleet.LicenseInfo{
Tier: fleet.TierPremium,
}
}
gitops, err := GitOpsFromFile(test.filePath, "./testdata", appConfig, nopLogf)
require.NoError(t, err)
if test.isTeam {
// Check team settings
assert.Equal(t, "Team1", *gitops.TeamName)
assert.Contains(t, gitops.TeamSettings, "webhook_settings")
webhookSettings, ok := gitops.TeamSettings["webhook_settings"].(map[string]interface{})
assert.True(t, ok, "webhook_settings not found")
assert.Contains(t, webhookSettings, "failing_policies_webhook")
failingPoliciesWebhook, ok := webhookSettings["failing_policies_webhook"].(map[string]interface{})
assert.True(t, ok, "webhook_settings not found")
assert.Contains(t, failingPoliciesWebhook, "enable_failing_policies_webhook")
enableFailingPoliciesWebhook, ok := failingPoliciesWebhook["enable_failing_policies_webhook"].(bool)
assert.True(t, ok)
assert.True(t, enableFailingPoliciesWebhook)
assert.Contains(t, gitops.TeamSettings, "host_expiry_settings")
assert.Contains(t, gitops.TeamSettings, "features")
assert.Contains(t, gitops.TeamSettings, "secrets")
secrets, ok := gitops.TeamSettings["secrets"]
assert.True(t, ok, "secrets not found")
require.Len(t, secrets.([]*fleet.EnrollSecret), 2)
assert.Equal(t, "SampleSecret123", secrets.([]*fleet.EnrollSecret)[0].Secret)
assert.Equal(t, "ABC", secrets.([]*fleet.EnrollSecret)[1].Secret)
require.Len(t, gitops.Software.Packages, 2)
require.Len(t, gitops.FleetSecrets, 6)
for _, pkg := range gitops.Software.Packages {
if strings.Contains(pkg.URL, "MicrosoftTeams") {
assert.Equal(t, "testdata/lib/uninstall.sh", pkg.UninstallScript.Path)
assert.Contains(t, pkg.LabelsIncludeAny, "a")
assert.Contains(t, pkg.Categories, "Communication")
assert.Empty(t, pkg.LabelsExcludeAny)
} else {
assert.Empty(t, pkg.UninstallScript.Path)
assert.Contains(t, pkg.LabelsExcludeAny, "a")
assert.Empty(t, pkg.LabelsIncludeAny)
}
}
require.Len(t, gitops.Software.FleetMaintainedApps, 2)
for _, fma := range gitops.Software.FleetMaintainedApps {
switch fma.Slug {
case "slack/darwin":
require.ElementsMatch(t, fma.Categories, []string{"Productivity", "Communication"})
require.Empty(t, fma.PreInstallQuery)
require.Empty(t, fma.PostInstallScript)
require.Empty(t, fma.InstallScript)
require.Empty(t, fma.UninstallScript)
case "box-drive/windows":
require.ElementsMatch(t, fma.Categories, []string{"Productivity", "Developer tools"})
require.NotEmpty(t, fma.PreInstallQuery)
require.NotEmpty(t, fma.PostInstallScript)
require.NotEmpty(t, fma.InstallScript)
require.NotEmpty(t, fma.UninstallScript)
default:
assert.FailNow(t, "unexpected slug found in gitops file", "slug: %s", fma.Slug)
}
}
} else {
// Check org settings
serverSettings, ok := gitops.OrgSettings["server_settings"]
assert.True(t, ok, "server_settings not found")
assert.Equal(t, "https://fleet.example.com", serverSettings.(map[string]interface{})["server_url"])
assert.EqualValues(t, 2000, serverSettings.(map[string]interface{})["query_report_cap"])
assert.Contains(t, gitops.OrgSettings, "org_info")
orgInfo, ok := gitops.OrgSettings["org_info"].(map[string]interface{})
assert.True(t, ok)
assert.Equal(t, "Fleet Device Management", orgInfo["org_name"])
assert.Contains(t, gitops.OrgSettings, "smtp_settings")
assert.Contains(t, gitops.OrgSettings, "sso_settings")
assert.Contains(t, gitops.OrgSettings, "integrations")
assert.Contains(t, gitops.OrgSettings, "mdm")
assert.Contains(t, gitops.OrgSettings, "webhook_settings")
assert.Contains(t, gitops.OrgSettings, "fleet_desktop")
assert.Contains(t, gitops.OrgSettings, "host_expiry_settings")
assert.Contains(t, gitops.OrgSettings, "activity_expiry_settings")
assert.Contains(t, gitops.OrgSettings, "features")
assert.Contains(t, gitops.OrgSettings, "vulnerability_settings")
assert.Contains(t, gitops.OrgSettings, "secrets")
secrets, ok := gitops.OrgSettings["secrets"]
assert.True(t, ok, "secrets not found")
require.Len(t, secrets.([]*fleet.EnrollSecret), 2)
assert.Equal(t, "SampleSecret123", secrets.([]*fleet.EnrollSecret)[0].Secret)
assert.Equal(t, "ABC", secrets.([]*fleet.EnrollSecret)[1].Secret)
activityExpirySettings, ok := gitops.OrgSettings["activity_expiry_settings"].(map[string]interface{})
require.True(t, ok)
activityExpiryEnabled, ok := activityExpirySettings["activity_expiry_enabled"].(bool)
require.True(t, ok)
require.True(t, activityExpiryEnabled)
activityExpiryWindow, ok := activityExpirySettings["activity_expiry_window"].(float64)
require.True(t, ok)
require.Equal(t, 30, int(activityExpiryWindow))
require.Len(t, gitops.FleetSecrets, 4)
// Check labels
require.Len(t, gitops.Labels, 2)
assert.Equal(t, "Global label numero uno", gitops.Labels[0].Name)
assert.Equal(t, "Global label numero dos", gitops.Labels[1].Name)
assert.Equal(t, "SELECT 1 FROM osquery_info", gitops.Labels[0].Query)
require.Len(t, gitops.Labels[1].Hosts, 2)
assert.Equal(t, "host1", gitops.Labels[1].Hosts[0])
assert.Equal(t, "2", gitops.Labels[1].Hosts[1])
}
// Check controls
_, ok := gitops.Controls.MacOSSettings.(fleet.MacOSSettings)
assert.True(t, ok, "macos_settings not found")
_, ok = gitops.Controls.WindowsSettings.(fleet.WindowsSettings)
assert.True(t, ok, "windows_settings not found")
_, ok = gitops.Controls.EnableDiskEncryption.(bool)
assert.True(t, ok, "enable_disk_encryption not found")
_, ok = gitops.Controls.MacOSMigration.(map[string]interface{})
assert.True(t, ok, "macos_migration not found")
assert.NotNil(t, gitops.Controls.MacOSSetup, "macos_setup not found")
_, ok = gitops.Controls.MacOSUpdates.(map[string]interface{})
assert.True(t, ok, "macos_updates not found")
_, ok = gitops.Controls.IOSUpdates.(map[string]interface{})
assert.True(t, ok, "ios_updates not found")
_, ok = gitops.Controls.IPadOSUpdates.(map[string]interface{})
assert.True(t, ok, "ipados_updates not found")
_, ok = gitops.Controls.WindowsEnabledAndConfigured.(bool)
assert.True(t, ok, "windows_enabled_and_configured not found")
_, ok = gitops.Controls.WindowsMigrationEnabled.(bool)
assert.True(t, ok, "windows_migration_enabled not found")
_, ok = gitops.Controls.EnableTurnOnWindowsMDMManually.(bool)
assert.True(t, ok, "enable_turn_on_windows_mdm_manually not found")
_, ok = gitops.Controls.WindowsEntraTenantIDs.([]any)
assert.True(t, ok, "windows_entra_tenant_ids not found")
_, ok = gitops.Controls.WindowsUpdates.(map[string]interface{})
assert.True(t, ok, "windows_updates not found")
assert.Equal(t, "fleet_secret", gitops.FleetSecrets["FLEET_SECRET_FLEET_SECRET_"])
assert.Equal(t, "secret_name", gitops.FleetSecrets["FLEET_SECRET_NAME"])
assert.Equal(t, "10", gitops.FleetSecrets["FLEET_SECRET_LENGTH"])
assert.Equal(t, "bread", gitops.FleetSecrets["FLEET_SECRET_BANANA"])
// Check agent options
assert.NotNil(t, gitops.AgentOptions)
assert.Contains(t, string(*gitops.AgentOptions), "\"distributed_denylist_duration\":0")
// Check reports
require.Len(t, gitops.Queries, 3)
assert.Equal(t, "Scheduled query stats", gitops.Queries[0].Name)
assert.Equal(t, "orbit_info", gitops.Queries[1].Name)
assert.Equal(t, "darwin,linux,windows", gitops.Queries[1].Platform)
assert.Equal(t, "osquery_info", gitops.Queries[2].Name)
// Check software
if test.isTeam {
require.Len(t, gitops.Software.Packages, 2)
if name == "team_config_with_paths_and_only_sha256" {
require.Empty(t, gitops.Software.Packages[0].URL)
require.True(t, gitops.Software.Packages[0].InstallDuringSetup.Value)
require.True(t, gitops.Software.Packages[1].InstallDuringSetup.Value)
} else {
require.Equal(t, "https://statics.teams.cdn.office.net/production-osx/enterprise/webview2/lkg/MicrosoftTeams.pkg", gitops.Software.Packages[0].URL)
}
require.Equal(t, "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef", gitops.Software.Packages[0].SHA256)
require.False(t, gitops.Software.Packages[0].SelfService)
require.Equal(t, "https://ftp.mozilla.org/pub/firefox/releases/129.0.2/mac/en-US/Firefox%20129.0.2.pkg", gitops.Software.Packages[1].URL)
require.True(t, gitops.Software.Packages[1].SelfService)
require.Len(t, gitops.Software.AppStoreApps, 1)
require.Equal(t, gitops.Software.AppStoreApps[0].AppStoreID, "123456")
require.False(t, gitops.Software.AppStoreApps[0].SelfService)
}
// Check policies
expectedPoliciesCount := 5
if test.isTeam {
expectedPoliciesCount = 9
}
require.Len(t, gitops.Policies, expectedPoliciesCount)
assert.Equal(t, "😊 Failing policy", gitops.Policies[0].Name)
assert.Equal(t, "Passing policy", gitops.Policies[1].Name)
assert.Equal(t, "No root logins (macOS, Linux)", gitops.Policies[2].Name)
assert.Equal(t, "🔥 Failing policy", gitops.Policies[3].Name)
assert.Equal(t, "linux", gitops.Policies[3].Platform)
assert.Equal(t, "😊😊 Failing policy", gitops.Policies[4].Name)
if test.isTeam {
assert.Equal(t, "Microsoft Teams on macOS installed and up to date", gitops.Policies[5].Name)
assert.NotNil(t, gitops.Policies[5].InstallSoftware)
if name == "team_config_with_paths_and_only_sha256" {
assert.Equal(t, "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef", gitops.Policies[5].InstallSoftware.HashSHA256)
} else {
assert.Equal(t, "./microsoft-teams.pkg.software.yml", gitops.Policies[5].InstallSoftware.PackagePath)
assert.Equal(t, "https://statics.teams.cdn.office.net/production-osx/enterprise/webview2/lkg/MicrosoftTeams.pkg", gitops.Policies[5].InstallSoftwareURL)
}
assert.Equal(t, "Slack on macOS is installed", gitops.Policies[6].Name)
assert.NotNil(t, gitops.Policies[6].InstallSoftware)
assert.Equal(t, "123456", gitops.Policies[6].InstallSoftware.AppStoreID)
assert.Equal(t, "Script run policy", gitops.Policies[7].Name)
assert.NotNil(t, gitops.Policies[7].RunScript)
assert.Equal(t, "./lib/collect-fleetd-logs.sh", gitops.Policies[7].RunScript.Path)
assert.Equal(t, "🔥 Failing policy with script", gitops.Policies[8].Name)
assert.NotNil(t, gitops.Policies[8].RunScript)
// . or .. depending on whether with paths or without
assert.Contains(t, gitops.Policies[8].RunScript.Path, "./lib/collect-fleetd-logs.sh")
}
},
)
}
}
func TestDuplicatePolicyNames(t *testing.T) {
t.Parallel()
config := getGlobalConfig([]string{"policies"})
config += `
policies:
- name: My policy
platform: linux
query: SELECT 1 FROM osquery_info WHERE start_time < 0;
- name: My policy
platform: windows
query: SELECT 1;
`
_, err := gitOpsFromString(t, config)
assert.ErrorContains(t, err, "duplicate policy names")
}
func TestManualLabelEmptyHostList(t *testing.T) {
t.Parallel()
config := getGlobalConfig([]string{})
config += `
labels:
- name: TestLabel
description: Label for testing
hosts:
label_membership_type: manual`
gitops, err := gitOpsFromString(t, config)
require.NoError(t, err)
assert.NotNil(t, gitops.Labels[0].Hosts)
}
func TestDuplicateQueryNames(t *testing.T) {
t.Parallel()
config := getGlobalConfig([]string{"reports"})
config += `
reports:
- name: orbit_info
query: SELECT * from orbit_info;
interval: 0
platform: darwin,linux,windows
min_osquery_version: all
observer_can_run: false
automations_enabled: true
logging: snapshot
- name: orbit_info
query: SELECT 1;
interval: 300
platform: windows
min_osquery_version: all
observer_can_run: false
automations_enabled: true
logging: snapshot
`
_, err := gitOpsFromString(t, config)
assert.ErrorContains(t, err, "duplicate query names")
}
func TestUnicodeQueryNames(t *testing.T) {
t.Parallel()
config := getGlobalConfig([]string{"reports"})
config += `
reports:
- name: 😊 orbit_info
query: SELECT * from orbit_info;
interval: 0
platform: darwin,linux,windows
min_osquery_version: all
observer_can_run: false
automations_enabled: true
logging: snapshot
`
_, err := gitOpsFromString(t, config)
assert.ErrorContains(t, err, "query name must be in ASCII")
}
func TestUnicodeTeamName(t *testing.T) {
t.Parallel()
config := getTeamConfig([]string{"name"})
config += `name: 😊 TeamName`
_, err := gitOpsFromString(t, config)
assert.NoError(t, err)
}
func TestVarExpansion(t *testing.T) {
os.Setenv("MACOS_OS", "darwin")
os.Setenv("LINUX_OS", "linux")
os.Setenv("EMPTY_VAR", "")
t.Cleanup(func() {
os.Unsetenv("MACOS_OS")
os.Unsetenv("LINUX_OS")
os.Unsetenv("EMPTY_VAR")
})
config := getGlobalConfig([]string{"reports"})
config += `
reports:
- name: orbit_info \$NOT_EXPANDED \\\$ALSO_NOT_EXPANDED
query: "SELECT * from orbit_info; -- double quotes are escaped by YAML after Fleet's escaping of backslashes \\\\\$NOT_EXPANDED"
interval: 0
platform: $MACOS_OS,${LINUX_OS},windows$EMPTY_VAR
min_osquery_version: all
observer_can_run: false
automations_enabled: true
logging: snapshot
description: 'single quotes are not escaped by YAML \\\$NOT_EXPANDED'
`
gitOps, err := gitOpsFromString(t, config)
require.NoError(t, err)
require.Len(t, gitOps.Queries, 1)
require.Equal(t, "darwin,linux,windows", gitOps.Queries[0].Platform)
require.Equal(t, `orbit_info $NOT_EXPANDED \$ALSO_NOT_EXPANDED`, gitOps.Queries[0].Name)
require.Equal(t, `single quotes are not escaped by YAML \$NOT_EXPANDED`, gitOps.Queries[0].Description)
require.Equal(t, `SELECT * from orbit_info; -- double quotes are escaped by YAML after Fleet's escaping of backslashes \$NOT_EXPANDED`, gitOps.Queries[0].Query)
config = getGlobalConfig([]string{"reports"})
config += `
reports:
- name: orbit_info $NOT_DEFINED
query: SELECT * from orbit_info;
interval: 0
platform: darwin,linux,windows
min_osquery_version: all
observer_can_run: false
automations_enabled: true
logging: snapshot
`
_, err = gitOpsFromString(t, config)
require.Error(t, err)
require.Contains(t, err.Error(), "environment variable \"NOT_DEFINED\" not set")
}
func TestMixingGlobalAndTeamConfig(t *testing.T) {
t.Parallel()
// Mixing org_settings and team name
config := getGlobalConfig(nil)
config += "name: TeamName\n"
_, err := gitOpsFromString(t, config)
assert.ErrorContains(t, err, "'org_settings' cannot be used with 'name', 'settings'")
// Mixing org_settings and settings (formerly settings)
config = getGlobalConfig(nil)
config += "settings:\n secrets: []\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "'org_settings' cannot be used with 'name', 'settings'")
// Mixing org_settings and team name and settings
config = getGlobalConfig(nil)
config += "name: TeamName\n"
config += "settings:\n secrets: []\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "'org_settings' cannot be used with 'name', 'settings'")
}
func TestInvalidGitOpsYaml(t *testing.T) {
t.Parallel()
// Bad YAML
_, err := gitOpsFromString(t, "bad:\nbad")
assert.ErrorContains(t, err, "failed to unmarshal")
for _, name := range []string{"global", "team"} {
t.Run(
name, func(t *testing.T) {
isTeam := name == "team"
getConfig := getGlobalConfig
if isTeam {
getConfig = getTeamConfig
}
if isTeam {
// Invalid top level key
config := getConfig(nil)
config += "unknown_key:\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "unknown top-level field")
// Invalid team name
config = getConfig([]string{"name"})
config += "name: [2]\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "expected type string but got array")
// Missing team name
config = getConfig([]string{"name"})
config += "name:\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "'name' is required")
// Invalid settings
config = getConfig([]string{"settings"})
config += "settings:\n path: [2]\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "expected type string but got array")
// Invalid settings in a separate file
tmpFile, err := os.CreateTemp(t.TempDir(), "*settings.yml")
require.NoError(t, err)
_, err = tmpFile.WriteString("[2]")
require.NoError(t, err)
config = getConfig([]string{"settings"})
config += fmt.Sprintf("%s:\n path: %s\n", "settings", tmpFile.Name())
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "expected type spec.BaseItem but got array")
// Invalid secrets 1
config = getConfig([]string{"settings"})
config += "settings:\n secrets: bad\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "must be a list of secret items")
// Invalid secrets 2
config = getConfig([]string{"settings"})
config += "settings:\n secrets: [2]\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "must have a 'secret' key")
// Missing settings (formerly settings).
config = getConfig([]string{"settings"})
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "'settings' is required when 'name' is provided")
// settings is now allowed on "no-team.yml" for webhook settings
config = getConfig([]string{"name", "settings"}) // Exclude settings with secrets
config += "name: No team\n"
noTeamPath1, noTeamBasePath1 := createNamedFileOnTempDir(t, "no-team.yml", config)
gitops, err := GitOpsFromFile(noTeamPath1, noTeamBasePath1, nil, nopLogf)
assert.NoError(t, err)
assert.NotNil(t, gitops)
// No team with valid webhook_settings should work
config = getConfig([]string{"name", "settings"})
config += "name: No team\nsettings:\n webhook_settings:\n failing_policies_webhook:\n enable_failing_policies_webhook: true\n"
noTeamPath2, noTeamBasePath2 := createNamedFileOnTempDir(t, "no-team.yml", config)
gitops, err = GitOpsFromFile(noTeamPath2, noTeamBasePath2, nil, nopLogf)
assert.NoError(t, err)
assert.NotNil(t, gitops)
// No team with invalid settings option should fail
config = getConfig([]string{"name", "settings"})
config += "name: No team\nsettings:\n features:\n enable_host_users: false\n"
noTeamPath3, noTeamBasePath3 := createNamedFileOnTempDir(t, "no-team.yml", config)
_, err = GitOpsFromFile(noTeamPath3, noTeamBasePath3, nil, nopLogf)
assert.ErrorContains(t, err, "unsupported settings option 'features' for 'No team' - only 'webhook_settings' is allowed")
// No team with multiple settings options (one valid, one invalid) should fail
config = getConfig([]string{"name", "settings"})
config += "name: No team\nsettings:\n webhook_settings:\n failing_policies_webhook:\n enable_failing_policies_webhook: true\n secrets:\n - secret: test\n"
noTeamPath4, noTeamBasePath4 := createNamedFileOnTempDir(t, "no-team.yml", config)
_, err = GitOpsFromFile(noTeamPath4, noTeamBasePath4, nil, nopLogf)
assert.ErrorContains(t, err, "unsupported settings option 'secrets' for 'No team' - only 'webhook_settings' is allowed")
// No team with host_status_webhook in webhook_settings should fail
config = getConfig([]string{"name", "settings"})
config += "name: No team\nsettings:\n webhook_settings:\n host_status_webhook:\n enable_host_status_webhook: true\n failing_policies_webhook:\n enable_failing_policies_webhook: true\n"
noTeamPath5a, noTeamBasePath5a := createNamedFileOnTempDir(t, "no-team.yml", config)
_, err = GitOpsFromFile(noTeamPath5a, noTeamBasePath5a, nil, nopLogf)
assert.ErrorContains(t, err, "unsupported webhook_settings option 'host_status_webhook' for 'No team'; only 'failing_policies_webhook' is allowed")
// No team with vulnerabilities_webhook in webhook_settings should fail
config = getConfig([]string{"name", "settings"})
config += "name: No team\nsettings:\n webhook_settings:\n vulnerabilities_webhook:\n enable_vulnerabilities_webhook: true\n"
noTeamPath5b, noTeamBasePath5b := createNamedFileOnTempDir(t, "no-team.yml", config)
_, err = GitOpsFromFile(noTeamPath5b, noTeamBasePath5b, nil, nopLogf)
assert.ErrorContains(t, err, "unsupported webhook_settings option 'vulnerabilities_webhook' for 'No team'; only 'failing_policies_webhook' is allowed")
// 'No team' file with invalid name.
config = getConfig([]string{"name", "settings"})
config += "name: No team\n"
noTeamPath6, noTeamBasePath6 := createNamedFileOnTempDir(t, "foobar.yml", config)
_, err = GitOpsFromFile(noTeamPath6, noTeamBasePath6, nil, nopLogf)
assert.ErrorContains(t, err, fmt.Sprintf("file %q for 'No team' must be named 'no-team.yml'", noTeamPath6))
// Missing secrets
config = getConfig([]string{"settings"})
config += "settings:\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "'settings.secrets' is required")
} else {
// 'software' is not allowed in global config
config := getConfig(nil)
config += "software:\n packages:\n - url: https://example.com\n"
path1, basePath1 := createTempFile(t, "", config)
appConfig := fleet.EnrichedAppConfig{}
appConfig.License = &fleet.LicenseInfo{
Tier: fleet.TierPremium,
}
_, err = GitOpsFromFile(path1, basePath1, &appConfig, nopLogf)
assert.ErrorContains(t, err, "'software' cannot be set on global file")
// Invalid org_settings
config = getConfig([]string{"org_settings"})
config += "org_settings:\n path: [2]\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "expected type string but got array")
// Invalid org_settings in a separate file
tmpFile, err := os.CreateTemp(t.TempDir(), "*org_settings.yml")
require.NoError(t, err)
_, err = tmpFile.WriteString("[2]")
require.NoError(t, err)
config = getConfig([]string{"org_settings"})
config += fmt.Sprintf("%s:\n path: %s\n", "org_settings", tmpFile.Name())
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "expected type spec.BaseItem but got array")
// Invalid secrets 1
config = getConfig([]string{"org_settings"})
config += "org_settings:\n secrets: bad\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "must be a list of secret items")
// Invalid secrets 2
config = getConfig([]string{"org_settings"})
config += "org_settings:\n secrets: [2]\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "must have a 'secret' key")
// Missing secrets
config = getConfig([]string{"org_settings"})
config += "org_settings:\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "'org_settings.secrets' is required")
// Bad label spec (float instead of string in hosts)
config = getConfig([]string{"labels"})
config += "labels:\n - name: TestLabel\n description: Label for testing\n hosts:\n - 2.5\n label_membership_type: manual\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "hosts must be strings or integers, got float 2.5")
}
// Invalid agent_options
config := getConfig([]string{"agent_options"})
config += "agent_options:\n path: [2]\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "expected type string but got array")
// Invalid agent_options in a separate file
tmpFile, err := os.CreateTemp(t.TempDir(), "*agent_options.yml")
require.NoError(t, err)
_, err = tmpFile.WriteString("[2]")
require.NoError(t, err)
config = getConfig([]string{"agent_options"})
config += fmt.Sprintf("%s:\n path: %s\n", "agent_options", tmpFile.Name())
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "expected type spec.BaseItem but got array")
// Invalid controls
config = getConfig([]string{"controls"})
config += "controls:\n path: [2]\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "expected type string but got array")
// Invalid controls in a separate file
tmpFile, err = os.CreateTemp(t.TempDir(), "*controls.yml")
require.NoError(t, err)
_, err = tmpFile.WriteString("[2]")
require.NoError(t, err)
config = getConfig([]string{"controls"})
config += fmt.Sprintf("%s:\n path: %s\n", "controls", tmpFile.Name())
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "expected type spec.GitOpsControls but got array")
// Invalid policies
config = getConfig([]string{"policies"})
config += "policies:\n path: [2]\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "expected type []spec.Policy but got object")
// Invalid policies in a separate file
tmpFile, err = os.CreateTemp(t.TempDir(), "*policies.yml")
require.NoError(t, err)
_, err = tmpFile.WriteString("[2]")
require.NoError(t, err)
config = getConfig([]string{"policies"})
config += fmt.Sprintf("%s:\n - path: %s\n", "policies", tmpFile.Name())
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "expected type spec.Policy but got number")
// Policy name missing
config = getConfig([]string{"policies"})
config += "policies:\n - query: SELECT 1;\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "name is required")
// Policy query missing
config = getConfig([]string{"policies"})
config += "policies:\n - name: Test Policy\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "query is required")
// Invalid reports
config = getConfig([]string{"reports"})
config += "reports:\n path: [2]\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "expected type []spec.Query but got object")
// Invalid reports in a separate file
tmpFile, err = os.CreateTemp(t.TempDir(), "*reports.yml")
require.NoError(t, err)
_, err = tmpFile.WriteString("[2]")
require.NoError(t, err)
config = getConfig([]string{"reports"})
config += fmt.Sprintf("%s:\n - path: %s\n", "reports", tmpFile.Name())
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "expected type spec.Query but got number")
// Query name missing
config = getConfig([]string{"reports"})
config += "reports:\n - query: SELECT 1;\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "name is required")
// Query SQL query missing
config = getConfig([]string{"reports"})
config += "reports:\n - name: Test Query\n"
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "query is required")
},
)
}
}
func TestTopLevelGitOpsValidation(t *testing.T) {
t.Parallel()
tests := map[string]struct {
optsToExclude []string
shouldPass bool
isTeam bool
}{
"all_present_global": {
optsToExclude: []string{},
shouldPass: true,
},
"all_present_team": {
optsToExclude: []string{},
shouldPass: true,
isTeam: true,
},
"missing_all": {
optsToExclude: []string{"controls", "reports", "policies", "agent_options", "org_settings"},
},
"missing_reports": {
optsToExclude: []string{"reports"},
},
"missing_policies": {
optsToExclude: []string{"policies"},
},
"missing_agent_options": {
optsToExclude: []string{"agent_options"},
},
"missing_org_settings": {
optsToExclude: []string{"org_settings"},
},
"missing_name": {
optsToExclude: []string{"name"},
isTeam: true,
},
"missing_settings": {
optsToExclude: []string{"settings"},
isTeam: true,
},
}
for name, test := range tests {
t.Run(
name, func(t *testing.T) {
var config string
if test.isTeam {
config = getTeamConfig(test.optsToExclude)
} else {
config = getGlobalConfig(test.optsToExclude)
}
_, err := gitOpsFromString(t, config)
if test.shouldPass {
assert.NoError(t, err)
} else {
assert.ErrorContains(t, err, "is required")
}
},
)
}
}
func TestGitOpsNullArrays(t *testing.T) {
t.Parallel()
config := getGlobalConfig([]string{"reports", "policies"})
config += "reports: null\npolicies: ~\n"
gitops, err := gitOpsFromString(t, config)
assert.NoError(t, err)
assert.Nil(t, gitops.Queries)
assert.Nil(t, gitops.Policies)
}
func TestGitOpsPaths(t *testing.T) {
t.Parallel()
tests := map[string]struct {
isArray bool
isTeam bool
goodConfig string
}{
"org_settings": {
isArray: false,
goodConfig: "secrets: []\n",
},
"settings": {
isArray: false,
isTeam: true,
goodConfig: "secrets: []\n",
},
"controls": {
isArray: false,
goodConfig: "windows_enabled_and_configured: true\n",
},
"reports": {
isArray: true,
goodConfig: "[]",
},
"policies": {
isArray: true,
goodConfig: "[]",
},
"agent_options": {
isArray: false,
goodConfig: "name: value\n",
},
}
for name, test := range tests {
test := test
name := name
t.Run(
name, func(t *testing.T) {
t.Parallel()
getConfig := getGlobalConfig
if test.isTeam {
getConfig = getTeamConfig
}
// Test an absolute top level path
tmpDir := t.TempDir()
tmpFile, err := os.CreateTemp(tmpDir, "*good.yml")
require.NoError(t, err)
_, err = tmpFile.WriteString(test.goodConfig)
require.NoError(t, err)
config := getConfig([]string{name})
if test.isArray {
config += fmt.Sprintf("%s:\n - path: %s\n", name, tmpFile.Name())
} else {
config += fmt.Sprintf("%s:\n path: %s\n", name, tmpFile.Name())
}
_, err = gitOpsFromString(t, config)
assert.NoError(t, err)
// Test a relative top level path
config = getConfig([]string{name})
mainTmpFile, err := os.CreateTemp(tmpDir, "*main.yml")
require.NoError(t, err)
dir, file := filepath.Split(tmpFile.Name())
if test.isArray {
config += fmt.Sprintf("%s:\n - path: ./%s\n", name, file)
} else {
config += fmt.Sprintf("%s:\n path: ./%s\n", name, file)
}
err = os.WriteFile(mainTmpFile.Name(), []byte(config), 0o644)
require.NoError(t, err)
_, err = GitOpsFromFile(mainTmpFile.Name(), dir, nil, nopLogf)
assert.NoError(t, err)
// Test a bad path
config = getConfig([]string{name})
if test.isArray {
config += fmt.Sprintf("%s:\n - path: ./%s\n", name, "doesNotExist.yml")
} else {
config += fmt.Sprintf("%s:\n path: ./%s\n", name, "doesNotExist.yml")
}
err = os.WriteFile(mainTmpFile.Name(), []byte(config), 0o644)
require.NoError(t, err)
_, err = GitOpsFromFile(mainTmpFile.Name(), dir, nil, nopLogf)
assert.ErrorContains(t, err, "no such file or directory")
// Test a bad file -- cannot be unmarshalled
tmpFileBad, err := os.CreateTemp(t.TempDir(), "*invalid.yml")
require.NoError(t, err)
_, err = tmpFileBad.WriteString("bad:\nbad")
require.NoError(t, err)
config = getConfig([]string{name})
if test.isArray {
config += fmt.Sprintf("%s:\n - path: %s\n", name, tmpFileBad.Name())
} else {
config += fmt.Sprintf("%s:\n path: %s\n", name, tmpFileBad.Name())
}
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "failed to unmarshal")
// Test a nested path -- bad
tmpFileBad, err = os.CreateTemp(filepath.Dir(mainTmpFile.Name()), "*bad.yml")
require.NoError(t, err)
if test.isArray {
_, err = tmpFileBad.WriteString(fmt.Sprintf("- path: %s\n", tmpFile.Name()))
} else {
_, err = tmpFileBad.WriteString(fmt.Sprintf("path: %s\n", tmpFile.Name()))
}
require.NoError(t, err)
config = getConfig([]string{name})
dir, file = filepath.Split(tmpFileBad.Name())
if test.isArray {
config += fmt.Sprintf("%s:\n - path: ./%s\n", name, file)
} else {
config += fmt.Sprintf("%s:\n path: ./%s\n", name, file)
}
err = os.WriteFile(mainTmpFile.Name(), []byte(config), 0o644)
require.NoError(t, err)
_, err = GitOpsFromFile(mainTmpFile.Name(), dir, nil, nopLogf)
assert.ErrorContains(t, err, "nested paths are not supported")
},
)
}
}
func TestGitOpsGlobalPolicyWithInstallSoftware(t *testing.T) {
t.Parallel()
config := getGlobalConfig([]string{"policies"})
config += `
policies:
- name: Some policy
query: SELECT 1;
install_software:
package_path: ./some_path.yml
`
_, err := gitOpsFromString(t, config)
assert.ErrorContains(t, err, "install_software can only be set on team policies")
}
func TestGitOpsGlobalPolicyWithRunScript(t *testing.T) {
t.Parallel()
config := getGlobalConfig([]string{"policies"})
config += `
policies:
- name: Some policy
query: SELECT 1;
run_script:
path: ./some_path.sh
`
_, err := gitOpsFromString(t, config)
assert.ErrorContains(t, err, "run_script can only be set on team policies")
}
func TestGitOpsTeamPolicyWithInvalidInstallSoftware(t *testing.T) {
t.Parallel()
config := getTeamConfig([]string{"policies"})
config += `
policies:
- name: Some policy
query: SELECT 1;
install_software:
package_path: ./some_path.yml
`
_, err := gitOpsFromString(t, config)
assert.ErrorContains(t, err, "failed to read install_software.package_path file")
config = getTeamConfig([]string{"policies"})
config += `
policies:
- name: Some policy
query: SELECT 1;
install_software:
package_path:
`
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "install_software must include either a package_path, an app_store_id or a hash_sha256")
config = getTeamConfig([]string{"policies"})
config += `
policies:
- name: Some policy
query: SELECT 1;
install_software:
package_path: ./some_path.yml
app_store_id: "123456"
`
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "must have only one of package_path or app_store_id")
// Software has a URL that's too big
tooBigURL := fmt.Sprintf("https://ftp.mozilla.org/%s", strings.Repeat("a", 4000-23))
config = getTeamConfig([]string{"software"})
config += fmt.Sprintf(`
software:
packages:
- url: %s
`, tooBigURL)
appConfig := fleet.EnrichedAppConfig{}
appConfig.License = &fleet.LicenseInfo{
Tier: fleet.TierPremium,
}
path, basePath := createTempFile(t, "", config)
_, err = GitOpsFromFile(path, basePath, &appConfig, nopLogf)
assert.ErrorContains(t, err, fmt.Sprintf("software URL \"%s\" is too long, must be 4000 characters or less", tooBigURL))
// Software URL isn't a valid URL
config = getTeamConfig([]string{"software"})
invalidURL := "1.2.3://"
config += fmt.Sprintf(`
software:
packages:
- url: %s
`, invalidURL)
path, basePath = createTempFile(t, "", config)
_, err = GitOpsFromFile(path, basePath, &appConfig, nopLogf)
assert.ErrorContains(t, err, fmt.Sprintf("%s is not a valid URL", invalidURL))
// Software URL refers to a .exe but doesn't have (un)install scripts specified
config = getTeamConfig([]string{"software"})
exeURL := "https://download-installer.cdn.mozilla.net/pub/firefox/releases/136.0.4/win64/en-US/Firefox%20Setup%20136.0.4.exe?foo=bar"
config += fmt.Sprintf(`
software:
packages:
- url: %s
`, exeURL)
path, basePath = createTempFile(t, "", config)
_, err = GitOpsFromFile(path, basePath, &appConfig, nopLogf)
assert.ErrorContains(t, err, fmt.Sprintf("software URL %s refers to an .exe package, which requires both install_script and uninstall_script", exeURL))
// Software URL refers to a .tar.gz but doesn't have (un)install scripts specified (URL doesn't exist as Firefox is all .tar.xz)
config = getTeamConfig([]string{"software"})
tgzURL := "https://download-installer.cdn.mozilla.net/pub/firefox/releases/137.0.2/linux-x86_64/en-US/firefox-137.0.2.tar.gz?foo=baz"
config += fmt.Sprintf(`
software:
packages:
- url: %s
`, tgzURL)
path, basePath = createTempFile(t, "", config)
_, err = GitOpsFromFile(path, basePath, &appConfig, nopLogf)
assert.ErrorContains(t, err, fmt.Sprintf("software URL %s refers to a .tar.gz archive, which requires both install_script and uninstall_script", tgzURL))
// Policy references a VPP app not present on the team
config = getTeamConfig([]string{"policies"})
config += `
policies:
- name: Some policy
query: SELECT 1;
install_software:
app_store_id: "123456"
`
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "not found on team")
// Policy references a software installer not present in the team.
config = getTeamConfig([]string{"policies"})
config += `
policies:
- path: ./team_install_software.policies.yml
software:
packages:
- url: https://ftp.mozilla.org/pub/firefox/releases/129.0.2/mac/en-US/Firefox%20129.0.2.pkg
self_service: true
`
path, basePath = createTempFile(t, "", config)
err = file.Copy(
filepath.Join("testdata", "team_install_software.policies.yml"),
filepath.Join(basePath, "team_install_software.policies.yml"),
0o755,
)
require.NoError(t, err)
err = file.Copy(
filepath.Join("testdata", "microsoft-teams.pkg.software.yml"),
filepath.Join(basePath, "microsoft-teams.pkg.software.yml"),
0o755,
)
require.NoError(t, err)
_, err = GitOpsFromFile(path, basePath, &appConfig, nopLogf)
assert.ErrorContains(t, err,
"install_software.package_path URL https://statics.teams.cdn.office.net/production-osx/enterprise/webview2/lkg/MicrosoftTeams.pkg not found on team",
)
// Policy references a software installer file that has an invalid yaml.
config = getTeamConfig([]string{"policies"})
config += `
policies:
- path: ./team_install_software.policies.yml
software:
packages:
- url: https://ftp.mozilla.org/pub/firefox/releases/129.0.2/mac/en-US/Firefox%20129.0.2.pkg
self_service: true
`
path, basePath = createTempFile(t, "", config)
err = file.Copy(
filepath.Join("testdata", "team_install_software.policies.yml"),
filepath.Join(basePath, "team_install_software.policies.yml"),
0o755,
)
require.NoError(t, err)
err = os.WriteFile( // nolint:gosec
filepath.Join(basePath, "microsoft-teams.pkg.software.yml"),
[]byte("invalid yaml"),
0o755,
)
require.NoError(t, err)
appConfig = fleet.EnrichedAppConfig{}
appConfig.License = &fleet.LicenseInfo{
Tier: fleet.TierPremium,
}
_, err = GitOpsFromFile(path, basePath, &appConfig, nopLogf)
assert.ErrorContains(t, err, "file \"./microsoft-teams.pkg.software.yml\" does not contain a valid software package definition")
// Policy references a software installer file that has multiple pieces of software specified
config = getTeamConfig([]string{"policies"})
config += `
policies:
- path: ./multipkg.policies.yml
software:
packages:
- path: ./multiple-packages.yml
`
path, basePath = createTempFile(t, "", config)
err = file.Copy(
filepath.Join("testdata", "multipkg.policies.yml"),
filepath.Join(basePath, "multipkg.policies.yml"),
0o755,
)
require.NoError(t, err)
err = file.Copy(
filepath.Join("testdata", "software", "multiple-packages.yml"),
filepath.Join(basePath, "multiple-packages.yml"),
0o755,
)
require.NoError(t, err)
appConfig = fleet.EnrichedAppConfig{}
appConfig.License = &fleet.LicenseInfo{
Tier: fleet.TierPremium,
}
_, err = GitOpsFromFile(path, basePath, &appConfig, nopLogf)
assert.ErrorContains(t, err, "contains multiple packages, so cannot be used as a target for policy automation")
}
func TestGitOpsWithStrayScriptEntryWithNoPath(t *testing.T) {
t.Parallel()
config := getTeamConfig([]string{"controls"})
config += `
controls:
scripts:
-
`
_, err := gitOpsFromString(t, config)
assert.ErrorContains(t, err, `check for a stray "-"`)
}
func TestGitOpsTeamPolicyWithInvalidRunScript(t *testing.T) {
t.Parallel()
config := getTeamConfig([]string{"policies"})
config += `
policies:
- name: Some policy
query: SELECT 1;
run_script:
path: ./some_path.sh
`
_, err := gitOpsFromString(t, config)
assert.ErrorContains(t, err, "script file does not exist")
config = getTeamConfig([]string{"policies"})
config += `
policies:
- name: Some policy
query: SELECT 1;
run_script:
path:
`
_, err = gitOpsFromString(t, config)
assert.ErrorContains(t, err, "empty run_script path")
// Policy references a script not present in the team.
config = getTeamConfig([]string{"policies"})
config += `
policies:
- path: ./policies/script-policy.yml
software:
controls:
scripts:
- path: ./policies/policies2.yml
`
path, basePath := createTempFile(t, "", config)
err = file.Copy(
filepath.Join("testdata", "policies", "script-policy.yml"),
filepath.Join(basePath, "policies", "script-policy.yml"),
0o755,
)
require.NoError(t, err)
err = file.Copy(
filepath.Join("testdata", "lib", "collect-fleetd-logs.sh"),
filepath.Join(basePath, "lib", "collect-fleetd-logs.sh"),
0o755,
)
require.NoError(t, err)
appConfig := fleet.EnrichedAppConfig{}
appConfig.License = &fleet.LicenseInfo{
Tier: fleet.TierPremium,
}
_, err = GitOpsFromFile(path, basePath, &appConfig, nopLogf)
assert.ErrorContains(t, err,
"was not defined in controls for TeamName",
)
}
func getGlobalConfig(optsToExclude []string) string {
return getBaseConfig(topLevelOptions, optsToExclude)
}
func getTeamConfig(optsToExclude []string) string {
return getBaseConfig(teamLevelOptions, optsToExclude)
}
func getBaseConfig(options map[string]string, optsToExclude []string) string {
var config string
for key, value := range options {
if !slices.Contains(optsToExclude, key) {
config += value + "\n"
}
}
return config
}
func TestSoftwarePackagesUnmarshalMulti(t *testing.T) {
t.Parallel()
config := getTeamConfig([]string{"software"})
config += `
software:
packages:
- path: software/single-package.yml
- path: software/multiple-packages.yml
`
path, basePath := createTempFile(t, "", config)
for _, f := range []string{"single-package.yml", "multiple-packages.yml"} {
err := file.Copy(
filepath.Join("testdata", "software", f),
filepath.Join(basePath, "software", f),
os.FileMode(0o755),
)
require.NoError(t, err)
}
appConfig := fleet.EnrichedAppConfig{}
appConfig.License = &fleet.LicenseInfo{
Tier: fleet.TierPremium,
}
_, err := GitOpsFromFile(path, basePath, &appConfig, nopLogf)
require.NoError(t, err)
}
func TestSoftwarePackagesPathWithInline(t *testing.T) {
t.Parallel()
config := getTeamConfig([]string{"software"})
config += `
software:
packages:
- path: software/single-package.yml
icon:
path: ./foo/bar.png
`
path, basePath := createTempFile(t, "", config)
err := file.Copy(
filepath.Join("testdata", "software", "single-package.yml"),
filepath.Join(basePath, "software", "single-package.yml"),
os.FileMode(0o755),
)
require.NoError(t, err)
appConfig := fleet.EnrichedAppConfig{}
appConfig.License = &fleet.LicenseInfo{
Tier: fleet.TierPremium,
}
_, err = GitOpsFromFile(path, basePath, &appConfig, nopLogf)
assert.ErrorContains(t, err, "the software package defined in software/single-package.yml must not have icons, scripts, queries, URL, or hash specified at the team level")
}
func TestIllegalFleetSecret(t *testing.T) {
t.Parallel()
config := getGlobalConfig([]string{"policies"})
config += `
policies:
- name: $FLEET_SECRET_POLICY
platform: linux
query: SELECT 1 FROM osquery_info WHERE start_time < 0;
- name: My policy
platform: windows
query: SELECT 1;
`
_, err := gitOpsFromString(t, config)
assert.ErrorContains(t, err, "variables with \"FLEET_SECRET_\" prefix are only allowed")
}
func TestInvalidSoftwareInstallerHash(t *testing.T) {
appConfig := &fleet.EnrichedAppConfig{}
appConfig.License = &fleet.LicenseInfo{
Tier: fleet.TierPremium,
}
_, err := GitOpsFromFile("testdata/team_config_invalid_sha.yml", "./testdata", appConfig, nopLogf)
assert.ErrorContains(t, err, "must be a valid lower-case hex-encoded (64-character) SHA-256 hash value")
}
func TestSoftwareDisplayNameValidation(t *testing.T) {
t.Parallel()
appConfig := &fleet.EnrichedAppConfig{}
appConfig.License = &fleet.LicenseInfo{
Tier: fleet.TierPremium,
}
// Create a string with 256 'a' characters (exceeds 255 limit)
longDisplayName := strings.Repeat("a", 256)
t.Run("package_display_name_too_long", func(t *testing.T) {
config := getTeamConfig([]string{"name", "software"})
// Use hash instead of URL to avoid script validation before display_name validation
config += `name: Test Team
software:
packages:
- hash_sha256: "abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234"
display_name: "` + longDisplayName + `"
`
path, basePath := createTempFile(t, "", config)
_, err := GitOpsFromFile(path, basePath, appConfig, nopLogf)
assert.ErrorContains(t, err, "display_name is too long (max 255 characters)")
})
t.Run("app_store_display_name_too_long", func(t *testing.T) {
config := getTeamConfig([]string{"name", "software"})
config += `name: Test Team
software:
app_store_apps:
- app_store_id: "12345"
display_name: "` + longDisplayName + `"
`
path, basePath := createTempFile(t, "", config)
_, err := GitOpsFromFile(path, basePath, appConfig, nopLogf)
assert.ErrorContains(t, err, "display_name is too long (max 255 characters)")
})
t.Run("valid_display_name", func(t *testing.T) {
config := getTeamConfig([]string{"name", "software"})
// Use hash instead of URL to avoid network calls, and no scripts required
config += `name: Test Team
software:
packages:
- hash_sha256: "abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234abcd1234"
display_name: "Custom Package Name"
app_store_apps:
- app_store_id: "12345"
display_name: "Custom VPP App Name"
`
path, basePath := createTempFile(t, "", config)
result, err := GitOpsFromFile(path, basePath, appConfig, nopLogf)
require.NoError(t, err)
require.Len(t, result.Software.Packages, 1)
assert.Equal(t, "Custom Package Name", result.Software.Packages[0].DisplayName)
require.Len(t, result.Software.AppStoreApps, 1)
assert.Equal(t, "Custom VPP App Name", result.Software.AppStoreApps[0].DisplayName)
})
}
func TestWebhookPolicyIDsValidation(t *testing.T) {
t.Parallel()
appConfig := &fleet.EnrichedAppConfig{}
appConfig.License = &fleet.LicenseInfo{
Tier: fleet.TierPremium,
}
t.Run("no_team_invalid_policy_ids_as_number", func(t *testing.T) {
config := getTeamConfig([]string{"name", "settings"})
config += `name: No team
settings:
webhook_settings:
failing_policies_webhook:
enable_failing_policies_webhook: true
destination_url: https://webhook.site/test
policy_ids: 567
host_batch_size: 0
software:
packages: []
policies: []
`
noTeamPath, noTeamBasePath := createNamedFileOnTempDir(t, "no-team.yml", config)
_, err := GitOpsFromFile(noTeamPath, noTeamBasePath, appConfig, nopLogf)
assert.ErrorContains(t, err, "policy_ids' must be an array")
})
t.Run("no_team_invalid_policy_ids_as_string", func(t *testing.T) {
config := getTeamConfig([]string{"name", "settings"})
config += `name: No team
settings:
webhook_settings:
failing_policies_webhook:
enable_failing_policies_webhook: true
destination_url: https://webhook.site/test
policy_ids: "567"
host_batch_size: 0
software:
packages: []
policies: []
`
noTeamPath, noTeamBasePath := createNamedFileOnTempDir(t, "no-team.yml", config)
_, err := GitOpsFromFile(noTeamPath, noTeamBasePath, appConfig, nopLogf)
assert.ErrorContains(t, err, "policy_ids' must be an array")
})
t.Run("no_team_valid_policy_ids_as_array", func(t *testing.T) {
config := getTeamConfig([]string{"name", "settings"})
config += `name: No team
settings:
webhook_settings:
failing_policies_webhook:
enable_failing_policies_webhook: true
destination_url: https://webhook.site/test
policy_ids: [567, 890]
host_batch_size: 0
software:
packages: []
policies: []
`
noTeamPath, noTeamBasePath := createNamedFileOnTempDir(t, "no-team.yml", config)
gitops, err := GitOpsFromFile(noTeamPath, noTeamBasePath, appConfig, nopLogf)
assert.NoError(t, err)
assert.NotNil(t, gitops)
assert.True(t, gitops.IsNoTeam())
})
t.Run("no_team_valid_policy_ids_as_empty_array", func(t *testing.T) {
config := getTeamConfig([]string{"name", "settings"})
config += `name: No team
settings:
webhook_settings:
failing_policies_webhook:
enable_failing_policies_webhook: true
destination_url: https://webhook.site/test
policy_ids: []
host_batch_size: 0
software:
packages: []
policies: []
`
noTeamPath, noTeamBasePath := createNamedFileOnTempDir(t, "no-team.yml", config)
gitops, err := GitOpsFromFile(noTeamPath, noTeamBasePath, appConfig, nopLogf)
assert.NoError(t, err)
assert.NotNil(t, gitops)
})
t.Run("no_team_valid_policy_ids_as_yaml_list", func(t *testing.T) {
config := getTeamConfig([]string{"name", "settings"})
config += `name: No team
settings:
webhook_settings:
failing_policies_webhook:
enable_failing_policies_webhook: true
destination_url: https://webhook.site/test
policy_ids:
- 567
- 890
host_batch_size: 0
software:
packages: []
policies: []
`
noTeamPath, noTeamBasePath := createNamedFileOnTempDir(t, "no-team.yml", config)
gitops, err := GitOpsFromFile(noTeamPath, noTeamBasePath, appConfig, nopLogf)
assert.NoError(t, err)
assert.NotNil(t, gitops)
})
t.Run("regular_team_invalid_policy_ids_as_number", func(t *testing.T) {
config := getTeamConfig([]string{"settings"})
config += `settings:
secrets:
- secret: test123
webhook_settings:
failing_policies_webhook:
enable_failing_policies_webhook: true
destination_url: https://webhook.site/test
policy_ids: 567
host_batch_size: 0
`
_, err := gitOpsFromString(t, config)
assert.ErrorContains(t, err, "policy_ids' must be an array")
})
t.Run("regular_team_valid_policy_ids_as_array", func(t *testing.T) {
config := getTeamConfig([]string{"settings"})
config += `settings:
secrets:
- secret: test123
webhook_settings:
failing_policies_webhook:
enable_failing_policies_webhook: true
destination_url: https://webhook.site/test
policy_ids: [567, 890]
host_batch_size: 0
`
gitops, err := gitOpsFromString(t, config)
assert.NoError(t, err)
assert.NotNil(t, gitops)
assert.NotNil(t, gitops.TeamSettings["webhook_settings"])
})
}