mirror of
https://github.com/fleetdm/fleet
synced 2026-05-24 09:28:54 +00:00
6d708ef67e
## Summary Migrates CIS benchmark files from `ee/cis/` to `docs/solutions/cis/` in Fleet GitOps-compatible format, with configuration profiles, remediation scripts, and policy queries for macOS 13/14/15, Windows 10/11, and Windows 11 Intune. **The original AI-generated content (commits 1–4) contained critical errors that would cause silent deployment failures.** Commits 5–10 are a systematic review that validates every CSP path, Apple payload key, and script against official vendor documentation — fixing 30+ issues. --- ## What was wrong and why ### 1. Windows XML profiles used wrong SyncML format (would be rejected by Fleet) The AI generated full SyncML protocol envelopes (`<SyncML><SyncBody><CmdID>…<Final/>`), but Fleet expects flat `<Replace><Item>…</Item></Replace>` blocks — the same format used in [`docs/solutions/windows/configuration-profiles/`](https://github.com/fleetdm/fleet/tree/main/docs/solutions/windows/configuration-profiles). Every production Fleet profile (e.g., [`enforce device password and lock requirements – [Bundle].xml`](https://github.com/fleetdm/fleet/blob/main/docs/solutions/windows/configuration-profiles/enforce%20device%20password%20and%20lock%20requirements%20%E2%80%93%20%5BBundle%5D.xml)) uses this flat format. This is the same class of error [reported by @AdamBaali in #42748](https://github.com/fleetdm/fleet/issues/42748#issuecomment-4223794562) when testing the `numa` AI-generated security baseline. **Fix:** Rewrote all 13 XML files to match Fleet's expected format. ### 2. Six Audit CSP names were fabricated (`LogonLogoff_` prefix doesn't exist) The AI used `Audit/LogonLogoff_AuditLogon` etc., but the [Microsoft Audit Policy CSP documentation](https://learn.microsoft.com/en-us/windows/client-management/mdm/policy-csp-audit) specifies the prefix `AccountLogonLogoff_` — e.g., `Audit/AccountLogonLogoff_AuditLogon`. Using the wrong prefix returns HTTP 500 from the MDM server. **Fix:** Corrected all 6 across 3 platform variants (18 replacements). ### 3. Three Apple Intelligence profiles used fabricated payload keys The AI invented keys that don't exist in [Apple's device-management reference](https://github.com/apple/device-management/blob/release/mdm/profiles/com.apple.applicationaccess.yaml): | Profile | AI-generated (fabricated) | Correct per Apple reference | |---------|--------------------------|----------------------------| | Extensions | `allowIntelligenceExtensions` | `allowExternalIntelligenceIntegrations` | | Mail | `com.apple.mail` / `allowMailIntelligence` | `com.apple.applicationaccess` / `allowMailSmartReplies` + `allowMailSummary` | | Notes | `com.apple.mobilenotes` / `allowNotesIntelligence` | `com.apple.applicationaccess` / `allowNotesTranscription` + `allowNotesTranscriptionSummary` | These profiles would deploy silently but enforce nothing — the keys would be ignored by macOS. The corresponding policy SQL queries also referenced the fabricated keys, so compliance checks would never pass even with the profile applied. **Fix:** Corrected keys, domains, and the corresponding policy SQL queries so compliance checks match what the profiles actually enforce. ### 4. Firewall CSP used wrong URI prefix AI used `./Device/Vendor/MSFT/Firewall/MdmStore/…` but [Microsoft's Firewall CSP documentation](https://learn.microsoft.com/en-us/windows/client-management/mdm/firewall-csp) and Fleet's own production profiles use `./Vendor/MSFT/Firewall/MdmStore/…`. **Fix:** Updated all firewall URIs across 3 platform variants. ### 5. Wrong SIDs in user rights deny policies CIS 2.2.16 (Deny access from network) and 2.2.20 (Deny RDP logon) require denying both **Guests** (`S-1-5-32-546`) and **Local account** (`S-1-5-113`). The AI only included Guests. This is the same error [identified by @AdamBaali in #42748](https://github.com/fleetdm/fleet/issues/42748#issuecomment-4223794562) where the `numa` baseline used `S-1-2-0` (wrong SID entirely). **Fix:** Added `S-1-5-113` to both XML profiles and PS1 scripts. ### 6. macOS scripts had runtime-breaking bugs Scripts copied from `ee/cis/` originals contained issues that would cause failures when deployed via Fleet: | Issue | Scripts affected | Impact | |-------|-----------------|--------| | Hardcoded `<username>` placeholder | CIS_6.1.1, CIS_6.3.6, CIS_2.11.1, CIS_2.6.2 | Shell error — `<username>` parsed as redirect | | Missing `chmod 0440` on sudoers drop-ins | CIS_5.4, CIS_5.5 | macOS sudo silently ignores files without 0440 perms | | `sudo IFS=$'\n'` doesn't set IFS in current shell | CIS_5.1.5, CIS_5.1.6, CIS_5.1.7 | Word splitting breaks on paths with spaces | | Unsafe temp files (`./tmp.txt` in CWD) | CIS_3.2, CIS_3.3, CIS_3.4 | Race condition, fails in read-only CWD | | Missing shebang | CIS_2.6.1.2 | May use wrong shell interpreter | | `not_always_working` scripts with `<password>` | CIS_2.10.1, CIS_2.10.2 | Could never work in automated deployment | **Fix:** Rewrote scripts with dynamic user enumeration, `mktemp`, proper IFS, `visudo` validation. Removed `not_always_working` scripts. ### 7. Policy YAML formatting bugs and spelling - **Computer Browser policy** (win-10, win-11): `resolution: |` was on the same line as description text, causing YAML to swallow the entire resolution into the description — no remediation steps shown. - **Set Time and Date policy** (macOS 13/14/15): Empty `description:` parsed as `null`. - 14 spelling errors across macOS and Windows YAML (`existance`, `Extention` ×8, `recomendation`, `bellow`, `enableds`, `addess` ×2, etc.) - Filename inconsistencies: `on-device-dictiation-enabled` (typo), `2.8.1.disable` (dots vs dashes) --- ## Verification methodology Every CSP OMA-URI and Apple payload key was validated against official vendor documentation: | Category | Count | Source | Result | |----------|-------|--------|--------| | [Audit CSP](https://learn.microsoft.com/en-us/windows/client-management/mdm/policy-csp-audit) | 27 | Microsoft docs | 6 corrected | | [UserRights CSP](https://learn.microsoft.com/en-us/windows/client-management/mdm/policy-csp-userrights) | 27 | Microsoft docs | All verified correct | | [LocalPoliciesSecurityOptions CSP](https://learn.microsoft.com/en-us/windows/client-management/mdm/policy-csp-localpoliciessecurityoptions) | 27 | Microsoft docs | 1 format corrected | | [Firewall CSP](https://learn.microsoft.com/en-us/windows/client-management/mdm/firewall-csp) | 14 | Microsoft docs | All correct (after path fix) | | [Defender CSP](https://learn.microsoft.com/en-us/windows/client-management/mdm/policy-csp-defender) | 9 | Microsoft docs | All verified correct | | [Apple applicationaccess](https://github.com/apple/device-management/blob/release/mdm/profiles/com.apple.applicationaccess.yaml) | 13 profiles | Apple device-management repo | 3 corrected | | Fleet GitOps YAML | 1,857 policies | [Fleet GitOps docs](https://fleetdm.com/docs/configuration/yaml-files) | 2 format bugs fixed | | Fleet XML format | 13 files | [`docs/solutions/windows/`](https://github.com/fleetdm/fleet/tree/main/docs/solutions/windows/configuration-profiles) | Reformatted to match | | Fleet blocked CSPs | — | `server/fleet/windows_mdm.go` | No conflicts | | [#43598](https://github.com/fleetdm/fleet/issues/43598) glob bug | 305 files | Filename scan | No `*`, `?`, `[`, `{` in any filename | ## Commits 1. `89d9418` — Copy CIS benchmarks to docs/solutions/cis/ in GitOps-compatible format 2. `bbabb13` — Restructure with policies/, configuration-profiles/, scripts/ subdirs 3. `00004f4` — Promote test artifacts, add Contents to READMEs 4. `786591f` — Add configuration profiles and scripts to all CIS benchmark platforms 5. `0a47a53` — Fix macOS scripts (username placeholders, sudoers perms, IFS, temp files, shebang, dictation typo, remove broken scripts) 6. `4efbd40` — Reformat all 13 Windows XML profiles to Fleet's flat Replace/Item format, fix Firewall CSP paths, fix SIDs 7. `f0abb66` — Fix 6 Audit CSP names: `LogonLogoff_` → `AccountLogonLogoff_` 8. `bfb0f95` — Fix `SmartCardRemovalBehavior` format type (`int` → `chr`) 9. `4cbb33e` — Fix 3 Apple Intelligence profiles (fabricated keys → real Apple keys), fix YAML formatting bugs 10. `d3a0031` — Fix spelling (14 corrections), naming conventions, policy-profile key alignment ## Current state | Platform | Policies | Config Profiles | Scripts | |----------|----------|-----------------|---------| | macOS 13 | 1 YAML (111 policies) | 46 .mobileconfig | 43 .sh | | macOS 14 | 1 YAML (111 policies) | 47 .mobileconfig | 43 .sh | | macOS 15 | 1 YAML (113 policies) | 43 .mobileconfig | 44 .sh | | Windows 10 | 1 YAML (510 policies) | 4 SyncML XML | 5 .ps1 | | Windows 11 | 1 YAML (555 policies) | 4 SyncML XML | 5 .ps1 | | Windows 11 Intune | 3 YAML (457 policies) | 5 SyncML XML | 2 .ps1 | The source files in `ee/cis/` are intentionally left untouched. https://claude.ai/code/session_01DUqJK6iJ8MWMdz2d25ZTNW --------- Co-authored-by: Claude <noreply@anthropic.com>
2.5 KiB
2.5 KiB
macOS 13.0 Ventura benchmark
Fleet's policies have been written against v3.1.0 of the benchmark. You can refer to the CIS website for full details about this version.
For requirements and usage details, see the CIS Benchmarks documentation.
Contents
| Folder | Description |
|---|---|
policies/ |
GitOps-compatible policy YAML — import via fleetctl apply or reference with - path: in fleet.yml |
configuration-profiles/ |
Apple .mobileconfig profiles — upload via Fleet UI or fleetctl apply to enforce the settings checked by the policies |
scripts/ |
Shell scripts — upload via Fleet UI or fleetctl apply and link as run_script remediation in the corresponding policy |
Limitations
The following CIS benchmarks cannot be checked with a policy in Fleet:
- 2.1.2 Audit App Store Password Settings
- 2.3.3.12 Ensure Computer Name Does Not Contain PII or Protected Organizational Information
- 2.6.6 Audit Lockdown Mode
- 2.11.2 Audit Touch ID and Wallet & Apple Pay Settings
- 2.13.1 Audit Passwords System Preference Setting
- 2.14.1 Audit Notification & Focus Settings
- 3.7 Audit Software Inventory
- 6.2.1 Ensure Protect Mail Activity in Mail Is Enabled
Checks that require decision
CIS has left the parameters of the following checks up to the benchmark implementer. CIS recommends that an organization make a conscious decision for these benchmarks, but does not make a specific recommendation.
Fleet has provided both an "enabled" and "disabled" version of these benchmarks. When both policies are added, at least one will fail. Once your organization has made a decision, you can delete one or the other policy.
The policy will be appended with a -enabled or -disabled label, such as 2.1.1.1-enabled.
- 2.1.1.1 Audit iCloud Keychain
- 2.1.1.2 Audit iCloud Drive
- 2.5.1 Audit Siri
- 2.8.1 Audit Universal Control
Furthermore, CIS has decided to not require the following password complexity settings:
- 5.2.3 Ensure Complex Password Must Contain Alphabetic Characters Is Configured
- 5.2.4 Ensure Complex Password Must Contain Numeric Character Is Configured
- 5.2.5 Ensure Complex Password Must Contain Special Character Is Configured
- 5.2.6 Ensure Complex Password Must Contain Uppercase and Lowercase Characters Is Configured
However, Fleet has provided these as policies. If your organization declines to implement these, simply delete the corresponding policies.