fleet/cmd/gitops-migrate/README.md
Anthony Maxwell 288ea58bce
Feat: GitOps YAML Migration Tool (#32237)
# Overview

This pull request resolves #31165, implementing command-line tooling to
migrate GitOps YAML files following the [changes introduced in the
upcoming 4.74
release](https://github.com/fleetdm/fleet/pull/32237/files#diff-8769f6e90e8bdf15faad8f390fdf3ffb6fd2238b7d6087d83518c21464109119R7).

Aligning with the recommended steps in the `README`; [this is an example
of the first step](https://github.com/Illbjorn/fleet/pull/3/files)
(`gitops-migrate format`) and [this is an example of the second
step](https://github.com/Illbjorn/fleet/pull/4/files) (`gitops-migrate
migrate`).

---------

Signed-off-by: Illbjorn <am@hades.so>
Co-authored-by: Ian Littman <iansltx@gmail.com>
2025-09-08 12:42:25 -04:00

8.3 KiB

Overview

This directory contains the gitops-migrate tool, designed leading up to the 4.74 Fleet release to automate necessary GitOps YAML transformations.

4.74 YAML Changes

The 4.74 release moves GitOps YAML keys: self_service, categories, labels_exclude_any, labels_include_any and setup_experience from the software files (example) to the team files (example).

Installation

Download the appropriate binary for your operating system and architecture:

Operating System Architecture Download Link
macos arm64 Download(Hash)
macos amd64 Download(Hash)
windows amd64 Download(Hash)
windows arm64 Download(Hash)
linux amd64 Download(Hash)
linux arm64 Download(Hash)

Method 2: Go Install

Install Go and install gitops-migrate by running:

$ go install github.com/fleetdm/fleet/v4/cmd/gitops-migrate@latest

You can verify the installation was successful by running gitops-migrate usage which should display the help text.

Note

If the go install is successful but you're not able to run gitops-migrate, you may need to add your GOBIN directory to PATH in the way appropriate for your operating system (Windows, Mac, Linux). The path to add can be found by running: go env GOBIN.

Running the Migration

The migration will unfold in two primary steps: format and migrate.

Important

If your GitOps files are version-controlled (stored in GitHub or similar) it is recommended to perform these steps in order, opening a pull request, moving through your standard review process and merging that pull request before moving to the next step.

Step 1: Format

Overview

When manipulating YAML files with this tool, the output will always alphabetize the keys. This means the following YAML file:

a: []
c: []
b: []

Will become:

a: []
b: []
c: []

This can mean, if your GitOps files are version-controlled (stored in GitHub or similar), you could see a very large number of changed lines which might make it more difficult to spot the actual transformations.

Considering the above, we recommend running the gitops-migrate format command before performing the migration.

Steps

Run the gitops-migrate tool, specifying the format command followed by the path to your GitOps YAML files.

Linux/Mac:

# If 'gitops-migrate' is in the current working directory.
$ ./gitops-migrate format ./fleet_gitops
# If 'gitops-migrate' is in PATH.
$ gitops-migrate format ./fleet_gitops

Windows:

# If 'gitops-migrate' is in the current working directory.
PS> .\gitops-migrate format .\fleet_gitops
# If 'gitops-migrate' is in PATH.
PS> gitops-migrate format .\fleet_gitops

Your YAML files should now all be alphabetized!

Tip

It's recommended to pause here, commit changes, then open a pull request for the formatting changes only. Then move onto the next section once that pull request has been reviewed and merged.

Step 2: Migrate

Overview

Now we'll run the gitops-migrate migrate command which will:

  • Perform a backup of your GitOps files, outputting an archive to your operating system's TEMP directory (the path to this backup will be shown at the start of the output of the command, be sure to take note of it).
  • Migrate all YAML files in the provided directory (changes outlined above).

Steps

Run the gitops-migrate tool, specifying the migrate command and the path to your GitOps YAML files.

Linux/Mac:

# If 'gitops-migrate' is in the current working directory.
$ ./gitops-migrate migrate ./gitops_files
# If 'gitops-migrate' is in PATH.
$ gitops-migrate migrate ./gitops_files

Windows:

# If 'gitops-migrate' is in the current working directory.
PS> .\gitops-migrate migrate .\gitops_files
# If 'gitops-migrate' is in PATH.
PS> gitops-migrate migrate .\gitops_files

Did it work?

In the command output, you should see messages like the following:

> Successfully applied transforms to team file.
┣━ [Team File]=>[it-and-security/teams/workstations.yml]
┗━ [Count]=>[39]

In cases where the team file previously contained software packages which referenced software files containing the fields described above, you can spot-check the results by confirming these fields are now present in the software packages array items, right alongside the path key(s).

When looking at a git diff you should see changes similar to the following:

Software file (./slack.yml):

url: https://downloads.slack-edge.com/desktop-releases/linux/x64/4.41.105/slack-desktop-4.41.105-amd64.deb
- self_service: true
- categories:
-  - Productivity
-  - Communication
- labels_include_any:
-  - "Debian-based Linux hosts"

Team File (./my_team.yml):

software:
  packages:
    - path: slack.yml
+     self_service: true
+     categories:
+       - Productivity
+       - Communication
+     labels_include_any:
+       - "Debian-based Linux hosts"

Help, something has gone wrong!

In the event you've attempted the migration and encounter any issues, you can quickly revert your GitOps file states by simply restoring the backup taken automatically during the migrate process.

To do this, locate the backup archive path in the log output:

> Performing Fleet GitOps file backup.
┣━ [Source]=>[fleet_gitops]
┗━ [Destination]=>[/tmp/fleet-gitops-1916163188/fleet-gitops-backup-8-31-2025_4-47-29.tar.gz] # <-- Here

Then simply run the gitops-migrate restore command to restore this backup, specifying the archive path as the first arg and the path to restore the archive to as the second arg:

Linux/Mac:

# If 'gitops-migrate' is in the current working directory.
$ ./gitops-migrate restore /tmp/fleet-gitops-1916163188/fleet-gitops-backup-8-31-2025_4-47-29.tar.gz ./fleet_gitops
# If 'gitops-migrate' is in PATH.
$ gitops-migrate restore /tmp/fleet-gitops-1916163188/fleet-gitops-backup-8-31-2025_4-47-29.tar.gz ./fleet_gitops

Windows:

# If 'gitops-migrate' is in the current working directory.
PS> .\gitops-migrate restore 'C:\Users\am\AppData\Local\Temp\fleet-gitops-1916163188/fleet-gitops-backup-8-31-2025_4-47-29.tar.gz' .\fleet_gitops
# If 'gitops-migrate' is in PATH.
PS> gitops-migrate restore 'C:\Users\am\AppData\Local\Temp\fleet-gitops-1916163188/fleet-gitops-backup-8-31-2025_4-47-29.tar.gz' .\fleet_gitops