Elgato_dark/argo-cd
1
0
Fork 0
mirror of https://github.com/argoproj/argo-cd synced 2026-04-21 17:07:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 22

feat(helm): support wildcard glob patterns for valueFiles (cherry-pick #26768 for 3.4) (#26919)

Browse source 
Signed-off-by: nitishfy <justnitish06@gmail.com>
Co-authored-by: Nitish Kumar <justnitish06@gmail.com>
This commit is contained in:
argo-cd-cherry-pick-bot[bot] 2026-03-19 15:48:06 -04:00 committed by GitHub
parent e70034a44b
commit 3157fb15a4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
11 changed files with 1021 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

235
docs/user-guide/helm.md
View file

@ -90,6 +90,241 @@ source:
ignoreMissingValueFiles: true
```
## Glob Patterns in Value Files
Glob patterns can be used in `valueFiles` entries to match multiple files at once. This is useful
when the set of environment-specific override files is not known in advance, or when you want to
pick up new files automatically without updating the Application spec.
```bash
# Single quotes prevent the shell from expanding the glob before Argo CD receives it
argocd app set helm-guestbook --values 'envs/*.yaml'
```
In the declarative syntax:
```yaml
source:
helm:
valueFiles:
- envs/*.yaml
```
### Supported pattern syntax
Glob expansion uses the [doublestar](https://github.com/bmatcuk/doublestar) library.
| Pattern | Description |
|---------|-------------|
| `*` | Matches any sequence of non-separator characters within a single directory level |
| `?` | Matches any single non-separator character |
| `[abc]` | Matches one of the characters listed inside the brackets |
| `[a-z]` | Matches any character in the given range |
| `**` | Matches any sequence of characters including `/` (recursive across directory levels) |
### How files are passed to Helm
Each matched file is passed to `helm template` as a separate `--values <path>` flag, in the same
order they appear after expansion. This is identical to listing each file individually in
`valueFiles`. Argo CD does the expansion before invoking Helm.
Matched files are expanded **in-place** within the `valueFiles` list and sorted in **lexical
(alphabetical) order**. Because Helm gives higher precedence to later `--values` flags, lexical
order determines which file wins when the same key appears in multiple files.
```
envs/
a.yaml # sets foo: a-value
b.yaml # sets foo: b-value
```
```yaml
# envs/*.yaml expands to: envs/a.yaml, envs/b.yaml (lexical order)
# b.yaml is last → foo = "b-value"
source:
helm:
valueFiles:
- envs/*.yaml
```
When you have multiple entries in `valueFiles`, the relative order between entries is preserved.
Glob expansion only reorders files within a single pattern:
```yaml
valueFiles:
- base.yaml # passed first
- overrides/*.yaml # expanded in lexical order, passed after base.yaml
- final.yaml # passed last, highest precedence
```
### Recursive matching with `**`
Use `**` to match files at any depth below a directory:
```yaml
# envs/**/*.yaml processes each directory's own files before descending into subdirectories,
# with directories and files sorted alphabetically at each level.
#
# envs/a.yaml ← 'a' (flat file in envs/)
# envs/z.yaml ← 'z' (flat file in envs/, processed before descending)
# envs/nested/c.yaml ← inside envs/nested/, processed after envs/ flat files
#
# nested/c.yaml is last → foo = "nested-value"
source:
helm:
valueFiles:
- envs/**/*.yaml
```
> [!NOTE]
> `**` matches zero or more path segments, so `envs/**/*.yaml` also matches files directly
> inside `envs/` (not just subdirectories). doublestar traverses directories in lexical order
> and processes each directory's own files (alphabetically) before descending into its
> subdirectories. This means `envs/z.yaml` always comes before `envs/nested/c.yaml`, even
> though `'n' < 'z'` alphabetically. To make ordering fully explicit and predictable,
> use numeric prefixes (see [Naming conventions](#naming-conventions)).
### Using environment variables in glob patterns
[Build environment variables](./build-environment.md) are substituted **before** the glob is
evaluated, so you can construct patterns dynamically:
```yaml
source:
helm:
valueFiles:
- envs/$ARGOCD_APP_NAME/*.yaml
```
This lets a single Application template expand to the right set of files per app name.
### Glob patterns with multiple sources
Glob patterns work with [value files from an external repository](./multiple_sources.md#helm-value-files-from-external-git-repository).
The `$ref` variable is resolved first to the external repo's root, and the rest of the pattern is
evaluated within that repo's directory tree:
```yaml
sources:
- repoURL: https://git.example.com/my-configs.git
ref: configs
- repoURL: https://git.example.com/my-chart.git
path: chart
helm:
valueFiles:
- $configs/envs/*.yaml # matches files in the 'my-configs' repo under envs/
```
### Naming conventions
Because files are sorted lexically, the sort order controls merge precedence. A common pattern is
to use a numeric prefix to make the intended order explicit:
```
values/
00-defaults.yaml
10-region.yaml
20-env.yaml
30-override.yaml
```
```yaml
valueFiles:
- values/*.yaml
# expands to: 00-defaults.yaml, 10-region.yaml, 20-env.yaml, 30-override.yaml
# 30-override.yaml has the highest precedence
```
Without a prefix, pure alphabetical ordering applies. Be careful with names that sort
unexpectedly, for example `values-10.yaml` sorts before `values-9.yaml` because `"1"` < `"9"`
lexically.
### Constraints and limitations
**Path boundary**: Glob patterns cannot match files outside the repository root, even with
patterns like `../../secrets/*.yaml`. Argo CD resolves the pattern's base path against the
repository root before expanding it, and any match that would escape the root is rejected.
**Symlinks**: Argo CD follows symlinks when checking the path boundary. A symlink that lives
inside the repository but points to a target outside the repository root is rejected, even though
the symlink's own path is within the repo. This check applies to every file produced by glob
expansion, including multi-hop symlink chains. Symlinks that resolve to a target still inside the
repository are allowed.
**Absolute paths**: A path starting with `/` is treated as relative to the **repository root**,
not the filesystem root. The pattern `/configs/*.yaml` matches files in the `configs/` directory
at the top of the repository.
**Remote URLs are not glob-expanded**: Entries that are remote URLs (e.g.
`https://raw.githubusercontent.com/.../values.yaml`) are passed to Helm as-is. Glob characters
in a URL have no special meaning and will cause the URL to fail if the literal characters are not
part of the URL.
**Shell quoting on the CLI**: Shells expand glob patterns before passing arguments to programs.
Always quote patterns to prevent unintended shell expansion:
```bash
# Correct: single quotes pass the literal pattern to Argo CD
argocd app set myapp --values 'envs/*.yaml'
# Incorrect: the shell expands *.yaml against the current directory first
argocd app set myapp --values envs/*.yaml
```
### Deduplication
Each file is included only once, but **explicit entries take priority over glob matches** when
determining position. If a file appears both in a glob pattern and as an explicit entry, the glob
skips it and the explicit entry places it at its declared position.
```yaml
valueFiles:
- envs/*.yaml # expands to base.yaml, prod.yaml — but prod.yaml is listed explicitly below,
# so the glob skips it: only base.yaml is added here
- envs/prod.yaml # placed here at the end, giving it highest Helm precedence
```
This means you can use a glob to pick up all files in a directory and then pin a specific file to
the end (highest precedence) by listing it explicitly after the glob.
If the same file (same absolute path) is matched by two glob patterns, it is included at the
position of the first match. Subsequent glob matches for that exact path are silently dropped.
Files with the same name but at different paths are treated as distinct files and are always included.
```yaml
valueFiles:
- envs/*.yaml # matches envs/base.yaml, envs/prod.yaml
- envs/**/*.yaml # envs/prod.yaml already matched above and is skipped;
# envs/nested/prod.yaml is a different path and is still included
```
### No-match behavior
If a glob pattern matches no files, Argo CD saves the Application spec (the spec is not invalid and
the files may be added to the repository later) and surfaces a `ComparisonError` condition on the
Application:
```
values file glob "nonexistent/*.yaml" matched no files
```
The app will remain in a degraded state until the pattern matches at least one file or the pattern
is removed. No spec update is required once the files are added to the repository.
To silently skip a pattern that matches no files instead of raising an error, combine the glob with
`ignoreMissingValueFiles`:
```yaml
source:
helm:
valueFiles:
- envs/*.yaml
ignoreMissingValueFiles: true
```
This is useful for implementing a default/override pattern where override files may not exist in
every environment.
## Values
Argo CD supports the equivalent of a values file directly in the Application manifest using the `source.helm.valuesObject` key.

134
reposerver/repository/repository.go
View file

@ -18,6 +18,7 @@ import (
"time"
"github.com/TomOnTime/utfutil"
"github.com/bmatcuk/doublestar/v4"
imagev1 "github.com/opencontainers/image-spec/specs-go/v1"
"sigs.k8s.io/yaml"
@ -656,6 +657,13 @@ func (s *Service) GenerateManifest(ctx context.Context, q *apiclient.ManifestReq
return nil, err
}
}
// Convert typed errors to gRPC status codes so callers can use status.Code()
// rather than string matching.
var globNoMatch *GlobNoMatchError
if errors.As(err, &globNoMatch) {
return nil, status.Error(codes.NotFound, err.Error())
}
return res, err
}
@ -1376,19 +1384,55 @@ func getResolvedValueFiles(
gitRepoPaths utilio.TempPaths,
ignoreMissingValueFiles bool,
) ([]pathutil.ResolvedFilePath, error) {
// Pre-collect resolved paths for all explicit (non-glob) entries. This allows glob
// expansion to skip files that also appear explicitly, so the explicit entry controls
// the final position. For example, with ["*.yaml", "c.yaml"], c.yaml is excluded from
// the glob expansion and placed at the end where it was explicitly listed.
explicitPaths := make(map[pathutil.ResolvedFilePath]struct{})
for _, rawValueFile := range rawValueFiles {
referencedSource := getReferencedSource(rawValueFile, refSources)
var resolved pathutil.ResolvedFilePath
var err error
if referencedSource != nil {
resolved, err = getResolvedRefValueFile(rawValueFile, env, allowedValueFilesSchemas, referencedSource.Repo.Repo, gitRepoPaths)
} else {
resolved, _, err = pathutil.ResolveValueFilePathOrUrl(appPath, repoRoot, env.Envsubst(rawValueFile), allowedValueFilesSchemas)
}
if err != nil {
continue // resolution errors will be surfaced in the main loop below
}
if !isGlobPath(string(resolved)) {
explicitPaths[resolved] = struct{}{}
}
}
var resolvedValueFiles []pathutil.ResolvedFilePath
seen := make(map[pathutil.ResolvedFilePath]struct{})
appendUnique := func(p pathutil.ResolvedFilePath) {
if _, ok := seen[p]; !ok {
seen[p] = struct{}{}
resolvedValueFiles = append(resolvedValueFiles, p)
}
}
for _, rawValueFile := range rawValueFiles {
isRemote := false
var resolvedPath pathutil.ResolvedFilePath
var err error
referencedSource := getReferencedSource(rawValueFile, refSources)
// effectiveRoot is the repository root used for the symlink boundary check
// on glob matches. For ref-source paths this is the external repo's checkout
// directory; for local paths it is the main repo root.
effectiveRoot := repoRoot
if referencedSource != nil {
// If the $-prefixed path appears to reference another source, do env substitution _after_ resolving that source.
resolvedPath, err = getResolvedRefValueFile(rawValueFile, env, allowedValueFilesSchemas, referencedSource.Repo.Repo, gitRepoPaths)
if err != nil {
return nil, fmt.Errorf("error resolving value file path: %w", err)
}
if refRepoPath := gitRepoPaths.GetPathIfExists(git.NormalizeGitURL(referencedSource.Repo.Repo)); refRepoPath != "" {
effectiveRoot = refRepoPath
}
} else {
// This will resolve val to an absolute path (or a URL)
resolvedPath, isRemote, err = pathutil.ResolveValueFilePathOrUrl(appPath, repoRoot, env.Envsubst(rawValueFile), allowedValueFilesSchemas)
@ -1397,6 +1441,38 @@ func getResolvedValueFiles(
}
}
// If the resolved path contains a glob pattern, expand it to all matching files.
// doublestar.FilepathGlob is used (consistent with AppSet generators) because it supports
// ** for recursive matching in addition to all standard glob patterns (*,?,[).
// Matches are returned in lexical order, which determines helm's merge precedence
// (later files override earlier ones). Glob patterns are only expanded for local files;
// remote value file URLs (e.g. https://...) are passed through as-is.
// If the glob matches no files and ignoreMissingValueFiles is true, skip it silently.
// Otherwise, return an error — consistent with how missing non-glob value files are handled.
if !isRemote && isGlobPath(string(resolvedPath)) {
matches, err := doublestar.FilepathGlob(string(resolvedPath))
if err != nil {
return nil, fmt.Errorf("error expanding glob pattern %q: %w", rawValueFile, err)
}
if len(matches) == 0 {
if ignoreMissingValueFiles {
log.Debugf(" %s values file glob matched no files", rawValueFile)
continue
}
return nil, &GlobNoMatchError{Pattern: rawValueFile}
}
if err := verifyGlobMatchesWithinRoot(matches, effectiveRoot); err != nil {
return nil, fmt.Errorf("glob pattern %q: %w", rawValueFile, err)
}
for _, match := range matches {
// Skip files that are also listed explicitly - they will be placed
// at their explicit position rather than the glob's position.
if _, isExplicit := explicitPaths[pathutil.ResolvedFilePath(match)]; !isExplicit {
appendUnique(pathutil.ResolvedFilePath(match))
}
}
continue
}
if !isRemote {
_, err = os.Stat(string(resolvedPath))
if os.IsNotExist(err) {
@ -1407,8 +1483,9 @@ func getResolvedValueFiles(
}
}
resolvedValueFiles = append(resolvedValueFiles, resolvedPath)
appendUnique(resolvedPath)
}
log.Infof("resolved value files: %v", resolvedValueFiles)
return resolvedValueFiles, nil
}
@ -1478,6 +1555,61 @@ func getRepoCredential(repoCredentials []*v1alpha1.RepoCreds, repoURL string) *v
return nil
}
// GlobNoMatchError is returned when a glob pattern in valueFiles matches no files.
// It is a runtime condition (the files may be added later), not a spec error.
type GlobNoMatchError struct {
Pattern string
}
func (e *GlobNoMatchError) Error() string {
return fmt.Sprintf("values file glob %q matched no files", e.Pattern)
}
// isGlobPath reports whether path contains any glob metacharacters
// supported by doublestar: *, ?, or [. The ** pattern is covered by *.
func isGlobPath(path string) bool {
return strings.ContainsAny(path, "*?[")
}
// verifyGlobMatchesWithinRoot resolves symlinks for each glob match and verifies
// that the resolved target is within effectiveRoot. It protects against symlinks
// inside the repository that point to targets outside it.
//
// doublestar.FilepathGlob uses os.Lstat, so it returns the path of the symlink
// itself (which lives inside the repo) rather than the symlink target. If the
// target is outside the repo, Helm would still follow the link and read the
// external file. This function catches that case before the paths reach Helm.
//
// Both effectiveRoot and each match are canonicalized via filepath.EvalSymlinks
// so the prefix comparison is correct on systems where the working directory is
// itself under a symlink chain (e.g. /var -> /private/var on macOS).
func verifyGlobMatchesWithinRoot(matches []string, effectiveRoot string) error {
absRoot, err := filepath.Abs(effectiveRoot)
if err != nil {
return fmt.Errorf("error resolving repo root: %w", err)
}
canonicalRoot, err := filepath.EvalSymlinks(absRoot)
if err != nil {
return fmt.Errorf("error resolving symlinks in repo root: %w", err)
}
requiredRootPath := canonicalRoot
if !strings.HasSuffix(requiredRootPath, string(os.PathSeparator)) {
requiredRootPath += string(os.PathSeparator)
}
for _, match := range matches {
realMatch, err := filepath.EvalSymlinks(match)
if err != nil {
return fmt.Errorf("error resolving symlink for glob match %q: %w", match, err)
}
// Allow the match to resolve exactly to the root (realMatch+sep == requiredRootPath)
// or to any path beneath it (HasPrefix).
if realMatch+string(os.PathSeparator) != requiredRootPath && !strings.HasPrefix(realMatch, requiredRootPath) {
return fmt.Errorf("glob match %q resolved to outside repository root", match)
}
}
return nil
}
type (
GenerateManifestOpt func(*generateManifestOpt)
generateManifestOpt struct {

561
reposerver/repository/repository_test.go
View file

@ -3895,6 +3895,567 @@ func Test_getResolvedValueFiles(t *testing.T) {
}
}
func Test_getResolvedValueFiles_glob(t *testing.T) {
t.Parallel()
tempDir := t.TempDir()
paths := utilio.NewRandomizedTempPaths(tempDir)
paths.Add(git.NormalizeGitURL("https://github.com/org/repo1"), path.Join(tempDir, "repo1"))
// main-repo files
require.NoError(t, os.MkdirAll(path.Join(tempDir, "main-repo", "prod", "nested"), 0o755))
require.NoError(t, os.MkdirAll(path.Join(tempDir, "main-repo", "staging"), 0o755))
require.NoError(t, os.WriteFile(path.Join(tempDir, "main-repo", "prod", "a.yaml"), []byte{}, 0o644))
require.NoError(t, os.WriteFile(path.Join(tempDir, "main-repo", "prod", "b.yaml"), []byte{}, 0o644))
require.NoError(t, os.WriteFile(path.Join(tempDir, "main-repo", "prod", "nested", "c.yaml"), []byte{}, 0o644))
require.NoError(t, os.WriteFile(path.Join(tempDir, "main-repo", "prod", "nested", "d.yaml"), []byte{}, 0o644))
require.NoError(t, os.WriteFile(path.Join(tempDir, "main-repo", "staging", "e.yaml"), []byte{}, 0o644))
// main-repo envs: used to verify depth-order with ** (z.yaml sorts after nested/ alphabetically
// but is still returned before nested/c.yaml because doublestar matches depth-0 files first).
require.NoError(t, os.MkdirAll(path.Join(tempDir, "main-repo", "envs", "nested"), 0o755))
require.NoError(t, os.WriteFile(path.Join(tempDir, "main-repo", "envs", "a.yaml"), []byte{}, 0o644))
require.NoError(t, os.WriteFile(path.Join(tempDir, "main-repo", "envs", "z.yaml"), []byte{}, 0o644))
require.NoError(t, os.WriteFile(path.Join(tempDir, "main-repo", "envs", "nested", "c.yaml"), []byte{}, 0o644))
// repo1 files
require.NoError(t, os.MkdirAll(path.Join(tempDir, "repo1", "prod", "nested"), 0o755))
require.NoError(t, os.WriteFile(path.Join(tempDir, "repo1", "prod", "x.yaml"), []byte{}, 0o644))
require.NoError(t, os.WriteFile(path.Join(tempDir, "repo1", "prod", "y.yaml"), []byte{}, 0o644))
require.NoError(t, os.WriteFile(path.Join(tempDir, "repo1", "prod", "nested", "z.yaml"), []byte{}, 0o644))
tests := []struct {
name string
rawPath string
env *v1alpha1.Env
refSources map[string]*v1alpha1.RefTarget
expectedPaths []string
ignoreMissingValueFiles bool
expectedErr bool
}{
{
name: "local glob matches multiple files",
rawPath: "prod/*.yaml",
env: &v1alpha1.Env{},
refSources: map[string]*v1alpha1.RefTarget{},
expectedPaths: []string{
// the order is a.yaml before b.yaml
// since doublestar.FilepathGlob returns lexical order
path.Join(tempDir, "main-repo", "prod", "a.yaml"),
path.Join(tempDir, "main-repo", "prod", "b.yaml"),
},
},
{
name: "local glob matches no files returns error",
rawPath: "dev/*.yaml",
env: &v1alpha1.Env{},
refSources: map[string]*v1alpha1.RefTarget{},
expectedPaths: nil,
expectedErr: true,
},
{
name: "local glob matches no files with ignoreMissingValueFiles set to true",
rawPath: "dev/*.yaml",
env: &v1alpha1.Env{},
refSources: map[string]*v1alpha1.RefTarget{},
ignoreMissingValueFiles: true,
expectedPaths: nil,
},
{
name: "referenced glob matches multiple files in external repo",
rawPath: "$ref/prod/*.yaml",
env: &v1alpha1.Env{},
refSources: map[string]*v1alpha1.RefTarget{
"$ref": {
Repo: v1alpha1.Repository{
Repo: "https://github.com/org/repo1",
},
},
},
expectedPaths: []string{
path.Join(tempDir, "repo1", "prod", "x.yaml"),
path.Join(tempDir, "repo1", "prod", "y.yaml"),
},
},
{
name: "ref glob with env var in path",
rawPath: "$ref/$ENV/*.yaml",
env: &v1alpha1.Env{
&v1alpha1.EnvEntry{
Name: "ENV",
Value: "prod",
},
},
refSources: map[string]*v1alpha1.RefTarget{
"$ref": {
Repo: v1alpha1.Repository{
Repo: "https://github.com/org/repo1",
},
},
},
expectedPaths: []string{
path.Join(tempDir, "repo1", "prod", "x.yaml"),
path.Join(tempDir, "repo1", "prod", "y.yaml"),
},
},
{
name: "local glob single match",
rawPath: "prod/a*.yaml",
env: &v1alpha1.Env{},
refSources: map[string]*v1alpha1.RefTarget{},
expectedPaths: []string{path.Join(tempDir, "main-repo", "prod", "a.yaml")},
},
{
name: "recursive glob matches files at all depths under a subdirectory",
// ** matches zero or more path segments, so prod/**/*.yaml covers both
// prod/*.yaml (zero intermediate segments) and prod/nested/*.yaml (one segment), etc.
rawPath: "prod/**/*.yaml",
env: &v1alpha1.Env{},
refSources: map[string]*v1alpha1.RefTarget{},
// lexical order: prod/a.yaml, prod/b.yaml, prod/nested/c.yaml, prod/nested/d.yaml
expectedPaths: []string{
path.Join(tempDir, "main-repo", "prod", "a.yaml"),
path.Join(tempDir, "main-repo", "prod", "b.yaml"),
path.Join(tempDir, "main-repo", "prod", "nested", "c.yaml"),
path.Join(tempDir, "main-repo", "prod", "nested", "d.yaml"),
},
},
{
name: "recursive glob from repo root matches yaml files across all directories",
rawPath: "**/*.yaml",
env: &v1alpha1.Env{},
refSources: map[string]*v1alpha1.RefTarget{},
// doublestar traverses directories in lexical order, processing each directory's
// own files before its subdirectories. So the order is:
// envs/ flat files → envs/nested/ files → prod/ flat files → prod/nested/ files → staging/ files
expectedPaths: []string{
path.Join(tempDir, "main-repo", "envs", "a.yaml"),
path.Join(tempDir, "main-repo", "envs", "z.yaml"),
path.Join(tempDir, "main-repo", "envs", "nested", "c.yaml"),
path.Join(tempDir, "main-repo", "prod", "a.yaml"),
path.Join(tempDir, "main-repo", "prod", "b.yaml"),
path.Join(tempDir, "main-repo", "prod", "nested", "c.yaml"),
path.Join(tempDir, "main-repo", "prod", "nested", "d.yaml"),
path.Join(tempDir, "main-repo", "staging", "e.yaml"),
},
},
{
name: "recursive glob anchored to a named subdirectory matches at any depth",
rawPath: "**/nested/*.yaml",
env: &v1alpha1.Env{},
refSources: map[string]*v1alpha1.RefTarget{},
expectedPaths: []string{
path.Join(tempDir, "main-repo", "envs", "nested", "c.yaml"),
path.Join(tempDir, "main-repo", "prod", "nested", "c.yaml"),
path.Join(tempDir, "main-repo", "prod", "nested", "d.yaml"),
},
},
{
name: "recursive glob with no matches and ignoreMissingValueFiles skips silently",
rawPath: "**/nonexistent/*.yaml",
env: &v1alpha1.Env{},
refSources: map[string]*v1alpha1.RefTarget{},
ignoreMissingValueFiles: true,
expectedPaths: nil,
},
{
name: "recursive glob with no matches returns error",
rawPath: "**/nonexistent/*.yaml",
env: &v1alpha1.Env{},
refSources: map[string]*v1alpha1.RefTarget{},
expectedPaths: nil,
expectedErr: true,
},
{
// z.yaml sorts after "nested/" alphabetically by full path, but doublestar processes
// each directory's own files before descending into subdirectories. So for envs/**/*.yaml:
// envs/ flat files (a, z) come before envs/nested/ files (c), giving:
// a.yaml, z.yaml, nested/c.yaml — not a.yaml, nested/c.yaml, z.yaml.
name: "** depth-order: flat files before nested even when flat file sorts after nested/ alphabetically",
rawPath: "envs/**/*.yaml",
env: &v1alpha1.Env{},
refSources: map[string]*v1alpha1.RefTarget{},
expectedPaths: []string{
path.Join(tempDir, "main-repo", "envs", "a.yaml"),
path.Join(tempDir, "main-repo", "envs", "z.yaml"),
path.Join(tempDir, "main-repo", "envs", "nested", "c.yaml"),
},
},
{
name: "recursive glob in external ref repo",
rawPath: "$ref/prod/**/*.yaml",
env: &v1alpha1.Env{},
refSources: map[string]*v1alpha1.RefTarget{
"$ref": {
Repo: v1alpha1.Repository{
Repo: "https://github.com/org/repo1",
},
},
},
expectedPaths: []string{
// doublestar matches zero path segments before recursing into subdirectories,
// so flat files (x, y) come before nested ones (nested/z).
path.Join(tempDir, "repo1", "prod", "x.yaml"),
path.Join(tempDir, "repo1", "prod", "y.yaml"),
path.Join(tempDir, "repo1", "prod", "nested", "z.yaml"),
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
repoPath := path.Join(tempDir, "main-repo")
resolvedPaths, err := getResolvedValueFiles(repoPath, repoPath, tt.env, []string{}, []string{tt.rawPath}, tt.refSources, paths, tt.ignoreMissingValueFiles)
if tt.expectedErr {
require.Error(t, err)
return
}
require.NoError(t, err)
require.Len(t, resolvedPaths, len(tt.expectedPaths))
for i, p := range tt.expectedPaths {
assert.Equal(t, p, string(resolvedPaths[i]))
}
})
}
// Deduplication: first occurrence of a resolved path wins. Subsequent references to the
// same file, whether explicit or via glob are silently dropped. This preserves the
// merge-precedence position set by the first mention of each file.
t.Run("glob then explicit: explicit entry placed at end, giving it highest Helm precedence", func(t *testing.T) {
t.Parallel()
repoPath := path.Join(tempDir, "main-repo")
resolvedPaths, err := getResolvedValueFiles(
repoPath, repoPath,
&v1alpha1.Env{}, []string{},
[]string{
"envs/*.yaml", // glob - z.yaml is explicit so skipped; only a.yaml added
"envs/z.yaml", // explicit - placed last, highest precedence
},
map[string]*v1alpha1.RefTarget{}, paths, false,
)
require.NoError(t, err)
require.Len(t, resolvedPaths, 2)
assert.Equal(t, path.Join(tempDir, "main-repo", "envs", "a.yaml"), string(resolvedPaths[0]))
assert.Equal(t, path.Join(tempDir, "main-repo", "envs", "z.yaml"), string(resolvedPaths[1]))
})
t.Run("explicit path before glob: explicit position is kept, glob re-match is dropped", func(t *testing.T) {
t.Parallel()
repoPath := path.Join(tempDir, "main-repo")
resolvedPaths, err := getResolvedValueFiles(
repoPath, repoPath,
&v1alpha1.Env{}, []string{},
[]string{
"prod/a.yaml", // explicit locks in position 0
"prod/*.yaml", // glob - a.yaml already seen, only b.yaml is new
},
map[string]*v1alpha1.RefTarget{}, paths, false,
)
require.NoError(t, err)
require.Len(t, resolvedPaths, 2)
assert.Equal(t, path.Join(tempDir, "main-repo", "prod", "a.yaml"), string(resolvedPaths[0]))
assert.Equal(t, path.Join(tempDir, "main-repo", "prod", "b.yaml"), string(resolvedPaths[1]))
})
t.Run("glob before explicit path: explicit position wins, glob skips the explicitly listed file", func(t *testing.T) {
t.Parallel()
repoPath := path.Join(tempDir, "main-repo")
resolvedPaths, err := getResolvedValueFiles(
repoPath, repoPath,
&v1alpha1.Env{}, []string{},
[]string{
"prod/*.yaml", // glob - a.yaml is explicit so skipped; only b.yaml added (pos 0)
"prod/a.yaml", // explicit - placed here at pos 1 (highest precedence)
},
map[string]*v1alpha1.RefTarget{}, paths, false,
)
require.NoError(t, err)
require.Len(t, resolvedPaths, 2)
assert.Equal(t, path.Join(tempDir, "main-repo", "prod", "b.yaml"), string(resolvedPaths[0]))
assert.Equal(t, path.Join(tempDir, "main-repo", "prod", "a.yaml"), string(resolvedPaths[1]))
})
t.Run("two overlapping globs: second glob only adds files not matched by first", func(t *testing.T) {
t.Parallel()
repoPath := path.Join(tempDir, "main-repo")
resolvedPaths, err := getResolvedValueFiles(
repoPath, repoPath,
&v1alpha1.Env{}, []string{},
[]string{
"prod/*.yaml", // adds a.yaml, b.yaml
"prod/**/*.yaml", // a.yaml, b.yaml already seen; adds nested/c.yaml, nested/d.yaml
},
map[string]*v1alpha1.RefTarget{}, paths, false,
)
require.NoError(t, err)
require.Len(t, resolvedPaths, 4)
assert.Equal(t, path.Join(tempDir, "main-repo", "prod", "a.yaml"), string(resolvedPaths[0]))
assert.Equal(t, path.Join(tempDir, "main-repo", "prod", "b.yaml"), string(resolvedPaths[1]))
assert.Equal(t, path.Join(tempDir, "main-repo", "prod", "nested", "c.yaml"), string(resolvedPaths[2]))
assert.Equal(t, path.Join(tempDir, "main-repo", "prod", "nested", "d.yaml"), string(resolvedPaths[3]))
})
t.Run("explicit paths take priority: globs skip explicitly listed files, which are placed at their explicit positions", func(t *testing.T) {
t.Parallel()
repoPath := path.Join(tempDir, "main-repo")
resolvedPaths, err := getResolvedValueFiles(
repoPath, repoPath,
&v1alpha1.Env{}, []string{},
[]string{
"prod/a.yaml", // explicit - pos 0
"prod/*.yaml", // a.yaml and b.yaml are both explicit, skipped entirely
"prod/b.yaml", // explicit - pos 1
"prod/**/*.yaml", // a.yaml, b.yaml, nested/c.yaml all explicit and skipped; nested/d.yaml added - pos 2
"prod/nested/c.yaml", // explicit - pos 3
},
map[string]*v1alpha1.RefTarget{}, paths, false,
)
require.NoError(t, err)
require.Len(t, resolvedPaths, 4)
assert.Equal(t, path.Join(tempDir, "main-repo", "prod", "a.yaml"), string(resolvedPaths[0]))
assert.Equal(t, path.Join(tempDir, "main-repo", "prod", "b.yaml"), string(resolvedPaths[1]))
assert.Equal(t, path.Join(tempDir, "main-repo", "prod", "nested", "d.yaml"), string(resolvedPaths[2]))
assert.Equal(t, path.Join(tempDir, "main-repo", "prod", "nested", "c.yaml"), string(resolvedPaths[3]))
})
}
func Test_verifyGlobMatchesWithinRoot(t *testing.T) {
t.Parallel()
tempDir := t.TempDir()
repoDir := filepath.Join(tempDir, "repo")
outsideDir := filepath.Join(tempDir, "outside")
require.NoError(t, os.MkdirAll(filepath.Join(repoDir, "values", "sub"), 0o755))
require.NoError(t, os.MkdirAll(outsideDir, 0o755))
// Files used as symlink targets
inRepoFile := filepath.Join(repoDir, "values", "real.yaml")
outsideFile := filepath.Join(outsideDir, "secret.yaml")
require.NoError(t, os.WriteFile(inRepoFile, []byte{}, 0o644))
require.NoError(t, os.WriteFile(outsideFile, []byte("password: hunter2"), 0o644))
// Symlink inside repo → file inside repo (safe)
inRepoLink := filepath.Join(repoDir, "values", "inrepo-link.yaml")
require.NoError(t, os.Symlink(inRepoFile, inRepoLink))
// Symlink inside repo → file outside repo (escape)
escapeLink := filepath.Join(repoDir, "values", "escape-link.yaml")
require.NoError(t, os.Symlink(outsideFile, escapeLink))
// Two-hop symlink: inside repo → another symlink (still inside) → file inside repo
hop1 := filepath.Join(repoDir, "values", "hop1.yaml")
require.NoError(t, os.Symlink(inRepoLink, hop1)) // hop1 → inRepoLink → real.yaml
// Two-hop symlink: inside repo → another symlink (inside repo) → file outside repo
hop2 := filepath.Join(repoDir, "values", "hop2.yaml")
require.NoError(t, os.Symlink(escapeLink, hop2)) // hop2 → escape-link → secret.yaml
tests := []struct {
name string
matches []string
expectErr bool
errContains string
}{
{
name: "regular file inside root passes",
matches: []string{inRepoFile},
},
{
name: "symlink inside root pointing to file inside root passes",
matches: []string{inRepoLink},
},
{
name: "two-hop chain that stays within root passes",
matches: []string{hop1},
},
{
name: "symlink pointing directly outside root is rejected",
matches: []string{escapeLink},
expectErr: true,
errContains: "resolved to outside repository root",
},
{
name: "two-hop chain that escapes root is rejected",
matches: []string{hop2},
expectErr: true,
errContains: "resolved to outside repository root",
},
{
name: "multiple matches all inside root pass",
matches: []string{inRepoFile, inRepoLink, hop1},
},
{
name: "one bad match in a list fails the whole call",
matches: []string{inRepoFile, escapeLink},
expectErr: true,
errContains: "resolved to outside repository root",
},
{
name: "empty matches list is a no-op",
matches: []string{},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
err := verifyGlobMatchesWithinRoot(tt.matches, repoDir)
if tt.expectErr {
require.Error(t, err)
assert.Contains(t, err.Error(), tt.errContains)
} else {
require.NoError(t, err)
}
})
}
}
// Test_getResolvedValueFiles_glob_symlink_escape is an integration-level check
// that verifyGlobMatchesWithinRoot is wired into glob expansion correctly: a
// symlink inside the repo pointing outside must cause getResolvedValueFiles to
// return an error rather than silently including the external file.
func Test_getResolvedValueFiles_glob_symlink_escape(t *testing.T) {
t.Parallel()
tempDir := t.TempDir()
paths := utilio.NewRandomizedTempPaths(tempDir)
repoDir := filepath.Join(tempDir, "repo")
outsideDir := filepath.Join(tempDir, "outside")
require.NoError(t, os.MkdirAll(filepath.Join(repoDir, "values"), 0o755))
require.NoError(t, os.MkdirAll(outsideDir, 0o755))
require.NoError(t, os.WriteFile(filepath.Join(repoDir, "values", "base.yaml"), []byte{}, 0o644))
require.NoError(t, os.WriteFile(filepath.Join(outsideDir, "secret.yaml"), []byte("password: hunter2"), 0o644))
require.NoError(t, os.Symlink(filepath.Join(outsideDir, "secret.yaml"), filepath.Join(repoDir, "values", "escape.yaml")))
_, err := getResolvedValueFiles(repoDir, repoDir, &v1alpha1.Env{}, []string{}, []string{"values/*.yaml"}, map[string]*v1alpha1.RefTarget{}, paths, false)
require.Error(t, err)
assert.Contains(t, err.Error(), "resolved to outside repository root")
}
func Test_isGlobPath(t *testing.T) {
tests := []struct {
path string
expected bool
}{
{
path: "prod/*.yaml",
expected: true,
},
{
path: "prod/?.yaml",
expected: true,
},
{
path: "prod[ab].yaml",
expected: true,
},
{
path: "prod/**/*.yaml",
expected: true,
},
{
path: "prod/values.yaml",
},
{
path: "values.yaml",
},
{
path: "",
},
{
path: "/absolute/path/to/*.yaml",
expected: true,
},
{
path: "/absolute/path/to/values.yaml",
},
{
path: "*",
expected: true,
},
{
path: "?",
expected: true,
},
{
path: "[",
expected: true,
},
}
for _, tt := range tests {
t.Run(tt.path, func(t *testing.T) {
assert.Equal(t, tt.expected, isGlobPath(tt.path))
})
}
}
func Test_getReferencedSource(t *testing.T) {
t.Parallel()
refTarget := &v1alpha1.RefTarget{
Repo: v1alpha1.Repository{
Repo: "https://github.com/org/repo1",
},
}
tests := []struct {
name string
rawValueFile string
refSources map[string]*v1alpha1.RefTarget
expected *v1alpha1.RefTarget
}{
{
name: "ref with file path found in map",
rawValueFile: "$ref/values.yaml",
refSources: map[string]*v1alpha1.RefTarget{
"$ref": refTarget,
},
expected: refTarget,
},
{
name: "ref with file path not in map",
rawValueFile: "$ref/values.yaml",
refSources: map[string]*v1alpha1.RefTarget{},
expected: nil,
},
{
name: "bare ref without file path found in map",
rawValueFile: "$ref",
refSources: map[string]*v1alpha1.RefTarget{
"$ref": refTarget,
},
expected: refTarget,
},
{
name: "empty string returns nil",
rawValueFile: "",
refSources: map[string]*v1alpha1.RefTarget{
"$ref": refTarget,
},
expected: nil,
},
{
name: "no $ prefix returns nil",
rawValueFile: "values.yaml",
refSources: map[string]*v1alpha1.RefTarget{
"$ref": refTarget,
},
expected: nil,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
result := getReferencedSource(tt.rawValueFile, tt.refSources)
assert.Equal(t, tt.expected, result)
})
}
}
func TestErrorGetGitDirectories(t *testing.T) {
// test not using the cache
root := "./testdata/git-files-dirs"

73
test/e2e/helm_test.go
View file

@ -156,6 +156,79 @@ func TestHelmIgnoreMissingValueFiles(t *testing.T) {
Expect(ErrorRegex("Error: open .*does-not-exist-values.yaml: no such file or directory", ""))
}
// TestHelmGlobValueFiles verifies that a glob pattern in valueFiles expands to all matching
// files and that they are applied in lexical order (last file wins in helm merging).
// envs/*.yaml expands to envs/a.yaml then envs/b.yaml - b.yaml is last, so foo = "b-value".
func TestHelmGlobValueFiles(t *testing.T) {
fixture.SkipOnEnv(t, "HELM")
ctx := Given(t)
ctx.Path("helm-glob-values").
When().
CreateApp().
AppSet("--values", "envs/*.yaml").
Sync().
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(HealthIs(health.HealthStatusHealthy)).
Expect(SyncStatusIs(SyncStatusCodeSynced)).
And(func(_ *Application) {
val := errors.NewHandler(t).FailOnErr(fixture.Run(".", "kubectl", "-n", ctx.DeploymentNamespace(),
"get", "cm", "my-map", "-o", "jsonpath={.data.foo}")).(string)
assert.Equal(t, "b-value", val)
})
}
// TestHelmRecursiveGlobValueFiles verifies that the ** double-star pattern recursively
// matches files at any depth. envs/**/*.yaml expands (zero-segments first) to:
// envs/a.yaml, envs/b.yaml, envs/nested/c.yaml - c.yaml is last, so foo = "c-value".
func TestHelmRecursiveGlobValueFiles(t *testing.T) {
fixture.SkipOnEnv(t, "HELM")
ctx := Given(t)
ctx.Path("helm-glob-values").
When().
CreateApp().
AppSet("--values", "envs/**/*.yaml").
Sync().
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(HealthIs(health.HealthStatusHealthy)).
Expect(SyncStatusIs(SyncStatusCodeSynced)).
And(func(_ *Application) {
val := errors.NewHandler(t).FailOnErr(fixture.Run(".", "kubectl", "-n", ctx.DeploymentNamespace(),
"get", "cm", "my-map", "-o", "jsonpath={.data.foo}")).(string)
assert.Equal(t, "c-value", val)
})
}
// TestHelmGlobValueFilesNoMatch verifies that a glob pattern with no matching files
// surfaces as a comparison error on the application.
func TestHelmGlobValueFilesNoMatch(t *testing.T) {
fixture.SkipOnEnv(t, "HELM")
Given(t).
Path("helm-glob-values").
When().
CreateApp().
AppSet("--values", "nonexistent/*.yaml").
Then().
Expect(Condition(ApplicationConditionComparisonError, `values file glob "nonexistent/*.yaml" matched no files`))
}
// TestHelmGlobValueFilesIgnoreMissing verifies that a non-matching glob pattern is
// silently skipped when ignoreMissingValueFiles is set, and the app syncs successfully.
func TestHelmGlobValueFilesIgnoreMissing(t *testing.T) {
fixture.SkipOnEnv(t, "HELM")
Given(t).
Path("helm-glob-values").
When().
CreateApp().
AppSet("--values", "nonexistent/*.yaml", "--ignore-missing-value-files").
Sync().
Then().
Expect(OperationPhaseIs(OperationSucceeded)).
Expect(HealthIs(health.HealthStatusHealthy)).
Expect(SyncStatusIs(SyncStatusCodeSynced))
}
func TestHelmValuesMultipleUnset(t *testing.T) {
Given(t).
Path("helm").

3
test/e2e/testdata/helm-glob-values/Chart.yaml vendored Normal file
View file

@ -0,0 +1,3 @@
apiVersion: v2
version: 1.0.0
name: helm-glob-values

1
test/e2e/testdata/helm-glob-values/envs/a.yaml vendored Normal file
View file

@ -0,0 +1 @@
foo: a-value

1
test/e2e/testdata/helm-glob-values/envs/b.yaml vendored Normal file
View file

@ -0,0 +1 @@
foo: b-value

1
test/e2e/testdata/helm-glob-values/envs/nested/c.yaml vendored Normal file
View file

@ -0,0 +1 @@
foo: c-value

6
test/e2e/testdata/helm-glob-values/templates/config-map.yaml vendored Normal file
View file

@ -0,0 +1,6 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: my-map
data:
foo: {{.Values.foo}}

1
test/e2e/testdata/helm-glob-values/values.yaml vendored Normal file
View file

@ -0,0 +1 @@
foo: default

6
util/argo/argo.go
View file

@ -910,6 +910,12 @@ func verifyGenerateManifests(
// and not whether it actually contains any manifests.
_, err = repoClient.GenerateManifest(ctx, &req)
if err != nil {
// A glob pattern matching no files is a runtime condition, not a spec error —
// the files may be added later. Skip adding an InvalidSpecError here and let
// the app controller surface it as a ComparisonError during reconciliation.
if status.Code(err) == codes.NotFound && strings.Contains(err.Error(), "matched no files") {
continue
}
errMessage := fmt.Sprintf("Unable to generate manifests in %s: %s", source.Path, err)
conditions = append(conditions, argoappv1.ApplicationCondition{
Type: argoappv1.ApplicationConditionInvalidSpecError,