Commit graph

305 commits

Author SHA1 Message Date
William Tam
9aae99cf7f
fix: Only processes need root uid will run as root inside test container (#4210)
* Add  "chown" to gpg/keys  in "start-local" target that so that repo-server can access gpg keys.

* * Set -u uid:gid in the docker run commands  so that test images are run under the current user.
* test Procfile processes will not need to perform "su"  to  default user (which has the current user's uid/gid)

* Remove chown in start-e2e-local

* clean up, remove  "bash -c"

* Test containers are run as uid 0 which allows uid_entrypoint.sh  to perform some user setup.  uid_entrypoint.sh creates a non-root user (default) and enables passwordless sudo for that user.    The container entry point command is run as the non-root user.   "goreman start"  does "sudo" to to the processes that need root permission including sshd, fcgiwrap, and nginix.   The other processes are running as the non-root user.

* use /bin/bash

* change back to sh

* Docker image to create  unpriveleged testuser and enable passwordless sudo for that user
2020-09-15 11:32:16 +02:00
May Zhang
cfb925c0d4
fix: app create with -f should not ignore other options (#4322) 2020-09-14 15:29:45 -07:00
Alexander Matyushentsev
df4002a987
fix: fix flaky TestTrackAppStateAndSyncApp e2e test (#4325) 2020-09-14 11:45:59 -07:00
Alexander Matyushentsev
aaab777d27
fix: optional 'name' flag of add-orphaned-ignore/remove-orphaned-ignore command should be named, not positional (#4062) 2020-08-06 15:28:22 -07:00
Alexander Matyushentsev
ba71ad934c
refactor: upgrade k8s client to 1.18 (#4046)
* refactor: upgrade k8s client to 1.18

* remove unnecessary go mod replacements
2020-08-05 11:36:40 -07:00
May Zhang
613af547c3
feat: adding validate for app create and app set (#4016)
* feat: adding disable-validation for app create and app set

* feat: adding disable-validation for app create and app set

* feat: change test func name

* feat: added support of app unset and app edit in addition to app create and app set.

* feat: remove extra space.
2020-07-30 13:28:36 -07:00
Darshan Chaudhary
9b99276d59
fix: skipped resources should not impact sync status (#3986)
* fix: skipped resources should not impact sync status

Signed-off-by: darshanime <deathbullet@gmail.com>
2020-07-30 10:09:12 -07:00
Alexander Matyushentsev
a6399e59e1
feat: support retrying failed sync attempts (#3997)
* feat: support retrying failed sync attempts

* fix: sync results should be cleared in retry sync attempt
2020-07-28 10:14:17 -07:00
May Zhang
067dcce88d
feat: auto create namespace (#3976)
* feat: auto create namespace

* feat: fixing yarn lint error.

* feat: fixing yarn lint error.

* update to the latest gitops-engine

* fix tidiness of go.sum
2020-07-27 13:33:08 -07:00
Liviu Costea
275daa7976
feat: Support applications with cluster name in the ui #1548 (#3944)
* feat: Support applications with cluster name in the ui

* Retrigger CI pipeline
2020-07-24 15:37:58 -07:00
Alexander Matyushentsev
1a36fd178a
chore: stop using pre-built docker images for testing/code generation (#3955) 2020-07-23 08:05:06 +02:00
rachelwang20
52926b7cb0
feat: Create argocd app resources CLI with various filters. (#3946)
* Include sub and and iat in PermissionDenied message

* Run test-local

* Fix typo

* Fix lint error

* iat fromat changing

* Adding MapClaims convertion

* Fix lint issue

* Fix golang lint

* Fix blank space

* Fix missing field

* Adding Orphaned exception list

* Fixed lint error

* Rebased on master

* Adding group kind label

* golangci-lint run

* Addressed comments

* Fixed lint errors

* Method rename

* orphaned ignore list cli support

* Orphaned resources cli support

* Display orphaned resource cli

* Added tests

* Create argocd app resources command

* Addressed comments
2020-07-20 13:29:25 -07:00
Alexander Matyushentsev
c8def406b0
refactor: upgrade gitops engine (#3962) 2020-07-20 10:11:17 -07:00
rachelwang20
561f30815c
feat: Orphaned ignore list cli support (#3922)
* Include sub and and iat in PermissionDenied message

* Run test-local

* Fix typo

* Fix lint error

* iat fromat changing

* Adding MapClaims convertion

* Fix lint issue

* Fix golang lint

* Fix blank space

* Fix missing field

* Adding Orphaned exception list

* Fixed lint error

* Rebased on master

* Adding group kind label

* golangci-lint run

* Addressed comments

* Fixed lint errors

* Method rename

* orphaned ignore list cli support
2020-07-14 14:53:56 -07:00
Hiroki Sakamoto
6f77d9b7bb
chore: pass required envs to repo server (#3929) 2020-07-14 09:38:30 +02:00
Alexander Matyushentsev
60637e6df2
fix: remove TestSyncOptionDryRun test (#3939) 2020-07-13 11:59:30 -07:00
rachelwang20
95820cf64f
feat: User can define Orphaned exception avoiding unnecessary warnings (#3900)
* Include sub and and iat in PermissionDenied message

* iat fromat changing

* Adding MapClaims convertion

* Adding Orphaned exception list

* Adding group kind label

* Fixed lint errors

* Method rename
2020-07-09 16:19:47 -07:00
Darshan Chaudhary
63acc26211
fix: don't refresh if dry run (#3891)
* fix: don't refresh if dry run

Signed-off-by: darshanime <deathbullet@gmail.com>

* fix: add e2e test for sync with dry-run

Signed-off-by: darshanime <deathbullet@gmail.com>

* fix: refresh only if not dryRun for sync command

Signed-off-by: darshanime <deathbullet@gmail.com>
2020-07-08 09:43:39 -07:00
Darshan Chaudhary
b3c118d4c0
feat: ignore status globally (#3754)
feat: ignore status globally (#3754)
2020-07-02 23:28:36 -07:00
May Zhang
d676209daa
fix: App Creation Requires Namespace Even When Repo Contains Only Global Custom Resource (#3874)
* fix: App Creation Requires Namespace Even When Repo Contains Only Global Custom Resource

* fix a typo

* fix a failed test
2020-06-29 16:33:13 -07:00
Alexander Matyushentsev
20eb8bbc4d
fix: application list page consumes too much CPU (#3849)
* fix: application list page consumes too much CPU

* add broadcasterHandler tests
2020-06-29 15:32:18 -07:00
May Zhang
7fbf51c346
fix: jwtTokens are reset when applying AppProjects (#3791)
* jwtTokens are reset when applying AppProjects

* fix unit tests in server_test.go
checking in generated.pb.go

* fix unit tests in rbackpolicy_test.go

* fix yarnl lint errors

* fix delete token in both spec and status

* add tests

* fixing failed test

* fixing failed test

* retry 3 times during update

* renamed CRD fields.
Updated nomalize method.

* fixed UI

* fixing merge conflicts

* fixing merge conflicts

* removed unused variables in UI
renamed a CRD field
updated combineToken logic using map

* Only update project which needs normalize token.

* Changed logging.

* check for nil

* Fix UI

* added project role get tests

* rename variables

* clean up

* fixing failed tests

* fixing failed tests

* fixing error handling for remove token

* log err when we have retried 3 times

* sort tokens

* sort tokens
2020-06-25 15:11:24 -07:00
jannfis
c76d7b9c7c
refactor: There were some more places needed for JSON marshaler migration (#3840) 2020-06-24 20:34:29 +02:00
Alexander Matyushentsev
5d5d6a4ad6
feat: Display cluster info on cluster details page (#3793)
* feat: Display cluster info on cluster details page

* Store cluster info in cache instead of secret
2020-06-22 17:51:20 -07:00
jannfis
be718e2b61
feat: GPG commit signature verification (#2492) (#3242)
* Add initial primitives and tests for GPG related operations

* More tests and test documentation

* Move gpg primitives to own module

* Add initial primitives for running git verify-commit and tests

* Improve and better comment test

* Implement VerifyCommitSignature() primitive for metrics wrapper

* More commentary

* Make reposerver verify gpg signatures when generating manifests

* Make signature validation optional

* Forbid use of local manifests when signature verification is enabled

* Introduce new signatureKeys field in project CRD

* Initial support for only syncing against signed revisions

* Updates to GnuPG primitives and more test cases

* Move signature verification to correct place and add tests

* Add signature verification result to revision metadata and display it in UI

* Add more primitives and move out some stuff to common module

* Add more testdata

* Add key management primitives to ArgoDB

* Move type GnuPGPublicKey to appsv1 package

* Add const ArgoCDGPGKeysConfigMapName

* Handle key operations with appsv1.GnuPGPublicKey

* Add initial API for managing GPG keys

* Remove deprecated code

* Add primitives for adding public keys to configuration

* Change semantics of ValidateGPGKeys to return more key information

* Add key import functionality to public key API

* Fix code quirks reported by linter

* More code quirks fixes

* Fix test

* Add primitives for deleting keys from configuration

* Add delete key operation to API and CLI

* Cosmetics

* Implement logic to sync configuration to keyring in repo-server

* Add IsGPGEnabled() primitive and also update trustdb on ownertrust changes

* Use gpg.IsGPGEnabled() instead of custom test

* Remove all keyring manipulating methods from DB

* Cosmetics/comments

* Require grpc methods from argoproj pkg

* Enable setting config path via ARGOCD_GPG_DATA_PATH

* Allow "no" and any cases in ARGOCD_GPG_ENABLED

* Enable GPG feature on start and start-e2e and set required environment

* Cosmetics/comments

* Cosmetics and commentary

* Update API documentation

* Fix comment

* Only run GPG related operations if GPG is enabled

* Allow setting ARGOCD_GPG_ENABLE from the environment

* Create GPG ConfigMap resource during installation

* Use function instead of constant to get the watcher path

* Re-watch source path in case it gets recreated. Also, error on finish

* Add End-to-End tests for GPG commit verification

* Introduce SignatureKey type for AppProject CRD

* Fix merge error from previous commit

* Adapt test for additional manifest (argocd-gpg-keys-cm.yaml)

* Fix linter issues

* Adapt CircleCI configuration to enable running tests

* Add wrapper scripts for git and gpg

* Sigh.

* Display gpg version in CircleCI

* Install gnupg2 and link it to gpg in CI

* Try to install gnupg2 in CircleCI image

* More CircleCI tweaks

* # This is a combination of 10 commits.
# This is the 1st commit message:

Containerize tests - test cycle

# This is the commit message #2:

adapt working directory

# This is the commit message #3:

Build before running tests (so we might have a cache)

# This is the commit message #4:

Test limiting parallelism

# This is the commit message #5:

Remove unbound variable

# This is the commit message #6:

Decrease parallelism to find out limit

# This is the commit message #7:

Use correct flag

# This is the commit message #8:

Update Docker image

# This is the commit message #9:

Remove build phase and increase parallelism

# This is the commit message #10:

Further increase parallelism

* Dockerize toolchain

* Add new targets to Makefile

* Codegen

* Properly handle permissions for E2E tests

* Remove gnupg2 installation from CircleCI configuration

* Limit parallelism of build

* Fix Yarn lint

* Retrigger CI for possible flaky test

* Codegen

* Remove duplicate target in Makefile

* Pull in pager from dep ensure -v

* Adapt to gitops-engine changes and codegen

* Use new health package for health status constants

* Add GPG methods to ArgoDB mock module

* Fix possible nil pointer dereference

* Fix linter issue in imports

* Introduce RBAC resource type 'gpgkeys' and adapt policies

* Use ARGOCD_GNUPGHOME instead of GNUPGHOME for subsystem configuration

Also remove some deprecated unit tests.

* Also register GPG keys API with gRPC-GW

* Update from codegen

* Update GPG key API

* Add web UI to manage GPG keys

* Lint updates

* Change wording

* Add some plausibility checks for supplied data on key creation

* Update from codegen

* Re-allow binary keys and move check for ASCII armoured to UI

* Make yarn lint happy

* Add editing signature keys for projects in UI

* Add ability to configure signature keys for project in CLI

* Change default value to use for GNUPGHOME

* Do not include data section in default gpg keys CM

* Adapt Docker image for GnuPG feature

* Add required configuration to installation manifests

* Add add-signature-key and remove-signature-key commands to project CLI

* Fix typo

* Add initial user documentation for GnuPG verification

* Fix role name - oops

* Mention required RBAC roles in docs

* Support GPG verification of git annotated tags as well

* Ensure CLI can build succesfully

* Better support verification on tags

* Print key type in upper case

* Update user documentation

* Correctly disable GnuPG verification if ARGOCD_GPG_ENABLE=false

* Clarify that this feature is only available with Git repositories

* codegen

* Move verification code to own function

* Remove deprecated check

* Make things more developer friendly when running locally

* Enable GPG feature by default, and don't require ARGOCD_GNUPGHOME to be set

* Revert changes to manifests to reflect default enable state

* Codegen
2020-06-22 18:21:53 +02:00
Daichi Sakaue
9192cd94c9
chore: Update Redis to 5.0.8 (#3734)
Signed-off-by: Daichi Sakaue <daichi-sakaue@cybozu.co.jp>
2020-06-11 14:41:18 -07:00
May Zhang
d040d9bf04
feat: Support additional metadata in Application sync operation (#3747)
* feat: Support additional metadata in Application sync operation

* regenerated generated.pb.go
2020-06-10 15:28:07 -07:00
Darshan Chaudhary
e102ec11ac
feat: Allow --local with automatic sync for --dry-run (#3675)
* feat: Allow --local with automatic sync for --dry-run

Signed-off-by: darshanime <deathbullet@gmail.com>

* feat: add e2e test for local sync with dry run

Signed-off-by: darshanime <deathbullet@gmail.com>
2020-06-10 14:05:37 +02:00
May Zhang
0a815be07a
fix: support partial sync with namespace. (#3705)
* support partial sync with namespace.

* corrected test folder name

* Trying to fix lint error

* 1. in test, delete ns after test
2. in test, created new methods for ResourceSyncStatusWithNamespaceIs and ResourceHealthWithNamespaceIs.

* reformat imports

* simplify code

* remove timeout
2020-06-08 15:59:59 -07:00
Alexander Matyushentsev
4a6fe4cd31
feat: upgrade kustomize to v3.6.1 version (#3696) 2020-06-02 09:37:10 -07:00
jannfis
80515a3b57
chore: Migrate CI toolchain to GitHub actions (#3677)
* chore: Migrate CI to GitHub actions

* Do not install golangci-lint, we use the action

* Integrate codecov.io upload

* Use some better names for analyze job & steps

* go mod tidy

* Update tools

* Disable CircleCI completely

* Satisfy CircleCI with a dummy step until it's disabled
2020-05-31 23:11:28 +02:00
jannfis
bc83719037
chore: Fix complaints of golang-ci lint v1.26.0 (#3673) 2020-05-30 18:54:14 -07:00
Henno Schooljan
7b2a95e83c
fix: Allow unsetting the last remaining values file (#3644) (#3645)
* fix: Allow unsetting the last values file (#3644)

Because the `setHelmOpt()` function does not act on empty inputs, it 
would do nothing when removing the last values file using `argocd app 
unset`.
The parameter overrides are actually being unset correctly, so this has 
been changed to work the same way by manipulating `app.Spec.Source.Helm` 
directly.

This fixes #3644.

* fix: Allow unsetting the last values file, add tests

* Retrigger CI pipeline
2020-05-29 22:27:40 -07:00
Henno Schooljan
36da074344
feat: CLI: Allow setting Helm values literal (#3601) (#3646)
* feat: CLI: Allow setting Helm values literal (#3601)

While you could already set values files using the `--values` flag with external data using the CLI with `argocd app create` and `argocd app set`, this was not yet possible for managing the literal `.spec.source.helm.values` value in an Application without resorting to a complicated `argocd app patch` escaped parameter or by generating the entire application YAML manifest by yourself.

Therefore, this PR adds a `--values-literal-file` flag to the `argocd app create` and `argocd app set` commands, which accepts a local file name or URL to a values file, which will be read and included as a multiline string in the application manifest. This is different from the `--helm-set-file` flag which expects the file in the chart itself.

The `argocd app unset` command is expanded with a `--values-literal` flag, so we can also unset this field again.

I hope I chose nice enough names for the flags, I wanted to make clear it expects a file name, but also distinguish it enough from the existing `--values` flag which actually points to values files.

Because the current `setHelmOpt()` functionality would not work for unsetting things to an empty value and it was difficult to do these changes independently, this PR also contains the fix for issue #3644. A separate PR has still been created for that one because I think it should end up as a separate issue in the release notes.

* feat: CLI: Allow setting Helm values literal, add tests
2020-05-29 09:00:28 -07:00
Alexander Matyushentsev
ee64a4d9ca
fix: upgrade gitops engine dependency (#3668) 2020-05-28 18:42:01 -07:00
Josh Soref
a724574ede
chore: Spelling (#3647)
chore: Spelling (#3647)
2020-05-27 10:22:13 -07:00
Alexander Matyushentsev
22dcc1e87d
fix: prevent possible null pointer dereference error in TestAddHelmRepoInsecureSkipVerify test (#3650) 2020-05-26 10:52:12 -07:00
Alexander Matyushentsev
fe8d47e0ea
feat: move engine code to argoproj/gitops-engine repo (#3599) 2020-05-15 14:39:29 -07:00
Alexander Matyushentsev
192ee93fc4
feat: Gitops engine (#3066)
* Move utils packages that are required for gitops engine under engine/pkg/utils package.
Following changes were implemented:
* util/health package is split into two parts: resource health assessement & resource health assessement and moved into engine/pkg/utils
* utils packages moved: Closer and Close method of util package moved into engine/pkg/utils/io package
* packages diff, errors, exec, json, kube and tracing moved into engine/pkg/utils

* Move single cluster caching into engine/kube/cache package

* move sync functionality to engine/kube/sync package

* remove dependency on metrics package from engine/pkg/utils/kube/cache

* move annotation label definitions into engine/pkg/utils/kube/sync

* make sure engine/pkg has no dependencies on other argo-cd packages

* allow importing engine as a go module

* implement a high-level interface that might be consumed by flux

* fix deadlock caused by cluster cache event handler

* ClusterCache should return error if requested group kind not found

* remove obsolete tests

* apply reviewer notes
2020-05-15 10:01:18 -07:00
Alexander Matyushentsev
31df9d11a9
feat: upgrade helm 3 to v3.2.0; user --insecure-verify-flag (#3514) 2020-04-29 21:04:25 -07:00
Alexander Matyushentsev
9b142c799a
fix: 'argocd sync' does not take into account IgnoreExtraneous annotation (#3486) 2020-04-28 08:49:07 -07:00
Alexander Matyushentsev
310b40aa20
feat: downgrade dex to 2.22.0 and revert bug workaround (#3468) 2020-04-22 17:34:58 -07:00
Alexander Matyushentsev
acc2369dc7
feat: upgrade dex to v2.23.0 (#3448)
* feat: upgrade dex to v2.23.0

* workaround for https://github.com/dexidp/dex/issues/1695
2020-04-20 22:22:19 -07:00
May Zhang
6ffd34dcf9
fix for helm repo add with flag --insecure-skip-server-verification (#3420) 2020-04-15 16:56:27 -07:00
May Zhang
3dbc330cf0
fix: app diff --local support for helm repo. #3151 (#3407)
* Fixing argocd app diff when using helm repo

* adding test code

* get rid of optional parameter

* get rid of optional parameter

* Added test case

* Fix failed tests
2020-04-14 08:13:38 -07:00
Alexander Matyushentsev
4c812576c1
chore: minor dev tools fixing for mac (#3330) 2020-04-09 15:01:45 +02:00
May Zhang
9e6c78d55c
Fix for jsonnet when it is localed in nested subdirectory and uses import (#3372) 2020-04-06 15:20:07 -07:00
Alexander Matyushentsev
5448466ddc
feat: support normalizing CRD fields that use known built-in K8S types (#3357)
* feat: support normalizing CRD fields that use known built-in K8S types

* apply reviewers notes

* fix codegen
2020-04-06 21:13:50 +02:00
Alexander Matyushentsev
e5452ff70e
fix: return 401 error code if username does not exist (#3369) 2020-04-06 11:15:15 +02:00
May Zhang
55bc144410
fix: Helm v3 CRD are not deployed (#3345)
* Fixing could not find plugin issue when app sync and app diff

* Fixing codegen error

* Revert "Fixing codegen error"

This reverts commit b2dcfb81

* Fixing codegen error

* If user is logged in, settings API would return ConfigManagementPlugins

* For helm3, add flag --include-crds when calling helm template to support helm3 crd

* Fixing typo.

* Added further assertion of ResourceSyncStatusIs for CRD resources.
2020-04-03 13:41:17 -07:00