* Adds consistent hashing with bound loads sharding algorithm
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
* Make the assignement consistent accross all clusters
- The assignment or running of the algorithm has to be consistent across all the clusters. Changed the function to return a map where the consistent hash will be used to build the map
- Modifications to the createConsistentHashsingWithBoundLoads function. This will create the map for cluster to shard. Note that the list must be consistent across all shards so that is why the cluster list must be sorted before going through the consistent hash algorithm
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
* Extracting constant and simplifying boolean expression
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
* Update docs: consistent-hashing sharding algorithm
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
---------
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
* Adding app list to sharding cache
Signed-off-by: Andrew Lee <andrewkl@enclavenet.com>
* Add shard by apps test
Signed-off-by: Andrew Lee <andrewkl@enclavenet.com>
* Fix lint
Signed-off-by: Andrew Lee <andrewkl@enclavenet.com>
* Add coverage to test
Signed-off-by: Andrew Lee <andrewkl@enclavenet.com>
* Fix lint
Signed-off-by: Andrew Lee <andrewkl@enclavenet.com>
* Converted cluster/app accesors to private, add apps-in-any-namespace suport in shardingcache init, added read lock to GetAppDistribution
Signed-off-by: Andrew Lee <andrewkl@enclavenet.com>
* Fix tests
Signed-off-by: Andrew Lee <andrewkl@enclavenet.com>
---------
Signed-off-by: Andrew Lee <andrewkl@enclavenet.com>
* fix: infer correct shard in statefulset setup
Signed-off-by: Lukas Wöhrl <lukas.woehrl@plentymarkets.com>
* fix the case if only a single replica
Signed-off-by: Lukas Wöhrl <lukas.woehrl@plentymarkets.com>
* fix: resolving pointer on shard compare
Signed-off-by: Lukas Wöhrl <lukas.woehrl@plentymarkets.com>
* fix: add readlock for cluster accessor
Signed-off-by: Lukas Wöhrl <lukas.woehrl@plentymarkets.com>
* fix: use defer to protect access of 'shard'
Signed-off-by: Lukas Wöhrl <lukas.woehrl@plentymarkets.com>
* fix: revert locking in getclusteraccessor
Signed-off-by: Lukas Wöhrl <lukas.woehrl@plentymarkets.com>
* fix: handle nil shard case
Signed-off-by: Lukas Wöhrl <lukas.woehrl@plentymarkets.com>
* fix: handle any nil shard value as false
Signed-off-by: Lukas Wöhrl <lukas.woehrl@plentymarkets.com>
* fix: handle nil case and fix another missing pointer dereference
Signed-off-by: Lukas Wöhrl <lukas.woehrl@plentymarkets.com>
* revert
Signed-off-by: Lukas Wöhrl <lukas.woehrl@plentymarkets.com>
* fix: added tests and fixed some behaviour bugs
Signed-off-by: Lukas Wöhrl <lukas.woehrl@plentymarkets.com>
* test: add test to validate that Shard value is not overriden
Signed-off-by: Lukas Wöhrl <lukas.woehrl@plentymarkets.com>
* fix: added tests and fixe the case when server is changed inside a secret
Signed-off-by: Lukas Wöhrl <lukas.woehrl@plentymarkets.com>
* tests: add test cases for infering the shard logic
Signed-off-by: Lukas Wöhrl <lukas.woehrl@plentymarkets.com>
---------
Signed-off-by: Lukas Wöhrl <lukas.woehrl@plentymarkets.com>
* feat(sharding): use a cache
Signed-off-by: Alexandre Gaudreault <alexandre.gaudreault@logmein.com>
* cluster cmd
Signed-off-by: Alexandre Gaudreault <alexandre.gaudreault@logmein.com>
* - Assign shard 0 to in-cluster cluster and nil check updates
- Caching clusters while sharding: Fixing unit tests
- Update generated docs
- Debug e2e tests
- Default the shardNumber to the number of replicas if it is calculated to a higher value
- defered Unlock only when a lock is set
- Disabling temporarly other versions of k3s to check if e2e passes
- Do not fail if hostname format is not abc-n
- Fix unit test and skip some e2e
- Skip TestGitSubmoduleHTTPSSupport test
- Remove breaking defer c.lock.Unlock()
- Reverting testing all k3s version
- Default sharding fix
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
* fixes related to code review: renaming structure param, moving db initialisation
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
* Code review
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
* Set default shard to 0
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
* Set different default value for Sts and Deployment mode
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
* Expose ClusterShardingCache
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
* Removing use of argoDB.db for DistributionFunction
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
* Update generated documentation
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
* Fix comment about NoShardingDistributionFunction and NoShardingAlgorithm
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
---------
Signed-off-by: Alexandre Gaudreault <alexandre.gaudreault@logmein.com>
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
Co-authored-by: Alexandre Gaudreault <alexandre.gaudreault@logmein.com>
* fix: Extraction of DistributionFunction to allow passing different type of functions to filter clusters by shard
- Adding unit tests for sharding
- Refresh clusters list on DistributionFunction call
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
Signed-off-by: ishitasequeira <ishiseq29@gmail.com>
* fix: Incorrect conversion of an integer with architecture-dependent bit size from [strconv.Atoi](1) to a lower bit size type uint32 without an upper bound check.
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
* Added config to switch to round-robin sharding
Signed-off-by: Raghavi Shirur <rshirur@redhat.com>
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
* Documenting sharding more, adding shuffling tests (skipped), re-enable sharding algo env var
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
* Allow configuration through argocd-cmd-params-cm configMap and key: controller.sharding.algorithm
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
* De-duplicate code, remove reflection for default case, shorten distribution methods name, ran codegen on manifests
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
---------
Signed-off-by: Akram Ben Aissi <akram.benaissi@gmail.com>
Signed-off-by: ishitasequeira <ishiseq29@gmail.com>
Signed-off-by: Raghavi Shirur <rshirur@redhat.com>
Co-authored-by: Raghavi Shirur <raghaviss11@gmail.com>
