fleet/server/platform/endpointer/extract_alias_rules_test.go

208 lines
5.5 KiB
Go
Raw Normal View History

Deprecate "team" and "query" API params (#39873) <!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** For #39344 # Details This PR builds on the previous PR (https://github.com/fleetdm/fleet/pull/39847) which added `renameto` tags to certain API parameters to mark them as deprecated. How this is used: ### In requests * When decoding requests, log a warning if a `json` or `query` param is used that has a `renameto` tag, e.g. if a `team_id` param is sent but the related struct has `renameto:"fleet_id"` in it. * If the `renamedto` version (e.g. `fleet_id`) is sent in the request, rewrite it to the deprecated name so that it can be unmarshalled into the struct * If both versions are sent (e.g. `team_id` AND `fleet_id`), throw an error and quit * URLs with deprecated terms have new aliases using `WithAltPaths` -- warning on using old URLSs a TODO that will be handled in a subsequent PR. ### In responses * Output _both_ the deprecated and new names for fields that have `renameto` tags, so that we don't break existing workflows expecting the old keys. Uses a shared `DuplicateJSONKeys` to do the duplication. * Most API responses are handled in `EncodeCommonResponse`. Exceptions are activities, failing policy webhooks and the streaming "list hosts" endpoints which call the function directly. ### In fleetctl * Similar to requests, log warnings when deprecated keys are used and rewrite the new keys internally so that they can be unmarshalled. * For `fleetctl get` and `fleetctl generate-gitops`, _only_ output the new names * The set of keys to replace is hardcoded in `fleetctl` rather than being dynamically generated as it is for API endpoints. Given the mixture of typed and untyped data and the level of nesting, dynamic map generation was very fragile and error-prone. ### Performance considerations * The biggest performance hit is the addition of the JSON key rewriter to the request pipeline. The rewriter buffers the entire request into memory before eventually passing it to the decoder than unmarshals the data into structs. I tried implementing this as a true streaming rewriter but encountered issues where the request would hang if the downstream reader (the decoder) encountered any errors. It's possible we could implement this in a streaming fashion if we replace our [current request decoder](https://github.com/fleetdm/fleet/blob/da43bf8371695382c4af0972d5da456c6b94bdaf/server/service/endpoint_utils.go#L108) with the v2 version, which is a bigger change requiring more thoughtful discussion in the engineering team. As it stands, memory usage for requests with deprecated fields will double while the request is being decoded. * The "alias rules" used to determine the old and new key names are cached per struct type and for most endpoints are generated on server start, so no performance impact is expected. * Some `fleetctl` commands may have an extra unmarshal/marshal step but as these are user-initiated and not performed in tight loops, the impact should be minimal. ### TODO * Log deprecation warnings when old URLs like "/fleet/teams" are used * Update API fields that the front-end uses to avoid deprecation warnings * Update `fleetctl apply` to accept/return `kind: fleet` rather than `kind: team` * Find/update any fleet server config vars with old language * Update all error messages that use old language # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) ## Testing - [X] Added/updated automated tests - [X] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [X] QA'd all new/changed functionality manually * Clicking around the front-end, no broken pages due to request ingestion errors or bad responses * Looking in network tab to verify that responses have both the old and new keys * Running `fleetctl generate-gitops` and verifying that the output looks correct and can be ingested by `fleetctl gitops` * Running `fleetctl get` and `fleetctl apply` --------- Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
2026-02-19 19:53:32 +00:00
package endpointer
import (
"testing"
"github.com/stretchr/testify/require"
"github.com/stretchr/testify/suite"
)
type extractAliasRulesSuite struct {
suite.Suite
}
func TestExtractAliasRules(t *testing.T) {
suite.Run(t, new(extractAliasRulesSuite))
}
// SetupTest runs before every Test* method, clearing the global cache.
func (s *extractAliasRulesSuite) SetupTest() {
aliasRulesCache.Range(func(key, _ any) bool {
aliasRulesCache.Delete(key)
return true
})
}
func (s *extractAliasRulesSuite) TestNilInput() {
rules := ExtractAliasRules(nil)
require.Nil(s.T(), rules)
}
func (s *extractAliasRulesSuite) TestNonStructInput() {
rules := ExtractAliasRules("hello")
require.Nil(s.T(), rules)
rules = ExtractAliasRules(42)
require.Nil(s.T(), rules)
}
func (s *extractAliasRulesSuite) TestStructWithNoRenametoTags() {
type plain struct {
Name string `json:"name"`
Age int `json:"age"`
}
rules := ExtractAliasRules(plain{})
require.Empty(s.T(), rules)
}
func (s *extractAliasRulesSuite) TestSingleRenametoTag() {
type singleAlias struct {
TeamID uint `json:"team_id" renameto:"group_id"`
}
rules := ExtractAliasRules(singleAlias{})
require.Equal(s.T(), []AliasRule{{OldKey: "team_id", NewKey: "group_id"}}, rules)
}
func (s *extractAliasRulesSuite) TestMultipleRenametoTags() {
type multiAlias struct {
TeamID uint `json:"team_id" renameto:"group_id"`
TeamName string `json:"team_name" renameto:"group_name"`
Other string `json:"other"`
}
rules := ExtractAliasRules(multiAlias{})
require.Equal(s.T(), []AliasRule{
{OldKey: "team_id", NewKey: "group_id"},
{OldKey: "team_name", NewKey: "group_name"},
}, rules)
}
func (s *extractAliasRulesSuite) TestJsonTagWithOmitempty() {
type omitemptyAlias struct {
TeamID uint `json:"team_id,omitempty" renameto:"group_id"`
}
rules := ExtractAliasRules(omitemptyAlias{})
require.Equal(s.T(), []AliasRule{{OldKey: "team_id", NewKey: "group_id"}}, rules)
}
func (s *extractAliasRulesSuite) TestJsonTagDashIsSkipped() {
type dashJSON struct {
Secret string `json:"-" renameto:"new_secret"`
}
rules := ExtractAliasRules(dashJSON{})
require.Empty(s.T(), rules)
}
func (s *extractAliasRulesSuite) TestNoJsonTagWithRenametoIsSkipped() {
type noJSON struct {
Field string `renameto:"new_field"`
}
rules := ExtractAliasRules(noJSON{})
require.Empty(s.T(), rules)
}
func (s *extractAliasRulesSuite) TestEmptyRenametoIsSkipped() {
type emptyRenameTo struct {
Field string `json:"field" renameto:""`
}
rules := ExtractAliasRules(emptyRenameTo{})
require.Empty(s.T(), rules)
}
func (s *extractAliasRulesSuite) TestNestedStruct() {
type inner struct {
InnerField string `json:"inner_field" renameto:"new_inner"`
}
type outer struct {
OuterField string `json:"outer_field" renameto:"new_outer"`
Nested inner
}
rules := ExtractAliasRules(outer{})
require.Equal(s.T(), []AliasRule{
{OldKey: "outer_field", NewKey: "new_outer"},
{OldKey: "inner_field", NewKey: "new_inner"},
}, rules)
}
func (s *extractAliasRulesSuite) TestDeeplyNestedStructs() {
type level2 struct {
Deep string `json:"deep" renameto:"new_deep"`
}
type level1 struct {
Mid string `json:"mid" renameto:"new_mid"`
Nested level2
}
type level0 struct {
Top string `json:"top" renameto:"new_top"`
Nested level1
}
rules := ExtractAliasRules(level0{})
require.Equal(s.T(), []AliasRule{
{OldKey: "top", NewKey: "new_top"},
{OldKey: "mid", NewKey: "new_mid"},
{OldKey: "deep", NewKey: "new_deep"},
}, rules)
}
func (s *extractAliasRulesSuite) TestDeduplicationAcrossNestedStructs() {
type shared struct {
TeamID uint `json:"team_id" renameto:"group_id"`
}
type parent struct {
TeamID uint `json:"team_id" renameto:"group_id"`
Child shared
}
rules := ExtractAliasRules(parent{})
// The same OldKey→NewKey pair should appear only once.
require.Equal(s.T(), []AliasRule{{OldKey: "team_id", NewKey: "group_id"}}, rules)
}
func (s *extractAliasRulesSuite) TestPointerToStructInput() {
type ptrInput struct {
Field string `json:"field" renameto:"new_field"`
}
rules := ExtractAliasRules(&ptrInput{})
require.Equal(s.T(), []AliasRule{{OldKey: "field", NewKey: "new_field"}}, rules)
}
func (s *extractAliasRulesSuite) TestNestedThroughPointerField() {
type pointed struct {
Inner string `json:"inner" renameto:"new_inner"`
}
type wrapper struct {
Ptr *pointed
}
rules := ExtractAliasRules(wrapper{})
require.Equal(s.T(), []AliasRule{{OldKey: "inner", NewKey: "new_inner"}}, rules)
}
func (s *extractAliasRulesSuite) TestNestedThroughSliceField() {
type elem struct {
Val string `json:"val" renameto:"new_val"`
}
type sliceWrapper struct {
Items []elem
}
rules := ExtractAliasRules(sliceWrapper{})
require.Equal(s.T(), []AliasRule{{OldKey: "val", NewKey: "new_val"}}, rules)
}
func (s *extractAliasRulesSuite) TestNestedThroughMapField() {
type mapVal struct {
Key string `json:"key" renameto:"new_key"`
}
type mapWrapper struct {
Data map[string]mapVal
}
rules := ExtractAliasRules(mapWrapper{})
require.Equal(s.T(), []AliasRule{{OldKey: "key", NewKey: "new_key"}}, rules)
}
func (s *extractAliasRulesSuite) TestDeduplicationSameStructViaMultiplePaths() {
type common struct {
ID string `json:"old_id" renameto:"new_id"`
}
type branch1 struct {
C common
}
type branch2 struct {
C common
}
type root struct {
B1 branch1
B2 branch2
}
rules := ExtractAliasRules(root{})
// common's rule should appear only once even though reachable via two paths.
require.Equal(s.T(), []AliasRule{{OldKey: "old_id", NewKey: "new_id"}}, rules)
}