mirror of
https://github.com/fleetdm/fleet
synced 2026-05-23 08:58:41 +00:00
BYO-Firehose module updates (#11333)
feat(firehose): add Terraform documentation to README.md feat(firehose): add Terraform module for IAM policy feat(firehose): add Terraform output for IAM policy ARN docs(byo-firehose-logging-destination): add introduction and explanation of IAM role and policy This commit adds an introduction and explanation of the IAM role and policy defined in the Terraform code. Specifically, it explains that the IAM role named `fleet_role` is being defined in the AWS account, and that it will be assumed by the Fleet application being hosted. The commit also explains that the IAM role is being given specific permissions to perform certain actions on the Firehose service, and that the associated IAM policy specifies the minimum allowed permissions. Additionally, the commit explains that the Firehose service is KMS encrypted, and that the IAM role needs permission to the KMS key being used to encrypt the data going into Firehose. Finally, the commit explains that the code sets up a secure and controlled environment for the Fleet application to perform its necessary actions on the Firehose service within the AWS account. refactor(byo-firehose-logging-destination): reformat table of resources and inputs feat(byo-firehose-logging-destination): add KMS key resource for firehose encryption feat(byo-firehose-logging-destination): add S3 bucket resource for logging destination feat(byo-firehose-logging-destination): add IAM policy and role resources for firehose feat(byo-firehose-logging-destination): add IAM policy attachment resource for fleet-firehose policy feat(byo-firehose-logging-destination): add data source for current AWS region feat(byo-firehose-logging-destination): add data source for KMS alias feat(byo-firehose-logging-destination): add data source for IAM policy documents feat(byo-firehose-logging-destination): add outputs for firehose IAM role, delivery streams, and S3 bucket fix(iam.tf): change aws_iam_policy and aws_iam_policy_attachment resource names to include fleet prefix closes https://github.com/fleetdm/fleet/issues/11331
This commit is contained in:
Benjamin Edwards
GitHub
parent
09e166d973
commit
1208c0151e
6 changed files with 97 additions and 59 deletions
|
|
@ -1 +0,0 @@
|
|||
header-from: .header.md
|
||||
|
|
@ -1,15 +1,16 @@
|
|||
# Logging Destination: Firehose
|
||||
This addon provides a Kinesis Firehose logging destination for Fleet with support for cross account S3 delivery.
|
||||
|
||||
<!-- BEGIN_TF_DOCS -->
|
||||
## Requirements
|
||||
|
||||
Apply module `target-account` to provision destination firehose, bucket, kms key, and IAM role/policies.
|
||||
| Name | Version |
|
||||
|------|---------|
|
||||
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.3.7 |
|
||||
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.52.0 |
|
||||
|
||||
## Providers
|
||||
|
||||
| Name | Version |
|
||||
|---------------------------------------------------|---------|
|
||||
| <a name="provider_aws"></a> [aws](#provider\_aws) | 4.52.0 |
|
||||
| Name | Version |
|
||||
|------|---------|
|
||||
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.52.0 |
|
||||
|
||||
## Modules
|
||||
|
||||
|
|
@ -17,19 +18,24 @@ No modules.
|
|||
|
||||
## Resources
|
||||
|
||||
No resources.
|
||||
| Name | Type |
|
||||
|------|------|
|
||||
| [aws_iam_policy.fleet-assume-role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
|
||||
| [aws_iam_policy_document.fleet-assume-role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
|
||||
| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
|
||||
|
||||
## Inputs
|
||||
|
||||
| Name | Description | Type | Default | Required |
|
||||
|-------------------------------|-----------------------------------------------------------|----------|---------------------|:--------:|
|
||||
| firehose_results_name | n/a | `string` | no default provided | yes |
|
||||
| firehose_status_name | n/a | `string` | no default provided | yes |
|
||||
| iam_role_arn | IAM Role used to write to target firehose delivery stream | `string` | no default provided | yes |
|
||||
|
||||
| Name | Description | Type | Default | Required |
|
||||
|------|-------------|------|---------|:--------:|
|
||||
| <a name="input_firehose_results_name"></a> [firehose\_results\_name](#input\_firehose\_results\_name) | name of the firehose delivery stream for osquery results logs | `string` | n/a | yes |
|
||||
| <a name="input_firehose_status_name"></a> [firehose\_status\_name](#input\_firehose\_status\_name) | name of the firehose delivery stream for osquery status logs | `string` | n/a | yes |
|
||||
| <a name="input_iam_role_arn"></a> [iam\_role\_arn](#input\_iam\_role\_arn) | IAM Role ARN to use for Firehose destination logging | `string` | n/a | yes |
|
||||
|
||||
## Outputs
|
||||
|
||||
| Name | Description |
|
||||
|-----------------------------------------------------------------------------------------------------------------|-------------|
|
||||
| <a name="output_fleet-extra-env-variables"></a> [fleet-extra-env-variables](#output\_fleet-extra-env-variables) | n/a |
|
||||
| Name | Description |
|
||||
|------|-------------|
|
||||
| <a name="output_fleet_extra_environment_variables"></a> [fleet\_extra\_environment\_variables](#output\_fleet\_extra\_environment\_variables) | n/a |
|
||||
| <a name="output_fleet_extra_iam_policies"></a> [fleet\_extra\_iam\_policies](#output\_fleet\_extra\_iam\_policies) | n/a |
|
||||
<!-- END_TF_DOCS -->
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
data "aws_iam_policy_document" "fleet-assume-role" {
|
||||
statement {
|
||||
effect = "Allow"
|
||||
actions = ["sts:AssumeRole"]
|
||||
resources = [var.iam_role_arn]
|
||||
}
|
||||
}
|
||||
|
||||
resource "aws_iam_policy" "fleet-assume-role" {
|
||||
policy = data.aws_iam_policy_document.fleet-assume-role.json
|
||||
}
|
||||
|
|
@ -8,4 +8,10 @@ output "fleet_extra_environment_variables" {
|
|||
FLEET_OSQUERY_STATUS_LOG_PLUGIN = "firehose"
|
||||
FLEET_OSQUERY_RESULT_LOG_PLUGIN = "firehose"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
output "fleet_extra_iam_policies" {
|
||||
value = [
|
||||
aws_iam_policy.fleet-assume-role.arn
|
||||
]
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,15 +1,25 @@
|
|||
# Logging Destination: S3
|
||||
This module will provision necessary resources to feed osquery results/status logs into S3.
|
||||
## Introduction
|
||||
|
||||
In this Terraform code, we are defining an IAM Role named `fleet_role` in our AWS Account, that will be assumed by the Fleet application we are hosting. We are only allowing this specific IAM Role (identified by its ARN) to perform certain actions on the Firehose service, such as `DescribeDeliveryStream`, `PutRecord`, and `PutRecordBatch`.
|
||||
|
||||
The reason we need a local IAM role in your account is so that we can assume role into it, and you have full control over the permissions it has. The associated IAM policy in the same file specifies the minimum allowed permissions.
|
||||
|
||||
The Firehose service is KMS encrypted, so the IAM Role we assume into needs permission to the KMS key that is being used to encrypt the data going into Firehose. Additionally, if the data is being delivered to S3, it will also be encrypted with KMS using the AWS S3 KMS key that is managed by AWS. This is because only customer managed keys can be shared across accounts, and the Firehose delivery stream is actually the one writing to S3.
|
||||
|
||||
Overall, this code sets up a secure and controlled environment for the Fleet application to perform its necessary actions on the Firehose service within your AWS Account.
|
||||
<!-- BEGIN_TF_DOCS -->
|
||||
## Requirements
|
||||
|
||||
None
|
||||
| Name | Version |
|
||||
|------|---------|
|
||||
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.3.7 |
|
||||
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.52.0 |
|
||||
|
||||
## Providers
|
||||
|
||||
| Name | Version |
|
||||
|---------------------------------------------------|---------|
|
||||
| <a name="provider_aws"></a> [aws](#provider\_aws) | 4.52.0 |
|
||||
| Name | Version |
|
||||
|------|---------|
|
||||
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.52.0 |
|
||||
|
||||
## Modules
|
||||
|
||||
|
|
@ -17,40 +27,46 @@ No modules.
|
|||
|
||||
## Resources
|
||||
|
||||
| Name | Type |
|
||||
|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------|
|
||||
| [aws_s3_bucket.osquery-destination](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) | resource |
|
||||
| [aws_s3_bucket_acl.osquery-destination](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_acl) | resource |
|
||||
| [aws_s3_bucket_public_access_block.osquery-destination](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_public_access_block) | resource |
|
||||
| [aws_s3_bucket_server_side_encryption_configuration.osquery-destination](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_server_side_encryption_configuration) | resource |
|
||||
| [aws_iam_policy.firehose](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
|
||||
| [aws_iam_role.fleet_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
|
||||
| [aws_iam_role.firehose](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
|
||||
| [aws_iam_role_policy_attachment.firehose](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
|
||||
| [aws_kinesis_firehose_delivery_stream.osquery_results](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kinesis_firehose_delivery_stream) | resource |
|
||||
| [aws_kinesis_firehose_delivery_stream.osquery_status](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kinesis_firehose_delivery_stream) | resource |
|
||||
| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
|
||||
| [aws_iam_policy_document.osquery_firehose_assume_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
|
||||
| [aws_iam_policy_document.osquery_results_policy_doc](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
|
||||
| [aws_iam_policy_document.osquery_status_policy_doc](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
|
||||
| Name | Type |
|
||||
|------|------|
|
||||
| [aws_iam_policy.firehose](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
|
||||
| [aws_iam_policy.fleet-firehose](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
|
||||
| [aws_iam_policy_attachment.fleet-firehose](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy_attachment) | resource |
|
||||
| [aws_iam_role.firehose](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
|
||||
| [aws_iam_role.fleet_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
|
||||
| [aws_iam_role_policy_attachment.firehose](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
|
||||
| [aws_kinesis_firehose_delivery_stream.osquery_results](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kinesis_firehose_delivery_stream) | resource |
|
||||
| [aws_kinesis_firehose_delivery_stream.osquery_status](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kinesis_firehose_delivery_stream) | resource |
|
||||
| [aws_kms_key.firehose](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_key) | resource |
|
||||
| [aws_s3_bucket.destination](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) | resource |
|
||||
| [aws_s3_bucket_acl.destination](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_acl) | resource |
|
||||
| [aws_s3_bucket_public_access_block.destination](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_public_access_block) | resource |
|
||||
| [aws_s3_bucket_server_side_encryption_configuration.destination](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_server_side_encryption_configuration) | resource |
|
||||
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
|
||||
| [aws_iam_policy_document.assume_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
|
||||
| [aws_iam_policy_document.firehose](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
|
||||
| [aws_iam_policy_document.firehose_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
|
||||
| [aws_iam_policy_document.osquery_firehose_assume_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
|
||||
| [aws_kms_alias.s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/kms_alias) | data source |
|
||||
| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
|
||||
|
||||
## Inputs
|
||||
|
||||
| Name | Description | Type | Default | Required |
|
||||
|-----------------------------------------|----------------------------------------------------------|----------|---------------------|:--------:|
|
||||
| osquery_logging_destination_bucket_name | name of the bucket for osquery logging | `string` | no default provided | yes |
|
||||
| firehose_results_name | name of the firehose delivery stream for results logging | `string` | `osquery_results` | no |
|
||||
| firehose_status_name | name of the firehose delivery stream for status logging | `string` | `osquery_status` | no |
|
||||
| results_prefix | s3 object prefix to give to results logs | `string` | `results/` | no |
|
||||
| status_prefix | s3 object prefix to give status logs | `string` | `status/` | no |
|
||||
| fleet_iam_role_arn | the role ARN from Fleet Cloud | `string` | no default provided | yes |
|
||||
|
||||
|
||||
| Name | Description | Type | Default | Required |
|
||||
|------|-------------|------|---------|:--------:|
|
||||
| <a name="input_firehose_results_name"></a> [firehose\_results\_name](#input\_firehose\_results\_name) | firehose delivery stream name for osquery results logs | `string` | `"osquery_results"` | no |
|
||||
| <a name="input_firehose_status_name"></a> [firehose\_status\_name](#input\_firehose\_status\_name) | firehose delivery stream name for osquery status logs | `string` | `"osquery_status"` | no |
|
||||
| <a name="input_fleet_iam_role_arn"></a> [fleet\_iam\_role\_arn](#input\_fleet\_iam\_role\_arn) | the arn of the fleet role that firehose will assume to write data to your bucket | `string` | n/a | yes |
|
||||
| <a name="input_osquery_logging_destination_bucket_name"></a> [osquery\_logging\_destination\_bucket\_name](#input\_osquery\_logging\_destination\_bucket\_name) | name of the bucket to store osquery results & status logs | `string` | n/a | yes |
|
||||
| <a name="input_results_prefix"></a> [results\_prefix](#input\_results\_prefix) | s3 object prefix to give to results logs | `string` | `"results/"` | no |
|
||||
| <a name="input_status_prefix"></a> [status\_prefix](#input\_status\_prefix) | s3 object prefix to give status logs | `string` | `"status/"` | no |
|
||||
|
||||
## Outputs
|
||||
|
||||
| Name | Description |
|
||||
|-------------------|---------------------------------------------------------------------------------|
|
||||
| firehose_iam_role | IAM Role ARN fleet cloud will assume to write data to firehose delivery streams |
|
||||
| firehose_results | name of the firehose delivery stream for results logs |
|
||||
| firehose_status | name of the firehose delivery stream for status logs |
|
||||
| Name | Description |
|
||||
|------|-------------|
|
||||
| <a name="output_firehose_iam_role"></a> [firehose\_iam\_role](#output\_firehose\_iam\_role) | n/a |
|
||||
| <a name="output_firehose_results"></a> [firehose\_results](#output\_firehose\_results) | n/a |
|
||||
| <a name="output_firehose_status"></a> [firehose\_status](#output\_firehose\_status) | n/a |
|
||||
| <a name="output_s3_destination"></a> [s3\_destination](#output\_s3\_destination) | n/a |
|
||||
<!-- END_TF_DOCS -->
|
||||
|
|
@ -35,11 +35,11 @@ data "aws_iam_policy_document" "firehose" {
|
|||
|
||||
}
|
||||
|
||||
resource "aws_iam_policy" "firehose" {
|
||||
resource "aws_iam_policy" "fleet-firehose" {
|
||||
policy = data.aws_iam_policy_document.firehose.json
|
||||
}
|
||||
|
||||
resource "aws_iam_policy_attachment" "firehose" {
|
||||
resource "aws_iam_policy_attachment" "fleet-firehose" {
|
||||
name = aws_iam_role.fleet_role.name
|
||||
policy_arn = aws_iam_policy.firehose.arn
|
||||
policy_arn = aws_iam_policy.fleet-firehose.arn
|
||||
}
|
||||
Loading…
Reference in a new issue