diff options
Diffstat (limited to 'content/blog/aws')
| -rw-r--r-- | content/blog/aws/ansible-fact-metadata.md | 88 | ||||
| -rw-r--r-- | content/blog/aws/defaults.md | 254 | ||||
| -rw-r--r-- | content/blog/aws/secrets.md | 136 |
3 files changed, 478 insertions, 0 deletions
diff --git a/content/blog/aws/ansible-fact-metadata.md b/content/blog/aws/ansible-fact-metadata.md new file mode 100644 index 0000000..3c48f1c --- /dev/null +++ b/content/blog/aws/ansible-fact-metadata.md @@ -0,0 +1,88 @@ +--- +title: 'Shell script for gathering imdsv2 instance metadata on AWS ec2' +description: 'An ansible fact I wrote' +date: '2024-10-12' +tags: +- ansible +- aws +--- + +## Introduction + +I wrote a shell script to gather ec2 instance metadata with an ansible fact. + +## The script + +I am using POSIX `/bin/sh` because I wanted to support a variety of operating systems. Besides that, the only dependency is `curl`: + +``` shell +#!/bin/sh +set -eu + +metadata() { + local METHOD=$1 + local URI_PATH=$2 + local TOKEN="${3:-}" + local HEADER + if [ -z "${TOKEN}" ]; then + HEADER='X-aws-ec2-metadata-token-ttl-seconds: 21600' # request a 6 hours token + else + HEADER="X-aws-ec2-metadata-token: ${METADATA_TOKEN}" + fi + curl -sSfL --request "${METHOD}" \ + "http://169.254.169.254/latest${URI_PATH}" \ + --header "${HEADER}" +} + +METADATA_TOKEN=$(metadata PUT /api/token) +KEYS=$(metadata GET /meta-data/tags/instance "${METADATA_TOKEN}") +PREFIX='{' +for KEY in $KEYS; do + VALUE=$(metadata GET "/meta-data/tags/instance/${KEY}" "${METADATA_TOKEN}") + printf '%s"%s":"%s"' "${PREFIX}" "${KEY}" "${VALUE}" + PREFIX=',' +done +printf '}' +``` + +## Bonus version without depending on curl + +Depending on curl can be avoided. If you are willing to use netcat instead and be declared a madman by your colleagues, you can rewrite the function with: + +``` shell +metadata() { + local METHOD=$1 + local URI_PATH=$2 + local TOKEN="${3:-}" + local HEADER + if [ -z "${TOKEN}" ]; then + HEADER='X-aws-ec2-metadata-token-ttl-seconds: 21600' # request a 6 hours token + else + HEADER="X-aws-ec2-metadata-token: ${METADATA_TOKEN}" + fi + printf "${METHOD} /latest${URI_PATH} HTTP/1.0\r\n%s\r\n\r\n" \ + "${HEADER}" \ + | nc -w 5 169.254.169.254 80 | tail -n 1 +} +``` + +## Deploying an ansible fact + +I deploy the script this way: +``` yaml +- name: 'Deploy ec2 metadata fact gathering script' + copy: + src: 'ec2_metadata.sh' + dest: '/etc/ansible/facts.d/ec2_metadata.fact' + owner: 'root' + mode: '0500' + register: 'ec2_metadata_fact' + +- name: 'reload facts' + setup: 'filter=ansible_local' + when: 'ec2_metadata_fact.changed' +``` + +## Conclusion + +It works, is simple and I like it. I am happy! diff --git a/content/blog/aws/defaults.md b/content/blog/aws/defaults.md new file mode 100644 index 0000000..454b325 --- /dev/null +++ b/content/blog/aws/defaults.md @@ -0,0 +1,254 @@ +--- +title: Securing AWS default VPCs +description: With terraform/OpenTofu +date: 2024-09-10 +tags: +- aws +- OpenTofu +- terraform +--- + +## Introduction + +AWS offers some network conveniences in the form of a default VPC, default security group (allowing access to the internet) and default routing table. These exist in all AWS regions your accounts have access to, even if never plan to deploy anything there. And yes most AWS regions cannot be disabled entirely, only the most recent ones can be. + +I feel the need to clean up these resources in order to prevent any misuse. Most people do not understand networking and some could inadvertently spawn instances with public IP addresses. By making the default VPC inoperative, these people need to come to someone more knowledgeable before they do anything foolish. + +## Module + +The special default variants of the following AWS terraform resources are quirky: defining them does not create anything but automatically import the built-in aws resources and then edit their attributes to match your configuration. Furthermore, destroying these resources would only remove them from your state. + +``` hcl +resource "aws_default_vpc" "default" { + tags = { Name = "default" } +} + +resource "aws_default_security_group" "default" { + ingress = [] + egress = [] + tags = { Name = "default" } + vpc_id = aws_default_vpc.default.id +} + +resource "aws_default_route_table" "default" { + default_route_table_id = aws_default_vpc.default.default_route_table_id + route = [] + tags = { Name = "default - empty" } +} +``` + +The key here (and initial motivation for this article) is the `ingress = []` expression syntax (or `egress` or `route`): while these attributes are normally block attributes, you can also use them in a `= []` expression in order to express that you want to enforce the resource not having any ingress, egress or route rules. Defining the resources without any block rules would just leave these attributes untouched. + +## Iterating through all the default regions + +As I said, most AWS regions cannot be disabled entirely, only the most recent ones can be. It is currently not possible to instanciate terraform providers on the fly, but thankfully it is coming in a future OpenTofu release! In the meantime, we need to do these kinds of horrors: + +``` hcl +provider "aws" { + alias = "ap-northeast-1" + profile = var.environment + region = "ap-northeast-1" + default_tags { tags = { "managed-by" = "tofu" } } +} + +provider "aws" { + alias = "ap-northeast-2" + profile = var.environment + region = "ap-northeast-2" + default_tags { tags = { "managed-by" = "tofu" } } +} + +provider "aws" { + alias = "ap-northeast-3" + profile = var.environment + region = "ap-northeast-3" + default_tags { tags = { "managed-by" = "tofu" } } +} + +provider "aws" { + alias = "ap-south-1" + profile = var.environment + region = "ap-south-1" + default_tags { tags = { "managed-by" = "tofu" } } +} + +provider "aws" { + alias = "ap-southeast-1" + profile = var.environment + region = "ap-southeast-1" + default_tags { tags = { "managed-by" = "tofu" } } +} + +provider "aws" { + alias = "ap-southeast-2" + profile = var.environment + region = "ap-southeast-2" + default_tags { tags = { "managed-by" = "tofu" } } +} + +provider "aws" { + alias = "ca-central-1" + profile = var.environment + region = "ca-central-1" + default_tags { tags = { "managed-by" = "tofu" } } +} + +provider "aws" { + alias = "eu-central-1" + profile = var.environment + region = "eu-central-1" + default_tags { tags = { "managed-by" = "tofu" } } +} + +provider "aws" { + alias = "eu-north-1" + profile = var.environment + region = "eu-north-1" + default_tags { tags = { "managed-by" = "tofu" } } +} + +provider "aws" { + alias = "eu-west-1" + profile = var.environment + region = "eu-west-1" + default_tags { tags = { "managed-by" = "tofu" } } +} + +provider "aws" { + alias = "eu-west-2" + profile = var.environment + region = "eu-west-2" + default_tags { tags = { "managed-by" = "tofu" } } +} + +provider "aws" { + alias = "eu-west-3" + profile = var.environment + region = "eu-west-3" + default_tags { tags = { "managed-by" = "tofu" } } +} + +provider "aws" { + alias = "sa-east-1" + profile = var.environment + region = "sa-east-1" + default_tags { tags = { "managed-by" = "tofu" } } +} + +provider "aws" { + alias = "us-east-1" + profile = var.environment + region = "us-east-1" + default_tags { tags = { "managed-by" = "tofu" } } +} + +provider "aws" { + alias = "us-east-2" + profile = var.environment + region = "us-east-2" + default_tags { tags = { "managed-by" = "tofu" } } +} + +provider "aws" { + alias = "us-west-1" + profile = var.environment + region = "us-west-1" + default_tags { tags = { "managed-by" = "tofu" } } +} + +provider "aws" { + alias = "us-west-2" + profile = var.environment + region = "us-west-2" + default_tags { tags = { "managed-by" = "tofu" } } +} + +module "ap-northeast-1" { + providers = { aws = aws.ap-northeast-1 } + source = "../modules/defaults" +} + +module "ap-northeast-2" { + providers = { aws = aws.ap-northeast-2 } + source = "../modules/defaults" +} + +module "ap-northeast-3" { + providers = { aws = aws.ap-northeast-3 } + source = "../modules/defaults" +} + +module "ap-south-1" { + providers = { aws = aws.ap-south-1 } + source = "../modules/defaults" +} + +module "ap-southeast-1" { + providers = { aws = aws.ap-southeast-1 } + source = "../modules/defaults" +} + +module "ap-southeast-2" { + providers = { aws = aws.ap-southeast-2 } + source = "../modules/defaults" +} + +module "ca-central-1" { + providers = { aws = aws.ca-central-1 } + source = "../modules/defaults" +} + +module "eu-central-1" { + providers = { aws = aws.eu-central-1 } + source = "../modules/defaults" +} + +module "eu-north-1" { + providers = { aws = aws.eu-north-1 } + source = "../modules/defaults" +} + +module "eu-west-1" { + providers = { aws = aws.eu-west-1 } + source = "../modules/defaults" +} + +module "eu-west-2" { + providers = { aws = aws.eu-west-2 } + source = "../modules/defaults" +} + +module "eu-west-3" { + providers = { aws = aws.eu-west-3 } + source = "../modules/defaults" +} + +module "sa-east-1" { + providers = { aws = aws.sa-east-1 } + source = "../modules/defaults" +} + +module "us-east-1" { + providers = { aws = aws.us-east-1 } + source = "../modules/defaults" +} + +module "us-east-2" { + providers = { aws = aws.us-east-2 } + source = "../modules/defaults" +} + +module "us-west-1" { + providers = { aws = aws.us-west-1 } + source = "../modules/defaults" +} + +module "us-west-2" { + providers = { aws = aws.us-west-2 } + source = "../modules/defaults" +} +``` + +## Conclusion + +Terraform is absolutely quirky at times, but it is not its fault here: the AWS provider and their magical default resources are. diff --git a/content/blog/aws/secrets.md b/content/blog/aws/secrets.md new file mode 100644 index 0000000..a25f9ef --- /dev/null +++ b/content/blog/aws/secrets.md @@ -0,0 +1,136 @@ +--- +title: Managing AWS secrets +description: with the CLI and with terraform/OpenTofu +date: 2024-08-13 +tags: +- aws +- OpenTofu +- terraform +--- + +## Introduction + +Managing secrets in AWS is not an everyday task that allows me to naturally remember the specifics when I need them, especially the `--name` and `--secret-id` CLI inconsistency. I found I was lacking some simple notes that would prevent me from having to search the web in the future, here they are. + +## CLI + +### Creating secrets + +From a simple string: + +``` shell +aws --profile common secretsmanager create-secret \ + --name test-string \ + --secret-string 'test' +``` + +From a text file: + +``` shell +aws --profile common secretsmanager create-secret \ + --name test-text \ + --secret-string "$(cat ~/Downloads/adyxax.2024-07-31.private-key.pem)" +``` + +For binary file we `base64` encode the data: + +``` shell +aws --profile common secretsmanager create-secret \ + --name test-binary \ + --secret-binary "$(cat ~/Downloads/some-blob|base64)" +``` + +### Updating secrets + +Beware that all the other aws secretsmanager commands use the `--secret-id` flag instead of the `--name` we needed when creating the secret. + +Update a secret string with: + +``` shell +aws --profile common secretsmanager update-secret \ + --secret-id test-string \ + --secret-string 'test' +``` + +### Reading secrets + +Listing: + +``` shell +aws --profile common secretsmanager list-secrets | jq -r '[.SecretList[].Name]' +``` + +Getting a secret value: + +``` shell +aws --profile common secretsmanager get-secret-value --secret-id test-string +``` + +### Deleting secrets + +``` shell +aws --profile common secretsmanager delete-secret --secret-id test-string +``` + +## Terraform + +### Resource + +Secret string: + +``` hcl +resource "random_password" "main" { + length = 64 + special = false + lifecycle { + ignore_changes = [special] + } +} + +resource "aws_secretsmanager_secret" "main" { + name = "grafana-admin-password" +} + +resource "aws_secretsmanager_secret_version" "main" { + secret_id = aws_secretsmanager_secret.main.id + secret_string = random_password.main.result +} +``` + +Secret binary: + +``` hcl +resource "random_bytes" "main" { + length = 32 +} + +resource "aws_secretsmanager_secret" "main" { + name = "data-encryption-key" +} + +resource "aws_secretsmanager_secret_version" "main" { + secret_id = aws_secretsmanager_secret.main.id + secret_binary = random_bytes.main.base64 +} +``` + +### Datasource + +``` hcl +data "aws_secretsmanager_secret_version" "main" { + secret_id = "test" +} +``` + +Using the datasource differs if it contains a `secret_string` or a `secret_binary`. In most cases you will know your secret data therefore know which one to use. If for some reason you do not, this might be one of the rare legitimate use cases for the [try function](https://developer.hashicorp.com/terraform/language/functions/try): + +``` hcl +try( + data.aws_secretsmanager_secret_version.main.secret_binary, + data.aws_secretsmanager_secret_version.main.secret_string, +) +``` + +## Conclusion + +Once upon a time I wrote many small and short articles like this one but for some reason stopped. I will try to take on this habit again. |