Update: See the follow up Installing kube2iam in AWS Kubernetes EKS Cluster
kube2iam allows a Kubernetes cluster in AWS to use different IAM roles for each pod, and prevents pods from accessing EC2 instance IAM roles.
Installation
Edit your kops cluster with kops edit cluster
to allow nodes to assume different roles, changing the account id 123456789012
to yours
spec: additionalPolicies: nodes: | [ { "Effect": "Allow", "Action": [ "sts:AssumeRole" ], "Resource": [ "arn:aws:iam::123456789012:role/k8s-*" ] } ]
Install kube2iam using the helm chart
helm install stable/kube2iam --namespace kube-system --name my-release \ --set=extraArgs.base-role-arn=arn:aws:iam::123456789012:role/,extraArgs.default-role=kube2iam-default,host.iptables=true,host.interface=cbr0,rbac.create=true
A curl to the metadata server from a new pod should return kube2iam
$ curl http://169.254.169.254/latest/meta-data/iam/security-credentials/ kube2iam
Role configuration
Create the roles that the pods can assume. They must start with k8s-
(see the wildcard we set in the Resource
above) and contain a trust relationship to the node pool role.
For instance, to allow access to the S3 bucket mybucket
from a pod, create a role k8s-s3
.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "s3bucketActions", "Effect": "Allow", "Action": "s3:*", "Resource": "arn:aws:s3:::mybucket", } ] }
Then edit the trust relationship of the role to allow the node role (the role created by Kops for the Auto Scaling Goup) to assume this role.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "", "Effect": "Allow", "Principal": { "Service": "ec2.amazonaws.com" }, "Action": "sts:AssumeRole" }, { "Sid": "", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::123456789012:role/nodes.example.com" }, "Action": "sts:AssumeRole" } ] }
Test it by launching a pod with the right annotations
apiVersion: v1 kind: Pod metadata: name: aws-cli labels: name: aws-cli annotations: iam.amazonaws.com/role: arn:aws:iam::123456789012:role/k8s-s3 spec: containers: - image: fstab/aws-cli command: - "/home/aws/aws/env/bin/aws" - "s3" - "ls" - "some-bucket" name: aws-cli
Securing namespaces
kube2iam supports namespace restrictions so users can still launch pods but are limited to a predefined set of IAM roles that can assume.
apiVersion: v1 kind: Namespace metadata: annotations: iam.amazonaws.com/allowed-roles: | ["my-custom-path/*"] name: default
Why are you not using ECS and ALB to manage the load?
wdym ? kube2iam doesn’t have anything to do with load
Pingback: Sending Kubernetes Logs to CloudWatch Logs using Fluentd | Carlos Sanchez's Weblog
“A similar setup should also work with AWS EKS.”
Carlos, where you able to get it working in EKS? I have not succeeded and would like to see if you were able to get it to work.
I have not yet tried 😦
It should work. Maybe this workshop can help you out: https://github.com/aws-samples/aws-workshop-for-kubernetes/blob/master/04-path-security-and-networking/402-authentication-and-authorization/readme.adoc#iam-container-roles-using-kube2iam
I believe one important part of the setup is that trust relationship that allows Node to assume roles. In EKS, all nodes should have a Node Role and it should be added to the trust policy of the role you want to assume via kube2iam.
Note: There is a bug if you are using Calico as a Network manager. Workaround can be found at https://github.com/jtblin/kube2iam/issues/14
Pingback: Installing kube2iam in AWS Kubernetes EKS Cluster | Carlos Sanchez's Weblog
Hi! this tutorial was really helpful thanks you 🙂 I found one typo in the first snippet it should be node instead of nodes in the additionalPolicies configuration.
Original snippet
spec:
additionalPolicies:
nodes: |
[
{
“Effect”: “Allow”,
“Action”: [
“sts:AssumeRole”
],
“Resource”: [
“arn:aws:iam::123456789012:role/k8s-*”
]
}
]
Should be:
spec:
additionalPolicies:
node: |
[
{
“Effect”: “Allow”,
“Action”: [
“sts:AssumeRole”
],
“Resource”: [
“arn:aws:iam::123456789012:role/k8s-*”
]
}
]
Thank you, that was succinct and very helpful!
https://stackoverflow.com/questions/68749772/upgrading-kube2iam-to-10-11-is-throwing-error-getting-instance-id