EKS Bootstrap

SquareOps Technologies Your DevOps Partner for Accelerating cloud journey.

This module provides a set of reusable, configurable, and scalable AWS EKS bootstrap configurations. It enables users to easily deploy and manage a highly available EKS cluster using infrastructure as code. This module supports a wide range of features, including node termination handlers, VPC CNI add-ons, service monitors, Istio service meshes, Velero backups, and Karpenter provisioners. Users can configure these features using a set of customizable variables that allow for fine-grained control over the deployment. Additionally, this module is regularly updated to keep pace with the latest changes in the EKS ecosystem, ensuring that users always have access to the most up-to-date features and functionality.

Usage Example

module "eks_bootstrap" {
  source                        = "squareops/eks-bootstrap/aws"
  name                          = "skaf"
  vpc_id                        = "vpc-06e37f0786b7eskaf"
  environment                   = "production"
  ipv6_enabled                  = true
  kms_key_arn                   = "arn:aws:kms:region:222222222222:key/kms_key_arn"
  keda_enabled                  = true
  kms_policy_arn                = "arn:aws:iam::222222222222:policy/kms_policy_arn" ## eks module will create kms_policy_arn
  eks_cluster_name              = "cluster_name"
  reloader_enabled              = true
  karpenter_enabled             = true
  private_subnet_ids            = [""]
  single_az_sc_config           = [{ name = "infra-service-sc", zone = "zone-name" }]
  kubeclarity_enabled           = false
  kubeclarity_hostname          = ""
  kubecost_enabled              = false
  kubecost_hostname             = ""
  cert_manager_enabled          = true
  worker_iam_role_name          = "worker_iam_role_name"
  ingress_nginx_enabled         = true
  metrics_server_enabled        = true
  external_secrets_enabled      = true
  amazon_eks_vpc_cni_enabled    = true
  cluster_autoscaler_enabled    = true
  service_monitor_crd_enabled   = true
  istio_enabled                 = false
  istio_config = {
    ingress_gateway_enabled             = true
    ingress_gateway_namespace           = "istio-ingressgateway"
    egress_gateway_enabled              = true
    egress_gateway_namespace            = "istio-egressgateway"
    observability_enabled               = true
    envoy_access_logs_enabled           = true
    prometheus_monitoring_enabled       = true
    cert_manager_cluster_issuer_enabled = true
  }
  karpenter_provisioner_enabled = true
  karpenter_provisioner_config = {
    private_subnet_name    = "private_subnet_name"
    instance_capacity_type = ["spot"]
    excluded_instance_type = ["nano", "micro", "small"]
    instance_hypervisor    = ["nitro"]   ## Instance hypervisor is picked up only if IPv6 enable is chosen
  }
  cert_manager_letsencrypt_email                = "email@example.com"
  internal_ingress_nginx_enabled                = true
  efs_storage_class_enabled                     = true
  aws_node_termination_handler_enabled          = true
  amazon_eks_aws_ebs_csi_driver_enabled         = true
  cluster_propotional_autoscaler_enabled        = true
  single_az_ebs_gp3_storage_class_enabled       = true
  cert_manager_install_letsencrypt_http_issuers = true
  velero_enabled                                = false
  velero_config = {
    namespaces                      = "my-application" ## If you want full cluster backup, leave it blank else provide namespace.
    slack_notification_token        = "xoxb-slack-token"
    slack_notification_channel_name = "slack-notifications-channel"
    retention_period_in_days        = 45
    schedule_backup_cron_time       = "* 6 * * *"
    velero_backup_name              = "my-application-backup"
    backup_bucket_name              = "velero-cluster-backup"
  }
}

Compatibility

Release	Kubernetes 1.23	Kubernetes 1.24	Kubernetes 1.25	Kubernetes 1.26
Release 1.0.0	✔	✗	✗	✗
Release 1.0.1	✔	✔	✔	✗
Release 1.1.0	✔	✔	✔	✗
Release 2.0.0	✔	✔	✔	✗
Release 2.1.0	✔	✔	✔	✗
Release 3.0.0	✔	✔	✔	✔
Release 3.1.0	✔	✔	✔	✔

IAM Permissions

The required IAM permissions to create resources from this module can be found here

Addons Details

Kubernetes addons are additional components that can be installed in a Kubernetes cluster to provide extra features and functionality. They are designed to work seamlessly with the Kubernetes API and can be managed just like any other Kubernetes resource. Some common examples of Kubernetes addons include:

AWS ALB

Amazon Web Services (AWS) Application Load Balancer (ALB) is a highly available and scalable load balancing service that routes incoming application traffic to multiple Amazon EC2 instances, containers, and IP addresses. It automatically distributes incoming application traffic across multiple targets, ensuring that your applications are highly available and scalable.

With AWS ALB, you can handle increased traffic levels, automatically scale your applications, and improve the overall performance of your applications. ALB provides advanced routing capabilities, including content-based routing, host-based routing, and path-based routing, enabling you to route traffic to different target groups based on specific rules.

Node Termination Handler

In AWS, the Node termination handler can be used in Lambda functions or EC2 instances to handle the termination of the underlying instance or container. When an instance or container is terminated, the termination handler can be used to perform any necessary cleanup operations, such as closing open resources, before the instance or container is terminated. In an EC2 instance, the termination handler can be set by writing a script that runs on instance startup and sets the process.on('SIGTERM', callback) method. This script can be executed using a user data script or by adding it to the instance's startup script.

EBS

Amazon Elastic Block Store (Amazon EBS) storage classes are different levels of performance and cost for Amazon EBS volumes. The storage classes determine the type of storage, performance characteristics, and cost of each Amazon EBS volume.

There are currently four Amazon EBS storage classes:

Standard storage class: This is the default and most widely used storage class, offering a balance of low cost and high performance. It's suitable for a wide range of applications, including boot volumes, transactional databases, and big data workloads.

Provisioned IOPS (input/output operations per second) storage class: This class provides high-performance I/O for mission-critical and I/O-intensive workloads, such as large databases and I/O-bound applications.

Cold storage class: This class provides low-cost storage for infrequently accessed data, such as backups and archives. Cold storage is designed to deliver low cost and high durability.

Throughput Optimized HDD (hard disk drive) storage class: This class provides low-cost storage optimized for large, sequential workloads, such as big data and data warehouses.

Cert Manager

Cert Manager is a Kubernetes add-on that automates the management and issuance of TLS certificates from various issuing sources. It helps to eliminate manual steps in the certificate management process, provides cert renewals and integrates with other parts of the system.

Cluster Autoscalar

Cluster Autoscaler is a Kubernetes component that automatically adjusts the number of nodes in a cluster based on the demand for resources. This allows you to optimize the cost of running your workloads while ensuring that they have the resources they need to run effectively. The Cluster Autoscaler works by monitoring the resource usage of your pods and comparing it to the available capacity on the nodes in the cluster. If there are pods that cannot be scheduled because of resource constraints, the Cluster Autoscaler will increase the number of nodes in the cluster until there is enough capacity to schedule the pending pods. Similarly, if there are nodes in the cluster that are underutilized, the Cluster Autoscaler can decrease the number of nodes to optimize resource utilization and reduce costs. Cluster Autoscaler is supported by many cloud providers, including Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure. It can be easily integrated into your existing Kubernetes deployment and can be configured to use different scaling policies to meet the needs of your specific workloads.

EFS

Amazon Elastic File System (Amazon EFS) is a fully managed, scalable, and highly available file storage service for use with Amazon Elastic Compute Cloud (Amazon EC2) instances. It provides a simple and scalable file storage solution that can be used by multiple EC2 instances at the same time, making it ideal for use cases such as big data, content management, and media sharing.

Amazon EFS is easy to set up, manage, and scale, and it automatically replicates data across multiple Availability Zones for high durability and availability. The service is also highly performant, with low latency and high throughput, making it suitable for a wide range of workloads.

External Secrets

Kubernetes External Secrets is a feature in Kubernetes that allows secrets to be stored and managed outside of the cluster. External secrets are useful in scenarios where sensitive information, such as passwords or API keys, should not be stored directly in the cluster, but still needs to be used by applications running in the cluster. Kubernetes External Secrets can be stored in external systems such as Hashicorp Vault, AWS Secrets Manager, or GCP Secret Manager, and accessed by pods using a Kubernetes Secret object. The Secret object references the external secret and maps it to a Kubernetes Secret, which can then be used by pods in the same way as regular Kubernetes Secrets. By using External Secrets, organizations can ensure that sensitive information is securely managed and stored outside of the cluster, while still being able to use that information in their applications running in the cluster.

Istio

Istio is an open-source service mesh platform that provides a set of tools for managing and securing microservices applications. Istio is designed to work with containerized applications and is built on top of Kubernetes, making it easy to deploy and manage. Some of the key features of Istio include: Traffic management: Istio provides the ability to control the flow of traffic between microservices, including load balancing, fault tolerance, and canary releases. Security: Istio provides built-in security features, such as mutual TLS authentication, for securing communication between microservices. Observability: Istio provides robust observability features, including distributed tracing, metric collection, and logging, making it easy to monitor and debug microservices applications. Configurable policies: Istio provides a flexible policy framework for controlling the behavior of microservices, allowing for easy enforcement of security, observability, and traffic management policies. Istio is widely adopted and has a strong ecosystem of partners and contributors, making it a popular choice for organizations looking to build and manage microservices applications. By using Istio, organizations can improve the reliability and security of their microservices applications and simplify the process of managing and operating them.

Karpenter

Karpenter is a flexible, high-performance Kubernetes cluster autoscaler that helps improve application availability and cluster efficiency. Karpenter launches right-sized compute resources, (for example, Amazon EC2 instances), in response to changing application load in under a minute. Through integrating Kubernetes with AWS, Karpenter can provision just-in-time compute resources that precisely meet the requirements of your workload. Karpenter automatically provisions new compute resources based on the specific requirements of cluster workloads. These include compute, storage, acceleration, and scheduling requirements. Amazon EKS supports clusters using Karpenter, although Karpenter works with any conformant Kubernetes cluster.

Metrics Server

Metric Server is a Kubernetes add-on that collects resource usage data from the Kubernetes API server and makes it available to other components, such as the Horizontal Pod Autoscaler (HPA) and the Cluster Autoscaler. The Metric Server collects data on the CPU and memory usage of pods and nodes in a cluster, and provides this data to other components in a format that they can use to make scaling decisions. The HPA, for example, can use the data provided by the Metric Server to automatically scale the number of replicas of a deployment based on the resource usage of the pods. The Cluster Autoscaler can also use this data to determine when to add or remove nodes from a cluster based on the resource utilization of the pods and nodes. Metric Server provides a simple and effective way to collect resource usage data from a cluster and make it available to other components for scaling and resource optimization. It is an important component for ensuring that your Kubernetes applications run effectively and efficiently in the cloud.

Nginx Ingress Controller

Nginx Ingress Controller is a Kubernetes controller that manages external access to services running in a Kubernetes cluster. It provides load balancing, SSL termination, and name-based virtual hosting, among other features. The Nginx Ingress Controller works by using the Kubernetes API to dynamically configure an Nginx instance running outside of the cluster to route traffic to services within the cluster. This allows you to easily expose your services to external users and manage the routing of incoming traffic. The Nginx Ingress Controller provides a flexible and powerful way to manage incoming traffic to your Kubernetes applications. It is widely used in production environments and is well-suited for both simple and complex routing scenarios. Additionally, the Nginx Ingress Controller integrates with other Kubernetes components, such as the Kubernetes Ingress resource and cert-manager, to provide a complete solution for managing external access to your services.

Reloader

Reloader can watch changes in ConfigMap and Secret and do rolling upgrades on Pods with their associated DeploymentConfigs, Deployments, Daemonsets, Statefulsets and Rollouts.

Velero

Velero (previously known as Heptio Ark) is an open-source backup and disaster recovery solution for Kubernetes. Velero provides the ability to back up and restore Kubernetes cluster resources and persistent volumes, making it easy to recover from data loss or cluster failures. Some key features of Velero include: Cluster Backup and Restore: Velero allows users to create backups of their Kubernetes clusters and restore them to the same or different cluster. Persistent Volume Backup and Restore: Velero provides the ability to backup and restore persistent volumes, ensuring that data can be recovered even if the cluster fails. Incremental Backups: Velero supports incremental backups, which can be performed more frequently than full backups and reduce the amount of data transferred. Snapshot Integration: Velero integrates with cloud provider snapshot services, such as AWS EBS and GCE PD, to simplify the backup process and reduce the cost of storing backup data. Easy to Use CLI: Velero provides a user-friendly CLI that makes it easy to create, manage, and restore backups. Velero is designed to work with cloud native environments, making it a popular choice for organizations that run their applications in the cloud. By using Velero, organizations can improve the reliability and availability of their applications and ensure that they can recover from data loss or cluster failures.

KubeClarity

KubeClarity helps you to secure your cloud-native applications and infrastructure by offering features such as automated threat detection, policy enforcement, compliance reporting, and continuous monitoring. It allows you to enforce security policies across all your Kubernetes environments and provides automated remediation of security issues, ensuring that your deployments are always secure and compliant.

Kubecost

Kubecost provides real-time cost visibility and insights for teams using Kubernetes, helping you continuously reduce your cloud costs. Breakdown costs by any Kubernetes concepts, including deployment, service, namespace label, and more.

Notes

Before enabling the Kubecost addon for your Amazon EKS cluster, please make sure to subscribe to the Kubecost - Amazon EKS cost monitoring license.

Requirements

Name	Version
aws	>= 4.23
helm	>= 2.6
kubernetes	>= 2.13

Providers

Name	Version
aws	>= 4.23
helm	>= 2.6
kubernetes	>= 2.13
random	n/a

Modules

Name	Source	Version
efs	./addons/efs	n/a
istio	./addons/istio	n/a
k8s_addons	./EKS-Blueprint/modules/kubernetes-addons	n/a
karpenter_provisioner	./addons/karpenter_provisioner	n/a
service_monitor_crd	./addons/service_monitor_crd	n/a
single_az_sc	./addons/aws-ebs-storage-class	n/a
velero	./addons/velero	n/a

Resources

Name	Type
aws_eks_addon.kubecost	resource
aws_iam_instance_profile.karpenter_profile	resource
helm_release.cert_manager_le_http	resource
helm_release.coredns-hpa	resource
helm_release.internal_nginx	resource
helm_release.kubeclarity	resource
helm_release.metrics-server-vpa	resource
helm_release.vpa-crds	resource
kubernetes_ingress_v1.kubecost	resource
kubernetes_namespace.internal_nginx	resource
kubernetes_namespace.kube_clarity	resource
kubernetes_secret.kube_clarity	resource
kubernetes_secret.kubecost	resource
random_password.kube_clarity	resource
random_password.kubecost	resource
aws_eks_addon_version.kubecost	data source
aws_eks_cluster.eks	data source
aws_region.current	data source
kubernetes_service.internal-nginx-ingress	data source
kubernetes_service.istio-ingress	data source
kubernetes_service.nginx-ingress	data source

Inputs

Name	Description	Type	Default	Required
amazon_eks_aws_ebs_csi_driver_enabled	Whether to enable the EKS Managed AWS EBS CSI Driver add-on or not.	bool	false	no
amazon_eks_vpc_cni_enabled	Enable or disable the installation of the Amazon EKS VPC CNI addon.	bool	false	no
aws_load_balancer_version	Specify the version of the AWS Load Balancer Controller for Ingress	string	"1.4.4"	no
aws_node_termination_handler_enabled	Enable or disable node termination handler	bool	false	no
cert_manager_enabled	Enable or disable the cert manager add-on for EKS cluster.	bool	false	no
cert_manager_install_letsencrypt_http_issuers	Enable or disable the HTTP issuer for cert-manager	bool	false	no
cert_manager_install_letsencrypt_r53_issuers	Enable or disable the creation of Route53 issuer while installing cert manager.	bool	false	no
cert_manager_letsencrypt_email	Specifies the email address to be used by cert-manager to request Let's Encrypt certificates	string	""	no
cluster_autoscaler_chart_version	Version of the cluster autoscaler helm chart	string	"9.29.0"	no
cluster_autoscaler_enabled	Whether to enable the Cluster Autoscaler add-on or not.	bool	false	no
cluster_issuer	Specify the letsecrypt cluster-issuer for ingress tls.	string	"letsencrypt-prod"	no
cluster_propotional_autoscaler_enabled	Enable or disable Cluster propotional autoscaler add-on	bool	false	no
core_dns_hpa_config	Configuration to provide settings of hpa over core dns	any	{ "corednsdeploymentname": "coredns", "maxReplicas": 10, "minReplicas": 2, "targetCPUUtilizationPercentage": 80, "targetMemoryUtilizationPercentage": "150Mi" }	no
efs_storage_class_enabled	Enable or disable the Amazon Elastic File System (EFS) add-on for EKS cluster.	bool	false	no
eks_cluster_name	Fetch Cluster ID of the cluster	string	""	no
enable_aws_load_balancer_controller	Enable or disable AWS Load Balancer Controller add-on for managing and controlling load balancers in Kubernetes.	bool	false	no
environment	Environment identifier for the Amazon Elastic Kubernetes Service (EKS) cluster.	string	""	no
external_secrets_enabled	Enable or disable External Secrets operator add-on for managing external secrets.	bool	false	no
ingress_nginx_enabled	Enable or disable Nginx Ingress Controller add-on for routing external traffic to Kubernetes services.	bool	false	no
ingress_nginx_version	Specify the version of the NGINX Ingress Controller	string	"4.7.0"	no
internal_ingress_nginx_enabled	Enable or disable the deployment of an internal ingress controller for Kubernetes.	bool	false	no
ipv6_enabled	whether IPv6 enabled or not	bool	false	no
istio_config	Configuration to provide settings for Istio	any	{ "cert_manager_cluster_issuer_enabled": false, "egress_gateway_enabled": false, "egress_gateway_namespace": "istio-egressgateway", "envoy_access_logs_enabled": false, "ingress_gateway_enabled": true, "ingress_gateway_namespace": "istio-ingressgateway", "observability_enabled": true, "prometheus_monitoring_enabled": false }	no
istio_enabled	Enable istio for service mesh.	bool	false	no
karpenter_enabled	Enable or disable Karpenter, a Kubernetes-native, multi-tenant, and auto-scaling solution for containerized workloads on Kubernetes.	bool	false	no
karpenter_provisioner_config	Configuration to provide settings for Karpenter, including which private subnet to use, instance capacity types, and excluded instance types.	any	{ "excluded_instance_type": [ "nano", "micro", "small" ], "instance_capacity_type": [ "spot" ], "instance_hypervisor": [ "nitro" ], "private_subnet_name": "" }	no
karpenter_provisioner_enabled	Enable or disable the installation of Karpenter, which is a Kubernetes cluster autoscaler.	bool	false	no
keda_enabled	Enable or disable Kubernetes Event-driven Autoscaling (KEDA) add-on for autoscaling workloads.	bool	false	no
kms_key_arn	ARN of the KMS key used to encrypt AWS resources in the EKS cluster.	string	""	no
kms_policy_arn	Specify the ARN of KMS policy, for service accounts.	string	""	no
kubeclarity_enabled	Enable or disable the deployment of an kubeclarity for Kubernetes.	bool	false	no
kubeclarity_hostname	Specify the hostname for the Kubeclarity.	string	""	no
kubeclarity_namespace	Name of the Kubernetes namespace where the kubeclarity deployment will be deployed.	string	"kubeclarity"	no
kubecost_enabled	Enable or disable the deployment of an Kubecost for Kubernetes.	bool	false	no
kubecost_hostname	Specify the hostname for the kubecsot.	string	""	no
metrics_server_enabled	Enable or disable the metrics server add-on for EKS cluster.	bool	false	no
metrics_server_helm_version	Version of the metrics server helm chart	string	"3.8.2"	no
metrics_server_vpa_config	Configuration to provide settings of vpa over metrics server	any	{ "maxCPU": "100m", "maxMemory": "500Mi", "metricsServerDeploymentName": "metrics-server", "minCPU": "25m", "minMemory": "150Mi" }	no
name	Specify the name prefix of the EKS cluster resources.	string	""	no
node_termination_handler_version	Specify the version of node termination handler	string	"0.21.0"	no
private_subnet_ids	Private subnets of the VPC which can be used by EFS	list(string)	[ "" ]	no
reloader_enabled	Enable or disable Reloader, a Kubernetes controller to watch changes in ConfigMap and Secret objects and trigger an application reload on their changes.	bool	false	no
service_monitor_crd_enabled	Enable or disable the installation of Custom Resource Definitions (CRDs) for Prometheus Service Monitor.	bool	false	no
single_az_ebs_gp3_storage_class_enabled	Whether to enable the Single AZ storage class or not.	bool	false	no
single_az_sc_config	Name and regions for storage class in Key-Value pair.	list(any)	[]	no
velero_config	Configuration to provide settings for Velero, including which namespaces to backup, retention period, backup schedule, and backup bucket name.	any	{ "backup_bucket_name": "", "namespaces": "", "retention_period_in_days": 45, "schedule_backup_cron_time": "", "slack_notification_channel_name": "", "slack_notification_token": "", "velero_backup_name": "" }	no
velero_enabled	Enable or disable the installation of Velero, which is a backup and restore solution for Kubernetes clusters.	bool	false	no
vpc_id	ID of the VPC where the cluster and its nodes will be provisioned	string	""	no
worker_iam_role_arn	Specify the IAM role Arn for the nodes	string	""	no
worker_iam_role_name	Specify the IAM role for the nodes that will be provisioned through karpenter	string	""	no

Outputs

Name	Description
ebs_encryption_enable	Whether Amazon Elastic Block Store (EBS) encryption is enabled or not.
efs_id	ID of the Amazon Elastic File System (EFS) that has been created for the EKS cluster.
environment	Environment Name for the EKS cluster
internal_nginx_ingress_controller_dns_hostname	DNS hostname of the NGINX Ingress Controller that can be used to access it from within the cluster.
istio_ingressgateway_dns_hostname	DNS hostname of the Istio Ingress Gateway.
kubeclarity	Kubeclarity_Info
kubecost	Kubecost_Info
nginx_ingress_controller_dns_hostname	DNS hostname of the NGINX Ingress Controller.

Contribution & Issue Reporting

To report an issue with a project:

1. Check the repository's issue tracker on GitHub
2. Search to see if the issue has already been reported
3. If you can't find an answer to your question in the documentation or issue tracker, you can ask a question by creating a new issue. Be sure to provide enough context and details so others can understand your problem.
4. Contributing to the project can be a great way to get involved and get help. The maintainers and other contributors may be more likely to help you if you're already making contributions to the project.

License

Apache License, Version 2.0, January 2004 (http://www.apache.org/licenses/).

Support Us

To support a GitHub project by liking it, you can follow these steps:

1. Visit the repository: Navigate to the GitHub repository.

2. Click the "Star" button On the repository page, you'll see a "Star" button in the upper right corner. Clicking on it will star the repository, indicating your support for the project.

3. Optionally, you can also leave a comment on the repository or open an issue to give feedback or suggest changes.

Starring a repository on GitHub is a simple way to show your support and appreciation for the project. It also helps to increase the visibility of the project and make it more discoverable to others.

Who we are

We believe that the key to success in the digital age is the ability to deliver value quickly and reliably. That’s why we offer a comprehensive range of DevOps & Cloud services designed to help your organization optimize its systems & Processes for speed and agility.

1. We are an AWS Advanced consulting partner which reflects our deep expertise in AWS Cloud and helping 100+ clients over the last 5 years.
2. Expertise in Kubernetes and overall container solution helps companies expedite their journey by 10X.
3. Infrastructure Automation is a key component to the success of our Clients and our Expertise helps deliver the same in the shortest time.
4. DevSecOps as a service to implement security within the overall DevOps process and helping companies deploy securely and at speed.
5. Platform engineering which supports scalable,Cost efficient infrastructure that supports rapid development, testing, and deployment.
6. 24*7 SRE service to help you Monitor the state of your infrastructure and eradicate any issue within the SLA.

We provide support on all of our projects, no matter how small or large they may be.

You can find more information about our company on this squareops.com, follow us on Linkedin, or fill out a job application. If you have any questions or would like assistance with your cloud strategy and implementation, please don't hesitate to contact us.