cnvrg.io docs

# Set Up an AWS EKS Cluster

cnvrg can natively run using an EKS cluster hosted on your own AWS account and an EKS cluster can also be used to run all your cnvrg job. This allows you to leverage cnvrg and Kubernetes to create one flexible compute and DS environment.

In this guide, you will learn how to:

  • Create a EKS cluster using eksctl and the AWS CLI

# Create a User in AWS with the Correct Permissions

Creating an EKS cluster requires certain permissions within AWS.

To create an AWS user account with the necessary permissions, first create a new Access Policy in your AWS Console. Copy the following JSONs and use it to create the new policy:

# AmazonEC2FullAccess (AWS Managed Policy)

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "ec2:*",
            "Effect": "Allow",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "elasticloadbalancing:*",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "cloudwatch:*",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "autoscaling:*",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "iam:CreateServiceLinkedRole",
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "iam:AWSServiceName": [
                        "autoscaling.amazonaws.com",
                        "ec2scheduled.amazonaws.com",
                        "elasticloadbalancing.amazonaws.com",
                        "spot.amazonaws.com",
                        "spotfleet.amazonaws.com",
                        "transitgateway.amazonaws.com"
                    ]
                }
            }
        }
    ]
}

# AWSCloudFormationFullAccess (AWS Managed Policy)

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "cloudformation:*"
            ],
            "Resource": "*"
        }
    ]
}

# EksAllAccess

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "eks:*",
            "Resource": "*"
        },
        {
            "Action": [
                "ssm:GetParameter",
                "ssm:GetParameters"
            ],
            "Resource": [
                "arn:aws:ssm:*:<account_id>:parameter/aws/*",
                "arn:aws:ssm:*::parameter/aws/*"
            ],
            "Effect": "Allow"
        },
        {
             "Action": [
               "kms:CreateGrant",
               "kms:DescribeKey"
             ],
             "Resource": "*",
             "Effect": "Allow"
        }
    ]
}

# IamLimitedAccess

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:CreateInstanceProfile",
                "iam:DeleteInstanceProfile",
                "iam:GetInstanceProfile",
                "iam:RemoveRoleFromInstanceProfile",
                "iam:GetRole",
                "iam:CreateRole",
                "iam:DeleteRole",
                "iam:AttachRolePolicy",
                "iam:PutRolePolicy",
                "iam:ListInstanceProfiles",
                "iam:AddRoleToInstanceProfile",
                "iam:ListInstanceProfilesForRole",
                "iam:PassRole",
                "iam:DetachRolePolicy",
                "iam:DeleteRolePolicy",
                "iam:GetRolePolicy",
                "iam:GetOpenIDConnectProvider",
                "iam:CreateOpenIDConnectProvider",
                "iam:DeleteOpenIDConnectProvider",
                "iam:ListAttachedRolePolicies",
                "iam:TagRole"
            ],
            "Resource": [
                "arn:aws:iam::<account_id>:instance-profile/eksctl-*",
                "arn:aws:iam::<account_id>:role/eksctl-*",
                "arn:aws:iam::<account_id>:oidc-provider/*",
                "arn:aws:iam::<account_id>:role/aws-service-role/eks-nodegroup.amazonaws.com/AWSServiceRoleForAmazonEKSNodegroup",
                "arn:aws:iam::<account_id>:role/eksctl-managed-*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:GetRole"
            ],
            "Resource": [
                "arn:aws:iam::<account_id>:role/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:CreateServiceLinkedRole"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "iam:AWSServiceName": [
                        "eks.amazonaws.com",
                        "eks-nodegroup.amazonaws.com",
                        "eks-fargate.amazonaws.com"
                    ]
                }
            }
        }
    ]
}

Now, add the policy to the user who will be creating the cluster.

Of course, any permission level higher than contributor should also be suitable.

# Prerequisites: Prepare Your Local Environment

Before you can complete the installation, you must install and prepare the following dependencies on your local machine:

# Use the AWS CLI to Connect to your Account

::: NOTE You can skip this step if you have already logged in to the AWS CLI. :::

To connect with your AWS account, use the AWS configure command:

aws configure

The AWS website has information for using the above command.

# Create the YAML Recipe for the Cluster

We will use eksctl to create the cluster in AWS. To use eksctl, you will need to create a YAML file that provides the necessary configuration for the cluster.

In the file, you must set:

  • name: the name for the cluster you will create
  • region: the name of the Amazon region you want to create the cluster in.
  • vpc: The VPC to set up the cluster in. If you do not set this, a new VPC will be created automatically.

The rest of the configuration is already set within the file to the default recommended values for running cnvrg. However, excluding the lines commented as Required., you can edit the rest of the configuration to your needs. For example, you could add new node pools or change the instance types for the node pools. You could also change the lower and upper bounds of the node pools.

Copy the following text into a new file called cluster.yaml and then edit it accordingly:

---
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
  name: cnvrg-core
  region: us-east-1
availabilityZones: ['us-east-1a', 'us-east-1b']
nodeGroups:
- name: cnvrg-app-core
  instanceType: m5a.2xlarge
  volumeSize: 100
  minSize: 2
  maxSize: 4
  desiredCapacity: 2
  privateNetworking: true
  iam:
    attachPolicyARNs:
    - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
    - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
    - arn:aws:iam::aws:policy/AmazonS3FullAccess  #you can create policy specfic for bucket created
    withAddonPolicies:
      autoScaler: true
  tags:
    k8s.io/cluster-autoscaler/enabled: 'true'
  availabilityZones: ['us-east-1b']

# VPC information (advanced)

When using the above YAML, eksctl will automatically configure a new VPC in your AWS account with all of the configurations and setup required for a EKS VPC.

If you wish to use an existing VPC, add the following section to the YAML with the required details. Please note, additional setup is required to configure the VPC correctly.

vpc: 
  id: "<vpc-id>" # This is the id of your VPC in AWS. 
  subnets: # In this section, include all the subnets of your AWS VPC. Follow the example format below.
    private: # Private subnet details. Add an entry for each region of your VPC.
      <subnet-region1>: { id: <subnet-id> } # Change to be the subnet region and subnet id.
      <subnet-region2>: { id: <subnet-id> } # Change to be the subnet region and subnet id.
    public: # Public subnet details. Add an entry for each region of your VPC.
      <subnet-region1>: { id: <subnet-id> } # Change to be the subnet region and subnet id.
      <subnet-region2>: { id: <subnet-id> } # Change to be the subnet region and subnet id.

# Create the cluster using the YAML file

To create the cluster based on your prepared cluster.yaml file use the following command:

eksctl create cluster -f cluster.yaml

# Conclusion

Congratulations, you have now set up a working EKS cluster in AWS! You can now use Helm to finish the setup and get the cluster ready to be used for deploying the cnvrg app or running cnvrg jobs.

Follow this guide to complete the setup.

Last Updated: 1/27/2021, 5:25:30 PM

← Add Node Pools →