# Set Up an GCP GKE Cluster

cnvrg can natively run using an EKS cluster hosted on your own AWS account and an EKS cluster can also be used to run all your cnvrg job. This allows you to leverage cnvrg and Kubernetes to create one flexible compute and DS environment.

In this guide, you will learn how to:

  • Create a GKE cluster using gcloud command line tools

# Create a User in GCP with the Correct Permissions

Creating an GKE cluster requires certain permissions within GCP.

To create a user with the necessary permissions to create a cluster and follow the rest of the guide, add the following GCP permissions to the relevant user account:

  1. Service Usage Admin
  2. Create Service Accounts
  3. Security Admin
  4. Compute Admin
  5. Kubernetes Engine Admin
  6. Service Account User
  7. Service Account Key Admin

# Prerequisites: Prepare Your Local Environment

Before you can complete the installation, you must install and prepare the following dependencies on your local machine:

# Use the gcloud CLI to Log In to your Account

NOTE

You can skip this step if you have already logged in to the gcloud CLI.

Use the following command to log in to your GCP account.

gcloud auth login

NOTE

Make sure your user account has the necessary privileges or is an administrator.

# Set the gcloud Project for the Setup

NOTE

You can skip this step if you have already configured the project in the gcloud CLI.

Make sure your gcloud CLI is communicating with the GCP project you would like to use for the cluster.

Specify the project with the following command. Fill in the variable at the beginning with the correct project id:

PROJECT=<project-id>

gcloud config set project ${PROJECT}

# Setup glcoud Services

NOTE

You can skip this step if you have already necessary services in the gcloud CLI.

Use the following command to set up the services as need for the rest of the process:

gcloud services enable compute.googleapis.com
gcloud services enable servicenetworking.googleapis.com
gcloud services enable cloudresourcemanager.googleapis.com
gcloud services enable container.googleapis.com
gcloud services enable containerregistry.googleapis.com

# Create a Service Account for the Cluster

NOTE

You can skip this step if you have created a service account.

# Create the service account

To be able to properly manage and use the cluster, we need to create a service account. Use the following command to create the service account as needed:

PROJECT=<project-id>

gcloud iam service-accounts create cnvrgio --display-name cnvrgio --project ${PROJECT}

# Delegate the permissions for the service account

Next, we will prescribe the correct permissions to our new service account:

PROJECT=<project-id>

gcloud projects add-iam-policy-binding ${PROJECT}   --member serviceAccount:cnvrgio@${PROJECT}.iam.gserviceaccount.com   --role roles/container.admin
gcloud projects add-iam-policy-binding ${PROJECT}   --member serviceAccount:cnvrgio@${PROJECT}.iam.gserviceaccount.com   --role roles/compute.admin
gcloud projects add-iam-policy-binding ${PROJECT}   --member serviceAccount:cnvrgio@${PROJECT}.iam.gserviceaccount.com   --role roles/iam.serviceAccountUser
gcloud projects add-iam-policy-binding ${PROJECT}   --member serviceAccount:cnvrgio@${PROJECT}.iam.gserviceaccount.com   --role roles/resourcemanager.projectIamAdmin
gcloud projects add-iam-policy-binding ${PROJECT}   --member serviceAccount:cnvrgio@${PROJECT}.iam.gserviceaccount.com   --role roles/storage.admin

# Create a VPC for the cluster

NOTE

If you have an existing VPC in GCP, you can skip this step.

Creating a VPC allows you to ensure everything is hosted securely and can only be accessed by those with the correct permissions.

You will need to decide which region you will be launching the cluster in. This will affect where the cluster is located and which machine types can be used. Using a region will create a multi-zonal cluster which is more reliant than a single zoned cluster. Read here to find out more.

Use the following command to create the VPC. Fill in the variables at the beginning with the correct information for your cluster:

PROJECT=<project-id>  
REGION=<region>

gcloud compute networks create cnvrg --subnet-mode custom --project ${PROJECT}
gcloud compute networks subnets create cnvrg-app --network cnvrg --region ${REGION} --range 10.10.0.0/16 --project ${PROJECT}

# Create the GKE cluster

We will now create the Kubernetes cluster. Use the following command to initialize the cluster in GCP. Fill in the variables at the beginning with the correct information for your cluster:

PROJECT=<project-id>
CLUSTER_NAME=<cluster-name>
REGION=<region>

gcloud --project ${PROJECT} container clusters create ${CLUSTER_NAME}  \
	--num-nodes 2 \
	--enable-ip-alias \
	--region ${REGION} \
	--machine-type n1-standard-8 \
	--image-type ubuntu \
	--scopes=storage-rw \
	--node-locations ${REGION} \
	--no-enable-autoupgrade

NOTE

The above command will create the cluster using the default service account, network and VPC. You can set those options by adding the following commands and editing them accordingly:

	--service-account cnvrgio@${PROJECT}.iam.gserviceaccount.com \
  	--network projects/${PROJECT}/global/networks/cnvrg \
	--subnetwork cnvrg-app

# Retrieve the Credentials (kubeconfig) for the Newly Created Cluster

Running the following command will save the kubeconfig for the new cluster locally. This allows you to be able to interact with it using kubectl.

The kubeconfig will be saved in ~/.kube/config

PROJECT=<project-id>
CLUSTER_NAME=<cluster-name>
REGION=<region>

gcloud container clusters get-credentials ${CLUSTER_NAME} \
    --region ${REGION} \
    --project ${PROJECT}

# Check the Cluster is Working Correctly

Run the following command to get a list of the nodes in the cluster. If this command runs correctly and returns the list of nodes, everything is working as expected:

kubectl get nodes

# Conclusion

Congratulations, you have now set up a working GKE cluster in GCP! You can now use Helm to finish the setup and get the cluster ready to be used for deploying the cnvrg app or running cnvrg jobs.

Follow this guide to complete the setup.

Last Updated: 7/6/2020, 9:58:38 AM