# cnvrg CLI

CLI v1 is deprecated on cnvrg v4.8.0
Please use CLI v2 instead.

The topics in this page:

Download and Install the cnvrg CLI
CLI Help
Environment Setup Commands
Project Commands
Dataset Commands
Workspace Commands
- Run a remote SSH session
Experiment Commands
- Run a new experiment
Flow Commands
AI Library Commands
Miscellaneous Commands
Troubleshooting the cnvrg CLI

# Download and Install the cnvrg CLI

# Install Ruby

cnvrg CLI was built with Ruby. Ruby is a great programming language and can be very easily installed.

WARNING

cnvrg currently supports Ruby versions 2.3 - 2.6.6. Ruby 2.7.0 is not currently supported.

# Install the cnvrg CLI

Now that you have Ruby configured, move to the next step and install cnvrg CLI.

Check your version of Ruby by running the following command:

ruby -v

Open the terminal and run the corresponding command for your version of Ruby.

Verify cnvrg is installed:

cnvrg --version

If the output was a version number, then the cnvrg CLI has been succesfully installed.

# Setup and log in

After successfully installing the CLI, you now need to set the API URL and log in.

To allow your CLI to communicate with your cnvrg environment, set the API URL using the cnvrg --api command:

cnvrg --api '/api'

You can copy the above code exactly to set your correct API. If you're still not sure what your unique Web URL is, log on via the Web and copy the URL from your browser, or contact support for additional help.

Now log in and authenticate using the cnvrg login command:

cnvrg login

You will be prompted to enter your email and password.

After succesfully logging in, you will have everything set up and can start using all the CLI commands.

# CLI Help

To find out information about any of the CLI commands, use cnvrg help [command] or cnvrg --help [command]

# Environment Setup Commands

# Connect to your private environment (set the API URL)

To ensure the CLI can communicate with your cnvrg environment, you must set the API URL using the cnvrg --api command:

cnvrg --api '/api'

# Log in

cnvrg login

# Log out

To unauthenticate your system:

cnvrg logout

# Check the logged in user

Use the me command to see which account is currently logged on to cnvrg CLI:

cnvrg me

# Switching between multiple organizations

If you are running on various organizations you can switch between them by configuring the config.yml file. In order to edit the yaml file run:

vim ~/.cnvrg/config.yml

Within the file you will find a field titled owner which is the name of your organization. Change the owner field to the organization you would like to have your CLI connected to. Once edited save and exit the file. Now when running a command the CLI will automatically work with the organization you assigned in the file.

# Check the cnvrg CLI version

Use the version command to check the currently installed version of cnvrg CLI:

cnvrg version

# Project Commands

# Create an empty project

To simply create a new, empty cnvrg project, use the new command. This command will create a new folder with the default tree of files:

cnvrg new 'project_name'

# Link a local directory to a new online project

You can create a project and sync a directory to it with a single command.

Navigate into the directory containing the files to upload. From there, run the the link command:

cnvrg link --title='project_name' --sync

Flags	Type	Description
--title=TITLE	string	Title of project. Default is the directory name.
--sync	boolean	Syncs the files with the online directory.

# Link an existing project with a Git directory

If you have created a project online already but now would like to link a directory connected with Git, use the link_git command:

 cnvrg link_git "cnvrg_project_url"

The project url is the full url path to the project in your cnvrg deployment.

If your project was at https://app.cnvrg.io/org/project/project_name/, in the browser then this url is what you use in the cli command. :::

# Clone an existing project

To clone or replicate an already existing project:

 cnvrg clone "cnvrg_project_url"

Flags	Type	Description
--commit=COMMIT	string	specify which commit the new project is cloning from

# Unlink your directory from the cnvrg project

Use the unlink command to unlink your local directory from the cnvrg project:

 cnvrg unlink

# Other project commands

Use the jump command to change to a specific commit:
```
 cnvrg jump commit_sha1
```

File Commands:

Use the download command to retrieve files from the cnvrg server:
```
cnvrg download
```
Use the upload command to send files from your local machine to the cnvrg server:
```
cnvrg upload
```
Use the sync command to update the server about any file changes in your local directory:
```
cnvrg sync
```
TIP

You can use cnvrg sync -f to commit directly to master.

::: note Run cnvrg sync command from your project root directory, and not from any of its subfolders. :::

# Dataset Commands

# Dataset ID

In many commands, you will need to use a dataset ID. The dataset ID is the name used for the dataset in its URL.

For example, if you have a dataset that lives at: https://app.cnvrg.io/my_org/datasets/dataset_name, the dataset ID is dataset_name.

Then to use cnvrg data link, the command would be cnvrg data link dataset_name.

# Initialize a new dataset with a local directory

Navigate into the directory for the dataset and use the data init command:

cnvrg data init

Using this command will create a new empty dataset with the same name as your local folder.

NOTE

If you would like to set a custom title of the dataset, simply add --title="custom_title":

cnvrg data init --title="custom_title"

Keep in mind that you cannot use whitespaces in the dataset's title.

# Link a local directory to an existing remote dataset

First, go to the dataset inside cnvrg's Web UI and copy the dataset ID from the URL.
Then navigate into the local directory that you are linking to the existing directory and use the data link command:

cnvrg data link dataset_id

TIP

The dataset id is usually the full name of the dataset. You can also use the full URL instead.
If your dataset was at https://app.cnvrg.io/org/datasets/dataset_name, you could either use the full URL or just dataset_name.

# Upload files to a dataset

To upload a file or files into a remote dataset, use the data put command:

cnvrg data put [--commit=SHA1/latest] [--message='MESSAGE'] [--threads=1000] dataset_id file_path/file

Flags	Type	Description
--commit=COMMIT	string	The commit to upload the file or directory to. Can be either the sha1 of a specific commit or latest. If the flag is not used, it will create a new commit.
--message='MESSAGE'	string	Add a message to the commit.
--threads=THREADS	integer	Configure the amount of CPU threads to use for the upload. Default is 15.
--chunk-size=CHUNK-SIZE	integer	The size of each portion of the upload that files are split into. Default is 1000.
--force	flag	Create a new commit with only the files included in the current upload.
--override	flag	Use this flag to replace existing copies even if they are identical. Every file included in the command will be uploaded.
--dir	string	Upload the files to a specific subdirectory in the remote dataset. For example, `--dir="images"` will upload the files to the `images/` folder. It will create the directory if it does not exist.

The data put command will upload the specified files to the remote dataset.

NOTE

In CLI versions below 1.9.5, you must use the full dataset URL. For example: cnvrg data put https://app.cnvrg.io/my_org/datasets/mnist image.png

Examples:

Upload the entire current folder recursively (including all files & sub-folders & folder structure):
```
cnvrg data put dataset_id .
```
Upload a specific folder and its content (not including sub-folders):
```
cnvrg data put dataset_id folder/*.*
```
Upload a specific file from a path (the path will be matched):
```
cnvrg data put dataset_id folder1/folder2/file.txt
```

# Remove files from a dataset

To remove a file that exists in a dataset, use the data rm command:

cnvrg data rm [--message='MESSAGE'] dataset_id file_path/file

Flags	Type	Description
--message='MESSAGE'	string	Add a message to the commit.

The data rm will remove the matching files from the dataset.

Examples:

Delete an entire folder and all its contents:
```
cnvrg data rm dataset_id folder/
```
Delete a specific file:
```
cnvrg data rm dataset_id file_name
```

# Commit caching and clearing commands

To cache a commit onto an NFS disk, use the data cache command:

cnvrg data cache commit_sha1 disk_slug

To clear a commit currently cached on an NFS disk, use the data clear command:

cnvrg data clear commit_sha1 disk_slug

You can also list the connected external disks using the external_disk_list command:

cnvrg external_disk_list

# Other dataset commands

To update a local version of a dataset with files from the remote version, use the data download command:

cnvrg data download

To clone the dataset from the server into your local environment:

cnvrg data [--commit=SHA1/latest] clone dataset_id [--only_tree] [--query='query_name']

Flags	Type	Description
--commit=COMMIT	string	the commit to clone the dataset from
--only_tree	boolean	whether to download only the tree of directories
--query	string	downloads only the files that match this query

To see a list of all datasets, use the data list command:
```
cnvrg data list
```
To see a list of the files associated with a specific dataset, use the data files command:
```
cnvrg data files dataset_id
```
To retrieve a list of existing commits (and cache information) for the current dataset, use the data commits command:
```
cnvrg data commits dataset_id
```
To retrieve a list of existing subsets and queries for the current dataset, use the data queries command:
```
cnvrg data queries dataset_id
```
To see a list of the files associated with a specific query on the current dataset, use the data query_files command:
```
cnvrg data query_files dataset_id 'QUERY_NAME'
```

# Workspace Commands

# Run a remote SSH session

To start a remote SSH session with an open workspace inside cnvrg, use the cnvrg ssh start command:

cnvrg ssh start [--kubeconfig=/path/to/kubeconfig] [--user=username] [--port=desired_local_port] [--password=desired_password] workspace_id

Flags	Type	Description
--kubeconfig=/path/to/kubeconfig	path	Set a specific kubeconfig. By default the kubeconfig is taken from the `KUBECONFIG` environment variable. If you have multiple contexts in the kubeconfig, maje sure you have the current context set to the workspace's cluster.
--user=username	string	Choose a specific username for SSH session. Default is `root`. The user must exist in the workspace's container.
--port=desired_local_port	int	Choose a specific local port to connect the SSH session to. Default is `2222`.
--password=desired_password	string	Choose a specific password for the SSH session. The default is a randomly generated string. You can use this flag to make the password consistent between sessions.

The SSH session will stay live until the command is interrupted. If the workspace stops, the command will not automatically stop and the user must stop it themselves.

NOTE

To disable host key checking for ssh by default add the following lines in your ssh config file (usually located at ~/.ssh/config):

Host localhost
  StrictHostKeyChecking no

If you would like to disable it for all hosts, change localhost to *, such that:

Host *
  StrictHostKeyChecking no

TIP

The workspace_id can be found in the URL for the workspace.

For example, if your workspace is live at https://app.cnvrg.io/acme/projects/recommender/notebook_sessions/show/ekqjsnzerv9x5t62y7yd, then the workspace_id would be ekqjsnzerv9x5t62y7yd.

The corresponding example command would be: cnvrg ssh start ekqjsnzerv9x5t62y7yd.

# Experiment Commands

# Run a new experiment

Use the run command to run a new experiment. Ensure the experiment command (for example python3 train.py) is the last parameter of the command and any flags you are using are entered before the command.

For example:

cnvrg run [-t='experiment_title'] [--data='dataset_name'] [--machine='machine_name'] command

Flags	Type	Description
--machine=MACHINE	string	Set the compute. For Kubernetes compute it should be in the format: `cluster_name.template_name`. For example, `workers.medium`. You can leave out `cluster_name` to use the default cluster. For an on-premise machine, simply write the name of the machine. For example, `worker1`.
--sync_before	boolean	Sync the directory before running the experiment
--sync_after	boolean	Sync the directory after running the experiment. Default is True.
--image="IMAGE_NAME"	string	Docker image to use.
--title=TITLE	string	Set the title of the experiment.
--log	boolean	Display the log of the experiment.
--wait	flag	Wait for the experiment to complete. The command will return the exit status of the experiment.
--commit=COMMIT	string	Choose a specific commit/version id of the project.
--data=DATASET_NAME	string	Choose a dataset to run the experiment with.
--data_commit=DATA_COMMIT	string	Choose the specific commit/version id of the dataset to run the experiment on
--datasets=DATASETS	array of dictionaries	A list of datasets and commits/queries (that is: `[{"id": "dataset_id-1", "commit": "commit_sha1-1", "query": "query_name-1", "tree_only": true, "use_cached": true}, {dataset_dict_2}, ...]`) to attach to the experiment. You can use either `commit` or `query` and not both. `use_cached` is only relevant when integrated to NFS and is true by default.
--grid='grid.yaml'	path	Run a grid search using a YAML file outlining the parameters.
--output_dir=OUTPUT_DIR	string	When running remote experiment in git project, sets the output folder for cnvrg to track experiment artifacts.
--git-branch=GIT_BRANCH	string	Specific branch to use when running Experiment in git project.
--git-commit=GIT_COMMIT	string	Specific commit to use when running Experiment in git project.
--git_diff=True	boolean	When git_diff is enabled cnvrg will also sync the git difference, saving the changes relative to your git index along with the experiment. Default is False.
--local_folders=LOCAL_FOLDER_PATH	string	Absolute path of the local folders which need to be mounted.
--prerun=False	boolean	Flag for whether or not to run the prerun.sh script. Default is True.
--requirements=False	boolean	Flag for whether or not to install the packages in the requirements.txt file. Default is True.
--schedule="dd.mm.yyyy hh:mm<PM/AM>" or --schedule="yyyy.dd.mm hh:mm<PM/AM>"	string	Set a time to execute the experiment. The time is relative to the timezone defined for the current user. You can specify the time using either a 24- or 12-hour clock format. For example: To specify 4:25 PM on the 31st of January 2012, you can specify any of these: `31.01.2021 16:25`, `31.01.2021 4:25PM`, `2021.01.31 16:25`, `2021.01.31 4:25PM`.
--recurring="* * * * *"	string	Set a schedule for executing a recurring experiment. Specify the schedule using Cron format. For example, to execute the experiment at 30 minutes past every hour, specify: `"30 * * * *"`
--notify_on_success	boolean	Receive an email when the experiment ends successfully. Default is False.
--notify_on_error	boolean	Receive an email if an error occurs while your experiment is running. Default is False.
--emails=EMAILS	string	Add extra recipients for email notifications for the experiment. The string should be a list of comma-separated emails. For example, `"person1@email.com,person2@email.com"`.

::: note TIP The --git_diff flag is useful when developing in a workspace. With it, you can fiddle with your code and run experiments without first syncing to git. The code you run with will still be synced so you can eventually push back the code that your are most happy with. :::

# Example YAML file for grid searches

parameters:
    - param_name: "learning_rate"
      type: "discrete" # An array of numerical values
      values: [0.1, 0.01 ,0.001]

    - param_name: "kernel"
      type: "categorical" # An array of string values
      values: ["linear", "rbf"]

    - param_name: "epochs"
      type: "integer"
      min: 10 # inclusive
      max: 200 # not inclusive
      scale: "linear"
      steps: 10 # The number of linear steps to produce.

    - param_name: "fractions"
      type: "float" # Precision is 9 after period
      min: 0.1 # inclusive
      max: 0.2 # not inclusive
      scale: "log2"
      steps: 10 # The number of linear steps to produce.

TIP

The scale can be linear, log2 or log10.

# Flow Commands

You can use Flow YAML files to build and run Flows from the CLI. To see how to construct a Flow YAML visit the Flows section.

# Create a Flow online using a local YAML file

cnvrg flow import --file='flow.yaml'

This command will create a new Flow in the Project from the data within the specified file.

# Download an online Flow as a YAML file

cnvrg flow export --title='Flow_Title' [--version='version'] [--file='name_of_file_to_save.yaml']

This command will download the specified Flow as a YAML and save it in your local folder.

Flags	Type	Description
--title='TITLE'	string	Required. Sets the name of the online Flow you will be downloading.
--version='VERSION'	string	Sets the version of the Flow you will be downloading. Default is latest.
--file='FILE_NAME'	string	Optionally sets the name of the downloaded Flow YAML.

# Run a Flow using a local Flow YAML file

cnvrg flow run --file='flow.yaml'

This command will run a flow using the specifications within the supplied Flow YAML file.

TIP

If you would like to run a Flow that you created online using the CLI, first use cnvrg flow export to download the Flow's YAML. Then use cnvrg flow run to run the Flow using th downloaded file.

# AI Library Commands

You can create a new component or update an existing component in the AI Library using the library import command. This command uses a library.yaml file and other accompanying files to create a library.

Run the library import command in a directory containing the library.yaml and the files needed to construct your library component.

cnvrg library import

# Miscellaneous Commands

To print the cnvrg CLI logs file to your screen, use the logs command:
```
cnvrg logs
```
To retrieve a list of comes or machines, use the list_machines command:
```
cnvrg list_machines
```
To receive usage tips for any cnvrg CLI commands, use the help command:
```
cnvrg help [COMMAND]
```
To set the compression path, use the set_compression_path command:
```
cnvrg set_compression_path 'path'
```
To set the current logged on account as the default owner of the current project, use the set_default_owner command:
```
cnvrg set_default_owner
```
To see the status of the working tree and ensure your local and remote projects are in sync use the status command:
```
cnvrg status
```

# Troubleshooting the cnvrg CLI

If you experience issues in running CLI commands, troubleshoot by performing the following checks:

# Update the cnvrg CLI

Sometimes updating the CLI can help resolve the issue.

To update the CLI, run the command:

gem update cnvrg

If you already have the latest version you will get the following response:

Updating installed gems
Nothing to update

If you do not have the latest version, the newest version will be downloaded and installed.

After updating the CLI, check if your issue persists.

# Check you are using the correct API

cnvrg needs to communicate with your specific workspace. To do so, you need to provide it a link to the correct URL API.
As part of the set up process, you should have set your API to the correct URL, but often this may be the root cause of common CLI issues.

Your specific URL API is: /api

NOTE

It should simply be the URL you use to interact with cnvrg online followed by /api.
For example, https://app.cnvrg.io/api, https://cnvrg.mycompany.com/api and so on.
Make sure you have not included your organization's name in the API link.

You can check the URL API you are currently using by running the command: cnvrg me.

To set your API correctly first log out:

cnvrg logout

Then set the API:

cnvrg --api '/api'

Finally, log in again:

cnvrg login

You should now have correctly set your API. Check if your issue persists.

# SSL verification error (Windows)

When you attempt to install the cnvrg CLI on a PC running Windows, you may receive the following error:

ERROR: SSL verification error at depth 2: self signed certificate in certificate chain (19) ERROR: Root certificate is not trusted

To resolve the issue, run the following two commands:

gem sources --remove https://rubygems.org
gem sources --add http://rubygems.org

Now, try again to install the cnvrg CLI by running the following command:

gem install cnvrg

# Install Xcode Command Line Tools (MacOS)

If you are having repeated errors installing Ruby, you may not have the Xcode Command Line Tools installed on your MacOS machine. Run the following command and then try again:

xcode-select --install

NOTE

Sometimes, updating your version of MacOS requires a reinstallation of the Xcode Command Line Tools to fix common command line issues.

# Put the gem in your PATH

After installing the CLI, it is possible that the CLI's gem is not in your PATH. This may be the case, if after installing the CLI, when using a command you get the following error:

bash: cnvrg: command not found

To fix this issue:

Run the following command:

gem environment

Copy the path from EXECUTABLE DIRECTORY. It should end with /bin. For example, /usr/local/bundle/bin)
Run the following command, with the copied path from the above step:

export PATH=/path/to/executable/directory/bin:$PATH

For example, using the above example: export PATH=/usr/local/bundle/bin:$PATH
4. To ensure this problem is solved (and you do not need to repeat these steps), run the following command (with the same path used above) to automatically export the CLI's gem to the PATH every time you run a new terminal session:

# if using bash
echo 'export PATH="/path/to/executable/directory/bin:$PATH"' >> ~/.bashrc 

#if using zsh
echo 'export PATH="/path/to/executable/directory/bin:$PATH"' >> ~/.zshrc

# ContactingAI Studio

If the above methods have not resolved your issue, please contact theAI Studio using the online chat bubble at the bottom right of the cnvrg web app.

For quick and efficient support, please include the output of the following command in your request:

cnvrg logs

Support will help resolve your issue as quickly as possible.

← Python SDK v1 Tutorials and Examples →