Release Notes

Version v4.7.48

• Fix: Dataset Caching - Can't install cnvrg sdk from private PYPI repository

Version v4.7.47

• Fix: Containers - Can't pull image when default compute template is missing

Version v4.7.46

• Fix: Dataset - Can't clear cached commit
• Fix: Workspace - Metrics aren't displayed in Modin/Ray workspace
• Fix: Resource - labels displayed wrong in the status page

Version v4.7.45

• Fix: Workspace - Tensorboard process doesn't re-launch in case of workspace restarts

Version v4.7.44

• Fix: Dataset - Filepath query includes every file that ends with the provided text(regex: "*Text"), not only the specified file

Version v4.7.43

• Improvement: AWS resource - Show availability zones by instance type and region
• Fix: SDK - can't pull artifacts from aborted tasks using SDK
• Fix: Experiments - End sync gets aborted before it finishes
• Fix: Workspace - Resume Jupyter workspaces when they get killed due to oom
• Fix: Bluprints - Input example is not working
• Fix: MetaCloud organizations can't use AI Libraries
• Fix: Dashboard - Filter by Active Status from top bar of dashboard doesn't return correct results

Version v4.7.41

• Fix: Experiments - ray images are not displayed for jobs

Version v4.7.40

• Improvement: SDK - Add the option to create registries

Version v4.7.39

• Improvement: Define custome prometheus URL in worker cluster (built-in monitoring stack)
• Fix: SDK - Cached commit can be deleted using SDK when it is in use
• Fix: Experiment - Objective_reached tag is missing when running Hyper Optimization

Version v4.7.36

• New Feature: Compute Resource - Change the text to "bring your own kubernetes" on on premise resources
• Fix: AI Libraries - yaml files are corrupted

Version v4.7.32

• New Feature: SDK - add the option to create images
• Fix: SDK - Can't update default image in project's settings
• Fix: Containers - Disable the option to build image until a registry is added
• Fix: Endpoint - Endpoint keeps exporteing data after disabling Feedback loop
• Fix: Project - Loading project page stalls when it has a lot of activities
• Fix: Endpoint - Model upate isn't working for Endpoint which are created through SDK
• Fix: Experiment Table - Dictionary values in exported CSV files is against CSV format
• Fix: Endpoint - Stop and rollback isn't working in batch endpoint in case of canary rollout

Version v4.7.31

• New Feature: Compute Resource - Support eastwest integration for clusters
• Fix: Experiments - Experiments remains in debug status after the debug time is completed when running many jobs
• Fix: Workspace - Workspace isn't shut off on idle time (gpu metrics were not transmitted)

Version v4.7.29

• Fix: Experiment - "system" tab doesn't show HPU utilization metric in distributed jobs
• Fix: Experiment - "system" tab charts show unix time instead of human time on X-axis

Version v4.7.27

• Fix: Blueprints - Inference blueprints displays errors when chosen compute is not on the default cluster

Version v4.7.26

• Fix: General - User settings are not functioning in air-gapped environments
• Fix: General - Fix cnvrg logo
• Fix: Experiments - Allow managers to approve Human in the loop tasks

Version v4.7.23

• New Feature: SDK - Add caching functionality

Version v4.7.22

• New Feature: Git wizard - A new walkthrough connecting Git repositories to cnvrg
• New Feature: Project - Support Bitbucket token on git integration
• Fix: Flow - Custom tasks get the same default title ("Task 1")
• Fix: Experiments - Stop button is disable for experiments that are in 'pending approval' status

Version v4.7.20

• Improvement: Compute Resource - Add terms and usage for partners compute

Version v4.7.19

• Improvement: Projects index page - Improve load time
• Fix: Blueprints - Typo in Training flow welcome popup
• Fix: all "max duration reached" messages refer to endpoint
• Fix: Experiment table - Filtering by flow or flow version shows unrelated experiments
• Fix: SDK - Error when logging a tag with float or integer value
• Fix: Workspace - creating image from workspace text is white over white background
• Fix: Papers - Papers section can not be accessed after manipulating the chart
• Fix: Workspace - Tap on commit sha1 from workspace configuration tab redirects to same workspace in new tab
• Fix: SDK - Can't update max_restarts value on project's settings

Version v4.7.18

• Improvement: SDK - Add the option to create user and membership

Version v4.7.17

• Fix: Compute Resource - Delete resource button is diabled while there are running jobs on. instead it should pop-up message to confirm deletion
• Fix: Papers - Text is disappearing after opening the same paper url in another tab

Version v4.7.15

• Fix: Flow - User list is not returning on human validation condition

Version v4.7.14

• New Feature: SDK - Support e.pull_artifacts(wait_until_success=True)
• Improvement: Compute templates - Add the option to increase pod shared memory
• Fix: Experiment - Objective_reached tag is missing when running Hyper Optimization
• Fix: Compute Resource - Text typo on Min Workers field when Autoscaling is on

Version v4.7.11

• New Feature: SDK - Create cache link functionality
• New Feature: SDK - add the option to "merge commit to master" via sdk2
• Fix: SDK - dataset clone(),download() ignores empty folders

Version v4.7.10

• Fix: Dashboard - Can't filter by 'Online' status

Version v4.7.8

• Improvement: Restore cnvrg logo when pride month is finished (but will continue to be in our hearts)

Version v4.7.7

• Improvement: Endpoints - Add a warning when deleting a deploy task from the experiments table
• Improvement: Support organization names that are case-sensitive
• Fix: Compute - Fix nodegroup tags/taints/labels key value display in AWS resources
• Fix: Compute Dashboard - Pending mode jobs are not filtered as active jobs
• Fix: Endpoints - Endpoint type does not change after updating version

Version v4.7.6

• Improvement: Compute - Support adding shared memory (/dev/shm) to jobs

Version v4.7.5

• Improvement: SDK - Add an option to add users to the organization

Version v4.7.4

• New Feature: Support Partners compute integration to cnvrg Metacloud
• Improvement: Endpoint - Add an option to send files in "Try it Live" section

Version v4.7.3

• Fix: Compute - Compute type is displayed as "other" for default resource
• Fix: Workspaces - Artifacts font size is very small
• Fix: Compute Dashboard - Table is missing padding

Version v4.7.2

• New Feature: Support Partners compute integration to cnvrg Metacloud

Version v4.7.1

• New Feature: Support Partners compute integration to cnvrg Metacloud
• Fix: Experiments - "Stop Flow" button is missing in the flow diagram

Version v4.7.0

• New Feature: Bring Your Own Storage: GCS (Available in cnvrg Metacloud)
• Fix: In case of uppercase letters Flows creates duplicate endpoints with the same name, url doesn't work
• Fix: SDK - Can't upload files to public datasets

Version v4.6.0

• New Feature: Bring Your Own Storage: Azure Containers (Available in cnvrg Metacloud)
• New Feature: MetaGPU release: Allow sharing gpu between gpus in the same job

Version v4.5.0

• New Feature: Papers Release: A new space to analyze and share your work

Version v4.4.0

• New Feature: Blueprints Release: Open-source pre-configured ML pipelines that can be quickly integrated with any application (Available in cnvrg Metacloud)
• New Feature: Endpoints - Add an option to send request to endpoint from the UI

Version v4.3.0

New Feature: Billing and plans integration (Available in cnvrg Metacloud)

Version v4.2.0

• New Feature: Compute - Support Partners compute integration to cnvrg metacloud

Version v4.1.0

• New Feature: Metacloud Release
• New Feature: Bring Your Own Compute: AWS EKS (Available in cnvrg Metacloud)
• New Feature: Bring Your Own Storage: AWS S3 (Available in cnvrg Metacloud)
• Improvement: Compute - Add Hugepages option to compute templates
• Fix: User Settings - Not able to add user profile image from account settings

Version v4.0.0

• New Feature: Metacloud Release
• New Feature: Bring Your Own Compute: AWS EKS
• New Feature: Bring Your Own Storage: AWS S3

Version v3.12.26

• Improvement: Compute - Change default Hugepages in Gaudi templates to 1000Mi
• Fix: Endpoints - Endpoint should have "Online" status when its running

Version v3.12.25

• Fix: Experiments - Export experiments results with an error
• Fix: Experiments - Debug mode buttons are not working

Version v3.12.24

• Fix: Experiments - When running pytorch distributed jobs end sync gets aborted

Version v3.12.23

• Fix: Experiments - "Periodic sync" is not working

Version v3.12.22

• Fix: Experiments - When using "Last 7 days" filter it doesn't show results
• Fix: Flows - Max duration doesn't work
• Fix: Experiments - filter for: "lt,gt,=" doesn't work
• Fix: Dataset - "active N hours/weeks/months ago" doesn't change
• Fix: Endpoints - Wrong compute value is showing during stop and rollback in endpoint

Version v3.12.21

• Fix: Experiments - Can't run pytorch distributed jobs

Version v3.12.20

• Fix: Compute Dashboard - Node name is missing for jobs running on on-premise resources
• Fix: Workspaces - "Restart when idle" is not working when using gpu

Version v3.12.19

• Fix: Experiments - Can't filter experiments by tags that have a dot in the tag name
• Fix: Experiments - Running multiple experiments is slow when you have a project with 10k+ experiments
• Fix: Flows - Deploy task is stuck on "Initializing" state when trying to update an existing endpoint

Version v3.12.18

• Fix: Experiments - Downloading artifacts as a zip file ignores file's paths
• Fix: Workspaces - Fix permissions issue for users when running CVAT workspaces

Version v3.12.17

• Improvement: Endpoints - Include port in the request example for python
• Fix: SDK - Uploading files from the sdk while having an empty idx file causes files to not show up in the UI
• Fix: Project Settings - Update environment variables sometimes don't work
• Fix: Compute - Taints aren't passed to the pods when running pytorch distributed templates

Version v3.12.16

• Fix: Projects - Activity feed doesn't show latest feed

Version v3.12.15

• Improvement: Endpoints - Increase mx duration time to be 60 minutes
• Improvement: Projects - Improve projects home page loading time
• Fix: Experiments table - tag values for custom columns not displayed in UI

Version v3.12.14

• Fix: Compute - Can't build image on newly added compute resources
• Fix: Experiments - Custom columns are not displayed in the UI

Version v3.12.13

• Fix: SDK - Restart webapps command doesn't work
• Fix: Experiments - Chart doesn't load when using an uppercase in the title of the chart
• Fix: Compute Dashboard - Metrics are not populated when using internal prometheus

Version v3.12.12

• Improvement: Flows - Continual learning trigger for updated datasets takes latest commit by default
• Fix: Flows - Flow which includes only deploy tasks transitions to an empty experiment's page
• Fix: Workspaces - Search for datasets in a running workspace doesn't show all relevant results

Version v3.12.11

• Fix: SDK - Can't clone specific project's commit

Version v3.12.10

• Improvement: Images - Support dynamic spark workers images

Version v3.12.9

• Fix: Flows - Running flow with a single deploy task transitions to an empty experiments page
• Fix: Experiments - Endpoint task is displayed before the rest of the tasks, even though it was the last task to run
• Fix: Workspaces - When adding datasets to a workspace, not all datasets are searchable or visible in the drop down
• Fix: Endpoints - Continual learning alerts aren't triggered when condition is met

Version v3.12.8

• Improvement: System - Support parameter based prefixes for cnvrg subdomains when installing cnvrg

Version v3.12.7

• Fix: CLI - Running experiments with the --wait flag doesn't output logs to the terminal

Version v3.12.6

• Improvement: SDK - Add or delete secrets using the sdk
• Improvement: Experiments - Support integer and float params when running Bayesian Optimization
• Fix: SDK: Date is incorrect when updating secrets through the SDK
• Fix: SDK - e.log_param() doesn't support boolean values, and could cause a flow to be stuck
• Fix: Workspaces - Deleting a workspace will delete commits related to that workspace

Version v3.12.5

• Fix: Flows - Time is missing from UI when scheduling a flow

Version v3.12.4

• Improvement: SDK - Add an option to clone a specific project's commit
• Fix: Webapps - When SSO is enabled, webapp UI is not visible
• Fix: User Setting - changing data in user page has no validators
• Fix: Flows - Support clock changes for scheduled flows (Daylight Saving Time)

Version v3.12.3

• Improvement: System - Allow changing user address for emails sent by cnvrg system

Version v3.12.2

• Fix: Workspaces - Allow workspaces to start from experiments commit and sync changes to the project

Version v3.12.1

• Fix: Workspaces - "max duration exceeded" status remains after starting and stopping workspaces

Version v3.12.0

• New Feature: Endpoints - Added integration for Openvino model server

Version v3.11.14

• New Feature: Project Settings - Enable "local folder" to be visible only to admin ( Enabled by a feature flag)
• Improvement: Organization Settings - Improve creation of private pypi conf for all jobs
• Fix: Experiments - When syncing from sdk with local experiments, artifacts tab shows files from previous sync command

Version v3.11.13

• Fix: Project Home page - User's icons are not align when one of the users has image

Version v3.11.12

• Fix: Experiments - "copy command" does not include notify on error/success with emails
• Fix:Workspaces - Workspaces are stuck in a restarting loop when using environment variables that include space in its key

Version v3.11.11

• Improvement: Add option to add preconfigured registry and images when installing cnvrg
• Fix: Users - Forget password page is missing the button to reset password
• Fix: Experiments - "cnvrg_auth" command gets stuck in initializing stage

Version v3.11.10

• Fix: Workspaces - cvat tab is not clickable
• Fix: Experiments - Experiment sometime is stuck on "terminating" after end commit

Version v3.11.9

• Improvement: Dashboard - Improve metrics for organization's dashboard

Version v3.11.8

• Improvement: Workspaces - Update install commands for Vscode to support version v4.0.1+

Version v3.11.7

• Fix: Workspaces - webapps can't run from inside the workspace
• Fix: SDK - Allow updating image from project settings
• Fix: Experiments - Memory metrics can exceed 100%

Version v3.11.6

• Fix: Experiments - Load config table with empty parameters

Version v3.11.5

• Improvement: Experiments - Faster loading of hyper parameters experiments in the experiments' table

Version v3.11.4

• Improvement: Endpoints - Support endpoint with python 2.7
• Fix: Experiments - Wrong values when running grid search with scale log2
• Fix: Experiments - Deleting of experiment popup has wrong description
• Fix: Experiments - Wrong values when running grid search with scale log10 and floats
• Fix: Workspaces - Workspace is sometime stuck on "stopping"
• Fix: Datasets - Fix creating file from UI in a folder with + sign creates 2 folders

Version v3.11.3

• Fix: Existing volumes are not visible in the Volume drop down
• Fix: Drop down is not available in Human Validation in flows

Version v3.11.2

• Improvement: Flow - Allow task parameters to contain other task's variable and string to transfer dynamic parameters between tasks
• Fix: Endpoint - Continual learning alerts aren't triggered when condition is met
• Fix: Experiment - Can't view last few charts (out of 25) in "visual" tab
• Fix: Flow - Creating task in a flow with title containing dot can break the flow
• Fix: Dashboard - "no data" when super-admin from 2nd org views dashboard in 1st org

Version v3.11.1

• Improvement: Workspace - Sync automatically master and workers files when using OpenMPI
• Improvement: Add log_param functionality to experiment object
• Fix: Endpoint - Add Rabbitmq option to to the UI
• Fix: Deprecated jobs cab get stuck on GPU
• Fix: Project - Can't add administrator to project's collaborators
• Fix: Remove commits request from files page
• Fix: Changing the 'Working Directory' in project settings doesn't affect flow tasks
• Fix: Containers - "Build Image" job ignores template selected and takes default system-template instead

Version v3.11.0

• New Feature: Refactor Experiments table page - Improve loading time and usability

Version v3.10.15

• Improvement: Internal - Update release tags in cnvrg releases

Version v3.10.14

• Fix: Double v in cnvrg version's icon

Version v3.10.13

• Improvement: Change pip.conf creation from "append" to "overwrite"

Version v3.10.12

• Improvement: VScode workspace - install jupyter extension manually

Version v3.10.11

• Fix: WebApps - voila app strip-sources isn't visible in UI

Version v3.10.10

• Fix: Workspace - commits with large amount of files can cause slow load times for workspace page

Version v3.10.9

• New Feature: Experiment - Support HTML format (line breaks,bold,etc) for alerts messages

Version v3.10.8

• Fix: Verify experiments exit status is updated correctly in load

Version v3.10.7

• Fix: Delete flow url is shown wrong

Version v3.10.6

• Fix: hpu is not shown in the templates list as a description ( also for mpi )

Version v3.10.5

• Improvement: SDK: Deleting flow having ongoing task using flows.delete() not deleting the task
• Fix: Flow - timezone displayed in the 'schedule' tab is utc
• Fix: SDK - e.alert not working, 404 error
• Fix: Experiment - Printing massive amount of logs can cause idle time

Version v3.10.4

• Fix: Add liveness checks to webapps

Version v3.10.3

• Fix: "Tensorflow" endpoints can not be accessed from Serving Tab.

Version v3.10.2

• Fix: Add support for region when setting up an on premise bucket

Version v3.10.1

• New Feature: Add support for triton endpoints

Version v3.10.0

• New Feature: Add Queues and jobs priority - Add the ability to prioritize jobs and preempt others
• New Feature: Add option to hard delete artifacts, projects and datasets
• Improvement: Add auto update to all tables

Version v3.9.75

• New Feature: Refactor Dataset selector
• Improvement: Experiment Show - Add timestamp to Artifacts/Commits list
• Improvement: Add filter to volumes list
• Fix: Allocated cpu and memory are showing wrong in Dashboard when cpu and memory was edited in the template
• Fix: Table: Sorting for project field in dashboard and volumes table is working in reverse order
• Fix: Add loader in all tables to when switching states / filter is loading / refreshing page
• Fix: Type is showing empty for default cluster on list page

Version v3.9.74

• Fix: Auth - cant login with sso, blank page.

Version v3.9.73

• Fix: Remote SSH - global cluster-scope environment variables not available in remote ssh terminal

Version v3.9.72

• Fix: Template/Caching - Cant cache: System Template not created for a newly/old Cluster.

Version v3.9.71

• Fix: Experiment - using print("cnvrg_tag_:value) only extract small amount of the metrics

Version v3.9.70

• Fix: sdk - Experiment.list() returns error

Version v3.9.69

• Fix: Flow - duplicate tasks when running webhook/flow_version_runner

Version v3.9.68

• Fix: Remote SSH - global cluster-scope environment variables not available in remote ssh terminal

Version v3.9.67

• Fix: Endpoint - 2 Emails are sent when updating to new model in kafka endpoints

Version v3.9.66

• New Feature: organization-wide settings option to auto attach an nfs
• Improvement: Add commits table to experiment artifacts
• Improvement: Dataset selector - search for commits and queries
• Improvement: Workspace makeover of top bar
• Fix: "Batch predict" library goes to DEBUG mode, when we try to make a Batch endpoint live
• Fix: "End commit" is not showing for Experiment
• Fix: Search Issues in Experiment's table
• Fix: Flow- deploy task of rabbitmq UI BUG image is not resized on select
• Fix: Experiment - Categorical value separated with whitespaces accept only last char
• Fix: Sorting not working for fields "Architecture" and "Word embedding" in the experiment's table
• Fix: Experiment visuals - can't zoom in when trying to capture first data point
• Fix: Flow - 'emails_to_notify' field exist in the exec_task but doesn't reach the experiment

Version v3.9.65

• Fix: Experiment - Dataset link leads to 404 if there is a Hyphen in the dataset name
• Fix: Endpoint - is_deployment_running return false even if there is model up
• Fix: User that isn't a collaborator can't start jobs in a public project

Version v3.9.64

• New Feature: Show workspace names in delete confirmation message
• Improvement: Add support to "openssl_verify_mode" param in SMTP settings
• Fix: SSO: trying to add a user that has a dash in its username fails
• Fix: Workspace - MPI Template slotsPerWorker not effected by the configuration
• Fix: Endpoints - Logs are not pulled if a Prometheus authentication on the cluster not defined
• Fix: SDK- Update model will fail if there is uppercase letter in the organization name

Version v3.9.63

• New Feature: SDK - support new python SDK functionality
• Improvement: Add activity feed for deletion and uploading of files
• Fix: Workspace - sync indication in progress, not presented to user
• Fix: Unable to Rename workspace from workspace overview page
• Fix: Experiments are not available in tensorboard iframe
• Fix: Schedule Flow: In Every hr, Next run is not showing correct over DAG, also the time changes in flow
• Fix: Single_bar_chart and multi_bar_chart are not loading in visual tab for experiment
• Fix: On using deleted hyper experiment's volume in a new experiment, experiment overview page is not showing
• Fix: Webapps are not getting public when using SSO
• Fix: Flow: Pending approval is requested, even if flow version chosen does not have any pending approval in it
• Fix: Flow: Status of flow remains "running" on DAG forever, even the Deploy Task is success
• Fix: Favorites are not getting generated in the "Visual-quick view" of experiment overview page, when creating a new experiment.
• Fix: Experiments running from flow are not presented in experiment table
• Fix: Toggle webhook is not working in flows
• Fix: Charts of the same task name doesn't show names in compare

Version v3.9.62

• Fix: SDK- Update model will fail if there is uppercase letter in the organization name

Version v3.9.61

• Fix: SDK - endpoint.is_deployment_running() failing with streaming endpoint

Version v3.9.60

• Improvement: Spark- Share /data and files across workers and master

Version v3.9.59

• Fix: Dataset - "cnvrg data put" to a public dataset gets "unauthorized" error
• Fix: NEW SDK - User gets "server error" on cnvrgp dataset clone, when a second user is pending registration
• Fix: File previewer in 'Files' doesn't render non-english text properly
• Fix: Workspace - "copy clone command" from commit copies "undefined" to clipboard
• Fix: Endpoints fail to update version on nginx deployments

Version v3.9.58

• Improvement: Dashboard export optimization
• Improvement: Add claim name to external disk api
• Fix: Private pypi doesn't affect all the dependencies in a job
• Fix: Experiments Compare doesn't work when started from the SDK using Experiment.init()

Version v3.9.57

• Fix: Rabbitmq endpoint - 'Prefetch Count' params is getting reset to nil with flow/deploy task

Version v3.9.56

• Improvement: support for random namespace in remote cluster
• Improvement: Change user logo from initials to full username
• Fix: Not able to cache the dataset
• Fix: Endpoint - Rabbit endpoint stuck on "updating in progress" when Updating model from flow
• Fix: System charts fail to load (UI race condition)
• Fix: Experiment - Accuracy value and tag not pulled from logs
• Fix: Grid Search - objective name doesn't accept underscores
• Fix: Dataset - Caching commit when user doesn't have permissions to the template

Version v3.9.55

• Fix: Flow - parsing error when connecting 2 tasks
• Fix: Flow - task got terminated by the monitor, following task hangs on "initializing" with no logs
• Fix: System charts fail to load in some cases

Version v3.9.54

• Fix: Experiment - debug mode isn't working in spark experiments

Version v3.9.53

• FIx: Docker container id was missing from grid experiments
• Improvement: Dataset - change get started with cnvrg CLI example

Version v3.9.52

• Fix: Flow - Emails on error sent only the user who lunched the flow

Version v3.9.51

• Improvement: Parse logs field in elasticsearch as JSON

Version v3.9.50

• Fix: Experiment - Parallel jobs are not working in bayesian optimization

Version v3.9.49

• Improvement: improve load times when updating experiments fields

Version v3.9.48

• Fix: User that isn't an admin can't cache a public dataset

Version v3.9.47

• Improvement: Support R endpoints with latest rocker/rstudio image

Version v3.9.46

• Fix: Containers- Pulling image without default compute template results with 401 error

Version v3.9.45

• Fix: Public projects are not visible to all users

Version v3.9.44

• Fix: Load on the DB caused experiment that ended with status code 0 to enter debug mode

Version v3.9.43

• Improvement: Private PyPi repo configure automatically when connecting to a workspace

Version v3.9.42

• Fix: CLI - Running a flow with deploy task fails
• Fix: Experiment jupyter debug dot is still shown in success
• Improvement: Compute Dashboard export optimization

Version v3.9.41

• Fix: Install R dependencies for endpoints

Version v3.9.40

• Improvement: Export dashboard CSV using stream

Version v3.9.39

• Improvement: GridSearch default values for parallel jobs should be unlimited

Version v3.9.38

• Fix: Hyper view should show scheduled time with regards to the user's time zone

Version v3.9.37

• Improvement: GridSearch default values for parallel jobs should be unlimited

Version v3.9.36

• Fix: Remove validations from experiment hyper search form

Version v3.9.35

• Fix: Emails notifications from cli are not saved

Version v3.9.34

• Fix: Fix dynamic routes in tasks

Version v3.9.33

• Fix: Experiment running rscript doesn't install jupyter for debug mode

Version v3.9.32

• Fix: SDK - No email validation applies when registering a new user

Version v3.9.31

• Improvement: Add reindex button to dataset commit
• Fix: Compare different versions of a files doesn't work properly
• Fix: Experiment - Can't filter by tags that have dot in their name
• Fix: Experiment - Changing experiment's title (via stdout) doesn't support whitespaces
• Fix: Endpoint - New Alert pop up is missing flow options
• Fix: Experiment - Debug icon is still on when experiment is in error
• Fix: Endpoint - "Invalid Date" on time column when starting an endpoint
• Fix: Experiment - Can't right-click save-as to download file in the artifacts tab
• Fix: Ray compute can't start with volume/pv
• Fix: Endpoint - Flow name is missing in continual learning tab when we create new trigger for flow
• Fix: "Default Homepage" drop down does not show all the organizations name.
• Fix: Experiment - "stop & sync" should not be available for local experiments
• Fix: SDK - The function e.finish(exit_status=-1) exits with wrong status
• Fix: Experiment - Charts take long time to load when there are a lot of logs
• Fix: Experiment - Search in config and metrics tab of experiment's overview page is case sensitive
• Fix: Compute Templates - amount of HPU in worker spec disappears from OpenMPI template
• Fix: Experiment - Clone artifacts fails on "Not Authorize, Are you logged in?"
• Fix: After clicking "Link to git repository", it goes to "General" tab instead of "Git Integration".
• Fix: Volumes- Filter by size removes the volumes from the UI
• Fix: Dashboard: Endpoints with status "stopped" due to "max duration" are not shown in the list of jobs
• Fix: Build image from workspace - agent container ID doesn't update when re-running workspace
• Fix: Missing validation when deleting a compute template that is used as the default compute for the organization

Version v3.9.30

• Improvement: Support multi tenant terms of use

Version v3.9.29

• Improvement: SDK - Support remove files and directories commands

Version v3.9.28

• Improvement: Add support to multi tenant terms of use
• Fix: User setting doesn't update automatically after saving Git Access Token

Version v3.9.27

• Fix: Can't pull agent container from default docker hub registry

Version v3.9.26

• Fix: Training Flow is missing in continual learning Project

Version v3.9.25

• Fix: SDK - When running experiments with grid search parameters, parameters are not showing in the UI
• Fix: Dashboard - exported csv file broken when a compute has multiple node-selectors

Version v3.9.24

• Improvement: Add 2 containers to the default intel registry

Version v3.9.23

• Fix: Folder names with special characters aborts sync
• Fix: Experiment - wrong parsing when printing tags with parenthesis in the key name
• Fix: Compute template that has a dot in it's name fails to launch flow using cli/trigger
• Fix: Volumes - volumes aren't deleted when a project is deleted
• Fix: Color of recurring status is wrong when creating hyper experiment with recurring option
• Fix: Tensorboard icon showing in enable mode when workspace is offline
• Fix: Workspace - Can't start a workspace with a volume that was deleted
• Fix: Periodic sync is not working in flows

Version v3.9.22

• Fix: Endpoint scale horizontally functionality doesn't work in RabbitMQ endpoints

Version v3.9.21

• Fix: Flow - Show an error if a flow is executed with parameters that don't have a key
• Fix: Flow - Failed validation closes the YAML editor
• Fix: SDK - flow.pull_artifacts returns "internal server error"

Version v3.9.20

• Improvement: Add support to global secrets
• Improvement: Change the acknowledgment of the receiving messages in RabbitMQ
• Fix: Users API should return access_level as a string instead of a number
• Fix: Data scientist can't clone public datasets that were created by admin

Version v3.9.19

• Fix: Support CVAT with https

Version v3.9.18

• Fix: Fix compare commits

Version v3.9.17

• Improvement: Support MIG configuration of 10GB
• Improvement: Support filter by node in the dashboard
• Improvement: Filter in experiments is not saved when going back and forth in the web
• Fix: Title of experiment is not link to experiment show page
• Fix: Can't see all workspace logs after workspace restart
• Fix: Experiments charts from sdk aren't parsed correctly
• Fix: Endpoint's predictions are not showing after 10k of predictions
• Fix: Endpoints log is in endless loop in case elasticsearch returns an error code

Version v3.9.16

• Improvement: Support labels from config-map
• Fix: Timestamp od last workspace sync is not shown
• Fix: Restarting endpoint with many logs loads a blank page
• Fix: Experiments list is taking too long to load
• Fix: Endpoint metrics are being polled in an asc order and not desc
• Fix: Experiment's duration is squeezed to multiple lines when it's too long

Version v3.9.15

• Fix: CVAT server doesn't run from within a script

Version v3.9.14

• Improvement: Wrong error when inviting a user and the user doesnt exist
• Fix: Can't create tag with the name: "Accuracy"
• Fix: Missing validation on environment variables keys ( allows empty values)
• Fix: Metric with underscore is not shown in the GridSearch view
• Fix: Running experiment with parameter that includes a comma runs as Grid Search
• Fix: SDK method fv.info() doesn't returns a response
• Fix: Experiment that reached idle time while initializing gets stuck trying to restart

Version v3.9.13

• Fix: Experiment - "zoom out" or "reset zoom" button is missing in charts

Version v3.9.12

• Improvement: Disable git clone in Tensorboard compare session

Version v3.9.11

• Fix: Experiment - Charts refresh is too fast

Version v3.9.10

• Fix: Support existing users in cvat

Version v3.9.9

• Fix: Flow - Environment variables of artifacts input folders are not passed to all the connected flows

Version v3.9.8

• Improvement: Run updates of cli asynchronously

Version v3.9.7

• Fix: Clone projects is not necessary in CVAT workspaces

Version v3.9.6

• Improvement: Support ssh to job in an openshift environment

Version v3.9.5

• Improvement: Support Prometheus and Elasticsearch secrets

Version v3.9.4

• Fix: Export Dashboard is very slow

Version v3.9.3

• Improvement: Support specific annotations to cnvrg jobs

Version v3.9.2

• Improvement: Automatically recognize when Kafka is offline without affecting the running endpoint
• Improvement: Support SDK v2 workflow version

Version v3.9.1

• Improvement: Run with the same docker user in the agent container

Version v3.8.5

• Improvement: Remove 'source' column from artifacts table

Version v3.8.2

• Feature: CVAT integration in workspace
• Fix: mpi/pytorch templates with gpu doesnt show gpu stats

Version v3.8.1

• Improvement: Improve organization dashboard stats load time
• Fix: File preview is not shown in Input and Output artifacts
• Fix: Fix pytorch distributed gpu count in command
• Fix: Remove auto-complete from dataset selector
• Fix: Workspace - collaborators can't run someone else's workspace

Version v3.8

• Feature: Support multi tenancy integration
• Feature: Support OpenShift integration
• Feature: Support ML Infra Dashboard
• Improvement: Support secrets with prometheus
• Improvement: Support vhost in rabbitmq
• Fix: Example project yaml files are invalid
• Fix: Utilized gpu/hpu is showing "NAN%" instead of 0%.
• Fix: Experiments charts are hidden when playing with the order & size

Version v3.7

• Feature: Support Rabbitmq in endpoints
• Feature: ADD permissions to compute templates
• Fix: Experiment notifications not sent to slack webhook
• Fix: Export experiments table to a csv generates too many column of grid search IDs
• Fix: Fix compare of commit without parent
• Fix: Flow - no validation of data types when directly pasting flow YAML
• Fix: support download specific commit
• Fix: sdk/not able to stop webapps, workspace, and experiments
• Fix: support all types of mig devices

Version v3.6.13

• Feature: Support TFServing in endpoints
• Improvement: Add files sizes to get commit files response
• Improvement: Library show page fails on missing compute
• Fix: Access token doesn't update when re-logging in to a different user
• Fix: Webapps GET requests sent from within the webapp fails on timeout
• Fix: Datasets page appear blank in safari
• Fix: SDK: flow.pull_artifacts returns an error
• Fix: SDK: e.pull_artifacts() returns an error

Version v3.6.12

• Feature: Replace terminal section on sidebar to jupyter in experiments
• Improvement: Add compute name in volumes selector
• : Webapp iframe url stays consistent throughout deployments
• Fix: Webhook in flow changes parameter type from categorical to discrete
• Fix: Experiment with parenthesis in key name gets stuck in a restarting loop
• Fix: Organization Settings - colon in pypi repo URL doesnt pass validation
• Fix: Editing a file through the UI works only the first time

Version v3.6.10

• New Feature: Experiment New Dashboard
• New Feature: Add flow dag to experiment page
• New Feature: Change mount path when attaching PVC as local folder
• New Feature: Support params to flows from webhook
• Improvement: Allow custom pypi to be used for installing requirements
• Improvement: Support log images in experiments
• Improvement: add ability to drag charts in quick view
• Improvement: add search and sort to visuals tab
• Improvement: Support dot in chart title
• Fix: Validation in tasks is not working if parallel job count field is greater than max job field
• Fix: Creating template using specific node label raises an error
• Fix: Saving a task as a library raises an error
• Fix: Flow - notify on success/error set to true in yaml but email is not sent
• Fix: cant push new images if repo exist in ECR
• Fix: Flow- 2 compute names from different clusters causes the flow to not run
• Fix: SDK: running an experiment with arguments ignores the compute
• Fix: SDK: running an experiment with non-existent compute templates defaults to organization settings instead of project settings
• Fix: Compute is showing as "unassigned" for scheduled experiment until manual refresh
• Fix: Datasets is not downloaded in endpoint when endpoint is run from flow

Version v3.6.9

• Feature: Support Oauth Proxy
• Feature: ADD mig (Multi instance gpu) Support
• Feature: ADD Ray integration
• Feature: ADD permissions to compute templates
• Improvement: Dataset - JSON files displayed without indentation
• Improvement: Retrieve organization name from URL In the websocket
• Improvement: Support cluster restart and restart pods lifecycle
• Improvement: SDK: Support Experiment.alert from sdk
• Improvement: CLi: improve mechanism of removing many folders in dataset
• Improvement: Update workspace title to be Workspace
• Improvement: support global gateway
• Improvement: Add Custom Scheduler option
• Fix: Can't restart jobs with PCV
• Fix: Dashboard graph is not presented
• Fix: mpi workers don't pull secrets

Version v3.6.8

• Feature: Support HyperParameter tuning for experiments and flows: Grid Search, Random Search, Bayesian
• Fix: CLI: Experiments are always taking folder as output even if we are running exp on custom folder
• Improvement: Change error message when attempting to delete a predefined image
• Fix: Fix View of commit in the browser
• Fix: flow is starting experiments in "waiting mode"
• Fix: Fix default timezone for users
• Fix: SDK: Can't authenticate in windows
• Fix: SDK: Fetch all files in dataset returns error