cnvrg.io docs

# Release Notes

# Version v3.11.6

  • Fix: Experiments - Load config table with empty parameters

# Version v3.11.5

  • Improvement: Experiments - Faster loading of hyper paramaters experiments in the experiemnt's table

# Version v3.11.4

  • Improvement: Endpoints - Support endpoint with python 2.7
  • Fix: Experiments - Wrong values when running grid search with scale log2
  • Fix: Experiments - Deleting of experiment popup has wrong description
  • Fix: Experiments - Wrong values when running grid search with scale log10 and floats
  • Fix: Workspaces - Workspace is sometime stuck on "stopping"
  • Fix: Datasets - Fix creating file from UI in a folder with + sign creates 2 folders

# Version v3.11.3

  • Fix: Existing volumes are not visible in the Volume drop down
  • Fix: Drop down is not available in Human Validation in flows

# Version v3.11.2

  • Improvement: Flow - Allow task parameters to contain other task's variable and string to transfer dynamic parameters between tasks
  • Fix: Endpoint - Continual learning alerts aren't triggered when condition is met
  • Fix: Experiment - Can't view last few charts (out of 25) in "visual" tab
  • Fix: Flow - Creating task in a flow with title containing dot can break the flow
  • Fix: Dashboard - "no data" when super-admin from 2nd org views dashboard in 1st org

# Version v3.11.1

  • Improvement: Workspace - Sync automatically master and workers files when using OpenMPI
  • Improvement: Add log_param functionallity to experiment object
  • Fix: Endpoint - Add Rabbitmq option to to the UI
  • Fix: Deprecated jobs cab get stuck on GPU
  • Fix: Project - Can't add administrator to project's collaborators
  • Fix: Remove commits request from files page
  • Fix: Changing the 'Working Directory' in project settings doesn't affect flow tasks
  • Fix: Containers - "Build Image" job ignores template selected and takes default system-template instead

# Version v3.11.0

  • New Feature: Refactor Experiments table page - Improve loading time and usability

# Version v3.10.15

  • Improvement: Internal - Update release tags in cnvrg releases

# Version v3.10.14

  • Fix: Double v in cnvrg version's icon

# Version v3.10.13

  • Improvement: Change pip.conf creation from "append" to "overwrite"

# Version v3.10.12

  • Improvement: VScode workspace - install jupyter extension manually

# Version v3.10.11

  • Fix: WebApps - voila app strip-sources isn't visible in UI

# Version v3.10.10

  • Fix: Workspace - commits with large amount of files can cause slow load times for workspace page

# Version v3.10.9

  • New Feature: Experiment - Support HTML format (line breaks,bold,etc) for alerts messages

# Version v3.10.8

  • Fix: Verify experiments exit status is updated correctly in load

# Version v3.10.7

  • Fix: Delete flow url is shown wrong

# Version v3.10.6

  • Fix: hpu is not shown in the templates list as a description ( also for mpi )

# Version v3.10.5

  • Improvement: SDK: Deleting flow having ongoing task using flows.delete() not deleting the task
  • Fix: Flow - timezone displayed in the 'schedule' tab is utc
  • Fix: SDK - e.alert not working, 404 error
  • Fix: Experiment - Printing massive amount of logs can cause idle time

# Version v3.10.4

  • Fix: Add liveness checks to webapps

# Version v3.10.3

  • Fix: "Tensorflow" endpoints can not be accessed from Serving Tab.

# Version v3.10.2

  • Fix: Add support for region when setting up an on premise bucket

# Version v3.10.1

  • New Feature: Add support for triton endpoints

# Version v3.10.0

  • New Feature: Add Queues and jobs priority - Add the ability to prioritize jobs and preempt others
  • New Feature: Add option to hard delete artifacts, projects and datasets
  • Improvement: Add auto update to all tables

# Version v3.9.75

  • New Feature: Refactor Dataset selector
  • Improvement: Experiment Show - Add timestamp to Artifacts/Commits list
  • Improvement: Add filter to volumes list
  • Fix: Allocated cpu and memory are showing wrong in Dashboard when cpu and memory was edited in the template
  • Fix: Table: Sorting for project field in dashboard and volumes table is working in reverse order
  • Fix: Add loader in all tables to when switching states / filter is loading / refreshing page
  • Fix: Type is showing empty for default cluster on list page

# Version v3.9.74

  • Fix: Auth - cant login with sso, blank page.

# Version v3.9.73

  • Fix: Remote SSH - global cluster-scope environment variables not available in remote ssh terminal

# Version v3.9.72

  • Fix: Template/Caching - Cant cache: System Template not created for a newly/old Cluster.

# Version v3.9.71

  • Fix: Experiment - using print("cnvrg_tag_:value) only extract small amount of the metrics

# Version v3.9.70

  • Fix: sdk - Experiment.list() returns error

# Version v3.9.69

  • Fix: Flow - duplicate tasks when running webhook/flow_version_runner

# Version v3.9.68

  • Fix: Remote SSH - global cluster-scope environment variables not available in remote ssh terminal

# Version v3.9.67

  • Fix: Endpoint - 2 Emails are sent when updating to new model in kafka endpoints

# Version v3.9.66

  • New Feature: organization-wide settings option to auto attach an nfs
  • Improvement: Add commits table to experiment artifacts
  • Improvement: Dataset selector - search for commits and queries
  • Improvement: Workspace makeover of top bar
  • Fix: "Batch predict" library goes to DEBUG mode, when we try to make a Batch endpoint live
  • Fix: "End commit" is not showing for Experiment
  • Fix: Search Issues in Experiment's table
  • Fix: Flow- deploy task of rabbitmq UI BUG image is not resized on select
  • Fix: Experiment - Categorical value separated with whitespaces accept only last char
  • Fix: Sorting not working for fields "Architecture" and "Word embedding" in the experiment's table
  • Fix: Experiment visuals - can't zoom in when trying to capture first data point
  • Fix: Flow - 'emails_to_notify' field exist in the exec_task but doesn't reach the experiment

# Version v3.9.65

  • Fix: Experiment - Dataset link leads to 404 if there is a Hyphen in the dataset name
  • Fix: Endpoint - is_deployment_running return false even if there is model up
  • Fix: User that isn't a collaborator can't start jobs in a public project

# Version v3.9.64

  • New Feature: Show workspace names in delete confirmation message
  • Improvement: Add support to "openssl_verify_mode" param in SMTP settings
  • Fix: SSO: trying to add a user that has a dash in its username fails
  • Fix: Workspace - MPI Template slotsPerWorker not effected by the configuration
  • Fix: Endpoints - Logs are not pulled if a Prometheus authentication on the cluster not defined
  • Fix: SDK- Update model will fail if there is uppercase letter in the organization name

# Version v3.9.63

  • New Feature: SDK - support new python SDK functionality
  • Improvement: Add activity feed for deletion and uploading of files
  • Fix: Workspace - sync indication in progress, not presented to user
  • Fix: Unable to Rename workspace from workspace overview page
  • Fix: Experiments are not available in tensorboard iframe
  • Fix: Schedule Flow: In Every hr, Next run is not showing correct over DAG, also the time changes in flow
  • Fix: Single_bar_chart and multi_bar_chart are not loading in visual tab for experiment
  • Fix: On using deleted hyper experiment's volume in a new experiment, experiment overview page is not showing
  • Fix: Webapps are not getting public when using SSO
  • Fix: Flow: Pending approval is requested, even if flow version chosen does not have any pending approval in it
  • Fix: Flow: Status of flow remains "running" on DAG forever, even the Deploy Task is success
  • Fix: Favorites are not getting generated in the "Visual-quick view" of experiment overview page, when creating a new experiment.
  • Fix: Experiments running from flow are not presented in experiment table
  • Fix: Toggle webhook is not working in flows
  • Fix: Charts of the same task name doesn't show names in compare

# Version v3.9.62

  • Fix: SDK- Update model will fail if there is uppercase letter in the organization name

# Version v3.9.61

  • Fix: SDK - endpoint.is_deployment_running() failing with streaming endpoint

# Version v3.9.60

  • Improvement: Spark- Share /data and files across workers and master

# Version v3.9.59

  • Fix: Dataset - "cnvrg data put" to a public dataset gets "unauthorized" error
  • Fix: NEW SDK - User gets "server error" on cnvrgp dataset clone, when a second user is pending registration
  • Fix: File previewer in 'Files' doesn't render non-english text properly
  • Fix: Workspace - "copy clone command" from commit copies "undefined" to clipboard
  • Fix: Endpoints fail to update version on nginx deployments

# Version v3.9.58

  • Improvement: Dashboard export optimization
  • Improvement: Add claim name to external disk api
  • Fix: Private pypi doesn't affect all the dependencies in a job
  • Fix: Experiments Compare doesn't work when started from the SDK using Experiment.init()

# Version v3.9.57

  • Fix: Rabbitmq endpoint - 'Prefetch Count' params is getting reset to nil with flow/deploy task

# Version v3.9.56

  • Improvement: support for random namespace in remote cluster
  • Improvement: Change user logo from initials to full username
  • Fix: Not able to cache the dataset
  • Fix: Endpoint - Rabbit endpoint stuck on "updating in progress" when Updating model from flow
  • Fix: System charts fail to load (UI race condition)
  • Fix: Experiment - Accuracy value and tag not pulled from logs
  • Fix: Grid Search - objective name doesn't accept underscores
  • Fix: Dataset - Caching commit when user doesn't have permissions to the template

# Version v3.9.55

  • Fix: Flow - parsing error when connecting 2 tasks
  • Fix: Flow - task got terminated by the monitor, following task hangs on "initializing" with no logs
  • Fix: System charts fail to load in some cases

# Version v3.9.54

  • Fix: Experiment - debug mode isn't working in spark experiments

# Version v3.9.53

  • FIx: Docker container id was missing from grid experiments
  • Improvement: Dataset - change get started with cnvrg CLI example

# Version v3.9.52

  • Fix: Flow - Emails on error sent only the user who lunched the flow

# Version v3.9.51

  • Improvement: Parse logs field in elasticsearch as JSON

# Version v3.9.50

  • Fix: Experiment - Parallel jobs are not working in bayesian optimization

# Version v3.9.49

  • Improvement: improve load times when updating experiments fields

# Version v3.9.48

  • Fix: User that isn't an admin can't cache a public dataset

# Version v3.9.47

  • Improvement: Support R endpoints with latest rocker/rstudio image

# Version v3.9.46

  • Fix: Containers- Pulling image without default compute template results with 401 error

# Version v3.9.45

  • Fix: Public projects are not visible to all users

# Version v3.9.44

  • Fix: Load on the DB caused experiment that ended with status code 0 to enter debug mode

# Version v3.9.43

  • Improvement: Private PyPi repo configure automatically when connecting to a workspace

# Version v3.9.42

  • Fix: CLI - Running a flow with deploy task fails
  • Fix: Experiment jupyter debug dot is still shown in success
  • Improvement: Compute Dashboard export optimization

# Version v3.9.41

  • Fix: Install R dependencies for endpoints

# Version v3.9.40

  • Improvement: Export dashboard CSV using stream

# Version v3.9.39

  • Improvement: GridSearch default values for parallel jobs should be unlimited

# Version v3.9.38

  • Fix: Hyper view should show scheduled time with regards to the user's time zone

# Version v3.9.37

  • Improvement: GridSearch default values for parallel jobs should be unlimited

# Version v3.9.36

  • Fix: Remove validations from experiment hyper search form

# Version v3.9.35

  • Fix: Emails notifications from cli are not saved

# Version v3.9.34

  • Fix: Fix dynamic routes in tasks

# Version v3.9.33

  • Fix: Experiment running rscript doesn't install jupyter for debug mode

# Version v3.9.32

  • Fix: SDK - No email validation applies when registering a new user

# Version v3.9.31

  • Improvement: Add reindex button to dataset commit
  • Fix: Compare different versions of a files doesn't work properly
  • Fix: Experiment - Can't filter by tags that have dot in their name
  • Fix: Experiment - Changing experiment's title (via stdout) doesn't support whitespaces
  • Fix: Endpoint - New Alert pop up is missing flow options
  • Fix: Experiment - Debug icon is still on when experiment is in error
  • Fix: Endpoint - "Invalid Date" on time column when starting an endpoint
  • Fix: Experiment - Can't right-click save-as to download file in the artifacts tab
  • Fix: Ray compute can't start with volume/pv
  • Fix: Endpoint - Flow name is missing in continual learning tab when we create new trigger for flow
  • Fix: "Default Homepage" drop down does not show all the organizations name.
  • Fix: Experiment - "stop & sync" should not be available for local experiments
  • Fix: SDK - The function e.finish(exit_status=-1) exits with wrong status
  • Fix: Experiment - Charts take long time to load when there are a lot of logs
  • Fix: Experiment - Search in config and metrics tab of experiment's overview page is case sensitive
  • Fix: Compute Templates - amount of HPU in worker spec disappears from OpenMPI template
  • Fix: Experiment - Clone artifacts fails on "Not Authorize, Are you logged in?"
  • Fix: After clicking "Link to git repository", it goes to "General" tab instead of "Git Integration".
  • Fix: Volumes- Filter by size removes the volumes from the UI
  • Fix: Dashboard: Endpoints with status "stopped" due to "max duration" are not shown in the list of jobs
  • Fix: Build image from workspace - agent container ID doesn't update when re-running workspace
  • Fix: Missing validation when deleting a compute template that is used as the default compute for the organization

# Version v3.9.30

  • Improvement: Support multi tenant terms of use

# Version v3.9.29

  • Improvement: SDK - Support remove files and directories commands

# Version v3.9.28

  • Improvement: Add support to multi tenant terms of use
  • Fix: User setting doesn't update automatically after saving Git Access Token

# Version v3.9.27

  • Fix: Can't pull agent container from default docker hub registry

# Version v3.9.26

  • Fix: Training Flow is missing in continual learning Project

# Version v3.9.25

  • Fix: SDK - When running experiments with grid search parameters, parameters are not showing in the UI
  • Fix: Dashboard - exported csv file broken when a compute has multiple node-selectors

# Version v3.9.24

  • Improvement: Add 2 containers to the default intel registry

# Version v3.9.23

  • Fix: Folder names with special characters aborts sync
  • Fix: Experiment - wrong parsing when printing tags with parenthesis in the key name
  • Fix: Compute template that has a dot in it's name fails to launch flow using cli/trigger
  • Fix: Volumes - volumes aren't deleted when a project is deleted
  • Fix: Color of recurring status is wrong when creating hyper experiment with recurring option
  • Fix: Tensorboard icon showing in enable mode when workspace is offline
  • Fix: Workspace - can't start a workspace with a volume that doesn't exist
  • Fix: Periodic sync is not working in flows

# Version v3.9.22

  • Fix: Endpoint scale horizontally functionality doesn't work in RabbitMQ endpoints

# Version v3.9.21

  • Fix: Flow - Show an error if a flow is executed with parameters that don't have a key
  • Fix: Flow - Failed validation closes the YAML editor
  • Fix: SDK - flow.pull_artifacts returns "internal server error"

# Version v3.9.20

  • Improvement: Add suport to global secrets
  • Improvement: Change the acknowledgment of the receiving messages in RabbitMQ
  • Fix: Users API should return access_level as a string instead of a number
  • Fix: Data scientist can't clone public datasets that were created by admin

# Version v3.9.19

  • Fix: Support CVAT with https

# Version v3.9.18

  • Fix: Fix compare commits

# Version v3.9.17

  • Improvement: Support MIG configuration of 10GB
  • Improvement: Support filter by node in the dashboard
  • Improvement: Filter in experiments is not saved when going back and forth in the web
  • Fix: Title of experiment is not link to experiment show page
  • Fix: Can't see all workspace logs after workspace restart
  • Fix: Experiments charts from sdk aren't parsed correctly
  • Fix: Endpoint's predictions are not showing after 10k of predictions
  • Fix: Endpoints log is in endless loop in case elasticsearch returns an error code

# Version v3.9.16

  • Improvement: Support labels from config-map
  • Fix: Timestamp od last workspace sync is not shown
  • Fix: Restarting endpoint with many logs loads a blank page
  • Fix: Experiments list is taking too long to load
  • Fix: Endpoint metrics are being polled in an asc order and not desc
  • Fix: Experiment's duration is squeezed to multiple lines when it's too long

# Version v3.9.15

  • Fix: CVAT server doesn't run from within a script

# Version v3.9.14

  • Improvement: Wrong error when inviting a user and the user doesnt exist
  • Fix: Can't create tag with the name: "Accuracy"
  • Fix: Missing validation on environment variables keys ( allows empty values)
  • Fix: Metric with underscore is not shown in the GridSearch view
  • Fix: Running experiment with parameter that includes a comma runs as Grid Search
  • Fix: SDK method fv.info() doesn't returns a response
  • Fix: Experiment that reached idle time while initializing gets stuck trying to restart

# Version v3.9.13

  • Fix: Experiment - "zoom out" or "reset zoom" button is missing in charts

# Version v3.9.12

  • Improvement: Disable git clone in Tensorboard compare session

# Version v3.9.11

  • Fix: Experiment - Charts refresh is too fast

# Version v3.9.10

  • Fix: Support existing users in cvat

# Version v3.9.9

  • Fix: Flow - Environment variables of artifacts input folders are not passed to all the connected flows

# Version v3.9.8

  • Improvement: Run updates of cli asynchronously

# Version v3.9.7

  • Fix: Clone projects is not necessary in CVAT workspaces

# Version v3.9.6

  • Improvement: Support ssh to job in an openshift environment

# Version v3.9.5

  • Improvement: Support Prometheus and Elasticsearch secrets

# Version v3.9.4

  • Fix: Export Dashboard is very slow

# Version v3.9.3

  • Improvement: Support specific annotations to cnvrg jobs

# Version v3.9.2

  • Improvement: Automatically recognize when Kafka is offline without affecting the running endpoint
  • Improvement: Support SDK v2 workflow version

# Version v3.9.1

  • Improvement: Run with the same docker user in the agent container

# Version v3.8.5

  • Improvement: Remove 'source' column from artifacts table

# Version v3.8.2

  • Feature: CVAT integration in workspace
  • Fix: mpi/pytorch templates with gpu doesnt show gpu stats

# Version v3.8.1

  • Improvement: Improve organization dashboard stats load time
  • Fix: File preview is not shown in Input and Output artifacts
  • Fix: Fix pytorch distributed gpu count in command
  • Fix: Remove auto-complete from dataset selector
  • Fix: Workspace - collaborators can't run someone else's workspace

# Version v3.8

  • Feature: Support multi tenancy integration
  • Feature: Support OpenShift integration
  • Feature: Support ML Infra Dashboard
  • Improvement: Support secrets with prometheus
  • Improvement: Support vhost in rabbitmq
  • Fix: Example project yaml files are invalid
  • Fix: Utilized gpu/hpu is showing "NAN%" instead of 0%.
  • Fix: Experiments charts are hidden when playing with the order & size

# Version v3.7

  • Feature: Support Rabbitmq in endpoints
  • Feature: ADD permissions to compute templates
  • Fix: Experiment notifications not sent to slack webhook
  • Fix: Export experiments table to a csv generates too many column of grid search IDs
  • Fix: Fix compare of commit without parent
  • Fix: Flow - no validation of data types when directly pasting flow YAML
  • Fix: support download specific commit
  • Fix: sdk/not able to stop webapps, workspace, and experiments
  • Fix: support all types of mig devices

# Version v3.6.13

  • Feature: Support TFServing in endpoints
  • Improvement: Add files sizes to get commit files response
  • Improvement: Library show page fails on missing compute
  • Fix: Access token doesn't update when re-logging in to a different user
  • Fix: Webapps GET requests sent from within the webapp fails on timeout
  • Fix: Datasets page appear blank in safari
  • Fix: SDK: flow.pull_artifacts returns an error
  • Fix: SDK: e.pull_artifacts() returns an error

# Version v3.6.12

  • Feature: Replace terminal section on sidebar to jupyter in experiments
  • Improvement: Add compute name in volumes selector
  • : Webapp iframe url stays consistent throughout deployments
  • Fix: Webhook in flow changes parameter type from categorical to discrete
  • Fix: Experiment with parenthesis in key name gets stuck in a restarting loop
  • Fix: Organization Settings - colon in pypi repo URL doesnt pass validation
  • Fix: Editing a file through the UI works only the first time

# Version v3.6.10

  • New Feature: Experiment New Dashboard
  • New Feature: Add flow dag to experiment page
  • New Feature: Change mount path when attaching PVC as local folder
  • New Feature: Support params to flows from webhook
  • Improvement: Allow custom pypi to be used for installing requirements
  • Improvement: Support log images in experiments
  • Improvement: add ability to drag charts in quick view
  • Improvement: add search and sort to visuals tab
  • Improvement: Support dot in chart title
  • Fix: Validation in tasks is not working if parallel job count field is greater than max job field
  • Fix: Creating template using specific node label raises an error
  • Fix: Saving a task as a library raises an error
  • Fix: Flow - notify on success/error set to true in yaml but email is not sent
  • Fix: cant push new images if repo exist in ECR
  • Fix: Flow- 2 compute names from different clusters causes the flow to not run
  • Fix: SDK: running an experiment with arguments ignores the compute
  • Fix: SDK: running an experiment with non-existent compute templates defaults to organization settings instead of project settings
  • Fix: Compute is showing as "unassigned" for scheduled experiment until manual refresh
  • Fix: Datasets is not downloaded in endpoint when endpoint is run from flow

# Version v3.6.9

  • Feature: Support Oauth Proxy
  • Feature: ADD mig (Multi instance gpu) Support
  • Feature: ADD Ray integration
  • Feature: ADD permissions to compute templates
  • Improvement: Dataset - JSON files displayed without indentation
  • Improvement: Retrieve organization name from URL In the websocket
  • Improvement: Support cluster restart and restart pods lifecycle
  • Improvement: SDK: Support Experiment.alert from sdk
  • Improvement: CLi: improve mechanism of removing many folders in dataset
  • Improvement: Update workspace title to be Workspace
  • Improvement: support global gateway
  • Improvement: Add Custom Scheduler option
  • Fix: Can't restart jobs with PCV
  • Fix: Dashboard graph is not presented
  • Fix: mpi workers don't pull secrets

# Version v3.6.8

  • Feature: Support HyperParameter tuning for experiments and flows: Grid Search, Random Search, Bayesian
  • Fix: CLI: Experiments are always taking folder as output even if we are running exp on custom folder
  • Improvement: Change error message when attempting to delete a predefined image
  • Fix: Fix View of commit in the browser
  • Fix: flow is starting experiments in "waiting mode"
  • Fix: Fix default timezone for users
  • Fix: SDK: Can't authenticate in windows
  • Fix: SDK: Fetch all files in dataset returns error
Last Updated: 1/24/2022, 8:03:52 PM