# Advanced Model Tracking with Keras Callbacks

cnvrg.io provides an easy way to track various metrics when training and developing machine learning models. Keras is one of the most popular frameworks for building deep-learning applications. In the following guide we will use Keras Callbacks with the cnvrg Python SDK to track and visualize training metrics.

# Create and Use Keras Callbacks

Callbacks in Keras are functions to be applied at given stages of the training procedure. You can use callbacks to store model checkpoints (using the ModelCheckpoint callback) and to get a full view on internal states and statistics of the model during training. When using the .fit method, you can pass a list of callbacks as an argument. The relevant methods of the callbacks will then be called at each stage of the training.

To create your own Keras callback function, simply create a new class and have it inherit from keras.callbacks.Callback. Then, chose metrics, parameters and statistics you would like to send back to cnvrg for plotting, dashboarding and visualizations in real-time

import keras
from cnvrg import Experiment

class MetricsCollector(keras.callbacks.Callback):

    def __init__(self, experiment):
        # Load the cnvrg Experiment context
        self.experiment = experiment

    def on_batch_end(self, batch, logs={}):
        # Log loss metric for each batch and send it to the set experiment
        # to view in cnvrg dashboard
        self.experiment.log_metric('loss', [logs.get('loss').item()])

There are additional methods you can utilize in your new Keras callback:

on_epoch_begin: called at the beginning of every epoch.
on_epoch_end: called at the end of every epoch.
on_batch_begin: called at the beginning of every batch.
on_batch_end: called at the end of every batch.
on_train_begin: called at the beginning of model training.
on_train_end: called at the end of model training.

# Training using `fit` and the new created callback

To use the created callback in your Keras training code, simply instantiate the experiment and the MetricsCollector class and add the function to the fit method in the callbacks argument, as specified below:


# Loading the cnvrg experiment instance
from cnvrg import Experiment
experiment = Experiment()

# Creating the MetricsCollector instance, and providing the cnvrg 
# experiment context.
metrics_collector = MetricsCollector(experiment)

# fit command with Keras Callbacks, and the metrics collector
model.fit(x_train, y_train,
          batch_size=batch_size,
          epochs=epochs,
          verbose=1,
          validation_data=(x_test, y_test),
          callbacks=[metrics_collector])

# Advanced Model Tracking with Keras Callbacks

# Create and Use Keras Callbacks

# Training using fit and the new created callback

# Training using `fit` and the new created callback