Model Architectures
  • Dark
    Light
  • PDF

Model Architectures

  • Dark
    Light
  • PDF

Dataloop's model-management module enables you to use public model architectures or connect your own models, create trained models using annotations from data items in datasets, review training metrics and use model versions for pre-annotations or production uses, by triggering them directly or embedding into Pipelines.

To install, train and deploy models using Dataloop SDK, read this tutorial on our SDK documentation site.

To access Model-Management module - Select "Model Management" menu item from the left-side navigation menu.
image.png

Model Architectures

Model architectures are the foundation to beginning model versioning and training with various data. Use a public model architecture or add your own from GIT to create the first model version. Train it with data, evaluate results and performances, and continue to create additional versions with more data, using the base version or any trained model as your starting point.

Public Models

The Dataloop system has several pre-installed and publicly available model architectures, which can be installed in your project and trained with your data to achieve pre-annotations of your labels.

  • ResNet - For image classification
  • yolo-v5 - For box annotations
  • unet - for mask/segmentation (COMING SOON)

Add Your Own Model

Connect your own models to the dataloop system by placing them in GIT repository and allowing Dataloop to access it.

Training models

To train your model in the Dataloop system, it should have a Train function that facilitates the training process you wish to facilitate.

Model Adaptor

The Model Adapter is a python class to create a single API between Dataloop’s platform and your Model. By inheriting from a Dataloop class dl.BaseModelAdapter that facilitates all the necessary functions, all that is left is to create these 4 methods:

  • Train
  • Predict
  • Load (to load model weights)
  • Save (to save model weights)

For example, your model adapter can be:

import dtlpy as dl
import torch
import os
class SimpleModelAdapter(dl.BaseModelAdapter):
    def load(self, local_path, **kwargs):
        print('loading a model')
        self.model = torch.load(os.path.join(local_path, 'model.pth'))
    def save(self, local_path, **kwargs):
        print('saving a model to {}'.format(local_path))
        torch.save(self.model, os.path.join(local_path, 'model.pth'))
    def train(self, data_path, output_path, **kwargs):
        print('running a training session')
    def predict(self, batch, **kwargs):
        print('predicting batch of size: {}'.format(len(batch)))
        preds = self.model(batch)
        return preds

Install Your Model (GIT Codebase)

  1. Click on "Add Model"
    image.png
  2. In the "New Model" popup, enter the relevant information to allow connecting to your code on GIT
    1. Model Name - The name you wish to assign to the model when represented and referred to in the Dataloop system
    2. GIT URL
    3. Model description (optional)
    4. Class name (optiona)- the class where you have the model adaptor
    5. Entry poin (optiona) - the python file to use as entry point
    6. Input type expected by the model (image, video etc)
    7. Output expected from the model (box annotation, classification etc)
    8. Tags - optional tags to assign to the model

What's Next