SDK Cheat Sheet
-
Print
-
DarkLight
The Dataloop Python dtlpy package enables a Python connection to Dataloop's environment.
Please verify you have Python installed with our dtlpy package. You can find the needed information here
Projects
To understand more about Dataloop's projects go to Projects.
project = dl.projects.create(project_name='my-new-project')
dl.projects.list().print()
project.open_in_web()
Get
dl.projects.list()
project = dl.projects.get(project_name='my-project')
project = dl.projects.get(project_id='my-project-id')
More about projects SDK on Project.
Datasets
To understand more about Dataloop's Datasets go to Datasets.
dataset = project.datasets.create(dataset_name='my-dataset-name')
dataset.print()
dataset.open_in_web()
Get
datasets = project.datasets.list()
dataset = project.datasets.get(dataset_name='my-dataset-name')
dataset = project.datasets.get(dataset_id='my-dataset-id')
More about datasets SDK on Datasets.
Items
To understand more about Dataloop's Items go to Items.
- Upload items
- Upload items to a specific folder
- Download items
- Download items from a specific folder
- Open item in the web
- Clone Items
dataset.items.upload(local_path="/path/to/image.jpg")
dataset.items.upload(local_path="/path/to/image.jpg",remote_path="/path/to/dataset/folder")
item.download(local_path="/where/to/save")
item.download(local_path="/where/to/save",remote_path="/from/where/to/download")
item.open_in_web()
# dst_dataset_id - ID of your desired destination folder to clone the item at. # with_annotations - assign - True to clone the item with it's annotations. # assign - False True to clone the item without it's annotations. # with_metadata - assign - True to clone the item with it's metadata. #assign - False True to clone the item without it's metadata. item.clone(dst_dataset_id='your-dataset-id-number', with_annotations=True, with_metadata=True, with_task_annotations_status=False)
Get
item = dataset.items.get (item_id = 'my_item_Id')
More about Items SDK on Items section.
Metadata
To understand more about Dataloop's Metadata go to Metadata.
item.metadata [ 'user' ] = dict () item.metadata [ 'user'][ 'MyKey' ] = 'MyValue' item.update()
More about metadata SDK on Metadata.
Recipe And Ontology
To understand more about Dataloop's Recipe And Ontology go to Recipes & Ontologies.
- Add a label
- Delete labels
- Copy datasets ontology to a new dataset
- Create a new recipe
- Update recipe changes
- Delete recipe
- Change annotation label to a different label
dataset.add_label(label_name='person', color=(34, 6, 231), attributes=['big', 'small'])
dataset.delete_labels(label_names=['myLabel1', 'Mylabel2'])
new_dataset = project.datasets.create( dataset_name='new_dataset_with_ontology', ontology_ids=dataset.ontology_ids)
recipe = dataset.recipes.create(recipe_name='My Recipe', labels=labels))
recipe.update()
# This command is only for DELETED datasets dataset.recipes.get(recipe_id='my_recipe_id').delete())
annotation.label = "label2" annotation.update()
Get
recipe = dataset.recipes.get(recipe_id='my_recipe_id')
ontology= recipe.ontologies.get(ontology_id='my_ontology_id')
childLabel = ontology.labels[x].children[y].children[z]
More about Recipe SDK on Recipe.
More about Ontology SDK on Ontology.
Annotation
To understand more about Dataloop's Annotation go to Annotations.
- Upload annotations
- Update annotation changes
- Create annotation builder
- Add box annotation with label person
- Add point annotation with attribute
- Add annotations of type segmentation
- Add polyline annotation with label person
- Create a mask for semantic segmentation
- Upload annotations to the item
- Delete Annotation
annotations = item.annotations.upload(annotations=local_annotation_path)
annotation = annotation.update()
builder = item.annotations.builder()
builder.add(annotation_definition=dl.Box(top=10,left=10,bottom=100, right=100,label='person')) item.annotations.upload(builder)
builder.add(annotation_definition=dl.Point(x=80,y=80,label='label1',attributes=['attribute1']))
builder.add(annotation_definition=dl.Segmentation(geo=mask,label='label1')) # Using the mask from previous step
builder.add(annotation_definition=dl.Polyline(geo=[[80, 40],[100, 120],[110, 130]], label='person'))
mask = np.zeros(shape=(item.height, item.width), dtype=np.uint8) mask[50:100, 200:250] = 1
item.annotations.upload(builder) # To upload the annotations added to the builder above
annotation = item.annotations.get(annotation_id='your-annotation-id-number') annotation.delete()
Get
annotations = item.annotations.list()
annotations = annotation = item.annotations.get(annotation_id='your-annotation-id-number')
More about Annotation SDK on the Annotations section.
Filters
To understand more about Datatloop's Filters go to Sort & Filters.
- Initiate Filter
- Filter only files
- Filter only annotated items
- Filter items name that includes 'dog'
- Filter items name or directory that includes 'dog'
- Filter specific directory
- Filter directory items with all subdirectories
- Update filtered items
- Delete filtered items
- Filter all items with the annotation type - Box
filter = dl.Filters()
filter.add(field='type', values='file')
filter.add(field='annotated', values=True)
filter.add(field='name', values='dog')
filter.add(field='filename', values='dog')
filter.add(field='dir', values='/myDirectory')
filter.add('dir', '/myDirectory', method='or') filter.add('dir', '/myDirectory/*', method='or')
pages = dataset.items.update(filters=filter, update_values=update_values)
dataset.items.delete(filters=filter)
filters.resource = 'items' filters.add(field='type', values='file') filters.add_join(field='type', values='box') list = dataset.items.list(filters=filters)
Get
list = dataset.items.list( filters = filter )
More about Filters SDK on Filters.
Task and Assignments
To understand more about Dataloop's tasks and assignments go to Tasks & Assignments.
Some tasks & assignments commands require datetime import
import datetime
Tasks
- Remove items from the task
- Delete a task
- Single status update
- Create a task
- Create QA task
- Add items to the task
task.remove_items( filters = filter)
task. delete()
# Mark single item as completed item = dataset.items.get( item_id= 'my_item_id') item.update_status( status= dl.ItemStatus.COMPLETED) # In the same way you can update to another status item. update_status( status= dl.ItemStatus.APPROVED) item. update_status( status= dl.ItemStatus.DISCARDED)
task = dataset.tasks.create( task_name= 'task_name', due_date = datetime.datetime(day= 1, month= 1, year= 2020).timestamp(), assignee_ids =[ 'annotator1@dataloop.ai', 'annotator2@dataloop.ai'])
task.create_qa_task( due_date=datetime.datetime(day= 1, month= 1, year= 2020).timestamp(), assignee_ids=[ 'annotator1@dataloop.ai', 'annotator2@dataloop.ai'] )
filters = dl. Filters( field= 'dir', values= '/my/folder/directory') task. add_items( filters=filters, assignee_ids=[ 'annotator1@dataloop.ai', 'annotator2@dataloop.ai'])
Get
project.tasks.get(task_id= 'my-Task-Id')
items_list= task.get_items()
task_list = project.tasks.list()
Assignments
assignment. remove_items( filters= filter)
assignment. redistribute( assignee_ids=[ 'annotator1@dataloop.ai', 'annotator2@dataloop.ai'] )
assignment.reassign( assignee_ids['annotator1@dataloop.ai'] )
Get
- Get assignment
- Get assignment items
- Get the assignment list of a task
- Get assignments list of a project
task.assignments.get(assignment_id= 'my_assignment_Id')
items_list= assignment.get_items()
task.assignments.list()
project.assignments.list()
More about tasks and assignments SDK on the Tasks and Assignments section.
Applications
To understand more about Dataloop's Applications go to Applications.
item1 = dataset.items.get(item_id='my_item_Id') item2 = dataset.items.get(item_id='my_2_item_Id') item1.modalities.create(name='my modality', modality_type=dl.ModalityTypeEnum.OVERLAY, ref=item2.id) item1.update()
target_item = dataset.items.get(filepath='/remote/path/target_item.png') ref_item1 = dataset.items.get(filepath='/remote/path/product1.jpeg') ref_item2 = dataset.items.get(filepath='/remote/path/product2.jpeg') ref_item3 = dataset.items.get(filepath='/remote/path/product3.jpeg') similarity = dl.Similarity(name='my-product-similarity', ref=target_item.id, items=[ref_item1, ref_item2, ref_item3]) dataset.items.upload(local_path=similarity, remote_path='/products_similarities')
item = dataset.items.get(filepath='/remote/path/original_item.png') item2 = dataset.items.get(filepath='/remote/path/second_item.png') multiview = dl.MultiView(name='my-items-multiview', items=[item, item2]) dataset.items.upload(local_path=multiview, remote_path='/remote/path')
More about Applications SDK in the Applications section.
FaaS
To understand more about Dataloop's FaaS go to FaaS (Function as a Service).
For a guide to getting started with FaaS go to Pre-Requisites.
Packages
To understand more about Dataloop's packages go to the package page.
package.delete()
# push the package again # this action will push the new codebase and bump the version of the package project.packages.push(src_path='/path_to_my_edited code', package_name=package.name)
# to see the versions package.versions # to see the latest version package.version
package.deploy()
Get
package = dl.packages.get(package_id='my_package_Id')
pack_list = dl.packages.list()
package.open_in_web()
Services
To understand more about Dataloop's services go to the service page.
service.delete()
service.open_in_web()
# to pause the service service.pause() # to resume the service service.resume()
bot = project.bots.create('bot_name')
Get
service = dl.services.get(service_name="my_service_name")
dl.services.list().print()
Triggers
To understand more about Dataloop's triggers go to the trigger page.
trigger.delete()
filters = dl.Filters(field='dir', values='/input') trigger = service.triggers.create(function_name='run', resource=dl.TriggerResource.ITEM, actions=dl.TriggerAction.CREATED, name='items-created-trigger', filters=filters)
Get
service = dl.triggers.get(trigger_name="my_trigger_name")
dl.triggers.list().print()