iMerit is a leading AI data solutions company providing high-quality data across computer vision, natural language processing, and content services that power machine learning and artificial intelligence applications for large enterprises. The company emphasizes the need for solutions encompassing “technology, talent and technique”, to solve the machine learning (ML) data problem at scale for enterprises. iMerit provides end-to-end ML data labeling solutions to Fortune 500 companies in a wide array of industries. The company works on data for transformative technologies such as advancing cancer cell research, optimizing crop yields, and training driverless cars to understand their environment.
Optimizing Your Data Efforts with iMerit
Today, iMerit is regarded as a leading enterprise-grade data annotation and ML data validation solution. With over 5,000 employees worldwide, iMerit’s goal is to uncover hidden value within unstructured datasets, thereby allowing their customers to focus on what they are directly trying to achieve.
iMerit’s end customers are enterprises, which require scalable, secure, and flexible solutions to keep pace with their sprawling machine learning needs. Consistently extracting and managing high-quality data intelligence at scale requires a powerful end-to-end solution while keeping in mind the cost to the end-user. With an output accuracy above 98%, iMerit is recognized as a strong leader in consistently delivering quality, cost-effective projects on time, and on budget.
More importantly, the expert-trained in-house workforce is capable of working on increasingly nuanced and complex data in order to push the boundaries of data-centric ML. Additionally, working with high-performance tools and using the assistance of automation provides the best value for the customer’s investment in large-scale data-centric ML.
Data-centric ML inherently requires a lot of manual effort from experts which can become costly and messy. Annotation software is an additional component which can greatly enhance the data annotation pipeline . A company like iMerit stands out by uniquely addressing the scaling and quality challenges for enterprises and also by looking at the entire lifecycle of data-centric ML, from training the model to monitoring, auditing and edge case analysis.
The way to manage this is through:
Visibility: Having real-time visibility into the multiple data workflows is critical to managing optimally at large scale. Metrics like the annotation progress, workforce productivity, time spent per item or annotation, labeling outcome, performance, cost usage, reporting, etc. are often ignored in simpler tools but can influence outcomes significantly. Such as cost, time, and quality. Instant workflow transparency, particularly of productivity and accuracy stats, help managers improve data preparation in real-time. Additionally, managing your unstructured data in a single and secure platform allows you to gain visibility into data such as images, videos, audio, text, etc.
People: Gain full transparency into your workforce by grading, scoring, tracking, and managing your workforce through intelligent analytics and reporting in real-time and designated for each individual user. Evaluate quality at the individual and group level, create quality and escalation workflows, and allow project managers and subject experts to oversee the data.
Insights: ML engineers derive huge value from qualitative and quantitative insights around their data. In the initial phases, edge cases and unanticipated situations serve to update the annotation guidelines, taxonomies and feature sets. During testing and deployment, anomaly handling and exception monitoring serve to close the gap between development and deployment. Insights delivered by experts in a common platform are disproportionately valuable as the development progresses.
How did iMerit achieve success?
iMerit developed their framework of “technology, talent and technique” to address the problem. They recognized that no single component is sufficient and all three are needed to properly meet the needs of enterprise customers moving fast in a dynamic field.
Technology: iMerit partnered deeply with Dataloop to achieve their tooling and data management needs for computer vision customers across image and video. They invested in building internal expertise among their team and also worked closely to leverage Dataloop on behalf of customers. Lastly, iMerit utilized Dataloop’s automation in order to cut time, lower costs, and ultimately accelerate and automate the annotation process to supercharge their manual labelers.
Talent: iMerit has developed an advanced expert workforce and strong project and skill management that has allowed them to optimize every operational step of their organization. The iMerit team increasingly works on complex and nuanced problems, often requiring domain expertise, and efficiently achieves project management through planning and tracking the deliveries. iMerit provides advanced insights around edge cases, anomalies and exceptions that are tracked through the Dataloop platform, and then shared to customers, in order to enhance the ML model.
Technique: iMerit leads customer engagements with highly experienced solution architects who implement iMerit’s best practices to scope and structure the customer’s use case, often iteratively. The project often starts with a rapid pilot spin-up to explore and experiment with the customer and to discover every aspect of the customer’s problem, often surfacing new approaches and insights. The architects are empowered to work quickly as they are well-familiarized with the Dataloop platform.
The project and delivery team then applies their expertise to create the data intelligence at high quality. Delivering rapid early wins is often the key for the customer to confidently double down on a long-term machine learning journey run on the Dataloop platform by the iMerit teams.
iMerit has many clients who needed a quick spin-up on computer vision projects in order to complete a Proof-of-Concept (PoC), incorporate the findings, and then move forward. This is an interactive and rapid process which needs a good interface. Dataloop helped us solve the problem and spin up projects faster.
Managing & Collaborating with a Massive Data Labeling Workforce
iMerit is well-known for managing its workforce of people. In 2020, they hired over 1,200 people across their teams despite the global disruption caused by Covid-19. What is even more impressive is their success in people planning and talent matching capabilities as iMerit proudly attests to achieving a 90% retention rate. With their mammoth workforce, they’re managing projects on a constant basis and keeping pace with the increasingly complex problems being solved by customers spanning fields like autonomous vehicles, medical AI and audio AI .
Dataloop stepped in and worked closely with iMerit to implement detailed real-time dashboards to help track their projects progress on the annotator level.
This allowed them to:
- Gain data labeling work status and performance observability to improve productivity.
- Manage complex QA processes, learning curves, and training cycles with ease.
- Distribute the work easily among the team using a no-code automation process.
iMerit found real-time dashboards heavily contributed to better quality and efficiency. Additionally, data scientists are data-driven and want to know the rate they’re getting their work accomplished, so having full visibility is a major bonus.
According to iMerit, the workflow, task design and allocations enabled them to make a number of quick PoC’s and customer projects which then scaled into full customer engagements.
Trying to align iMerit’s workforce management before Dataloop proved to be a very difficult task when factoring in external third parties together with the organic teams. Occasionally, on certain projects, iMerit, though their workforce is vast, sometimes had clients that required work done within short deadlines which required them to bring in these third parties to help with the workload. However, with Dataloop we were able to provide them with a workforce management solution that kept everything under one roof, allowing iMerit to run their in-house projects seamlessly.
Being able to have full visibility of different companies and partners from the side of workforce management allowed iMerit to work in the most efficient manner.
iMerit Wins Big with Workforce Management:
Removed the need to do manual daily analysis tracking and manual report creation by Project Managers and Team Leaders.
Time was now spent on the daily/weekly auto-generated reports that are customizable by date and time.
Project Managers and Team Leaders (depending on the team size) saved around 2 hours on project management hours which is now better spent with the team.
Ensuring Annotator Accuracy
iMerit utilizes many different QA workflows that involve humans and ML to ensure the highest data accuracy. The QA workflow will generally depend on how robust the project is and how in depth you want the workflow verification to be. iMerit deploys two main workflows, one which is more basic, and the other a more in-depth workflow. The first workflow consists of the production task where for example, bounding boxes are applied. The second step involves iMerit checking those annotations. When you’re dealing with a more robust workflow for example, with a medical AI pod, this type of consensus workflow is utilized. A consensus workflow allows you to assign two people (or more) to the same task and then their work is compared and measured for differences. If they have the same results, then it’s assumed to be correct. The third step is then bringing in the client and having them QA on the Dataloop platform.
iMerit utilizes this process to train their annotators into becoming domain experts, even if they aren’t considered experts in that specific industry. It’s a dynamic process to try to match the skills with the experience and where they have the availability with the right people. It also plays into the equation of being able to process efficiently, cost effectively, and if you’re getting the right throughput for the right level of data.
Before Dataloop, iMerit tracked their annotators manually. The problem with manual capture is that it is error prone. Most can relate to this and it is not a helpful, scable, or reliable reporting mechanism to send back to your client. At the end of the day, the clients want to see this information, but they want to see the data collated, and easily at their fingertips. iMerit finds it incredibly helpful to have reliable data coming out of Dataloop’s analytics platform, in fact they say it’s been “game-changing” for their project management team. iMerit’s time is no longer about capturing accurate data, now they can deep dive into that data and analyze it and make decisions accordingly.
The Data Stack Supporting iMerit’s Needs
The goal is to get your customers onto a tool and use all of its functionality to the best of its ability. It’s a learning curve to acquire a tool and use it to its full potential. But in order to get there, you need time to run projects. iMerit utilizes Dataloop’s AI-assisted annotation platform, Data Management, Automation, and Production Pipelines to achieve this goal. Generally, if you use different tools you’re going to need to invest time into learning each and every tool from scratch until you know it like the back of your hand.
With Dataloop, iMerit has removed that learning curve, so they can focus rather on learning the data instead of learning the tool, which is where iMerit wants to be. They want to learn about their client’s data and gain insights and how to navigate accordingly. Dataloop’s data management and pipeline features are timesavers for iMerit’s teams. It only requires their clients to do an initial setup, versus setting it up every single time. There is a streamlining of work, time, and actions, thereby removing the chance of duplications.
Dataloop provides iMerit with a one-stop shop for all its tooling needs. In fact, when iMerit started with Dataloop, 80% of their workload was images. Today, they’ve expanded to NLP, LiDAR, video, etc. When you’re switching between different tools typically your efficiency drops. However, we provide iMerit and its customers with tools to annotate and manage their workforce and pipelines. This ensures their entire workflow and user experience are consistent. Allowing them to manage people better, save time, and also support any type of data.
Annotating with Automation Tools
When applying automation you’re shifting the focus from “just labeling”, to focusing more on validating your labeling process. Think of it as looking at pre-labeled data and you’re involved in the auditing of that labeling. This allows you to focus on quality, which is helpful for both the customer and iMerit. This is a much more refined way of turning a workflow into a quality controlled workflow, where you’re no longer looking at just putting labeling onto an image/video but rather at the accuracy gains. It’s incredibly helpful when you’re looking for extremely high-quality workflows and the margins of error are very small. It also leverages an expert team much better to work on the more nuanced information.
iMerit compared and tested projects to gauge the difference between a purely manual annotation process to automated assisted processing. The contrast was stark showing greater achievement gains in output. They compared a pure manual annotation process with an automation-assisted process, on an identical data set. They tested for both throughput and accuracy. According to iMerit, what their clients are really looking for is a wealth of accurately labeled data at large volume. Accuracy and volume is the crux of it all when it comes to training models.
As Natasha Montagu at iMerit says, “if you can accomplish accuracy and volume, then you’ve really changed the ball game with automation”.
A Partnership That Goes Beyond
Dataloop provides iMerit with a partnership marrying iMerit’s expertise of managing and training people with Dataloop’s expertise of technology. It’s a genuine partnership built on trust in Dataloop which enabled us to be a part of iMerits stack of solutions for its customers. It’s not just a software subscription, it’s a full-on relationship.
LET'S TALK ABOUT IMERIT'S GROWTH WITH DATALOOP
Thanks to Dataloop, iMerit was able to improve the spin-up time of client pilots and reduce their sales cycles. The solution architects were able to assess datasets and manage their time and scope studies more quicker and more accurately. Additionally, Dataloop helped iMerit with larger projects due to our strong dashboards and data management.
iMerit experienced massive growth in the past year and they found that most of their customers needed that “white glove experience”. iMerit’s partnership with Dataloop enabled them to scale up their pipeline of projects and speed up time to revenue on those projects. It was also easier for iMerit’s team of solution architects to manage a number of pilots in a relatively straightforward fashion. iMerit worked closely with our team at Dataloop, to implement feature improvements and custom requirements from their clients, all of which contributed to a high NPS (Net Promoter Score) in the past year.
Dataloop’s extensive infrastructure helps large companies like iMerit scale their production. The Dataloop platform allows you to start building labeling pipelines and leap your labeling efficiency towards production, with our significant automation features. Overall, the combination of iMerit’s know-how and Dataloop’s technology allowed iMerit to provide a strong offering to their customers’ ever-changing needs.