The last 30 years have been revolutionary in every aspect of our lives. Technological breakthroughs have enabled massive changes to the way we consume content, purchase goods, and communicate with people across the globe. The coming changes such as Autonomous Vehicles, Precision Agriculture, and Cashierless Checkouts are already in hands reach. These new solutions require mass amounts of quality and diverse data. I’d like to take this opportunity to explain why and how you can optimize this data usage.
Milestones
There is one fact that no one can dispute, the Internet is the backbone for most of the world’s globalization, and has given us the ability to connect and communicate with individuals for any reason under the sun and it’s easier than ever before. Millions around the world started creating content or digitized it in order to attract the general public into buying their products or services and the result was data and a lot of it. This data was mostly collective, textual, and numeric which until the beginning of the Millennium was used for general market understanding or for financial predictions.
That brings us to 2007 (after the dot com and Google surge) when Steve Jobs introduced the first iPhone to the world. Apple took the simplicity of a phone and turned it into a necessity that is connected to every facet of your life. This was a massive change in the way we consumed content, and though it was already available prior to 2007, it was now much more interactive, approachable, user-friendly, and most importantly, accessible to everyone and everywhere. The small device gave companies the “individual” perspective to the collected data that although always existed, could now be further understood. The data that organizations obtain became personal and visual and not only textual and numeric as previously.
The last milestone I’d like to address was in 2011 when the first convolutional neural network (AlexNet) was trained and produced amazing results. The increased speed of GPUs made it possible to train convolutional neural networks “without” the layer-by-layer pre-training. Visualization of a deep network explains the strengths nicely, but the main requirement was and still is tons of data. The world started to understand that image and video data can now be used to train models accurately. Infrastructure was built, public code libraries started, (Python users tripled over 3 years) and the race for production use cases began.
Become a Participant
Each one of the milestones accumulated hundreds of great companies into multiple verticals, but I’d like to emphasize here the enablers; the platforms that let other companies participate in the race. Over time, different platforms became crucial.
These platforms understand their customer requirements and help them to fulfill their needs with expertise without diverting from their core business.
Wix
Web presence is a must in today’s world. In 2006, when there were already 85 million sites in the world, Wix founders understood that it was not going to stop. Websites are a massive business enabler and as we know today, if you’re not online, you don’t exist. By allowing users to create, design, and manage their websites easily they take many of the unrelated core business tasks like coding websites off their customer’s shoulders. As a result, today, there are 160 million Wix sites, almost double the number of sites in the entire world in 2006. These kinds of platforms are huge contributors to the massive growth in online content.
Tableau
The company that was acquired by Salesforce in 2019 in a $15 billion dollar deal is a great example of the increasing need for data analysis and visualization. Their market value in 2014 was less than $4.8 billion.
Tableau products query databases, online analytical processing cubes, cloud databases, and spreadsheets to generate graph-type data visualizations and enable companies to get critical insights into their customers.
They understand the value of data and enable key capabilities such as data visualization, exploration, and organization for a huge amount of data. By providing this to their customers they enabled customers to use their data in an effective and actionable way.
Current Market Needs
Nowadays we see the first batch of vision-based companies. The amount of use cases seems to be infinite and the sky’s the limit, whether it’s medical robotics, ADAS, or disease detection for plants. Cloud and deep learning are a solid foundation so the vision solutions are not limited to big corporations and we see an increasing number of startups with unique solutions. As long as you’re able to get the data, you’re good to go. Managing huge amounts of unstructured data is now the biggest challenge for most of those companies, for example, an average autonomous car provides 4 TB of data per day and those numbers will only increase.
For those reasons you have to manage the data in a way that enables you to OBSERVE it:
- Organize — Different tasks require different data, therefore it is essential to organize it in an easy-to-use way according to the task.
- Browse — Enable one pane of glass of the data to different teams with different uses and permissions (even your external labeling services).
- Structure — Use the existing structure in order to optimize workflow, make sure you can connect your platform to your code.
- Explore — Sort, filter, and query the data based on any information you have such as annotations, item data, and even metadata.
- Research — Get insights on improving model performance on specific use cases based on the training data and analytics.
- Visualize — Since the data is no longer numeric you have to see it for what it is, run different visualization techniques, and understand data structures.
- Extend — Open source and internally built tools tend to fail in scaling up; when you go to production, by default you scale up exponentially.
Join the Revolution
My advice to you? Continue changing our world. Every day that goes by we get closer to automated roads, safer schools, quicker surgeries, and better food. The number of companies trying to do that is uncanny and if 5 years from today even 10% of them are successful, our world will be totally different. Data has always been a major part of our lives, but the current usage requires a different treatment than before. Therefore, we need to take care of it, explore it, and let it become Artificial Intelligence…
If you liked this article, please feel free to share it on social media.
