Bench Talk

3 Emerging Trends Every AI Engineer Needs to Explore

(Source: Omelchenko/Shutterstock.com)

In late 2019, International Data Corporation predicted that global spending for artificial intelligence (AI) systems would grow to $97.9 billion (USD) by 2023, almost two-and-a-half times 2019’s $37.5 billion (USD). Even the disruption the COVID-19 pandemic brought to 2020 has not slowed the inflection point for AI.

It’s a good idea for AI specialists to be on the lookout for emerging trends and opportunities for skill diversification so that you can take advantage of these new trends. We have identified three areas for your consideration. Each has the potential to better prepare you for the growth explosion that experts predict.

Until recently, developers only considered doing serious processing in a data center. Processing locally was not an option. Now, supercomputer performance has crossed important size, cost, and power requirement thresholds. NVIDIA’s Jetson Xavier NX development kit, released in May 2020, costs under $400 (USD) and can run using only 10W of power. It delivers up to 21 terra operations per second (TOPS) of computing, real power capable of running modern neural networks in parallel and processing data from multiple high-resolution sensors, exactly what is needed for a full AI system. Other competitors are not far behind.

These advancements greatly simplify the development of cloud-connected AI and ML multi-modal applications (Figure 1). Imagine building an advanced live video and thermal image processing system that detects potential threats from visitors with a high fever or individuals who have been placed on a do-not-admit list. Even with 5G, it’s not practical to upload high-resolution live video and thermal sensor data to the cloud for processing in a remote server farm, then waiting for the facial-recognition and risk-assessment scoring algorithms to do their work and results to be transmitted back, perhaps minutes later. The only solution is to process the live data locally so that the system can quickly assess the threat level.

Figure 1: Cloud-connected AI applications are greatly advancing (Source: Phonlamai Photo/Shutterstock.com)

This theoretical system is now possible with standard cameras, thermal-imaging devices, and the 8-core NVIDIA device that sells as a complete development kit. The common AI application building blocks such as object detection, text-to-speech, classification, recommender engines, language translation, sentiment analysis, and more can now be found in pre-trained models. Developing applications with these capabilities is much faster when you start with these models and then tune them for specific use cases. The NVIDIA GPU Cloud (NGC) library offers hundreds of these models to get a jump-start on common AI tasks. They are already optimized for the Jetson Xavier and the company’s other offerings. Moreover, once you get your project running, you can easily re-train these models by updating just a few layers, saving hours.

These power-packed supercomputers are perfect for smart camera projects such the one described above. They also can be leveraged to build an unlimited number of other high-performance AI systems such as medical instruments, automated optical inspection, robots for commercial applications, and more.

Significant advances in natural language generation, specifically Microsoft’s Turing Natural Language Generation (T-NLG), have opened some doors previously inaccessible for AI developers. But this was only possible with some resourceful engineering.

Even with 32GB of RAM, a single graphics processing unit (GPU) cannot fit models with over 1.3 billion parameters. The Microsoft team solved this problem by taking advantage of several hardware and software breakthroughs. Microsoft leveraged a hardware setup from NVIDIA with superfast communication between GPUs and applied tensor slicing to shard the model across four GPUs. The model-parallelism degree was reduced from 16 to 4, increasing the batch size per node by 400 percent, and increasing training speed threefold. The result, DeepSpeed, trains at a batch size of 512 GPUs with only 256 GPUs compared to 1024 GPUs needed using previous configurations. The great thing is that you don’t have to understand this past paragraph to start using the technology.

The February 2020 beta release of T-NLG makes it possible to answer search queries with complete sentences, generating conversational language in real-time. Imagine a chatbot that is intelligent enough to speak your language and respond in a way that makes complete sense.

T-NLG is the largest model ever published at 17 billion parameters making it possible to create applications such as those that can assist authors in composing their content or summarizing a long text excerpt. Or, imagine improving customer experience with a digital assistant that is significantly smarter than the ones we are all so used to. We believe that exploring natural language generation tools will help any AI developer cultivate new skills that will be highly marketable in the next few years.

Getting started with T-NLG is easier than you might think, especially if you are already using Python. Head over to GitHub and explore the DeepSpeed library repo (compatible with PyTorch) and dive right in. This API library makes distributed training easy, efficient, and effective with deep-learning models 10 times larger and five times faster, letting you leverage a hundred billion parameters to train models in record time.

A database is typically the bottleneck in AI systems and applications. It’s a problem that can’t easily be solved by applying more processing power or memory. One trend to watch is the application of neural networks to database system design.

The Massachusetts Institute of Technology (MIT) and Google’s experimental data management design swaps in a neural network for the core components, increasing performance over cache-optimized B-Trees by up to 70 percent while reducing memory requirements.

The learned indexes quickly learn the structure of the database’s lookup keys, using them to predict records’ position. Neural nets can impact future systems designs significantly, so we feel it’s a topic worth exploring. So pick a topic: Artificial Intelligence of Things (AIoT), natural-language generation (NLG), or neural computing–or pick them all–and prepare to ride what is sure to be an exciting wave of AI innovation.