According to a new research report from the IoT analyst firm Berg Insight, the on-device AI market reached US$ 10.1 billion in 2024, an increase of around 22 percent from 2023. This figure includes AI SoCs/SoMs, AI accelerators, AI MCUs and specialized on-device AI software and platforms, but excludes revenues generated by non-IoT applications such as smartphones, tablets and personal computers. The market is expected to grow to US$ 30.6 billion in 2029, representing a compound annual growth rate (CAGR) of 25 percent.
“Over the past decade, the on-device AI market has been driven primarily by traditional machine learning use cases such as computer vision and anomaly detection, which have seen steady annual growth of around the 10 percent range”, said Melvin Sorum, IoT analyst at Berg Insight. “In recent years, the market has reached an inflexion point as emerging technologies and applications in generative AI, robotics and autonomous driving have opened up new dimensions of growth. These developments are expected to accelerate market growth and give rise to entirely new use cases and product categories”, continued Mr. Sorum.
The market for on-device AI solutions is characterized by a high degree of heterogeneity in both technologies and applications, in contrast to cloud-based AI where the hardware is typically designed around predefined use cases and centralized infrastructure. Embedded AI processing can be architected in numerous ways depending on the end use case, and it can be integrated into an almost limitless range of devices across consumer, industrial and automotive domains. This leads to a differentiated market landscape, with unique design constraints, performance requirements and optimization strategies. However, the overarching objective is typically the same for all vendors – to achieve the highest possible performance per watt for the intended use case.
Berg Insight has identified 40 key companies that shape the on-device AI landscape. The market can broadly be divided into two layers. The first encompasses hardware categories such as AI system-on-chips (SoCs) or system-on-modules (SoMs), AI accelerators and AI microcontroller units (MCUs), each optimized for different levels of performance, power efficiency and integration. AI SoCs typically integrate components such as general-purpose and specialized AI compute cores, on-chip memory and connectivity on a single chip, while SoMs extend this design by including external system memory, storage and interface components on a larger board, targeting more advanced use cases. AI accelerators are specialized chips or modules designed to enhance AI inference efficiency in existing systems, typically working alongside a separate host processor in embedded applications. AI MCUs serve lower-power devices by bringing neural network capabilities to sensors, wearables and IoT endpoints where energy efficiency and cost are most critical. The second layer consists of on-device AI platforms that combine hardware, software and developer tools to simplify model deployment and optimization.