Running machine learning on a cloud server is easy. Running it on a device that must survive for years on a coin-cell battery is not. TinyML — the practice of deploying machine learning models on microcontrollers and ultra-low-power processors — exists precisely to solve this problem.
At scale, the real constraint isn’t compute capability but energy. For many industrial, environmental, and consumer sensor networks, the practical power budget per node is under 10 milliwatts. In remote deployments, even that can be too high. Devices powered by small batteries or energy harvesting must spend most of their time asleep and wake only briefly to sense and infer.
Quantization is the single most important technique that makes this possible.
Why Floating-Point Models Don’t Work on Sensors
Most machine learning models are trained using 32-bit floating-point numbers. This format is precise but expensive:
- Large memory footprint
- High computational cost
- Significant energy consumption
- Often requires hardware FPUs
Microcontrollers used in TinyML deployments typically have tens or hundreds of kilobytes of RAM, not gigabytes. Many lack floating-point units entirely.
A single small neural network in FP32 format can exceed the entire memory of the device.
Even when memory is sufficient, moving data between memory and CPU dominates energy usage. In ultra-low-power systems, data movement can consume more power than arithmetic itself.
What Quantization Actually Does
Quantization reduces the numerical precision of model weights and activations. Instead of 32-bit floating-point values, models use smaller integer representations.
Common formats include:
- INT8 (8-bit integers)
- INT4 (4-bit integers)
- Binary or ternary weights in extreme cases
An INT8 model requires four times less memory than FP32. More importantly, integer operations are dramatically more energy-efficient on embedded processors.
In practical deployments, switching from FP32 to INT8 often reduces inference energy by 3–10×.
Post-Training Quantization vs Quantization-Aware Training
There are two main ways to quantize a model.
Post-Training Quantization (PTQ)
This method converts a trained floating-point model into a lower-precision version without retraining. It is fast and convenient but can reduce accuracy, especially for complex models.
PTQ works best for:
- Simple CNNs
- Sensor classification tasks
- Anomaly detection models
- Well-behaved datasets
Quantization-Aware Training (QAT)
QAT simulates quantization effects during training. The model learns to operate under reduced precision from the start.
This approach typically preserves accuracy far better, especially when pushing below 8-bit precision.
For sub-10 mW deployments, QAT is often necessary.
Memory Bandwidth: The Hidden Power Sink
In ultra-low-power systems, energy isn’t dominated by math operations but by memory access. Fetching weights from flash or SRAM can cost orders of magnitude more energy than performing a multiply-accumulate.
Quantization reduces not only storage size but also memory traffic. Smaller weights mean fewer bytes transferred per inference.
Some hardware platforms also support:
- Weight compression in flash
- On-chip SRAM caching
- Direct memory access optimization
- Specialized low-power neural accelerators
These features compound the gains from quantization.
Sub-10 mW Deployment Strategies
Achieving reliable inference under 10 mW usually requires multiple optimizations working together.
1. Aggressive Model Pruning
Removing redundant connections reduces both computation and memory requirements. Sparse models can be extremely efficient when supported by hardware.
2. Event-Driven Operation
Instead of continuous inference, sensors wake only when something interesting happens — detected via threshold triggers or simpler algorithms.
For example, an audio sensor may run a tiny wake-word detector continuously, activating a larger model only when speech is detected.
3. Duty Cycling
The processor sleeps most of the time. Even a model that consumes 50 mW during inference can fit within a 10 mW average budget if it runs briefly and infrequently.
4. Hardware Acceleration
Modern microcontrollers increasingly include neural processing units designed for integer operations. These accelerators can execute INT8 inference at a fraction of CPU power consumption.
Pushing Below INT8: INT4 and Beyond
For the most constrained devices, 8-bit precision may still be too expensive. Researchers are actively deploying models using 4-bit or even binary weights.
The trade-off is accuracy. Lower precision increases quantization noise and reduces representational capacity.
However, many sensor tasks are surprisingly tolerant of imprecision. Detecting vibration anomalies, classifying simple sounds, or monitoring environmental conditions often requires far less model complexity than image recognition.
Hybrid schemes are also common, such as:
- INT4 weights with INT8 activations
- Mixed-precision layers
- Per-channel quantization
These techniques squeeze additional efficiency without catastrophic accuracy loss.
Real-World Applications Driving Scale
TinyML at sub-10 mW is already being deployed across industries:
- Predictive maintenance sensors in factories
- Wildlife monitoring devices in remote locations
- Smart agriculture systems
- Wearable health trackers
- Infrastructure monitoring (bridges, pipelines)
In many of these cases, replacing batteries is impractical or expensive. Devices must operate autonomously for years.
The Future: Always-On Intelligence
As quantization techniques improve, the line between “sensor” and “computer” continues to blur. Sensors are no longer passive data collectors; they perform local reasoning.
Future networks may consist of millions of tiny intelligent nodes filtering data before anything reaches the cloud. This reduces bandwidth, latency, and privacy risks while dramatically lowering energy consumption.
In that sense, quantization is not just an optimization trick — it is an enabling technology for distributed intelligence at planetary scale.
Q&A
Q: Can TinyML models match the accuracy of cloud AI models?
Not for complex tasks like large-scale image recognition or language processing. However, for narrowly defined sensor tasks, TinyML models can achieve comparable accuracy while using a tiny fraction of the energy.
Q: Why not simply use a more powerful processor?
Higher performance usually means higher power consumption. In remote or battery-powered deployments, energy availability is the dominant constraint, not compute capability.
Q: Is INT8 the standard for TinyML today?
Yes. INT8 offers a strong balance between efficiency and accuracy and is widely supported by hardware accelerators and software frameworks.
Q: How long can a sub-10 mW sensor run on a coin-cell battery?
Depending on duty cycle and battery capacity, operation can range from several months to multiple years.
Q: Will future hardware eliminate the need for quantization?
Unlikely. Energy efficiency will remain critical, and lower-precision computation inherently consumes less power. Quantization will continue to be central to ultra-low-power AI.
References
- Google AI. (2025). Quantization Techniques for Ultra-Low-Power TinyML Devices. Google Research Blog.
- ARM Holdings. (2024). Optimizing Machine Learning for Cortex-M0+ Class Devices. ARM Developer Documentation.