These days, everywhere you go, people are talking about AI. Almost everyone. And when I say people, I mean every living thing in our society — from the engineer who uses it every day, or builds products with it for customers, to the retired neighbor, the teacher, the student. Everyone.
I don’t know the language of other living things, like birds. But I think if you could translate the song of birds, they would probably be talking about AI too. Some praise it. Some curse at it. Some are addicted to it.
Before, when people said “AI” they meant the AI running on servers, on the cloud, on big systems with huge resources. But the game has changed. AI came down into embedded systems too. And thanks to that, we now have edge AI — models deployed on embedded boards, running locally. LLM, SLM, vision, whatever fits.
This post is in two parts. It’s the result of my own investigation over the last few months — mostly to answer my own questions about how this stuff actually works. But then when I shared the first version, I got feedback. Good feedback, sharp questions. Some of what you’re going to read is the answer to that feedback. I hope it’s useful to you.
In Part 1, I want to cover what’s actually inside the TDA4VM, where the famous “8 TOPS” number comes from in real silicon, and one big question I kept getting from readers: if the chip can’t learn, is it really AI?
Part 2 will be the harder stuff — how precision changes everything, why memory bandwidth is the real bottleneck, the software stack, and why 8 TOPS becomes around 2 TOPS in real life.
Let’s start with what’s on the chip.
What’s actually inside the TDA4VM
The BeagleBone AI-64 sits on top of Texas Instruments’ TDA4VM, part of the Jacinto 7 family. This SoC was originally designed for cars — driver assistance, ADAS, industrial vision. That matters, because it shapes how the whole chip is built. It’s not a phone SoC. It’s not a desktop CPU. It’s a chip built around the idea that different jobs need different silicon, all on the same die.
Here’s what you actually get:
| Block | Count | Type | Frequency | What it’s for |
|---|---|---|---|---|
| Cortex-A72 | 2 | 64-bit application cores | 2.0 GHz | Linux, your applications |
| Cortex-R5F | 6 | 32-bit real-time cores | 1.0 GHz | Real-time control, safety |
| C7x DSP + MMA | 1 | Vector DSP + matrix engine | 1.0 GHz | The AI engine — 8 TOPS (INT8) |
| C66x DSP | 2 | Floating-point DSP | 1.35 GHz | Signal processing, pre-processing |
| PowerVR GPU | 1 | Rogue 8XE GE8430 | 750 MHz | Graphics |
The Cortex-A72 cluster is what you talk to when you SSH into the board. It runs Debian Linux. It feels normal — like a slightly slow laptop. That’s by design. Linux on the A72 is the friendly face of the chip.
But the part of the chip that does the actual AI work is the C7x DSP and the MMA sitting next to it. That’s where the 8 TOPS comes from. Everything else is support staff.
This is one of the things people miss when they read “BeagleBone AI-64, 8 TOPS” on a spec sheet. The 8 TOPS is not spread across the chip. It lives in one block. The rest of the chip is there to make sure that block has clean data to work on.
In the previous post I wrote about the Cortex-R5F side of this chip — the real-time cores. Now we’re looking at the AI side. Same SoC, totally different world.
The C7x DSP — the math engine
The C7x is the latest generation DSP from TI. It replaces the older C66x DSP and the EVE (Embedded Vision Engine) and combines them into one core. So when you see “C7x”, think: vector math machine.
A few details that matter:
The C7x is a VLIW processor — Very Long Instruction Word. That means it can pack many small operations into a single instruction. With its wide SIMD path, it can do up to 64 operations per cycle on vector data. That’s why it’s called a vector DSP — it doesn’t process numbers one at a time, it processes them in groups.
There’s another detail that’s easy to miss. The C7x has a split datapath: an A-side and a B-side.
- The A-side handles scalar work — control flow, branches, memory loads, and stores.
- The B-side handles vector math — the actual heavy lifting.
This split matters because it means the math doesn’t have to wait for the program logic. While the A-side is loading the next chunk of data, the B-side is already crunching the previous chunk. If the compiler does its job well, the pipeline never stops.
But the C7x by itself is “only” around 80 GFLOPS in floating point. Respectable, but not 8 TOPS. So, where does the 8 TOPS actually come from?
Where 8 TOPS actually comes from
Right next to the C7x sits the MMA — Matrix Multiply Accelerator. This is a dedicated hardware block, tightly coupled to the C7x. It’s not programmable like a CPU. It’s a fixed-function machine that does one thing: large matrix multiplication on 8-bit integer data.
This is the actual AI engine. And the math behind the 8 TOPS is simple once you see it.
The MMA contains a 64×64 MAC array. MAC = Multiply-Accumulate. So in one cycle, the MMA can do 4,096 multiply-accumulate operations on INT8 data.
A single MAC counts as 2 operations (one multiply + one add), so:
4,096 MACs/cycle × 2 ops/MAC × 1 × 10⁹ cycles/sec = 8.192 × 10¹² ops/secThat’s 8.192 TOPS. Round it to 8 and that’s the marketing number. Now you know exactly where it comes from.
Notice what this means: the 8 TOPS is only for INT8 matrix math. It’s a peak number. It’s not a general-purpose number. If your workload doesn’t look like “lots of INT8 matrix multiplies”, you’re not getting 8 TOPS. We’ll come back to that in Part 2.
The question I kept getting — “but can it learn?”
After I shared the first version of this article, the most common pushback was some version of:
“If the C7x-MMA can’t actually train models, is it even AI? It’s not even ML.”
Honest question. And it took me a minute to realize it’s not really a hardware question. It’s a definition question.
Let me lay it out plainly.
Every neural network in production goes through two completely separate phases:
Training — the network learns. You feed it labeled data. It guesses. You measure how wrong it was. You adjust millions of weights to make it slightly less wrong. Repeat for hours, days, or weeks. Training needs:
- floating-point precision (FP32 or FP16),
- gradient computation (backpropagation),
- huge memory bandwidth,
- usually a GPU cluster in a datacenter.
Inference — the network runs. You take the trained weights, give it a new input, and it produces an answer. No learning happens. The weights don’t change. Inference can run with much less precision (INT8 is fine), no gradients, modest memory, and can sit on a tiny accelerator at the edge — sometimes on battery power.
These two phases are so different that the hardware industry has split into two completely different product categories:
- Training silicon lives in datacenters. NVIDIA H100, Google TPU, AMD MI300.
- Inference silicon lives in the field. Every NPU in every phone. Every accelerator on every camera. Every embedded AI chip in every car.
The TDA4VM’s MMA is inference silicon. By design.
This is the part where the definition question matters. If your definition of AI is “a chip that learns on its own”, then by that rule:
- Apple’s Neural Engine in every iPhone since 2017 → not AI.
- Google’s Edge TPU on the Coral → not AI.
- NVIDIA’s NVDLA on every Jetson → not AI.
- Hailo, Movidius, Hexagon NPU on Snapdragon (95% of the time) → not AI.
- The TDA4VM’s MMA → not AI.
That’s a definition that excludes basically every piece of edge AI hardware shipped in the last decade. It’s not a useful definition. The industry calls these chips inference accelerators, and they are absolutely AI/ML hardware. They run neural networks. The fact that the training of those networks happened on a different machine, somewhere else, doesn’t change what the chip is doing when it runs them.
A simple comparison: a CD player doesn’t record music. Nobody says a CD player isn’t a music device.
There’s also a deeper reason this confusion exists, and it’s worth saying out loud. Training requires backpropagation. Backprop means you have to keep all the intermediate activations from the forward pass in memory, so you can compute gradients on the way back. For modern networks this means gigabytes of high-bandwidth memory, often HBM. The TDA4VM has 8 MB of on-chip L3 SRAM and a 32-bit LPDDR4 interface to external RAM. There’s just no way that hardware can do efficient backprop for modern networks. Even the NPUs in flagship phones can’t really do it — when a Snapdragon does “on-device learning”, that work usually runs on the CPU or GPU, not on the Hexagon NPU.
So the honest answer to “but can it learn?” is:
No, it can’t. And neither can almost any other edge AI chip. They’re not supposed to. They’re supposed to run trained models very fast and very efficiently. That’s what edge AI is.
Why this distinction makes edge AI interesting
Here’s something that surprised me when I started looking at this seriously: inference is where the actual engineering happens.
Training, in a way, is the easy part. You throw money at GPUs in a datacenter, you write a PyTorch script, you wait. The bottlenecks are budget, data, and patience.
Inference at the edge is harder, because every constraint is real:
- The chip has 8 TOPS, not 800.
- The power budget is 5–10 watts, not 700.
- The latency target is 30 ms, not “next week.”
- The memory bandwidth is what’s on the board, not what’s in a server rack.
- There’s no swap. There’s no second chance. Either the frame gets processed before the next one arrives, or it doesn’t.
To make a model run well on the C7x-MMA, you have to quantize it from FP32 down to INT8 without losing too much accuracy. You have to schedule layers so weights stay on-chip. You have to fit within the operator set the toolchain supports, or rewrite the parts that don’t. You have to measure real-world FPS, latency, and power, and prove the system meets its requirements under thermal load.
That’s not “less than” training. It’s a different discipline. And it’s the discipline that decides whether AI actually works in the physical world.
Wrap-up of Part 1
So far we’ve seen:
- The TDA4VM is a heterogeneous SoC. Different cores for different jobs.
- The 8 TOPS comes from one specific block — the MMA, a 64×64 INT8 MAC array clocked at 1 GHz, paired with the C7x DSP.
- The chip is inference silicon. Like almost every edge AI accelerator. That’s not a limitation, it’s the category.
But this is only the marketing-friendly half of the story. 8 TOPS is a peak number, under ideal conditions. In real applications the number drops — and how much it drops depends on your model precision, your memory layout, and how well you use the software stack.
That’s what Part 2 is for.
Next time, in Part 2
Coming up:
- Why INT8 vs INT16 vs FP32 completely changes what the chip can deliver.
- The memory wall — the LPDDR4 bottleneck and why the 8 MB on-chip SRAM is the real currency.
- The TIDL software stack — what it does, where it shines, where it bites.
- Why a real YOLO model runs at around 2.24 TOPS effective, not 8 — and the architectural reasons behind that 28% number.
- The camera pipeline (VPAC + ISP) you can’t skip.
- And a quick word on the part of this chip nobody talks about: functional safety.
That’s enough for today. Part 2 is coming.