AI PCs Aren't Good at AI: The CPU Beats the NPU

In a recent article, Pete Warden expresses his disappointment with the AI performance of Qualcomm's NPUs on Windows tablets. Despite the chips' promise of up to 45 trillion operations per second, Warden's benchmark tests using foundational AI operations only achieved 573 billion ops/sec, which is less than 1.3% of the advertised capability, and four times less than an Nvidia RTX 4080 in his gaming laptop. He speculates that the issues may stem from the software stack and hopes for future improvements through software updates.

• Qualcomm's NPUs are advertised to deliver up to 45 trillion ops/sec.
• Warden's benchmarks show only 573 billion ops/sec on the NPUs.
• Performance on Windows is compared unfavorably to similar chips on Android.
• Warden has publicly shared a benchmark for others to test.
• He remains hopeful for hardware improvements through software updates.