Tuesday, March 1, 2016

Pi 3 Benchmarks: The Marketing Hype Is True

The spec bullet list for the latest Raspberry Pi begins as you've already heard: WiFi and Bluetooth, now standard. While this is impressive itself, it doesn't tell the whole story. The Pi 3, with an ARM Cortex A53, is up to 50% faster than the Pi 2 from last year. That's an astonishing improvement in just 12 short months.

In playing with the Pi 3 for a few hours, it's apparent the Pi 3 is fast. It passes a threshold of usability. The Raspberry Pi isn't a computer that just sits on a shelf and runs a few cron jobs and blinks LEDs anymore – this is a computer that's usable as a computer. But how fast is it? By stroke of luck, the official website for the Cortex A53 gives us a direct comparison between this chip and the CPU in the Raspberry Pi 2:

image credit: arm.com
image credit: arm.com

In real devices, the performance improvement from the Pi 2 to the Pi 3 is somewhere between 40 and 60 percent. At least that's what ARM and the Raspberry Pi foundation are claiming. Is this true? There are tests we can run, and the marketing speak, for once, isn't too terribly off the mark.

Benchmarks

The standard benchmarking tool for the Raspberry Pi and other Single Board computers is Roy Longbottom's Raspberry Pi benchmarks. These benchmarks are able to test the computational power of Pis in nearly every respect, from integer performance to floating point performance, to how fast NEON instructions (the same idea behind Pentium MMX) are executed. There are benchmarks for OpenGL, memory speed, and Java.

Many of these benchmarks can be disregarded. There's no need to test how fast the Pi 2 and Pi 3 can access memory; the RAM chip is the same. Java will die a slow and miserable death about a hundred years from now. The state of OpenGL drivers for the Pi is in flux right now. For these tests, I'll only concentrate on the integer and floating point performance of the Pi 2 and Pi 3.

Dhrystone

The Dhrystone benchmark is the standard measure of integer performance, producing a number of Millions of Instructions per Second (MIPS). Tests were conducted on the Raspberry Pi 2 running a 1 GHz, and the Raspberry Pi 3 running at 1.2 GHz. The Raspberry Pi 2 clocked in at 1822 MIPS, The Raspberry Pi 3 clocked in at 2451 MIPS, an increase of 25%.

Linpack Single

Linpak Double

The Linpack Double Precision and Linpack Single Precision benchmark is the standard measure of floating point performance, producing a number of Millions of Floating Point Instructions per Second (MFLOPS). Tests were conducted on the Raspberry Pi 2 running a 1 GHz, and the Raspberry Pi 3 running at 1.2 GHz. The Raspberry Pi 2 rated 151 and 157 MFLOPS for single precision and double precision Linpack, respectively. The Raspberry Pi 3 rated 181 and 192 MFLOPS for single precision and double precision Linpack, respectively. The Linpack performance increase for the Pi 3 over the Pi 2 was increased 17% for single precision and 18% for double precision.

Power

The choice of power connector for the original Raspberry Pi was a stroke of brilliance. In 2012, a micro USB connector was found in every good smartphone, and chargers were plentiful. You could walk into any dollar store and find something that would convert mains power to 5VDC into a USB receptacle. The quality of these cut rate power supplies wasn't really there, but it didn't matter.

The Raspberry Pi capitalized on the ubiquity of cell phone chargers and used this as the power connector. For the time, it was a great idea. A stock Raspberry Pi wasn't going to draw more than the 500 mA that were guaranteed to come from these power adapters. For a time, it worked.

PowerCon

Every USB port on the planet should be able to provide 500mA to peripherals, Raspberry Pis, and USB-powered paraphernalia. Going above this is possible, provided you have a good power supply. While the Pi 1, Pi 2, and Pi Zero can be used with any power supply, this is not true for the Pi 3. Compared to the Pi Zero, it's a power hog. Add in a few USB dongles and the new on-board WiFi adapter, and you'll need a 2.5A power supply at least.

Networking

Until now, there was exactly one networking option for the Pi. You could either use the on-board Ethernet, or grab a ubiquitous 802.11n WiFi adapter and use wireless networking. Notice I said one networking option.

The USB and Ethernet for the Raspberry Pi all go through one chip, the LAN9514 combination 10/100 Ethernet adapter and four-port USB hub. This chip connects to the CPU through the single USB port on the chipset, limiting bandwidth from the four USB ports and the Ethernet to around 480Mbps. Real world use cases will vary.

The Raspberry Pi 3 includes WiFi and Bluetooth, but it doesn't go through the USB port. Instead, the best way to connect a Pi 3 to the Internet now happens through the SDIO port on the CPU. This port is accessible through the GPIO pins and has been used for great effect in custom WiFi hats, but does it deliver more bandwidth?

SCROT

The standard way of testing a network interface on a local network is iperf; all you need to do is run an iperf server on one box, and run the test on the other. This is not a test that will give you the absolute throughput of a network interface. It'll be close if you're testing two computers connected with an Ethernet cable, but when measuring WiFi, iperf should only be used as a basis of comparison. That's exactly what we're trying to measure, anyway: is the Pi 3's built-in WiFi faster or slower than a USB WiFi adapter?

The results are insignificant. An Edimax 802.11n USB WiFi adapter was able to pull down 22.7 Mbps. The Pi 3's built-in WiFi managed 21.2 Mbps. That's more or less the same.

A Real World Test

Dhrystones, MIPS, Linpack, and other benchmarking tools are just that; benchmarks. They don't measure real-world performance of code you would write. There's an simple, standard way of figuring out how fast a little bit of Python will run: finding all the prime numbers below one million. There are 78,497, but how fast can the Pi 3 find them all?

Seive

The Raspberry Pi 3 found all the prime numbers below one million in one minute, 7.49 seconds. With the same Python script running on the same SD card, a Raspberry Pi 2 found all the prime numbers below one million in two minutes, five seconds.

The Marketing Hype Was True

The lede from the Raspberry Pi foundation in announcing the Raspberry Pi 3 was WiFi, Bluetooth, and a CPU that's about twice as fast as last year's Pi 2. The marketing materials from ARM provided a direct comparison between the Cortex A7 in the Pi 2 and the Cortex A53 in the Pi 3, with the latter being a 40 to 60% speed boost over the former.

The hype, for the most part, is true. In real-world tests, the Pi 3 may be about twice as fast. In the standardized benchmarks, the Pi 3 is about 20-40% faster. Networking speeds remain the same, though, but the Pi 3 still opens up a lot of doors to interesting applications. Personally, I have a USB Nintendo 64 controller on order.


Filed under: Featured, news, Raspberry Pi
Related Posts Plugin for WordPress, Blogger...