Sensor Hub, Motion Coprocessor or DSP?

In my last blog I referred to the return of digital signal processing in the form of a discrete, low-power chip that acts as co-processor to the main applications processor of a smartphone or mobile device. Taking this one step further raises some fundamental questions. How complex are these helping hands in terms of their signal processing capabilities?

Determining motion

Let’s take the iPhone as an example. The latest iPhone 6 uses the newer M8 motion coprocessor (an NXP Semiconductor LPC18B1 chip) in combination with Apple’s very own A8 / APL1011 applications processor as outlined in a recent teardown by TechInsights. The word motion provides the first clue regarding the primary purpose of the chip: monitoring movement to determine if the user is sitting, running, walking, cycling or driving. In fact iPhone apps interrogate this user activity status from the CMMotionActivity class offered by the iOS operating system. To determine user activity, the motion co-processor will most likely take acceleration readings in three axes (x, y and z) from the accelerometer sensor. Repetitively calculating the max and min values will enable distinction between sitting and running/cycling and driving, regardless if the user is holding the phone, has it stashed in a trouser pocket or nestled in a car’s device cradle.

However it’s far more difficult to distinguish between running or cycling as the acceleration in all three axes is quite similar for both activities. This is where measuring yaw (rotation) using the gyroscope sensor comes into play. Cycling has a smoother repetitive motion than jogging. By calculating the spectrum of the yaw rate using an FFT (Fast Fourier Transform), cycling will show a single dominant frequency as determined by the cadence of the cyclist. Of course calculations can be further relaxed if likelihoods are taken into consideration. For example, a measurement showing an impromptu change of states from cycling to driving is somewhat implausible. Statistical models as offered by Bayesian probability inference or Markov chains come to the rescue here. And if all goes wrong and a confident activity guess is out of question, iOS luckily provides the state unknown.

The M8 LPC18B1UK chip is based on ARM’s Cortex M3 core. In contrast to the follow-up Cortex M4 core, the M3 does not include a DSP instruction set. So, it ’s plausible that activity tracking calculations are performed at a low frequency. In fact the chip is clocked at only 0.15 GHz. That in turn makes it battery efficient so it can run constantly without ever taking a break. Even in the iPhone’s standby mode it collects, calculates and caches sensor data. The iPhone stores results for a maximum period of seven days in the LPC18B1UK’s on-chip flash of 1 MByte. Sampling rates of 14-bit accelerometer data are thus probably in the 1 to 2 Hz range. In other words, really slow.

From lightweight processing to heavy lifting

As the iPhone 6 example above underlines, motion detection is all about lightweight processing. Similarly Atmel’s sensor hub solution as found in several smartphones from Samsung (see Wikipedia) use Atmel’s SAM D20 which features an ARM Cortex M0+ core on-chip and no DSP functionality. However Atmel’s sensor hub roadmap points to the follow-up SAM G51/53 which is based on the DSP-rich ARM Cortex 4 core. Sensor hubs are evidently transitioning from lightweight DSP processing to heavy DSP lifting. Recent smartphones from HTC, Nokia, Samsung and Sony confirm this trend: they use Qualcomm’s Snapdragon 800 SoC (system-on-chip) family with an on-chip sensor engine based on their powerful 32-bit Hexagon DSP core that also offers floating-point support. Next-generation smartphones, wearables and other mobile electronics will thus not only capture both user and environmental sensor data but also combine these streams in an eclectic signal processing mix to provide never-seen-before smarts for the user.

Apple's Watch and Sensor Magic

Slated for release in early 2015, the Apple Watch will certainly raise eyebrows in many ways, possibly heralding in the consumerization of smartwatches. The company’s general mobile philosophy has been to cram more sensors into products than most of its competitors do. By integrating sensors, users not only carry a desktop computer in pocket format with them but a truly smart mobile device.
One of the more novel sensor solutions on-board the Watch can be seen on the back of its stainless steel and alumium casing: an optical system consisting of a combination of LEDs and photodiodes that gather data from both the visible and infrared spectrum.

Apple Watch’s LEDs and photodiodes for heart rate measurement

Photo courtesy of Apple, Inc.

With these sensors the Watch becomes a fitness device capable of measuring the user’s heart rate through optical sensing. In very basic terms this means shining a light through the skin of a user’s wrist and monitoring the change in blood flow in order to determine his or her pulse. It’s all about addressing today’s trend of the “Quantified Self” by continuously logging one’s daily activity as a means to improve basic fitness or even workout endurance over time. Electronic activity trackers are mostly sold as wearable bands today from the likes of Fitbit, Jawbone and Nike , but smartwatches and bands will converge at some future point. In 2014 around 10 million activity tracker bands will be sold worldwide and this figure is expected to triple in 2015. The market segment is set for exponential growth.

The smartwatch as a meaningful medical device

Lower-cost wrist bands are mostly attractive for those obsessed with tracking their activity and exercise. Yet will smartwatches such as Apple’s Watch also address real patients or the elderly who’s medication regime depends on regular and accurate monitoring of heart rate, blood pressure, blood oxygen levels (oximetry) or even blood sugar (glucose)? Two announcements provide some clues on this potential. Firstly, Apple has silently been hiring medical sensor experts over the past few years. The Apple Insider article identifies that a number of these hires were previously employed with serious medical companies. Secondly, Apple has filed a slew of patents related to medical monitoring over the past years. Patently Apple reports on several health and biometric-related patents in this area. Skeptics however purport that accurately measuring heart rate, as an example, is a big leap from the techniques that fitness trackers employ, maintaining that doctors rely on electrical not optical measurements for accuracy. Wearing a device on one’s wrist for correct pulse tracking requires it to be strapped very tightly so the sensor cannot move around during measurements. In addition, as users who measure blood pressure know, wrist measurements can be off from their true value by quite a bit because blood flows a lot slower by the time it reaches the body’s peripheral zones like the wrist. Many open questions and much room for speculation remain. Mike Nicholls of startup88 provides some “can’ts” why the Watch won’t cut it on the medical front. Are there any cans?

Many sensors make light work

Whilst accuracy imposes certain onerous requirements, it’s probably shortsighted to brush off the Watch’s possibly far-reaching medical potential. Point in case: today’s smartphones do not determine user location single-handedly by satellite (GPS) but in combination with input from other embedded sensors. Using complex calculations, data from their accelerometer, gyroscope, and magnetometer predict the user’s current position based on his previous one if a GPS signal is not available (dead reckoning). Multiple sensors termed as a sensor hub in combination with a power-friendly, always-on dedicated processor running signal processing algorithms can work sensor magic - possibly even medical magic. The Apple Watch is rumoured to have more than 10 sensors and will most likely also feature a motion coprocessor like the M7 (iPhone 5S) or M8 (iPhone 6). Combined with Apple’s talent pool of medical expertise it’s highly likely we can expect some surprises on the road ahead. If not in a 1st generation Watch, then for sure in its later product cycles.