4K Smartphones - Stand and Deliver

In my last post I ventured to predict that 4K displays will become mainstream on high-end smartphones at some point even though they’re unlikely to deliver a better viewing experience for the mobile user on average. To become a game changer, it takes more than just adding a impressive new feature to a device to attain that immersive cinematic experience. What about 4K video content? Is enough readily available? Are the wireless networks and mobile infrastructure in place to transport high-resolution content? What are the effects of 4K displays on the smartphone itself? It’s a tall order to provide a comprehensive picture in a blog, so I’ll attempt providing some vignettes of insight in a question and answer style. We’ll shortly probe the ecosystem from production through to transfer and finally consumption of 4K content to arrive at the bigger picture.

"Looks pretty much like 4K to me"

Seeing the difference

Spicing up the quality of images or video is not only a case of increasing resolution. It’s technical and personal matter at the same time.

What parameters influence the perceived quality of a display?

A combination of technical factors such as the screen size, display resolution, frame rate, color depth, dynamic range, viewing distance and other parameters make up for the quality of the viewing experience. In addition, your own eyesight plays an important role, in particular its ability to distinquish fine details, known as visual acuity. The resulting mix of objective and subjective factors makes it extremely difficult to pinpoint a single metric as being paramount in providing a superior viewing experience.

What does it take to produce 4K broadcast quality content?

Producing 4K content dictates a four times higher resolution than the prevalent HD (high-definition) standard, resulting in 4x the amount of data too. That translates to very expensive equipment in the production chain (cameras, lenses, switchers, encoders, storage disks, editing workstations) so that only a trickle of material is currently shot in 4K. Over time equipment pricing will erode and options will increase. Shooting in 4K will ultimately ensure that content is future proof, spurring on its adoption.

What content is available in 4K resolution to date?

YouTube Netflix, Amazon, Sony, DirectTV, Comcast are just some of the names who boast 4K UHD movies, TV shows, or video clips, all of which have caveats of sorts. A good overview of 4K UHD programming available in the USA currently (December 2014) can be found at Digital Trends. Indeed, it’s quite a limited offering but it’s an incoming tide that’s rising.

Squeezing 4K data

Getting 4K content to the end-user requires high-bandwidth transmission paths. 4K codecs that efficiently compress the material when it is captured (encoder) and decompress it again when viewed (decoder) play a pivotal role in easing bandwidth requirements and the cost of transfer.

What are the transport requirements for 4K video?

As can be seen from the listing of offerings at Digital Trends, most 4K content is streamed over the Internet and requires a 25 Mbps channel. Many experts contend that 36 Mbps is the best channel bandwidth for delivering Ultra HD, whilst others purport that 15 Mbps is sufficient for a decent 4K experience that is also commercially viable. Whatever the case, that’s certainly a lot more compared to delivering a HD (1K) stream at 3 Mbps. It also costs approximately 5x more.

	SD (720 x 576)	HD (1920 x 1080)	UHD (3840 x 2160)
Recommended Internet Speed	3 Mbps	5 Mbps	25 Mbps

Netflix’s recommendations on Internet speed for viewing their movies and TV shows

Why are differing minimum 4K bandwidth requirements quoted by experts?

This depends on the type of content and the codec used for compressing the video footage. For example live sports events with fast moving action will require higher frame rates (more data to encode) for a fluid viewing experience. In addition the encoder used to reduce the data must compress at a faster rate due the live nature of the event. Stated in simple terms, the more time a codec has available, the more it can squeeze the data without sacrificing quality. So it stands to reason that films shot and compressed in the studio require less transport bandwidth than live sports events broadcast on the fly.

What codecs are available for 4K video compression and do they differ in the quality and bit-rate?

Of late there’s been a pitting of solutions between Google’s VP9 that follows an open source approach, and MPEG / ITU groups’ HEVC (High Efficiency Video Codec, also known as H.265). The latter’s previous H.264 (MPEG) codec has been the defacto standard for HD content in the past, but Google is hot on the heels to change this for 4K. Then there’s another initiative from Mozilla and Xiph.org called Daala that claims it will beat both on technical merit. In a future post I’ll attempt to uncover some of the key differences of these codecs in their complexity, delivered quality, achievable bit-rate reduction, available content offerings and devices that support each.

Please, no frame freezes or fallouts

Many of us know how annoying a sudden and unexpected throttling of bandwidth can be, especially when enjoying streamed video or audio. Consistent channel bandwidth is of the essence, so let’s take a look at the options.

What channels are available for transmitting 4K Ultra-HD content?

Several options are available such as satellite, microwave, cellular and fixed networks using cable or DSL. Currently the majority of 4K content is available as video-on-demand (VoD) that is stored and then streamed through fixed networks (cable/DSL) to the end user. On the broadcast TV front, satellite providers like DirectTV and BSkyB will lead the 4K race. Terrestial TV broadcasting will require more time for standards to come into place and availability will largely depend on country initiatives in transitioning to newer technology with Japan and Korea at the forefront.

Will mobile networks have sufficient bandwidth to transmit 4K content?
4G cellular networks typically support download rates of up to 150 Mbps for LTE Cat 5 smartphones that are prevalent on the market . More than enough for one UHD channel at 25 Mbps one would think. Keep in mind though, the cellular broadband network is being shared by many people at the same time. In fact we’re dealing with a top theoretical download speed that most of us won’t ever witness in the real world. Even worse, the data rate may change unexpectedly depending on network usage, and your 4K video may suddenly freeze.

Can Wi-Fi reliably stream 4K video content?

The more recent WiFi standards that employ 40 MHz channel bands (802.11n or 802.11ac) provide sufficient data throughput to support 4K video transmission in theory. In practice however, performance is unpredictable as those of us who have tried streaming HD video at home will know. This is largely due to contention between neighbouring WiFi networks whose data rates start sagging as they counter the chatter of next-door access points on the same frequencies. In addition WiFi microwaves, especially those operating in the 5 MHz band, are attenuated by walls and floors, leading to a 80 - 90 % drop in peak rate compared to the access point being in the same room. In short, uninterrupted video delivery is simply not reliable enough.

What other wireless standards can handle 4K streaming and when will they become available?

Two new standards, WiGig and WirelessHD, are out there and both operate in the 60 GHz band. They are much faster than Wi-Fi 802.11ac or LTE mobile broadband technologies. Their formidable throughput rates (7.5 Gbps for WiGig) are targeted at the wireless delivery of high-definition content. Yet the 60 GHz signals they use cannot penetrate walls. It’s all about connecting computing and entertainment devices in the same room without cables. The idea is to turn mobile devices into media stations that wirelessly dispatch streams to 4K TVs and displays. Devices supporting the new standards are expected to start shipping this year and they will allow uninterrupted wireless streaming of 4K content, albeit in the same room.

Expensive power hogs

Smartphone innovation happens at a mindboggling rate and users often wait for announced models to be released before replacement. Even if 4K is one of those desirable new features, there are other factors to observe.

What are the drawbacks of smartphones with 4K displays?

Power and cost are two inhibitors in the uptake of 4K displays on smartphones. The display is one of the major culprits in draining the battery on a mobile phone and the amount of power consumed is directly related to the size of the display. 4K also means four times more data to store and process. More memory and busier units on the SoC (system-on-chip) will sap precious juice from your phone’s battery even faster. In addition, the display is the component that costs the most in a smartphone. Paying more and having to charge your phone more often don’t speak for a runaway market success.

Can smartphones record 4K video?

The minimum requirement for a smartphone camera to capture 4K content is a camera with at least 8 megapixels (MP) of resolution. Most mid-range devices already sport such pixel magic. But can available smartphones capture and encode 4K video? Suprisingly yes, even though most don’t feature 4K displays. For example, both Apple’s iPhone 6 and 6 Plus models (8 MP camera) support 4K video recording; Samsung’s Galaxy S5 (16 MP camera) and Sony’s Xperia Z2 (20.7 MP camera) both use Qualcomm’s Snapdragon 801 that supports video capture and playback using the HEVC / H.265 codec. In fact, an impressive list of smartphones supporting HEVC can be found at phoneArena.com which are able to record 4K videos.

A nascent technology gaining momentum

What do you do if 4K content is limited, seamless transfer paths still patchy, the first smartphones with 4K displays just around corner, yet many featuring 4K capture available right now? 4K’s marketing machine has the answer and is harping that user-generated 4K content on mobiles will bridge the gap. Smartphones will be the force to bring 4K into the limelight as users record 4K footage and either watch it on 4K TVs or upload it to YouTube. A bulletproof business model or high hopes on an act of faith? Whatever the case, it is apparent that the industry is working its fingers to the bone towards a single new display standard. At some point the law of large numbers will bring down costs and herald in the age of 4K phablets, one way or another, sooner or later. The market research company ABI Research forecasts almost 500 million 4K display-enabled mobile phones to be sold in 2019.

It’s still an HD world out there and will be for quite some time, but 4K UHD is on the move.

4K on Mobile - splash, bang or thud?

The new year has just begun and the mobile industry is bracing for the splash of 4K displays in the smartphone market. Will smartphones with Ultra-High-Definition (UHD - the acronym denoting 4K resolution) become a runaway success? They promise a four times higher resolution than the current high-end of small screen technology. They also require four times the amount of data to transfer and a lot more processing muscle to handle the extra workload in tasks such as compression and image enhancement .

 
Rumours have it that Samsung’s Galaxy Note 5 (to be released mid 2015) will feature a 4K screen resolution of 2160 x 3840 pixels on a 5.9 inch display delivering an astonishing pixel density of 746 ppi (pixels per inch). What does this all mean and does it make sense, or is it just one more superlative being added to the spec list of megapixels, octacores etc. to dazzle the consumer?

10-fold increase in 4K smartphone unit shipments

Not if you believe Qualcomm, the Nr. 1 chip supplier in the mobile handset world with a market share of over 50 % for baseband modems in cell phones according to Forward Concepts . A quick look at 4K support in their Snapdragon 805 system-on-chip reveals an unshakeable commitment and formidable investment in this latest display technology.

Smart Phone Display Trend – Source: IHS DisplaySearch, 2015 IHS DisplaySearch expect a 10-fold increase in annual shipments of 4K displays on smartphones by 2017. A modest 6 million units are forecast to ship this year (2015).

Why the 4K smartphone market differs from its TV counterpart

Most of us have witnessed how TV formats battled in moving from PAL/NTSC to high-definition HDTV and the time it took for content and broadcasting in this format to become common. Seers from the mobile world expect the change to 4K to be a completely different kettle of fish. Uptake of this superior technology will be swift . The premise is that mobile devices such as smartphones don’t need to wait for 4K content to become available on a large scale as they are already equipped with cameras and codecs that generate 4K video which can be shared with like-minded. The breakthrough will be further fueled by the fact that consumers buy new mobiles every other year. That’s a very different replacement cycle to PCs or TVs, driving down the cost of high resolution handsets at a much quicker rate. 

Your best viewing distance

 The proposition of enjoying an immersive cinematic experience, viewing an image in fine detail, or simply reading crisp text are undeniably attractive. Does 4K on a handheld deliver on this promise and improve the user experience compared to lower resolution displays? Image quality is an extremely subjective experience and its perception depends on a multitude of factors, not the least of which being the condition of your eyesight. Objectively speaking however, a higher resolution always makes perfect sense the closer you get to the screen or the bigger the screen gets. At some point your eyesight will notice the edges of rasterized text or displayed objects. But how close is close and how big is big? Smartphones are getting bigger by the day - sometimes that small hand can barely hold newer models. In addition, the phone, or shall we call it phablet (phone + tablet) , is at most an arm’s length away from your eyes, usually viewed less than elbow’s length away or some 40 cm / 16 inches. Experts from the display market rely on a simple but reliable rule of thumb for the “best” viewing distance based on screen resolution and size for 16:9 aspect ratios:

• for UHD (2160 x 3840) it’s 1.5 x height of the display
• for Full HD (1080 x 1920) it’s 3 x height of the display

Even the best-sighted of us will no longer be able to distinguish differences to the next lower-resolution display beyond this measure.

So, how does this translate to the latest phablets?

 

	Screen Diagonal	Screen Size	Resolution In pixels	Pixels-per-Inch (PPI)	Best Viewing Distance (landscape)
SAMSUNG Galaxy Note 5 **	15 cm 5.9 "	13.1. x 7.4 cm 5.2 x 2.9 "	3840 x 2160 UHD	746	11 cm 4.3 "
SAMSUNG Galaxy Note 4	14.5 cm 5.7 "	12.6 x 7.1 cm 5.0 x 2.8 "	2560 x 1450 WQHD	515	16 cm 6.3 "
Apple iPhone 6 Plus	14 cm 5.5 "	12.2 x 6.9 cm 4.8 x 2.7 "	1920 x 1080 Full HD	401	21 cm 8.3 "
Apple iPhone 6	12 cm 4.7 "	10.4 x 5.9 cm 4.1 x 2.3 "	1334 x 750	326	31 cm 12.2 "

       ** estimated release is mid 2015

As the table suggests, a cinema quality movie on a 4K smartphone will deliver a better user experience if viewed from a distance of less than 11 cm / 4.3  inches (landscape mode). That’s really close in front of the eyes. Of course, zooming ever further into a hi-res photo by pinching the touchscreen remains an attractive side benefit of 4K content on 4K displays. The “best viewing distance” matches Qualcomm’s affirmation on the average person being able to immediately appreciate the superior quality of a 344 ppi display if viewed closer than 10 inches (25 cm). However note, 344 ppi is not even Full HD resolution.

Notwithstanding, the mobile future is 4K

In the mid nineties when browsing the Internet and Search were at their genesis, I remember contending with technology peers that the future of the medium would remain text-based, more than enough for the emerging knowledge-based, networked society at the time. My rationale was that pictures, let alone videos, would never break through because of bandwidth and cost considerations. How wrong could I have been! As the saying goes “a picture says more than 1000 words”. And if a picture actually sells more than 1000 words, the dynamics of retail and commerce will drive adoption for certain. So, with this in mind and unencumbered by conventional wisdom, 4K on smartphones are destined to break in and break through, even if their added value may not be apparent to the mobile user at first glance.

The Charm of Personal Medical Electronics

Last week I attended the Medica right on my doorstep here in Düsseldorf. My goal was to investigate the state of mobile healthcare - mHealth - with a focus on personal medical electronics. We’ve seen lots of fitness devices becoming mainstream over the past years, but what about personal devices for those who have a chronic ailment? For the uninitiated: Medica is the world’s largest event in the medical sector and attracts over 130 000 visitors each year to see products and services from some 5000 exhibitors.

Photo courtesy of Messe Düsseldorf GmbH

Having steered well clear of pavilions covering surgical devices (not for the faint of heart), one thing that immediately caught my eye was the large number of country booths in each hall inhabited by many smaller companies with innovative products. After visiting around 10 of these, I was surprised to often find the conversation steering towards regulation, compliance and the lobbying power of large incumbents in the medical and pharmaceutical field. It quickly became clear that mHealth has an armoury of jaw dropping technology at its disposal to propel the industry forward, yet security, privacy concerns, slow regulation & legislation and the interests of industry heavyweights curb and constrain enthusiasm and progress in adopting new technologies. Mobile health (mHealth) technology circumvents the technical challenges of existing health systems and provides a more flexible way of enhancing patient self-care. mHealth ultimately equips the patient with completely new 24/7 self-monitoring capabilities that change the dynamics of the doctor-patient relationship.

A second observation from my visit was the number of “cloud based” solutions on offer. In particular for establishing new electronic medical record systems to receive and process information from mobile devices, moving them to “the cloud” so that physicians and other experts have “anytime” and “anywhere” access to a patient’s health status.

The smartphone is a welcome hub for many mHealth solutions. It records incoming data from specialized sensors monitoring a patient’s condition, and wirelessly moves this data to “the cloud”. Often the device’s display provides insightful visual feedback. If necessary, acoustic alarms or haptic prompts notify the patient to take a corresponding course of action.
On the sensor front there are a plethora of exciting solutions around, many if the form of “wearables” and some as “implants”. Two examples from my Medica visit, both from companies who license their technology, give an idea of what’s round the corner. The Israeli company Healthwatch Technologies showcased T-shirt-like garments with interwoven textile electrodes that enable hospital-grade ECG (electrocardiogram) monitoring for patients with heart arrhythmia or palpitations. The garments are comfortable and machine washable.

Photo courtesy of Healthwatch Technologies

The Swiss company Biovotion, award winner in Nokia’s recent Sensing X Challenge, demonstrated a device worn on the upper arm loaded with specialized sensors that monitor physiological parameters. The current version measures five parameters and the next-gen device expands this to thirteen.

Photo courtesy of biovotion

Future versions will also allow measurement of blood glucose levels for diabetes patients using patented dielectric and optical spectroscopy techniques. Non-invasively of course, meaning no piercing of the skin and no blood. Present day self-monitoring of glucose is mostly done using test strips and readers. Pharmaceutical incumbents in this multi-billion dollar “strip” market will certainly raise their eyebrows once such technology reaches the street.

The two examples above demonstrate what novel personal medical electronics can achieve based on available technology.

In general, personal monitoring of the following vital metrics is certain to provide telemedicine  with a fresh impetus going forward:

• heart rate
• body and/or skin temperature
• respiratory rate
• ECG (heart arrhythmia)
• blood pressure (hypertension)
• blood glucose levels (diabetes)
• lung volume/spirometry (asthma, COPD)
• oxygen level in blood (asthma, COPD, sleep apnea)
• sleep patterns (sleep apnea)
• sweat (anxiety and stress)

Health insurers will love this technology because it keeps patients out of hospitals or reduces the time they need to spend in hospital, saving costs where it hurts most.

Privacy and security concerns are among the greatest barriers hampering mHealth adoption. They slow down innovation in this field. Understandably. Who wants their health monitored by a smartphone app if the data could be purchased by an insurance company or other third party covertly. Their interest in optimizing profit or increasing revenue might well rank higher than the patient’s well-being. So long as no clear legislation is in place on both national and international fronts that ascribes such abuse as a criminal offense, this promising area of the medical industry will not be able to fully take advantage of the opportunities offered by consumer electronics and match the fast cycle of innovation in the latter field. mHealth is at an important intersection right now. The technology has the potential to radically and effectively change the treatment of many lifestyle diseases which plague industrial societies.

Sensor Hub, Motion Coprocessor or DSP?

In my last blog I referred to the return of digital signal processing in the form of a discrete, low-power chip that acts as co-processor to the main applications processor of a smartphone or mobile device. Taking this one step further raises some fundamental questions. How complex are these helping hands in terms of their signal processing capabilities?

Determining motion

Let’s take the iPhone as an example. The latest iPhone 6 uses the newer M8 motion coprocessor (an NXP Semiconductor LPC18B1 chip) in combination with Apple’s very own A8 / APL1011 applications processor as outlined in a recent teardown by TechInsights. The word motion provides the first clue regarding the primary purpose of the chip: monitoring movement to determine if the user is sitting, running, walking, cycling or driving. In fact iPhone apps interrogate this user activity status from the CMMotionActivity class offered by the iOS operating system. To determine user activity, the motion co-processor will most likely take acceleration readings in three axes (x, y and z) from the accelerometer sensor. Repetitively calculating the max and min values will enable distinction between sitting and running/cycling and driving, regardless if the user is holding the phone, has it stashed in a trouser pocket or nestled in a car’s device cradle.

However it’s far more difficult to distinguish between running or cycling as the acceleration in all three axes is quite similar for both activities. This is where measuring yaw (rotation) using the gyroscope sensor comes into play. Cycling has a smoother repetitive motion than jogging. By calculating the spectrum of the yaw rate using an FFT (Fast Fourier Transform), cycling will show a single dominant frequency as determined by the cadence of the cyclist. Of course calculations can be further relaxed if likelihoods are taken into consideration. For example, a measurement showing an impromptu change of states from cycling to driving is somewhat implausible. Statistical models as offered by Bayesian probability inference or Markov chains come to the rescue here. And if all goes wrong and a confident activity guess is out of question, iOS luckily provides the state unknown.

The M8 LPC18B1UK chip is based on ARM’s Cortex M3 core. In contrast to the follow-up Cortex M4 core, the M3 does not include a DSP instruction set. So, it ’s plausible that activity tracking calculations are performed at a low frequency. In fact the chip is clocked at only 0.15 GHz. That in turn makes it battery efficient so it can run constantly without ever taking a break. Even in the iPhone’s standby mode it collects, calculates and caches sensor data. The iPhone stores results for a maximum period of seven days in the LPC18B1UK’s on-chip flash of 1 MByte. Sampling rates of 14-bit accelerometer data are thus probably in the 1 to 2 Hz range. In other words, really slow.

From lightweight processing to heavy lifting

As the iPhone 6 example above underlines, motion detection is all about lightweight processing. Similarly Atmel’s sensor hub solution as found in several smartphones from Samsung (see Wikipedia) use Atmel’s SAM D20 which features an ARM Cortex M0+ core on-chip and no DSP functionality. However Atmel’s sensor hub roadmap points to the follow-up SAM G51/53 which is based on the DSP-rich ARM Cortex 4 core. Sensor hubs are evidently transitioning from lightweight DSP processing to heavy DSP lifting. Recent smartphones from HTC, Nokia, Samsung and Sony confirm this trend: they use Qualcomm’s Snapdragon 800 SoC (system-on-chip) family with an on-chip sensor engine based on their powerful 32-bit Hexagon DSP core that also offers floating-point support. Next-generation smartphones, wearables and other mobile electronics will thus not only capture both user and environmental sensor data but also combine these streams in an eclectic signal processing mix to provide never-seen-before smarts for the user.

Sensors and the Rebirth of the DSP

Back in the nineties the digital signal processor (DSP) had its heyday. It was a completely different beast compared with CISCs (complex instruction set computers) like Intel’s Pentium or RISCs (reduced instruction set computer) like ARM’s IP cores. DSPs allowed complex mathematical algorithms to be processed in real-time based on the fundamental mutliplier-accumulator structure of their internal architecture. Mostly these signal processors had a 16-bit fixed-point word length with a very basic feature set. Their distinctive clout was speed combined with low power - a boon for all types of embedded applications requiring real-time response. Yet their restricted word length made programming them an art, something for maths whizz kids who could work with integers just as well as with floating point numbers. And then at some point, stand-alone DSPs simply disappeared off the processor map. What happened?

Layed off by semiconductor advances

With each reduction in size of semiconductor processing nodes, clock speeds moved into the gigahertz range whilst supply voltages and power consumption dropped dramatically. Lower power levels and longer word lengths enabled newer processor types and categories to extend their reach into the domain of rigorous real-time requirements, formerly a unique terrain for digital signal processors. Multiple cores on the same dice were suddenly feasible and the stand-alone DSP simply got gobbled up in the process. What was once a stand-alone math starlet enjoying the limelight became reduced to a common (and essential) block on a larger system-on-chip (SoC).

Smartphones and their sensors

Recently I’ve noticed the term “DSP” appearing more frequently again in the semiconductor world. The trail leads back to sensors; in particular, sensors as they are used in smartphones. First -generation devices featured three or four sensors to ensure fluid interaction with their touch screens: a proximity sensor to turn off the display during a call for saving power and preventing contact with the ear or face; an ambient light sensor for the best reading experience under all types of lighting conditions; an accelerometer to sense the orientation of the phone and switch between portrait and landscape modes accordingly. With each new device generation, further sensors joined the fold. Recent smartphones are often blessed with over ten such environment watchdogs.

Each new smartphone generation features more sensors

Sensors provide information on what the user is currently doing. Combined with location intelligence, smartphones can react intelligently to the user’s activity and current surroundings. Continuous sampling, storage and processing of sensor data is necessary in order to keep track of what is happening to the device, its user and whereabouts. Keeping the phone’s main processor- one of the battery hogs - on all the time, makes no sense. Enter the power efficient coprocessor, often termed sensor hub, or motion processor and sometimes even DSP.

Fusing sensor data to predict location

By adding an additional processor with scanty energy demands operating separately from the main, power-hungry applications processor, the continuous flow of sensor data can be analyzed all the time, even when the phone itself is asleep. Sharp readers will contest why use a processor geared towards blistering speed (read DSP) if sensor data, like readings of the temperature or magnetic field, arrive at a snail’s pace? Yet most calculations are all about sensor fusion, or using multiple sensors inputs to determine something really useful. Take indoor navigation as an example. The usual satellite GPS signal may not be available yet seamless navigation might still be required. Using combined data from the accelerometer, gyroscope, magnetometer and pressure sensor, the DSP implements a mathematically complex Kalman filter to accurately estimate the user’s position from previous bearings (dead reckoning algorithm) and simultaneously compensates for a number of tricky sensor anomalies such as offset, gain, non-linearity and noise. Such an intelligent sensor hub provides rich soil for further smartphone differentiation to take root as algorithms and smartphone apps combine motion, physiological (e.g. heart rate, voice analysis, …) and environmental data in completely new ways. This new trend brings the digital signal processor (DSP) back into the spotlight, reestablishing it prowess from its former glory days as a highly power-efficient mathematical engine.

Apple's Watch and Sensor Magic

Slated for release in early 2015, the Apple Watch will certainly raise eyebrows in many ways, possibly heralding in the consumerization of smartwatches. The company’s general mobile philosophy has been to cram more sensors into products than most of its competitors do. By integrating sensors, users not only carry a desktop computer in pocket format with them but a truly smart mobile device.
One of the more novel sensor solutions on-board the Watch can be seen on the back of its stainless steel and alumium casing: an optical system consisting of a combination of LEDs and photodiodes that gather data from both the visible and infrared spectrum.

Apple Watch’s LEDs and photodiodes for heart rate measurement

Photo courtesy of Apple, Inc.

With these sensors the Watch becomes a fitness device capable of measuring the user’s heart rate through optical sensing. In very basic terms this means shining a light through the skin of a user’s wrist and monitoring the change in blood flow in order to determine his or her pulse. It’s all about addressing today’s trend of the “Quantified Self” by continuously logging one’s daily activity as a means to improve basic fitness or even workout endurance over time. Electronic activity trackers are mostly sold as wearable bands today from the likes of Fitbit, Jawbone and Nike , but smartwatches and bands will converge at some future point. In 2014 around 10 million activity tracker bands will be sold worldwide and this figure is expected to triple in 2015. The market segment is set for exponential growth.

The smartwatch as a meaningful medical device

Lower-cost wrist bands are mostly attractive for those obsessed with tracking their activity and exercise. Yet will smartwatches such as Apple’s Watch also address real patients or the elderly who’s medication regime depends on regular and accurate monitoring of heart rate, blood pressure, blood oxygen levels (oximetry) or even blood sugar (glucose)? Two announcements provide some clues on this potential. Firstly, Apple has silently been hiring medical sensor experts over the past few years. The Apple Insider article identifies that a number of these hires were previously employed with serious medical companies. Secondly, Apple has filed a slew of patents related to medical monitoring over the past years. Patently Apple reports on several health and biometric-related patents in this area. Skeptics however purport that accurately measuring heart rate, as an example, is a big leap from the techniques that fitness trackers employ, maintaining that doctors rely on electrical not optical measurements for accuracy. Wearing a device on one’s wrist for correct pulse tracking requires it to be strapped very tightly so the sensor cannot move around during measurements. In addition, as users who measure blood pressure know, wrist measurements can be off from their true value by quite a bit because blood flows a lot slower by the time it reaches the body’s peripheral zones like the wrist. Many open questions and much room for speculation remain. Mike Nicholls of startup88 provides some “can’ts” why the Watch won’t cut it on the medical front. Are there any cans?

Many sensors make light work

Whilst accuracy imposes certain onerous requirements, it’s probably shortsighted to brush off the Watch’s possibly far-reaching medical potential. Point in case: today’s smartphones do not determine user location single-handedly by satellite (GPS) but in combination with input from other embedded sensors. Using complex calculations, data from their accelerometer, gyroscope, and magnetometer predict the user’s current position based on his previous one if a GPS signal is not available (dead reckoning). Multiple sensors termed as a sensor hub in combination with a power-friendly, always-on dedicated processor running signal processing algorithms can work sensor magic - possibly even medical magic. The Apple Watch is rumoured to have more than 10 sensors and will most likely also feature a motion coprocessor like the M7 (iPhone 5S) or M8 (iPhone 6). Combined with Apple’s talent pool of medical expertise it’s highly likely we can expect some surprises on the road ahead. If not in a 1st generation Watch, then for sure in its later product cycles.

Who Needs a Smartwatch?

Touted as the next big wave of consumer devices to impact our lifestyles, smartwatches will supposedly change many of our daily habits. Or will this be just one more device that finds it way into some drawer of collectibles after a few weeks of use? 

Examining my own behaviour, I have to admit that I always wore a wrist watch until the day I became the proud owner of my first cell phone. On that veritable day my cheap wrist watch became a forever companion to the special-occassion premium model in my bedside drawer. Yet recently I rediscovered the convenience of strapping a regular watch to my wrist to tell the time. A quick glance instead of fumbling for that phone slate somewhere deep in a pocket.

Hands freed up at long last

If you're on the go, there's a certain magic to having both hands free from carrying things like a luggage item or a phone. Smartwatches have the potential to deliver on that promise. In addition they they will be so much more than just an elegant timepiece.  Having a launchpad for calls, guidance, management and diversion strapped to one's wrist is a tempting thought. And a few clicks, swipes or voice commands could initiate all the fun. If regular wrist watches with a brand name cost 100 Euros at the lower end of the scale, then surely 300 Euros is a consumer-friendly price tag for a watch with smarts?

I’d be an easy mark for a smartwatch if

• it offers full functionality without my smartphone, unless I want to make/take calls or access the internet.
• the casing and industrial design in no way resemble an electronic device. As all smartphones resemble rectangular slates, it’s a fitting time to break out of the uniformity mould and revive lifestyle regalia.
• it powers itself through a kinetic mechanism or energy harvesting. As long as I never have to charge it.
• it unfolds a whole new world of “hands-free” functionality for
  • phone calls (missing parts: the Bluetooth earpiece that invisibly clips behind an ear or integrated in an eyeglasses’ arm, and some kind of highly directional microphone that won't require any strange gestures so that the person on the other end understands me)
  • effortless and efficient contactless payment
  • finding my way whilst walking or cycling
  • helping improve fitness or basic monitoring of health (heart rate, blood pressure, glucose level...)

My short list above might one day appear quite meek. Equipped with a multitude of sensors, smartwatches will see unthought of functionality unfold as creative developers build new applications using development kits that the manufacturers provide. The beauty of “smart” in smartwatch means different designs, different functions, different uses for different types of users.

Apple Watch	Samsung Gear S	Moto 360	Asus ZenWatch	LG G Watch R
Photo courtesy of Apple	Photo courtesy of Samsung	Photo courtesy of Motorola	Photo courtesy of Asus	Photo courtesy of LG

From Swatch to Rolex: will smartwatches ever span the breadth from the low-cost to the up-market?

Visitors to the Mobile World Congress earlier this year could witness smartwatches and fitness gadgets galore on display. Apart from the trendy fitness band, I have yet to see any of my personal acquaintances sporting a smartwatch. Will the announcement of Apple’s Watch for next year change consumer behaviour at the flick of a switch as they so often have proven before? 2015 might indeed provide the spark to catapult smartwatches and wearables into mass adoption.

Smartwatch and Wearables Shipments

Smartwatches fall into the so-called category of "Wearables" that many technology market research companies track. Forecasts vary and inflated expectations are sure to traverse a temporary trough before certain mass adoption occurs. Various sources suggest that around 10 million wearables were sold in 2013 with sports and activity trackers topping the list. Looking further into the future, opinions differ on the potential market size. Bold seers predict over USD 100 billion of wearable device revenue for 2018, whilst the more cautious cap their forecast at USD 30 billion. Compared to 2013 smartphone revenues of roughly USD 300 billion (around 1 billion smartphones were sold at an average price of USD 300) smartphones overshadow wearables considerably, but for how long?
On the smartwatch front Generator Research provide a short yet comprehensive history and outlook for smartwatches (free). The market research outfit’s recent study Smart Watches: 2014 forecasts sales of 5.8 million devices worldwide in 2014, rising to 313 million by 2020.

Sample Market Research on Wearables

Wearables Service	CCS Insights	http://bit.ly/1ou2S2b
Wearable Computing: Technologies, Applications and Global Markets	BCC Research	http://bit.ly/1qReqfq
Investing in Wearables for Financial Services	Javelin Research	http://bit.ly/ZnzAMS
Wearable Computing: Fitness and Health in Style	Parks Associates	http://bit.ly/1nEvg71

On wi360 you’ll find many more report summaries that focus on wearables and smartwatches providing insight and opinions into a consumer market segment that is set to boom.