Digitizing Experiences

Digitization has dramatically changed our world.  From Apple’s retina display which packs 326 pixels in a square inch to encoded keys that let us into offices, breaking the world into finer and finer granules of 1’s and 0’s has changed our lives.

But for these devices to work they need a power source, logic software and most importantly a sensor.  Sensors provide the signal that defines the current state or change from it.  As computers shrink in size, the leading edge of innovation is shifting from the sensor itself to a rapidly expanding partnership between sensors and the software that figures out how to share and makes sense of our experiences.  This piece wades into this surging tide of digitizing experiences exploring the opportunities unleashed and where threats may linger.

This first piece is a toe dip with more to follow.  I invite you to contribute your thoughts as I begin with more questions than answers.

Visibility 2.0

One could think of the growing plethora of cameras in every city as an invasion of privacy.  They also provide a dramatic increase in visibility, at low cost and with a memory that aids in catching crooks ripping off folks at ATM’s, diagnosing traffic accident causes, etc.  Love ‘em or hate ‘em, the fact is we’re only adding more.

Cameras are fixed in place sensors.  There are also sensors in the streets and traffic signals triggering light changes.  Utilities and transportation systems rely on an array of sensors to signal location, alert security to the presence of toxins as well as the TSA flotilla of metal detectors, x-rays and yes, facial recognition.

Now add in the new wave: mobile sensors.  Every cell phone now has GPS so even if you’re not in a camera’s range, the likelihood someone can find you is high.  Do something worthy of attracting attention and don’t be surprised if someone hasn’t used their camera phone to record it.

Last week a JetBlue pilot became disruptive on a flight from New York to Las Vegas.  Passengers took him down and within a couple of hours after landing, the video was all over the Web.

Privacy fears compete with sensor technology advances.  Persistent visibility enables researchers to identify behavior patterns (and yes, profiles) that precede trouble to forewarn us.  Without sensors, we wouldn’t have advances such as non-invasive medicine.  Would you rather suffer the incision of exploratory surgery or have a fast and painless MRI or CT Scan?

Sensors are continually shrinking in size and power requirements.  The latest now sense from a greater distance.  Microsoft’s Kinect console detects body motion that enables game play without a keypad or joystick controller.   Build similar technology into an LCD projector and you’ll be changing slides with a sweep of your hand from the front of the room.

When sensors combine networking with embedded intelligence they change some of our most familiar sensors such as thermostats.  The new NEST thermostat illustrates not only where sensors are heading but the challenges facing them.

A Thermostat that Thinks It’s More than That

One look at the NEST thermostat signals its Apple design roots (see photo at top of page).  The device itself is drop dead gorgeous compared to the plastic beige plastic circles and squares with which we’re all familiar.  Founded by the  Apple executive who used to be in charge of Apple’s iPod and iPhone teams, the NEST thermostat incorporates sensors that detect your presence.  When no one is home, it automatically turns down the temperature.

The NEST thermostat is also networked.  If you have more than one thermostat in your home, they talk to each other over WIFI.  Should you go on holiday and forget to adjust for that, you can do so through a mobile app or web-based application.  This is ideal for a frequent traveler or if you own a second home.

With embedded artificial intelligence software, NEST learns your living patterns and if enabled, adjusts heating and cooling based on your experience.  And that’s my point, the leading edge of sensors is moving from simple motion detectors that open super market doors to sophisticated systems that track, network and analyze our life experience over limitless time.

Smart But Smart Enough?

My household lifestyle has nearly melted the artificial intelligence inside our NEST thermostat since our patterns are more like fractals with seemingly random changes.  Just when NEST has us figured out, we break the pattern.

Part of the problem is that while our thermostat is centrally located, it is not in an area that we pass by frequently so it thinks we’re not here when we are.  Another issue is that despite having a wide viewing range, different rooms have different sun and wind exposure such that any single sensor, no matter how smart, is handicapped to heat or cool at an average.  An alternative would be innocuously shaped “sensor pebbles” that we could distribute throughout the house.  This would increase the sensing range and increase the data input.

Undoubtedly, the folks at NEST are way ahead of me as they comb through the data they have gathered from each thermostat sold.

Think of this for a moment.  Most of us grew up and have used thermostats that never reached beyond the home or office.  NEST has information on our settings, usage, when we change temperature (or don’t), etc.  Now correlate that with weather, public utility usage patterns, etc. and who knows what surprises are in store?  A big gating issue is making sense of the experience data captured.

Big Data Analytics

Software analytics will leverage networking to achieve the next level.  That’s what all the fuss about “Big Data” analytics is about.  When we begin to combine data from multiple sources, patterns emerge that are not visible from single sources.

Imagine combining point of sale data with facial recognition, affinity networks such as airline mileage or supermarket clubs, weather, demographic and GPS.  Retailers such as Target have already figured out which of their customers are in the last trimester of their pregnancy by analyzing their purchasing behavior and shortly will be able to tell (let’s assume with your permission) where you are in their stores.

That’s what makes the work of MIT’s Sandy Pentland so intriguing.

A Badge that Blabs About You

In The New Science of Building Great Teams (Harvard Business Review, April 2012), Pentland describes a study using electronic badges that collected behavior on individuals’ voice tone, body language, whom they talked to, how frequently and more.  In their seventh version, the badges capture 100 data points a minute yet is no more intrusive than most corporate IDs.  In sum, the badges quickly become invisible and don’t appear to alter people’s behavior.

Pentland’s study weaves technology and sociology into fascinating insights.  His data demonstrates that patterns of communication are not only critical to success but as significant as factors such as individual intelligence, personality, skill and the substance of discussions combined. He’s identified productive team data signatures that are so consistent he can predict success without ever meeting the team.  It’s a worthy read that further illustrates the potential that digitizing experience offers.

Sensoring or Censoring?

Every new technology brings with it opportunities and concerns.  Let’s assume that electronic badges or NEST technologies shrink in size and power requirements, while increasing in computational strength and networking.  Let’s further assume that sensing from a distance and connecting wirelessly continues to improve as demonstrated by Microsoft’s Kinect gaming interface and Nike’s FIT athletic sensor, respectively.   And of course, Big Data analytics continue to drive forward.

What emerges is a digitization of experience that goes far beyond what IBM refers to as the Internet of Things.  IBM’s terminology suggests a networked world of fixed sensors such as traffic signals that adapt for weather or water main breaks based on networked sensors.  No doubt there are benefits to found in operating cost savings, preventive maintenance and emergency response.

But the implications of digitized experience leaves the Internet of Things in the dust.  Just as electronic digitization enabled us to control and manipulate visual elements to “Photoshop” pictures and create dazzling special effects in the movies, digitizing experience opens the door to better understanding and improving human experience.

For example, what might happen if Pentland’s  post experiment analysis was transformed into a real time display of communication efficiency in meetings or web conferences?  Think of it as bio-feedback for group discussions and team leaders.  It offers the potential to reduce the time-to-detect and time-to-correct a lousy meeting.

Now record this data over the life of the team or duration of the project/problem.  Then add big data analytics and consider the potential for leaders to track and analyze communication effectiveness over time.  Since tough problems and good teams take time to solve and groom respectively, we could identify who are the best team leaders, players and settings relative to specific situations.

Recognize that all of what’s described above we do today through observation and what designers refer to as ethnography.  The difference is these are usually one-off studies and the data analysis depends on the researcher.  With digitized experience, the same intelligence that has made Google search so valuable can be applied to knowledge work.

If we take to heart that our global economy is based on knowledge, then how effectively we process knowledge in teams, across cultural and technical boundaries unleashes dramatic potential.

Balancing Opportunity with Caution

Without question, the technology raises some fears as well as opportunities.  Without being dismissive, let’s stipulate that fear of misuse is part of every technological shift.  Some concerns deserve our full attention while others just reflect the anxiety of change.

Building on my above speculation, it’s not crazy to suggest that combing NEST with Pentland’s badge technology to build a real-time bullshit detector.  That sure sounds like big brother if your employer does it to you but how would you feel about using the same tool in a critical acquisition discussion?  Imagine what knowing when the people across are bluffing could do?

If only one party had access to the information it could be wielded for serious advantage.  Alternatively if both parties had the technology, it could take bullshit off the table.   Now apply it to the upcoming electoral season – just use your own imagination of what doors that might open!

Of course the above is speculative.  What is not is how fast the new world of digitized experience is approaching.