Tag Archives: Big data

Fred McClimans #140MTL State of Now Montreal Quebec

Pervasive Communications & Biological Big Data

I recently had the pleasure of speaking at the #140MTL State of Now conference, on May 15th, 2012, in Montreal, Quebec. It was a fantastic event, with some great speakers and wonderful attendees.

As part of the event, I had the opportunity to discuss a few concepts that are helping to shape global events and trends, and how we interpret them, including Pervasive Communications (see a great write up by Alan Berkson), Biological Big Data, and the value of ad hoc social interactions and the information they can reveal about each person’s unique, and contextual, perspective.

I hope some of these points resonate with you, and welcome your feedback and comments. This discussion is far from over.

Werner Heisenberg

Mr. Heisenberg meets #BigData?

1927 was a very good year for Werner Heisenberg, and, in an odd twist, those wrestling with Big Data and the identification of global events and trends that are shaping our future, a mere 85 years later.

Heisenberg was a brilliant physicist, yet his work on Quantum Theory and the Uncertainty Principle may help us shape how we look at many of the issues that we face today in the non-Brilliant-Scientist realm.

“One can never know with perfect accuracy both of those two important factors which determine the movement of one of the smallest particles—its position and its velocity. It is impossible to determine accurately both the position and the direction and speed of a particle at the same instant.” ~ Werner Heisenberg

Heisenberg’s statement has been quoted, mis-quoted, adapted and modified to suit any number of ideas over the decades, so excuse me if I twist it myself to make a point.

In 1926 and 1927, when Heisenberg was laying the foundation for, and publishing, the Uncertainty Principle, we were in a world where Big Data didn’t exist as we know it today. We were also far from being globally hyper-connected, and the idea of Pervasive Communications was a dream of the future.

TAKE A QUANTUM LEAP

I was recently having an interesting, and ongoing, Twitter discussion about Big Data and the value of Curation with some friends (Alan Berkson, Colin Hope-Murray, Peter Bordes and Robert Moore). In response to a question about the value of too much data, or data that was too old, I tweeted “old info doesn’t die, it reveals long-term trends”.

As I looked at what I had written, Heisenberg (oddly, also part of the ongoing discussion) kept coming to mind, ultimately prompting the question “How do we determine the long-term value of an event or data point, and ultimately the value of a trend if it lacks the right context?” This question became all the more important as the different perspectives that frame “context” began to come to light. No two people see the same particle or event from exactly the same personal perspective.

THE RIGHT STUFF

It became increasingly apparent that our discussion of “too much” Big Data was really about having the “right data”. But how do you determine the right data? In many cases, you can’t. We’ve plugged ourselves into this giant fire-hose of Social Media and can’t digest it all.

In the end, most of us can only “sample” off the feed. But in sampling, we get a very accurate description of what is happening at that particular moment, but we can’t tell where what we are sampling fits into the bigger picture. Is this data “byte” the beginning of a trend? Is it supporting a trend that already exists? Or is it perhaps signaling the evolution, or end, of a trend? Is it possible that we can’t answer these questions unless we are continuously sampling from the buffet that is available courtesy of Pervasive Communications and our always-on data feed?

THE MEANING OF LIFE

As we talked a bit about this issue offline (if you consider a couple of hours on a Skype video call “offline”), I came back around to the tweet about the value of old data revealing trends. Perhaps we’re looking at Big Data and the online fire-hose in the wrong way. Too often we think we already know the questions, or we already know the trends, and we look at data points as a way to support our pre-existing notions (numerologists often have a particular knack for this – you can find anything if you look hard enough in the wrong direction).

So rather than always trying to consume information to answer questions, what if we just taste the data, and let it help us form the right questions, regardless of the sector, the market or even what the data was originally supposed to represent? Why not let information from the Transportation sector be co-mingled with information from Politics, or Economics, or Energy. By doing so, we’re helping to erase preconceived notions about the value of the data, and the answers we expect to get.

Ultimately, that’s what it’s all about, isn’t it? As Alan pointed out in his recent post Big Data: Is The Answer 42?, answers are meaningless if you don’t understand the question, and with today’s glut of data, events and trends, figuring out the right question, and understanding why it’s the right question, is more difficult than ever.

FINDING THE INFLUENCE OF UNCERTAINTY

Heisenberg talked about particles, their position and their velocity. I’m talking about events, their impact and their influence (their ability to form trends). In either case, the more certain we are of something, the less certain we are of something else. To me, that raises the question of value in being uncertain, to an extent.

Knowing the present state of an event or data-point is extremely valuable, as is knowing the direction it is heading. But equally important is the value of knowing why it is where it is at a particular moment and why it is heading in a particular direction (what influenced it, what shaped it). Following that lead, it’s also important to know where it is heading and what it is going to hit (how will it influence something else).

“Why a trend exists is just as important a question as asking what impact will result from the trend. It’s all about context.”

As the data reveals more potential trends, so too does it raise more interesting questions:

  • What value do individual events have, either as singular events or as part of a larger data set?
  • How important are multiple layers of context and different perspectives?
  • How do you anticipate when or how trends may collide or intersect?

The next time you sift through the data, as you swim through the stream, try squinting your eyes a bit. Don’t focus so much on what you see, but rather let some uncertainty creep in, and see what patterns emerge when you see things just a bit “fuzzy”.

In the end, you might be surprised at what you do see, and the questions you start to ask.

ZeroKlout2

Klout, Big Data and the Meaning of “Opt Out”

Is it possible to have a Klout Score of Zero (K = 0)?

Why, you might ask, would anybody want to have such a score in the gamified realm of influence measurement, where higher scores indicate a higher level of perceived online influence?

The answer may lie in the way that Klout profiles you, branding you a Specialist, an Observer, or a Broadcaster. The answer may also lie in how people relate to Big Data, vaguely defined ranking algorithms, and the increased tendency of offline organizations to make some big, and potentially misleading, assumptions about the role of online influence in an offline world.

“Klout calculates billions of data points across over 100 million influencers every day.” ~ Klout.com

Whatever the reason, there are people who simply want out.  But opting out, and driving your score to a meaningless Zero, is apparently a bit more difficult in the Klout dimension than one might imagine.

I PRESENT TO YOU MR. SAM FIORELLA

Mr. Fiorella was recently referenced in a Wired.com article (What your Klout Score really means) that delved into an experience he had a while back with a potential employer, who eliminated Sam (and possibly others) from the list of candidates based on his perceived “sub-par” Klout Score. As listed on the Klout.com website…

It’s not the first time something in the online world has impacted a decision in the offline world, and it definitely won’t be the last (see Jeremiah Owyang’s post “How ‘Social Profiling’ Will Work In The Real World“).

Sam ultimately did improve his Klout Score (into the 70’s) but was never happy with the idea of being ranked (or branded) by an algorithm for online OR offline purposes. So when Klout offered an “opt out” option at the beginning of November, 2011, he promptly did just that. He opted out and initiated the deletion of his Klout profile, per the language on the Klout site:

Klout Opt Out

As far as Sam was concerned, he was satisfied that after opting out nobody would be able to view his Klout Score moving forward and that only trace data would remain in the system (for 180 days, after which it would be removed).

He also understood that Klout would continue track his activities on the public broadcast social site Twitter. 

Note: I wouldn’t be surprised if Klout NEEDS to track Sam privately in order to accurately determine the Klout Scores of others within his Twitter social graph. In essence, influencers who are not tracked become dark matter, or invisible thought leaders. They mess with what we perceive by influencing behavior in unseen ways. 

But there was also a level of expectation that the information gathered on Twitter (and his resulting private Klout Score) would to be kept private and OFF the Klout.com site.

Unfortunately, it didn’t work out that way.

I PRESENT TO YOU MR. SAM FIORELLA’S GHOST

In the name of full disclosure, I know Sam personally and I am a registered Klout user. I was also aware when he, and others, opted out of Klout last year. So when I read the Wired article, and the various other articles and posts that it spawned, the analyst in me was just a bit curious to see if Sam had in fact been removed from the site. So I search Klout.com and found no public profile or information on him.

But I did come across the profile of a friend of mine, and attached to that profile, in their Influencers list, was the smiling face of Sam Fiorella. On the site, exactly where it should not have been.

Apparently, the phrase “you will be removed from Klout.com within 24-48 hours” – as mentioned in the Klout opt out statement – may not mean what you think it means.

Sam opted out from Klout almost 6 months ago. Could this possibly be the “trace data” mentioned in the Klout “opt out” statement?  I don’t believe so, as his current Twitter avatar is on display along with an assumingly current Klout Score of 52 (which sounds plausible since Klout appears to be pulling his data only from Twitter, not the complete list of social sites that Sam previously had linked to his Klout account, and it increased to 53 last night).

But wait, there’s more (thank you Ron Popeil). While I did pass on the option to invite Sam back to Klout (he wouldn’t have accepted anyway), I couldn’t resist the chance to test the software and see if it would allow me to give him a +K in Blogging. It did:

I’m not sure the +K stuck (even though it does now show me a greyed out +K button for Sam and Blogging, it apparently didn’t decrement my +K counter).

But the mere fact that it allowed me to go through the action, give me a success notification and offer the option to Tweet the +K out, was more than just a bit interesting – it was a challenge to figure out what had gone wrong, how it might be corrected and to think strategically a bit about some of the larger (beyond Klout) implications it might have.

FOR YOUR CONSIDERATION

From a social perspective, you cannot deny that influence exists – marketing, advertising and sales people have been trying to identify and target influential consumers for years. Nor can we deny that our online and offline lives are colliding extremely fast, and influence in one medium can, and will, transcend to another.

From an online influence measurement perspective, there is a defined need to look for insights in online behavior (served by Klout and other firms such as PeerIndex, Twitalyzer, TweetLevel, etc.), and the people at Klout have been very honest and open with me, and others, about how and why they are undertaking this task. 

But there is a disconnect when a phrase like “removed” appears to mean “erased a bit” – not quite how I would interpret it.

CAN WE ACHIEVE ZERO?

When Sam opted out of Klout, he assumed that he would still have a Klout Score, but that his information would no longer be shared or visible to others – in essence giving him a public null Klout Score (K = 0) that he sought. While the data would still exist, and be interpreted by Klout, they would not share their interpretations with others.

So why is Sam Fiorella still appearing on Klout? Perhaps there is an issue that weaves around Klout’s interpretation of words, and the managing of expectations from a contractual Terms of Service (TOS) perspective. Or perhaps it has to do with the massive amounts of Big Data that we are crunching on an ongoing basis, with technology evolving at such a rapid pace that glitches and ghosts, while unacceptable, are going to occur. Either way, there is a flaw somewhere in the system, and Mr. Fiorella has become its poster child.

PRIVACY AND PERVASIVE COMMUNICATIONS

Sam’s issue with Klout is bigger than either Sam or Klout. Not to diminish what Sam is going through, neither Sam nor Klout are alone in facing issues regarding personal data, big data, privacy or changing technology. If anything, his dilemma is indicative of a much larger series of questions and issues that we face.

We live in an age of technology-enabled Pervasive Communications. Our ability to communicate with almost anyone, anywhere at any time, over a multitude of communications channels, is allowing us to unleash our DNA-driven need to create, share and consume content and information with others.

As we do this, our public actions are increasingly tracked, tagged, shared and mined by people and companies that we’ve never met. They’re sifting through piles of Big Data looking for patterns, for trends, for clues regarding what influences our decisions, and how our decisions influence – if at all – the decisions of others. This isn’t necessarily a bad thing, but when this activity lacks true transparency of both intent and use, the user is increasingly, and unknowingly, giving away far more than they are receiving in return.

“There is nothing in the dark that isn’t there when the lights are on.”

~ Rod Serling

The data is out there and it’s not going away. It may lose some of its relevance, but it will still be out there and is increasingly being linked with other data to create “new” data. The questions of who really owns our data (both pre and post-processing), how and when it can be shared and reused, and how much light (transparency) should be shined upon it, will likely be argued (and should be) for many years to come.

While many individuals may argue that they want their data out there (in an effort to achieve a richer, more engaging online experience), I do believe that there are different times and places for private and public, and, as individuals, businesses and governments, we need to continually ask ourselves:

  • What should the ground-rules be for how Terms of Service and ownership of data are defined?
  • How will we let these definitions and rules evolve and adapt to technology and human behavior patterns that don’t yet exist or have yet to be defined?
  • How can we provide true transparency (in simple terms) to online users regarding their data and its linkages with other data (there’s a business out there if you can create that infograph, BTW)? And,
  • How we are going play together in an ever shrinking sandbox where transparency has become a buzz-word and personal privacy continues to become increasingly elusive?

I also believe that when an “opt out” option is offered, as it was with Klout, it should be just that – a way for you to take yourself, and your data, OUT of the system. If not for your actions, the data wouldn’t exist in the first place.

 Note: Images adapted from Klout.com