In case you missed it over the past few weeks a new meme is picking up some stream on whether software-based filtering is good or bad and how it should evolve.
One thing is certain for all of us: There is no question information is coming at us or flowing past us faster than we can make sense of it—much less actually use it. Digital content syndication, social media and collaborative business applications have permanently changed the information supply chain. Different coping mechanisms are emerging, with social curation and algorithmic filtering both holding great potential, but also more than a few negative consequences.
Not that long ago, Clay Shirky characterized the problem as one of filter failure, citing how our traditional media filters no longer protect us from volumes of irrelevant information. Earlier this spring, Eli Pariser offered up the concept of Filter Bubbles during a TED talk, describing a personal account of how filter algorithms designed to help consume relevant information actually created web-based biases that negatively skewed his and his friends' perspectives on recent events in the Middle East. This sparked a debate summarized last week in Filtering in Social Software: Protective Bubble v. Serendipitous Awareness, by Larry Hawes. The tug of war between filter failure, filter bubbles and information good or bad is evidence of one aspect of information overload that I rarely hear discussed—the changing landscape of how we consume digital information.
Algorithmic attention processing is bound to, well, attract a lot of attention. In a tweet last week, John Battelle observed “When I open Twitter, the top five or six stream “results” should be the most relevant to me, in context of time and past habit. Not so now.”
Echoing my comment on Larry's post, regarding the filter bubble debate, software should able to present the user with filtered and unfiltered views of information to guard against inherent biases by empowering the consumer of information with the choice to control as much or as little of their own knowledge flow. It should also enable people to create adhoc filters—mostly due to the fact that context or topical interests can change quicker than most algorithms can adapt.
In order to provide the most effective filter system, the software tools built to address this must also overcome the bias created by their own content focus and the limited scope of human interactions with any individual application. The attention information needed for truly effective algorithmic filtering requires solutions that understand the broad nature of an individual's information consumption behavior, the consumption context of the individual paying attention to the information and the location where the information is being filtered and delivered.
This debate will continue and it will accelerate the innovation needed to make truly intelligent information consumption possible.