In this edition of Outside the DXperience we’re going to take a look at the Realtime web; find out what it is and how search differs between the Traditional Web and the Realtime Web; we’ll look at who the players are in realtime search and who amongst them, if any, has the upper hand; before looking at the question on everyone’s lips… has Google been out Googled?
What is the Realtime Web?
Well put simply, the Realtime Web is the here and now. Where the traditional web consists of pages from commercial sites, blog posts – both amateur and professional – news sites, weather sites and pages of information of every conceivable topic; they all have one thing in common: they are in the past tense. Someone has conceived them, written them, polished them and finally published them. By the time you read it the information contained in the pages of the Traditional Web is at least hours old.
The Realtime Web, on the other hand, is the web of the here and now. Fuelled by social media tools like Facebook and Twitter that allow users to post their thoughts at any given time, on any given topic, the Realtime Web is a window on the life of the global citizenry. Posts on the Realtime Web are never really conceived but are more off the cuff remarks; the thinking out loud of any given, ordinary, person. These posts are certainly never polished but, instead, are a measure of the writer’s emotion at that time. Posts on the Realtime Web are usually what we call “soft” posts, containing opinion rather than fact; emotion rather than measured response. They are useful because they are what people think without the “white lie” that most of us use to survive in our day to day social interactions.
Of course, saying posts on the Realtime web are “soft” is a generalisation, one that holds true in most circumstances certainly, but a generalisation nevertheless. One area where this is not true of course is in the area of citizen journalism. When some “Johnny on the spot” can post words and sometimes pictures too of events as they happen, in a way that no news corporation could do, unless they were very, very fortunate. One example of this has been the coverage of the Iranian election, where most, if not all, of the most interesting coverage has come from people inside the country getting the word out via Realtime Web tools like Twitter and YouTube. Another place where you see the Realtime Web excel compared to the Traditional Web is during a natural disaster, where news of earthquakes are commonly first heard on Twitter or FriendFeed before you hear about them on the mainstream news channels.
What is the difference Between Traditional Web Search and Realtime Web Search?
Traditional Web search works by having a “crawler” (a software agent) examine as many web pages as it can and index those pages in large server farms. Then, when someone makes a query, the index is examined and matching results are returned, based on an algorithm that each search engine company hopes will out perform the other. At the moment, the undisputed king of Traditional Web search is Google.
Searching the Realtime web works differently. You don’t have to be a genius to realise that it would not be possible to store and index every piece of Realtime Web information as it is generated. Instead Realtime Web search relies on making API calls to social media service providers and have them provide a list of “what just happened now and in the recent past”. Twitter, for example, limit the results of any search to the previous seven days.
Who are the Players in RealTime Web Search?
Until recently, the three major players in Realtime Web Search were Twitter, Facebook and Friendfeed. This, of course, has been whittled down to just two now that Friendfeed have been bought by Facebook. There are, of course, a number of smaller players, here are a few in their own words:
Scoopler is a real-time search engine. We aggregate and organize content being shared on the internet as it happens, like eye-witness reports of breaking news, photos and videos from big events, and links to the hottest memes of the day. We do this by constantly indexing live updates from services including Twitter, Flickr, Digg, Delicious and more. When you search for a topic on Scoopler, we give you the most relevant results, updated in real-time.
OneRiot crawls the links people share on Twitter, Digg and other social sharing services, then indexes the content on those pages in seconds. The end result is a search experience that allows users to find the freshest, most socially-relevant content from across the realtime web.
TweetMeme is a service which aggregates all the popular links on twitter to determine which links are popular. TweetMeme is able to categorize these links into categories and subcategories, making it easy to filter out the noise to find what your interested in.
Who has the Upper Hand?
Of the two big players, Twitter and Facebook, it is hard to say who has the upper hand at the moment as both have their advantages, which I’ll summarise here for you now:
Twitter
It was Twitter who was first to market and that gives them the all important “first mover advantage”. There are millions of people who know Twitter search and reach for it as those in the realm of the Traditional Web reach for Google. Add to that the fact that Twitter has more experience, they’ve been in the game for a year now, learning what works and – more importantly perhaps – what doesn’t. This means that they have the knowledge to respond quickly to whatever Facebook can bring to the market place. But, to my mind, Twitter’s main advantage is that it is an open platform.
Facebook
Despite what you might have heard, size does matter and, in those terms, Facebook is way ahead. Facebook has over 250 million users. With those people to draw on, Facebook can provide a more accurate picture of what people are talking about at any given moment. You are also, statistically, more likely to have a friend on Facebook than on Twitter. Facebook’s search covers a wider range of media; as well as statuses Facebook search can deliver video and pictures as well which, by the way, can be filtered.
Has Google Been Out Googled?
Let’s be honest, Google was a fluke. Don’t look at me like that, it was! They started out with the idea of indexing the web and somewhere along the road they found that they could make money by placing adverts on the result pages of people’s searches. Fair play to the lads at Google, they exploited that discovery to the max and made a fortune doing it. What Google did was to shift the “value base” of content. Before Google came along (and to a certain extend before the web) the “value base” was with the content providers. “He who writes the words earns the money” as it were. However, Google shifted that “value base” to be “he who finds the content earns the money”.
With the advent of the Realtime Web, that “value base” has shifted again to be “he who provides the tools earns the money”. We established above that no company – not even the mighty Google – can index all the Realtime Web content that is produced every second of every day, and we are going to become heavily dependant on the tool vendors to provide an API which we, or search engine companies, can access to gain results for a search query. Successful social media tool vendors can charge what they like to search engine companies like Google, Yahoo and Microsoft to have access to that API. This, I feel, is the most likely revenue stream for providers such as Twitter and if Google are not very careful here, they could very well find themselves out Googled.