Today’s search engines deliver results based on meta tags and “hits” which is largely an influence of advertising or popularity. Advertising and popularity does not necessarily equate to relevance.Relevance is a term used to describe how pertinent, connected, or applicable some information is to a given matter. It has unique significance in a variety of fields.
The idea of using computers to search for relevant pieces of information was popularized in an article As We May Think by Vannevar Bush in 1945. First implementations of information retrieval systems were introduced in the 1950s and 1960s. By 1990 several different techniques had been shown to perform well on small text corpora (several thousand documents).
Information retrieval (IR) is the science of searching for information in documents, searching for documents themselves, searching for metadata which describe documents, or searching within databases, whether relational stand-alone databases or hypertextually-networked databases such as the World Wide Web.
Automated IR systems are used to reduce information overload. Many universities and public libraries use IR systems to provide access to books, journals, and other documents. Web search engines such as Google, Yahoo search and Live Search (formerly MSN Search) are the most visible IR applications. However, the opposite of the intent is happening, information overload with much of the information not relevant.
Today’s search engines, although they are continually getting better, tend to be rather low tech. Some read meta tags, some read only the first few hundred words of text. Some read both and link them together, so that if you have meta tags which don’t relate to the words on the page, it won’t rank your page very highly.
What Are We Trying to Find?
When you boil down everything we search for it equates to two attributes, information and people. People generate information derived from their experience, education and relationships with others. Our information is contained in numerous forms of media much of it published on the internet. Yet finding the relevant information and people can be a big time suck using today’s search methodologies.
Kevin Kelly writes “When there are millions of books, millions of songs, millions of films, millions of applications, millions of everything requesting our attention — and most of it free — being found is valuable.”
“The giant aggregators such as Amazon and Netflix make their living in part by helping the audience find works they love. They bring out the good news of the “long tail” phenomenon, which we all know, connects niche audiences with niche productions. But sadly, the long tail is only good news for the giant aggregators, and larger mid-level aggregators such as publishers, studios, and labels.”
“The “long tail” is only lukewarm news to creators themselves. But since findability can really only happen at the systems level, creators need aggregators. This is why publishers, studios, and labels (PSL)will never disappear. They are not needed for distribution of the copies (the internet machine does that). Rather the PSL are needed for the distribution of the users’ attention back to the works. From an ocean of possibilities the PSL find, nurture and refine the work of creators that they believe fans will connect with. Other intermediates such as critics and reviewers also channel attention. Fans rely on this multi-level apparatus of findability to discover the works of worth out of the zillions produced. There is money to be made (indirectly for the creatives) by finding talent.”
Today’s social web activity is centric to people and information connecting. Recruiters use the social web to find people to fill job openings searching endlessly through profiles matching information with requirements. People search blogs, books and news releases looking for answers, experts and market intelligence. Conversational rivers abound but with limited search structure to match relevant people and information collectively.
Imagine being able to manage the structure of the conversation and capturing interactions within a consistent framework. Imagine search criteria enabled with ratings of expertise, relevant experiences, education and history of participation in matters of relevance.
It’s time to replace threaded discussions with a tapestry of human capacity and information. These solutions are being built and soon our search capabilities will mature. This increase in value and capacity will be like the rising tide lifting all ships.
What say you?