Not only is it about personalized search, but also the authors include Susan Dumais and Eric Horvitz from Microsoft Research.
The paper describes a user study that showed wide variation for the same query in user intent and in user perception of the most relevant search result. That data is used to motivate personalized search as a way to better capture searcher intent.
Our analysis shows that rank and rating were not perfectly correlated.The paper then discusses providing users with explicit customization or more powerful search tools to explicitly specify their intent, but dismisses that approach as too much work, too difficult for searchers to do accurately, and too unlikely to be helpful:
While Web search engines do a good job of ranking results to maximize their users' global happiness, they do not do a very good job for specific individuals.
If everyone rated the same currently low-ranked documents as highly relevant, effort should be invested in improving the search engine’s algorithm to rank those results more highly, thus making everyone happier. However ... our study demonstrated a great deal of variation in their rating of results.
We found that people rated the same results differently because they had different information goals or intentions associated with the same queries.
Rather than improving the results to a particular query, we can obtain significant boosts by working to improve results to match the intentions behind it.
One solution to ambiguity is to aid users in better specifying their interests and intents ... [Ask] users to build a profile of themselves ... [or] help users better express their informational goals through ... relevance feedback or query expansion.The paper goes back at that point to looking toward implicit personalization to improve search, saying that inferring "users' information goals automatically" is the most promising way to disambiguate the "range of intentions that people associate with queries."
While it appears people can learn to use these techniques ... they do not appear to improve overall success ... We agree with [Jakob] Nielsen, who cites the importance of not putting extra work on the users for personalization.
Even with additional work, it is not clear that users can be sufficiently expressive. Participants in our study had trouble fully expressing their intent even when asked ... In related work, people were found to prefer long search paths to expending the effort to fully specify their query.
Great stuff. I am curious how much of this work at MSR is making it over to MSN.
See also my thoughts on the Teevan et al. SIGIR 2005 paper where the same authors describe a prototype of keyword-based approach to personalizing web search.
See also my previous post, "Attention and life hacking", discussing a NYT article on some of Eric Horvitz's work on attention and linking to some more of Eric's papers.
See also my earlier posts ( ) about Google's Personalized Search.
See also the announcement of Findory's alpha test of personalized web search.
See also my May 2004 post, "Why do personalized search?"