Tuesday, March 24, 2009

What is a good recommendation algorithm?

I have an article on the new Communications of the ACM blog, blog@CACM, titled "What is a good recommendation algorithm?"

The post picks on the root mean squared error (RMSE) measure used for evaluating recommender systems in the Netflix Prize, talks about precision and why making recommendations is like search, and discusses some factors that impact people's perception of the usefulness and quality of the recommendations that are not captured by RMSE.

If you have thoughts on evaluating recommender systems, please go to the article and comment there. I left many questions unanswered in that post and was hoping to get a bit of a discussion going over on that new CACM blog.

The blog@CACM is just getting started, but the list of contributors is quite impressive and includes Peter Norvig, Daniel Reed, Michael Stonebraker, and many others. If you like, you can get the feed here.

By the way, if you are an ACM member or just remember it fondly from school, you might also go check out the new CACM website. It's been recently redesigned to emphasize news articles on the front page and in its many new feeds.

[Full disclosure: I'm on the CACM web board.]

3 comments:

Daniel Tunkelang said...

I'm psyched to see that CACM is embracing blogs. I subscribed as soon as I read this post. And then unsubscribed as soon as I saw that the feed only contains tiny excerpts.

A great thing about RSS is that it lets you aggregate feeds in one place, rapidly getting a sense of their content. The short excerpts aren't enough, and the cost of opening up another tab to find out if you want to read an article is simply too high.

I thought there was a consensus by now that, unless you're concerned with ad revenue, you should export full feeds. I realize the CACM site has features that won't be exposed in an RSS reader. But, at least in my view, users should be able to make that trade-off themselves.

Any other feedback from ACM members and readers?

Dave said...

Having to create an account to post on a blog is a hassle most people probably won't bother with :P Interesting article though. I find online record shops I use to be generally better at recommending than books or movie sites. Perhaps the media is important too.

Scott Wheeler said...

Greg thanks for writing this; I'm almost giddy reading it because it really touches at the core of what I've been preaching for a while now. Recommendations are about people. How they use and interact with those recommendations is how you know they're good; not how well you can predict their ratings on stuff they don't care or already know about.