Sunday, October 29, 2006

Netflix Prize Contest

I was very excited when the Netflix contest was first announced. I have been toying with that problem for a while, but I never had enough data to do anything really with so it was always a minor project of mine. As I watched people get interested in the contest many of them were spending the first week figuring out how to manage all the data that was given to them. The worst part was that after a week of doing this it wasn't very fun and they would stop, but yet they hadn't even worked on the real problem yet! I have been staying up late this week, but having a blast and created a little present for you all, the NetflixRecommenderFramework. It lets you very quickly start hacking on your algorithm ideas without having to bootstrap everything else first. It even includes autotests. Maybe you can win the million dollars.

On the Qt side of things one of the included tools is a small GUI application that can be used to explore the database. At over 100 million rows I am happy to say that Qt QTableView can handle it just fine and there is no startup delay. This is a followup to last years blog entry 66 million rows in QTreeView where I got it up to 66 million rows without any work. This time around I have actual data behind each row and column and not just a model that generates the data based upon the row and column. The bugs found before were still there if I set the height of the rows too large at which point their total height was greater then max int, but I have fixed those issues and they will be in the next Qt release. Anyone know if Gtk can do this?



Update: fixed link to project and source.

No comments:

Popular Posts