Recommendation Engin for One of a Kind "Products"

A market we are building allows people to list their stuff to sell but in batches / bags / boxes. We are looking to build in a recommendation engine for this, but most of the articles out there seem better suited for markets that "sell" large quantities of many products - ie amazon, netflix etc. Because every listing is somewhat unique, what is the best approach for a recommendation engine? Any relevant articles out there?

We know items people have bought in the past. We know the size 开发者_运维问答or age appropriateness they are looking for.

The listed bundles have categories, brands, sizes/ages, colors and free form text.

Any ideas to help us get started? Any particular language you think would be best if our data is stored in MySQL?

There are several things you can filter with a recommendation engine. You can filter on what a particular user has bought before (in your case, which features have been present in the products they have bought). You can also filter on social groupings--users like them, or on product groupings--other products like the ones you have sold before. I'd recommend that you first cluster the products, and then map the individual or groups to the features in that cluster of products. So, you'll end up with a recommendation engine that says: people who bought items with this feature also bought items with these features. Then, you can create an engine for known users: you tend to buy products with these features, here are some more items like those. Finally, you can build an engine for groups: people like this tend to buy products with these features.

With several models in hand, your system can turn to the appropriate one, depending on what they know at the moment: known user, known user group, or just known browsing history.

Since you are recommending batches of more unique products, you'll want to add an additional model after you get your recommendations that will filter out inappropriate recommendations. This model will represent compatibility. A new game using the same console that the user used before is more compatible than another console. If they bought a new car last month, you wouldn't recommend a new car, but maybe a package of ten car washes.

You could use several different concepts for this last model. If you are going to add explicit knowledge to your model that's in people's heads, you may want to build a belief network that filters out inappropriate recommendations. If you are going to use collective intelligence, you could use simple regression, a support vector machine, or an artificial neural network. I would go with the easiest to implement filter and not worry about choosing the first model you build. You'll probably build a handful of models before you settle on one giving you good results with appropriate effort.

Your filtering model will go through a test phase where you make a recommendation, filter it for appropriateness, then filter it again with some sort of human intervention--a set of "answers" you want your filter to learn, or just a human being double-checking results. Then you'll retrain your filter with the updated results, resample and test again.

As far as the recommendation engine goes, you can do SVD with the GNU scientific library (bindings available for about any platform). You could also choose the Mahout recommendation engine (part of the Hadoop world) if you are going to be using big data. For the filter, you may want to look at apophenia, libsvm, or FANN.

You could also choose to work in an analytics framework for a while until you feel like you've got a handle on things. Some to choose from are Weka, R, Octave, Matlab, Maple, and Mathematica. I think I've listed those in terms of price first, then ease of use.

As far as resources, there are a few good introductory books: Collective Intelligence, Mahout (MEAP from Manning), Data Mining (all about Weka), and Modeling with Data (apophenia reference).

My last thought is that however sophisticated you do or don't get with your recommendation engine, most of the value is in the user experience. One of the people from Amazon wrote that their recommendation engines worked best when they told the user why they were making a recommendation. That helps the user quickly adopt your reasoning (an emotive response to their old and good purchase), or reject it and keep going (they already have something like that, they don't need another one).

Personally I prefer Ruby, but Ruby, Python and Perl can easily connect to MySQL.

One of the reasons I like Ruby is its Sequel gem which is a very powerful ORM, making database access very easy to manage. If you go with a MVC, Ruby has Rails which favors ActiveRecord as its ORM, which also makes it easy to talk to MySQL. There's also Sinatra and Padrino, which are a bit lighter weight ORMs, but very capable too. They're more DB agnostic out of the box and nicely integrate with Sequel.