A/B testing on App Engine?

What would be the simplest implementation of an A/B testing system running on App engine?

I'm especially keen towards performance implications of using开发者_如何转开发 Datastore for back-end (with looong query times), and database design.

Have a look to Gae/Bingo, it's an A/B split-testing framework for App Engine inspired by A/Bingo.

More information here.

You could deploy two versions of your application:

appcfg.py update -V "A" mysiteA/
appcfg.py update -V "B" mysiteB/

And then create a third version that simply chooses whether to proxy a user to A.latest.mysite.appspot.com or B.latest.mysite.appspot.com.

It is now generally available in SDK 1.6.3 as Traffic Splitting feature: http://code.google.com/appengine/docs/adminconsole/trafficsplitting.html

Assuming you want to test different versions of your app, I would suggest using a simple bit of WSGI middleware. Build something that directs x% of users to one WSGI app, and the remainder to another, sharded by whatever suits - user ID, IP address, etcetera. This should be pretty straightforward to implement, and you can pile whatever you like on top of it.

A/B test requires to show page A to some users, while page B to some other users.

App Engine has nothing to do with it. App Engine is a way to deploy applications, not direct user along the pages.

It's the function of the web framework you use to serve one page or another based on user cookie/session.

In a simple way it could be done like this:

• Get user cookie
• Find it in datastore
• Found? Use the same set of pages (A or B) as the last time
• Not found? Choose A or B randomly, save the choice into datastore along with cookie
• (May be) Place the choice into session for fast access

Then, in specific controllers/views, based on selected A or B, serve/redirect user to page A or page B. Record the outcome (whatever your outcome is -- sale, registration, ...) into datastore.

That can be done for any web framework. You didn't even told which one you use ;)