in-app A|B Testing for Mobile

Is there a good solution for A|B Testing in mobile apps like online? I know with iOS it's against the TOS to have different user experiences with identical actions, but what about Android? And what about firms like Apsalar which 开发者_Go百科claim to offer A|B Testing in their analytics for apps? How would one implement that?

Artisan mobile makes an A/B testing solution for iOS and Android.

The basic idea is that you drop the SDK in your app and then put it out in the app store. You can use the service to create A/B tests and optimize your application without having to touch the code or go back through the app store for each test.

For mobile apps, A/B testing basically works by replacing static, hard-coded objects with dynamic objects that can be controlled from a remote server.

This methodology raises a potential performance issue: What if the end user's device is not connected to pull configuration data for an object being tested? We've built Splitforce (http://splitforce.com) to seamlessly setup and manage A/B testing in mobile apps while controlling for performance risk.

Los details

Once the SDK and experiment has been integrated, non-technical product or marketing folks can setup new tests or tweak existing tests on-the-fly - without having to resubmit to the app stores or hassle engineers.

On first app launch, the mobile app requests configuration data from the server and then caches that data locally on the device. This is to both ensure a consistent user experience on subsequent app launches, and prevent corrupt test results by guaranteeing accurate attribution of conversion events to variations.

If the end user's connection fails or is timed-out on first app launch, the library displays a hard-coded 'default' variation. And to make sure that everything is looking good before you go live, we've built a 'shake to preview' functionality in debug mode that does just that :-)

Once the app is deployed with Splitforce event data are stored locally and sent back to the website to be displayed for each variation alongside measurements of observed improvement and statistical confidence.

Instructions on integration of the SDKs and new tests can be found at https://splitforce.com/documentation.

And how is it used?

We've seen Splitforce used to A/B test: UI elements + layouts (color, text, images, ad/menu placements) UX workflows Game dynamics + rules Prices + promotions

We've also seen the tool used to control mobile apps remotely, by essentially setting one variation of a test subject to 100%.

1. Yes there is: E.g. the company Leanplum offers a Visual Interface Editor for iOS and Android: This requires no coding, and Leanplum will automatically detect the elements and allow you to change them. No engineers or app store resubmissions required.
2. Apple must have updated their TOS (https://developer.apple.com/app-store/review/guidelines/#user-interface) - At least I am not aware of anything that prohibits altering the UI in a way that the Leanplum Visual Editor is doing it.
3. Generally that is achieved by method swizzling (iOS) and reflection (Android).

To learn more about the Leanplum Visual Interface Editor, check out leanplum.com. They offer a free 30-day trial.

(Disclaimer: I am an Engineer at Leanplum.)

I wrote a small open source project called Switchboard. It let's you A/B test, remote configure and stage rollout things in your native mobile app. It contains a server component that specifies what information the application should have and 2 native clients for android and iOS.

You can find the codebase at github.com/keepsafe/switchboard and a blog post about how you can use it HERE

The new kid around the block is Arise.io. They provide an A/B testing service for iOS and Android.

I wrote MSActiveConfig, an extremely flexible framework to do remote configuration + A/B testing on iOS, with a portable format to be able to implement clients on other platforms: https://github.com/mindsnacks/MSActiveConfig.

This framework is being used in applications with more than 5 million users.

There have been a spate of new entrants in this field...you could check out Swerve, Appiterate, leanplum...all of them seem to be having SDKs for iOS as well, not really sure whether and how Apple TOS allows for that, but since there are some many of them doing it, there must be a way.

Yes, new entrants are showing up in app A/B testing practically every week! But, I think Appiterate has gone two steps ahead of other competitors by creating a visual interface, without any need to re-write code. I have seen their platform (you can ask for an invite. I got a demo within 12 hours) and believe me, it is actual WYSIWYG that they are providing.