Access log replay for load testing? Jmeter Pitfalls and Competitors

Context

We wish to use "replay" web server access logs to generate load tests. JMeter had come to mind as I'd recently read blog posts about using jmeter in the cloud (e.g. firing up a number of Amazon EC2 instances to generate the load)

For years I had heard of JMeter's ability to replay access logs, but in reviewing this feature I found the following.

Access Log Sampler

• DOES:

  • recreate sessions, i.e. handle the jsessionId token (thought it tries to approximate sessions by IP address);

• DOES NOT:

  • handle POST data (even if you could configure apache/tomcat to write out post data to the access log, jmeter access log sampler only handles 'common' log format).

Post data would go a long way toward recreating actual load.

Additionally, the documentation describes the Access Log Sampler开发者_运维技巧 as "alpha code" even though it's 8 years old. It doesn't seem actively maintained. (That's longer than Gmail's beta.)

HttpPerf

Another blog post pointed me to the httpperf tool. I've started to read up on it:

• blog: http://www.igvita.com/2008/09/30/load-testing-with-log-replay/
• httpperf: http://code.google.com/p/httperf/

Summary

• What's the best way to generate load testing 'scripts' from real user data?
• What has worked best for you?
• Pros and cons of various tools?

JMeter + HTTP Raw Request + Raw Data Source for me works well

I will describe how do we solve this problem using our own LT tool called Yandex Tank It can handle simple access.log but only 'GET' requests, too. When there's a need to make other types of requests, we use other ammo formats (ammo is a file containing all the requests that we gonna send to our server). Example:

342
POST / HTTP/1.1^M
Host: xxx.xxx.xxx.xxx:8080^M
Connection: keep-alive^M
Keep-Alive: 300^M
Content-Type: multipart/form-data; boundary=AGHTUNG^M
Content-Length: 1400^M
Connection: Close^M
^M
--AGHTUNG^M
Content-Disposition: form-data; name="fp"; filename="fp_tank"^M
Content-Type: application/octet-stream^M
Content-Transfer-Encoding: binary^M
...
--AGHTUNG--^M

A number ('342') on the first line is the size of a following request. Request is in it's raw format. You could write a simple script in your favourite language that generates such ammo files from your access.log and then use it for load testing.

Such ammo format makes it really flexible. For example, this code generates ammo from FCGI logs (POST bodies are encoded in Base64). But on the other hand you will need to handle sessions manually.

You can easily replay access logs with POST data using ZebraTester. It has many plugins similar to JMeter and also ability to add inline scripts using which you can easily target POST payload, URLs, timestamps, etc. from the access logs. You can run load tests directly from the tool locally or copy the recorded script to the SaaS portal to run massive million virtual user load tests