How do I run PHPUnit Selenium tests over a ssh connection?

I'm using the yii framework and trying to get its unit tests running while connected over ssh on a CentOS server. When I run phpunit, it开发者_开发百科 tries to launch Firefox, which fails with the error "no display specifiied"

General theory

Error: no display specified

To understand that error message you first have to understand how the X Windowing System works - that is the name of the framework used by Linux (and other types of Unix) systems used to display graphical user interfaces.

X consists of two parts - there is a client and a server. Client is the program that wants to draw the interface - in your case that would be Firefox. Server is a program that makes drawing possible. There are X servers available for all the major operating systems. Linuxes and OSX usually ship with one, on Windows you will have to find and install one - Cygwin/X is one option, but there are others.

So why is this client/server architecture even necessary?

Most of the time its not even needed. If you happen to run Linux locally, then you will not even notice that there is any kind of client/server communication happening somewhere - but there is.

Where X shines though is that this architecture means that network capabilities are built right into it. You can run a client (Firefox) on one machine and display the GUI on a completely different machine. You could run 10 different clients on 10 different machines and have them all display output on a single machine thanks to X. Think VNC or Remote Desktop - X is somewhat similar, but you could say that its on steroids compared to those. And X has had this ability for a really long time.

Whenever your start up a X client (a program that wants to display graphical user interface) it looks for an X server. One possibility for the client to find one is an environment variable named DISPLAY. I'm on OSX and this is what I see.

[~]> echo $DISPLAY
/tmp/launch-ihNtDq/org.x:0

This point to my local X server. It could point to any server on my local network. When the client finds this environment variable it will connect to it and the user interface pops up.

If client cannot find this environment variable - you will get the familiar

Error: no display specified

Back to Yii

Looks like Yii has Selenium tests bundled. PHPUnit needs to start up Selenium RC to control a Firefox instance to run those tests. Selenium RC (or maybe Firefox itself) fails to find DISPLAY environment variable. And dies with the above error.

How do you solve this issue?

There are 3 options

1) install Yii, PHPUnit and all their dependencies locally. Selenium runs just fine on Windows. It wont use the X protocol on Windows, so none of that business with X clients and X servers. And you can then run Yii testsuite locally.

2) install an X server on your Windows box. Then enable 'X Forwarding' in your ssh client settings (or use the -X command line parameter for ssh). When you do that, then there will be a DISPLAY variable set when you are logged in to that CentOS server. You can verify it by typing the echo command above. Then the X client on CentOS can talk (show GUI) to the X server on your Windows machine - all the X traffic is tunneled over the ssh connection. This however means that you need java (which selenium RC is built in) and Firefox on that CentOS server. You may or may not have them there.

3) use a virtual framebuffer - for example Xvfb - an X server that performs all drawing operations in memory, not showing any output anywhere.

What good is that? Selenium has commands for taking screenshots at any point during the test run and saving them to files. For example a typical Selenium test would check whether an element exist on the page - and to make a screenshot when it does not. Screenshot would then be saved in a file, which you can view later to determine what the reason for the failure was. Making screenshots works just fine with a virtual framebuffer.

Final clarification

Note that Selenium tests are merely one type of tests that PHPUnit can run. Selenium in not required to write PHPUnit tests, its an optional add-on. But Yii testsuite apparently relies upon it.

Last, but not least

Integration tests (which Selenium tests are) are not usually ran on production systems, because there is a chance that test data is left behind in your production database. Also, getting good test results means isolating from external factors as much as possible - the contents of your production database will be constantly changing and this may affect your tests.

Normally all tests will be executed somewhere else (your development machine, dedicated QA server, whatever you have), before the new code is deployed to production servers. After all the point of tests is to verify that the system works after the changes. There is not much value in running them on production systems - the code does not change after it has been deployed.

Of course - its up to you - if you see value in doing those tests on production system, go right ahead.

Its simpler than you think. Run Selenium locally on your desktop, make sure phpunit is setup on the remote server. Then start a reverse SSH tunnel in your SSH connection. This varies depending on your SSH client. In PuTTY, there is a setting for SSH tunnels and you can reverse the direction by selecting the remote option. Check out this page for details. With OpenSSH from the command line, its done like this:

ssh -R 4444:localhost:4444 user@remoteserver

This will listen on the remote server on port 4444 and forward it to your selenium server running on localhost on your desktop port 4444.

Once you've done that, you'll need to change the TEST_BASE_URL setting in yourproject/protected/tests/WebTestCase.php to go to the remote server's URL for your yii project.

The simplest way to run a gui test on from another agent machine on a Windows client is to use "psexec" (http://technet.microsoft.com/en-us/sysinternals/bb897553.aspx).