What is the best way of running shell commands from a web based interface?

Imagine a web application that allows a logged in user to run a shell command on the web server at the press of a button. This is relatively simple in most languages via some standard library os tools.

But if that command is long running you don't want your UI to hang. Again this is 开发者_JS百科relatively easy to deal with using some sort of background process or putting the command to be executed onto a message queue (and maybe saving the output and status somewhere for later consumption). Just return quickly saving we'll run that and get back to you.

What I'd like to do is show the output of said web ui triggered shell command as it happens. So vertically scrolling text like when running in a terminal.

I have a vague idea of how I might approach this, streaming the output to a websocket perhaps and simply printing the output to screen.

What I'd like to ask is:

Are their any plugins, libraries or applications that already do this. Something I can either use or read the source of. Ideally an open source python/django or ruby/rails tool, but other stacks would be interesting too.

I'm not sure if it's what you want, but there are some web based ssh clients out there. If you care about security and really just want dynamic feedback, you could look into comet or just have a frame with its own http session that doesn't end until it's done printing.

web-based ssh client would work, into the host (there are java ssh clients out there).

Ruby has a web-based terminal: http://tryruby.org (link to the source is at the bottom of the page). You could also embed Ruby via jruby: http://tim.lossen.de/2007/03/jruby/applet.html http://github.com/jruby/jruby/blob/master/samples/irb-applet.html

I haven't heard of any libraries that do this, but you'll need to setup the system command and call out to the system. You will then need to "pump" the sysout and syserr standard inputs and pipe that data back out to your web client.

As an example for this style of problem, look into code snippits of how people use ruby/python/etc to transcode a video, i.e. http://kpumuk.info/ruby-on-rails/encoding-media-files-in-ruby-using-ffmpeg-mencoder-with-progress-tracking/ - my example was taken from this blog post.

class MediaFormatException < StandardError
end

def execute_mencoder(command)
  progress = nil
  IO.popen(command) do |pipe|
    pipe.each("r") do |line|
      if line =~ /Pos:[^(]*(s*(d+)%)/
        p = $1.to_i
        p = 100 if p > 100
        if progress != p
          progress = p
          print "PROGRESS: #{progress}n"
          $defout.flush
        end
      end
    end
  end
  raise MediaFormatException if $?.exitstatus != 0
end

I don't know if this example is pulling data from both sysout and syserr, but you will definitely need to be pulling data from both of those interfaces, typically if the buffer fills up, the executing command might hang or fail (I have experienced this with Python). This method will also look different if the only thing you do is return line to the web client - in a terminal, the progress indicator of ffmpeg/mencoder remains stationary on the bottom line, but this method will give you a long list of progress indicator updates. Pipe line out to your terminal and you'll see what I'm referring to.

So, I've tried to answer my own question with code as I couldn't find anything to quite fit the bill. Hopefully it's useful to anyone coming across the same problem.

Redbeard 0X0A pointed me in the general direction, I was able to get a stand along ruby script doing what I wanted using popen. Extending this to using EventMachine (as it provided a convenient way of writing a websocket server) and using it's inbuilt popen method solved my problem.

More details here http://morethanseven.net/2010/09/09/Script-running-web-interface-with-websockets.html and the code at http://github.com/garethr/bolt/

Certainly not the best way to run shell commands, but likely the easiest:

#!/bin/sh

echo Content-Type: text/plain
echo

/usr/bin/uptime

http://www.sente.cc/scripts/uptime.cgi

Take a look at Galaxy (online demo) or Yabi.

Except from the requirement to be able to show output during the job run, they are both excellent solutions to this! They are also both written i Python (and Yabi even on django).

They were both built with bioinformatics in mind, but really are both general job runner/workflow tools.

They will let you specify parameters in a web interface, see queued/running/finished jobs in a separate column, and after the jobs are finished, inspect details and results, or re-run the job, with possibly changed parameters.

Galaxy is the easier one to install. The Galaxy installation boils down to downloading and run "sh run.sh"), and adding your own tool boils down to creating an XML file in the line of:

<tool id="mytool" name="My Tool" version="1.0.0">
  <description>Does this and that</description>
  <command>somecommand --aparam $aparam</command>
  <inputs>
    <param name="aparam" type="text" label="A parameter"/>
  </inputs>
  <outputs>
    <data name="outfile" format="tabular"/>
  </outputs>
</tool>

... and place it in the /tools folder, and add a line in the tool_conf.xml to tell galaxy of your new tool (There you can also get rid of the bioinformatics-tools, so they don't mess up your tools menu).

Yabi is more complicated to install (see the readme file), but the process might be smooth if you are on the right kind of system. On the other hand, it allows you even do the tool configuration in the web interface, rather than as an XML file like in Galaxy.

Galaxy still is the one with the biggest community though, which is reflected in the number of features/already integrated tools (See the toolshed for shared tools/wrapper).

websocketd looks like the perfect tool for that.