Ruby IO.popen STDOUT buffering

I'm working on a script that uses IO.popen to open another program and continually read the data. It's like this:

process = IO.popen(["/the/program", "argument", "argument"])

loop do
  line = process.gets
  puts "#{line}"
end

(The actual program does more than just printing the output, obviously - that's just an example.)

The issue I'm running into is that popen seems to be buffering STDOUT from the opened process. I've confirmed this by running the program directly from a shell and through popen, side-by-side, and the Ruby one never gets one line at a time. It always gets multiple lines at a time, and is delayed.

I've tried

STDOUT.sync = true

... before popen, but that hasn't changed anything.

The program in question is definitely using \n开发者_JAVA百科 as a new line, so that's not the issue.

Do you have the source to the other program? You either need to force the other program to flush its output, or make your script look like a pty (see the pty standard lib).

See this question for a good explanation of what's going on.

EDIT: pty sample code:

require 'pty'
PTY.spawn "some-command" do |r,w,p|
  loop { puts r.gets }
end

I suspect /the/program is buffering when it detects stdout is not a terminal - you can test by piping through cat, eg:

"/the/program" "argument" "argument" | cat

The answer above, will solve it if that is the problem, ie:

#!/usr/bin/env ruby

require 'pty'
PTY.spawn "./the-program testing one Two three" do |r,w,p|
  loop { puts "GOT: #{r.gets}" }  
end

Some languages (eg C) detect if stdout is a terminal and change to line buffered - see Is stdout line buffered, unbuffered or indeterminate by default?

As an example when it works, I used a simple bash script to output each argument and the time, one at a time, with 3 seconds in between between and the ruby script worked without a problem. I added eof detection for this example.

Modified script:

#!/usr/bin/env ruby

process = IO.popen(["./the-program", "testing", "one", "Two", "three"])

while !process.eof?
  line = process.gets
  puts "GOT: #{line}"
end

the-program contents:

#!/bin/bash

for arg
do
  echo $arg
  date
  sleep 3
done

I tried with ruby version 1.9.3 and 2.1.2

$ ruby ,p
GOT: testing
GOT: Mon Jun 16 06:19:00 EST 2014
GOT: one
GOT: Mon Jun 16 06:19:03 EST 2014
GOT: Two
GOT: Mon Jun 16 06:19:06 EST 2014
GOT: three
GOT: Mon Jun 16 06:19:09 EST 2014
$ 

If I use a C program instead, then the problem reoccurs:

#include <stdio.h>

main(int argc, char **argv)
{
        int i;

        for (i=0; i<argc; i++) {
                printf("%s\n", argv[i]);
                sleep(3);
        }
}