Linux: Formatting stream (pipe) output with column? (undetermined file size)

I would like to format text content that I am getting as a stdout stream using column, but I fail. For instance, using a single line, all works fine:

$ echo "1 12 123 1234 1 12 123 1234 " | column -t
1  12  123  1234  1  12  123  1234

.. however, if I try to simulate an "endless" stream:

$ while true; do echo "1 12 123 1234 1 12 123 1234 "; sleep 1; done | column -t
^C

... there is simply no response, until one exits with Ctrl-C.

(Note that the while thing is just meant to simulate reading an endless stream from a device, as in 'cat /dev/ttyUSB0')

This leads me to believe that, e开发者_Go百科ven if column by default should accept standard input, it needs a "complete" and finished file (that is, with a determined file size) in order to work; and in the case of an "endless" stream, that is not the case - so it never outputs anything.

Any suggestion on how I could achieve column formatting in this context? EDIT: Doesn't necesarily have to be the column program; anything else that formats text into columns would be fine (but I'm afraid, for instance awk also needs a complete file)...

Thanks in advance,

Cheers!

I, too, found column disappointing in its inability to stream, and sdaau's code in its inability to measure my columns. So here's a fix for both those problems. Maybe not terribly efficient, but it's just for debugging, anyway. I have it in my ~/bin folder as pcol (be sure to chmod +x it):

#!/usr/bin/env python
import sys

sep = ','
gutter = 2
for arg in sys.argv:
    if arg.startswith('-s'):
        sep = arg[2:]
    elif arg.startswith('-g'):
        gutter = int(arg[2:])

widths = []
while True:
    raw = sys.stdin.readline()
    line = raw.strip('\0').strip('\n')
    vals = line.split(sep)
    if len(vals) > len(widths):
        widths = [0] * len(vals)

    widths = [max(len(val) + gutter, width) for val, width in zip(vals, widths)]
    metaformat = '%%%ds' * len(widths)
    format = metaformat % tuple(widths)
    print format % tuple(vals)

There is no response because your second command creates an infinite loop and then asks that once done, the result be piped to column -t. Since the loop will never terminate, column -t doesn't do anything. I'm not sure what you're trying to accomplish, but a variation of 2nd command that does produce output at one second intervals (which is what I'm assuming you want) is this:

while true; do echo "1 12 123 1234 1 12 123 1234 " | column -t; sleep 1; done

EDIT: I see now what you're doing. For your situation, you could use sed to replace the spaces with tabs, which would automatically align your numbers to previous previous lines as long as number of digits doesn't exceed tab size:

echo "1 12 123 1234 1 12 123 1234 " | sed 's/ /\t/g'

OK, thanks to responses here, I managed to cook up a python script that will do that; obviously, you have to know (and enter in the script) the format of the columns a-priori; here's the script, columnizeline.py:

#!/usr/bin/env python

import sys;

#below no work:
#print sys.stdin;
#for line in sys.stdin:
#   sline=line.split(' ')
#   print sline

while 1:
  next = sys.stdin.readline()
  ns = next.strip('\0').split(' ') # remove null char
  ns.pop() # remove the last entry in the list, it is '\n'
  #~ nextf = "%4s %4s %4s %4s %4s %4s %4s %4s" % (ns[0], ns[1], ns[2], ns[3], ns[4], ns[5], ns[6], ns[7])
  nextf=""
  nsc = ns[0:6] # first 6 elements only
  for nsi in nsc:
    nextf = "%s%5s" % (nextf, nsi)
  print nextf

... and here is a little test:

$ while true; do echo "1 12 123 1234 1 12 123 1234 "; sleep 1; echo "100 1 123 12 1 12 123 1234 "; sleep 1; done | ./columnizeline.py 
   1   12  123 1234    1   12  123 1234
 100    1  123   12    1   12  123 1234
   1   12  123 1234    1   12  123 1234
 100    1  123   12    1   12  123 1234
   1   12  123 1234    1   12  123 1234
 100    1  123   12    1   12  123 1234
^CTraceback (most recent call last): [...]

Cheers!