File Streams & Piping Data

I can write an app that would copy all input from cin into a container (let's say a set), and then print the container contents back to stdout. In the case of a set, they'd be sorted.

If I did that and compiled it as say, mysort, I could go into Unix and use the program as part of a pipe.

cat myfile.txt | mysort

and the output would be the words from myfile sorted.

In this case, mysort couldn't really do anything until all the output from cat-ing myfile.txt was piped into it. How can I do something similar to process data being generated on the fly? (Ignore the sort; I know to sort I'd have to have all the data at once to sort it).

For example开发者_运维问答, if I ran a huge program (let's call it hugeprog) and wanted to analyze the stdout and stderr of that huge program while it's running, how would I handle the input to my analyzer program (myanalyzer)?

I'd want to execute this analysis along with the program run like so:

hugeprog 2>&1 | myanalyzer

how would I handle the input to my analyzer program (myanalyzer)?

By simply reading from stdin and writing to stdout you will make your program work with anything that pipes data into it / from it.

You just have to watch out for buffering when writing out.

EDIT

Given hugeprog is running and constantly producing data, how do I continuously read it in my app?

Just like you read input from the user. My C++ is not great but cin.getline should do. In C I would use getline / `fgets.

How do I know when the hugeprog will generate an end of stream character

It's not his job to generate that character. That's just an artificial way the OS is telling you there's no more input.

Also, what if theres a period of time where no output is being produced

You wait / establish a timer and when it runs out you exit with an error.

Will my cin calls just block until there is data or will it cause a problem

It will block (possibly forever).

I think you might still hit buffering issues. If you really want to be responsive, you can try using fcntl to set O_NONBLOCK on your stdin file handle.

fcntl(file_desc, F_SETFL, O_NONBLOCK); // <fcntl.h>

It will change some of the behavior. For example, reading from that file will always return right away but it may contain nothing or it may contain only a part of what is waiting to be read so you have to adjust your code to suit.