How to plot a realtime graph (histogram) using data obtained in a text file

I have a continuously generated data (text file) generated by a program on the server. I want to plot the data as a real-time graph just like powergrid does. This was my approach:

As the data is generated continuously on the server in a text file, I wrote a PHP script 开发者_如何学Cwhich reads that file(get_file_contents), outputs the data points and plot the graph using sparkline jQuery plugin. But the problem is that it reads the file all at once. Moreover, the text file keeps on growing. Can anyone suggest me a better approach?

As you're talking about using a Javascript plotting solution you do the following:

• on page load you create the current graph by reading the complete text file and remembering it's size.
• after the page is loaded you create a Javascript function that regularly polls a specific script on your server using AJAX-techniques (XMLHttpRequest) and passing the last-known filesize of your text file as a parameter.
• your polling script takes the filesize parameter, opens the text file, skips through the file until it reaches the point from which you last read the file (filesize-parameter).
• the polling script returns all the available data from filesize to the end of the file and the new filesite
• your Javascript reads in the AJAX response and adds the required plot points to your graph
• you can then start over polling your server-side script with the new filesize as a parameter

This procedure involves server-side as well as client-side programming but can be accomplished easily.

The following is a sample polling script that requires a index paramater that tells the script from which position to read the text file and returns a JSON-encoded list of plot points and the new index pointer.

// poll.php
$index = (isset($_GET['index'])) ? (int)$_GET['index'] : 0;
$file = fopen('path/to/your/file.txt', 'r');
$data = array(
    'index' => null,
    'data'  => array()
);
// move forward to the designated position
fseek($file, $index, SEEK_SET);
while (!feof($file)) {
    /*
     * assuming we have a file that looks like
     * 0,10
     * 1,15
     * 2,12
     * ...
     */
    list($x, $y) = explode(',', trim(fgets($handle)), 2);
    $data['data'][] = array('x' => $x, 'y' => $y);
}
// set the new index
$data['index'] = ftell($file);
fclose($file);

header('Content-Type: application/json');
echo json_encode($data);
exit();

The corresponding Javascript/jQuery snippet could be:

// the jQuery code to poll the script
var current = 0;
function pollData() {
    $.getJSON('poll.php', { 'index': current }, function(data) {
        current = data.index;
        for (var i= 0; i < data.data.length; i++) {
            var x = data.data[i].x;
            var y = data.data[i].y;
            // do your plotting here
        }
    });
}
// call pollData() every 5 seconds
var timer = window.setInterval(pollData, 5000);

Please be careful that this is only an example and that it lacks all error checking (e.g. concurrent calls to pollData() on the same page will be problematic).

It sounds like you have the visualization part mostly worked out. If the dataset is too large to re-calculate, you may want to look into techniques for maintaining incremental histograms. Here are a few papers that may help:

• The history of histograms (abridged)
• Random Sampling for Histogram Construction: How much is enough?
• Data-Streams and histograms
• Fast Incremental Maintenance of Approximate Histograms

First off, I would not generate the graph on the user side. That has simple reasons: Not everyone has JavaScript enabled (okay, depends on your target group) and it is probably not very fast.

Since you are already using PHP, I would therefore recommend using a package like pChart for creating the graphs on the server side. Another positive side-effect of this is that said package comes with caching, too. This would, for example, allow you to create the graph only when the data text file is changed (assuming you generate that with PHP, too - else you can simply check if the file was modified every time your PHP script is run) and thus saving loads of resources ;)