Dynamic creation of large html table in javascript performance

I have an application which is used for data analysis and I'm having a few performance issues with the creation of the table. The data is extracted from documents and it is important that all data is presented on one page (pagination is not an option unfortunately).

Using jQuery, I make an ajax request to the server to retrieve the data. On completion of the request, I pass the data to an output function. The output function loops through the data array using a for loop and concatenating the rows to a variable. Once the looping is complete, the variable containing the table is then appended to an existing div on the page and then I go on to bind events to the table for working with the data.

With a small set of data (~1000-2000 rows) it works relatively good but some of the data sets contain upwards of 10,000 rows which causes Firefox to either crash and close or become unresponsive.

My question is, is there a better way to accomplish what I am doing?

Here's some code:

//This function gets called by the interface with an id to retrieve a document
function loadDocument(id){
    $.ajax({
        method: "get",
        url: "ajax.php",
        data: {action:'loadDocument',id: id},
        dataType: 'json',
        cache: true,
        beforeSend: function(){
            if($("#loading").dialog('isOpen') != true){
                //Display the loading dialog
                $("#loading").dialog({
   开发者_开发技巧                 modal: true
                });
            }//end if
        },//end beforesend
        success: function(result){
            if(result.Error == undefined){
                outputDocument(result, id);
            }else{
                <handle error code>
            }//end if
            if($('#loading').dialog('isOpen') == true){
                //Close the loading dialog
                $("#loading").dialog('close');
            }//end if
        }//end success
    });//end ajax
};//end loadDocument();


//Output document to screen
function outputDocument(data, doc_id){

    //Begin document output
    var rows = '<table>';
    rows += '<thead>';
    rows += '<tr>';
    rows += '<th>ID</th>';
    rows += '<th>Status</th>';
    rows += '<th>Name</th>';
    rows += '<th>Actions</th>';
    rows += '<th>Origin</th>';
    rows += '</tr>';
    rows += '</thead>';
    rows += '<tbody>';

    for(var i in data){
        var recordId = data[i].id;
        rows += '<tr id="' + recordId + '" class="' + data[i].status + '">';
        rows += '<td width="1%" align="center">' + recordId + '</td>';
        rows += '<td width="1%" align="center"><span class="status" rel="' + recordId + '"><strong>' + data[i].status + '</strong></span></td>';
        rows += '<td width="70%"><span class="name">' + data[i].name + '</span></td>';
        rows += '<td width="2%">';
        rows += '<input type="button" class="failOne" rev="' + recordId + '" value="F">';
        rows += '<input type="button" class="promoteOne" rev="' + recordId + '" value="P">';
        rows += '</td>';
        rows += '<td width="1%">' + data[i].origin + '</td>';
        rows += '</tr>';
    }//end for

    rows += '</tbody>';
    rows += '</table>';
    $('#documentRows').html(rows);

I was initially using a jQuery each loop but switched to the for loop which shaved off some ms.

I thought of using something like google gears to try offloading some of the processing (if that's possible in this scenario).

Any thoughts?

joinHi,

The rendering is a problem, but there is also a problem with concatenating so many strings inside the loop, especially once the string gets very large. It would probably be best to put the strings into individual elements of an array then finally use "join" to create the huge string in one fell swoop. e.g.

var r = new Array();
var j = -1, recordId;
r[++j] =  '<table><thead><tr><th>ID</th><th>Status</th><th>Name</th><th>Actions</th><th>Origin</th></tr></thead><tbody>'; 
for (var i in data){
    var d = data[i];
    recordId = d.id;
    r[++j] = '<tr id="';
    r[++j] = recordId;
    r[++j] = '" class="';
    r[++j] = d.status;
    r[++j] = '"><td width="1%" align="center">';
    r[++j] = recordId;
    r[++j] = '</td><td width="1%" align="center"><span class="status" rel="';
    r[++j] = recordId;
    r[++j] = '"><strong>';
    r[++j] = d.status;
    r[++j] = '</strong></span></td><td width="70%"><span class="name">';
    r[++j] = d.name;
    r[++j] = '</span></td><td width="2%"><input type="button" class="failOne" rev="';
    r[++j] = recordId;
    r[++j] = '" value="F"><input type="button" class="promoteOne" rev="';
    r[++j] = recordId;
    r[++j] = '" value="P"></td><td width="1%">';
    r[++j] = d.origin;
    r[++j] = '</td></tr>';
}
r[++j] = '</tbody></table>';
$('#documentRows').html(r.join(''));

Also, I would use the array indexing method shown here, rather than using "push" since, for all browsers except Google Chrome it is faster, according to this article.

Displaying that many rows is causing the browser's rendering engine to slow down, not the JavaScript engine. Unfortunately there's not a lot you can do about that.

The best solution is to just not display so many rows at the same time, either through pagination, or virtual scrolling.

The way you are building your string will cause massive amounts of garbage collection.

As the string gets longer and longer the javascript engine has to keep allocating larger buffers and discarding the old ones. Eventually it will not be able to allocate sufficient memory without recycling the remains of all the old strings.

This problem gets worse as the string grows longer.

Instead try adding new elements to the DOM one at a time using the jQuery manipulation API

Also consider only rendering what is visible and implement your own scrolling.

You can do couple of things to increase the performance:

1. your rows variable is getting bigger and bigger so, don't store the html in one variable. solution can be $.each() function and each function you append the element into DOM. But this is minor adjustment.
2. Html generating is good, but you can try DOM creating and appending. Like $('<tr></tr>').
3. And finally, this will solve your problem for sure : use multiple ajax call in the first ajax call collect how many data is available and fetch approximately 1,000 or may be more data. And use other calls to collect remaining data. If you want, you can use synchronous call or Asynchronous calls wisely.

But try to avoid storing the value. Your DOM size will be huge but it should work on moder browsers and forget about IE6.

@fuel37 : Example

function outputDocumentNew(data, doc_id) {
    //Variable DOM's
    var rowSample = $('<tr></tr>').addClass('row-class');
    var colSample = $('<td></td>').addClass('col-class');
    var spanSample = $('<span></span>').addClass('span-class');
    var inputButtonSample = $('<input type="button"/>').addClass('input-class');

    //DOM Container 
    var container = $('#documentRows');
    container.empty().append('<table></table>');

    //Static part
    var head = '<thead>\
                <tr>\
                    <th width="1%" align="center">ID</th>\
                    <th width="1%" align="center">Status</th>\
                    <th width="70%">Name</th>\
                    <th width="2%">Actions</th>\
                    <th width="1%">Origin</th>\
                </tr>\
                </thead>';
    container.append(head);

    var body = $('<tbody></tbody>');
    container.append(body);

    //Dynamic part
    $.each(data, function (index, value) {
        var _this = this;

        //DOM Manupulation
        var row = rowSample.clone();

        //Actions
        var inpFailOne = inputButtonSample.clone().val('F').attr('rev', _this.id).addClass('failOne').click(function (e) {
            //do something when click the button.
        });
        var inpPromoteOne = inputButtonSample.clone().val('P').attr('rev', _this.id).addClass('promoteOne').click(function (e) {
            //do something when click the button.
        });

        row
        .append(colSample.clone().append(_this.id))
        .append(colSample.clone().append(spanSample.colne().addClass('status').append(_this.status)))
        .append(colSample.clone().append(spanSample.colne().addClass('name').append(_this.name)))
        .append(colSample.clone().append(inpFailOne).append(inpPromoteOne))
        .append(colSample.clone().append(_this.origin));

        body.append(row);
    });
}

in this process you need to create & maintain id's or classes for manipulation. You have the control to bind events and manipulate each elements there.

Answering to get formatting

What happens if you do

for(var i in data){
    var record = data[i];
    var recordId = record.id;
    rows += '<tr id="' + recordId + '" class="' + record.status + '">';
    rows += '<td width="1%" align="center">' + recordId + '</td>';
    rows += '<td width="1%" align="center"><span class="status" rel="' + recordId + '"><strong>' + data[i].status + '</strong></span></td>';
    rows += '<td width="70%"><span class="name">' + record.name + '</span></td>';
    rows += '<td width="2%">';
    rows += '<input type="button" class="failOne" rev="' + recordId + '" value="F">';
    rows += '<input type="button" class="promoteOne" rev="' + recordId + '" value="P">';
    rows += '</td>';
    rows += '<td width="1%">' + record.origin + '</td>';
    rows += '</tr>';
}//end for

Per others suggestions (I'm not reputable enough to comment yet, sorry!), you might try the TableSorter plugin to handle only displaying a usable amount of data at a time.

I don't know how it fares at very high numbers of rows, but their example data is 1000 rows or so.

This wouldn't help with JS performance but would keep the burden off the browser renderer.

Could try this...

Improve Loops

Improve String Concat

var tmpLst = [];

for (var i=0, il=data.length; i<il; i++) {
    var record = data[i];
    var recordId = record.id;

    tmpLst.push('<tr id="');
    tmpLst.push(recordId); 
    tmpLst.push('" class="'); 
    tmpLst.push(record.status);
    tmpLst.push('">');
    tmpLst.push('<td width="1%" align="center">');

...ect...


}
rows += tmpLst.join('');

This might squeeze an extra bit of performance...

var lstReset = i * lstReset.length;
tmpLst[lstReset + 1]='<tr id="';
tmpLst[lstReset + 2]=recordId; 
tmpLst[lstReset + 3]='" class="';