开发者

Upload a large file using nodejs

I have the following nodejs code which uploads a file by calling a server-side API (written by me) and passing the file content as a multi-part request. The problem is that my code works perfectly with small files but it fails with large files (1 MB or above). I'm pretty sure it's a problem in my code but I'm not able to find out what it is.

        // assume file content have been read into post_data array
        //Make post
        var google = http.createClient(443, host, secure = true);
        var filepath = '/v2_0/put_file/';
        var GMTdate = (new Date()).toGMTString();
        var fileName = encodeURIComponent(destination);
        console.log("fileName : " + fileName);
        console.log("Path : " + filepath);
        var header = {
            'Host': host,
            'Authorization': 'Basic ' + authStr,
            'Content-Type': 'multipart/form-data; boundary=0xLhTaLbOkNdArZ',
            'Last-Modified': GMTdate,
            'Filename': fileName,
            'Last-Access-By': username
        };
        var request = google.request('POST', filepath, header);
        for (var i = 0; i < post_data.length; i++) {
            request.write(post_data[i]);
        }
        request.end();
        request.addListener('response', function(response){
            var noBytest = 0;
            response.setEncoding('utf8');
            console.log('STATUS: ' + response);
            console.log('STATUS: ' + response.statusCode);
            console.log('HEADERS: ' + JSON.stringify(response.headers));
            console.log('File Size: ' + response.headers['content-length'] + " bytes.");

From the logs, I see that control comes to request.end(); but I do not see the last few logs written after request.addListener() block.

I've been pulling my hair off for last couple of days trying to understand why it works for small files but not for larger files. I don't see any timeouts and the code just seems to be hung till I kill it off.

Can anyone guide me as to what am I doing wrong?

UPDATE:

post_data is an array, here is what I'm doing

post_data = [];
console.log('ContentType =' + ContentType + 开发者_StackOverflow"\n\nEncoding Style =" + encodingStyle);
post_data.push(new Buffer(EncodeFilePart(boundary, ContentType, 'theFile', FileNameOnly), 'ascii'));
var file_contents = '';
var file_reader = fs.createReadStream(filename, {
    encoding: encodingStyle
});

file_reader.on('data', function(data){
    console.log('in data');
    file_contents += data;
});

file_reader.on('end', function(){
    post_data.push(new Buffer(file_contents, encodingStyle))
    post_data.push(new Buffer("\r\n--" + boundary + "--\r\n", 'ascii'));
     ...
        var request = google.request('POST', filepath, header);
        for (var i = 0; i < post_data.length; i++) {
            request.write(post_data[i]);
        }

I look forward to your suggestions.


You should be passing either an array or a string to request.write . Is post_data an array of strings, or an array of arrays?

Also, you are posting it as multipart/form-data, so that means you have to modify your data to that format. Have you done so, or is post_data just the raw data from a file?


checkout node-formidable and this post http://debuggable.com/posts/parsing-file-uploads-at-500-mb-s-with-node-js:4c03862e-351c-4faa-bb67-4365cbdd56cb

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜