Node.js HTTP GET not returning all data
Node's HTTP module is not returning all the expected data from this URL: http://itradedata.co.za/pmchart/prices/GetPricesTRI.asp?Shortname=ACE&numprices=5
There are 5 price records when viewing in a browser, but only one in Node's HTTP GET data. Packet sniffing shows all the data going in, so why is it not in the output?
var http = require('http'),
host = 'itradedata.co.za',
records = 5,
url = '/pmchart/prices/GetPricesTRI.asp?Shortname=ACE&numprices=' + records,
client = http.createClient(80, host),
headers = { 'host': host },
req;
req = client.request('GET', url, headers);
req.on('response', function(res) {
console.log(res.statusCode);
res.on('data', function (chunk) {
console.log(chunk.toString());
});
});
req.end();
The problem seems to be in the way the server returns data... cURL also does not show the data when running curl {url}
in terminal, but it writes everything to file when using curl {url} -o {file}
. What is going on here?
Expected result:
# Attempting to get 5 prices for theTicker: ACE<BR>
# Trying JSECODE<BR>
# Sending prices for Entity_ID: 17352 Shortname开发者_如何学C: ShareID: JSECode: <BR>
2011-8-15,46,46,46,46,0,08268
2011-8-12,46,46,46,46,51,0068
2011-8-11,46,46,46,46,51,0068
2011-8-10,46,46,46,46,51,0068
2011-8-8,46,46,46,46,51,00068
Actual result:
# Attempting to get 5 prices for theTicker: ACE<BR>
# Trying JSECODE<BR>
# Sending prices for Entity_ID: 17352 Shortname: ShareID: JSECode: <BR>
2011-8-8,46,46,46,46,51,00068
The four missing records are in the middle of the result.
I'm relatively sure that it is because that site sends \r
, but no \n
.
Buffer contents:
2c 30 30 0d 32 30
0d says "back to the begin of the line", 0a says "next line". So it always goes back, but never writes a new line. It just overwrites existing output.
You could try replacing 0d with 0a to see everything.
Edit: Tried it, it works. Code:
req = client.request('GET', url, headers);
req.on('response', function(res) {
console.log(res.statusCode);
res.on('data', function (chunk) {
for (var i=0; i<chunk.length; i++)
if (chunk[i] === 0xd) chunk[i] = 0xa
console.log(chunk.toString());
});
});
req.end();
精彩评论