Doing HTTP post of JSON object every second

I开发者_开发百科 have to do a HTTP post in java every second after building a json object. The json object is built from reading a CSV file which is huge (200Mbs+), so my problem is how do I read x number of lines build x objects and post that every second(as it is not possible to parse the whole 200mb file in less than a second) and continue reading the next x lines.

Please let me know your thoughts..

Can I use Java timer class, and keep reading the CSV file and at the same time post the json object to the server every second with the formed json?

It is hardly possible to read, parse, convert and send a 200 MB file once per second.

So you need to change your design:

My suggestion would be to only send changed lines, something like this:

{
"1" : {"field1":"value1","field2":"value2"},
"17" : {"field1":"value1","field2":"value2"}
}

Which of course gives you new problems:

The client needs to figure out which lines have changed, and the server needs to integrate the changed lines with the existing data.

I would make it depending on the file size and not depending on time.

BufferedReader fin = null; //create it
    Gson gson=new Gson();  //Google code open source library for JSON in Java  
    ArrayList<JSONObject> jsonList=new ArrayList<JSONObject>();    
    while (((line = fin.readLine()) != null)) {
                            if ( line.length()==0 ){
                                //"Blank line;
                            }else{
                                        currJSON=loadJSON(line);//You have to load it in a Java Object

                               if ( jsonList.size()<MAX_JSON){
                               jsonList.add(currJSON);
                            }

        if (JsonList.size()==MAX_JSON){ //Define the maximum size of the list you want to post
        gson.toJson(jsonList); //Convert to JSON
        //You should post your Json with some Http Connection to your server
    jsonList.clear();