Best practice for transferring files over web service C#

I have a bunch of files(nearly 1500, some of them are more then 500MB). Basically, we have an Defect Management System(Serena Business Mashups, some of them may know it) and I need to attach those files to related issues through their web service. the maximum file size which can be attached is 500mb. Therefore, I have focused on files which are less then 500Mb. This web service has a CreateAttachment call taking content of the files as base64binary ty开发者_运维问答pe. I have no problem with attaching small files as I red file into byte array and pass that byte array as a content which works fine. However, when it comes down to big files, I always get outofmemoryexception. I have tried nearly lots of the samples and read most of the articles but they did not help me... I have even tried to split files read them and merge arrays, or encoded strings... no success

I was just wondering whether you have better idea to achieve this task, or suggest me some best practices.

thanks very much in advance for your help.

Regards

Could you please specify what language you're talking about? Sounds like C# or Java.

If it's Java, you might want to increase the heap size of your vm.

For Java you could use parameters like this (there are better ways to get around that problem, but that would solve it for now):

java -Xms64m -Xmx1024m appname

Edit: As a general advice, don't load the file fully into memory then. Split it up as you described it, output the chunk to the socket and flush the cache after each chunk is read. Just don't close the socket/http connection.

Uploading files via plain HTTP might be unreliable. I suggest you look into Background Intelligent Transfer Service (BITS). Look into http://www.codeproject.com/KB/IP/sharpBITS.aspx?msg=2815376 for example how to use it from client side.

If you need to call a web service to pass some attributes and process the file, you can do this after BITS upload completes.

Edit: BITS uses HTTP for transfer, however it supports resume in case of network issues. Obviously, BITS is Windows only :)