HttpWebRequest asynchronous calls filling up the stack and stripped down Reactive Extensions on Windows Phone

I am writing some code that downloads a file from a network resource to disk. Reading from the network is done asynchronously, as well as writing. I am observing a problem that asynchronously calls are in fact being called synchronously, so that each new iteration creates a new function call on the stack. Here is the code:

private IsolatedStorageFileStream fileStream = null; 
private HttpWebRequest webRequest = null; 
private Stream responseStream = null; 
private long responsePosition = 0; 
private static int BufferSize = 4096; 
private byte[] bufferRead = new byte[BufferSize]; 

private void button4_Click(object sender, RoutedEventArgs e) 
{ 
    string fileName = "TestFile.mp3"; 

    using( var store = IsolatedStorageFile.GetUserStoreForApplication() ) 
    { 
        if( store.FileExists(fileName) ) 
        { 
            store.DeleteFile(fileName); 
        } 

        fileStream = store.OpenFile(fileName, FileMode.CreateNew, FileAccess.Write, FileShare.Read); 
        webRequest = WebRequest.Create(new Uri(mpsUri.Text)) as HttpWebRequest; 

        var observableRequest = Observable.FromAsyncPattern<WebResponse>(webRequest.BeginGetResponse, webRequest.EndGetResponse); 
        Observable.Timeout(observableRequest.Invoke开发者_StackOverflow社区(), TimeSpan.FromMinutes(2)) 
            .Subscribe(response => { ResponseCallback(response); }, exception => { TimeoutCallback(); }); 
    } 
} 

private void TimeoutCallback() 
{ 
    webRequest.Abort(); 
    MessageBox.Show("Request timed-out"); 
} 

private void ResponseCallback(WebResponse webResponse) 
{ 
    if( (webResponse as HttpWebResponse).StatusCode != HttpStatusCode.OK ) 
    { 
        MessageBox.Show("Download error1"); 
    } 
    else 
    { 
        responseStream = webResponse.GetResponseStream(); 
        if( responsePosition != 0 ) 
        { 
            responseStream.Position = responsePosition; 
        } 
        IAsyncResult readResult = responseStream.BeginRead(bufferRead, 0, BufferSize, new AsyncCallback(ReadCallback), null); 
        return; 
    } 
    webResponse.Close(); 
    MessageBox.Show("Download error2"); 
} 

private void ReadCallback(IAsyncResult asyncResult) 
{ 
    int bytes = responseStream.EndRead(asyncResult); 
    DLog.Info("store:{0}, current size:{1}", bytes, fileStream.Length); 
    if( bytes > 0 ) 
    { 
        fileStream.BeginWrite(bufferRead, 0, bytes, WriteCallback, null); 
        return; 
    } 
    responseStream.Close(); 
    MessageBox.Show("Download error3"); 
} 

private void WriteCallback(IAsyncResult asyncResult) 
{ 
    DLog.Info("Stored!"); 
    responseStream.BeginRead(bufferRead, 0, BufferSize, new AsyncCallback(ReadCallback), null); 
}

And here is a snippet of the (slightly edited) stack:

(...)
MainPage.ReadCallback(System.IAsyncResult asyncResult) Line 190 + 0x1b bytes    C#
Stream.BeginRead(byte[] buffer, int offset, int count, System.AsyncCallback callback, object state) + 0x6c bytes    
MainPage.WriteCallback(System.IAsyncResult asyncResult) Line 200 + 0x1f bytes    C#
Stream.BeginWrite(byte[] buffer, int offset, int count, System.AsyncCallback callback, object state) + 0x64 bytes    
FileStream.BeginWrite(byte[] array, int offset, int numBytes, System.AsyncCallback userCallback, object stateObject) + 0x7c bytes    
IsolatedStorageFileStream.BeginWrite(byte[] buffer, int offset, int numBytes, System.AsyncCallback userCallback, object stateObject) + 0x1a bytes    
MainPage.ReadCallback(System.IAsyncResult asyncResult) Line 190 + 0x1b bytes    C#
Stream.BeginRead(byte[] buffer, int offset, int count, System.AsyncCallback callback, object state) + 0x6c bytes    
MainPage.WriteCallback(System.IAsyncResult asyncResult) Line 200 + 0x1f bytes    C#
Stream.BeginWrite(byte[] buffer, int offset, int count, System.AsyncCallback callback, object state) + 0x64 bytes    
FileStream.BeginWrite(byte[] array, int offset, int numBytes, System.AsyncCallback userCallback, object stateObject) + 0x7c bytes    
IsolatedStorageFileStream.BeginWrite(byte[] buffer, int offset, int numBytes, System.AsyncCallback userCallback, object stateObject) + 0x1a bytes    
MainPage.ReadCallback(System.IAsyncResult asyncResult) Line 190 + 0x1b bytes    C#
Stream.BeginRead(byte[] buffer, int offset, int count, System.AsyncCallback callback, object state) + 0x6c bytes    
MainPage.WriteCallback(System.IAsyncResult asyncResult) Line 200 + 0x1f bytes    C#
Stream.BeginWrite(byte[] buffer, int offset, int count, System.AsyncCallback callback, object state) + 0x64 bytes    
FileStream.BeginWrite(byte[] array, int offset, int numBytes, System.AsyncCallback userCallback, object stateObject) + 0x7c bytes    
IsolatedStorageFileStream.BeginWrite(byte[] buffer, int offset, int numBytes, System.AsyncCallback userCallback, object stateObject) + 0x1a bytes    
MainPage.ReadCallback(System.IAsyncResult asyncResult) Line 190 + 0x1b bytes    C#
Stream.BeginRead(byte[] buffer, int offset, int count, System.AsyncCallback callback, object state) + 0x6c bytes    
MainPage.WriteCallback(System.IAsyncResult asyncResult) Line 200 + 0x1f bytes    C#
Stream.BeginWrite(byte[] buffer, int offset, int count, System.AsyncCallback callback, object state) + 0x64 bytes    
FileStream.BeginWrite(byte[] array, int offset, int numBytes, System.AsyncCallback userCallback, object stateObject) + 0x7c bytes    
IsolatedStorageFileStream.BeginWrite(byte[] buffer, int offset, int numBytes, System.AsyncCallback userCallback, object stateObject) + 0x1a bytes    
(...)

This stack shows that the asynchronous callback is called immediately and the execution doesn't return to the calling function until probably all iterations are finished. I would expect that when a function calls another function asynchronously, the callback isn't called until the calling function completes and is removed from the stack. The current behavior, however, causes the stack to overflow after a couple of hundred iterations.

I am trying to workaround this limitation (or shouldn't this really be called a bug?) by using reactive extensions. So I am trying to replace the iteration that reads the network stream by an observable pattern, e.g.:

var readerFunc = Observable.FromAsyncPattern<byte[], int, int, int>(responseStream.BeginRead, responseStream.EndRead);

But in here the problems is that the Windows Phone version of the reactive library has been stripped down to support only two parameters and the return parameter:

Observable.FromAsyncPattern<T1, T2, TResult>

So I can't define the reading function as I wanted above because that would require three parameters. Even the downloadable version of the library doesn't provide more parameters.

Finally, my questions are:

1. Is there any other way of working around the original problem with the asynchronous calls being called synchronously and filling up the stack, other than using Reactive Extensions?

2. If not, then how can I read from network stream and write to a file stream asynchronously using the limited version of Reactive Extensions that is available on Windows Phone?

Any help greatly appreciated!

Quite straightforward really... you need to check if IAsyncResult.CompletedSynchronously is true. If so, take action to break up the call stack. Perhaps you could ThreadPool.QueueUserWorkItem or the WP7 equivalent?

Here is an example for you. And yes, Windows Phone 7 has indeed a limited version of Rx that accepts only two parameters and a result (while in the actual version you can use 29 overloads and 14 possible parameters).