Strings appear to be sticking around too long

In short, I've got an application that converts a flat data file into an XML file. It does this by populating objects and then serializing them to XML.

The problem I'm running into is that the Garbage Collector does not seem to be taking care of the serialized strings. 3500 record files are running up to OutOfMemoryExceptions before 开发者_C百科they finish. Something is fishy, indeed.

When I take the serialization out of the mix and simply pass an empty string, the memory consumption remains as expected, so I've ruled out the possibility that my intermediate objects (between flat file and xml) are the problem here. They seem to be collected as expected.

Can anyone help? How do I make sure these strings are disposed of properly?

Update: Some sample code

// myObj.Serialize invokes an XmlSerializer instance to handle its work    
string serialized = myObj.Serialize();
myXmlWriter.WriteRaw(serialized);

This is basically where the problem is ocurring--if I take the string serialized out of play, the memory problems go away, too, even though I'm still transforming the flat file into objects, one at a time.

Update 2: Serialize method

public virtual string Serialize()
{
      System.IO.StreamReader streamReader = null;
      System.IO.MemoryStream memoryStream = null;

      using (memoryStream = new MemoryStream())
      {
          memoryStream = new System.IO.MemoryStream();
          Serializer.Serialize(memoryStream, this);

          memoryStream.Seek(0, System.IO.SeekOrigin.Begin);
          using (streamReader = new System.IO.StreamReader(memoryStream))
          {
              return streamReader.ReadToEnd();
          }
      }
}

You need to make sure they aren't referenced anywhere. Before an OutOfMemoryException is thrown, the GC is run. If it isn't recovering that memory, that means something is still holding on to it. Like others said, if you post some code, we might be able to help. Otherwise you can use a profiler or WinDbg/SOS to help figure out what is holding onto your strings.

Very curious indeed. I added the following dandy after each serialized record writes to the XmlWriter:

if (GC.GetTotalMemory(false) > 104857600)
{
     GC.WaitForPendingFinalizers();
}

and wouldn't you know it, it's keeping it in check and it's processing without incident, never getting too far above the threshold I set. I feel like there should be a better way, but it almost seems like the code was executing too fast for the garbage collector to reclaim the strings in time.

Do you have an example of your code - how you're creating these strings? Are you breaking out into unmanaged code anywhere (which means you would be required to clean-up after yourself).

Another thought is how you are converting flat data file into XML. XML can be somewhat heavy depending on how you are building the file. If you are trying to hold the entire object in memory, it is very likely (easy to do, in fact) that you are running out of memory.

It sure looks like your method could be cleaned up to be just:

public virtual string Serialize()
{
    StringBuilder sb = new StringBuilder();
    using (StringWriter writer = new StringWriter(sb))
    {
        this.serializer.Serialize(writer, this);
    }

    return sb.ToString();
}

You are creating an extra MemoryStream for no reason.

But if you are writing the string to a file, then why don't you just send a FileStream to the Serialize() method?