Delphi - Check if memory is being released "on time"

I have a GUI application that doesn't have a memory leak. I have confirmed this with FastMM over numerous test cycles. On one particular client's server, I get random crashes. The server specs are well in line with that of our other clients (and we have actually tried on various hardware), and so are the files used by the program (as far as I can tell, There is some super-sensitive material that I can't get access to, but there doesn't seem to be anything out of the ordinary there).

I have tried the likes of EurekaLog and MadShi, to maybe narrow down the issue, but unfortunately, they only seem to catch an exception at the time of the crash on occasion, not all the time. When it does, it usually shows one or more "Out of memory" error prior to the crash.

So I'm thinking maybe some objects get freed "too late", i.e. only when the application is shutting down, as opposed to when I mean to free them? I've seen the FastMMUsageeTracker demo, but couldn't really make sense of it all. Is there documentation anywhere? Or could someone put in (somewhat accessible) words how I could go about checking for this?

Alternatively, what would be the best approach to detect that the application is approaching its "memory limit开发者_StackOverflow", so as to take some preventive action? If I understand properly, a regular Delphi app being 32bit, it should be fine handling up to 2Gb of memory (provided the hardware supports it of course), correct?

PS: Delphi 2009 or XE, if that's relevant

Thanks!

EDIT - Issue possibly solved

We were able to find an issue where a pop-up window that closes and frees itself automatically after a while was being created at a much faster rate than it was disappearing. This would eat a huge amount of memory over time, and then any memory allocation would basically bring it over the edge and trigger the "out of memory" problem.

This would explain why the stack traces where inconsistent.

I'm not entirely convinced this is our only issue, as, even though unlikely, this scenario could well have happened before in the years our application has been running, but somehow it hasn't. I'll do a lot more digging on this issue.

Thanks to all who responded, each answer actually has valuable info in it.

If you have Delphi XE, it comes with AQTime, and AQTime has a memory allocation profiler as part of its bag of tricks. If you run that on your program you might be able to see where your RAM is going.

Forget about "Windows" memory - what you want is the actual memory allocated by the application. This is the only way you can tell if you are allocating memory that is not being freed over time. For Delphi 2006+ with FastMM, this is what you need:

//------------------------------------------------------------------------------  
// CsiGetApplicationMemory  
//  
// Returns the amount of memory used by the application (does not include  
// reserved memory)  
//------------------------------------------------------------------------------  
function CsiGetApplicationMemory: Int64;  
var  
  lMemoryState: TMemoryManagerState;  
  lIndex: Integer;  
begin  
  Result := 0;  

  // get the state  
  GetMemoryManagerState(lMemoryState);  

  with lMemoryState do begin  
    // small blocks  
    for lIndex := Low(SmallBlockTypeStates) to High(SmallBlockTypeStates) do  
      Inc(Result,  
          SmallBlockTypeStates[lIndex].AllocatedBlockCount *  
          SmallBlockTypeStates[lIndex].UseableBlockSize);  

    // medium blocks  
    Inc(Result, TotalAllocatedMediumBlockSize);  

    // large blocks  
    Inc(Result, TotalAllocatedLargeBlockSize);  
  end;  
end;  

I log this on an interval (anywhere between 10 seconds and 10 minutes) to my log file, together with the difference from last time.

You can find out how much memory your application is using - see this About page. Summary:

uses PsAPI;

//current memory size of the current process in bytes
function CurrentMemoryUsage: Cardinal;
var
  pmc: TProcessMemoryCounters;
begin
  pmc.cb := SizeOf(pmc) ;
  if GetProcessMemoryInfo(GetCurrentProcess, @pmc, SizeOf(pmc)) then
    Result := pmc.WorkingSetSize
  else
    RaiseLastOSError;
end;
ShowMessage(FormatFloat('Memory used: ,.# K', CurrentMemoryUsage / 1024)) ;

If you log that value periodically in your server you'll at least get an idea of what is happening. There is more info in that result that should help you learn more about what your program is doing.

The fix will be to look at what is actually using the memory and manage that more aggressively. I suspect there will be somewhere that you're creating objects and only freeing them on shutdown, when you can (and should) free them as soon as you're finished with them.

One possible fix is to use the /3GB switch on the full version of FastMM and see if the problem takes longer to occur.

If you are spectacularly unlucky you will have "broken" FastMM's memory pool management algorithm so it never releases memory (a related question). Trying different memory managers might help you in that some are more aggressive about reclaiming unused memory. But if you're fragmenting your heap the only real solution is to work out how to avoid doing that. Which is a complex topic, so again: try the simple things above first.

Can you show us a stack trace when you get the error? How much memory does it consume at the time of the error?

I made a memory logger (for FastMM) some time ago which logs all memory between 2 points (using a "startlog" and "endlog" procedure) to find a "soft leak": I was allocation objects in a list but never clearing the list, only when closing the app, so FastMM reported no leak. By using my memory logger I could find those objects (it only logs new allocated memory that is not released before you execute the "endlog" procedure).
I will see if I can find this code.

Btw: you can get an "out of memory" in 3 other ways:

• FastMM gives this error when it detects an error while allocating. So not a real "out of memory" but more an internal fastmm error (due to corruption etc)
• Alloc a very big block (e.g. 1Gb). I once got this because of a stream read error (RemObjects) so it read a wrong size value for a string, so it tried to pre-allocate a (random) big string. Such an error looks weird because in my case my app had allocated about 150Mb so also no real "out of memory"
• Fragmentation: if you try to alloc a block of 10Mb but Windows cannot find one contineous block of 10Mb then Windows will give an "out of memory".

So please give a stack trace and amount of memory used at the time of the error!

With Delphi's limitation to 32 bit address space, such problems are becoming more common.

The first and simplest thing you can do is to run on a 64 bit OS and move from 2GB available address space (as you get on a 32 bit OS) to 4GB address. This doesn't happen automatically. You need to mark your application as being LARGEADDRESSAWARE. Do this by adding the following to your .dpr file:

const
  IMAGE_FILE_LARGE_ADDRESS_AWARE = $0020;

{$SetPEFlags IMAGE_FILE_LARGE_ADDRESS_AWARE}

The other common reason for out of memory errors is not that there is a shorage of memory, but that you are asking for a large block of contiguous memory and there is no single contiguous block of address space available.

Dealing with this problem is more difficult. You first need to identify the parts of your code that are currently demanding large contiguous blocks of memory. Next you have to modify the objects that are doing so and arrange that they instead ask for small chunks of memory which you can then "stitch together" to give the appearance of a larger block. This typically happens with code that uses dynamic arrays, in my experience.

When I have gotten an "Out of memory" error, it was due to a runaway loop. The loop typically would allocate memory and would not stop before all available memory had been used. Freeing memory was not an issue, because the program never go to that point. Types of code that have been bitten me are:

A “while not x.Eof do” loop without a “x.Next” to advance through the dataset, or

a recursive procedure or subroutine that never encountered the exit condition(s).

I would look for any kind of loop or recursion that could continue on “forever” under certain circumstances, such as building a data structure in memory that is massive.