Memory Continues to Increase when Loading and Releasing NSImage

I have a problem where my application aggressively consumes memory to a "thrashing point" with successive image file loads. For example, consider the following code, which repeatedly loads and releases a 15MB JPEG file (large file size for test purposes):

NSURL *inputUrl = [NSURL URLWithString:@"file:///Users/me/Desktop/15MBjpeg.jpg"];  
for(int i=0; i<1000; i++) {  
    NSImage *image = [[NSImage alloc] initWithContentsOfURL:inputUrl];  
    [image release];  
}

It performs quickly for the first several since there is plenty of free system memory, but eventually the system comes to its knees at what I'm calling the "thrashing point." Here, I believe the system relieves just enough memory to load the next image, so performance ends up being slow. Additionally, now other applications run slowly because the system has to free up this hogged, but now unused, memory.

What would make sense to me is if it would allocate the memory and then have the system free it so that my "Real Mem" statistic in Activity Monitor stays small rather than heading into the gigabytes with successive load/release iterations. In practice, I may never end up at this point, but it seems odd that my Activity Monitor "Real Mem" statistic eventually exceeds all other applications when the actual resident memory required at any time is generally small. I originally thought this was some sort of memory leak or caching issue, but it seems more related to aggressive memory allocation in the application and lazy memory freeing on the system (not that there's anything wrong if the OS has that policy--if that is in fact the way it works). Perhaps I'm missing something altogether though.

Is there a better way to repeatedly load images without this hogging-and-not-proactively-relieving memory usage behavior? Perhaps there is a way to force the application memory footprint down, or there is a way to be smarter about how images are loaded into the same objects or memory locations? My goal is to load an image, process it (get a thumbnail, change the image format, etc), get rid of it in memory, and then do it again--all without this observed memory growth.

--

FOLLOW-UP:

Bavarious, thanks. A wrapping NSAutoreleasePool does resolve the iterative memory growth when loading the same file:

NSURL *inputUrl = [NSURL URLWithString:@"file:///Users/me/Desktop/15MBjpeg.jpg"];  
for(int i=0; i<1000; i++) {  
    NSAutoreleasePool *apool = [[NSAutoreleasePool alloc] init];
    NSImage *image = [[NSImage alloc] initWithContentsOfURL:inputUrl];  
    [image release];
    [apool drain];
}

However, it does not solve the issue where the memory stays increased after the image is released (and the NSAutoreleasePool drained). For example, when loading my 15MB JPEG image, the "Real Mem" memory jumps from the 8MB steady state to about 25MB and then stays there. (My application only has an Interface Builder button that has a wired IBAction to a method that invokes only the for loop I copied). I would expect that after the for loop finishes (or if only one image is loaded and released) that the "Real Mem" statistic would decrease back to the nominal application level memory usage.

It seems reasonable that other things in the background can be loaded as well when invoking NSImage functionality, which can increase the memory. However, different sized images (15MB, 30MB, 50MB, etc) increase memory proportionally in the application, which leads me to believe it's more than such allocation.

Further, if I try to load separate images in succession (say, 15MBjpeg-1.jpg, 15MBjpeg-2.jpg, etc.) the memory sometimes compounds for each new image loaded. For example, if two images are loaded in succession, the "Real Mem" memory usage for the application after the loading/releasing is now roughly 50MB, and it never decreases--based on my observation. This behavior continues when loading subsequent images so that the application can creep into hundreds of MB of "Real Mem" memory usage after loading several large images.

Interestingly, if I reload the same image over and over again, the steady state memory does not increase. Does this indicate some sort of caching that's going on? Again, my goal is to batch process through several different image files without this growth in memory. Thanks in advance.

Oh, and I'm looking into the Heapshot Analysis article ATM, but wanted at least to post my progress and see if there's other input.

--

FOLLOW-UP #2

bbum, thanks for the great article. I ran Instruments Allocations with my test program and did not find any Heap Growth. As on the referenced blog post, my approach was to, 1) click my "Load and Release Image" button on my Interface Builder interface (which invokes the load/release behavior), 2) click "Mark Heap" several times every few seconds in Instruments Allocations, and then 3) repeat 1) and 2).

Using this approach, the Heapshots consistently reported 0 Bytes in the Heap Growth column over time (3 clicks every 5 sec for 15 seconds), meaning that there is no additional memory allocated from the baseline Heapshot. Additionally, in the Statistics pane, there is a Malloc of 13.25MB each time I click the "Load and Release Image" button, but Live Bytes is 0 Bytes, meaning that it has been fully released. If I click the "Load and Release Image" button three times, the Overall Bytes for the image reports 39.75MB (3 * 13.25MB), meaning that 39.75MB was allocated, but fully released since the Live Bytes is 0. The Allocations graph spikes up quickly and comes right back down, since it's a pretty quick operation. It all seems to make sense that there is no leak and no growth in the steady state use of memory.

But, now what do I do that my "Real Mem" statistic is still high? I know that Activity Monitor is not the standard for debugging memory issues. But, "Real Mem" still stays high until I close the program, and then the "Real Mem" all goes back into the "Free" category, which is bizarre to me.

I tested this same approach with two images (15MBjpeg-1.jpg, 15MBjpeg-2.jpg) by just duplicating the code in the same method, and I observe no Heap Growth again. Obviously more allocations were been made and released. However, now the "Real Mem" sits at roughly twice the increase as in the case of loading and releasing only one image. And again, it doesn't decrease in the test program's steady state.

Is there anything further I can do? Here is the test code for a single image load/release for anyone who wants to give it a try (just wire an IB button to openFiles):

#import "TestMemory.h" // only declares -(IBAction) openFiles:(id)sender;
@implementation TestMemory
-(IBAction) openFiles:(id)sender {
    NSURL *inputUrl = [NSURL URLWithString:@"file:///Users/me/Desktop/15MBjpeg.jpg"];
    NSAutoreleasePool *apool = [[NSAutoreleasePool alloc] init];
    NSImage *image = [[NSImage alloc] initWithContentsOfURL:inputUrl];
    [image release];
    [apool drain];
}
@end

Thanks for reading. :)

--

FOLLOW-UP #3

bbum, no I don't have garbage collection enabled (GC is set to Unsupported in the project settings). I investigated the memory using vmmap and the VM Tracker bar in Instruments Allocations. I assumed you meant the VM Tracker data when you said VM Instruments, since it reports the same information as vmmap. After opening a single image using my "Load and Release Image" button, some significant memory numbers include the following (from VM Tracker):

Type         %ofRes  ResSize  VirtSize Res%  %AllDirty  DirtySize

__TEXT       38%     33.84MB  80.45MB  42%   0%         0Bytes

*Dirty*     32%     28.23MB  114.99MB 24%   100%       17.11MB

MALLOC_LARGE 14%     13.25MB  13.25MB  100%  0%         4KB

Carbon       11%     9.86MB   9.86MB   100%  20%        3.46MB

VM_ALLOCATE  9%      8.43MB   48.17MB  18%   49%        8.43MB

...

Interestingly, subsequent load/releases of a single image increase the "Resident Size" of the Dirty and VM_ALLOCATE types by ~.3MB, and the "Dirty Size" of these types also increases over time. (VM_ALLOCATE seems to be a subset of Dirty). No other types appear to change with subsequent load/releases.

I am unsure exactly what to take away from this data, or how I can use it to make the program release the memory. It seems like 开发者_运维技巧the VM_ALLOCATE type may be the chunk that is not being freed, but that's only speculation. Is it possible that the some portion of the underlying NSImage init implementation saves a cache of the image file and won't release it? Again, as stated earlier, it intrigues me that each subsequent load/release of the same image file hardly consumes resources (CPU, disk grinding, etc) and wall clock time compared to the first load/release. Thanks in advance.

• What Bavarious said; have you tried surrounding it with an NSAutoreleasePool.

• That is a classic micro-benchmark. While it certainly indicates a problem, the problem may actually be that the benchmark is divergent from expected real world patterns so much that the bug lies with the benchmark.

• In an efficiently designed application, it would not read the same image data from disk more than once.

• This is a prime candidate for Heapshot analysis.

(Thanks for the follow-ups; helpful!)

The symptoms you describe sound like either a VM leak (something is consuming addresses without doing allocations; mapped memory, for example) or a cache that isn't being pruned (that contains VM allocations).

• do you have GC enabled? If so, this could easily be because the GC threshold isn't triggered. The collector doesn't know to credit Really Large Allocations from non-GC to the GC zone. If you force collections, it'll work around this particular edge case.

• Try having a look at the VM instrument or use vm_map at the command line to have a look at what is consuming the address space within your app.