application running slow after using large memory allocation

I am involved in cross platform software development over embedded Linux environment. I see that my application runs very slow once it starts using 30+ MB of memory.

I have followed开发者_JAVA技巧 two different approach but both end up in same results.

Approach 1

Allocate using valloc (requires aligned memory), I kept a count of memory allocated after reaching 30MB the application goes slow.

Approach 2

Allocate large amount of memory 40MB at initialization. Further allocations are done from this partition (never freed though out the program execution). Again the application goes slow once 30+ MB is used. However anything less than 30 MB application runs good.

PS : I could use this approach as uniform blocks of memory were allocated. Total available memory is 128MB.

I am wondering why my application slows down upon accessing the memory, even though allocation was successful.

# more /proc/meminfo
MemTotal:        92468 kB
MemFree:         50812 kB
Buffers:             0 kB
Cached:          22684 kB
SwapCached:          0 kB
Active:           4716 kB
Inactive:        18540 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:        92468 kB
LowFree:         50812 kB
SwapTotal:           0 kB
SwapFree:            0 kB
Dirty:               0 kB
Writeback:           0 kB
AnonPages:         600 kB
Mapped:            952 kB
Slab:             7256 kB
PageTables:        108 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:     46232 kB
Committed_AS:     1736 kB
VmallocTotal:   262136 kB
VmallocUsed:     10928 kB
VmallocChunk:   246812 kB

Total available memory is 128MB

How sure are you of that number? If it's the total system memory, some amount will be used up by the kernel, drivers, and other processes. It could be that after allocating 30MB of memory, you start swapping to disk. That would certainly explain a sudden slowness.

I am wondering why my application slows down upon accessing the memory, even though allocation was successful.

Possibly because Linux won't actually back the virtual allocation with physical pages until you write to them. So although your malloc() succeeds, it's only really updating the page tables until such time as you actually use that memory.

Looking at your meminfo dump above, I think you have around 50MB free on the system:

MemFree: 50812 kB

Furthermore, around 22MB are in use as cache:

Cached: 22684 kB

I wonder whether your app using over 30MB of memory might push the kernel's VM to the point that it decides to start freeing up cached data. If that should happen you might expect a slowdown if e.g. filesystem buffers that you might have been using get flushed from the cache.

I note you don't have swap enabled (SwapTotal is 0kB). If you did, your app could be causing the VM to thrash.

If I were you trying to debug this, I would try running "top" as my app hit the 30MB memory use point and see whether any kernel threads are suddenly getting busier. I would also use "vmstat" to track system i/o and memory cache/buffer allocations. Finally, I'd try having a bit of a poke around the /proc filesytem to see if you can glean anything there (for example, /proc/sys/vm might be worth looking at).