How to stop the flushing of mmaped pages to disk in RHEL5.2 and later?

If your application is mmapping too much memory, resulting in lots of pdflush write activity and thereby by affecting system performance, vm.flush_mmap_pages kernel parameter could be useful.

/proc/sys/vm/flush_mmap_pages is introduced in RHEL5.2 kernel.

Setting /proc/sys/vm/flush_mmap_pages to 0 will stop the flushing of dirty memory-mapped file pages to disk as long as the memory map is active.

All dirty file pages will be asynchronously flushed to disk only as soon as the memory map is deactivated.

As per RHEL5.2 release notes.. (http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5.2/html/Release_Notes/x86/ar01s05.html)

Added a new kernel parameter: /proc/sys/vm/flush_mmap_pages. This parameter specifies whether or not memory-mapped file pages should be flushed to disk by kupdate while the memory map is active. Valid values for this parameter are 1 (enable memory mapping by kupdate) and 0 (disable memory mapping by kupdate). The default value for this parameter is 1.

To configure this parameter, use echo [1 or 0] /proc/sys/vm/flush_mmap_pages. Setting this parameter to 0 does the following:

kupdate will not flush dirty memory-mapped file pages as long as the memory map is active.

All dirty file pages will be asynchronously flushed to disk only as soon as the memory map is deactivated.

If you set /proc/sys/vm/flush_mmap_pages to 0, it is advisable that you use another application to manually sync memory-mapped pages to disk.

High memory utilization and cache

Many people complains about high memory utilization in the output of free command, which in most situations caused by misreading of free command.

If we are taking following free output for instance,

total used free shared buffers cached
Mem: 8054896 7193712 861184 0 188348 5286732
-/+ buffers/cache: 1718632 6336264
Swap: 2096440 144 2096296

It is seen that most of the memory in the system is used by 'cached' (5286732kb) and buffers (188348kb). The system over time increases the size of the buffers and cache in its RAM. The cache stores various pieces of data from the hard disk such as data from files. When reading from the hard disk, the kernel first looks into cache for that data. This is more efficient, as reading from memory is much faster than reading from the disk. This memory is returned whenever it is required by other programs and should be calculated as being free along with the buffers.

This is an expected behaviour by Linux kernel to speed up the system.

Memory utilization need not be an issue unless there is an Out of Memory or a performance issue.

--

In RHEL5, If cache memory need to be flushed /proc/sys/vm/drop_caches could be used.To free pagecache:

# echo 1 > /proc/sys/vm/drop_caches

To free dentries and inodes:

# echo 2 > /proc/sys/vm/drop_caches

To free pagecache, dentries and inodes:

# echo 3 > /proc/sys/vm/drop_caches

This could cause a hike in I/O in some situations, as kernel will try to write all data to disk.

"sync" command should be run before the above commands.

In RHEL4 /proc/sys/vm/drop_caches is introduced in 2.6.9-67* kernel.

There is also a known bug in which the system hangs when cache is flushed using drop_caches. The bug is only reproduced in large systems with at least 64GB RAM and 8 cpus. The bug is fixed in 2.6.9-78* kernel. Please refer following bugzilla.

https://bugzilla.redhat.com/show_bug.cgi?id=449381

--

Cached memory is good for most of the work loads.

If you want to specifically opt for less cache usage, increasing the value of /proc/sys/vm/swappiness could help.

For Example:

# echo 100 > /proc/sys/vm/swappiness

You can make the values persistent over reboot by adding following to /etc/sysctl.conf.

vm.swappiness=

Please remember that this will increase the usage of swap, which might affect system performance.

--

Following are some other kernel tuning parameters that helps to manage cache.

vm.dirty_expire_centisecs=2000

It determines the amount of time for a page to be considered old enough for flushing to disk via the pdflush daemon. Expressed in 100'ths of a second.

vm.dirty_writeback_centisecs=400

It determines the interval at which the pdflush daemon wakes up to write dirty data to disk.. Expressed in 100'ths of a second.

vm.dirty_background_ratio=5

It determines the percentage of total system memory at which the pdflush background writeback daemon will start writing out dirty data.

vm.dirty_ratio=20

It determines percentage of total system memory at which a process which is generating disk writes will itself start writing out dirty
data.