[dm-crypt] md + dm-crypt + lvm2: I/O slows down system, I/O schedule ineffective

Günter Merz lotan_rm at hotmail.com
Thu Feb 23 11:19:52 CET 2012


Hi,

at the moment, I've got the following I/O stack on my system:

filesystems (ext4 and swap)
lvm2
dm-crypt
md raid 5
physical disks

other than that, I'm using a quadcore Athlon II X4 640 and 8 GB RAM.
I'm using a vanilla kernel (I/O scheduler: CFS).

I'm experiencing unresponsiveness and a general system slowdown when I'm unpacking big archives or building a big packages (during builds I'm only using max 2 cores). To counter this, I've been experimenting with nice and ionice. Unfortunately, ionice -c3 before big I/O traffic doesn't seem to make any difference. The responsiveness is still really bad at times.

I've been googling in reading in forums quite a lot and there seems to be an longstanding issue in the linux kernel (https://bugzilla.kernel.org/show_bug.cgi?id=12309) that hasn't been resolved for years.

Huge discussion threads seem to deal with this issue:

http://forums.gentoo.org/viewtopic-t-793263-start-100-postdays-0-postorder-asc-highlight-.html?sid=175e371c81170f23e7116626a6b26755
https://bbs.archlinux.org/viewtopic.php?id=93960

Some are suggesting to use different schedulers (BFQ, BFS).
Others are tinkering with the kernel I/O parameters.

Most of these counter measures seem to have little or no effect to others in follow-ups postings.

At this point I could have said: "Linux doesn't handle I/O well" and grudgingly live with it.

However, a different post caught my attention:

http://www.saout.de/pipermail/dm-crypt/2009-October/000329.html

Christian Pernegger did some tests in which he changed the vfs layers.

My layer (dm-crypt on top of md) came out worst in his tests.
dm-crypt directly on top of the physical disk seemed to be the best option.

Other people suggested that using dm-crypt renders the I/O scheduler of the kernel useless.

Since Christian Pernegger's post in 2009, some things have changed:
dm-crypt has become multi-threaded
the linux kernel has changed (VFS, CFS)

So my questions are:
- does the position of dm-crypt in the vfs layer affect I/O performance significantly nowadays?
- is there an explanation for that?
- can anyone suggest a diagnosis on my system that would prove or disprove that dm-crypt is in the wrong position in the vfs stack before I start reordering it on my system
 		 	   		  


More information about the dm-crypt mailing list