[dm-crypt] Bad performance with software RAID5 and LUKS encryption

Philipp Wendler ml at philippwendler.de
Tue Feb 5 22:09:36 CET 2013


Hi,

Am 05.02.2013 21:45, schrieb Arno Wagner:
> thanks for the info. I think I may add a "performance"
> section to the FAQ, and this could be one of the items.
> Do you have the reference where you found the info on the
> stripe_cache_size?

Actually, I first read about the concept of a stripe cache in this btrfs
announcement post:
http://article.gmane.org/gmane.comp.file-systems.btrfs/23006
Then I googled it and found some usages:
http://h3x.no/2011/07/09/tuning-ubuntu-mdadm-raid56
http://ubuntuforums.org/showthread.php?t=1494846

Unfortunately, I didn't find a real citable source.
The kernel documentation (Documentation/md.txt) also only contains 3
lines saying nothing about what it actually is.

Greetings, Philipp


> On Tue, Feb 05, 2013 at 09:02:21PM +0100, Philipp Wendler wrote:
>> Hello,
>>
>> Am 05.07.2011 08:45, schrieb Philipp Wendler:
>>> I have set up a Linux software RAID5 on three hard drives and want to
>>> encrypt it with cryptsetup/LUKS. My tests showed that the encryption
>>> leads to a massive performance decrease that I cannot explain.
>>>
>>> The RAID5 is able to write 187 MB/s [1] without encryption. With
>>> encryption on top of it, write speed is down to about 40 MB/s.
>>
>> Sorry for answering to a mail in this very old thread,
>> but it seems I finally found a solution,
>> and I know that there are some other people interested in the solution
>> as well (so if you got this email directly from me, I BCC'ed you because
>> you contacted me about this).
>> If you want to read up the full story, here's the link:
>> http://www.saout.de/pipermail/dm-crypt/2011-July/001773.html
>>
>> Today I read about the stripe_cache_size setting of md RAID, and tried
>> it out. With the default value of 256, the performance is slow as
>> described. With a value of 4096, I get a performance increase from about
>> 40-50 MB/s to 123 MB/s. For values >= 8192, I get 140 MB/s out of it.
>>
>> Background: The stripe cache stores recently written blocks. If data is
>> written continuously, it might happen that during a first write only a
>> part of one stripe is written. This means, the RAID code has to read the
>> complete stripe from disk, update it, and write it completely again. If
>> a second write comes in for another part of the same stripe, all this
>> would have to be done again. Now, if the cache is used and still
>> contains the data written by the first write, the read that was
>> necessary before the second write can be omitted.
>>
>>
>> Now it seems that dm-crypt always writes with small block size to the
>> underlying disk, even when I write with a big block size.
>> Could this be true?
>> Could this perhaps be improved? While I have found a solution for me,
>> this could probably solve performance problems for many people.
>>
>> Furthermore, dm-crypt write is still slower than unencrypted write,
>> although for reads the performance of encrypted and unencrypted are the
>> same. So I guess the small block size still has a performance penalty
>> (probably when first writing to a stripe and it is not yet in cache).
>>
>> My current setup is Ubuntu 12.04 (Linux 3.2).
>> Nothing else has changed compared to when I first asked about this.
>>
>> Greetings, Philipp
>> _______________________________________________
>> dm-crypt mailing list
>> dm-crypt at saout.de
>> http://www.saout.de/mailman/listinfo/dm-crypt
> 



More information about the dm-crypt mailing list