[dm-crypt] Bad performance with software RAID5 and LUKS encryption
arno at wagner.name
Tue Feb 5 22:16:39 CET 2013
thanks, if that explains it reasonably well, I will just
paraphrase. But first I have to understand it myself ;-)
On Tue, Feb 05, 2013 at 10:09:36PM +0100, Philipp Wendler wrote:
> Am 05.02.2013 21:45, schrieb Arno Wagner:
> > thanks for the info. I think I may add a "performance"
> > section to the FAQ, and this could be one of the items.
> > Do you have the reference where you found the info on the
> > stripe_cache_size?
> Actually, I first read about the concept of a stripe cache in this btrfs
> announcement post:
> Then I googled it and found some usages:
> Unfortunately, I didn't find a real citable source.
> The kernel documentation (Documentation/md.txt) also only contains 3
> lines saying nothing about what it actually is.
> Greetings, Philipp
> > On Tue, Feb 05, 2013 at 09:02:21PM +0100, Philipp Wendler wrote:
> >> Hello,
> >> Am 05.07.2011 08:45, schrieb Philipp Wendler:
> >>> I have set up a Linux software RAID5 on three hard drives and want to
> >>> encrypt it with cryptsetup/LUKS. My tests showed that the encryption
> >>> leads to a massive performance decrease that I cannot explain.
> >>> The RAID5 is able to write 187 MB/s  without encryption. With
> >>> encryption on top of it, write speed is down to about 40 MB/s.
> >> Sorry for answering to a mail in this very old thread,
> >> but it seems I finally found a solution,
> >> and I know that there are some other people interested in the solution
> >> as well (so if you got this email directly from me, I BCC'ed you because
> >> you contacted me about this).
> >> If you want to read up the full story, here's the link:
> >> http://www.saout.de/pipermail/dm-crypt/2011-July/001773.html
> >> Today I read about the stripe_cache_size setting of md RAID, and tried
> >> it out. With the default value of 256, the performance is slow as
> >> described. With a value of 4096, I get a performance increase from about
> >> 40-50 MB/s to 123 MB/s. For values >= 8192, I get 140 MB/s out of it.
> >> Background: The stripe cache stores recently written blocks. If data is
> >> written continuously, it might happen that during a first write only a
> >> part of one stripe is written. This means, the RAID code has to read the
> >> complete stripe from disk, update it, and write it completely again. If
> >> a second write comes in for another part of the same stripe, all this
> >> would have to be done again. Now, if the cache is used and still
> >> contains the data written by the first write, the read that was
> >> necessary before the second write can be omitted.
> >> Now it seems that dm-crypt always writes with small block size to the
> >> underlying disk, even when I write with a big block size.
> >> Could this be true?
> >> Could this perhaps be improved? While I have found a solution for me,
> >> this could probably solve performance problems for many people.
> >> Furthermore, dm-crypt write is still slower than unencrypted write,
> >> although for reads the performance of encrypted and unencrypted are the
> >> same. So I guess the small block size still has a performance penalty
> >> (probably when first writing to a stripe and it is not yet in cache).
> >> My current setup is Ubuntu 12.04 (Linux 3.2).
> >> Nothing else has changed compared to when I first asked about this.
> >> Greetings, Philipp
> >> _______________________________________________
> >> dm-crypt mailing list
> >> dm-crypt at saout.de
> >> http://www.saout.de/mailman/listinfo/dm-crypt
> dm-crypt mailing list
> dm-crypt at saout.de
Arno Wagner, Dr. sc. techn., Dipl. Inform., Email: arno at wagner.name
GnuPG: ID: CB5D9718 FP: 12D6 C03B 1B30 33BB 13CF B774 E35C 5FA1 CB5D 9718
One of the painful things about our time is that those who feel certainty
are stupid, and those with any imagination and understanding are filled
with doubt and indecision. -- Bertrand Russell
More information about the dm-crypt