[dm-crypt] aes-xts-plain with aes_x86_64 makes my SSD 5x slower than my encrypted HD
marc at merlins.org
Mon Jul 23 19:51:29 CEST 2012
On Mon, Jul 23, 2012 at 07:15:24PM +0200, Milan Broz wrote:
> > Mmmh, I have one possible thing. I have a preempt kernel. Could that be it?
> > http://marc.merlins.org/tmp/config-3.4.4-amd64-preempt.txt
> Can you send me your kernel .config then? Preempt should not be problem.
> Which kernel version? which architecture? Any additional patches over
> mainline code?
I just sent you my config, it was in the URL above :)
No patches, kernel 3.4.4 from kernel.org, see above.
I tried without preempt and with volprempt and duplicated the same slow
> > Disk /dev/sda: 512.1 GB, 512110190592 bytes
> > 255 heads, 63 sectors/track, 62260 cylinders, total 1000215216 sectors
> > Units = sectors of 1 * 512 = 512 bytes
> > Sector size (logical/physical): 512 bytes / 512 bytes
> > I/O size (minimum/optimal): 512 bytes / 512 bytes
> > Disk identifier: 0x09aaf50a
> > Device Boot Start End Blocks Id System
> > /dev/sda1 * 2048 502047 250000 83 Linux
> > /dev/sda2 502048 52930847 26214400 83 Linux
> > /dev/sda3 52930848 73902367 10485760 82 Linux swap / Solaris
> > /dev/sda4 73902368 1000215215 463156424 83 Linux
> Hm. sda1 is apparently aligned. But I think
> other partitions are not aligned properly.
I used fdisk -H32 -S32 /dev/sda as recomended on
> No idea which block size/page size your SSD internaly use, but to be safe,
> let's assume ideal is 256KiB (= 512 * 512 byte sectors).
> 73902368 seems not to be multiple of 512...
> Well, your sda2 configuration is created with +256MB, which is SI unit,
> so not multiple of 1024 but 1000!
Oh my, I can't believe they changed that (been using linux since 1993 back
then +1MB in fdisk was a real MB).
> It should be created with +256M, so you end with:
> Device Boot Start End Blocks Id System
> /dev/sda1 2048 526335 262144 83 Linux
> /dev/sda2 526336 ...
> But I think this is not the core problem but you should try it to fix.
> (I hope I did not mixed up numbers above :)
It would be a problem for erase blocks if it is wrong, but not for reading,
especially when I can do filesystem reads at 260MB/s on the filesystem
created on the encrypted device.
But I'm confused, I thought
fdisk -H32 -S32
would protect me from mis-alignment and fdisk/util-linux 2.20.1 was smart
enough to use proper boundaries on its own on SSDs anyway.
> ... Thinking about it, we can test it without partition change (for reads).
> This test is perhaps useless but maybe someone can find it useful later.
> Let's map properly aligned device to sda4 (read only). Result will be just
> garbage, but it is just for speed testing.
> For mapping above, properly aligned (1MiB alignment = 2048 sectors)
> should start on sector 7390412. This is +1760 sector shift for sda4.
> Map 1GiB device there:
> # dmsetup create test_plain -r --table "0 2097152 linear /dev/sda4 1760"
> Map null cryptsetup over it
> # echo "password" | cryptsetup create -c null test_crypt /dev/mapper/test_plain
> Now repeat read dd test for /dev/mapper/test_plain and /dev/mapper/test_crypt
gandalfthegreat:~# dmsetup create test_plain -r --table "0 2097152 linear /dev/sda4 1760"
gandalfthegreat:~# echo "password" | cryptsetup create -c null test_crypt /dev/mapper/test_plain
gandalfthegreat:~# dmsetup status test_crypt
0 2097152 crypt
gandalfthegreat:~# cryptsetup status test_crypt
/dev/mapper/test_crypt is active.
keysize: 0 bits
offset: 0 sectors
size: 2097152 sectors
gandalfthegreat:~# hdparm -t -T /dev/mapper/test_plain
Timing cached reads: 13978 MB in 2.00 seconds = 6997.30 MB/sec
Timing buffered disk reads: 958 MB in 3.00 seconds = 319.00 MB/sec
gandalfthegreat:~# hdparm -t -T /dev/mapper/test_crypt
Timing cached reads: 15434 MB in 2.00 seconds = 7725.50 MB/sec
Timing buffered disk reads: 76 MB in 3.03 seconds = 25.05 MB/sec
> > I also used:
> > cryptsetup luksFormat --align-payload=8192
> It will not help if underlying partition is misaligned.
That's correct of course :)
Now I'm going to have to do some more reading to see whether I'm really
misaligned or not. I had done all the relevant reading on SSDs to make sure
I did use proper alignment.
I wonder why fdisk -H32 -S32 failed me.
> I think this one, try to increase it to 8192 or so...
> > nr_requests:128
gandalfthegreat:/sys/block/sda/queue# echo 8192 > nr_requests
gandalfthegreat:/sys/block/sda/queue# hdparm -t -T /dev/mapper/cryptroot
Timing cached reads: 14970 MB in 2.00 seconds = 7492.34 MB/sec
Timing buffered disk reads: 72 MB in 3.08 seconds = 23.35 MB/sec
Thanks for the other suggestions. Hopefully we'll nail this somehow :)
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
.... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/
More information about the dm-crypt