[dm-crypt] dm-crypt on top of DRBD for live migration
arno at wagner.name
Mon Dec 12 15:11:38 CET 2011
I see. If you place dm-crypt above DRBD, then you will have
the same master key on both machines, as the same LUKS header
will be on both machines. There is nothing random here. You
will create the LUKS header on one machine, and just use it
on the other one.
As to the potential problems, you get the same from the
buffer-cache and filesystem. Hence a situation where M1 and M2
both write to disk already breaks a number of things, because
the filesystm and the buffer-cache both depend on having
exclusive access to the underlying block device. If you
add plain dm-crypt or LUKS as encryption, it just has the
If you look at the picture on the DRBD homepage, then you see that
it is somewhat misleading. The mirroring can always be active,
but only one service layer, filesystem and buffer-cache can.
And before you can migrate the service layer, you need to
teardown the filesystem and buffer-cache on the first machine,
by unmounting the device. Only then can you mount the device
on the second machine. And only after that can you start the
service layer on the second machine.
The exception here is if you have a proper cluster-filesystem.
These can have filesystem instances and buffer-cache instances
on several machines accessing the same block device. Here,
you must place encryption below DRBD on each machine, as
dm-crypt is not capable of supporting multiple copies of
itself accessing the same block device concurrently. It
may even work most of the time, but especially in LUKS
header management could fail, potentially destroying the
This (dm-crypt below DRBD) produces indeed two different
raw data sets on both machines, but DRBD only sees the
decrypted versions through dm-crypt and hence no problems
Hope this clears matters up a bit.
On Mon, Dec 12, 2011 at 09:51:21AM +0100, Berengar Lehr wrote:
> > On the migration issue, I do not understand the question.
> > What is your concern here?
> > Arno
> Hy Arno,
> thank you for your thoughts. Our concern about live migration from one
> machine to another is that using dm-crypt on two different machines
> might produce two different disc values even if to master keys are used.
> If both machines (they should be running in some kind of parallel mode
> during live migration) execute the same write-command writing the same
> data would result in the very same value on disc. Using DRBD would hence
> not result in write/read problems.
> We do not fully understand the way dm-crypt/LUKS works so we considered
> the following situation:
> Machine M1 is writing to decrypted disk DV1, dm-crypt is writing this
> data encrypted to PV1. This is done by generating a random key (RK1)
> used for encrypting the data D itself which is again encrypted by the
> master key (MK) and written to a special location on the disc (probably
> the first bytes of the sector).
> Now M2 was parallel writing the same data D to that sector before, using
> its own RK2. Now even if both disks have the same MK (which under any
> circumstances should be the case if we would be using the second setup).
> But due to the difference RK2 and RK1 both machines would write
> different data to disk and hence might run into a problem when DRBD
> synchronizes the data.
> This was the scenario we thought about (not knowing if dm-crypt/LUKS is
> using such randomized sector keys) but there might be other problems
> using dm-crypt outside the VM but above DRBD. Hope that helps to
> understand our considerations.
> Thank You
> B. Lehr & M. M?ller
> dm-crypt mailing list
> dm-crypt at saout.de
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno at wagner.name
GnuPG: ID: 1E25338F FP: 0C30 5782 9D93 F785 E79C 0296 797F 6B50 1E25 338F
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans
If it's in the news, don't worry about it. The very definition of
"news" is "something that hardly ever happens." -- Bruce Schneier
More information about the dm-crypt