[dm-crypt] LUKS in failover cluster

Arno Wagner arno at wagner.name
Wed Oct 12 05:13:29 CEST 2011

On Tue, Oct 11, 2011 at 04:34:21PM -0700, Sohl, Jacob (LNG-SEA) wrote:
> > You can map ("decrypt") the devices and never use them. The
> > LVM/mount/whatever is completely optional.
> > 
> > In fact the mapper tool (cryptsetup) does nothing except
> > to decrypt the raw encrypted device to a raw decrypted device
> > under /dev/mapper/.
> > 
> Ok, thank you. That seemed logical, but I just recently started working
> with encryption so I'm still learning.

That is fine. Making sure now will safe you grief later.
> > 
> > > Also, scalability is a requirement in my design, hence XFS. I was
> > > thinking I needed to use multiple LUKS PVs in LVM to grow the
> > > filesystem. But I would end up with multiple LUKS devices to keep
> > track
> > > of. I recently found out that LUKS can resize.
> > 
> > LUKS _cannot_ resize. The thing is that LUKS does not care about
> > device size. So if you enlarge a device/partition with an intact
> > LUKS header, "cryptsetup luksOpen" will just map the larger
> > device.
> > 
> > If you have multiple devices, you can "slave" the additional ones
> > to a "master" device using something like "decrypt_derived".
> > This just takes the master key from the opened "master" container,
> > runns it though a hash and uses this as key for the "slave"
> > device.
> > 
> > > Would it be better to
> > > create one LUKS device on top of LVM? Then create a filesystem on
> > that?
> > > (Though that would affect resource dependencies.)
> > 
> > Sre you sure you need LVM at all?
> Yes, I do need LVM. I'm dealing with ~50TB of data and the SAN admins
> can/will only attach storage in 4TB increments. So I have to combine all
> of the LUNS into one Logical Volume in order to create a 50TB+ XFS
> filesystem. That's why I ask about encrypting multiple physical LUNS vs
> a single Logical Volume.

50TB? Ok, that is a bit larger. I did have such a dataset
for research a few years back on tape. Took several days 
days to get 15TB of it to disk for my largest measurement.

Well, yes, I think LVM is the best way here.

> > 
> > > But basically:
> > >
> > > SAN LUNs > LVM > LUKS > XFS > Samba Server
> > >
> > >
> > >
> > > Other people will be accessing/managing this system, so I want
> > > manageability through simplicity.
> > 
> > Hence the question whether you actually need LVM.
> > It strikes me that typically LVM is not needed and
> > just complicates matters.
> > 
> > > Don't want to have the wrong volumes
> > > (re)encrypted, headers damaged, etc.
> > 
> > I think this setup is already too complicated for most people
> > to manage. A full backup should be part of your plans.
> > Resize with backup is actually easier, as you can just
> > backup, kill everything and do a clean new config.
> > 
> I will be making full backups with xfsdump. But it doesn't seem
> practical to be moving 50TB around. Both LVM and XFS allow for designed
> for online resize. I was planning to "pvcreate" an encrypted LUN then
> expand the Logical Volume and Filesystem.

For that volume, run a full compare as well. I found that
with good hardware, and an IBM Tape-Library I still had 
something like 1 unexplained bit-error every 10TB or so
when pulling the data from the tape-library again. That was 
a few years back, but still.

> > > Anyways, thanks for your help.
> > >
> > 
> > Just to make sure: You _have_ read the FAQ?
> Yes I have read the FAQ. 


> I guess I am just a little nervous about this project. 


> This is my first time working with encryption and 50TB of
> critical client data is at stake. Just being thorough and making sure I
> fully understand everything.

And that is all you can do. I think you are approaching this
perfectly right. I take it the 50TB are alredy on XFS+LVM
at this time and you are just adding the encryption layer?

Make sure to use XTS mode or plain64 with ESSIV for these
volume sizes.

And let us know what your expeiences are!

Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno at wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier 

More information about the dm-crypt mailing list