[dm-crypt] LUKS in failover cluster

Scott McCarty smccarty at redhat.com
Wed Oct 12 21:54:57 CEST 2011


That was a good laugh, maybe a chuckle even :-)

Scott McCarty, RHCE, RHCSA, LPIC-1
Solutions Architect
Email: smccarty at redhat.com
Phone: 312-660-3535
Web: http://people.redhat.com/smccarty


----- Original Message -----
From: "Jacob Sohl (LNG-SEA)" <jacob.sohl at applieddiscovery.com>
To: "Arno Wagner" <arno at wagner.name>, dm-crypt at saout.de
Cc: "LNG-SEA Systems Engineers ADI" <LNG-SEASystemsEngineersADI at lexisnexis.com>, "Scott McCarty" <smccarty at redhat.com>
Sent: Wednesday, October 12, 2011 3:20:51 PM
Subject: RE: [dm-crypt] LUKS in failover cluster



> -----Original Message-----
> From: dm-crypt-bounces at saout.de [mailto:dm-crypt-bounces at saout.de] On
> Behalf Of Arno Wagner
> Sent: Tuesday, October 11, 2011 8:13 PM
> To: dm-crypt at saout.de
> Subject: Re: [dm-crypt] LUKS in failover cluster
> 
> On Tue, Oct 11, 2011 at 04:34:21PM -0700, Sohl, Jacob (LNG-SEA) wrote:
> [...]
> > > You can map ("decrypt") the devices and never use them. The
> > > LVM/mount/whatever is completely optional.
> > >
> > > In fact the mapper tool (cryptsetup) does nothing except
> > > to decrypt the raw encrypted device to a raw decrypted device
> > > under /dev/mapper/.
> > >
> >
> > Ok, thank you. That seemed logical, but I just recently started
> working
> > with encryption so I'm still learning.
> 
> That is fine. Making sure now will safe you grief later.
> 
> > >
> > > > Also, scalability is a requirement in my design, hence XFS. I
was
> > > > thinking I needed to use multiple LUKS PVs in LVM to grow the
> > > > filesystem. But I would end up with multiple LUKS devices to
keep
> > > track
> > > > of. I recently found out that LUKS can resize.
> > >
> > > LUKS _cannot_ resize. The thing is that LUKS does not care about
> > > device size. So if you enlarge a device/partition with an intact
> > > LUKS header, "cryptsetup luksOpen" will just map the larger
> > > device.
> > >
> > > If you have multiple devices, you can "slave" the additional ones
> > > to a "master" device using something like "decrypt_derived".
> > > This just takes the master key from the opened "master" container,
> > > runns it though a hash and uses this as key for the "slave"
> > > device.
> > >
> > > > Would it be better to
> > > > create one LUKS device on top of LVM? Then create a filesystem
on
> > > that?
> > > > (Though that would affect resource dependencies.)
> > >
> > > Sre you sure you need LVM at all?
> >
> > Yes, I do need LVM. I'm dealing with ~50TB of data and the SAN
admins
> > can/will only attach storage in 4TB increments. So I have to combine
> all
> > of the LUNS into one Logical Volume in order to create a 50TB+ XFS
> > filesystem. That's why I ask about encrypting multiple physical LUNS
> vs
> > a single Logical Volume.
> 
> 50TB? Ok, that is a bit larger. I did have such a dataset
> for research a few years back on tape. Took several days
> days to get 15TB of it to disk for my largest measurement.
> 
> Well, yes, I think LVM is the best way here.
> 
> > >
> > > > But basically:
> > > >
> > > > SAN LUNs > LVM > LUKS > XFS > Samba Server
> > > >
> > > >
> > > >
> > > > Other people will be accessing/managing this system, so I want
> > > > manageability through simplicity.
> > >
> > > Hence the question whether you actually need LVM.
> > > It strikes me that typically LVM is not needed and
> > > just complicates matters.
> > >
> > > > Don't want to have the wrong volumes
> > > > (re)encrypted, headers damaged, etc.
> > >
> > > I think this setup is already too complicated for most people
> > > to manage. A full backup should be part of your plans.
> > > Resize with backup is actually easier, as you can just
> > > backup, kill everything and do a clean new config.
> > >
> >
> > I will be making full backups with xfsdump. But it doesn't seem
> > practical to be moving 50TB around. Both LVM and XFS allow for
> designed
> > for online resize. I was planning to "pvcreate" an encrypted LUN
then
> > expand the Logical Volume and Filesystem.
> 
> For that volume, run a full compare as well. I found that
> with good hardware, and an IBM Tape-Library I still had
> something like 1 unexplained bit-error every 10TB or so
> when pulling the data from the tape-library again. That was
> a few years back, but still.
> 

Thanks, I've never been very good at doing backups so any advice is
appreciated.

> > > > Anyways, thanks for your help.
> > > >
> > >
> > > Just to make sure: You _have_ read the FAQ?
> >
> > Yes I have read the FAQ.
> 
> Good!
> 
> > I guess I am just a little nervous about this project.
> 
> Understandable.
> 
> > This is my first time working with encryption and 50TB of
> > critical client data is at stake. Just being thorough and making
sure
> I
> > fully understand everything.
> 
> And that is all you can do. I think you are approaching this
> perfectly right. I take it the 50TB are alredy on XFS+LVM
> at this time and you are just adding the encryption layer?

Currently the data is spread across multiple ext3 volumes of 8TB or less
and is a pain to manage. That is the reason we are moving to XFS. The
encryption is currently being handled by a Brocade switch blade. The
encryption blade caused us major issues last year, resulting in a
multi-day outage. That is the why we are moving to LUKS.

> 
> Make sure to use XTS mode or plain64 with ESSIV for these
> volume sizes.

I don't know what this means. Haha. But I will look it up.

> 
> And let us know what your expeiences are!
> 

Sure thing. This project has been my first exposure to XFS, LUKS and
clustering. And I'm the main Linux guy on my team. I've been working
with Red Hat, but the solution architect laughed when he saw what we
were trying to do. Haha. But he's been doing research and talking to his
Red Hat contacts. And you guys have helped out tremendously! Thank you!

Jacob

> Arno
> --
> Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email:
> arno at wagner.name
> GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50
1E25
> 338F
> ----
> Cuddly UI's are the manifestation of wishful thinking. -- Dylan Evans
> 
> If it's in the news, don't worry about it.  The very definition of
> "news" is "something that hardly ever happens." -- Bruce Schneier
> _______________________________________________
> dm-crypt mailing list
> dm-crypt at saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt


More information about the dm-crypt mailing list