[dm-crypt] "not a valid LUKS device" after distro change

Arno Wagner arno at wagner.name
Tue Oct 28 19:34:39 CET 2014


This is really catatstrophic, agreed. But it seems to come
from a common cause and it is not a LUKS problem. It is a 
problem with something messing with partitions 
"auto-magically" and that "something" happens to not 
recognize LUKS containers as things to leave intact.

I think I said it before: Ubuntu has a tendency to not
recognize LUKS containers, see for example the installer
that deletes present LUKS containers without clear warning
(AFAIK still not fixed).

My take is this is possible some kind of automtic recognition
of the disk-setup that under some circumstances fails and
compounds ints inability by writing to the misidentified
partition. 

It would be pretty good finding out what this is, so I
can write an additional warning for the FAQ  and send 
the Ubuntu maintainers a "WTF do you thing you are 
doing?" email (if it is their fault) and, as importantly, 
get this fixed. 

So, here is some debugging ideas:

- Go though all the boot-scripts in /etc/init.d and
  see whether there are any calls to parted.
- Same with the root cron-jobs.
- Same for "partx" and "kpartx"

And more:

- Look in the logs (/var/log) for any instances of
  parted, partx, kpartx and lvm* and see whether it
  says anything about loo-mounting.
  Hot candidates are the minutes/hours right before 
  it failed.
- If this failed pretty soon after booting: Also look
  at the startup messages for any partition detection
  messeges or the like.

On the risk of repeating myself, this also nicely 
illustrates that LVM gives you too much flexibility
and not enough safety. It also shows that backup
needs to be tested in order to deserve any trust.

Gr"usse,
Arno

On Tue, Oct 28, 2014 at 16:35:30 CET, John Wells wrote:
> Note, if you read far enough back in the thread, the same thing happened to
> the original partition which failed...it suddenly showed up as a loopback
> device and wasn't recongized by cryptsetup.
> 
> On Tue, Oct 28, 2014 at 11:34 AM, John Wells <jbwellsiv at gmail.com> wrote:
> 
> > Hi guys, I know I went silent on this. I ended up being on the road a good
> > bit and just restored from back up and accepted it as an isolated incident.
> >
> > However, today I received some odd messages and the inability to write to
> > files.
> >
> > Now, after reboot, /dev/MORE_VG/MORE_LV is suddenly missing it's Luks info
> > as well.
> >
> > $ sudo cryptsetup luksOpen /dev/MORE_VG/MORE_LV cryptd
> > Device /dev/MORE_VG/MORE_LV is not a valid LUKS device.
> > $ sudo head -c 1024 /dev/MORE_VG/MORE_LV | hexdump -C
> > 00000000  47 4e 55 20 50 61 72 74  65 64 20 4c 6f 6f 70 62  |GNU Parted
> > Loopb|
> > 00000010  61 63 6b 20 30 00 00 00  00 00 00 00 00 00 00 00  |ack
> > 0...........|
> > 00000020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > *
> > 00000400
> >
> > Recall, before this happened, it looked like this:
> >
> > # head -c 1024 /dev/MORE_VG/MORE_LV | hexdump -C
> > 00000000  4c 55 4b 53 ba be 00 01  61 65 73 00 00 00 00 00
> >  |LUKS....aes.....|
> > 00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000020  00 00 00 00 00 00 00 00  78 74 73 2d 70 6c 61 69
> >  |........xts-plai|
> > 00000030  6e 36 34 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |n64.............|
> > 00000040  00 00 00 00 00 00 00 00  73 68 61 31 00 00 00 00
> >  |........sha1....|
> > 00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000060  00 00 00 00 00 00 00 00  00 00 10 00 00 00 00 40
> >  |...............@|
> > 00000070  4e 94 98 c0 51 a9 25 bb  74 73 88 a4 a7 6e c7 d3
> >  |N...Q.%.ts...n..|
> > 00000080  6f 18 71 fa 94 9a f4 5e  d1 e8 5b 1a 34 3a 9f 9f
> >  |o.q....^..[.4:..|
> > 00000090  1d 79 1f 61 f5 dd 98 09  e1 d6 2e ed c4 29 af 1a
> >  |.y.a.........)..|
> > 000000a0  23 c9 59 da 00 00 77 a1  36 63 63 31 38 38 64 62
> >  |#.Y...w.6cc188db|
> > 000000b0  2d 63 63 64 62 2d 34 63  38 66 2d 39 37 62 32 2d
> >  |-ccdb-4c8f-97b2-|
> > 000000c0  65 34 31 31 39 38 65 63  36 65 34 34 00 00 00 00
> >  |e41198ec6e44....|
> > 000000d0  00 ac 71 f3 00 01 df d7  79 85 dd f0 29 59 98 63
> >  |..q.....y...)Y.c|
> > 000000e0  0b 69 80 fe 48 61 8c 40  5b 3b 57 0f 82 9c ae 90  |.i..Ha.@
> > [;W.....|
> > 000000f0  36 57 45 e2 03 82 26 c5  00 00 00 08 00 00 0f a0
> >  |6WE...&.........|
> > 00000100  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000110  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000120  00 00 00 00 00 00 00 00  00 00 02 00 00 00 0f a0
> >  |................|
> > 00000130  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000140  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000150  00 00 00 00 00 00 00 00  00 00 03 f8 00 00 0f a0
> >  |................|
> > 00000160  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000170  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000180  00 00 00 00 00 00 00 00  00 00 05 f0 00 00 0f a0
> >  |................|
> > 00000190  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 000001a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 000001b0  00 00 00 00 00 00 00 00  00 00 07 e8 00 00 0f a0
> >  |................|
> > 000001c0  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 000001d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 000001e0  00 00 00 00 00 00 00 00  00 00 09 e0 00 00 0f a0
> >  |................|
> > 000001f0  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000200  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000210  00 00 00 00 00 00 00 00  00 00 0b d8 00 00 0f a0
> >  |................|
> > 00000220  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000230  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000240  00 00 00 00 00 00 00 00  00 00 0d d0 00 00 0f a0
> >  |................|
> > 00000250  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> >
> > This has happened even though I haven't changed OSes. This is on Ubuntu
> > 14.04.
> >
> > And, sadly for me, my backup software was silently malfunctioning, so I
> > have lost a good bit of data in this case.
> >
> > Any ideas why a LUKS device would suddenly appear as a loopback device?
> > I'm totally at a loss and not sure if I can trust luks any more with my
> > data, which is sad but understand considering what I paid for it. ;-)
> >
> > Thanks,
> > John
> >
> > On Fri, Aug 22, 2014 at 8:55 AM, Jonas Meurer <jonas at freesources.org>
> > wrote:
> >
> >> Am 22.08.2014 um 13:55 schrieb Arno Wagner:
> >> > On Fri, Aug 22, 2014 at 11:08:56 CEST, Jonas Meurer wrote:
> >> >> Hi John,
> >> >> Am 21.08.2014 um 22:46 schrieb John Wells:
> >> > [...]
> >> >> As Arno already wrote, the hexdump from your meant-to-be LUKS container
> >> >> on HOME_LV in FINALFRONTIER_VG doesn't look too promising. I've never
> >> >> heard about 'GNU Parted Loopback 0' before, but it sounds like
> >> something
> >> >> caused Parted to overwrite your LUKS header.
> >> >
> >> > Not necessarily. The other container had that in it the one time as
> >> > well, but it fixed itself ...
> >>
> >> I'm not sure about that one. Event though John wrote earlier, that »both
> >> had the same "GNU Parted Loopback 0" in the output of "head -c 1024
> >> /volume | hd"«, so maybe you're correct.
> >>
> >> >> Last hope is to find the LUKS header with an offset on the partition.
> >> >> Try to search for 'LUKS' by grepping the output of 'strings
> >> >> /dev/FINALFRONTIER_VG/HOME_LV'. If you don't find the string 'LUKS'
> >> >> followed by something describing your cipher and hash algorithm, I fear
> >> >> that this second partition is lost.
> >> >
> >> > I second searching for that. It may not be the only way though.
> >> > Easy way to search:
> >> >
> >> > hd <partition> | grep "LUKS"
> >> >
> >> > That gives you hex offset(s) to examine further.
> >>
> >> Maybe the best is to grep the whole physical partition for "LUKS"
> >> instead of just searching on the LVM logical volume device. If it's
> >> really linux-md or lvm that mix things up here, then one should not
> >> trust in alignment and layering by them.
> >>
> >> So, John, best would be to do
> >> # hd /dev/sdc | grep "LUKS"
> >> # hd /dev/sdd | grep "LUKS"
> >> (given that sdc and sdd are the disks with md raid-1 and lvm on top).
> >>
> >> If I got your setup right, then FINALFRONTIER_VG-HOME_LV is the only
> >> LUKS-encrypted lv on these disks. Thus, when you get a result, try to
> >> extract the LUKS header from it using the offset and check its validity.
> >>
> >> Kind regards,
> >>  jonas
> >>
> >>
> >> _______________________________________________
> >> dm-crypt mailing list
> >> dm-crypt at saout.de
> >> http://www.saout.de/mailman/listinfo/dm-crypt
> >>
> >
> >

> _______________________________________________
> dm-crypt mailing list
> dm-crypt at saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt


-- 
Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno at wagner.name
GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
----
A good decision is based on knowledge and not on numbers. -- Plato

If it's in the news, don't worry about it.  The very definition of 
"news" is "something that hardly ever happens." -- Bruce Schneier


More information about the dm-crypt mailing list