[dm-crypt] "not a valid LUKS device" after distro change

John Wells jbwellsiv at gmail.com
Fri Aug 22 05:52:00 CEST 2014


Thanks Arno. Assuming I don't want to give up LVM just yet, is there a
particular distro which tends to cautiously tie raid, lvm and luks together
reliably? Or is it that they're all doing this, but in incompatible ways?
On Aug 21, 2014 6:54 PM, "Arno Wagner" <arno at wagner.name> wrote:

> This looks to me like some massive issues with user-space tools,
> starting with the md-devices, that in Fedora are md0 and md2,
> but in Ubuntu are md126 and md127.
>
> This likely means they are of the completely braindead "new"
> RAID-superblock formats where the RAID assembly is not done
> by the RAID controller (i.e. the kernel) as it should be, but
> by something in userspace. Change the userspace, lose proper
> RAID assembly. The idiots responsible for that have not
> understood what layering is and why you do it and have
> implemented a massive step backwards.
>
> The way around this is to use the old superblock format 0.90
> with partition type 0xfd. With that and any kernel doing RAID
> auto-assembly, you always get the same evices. Of course, some
> distros are utterly brain-dead and re-map the 0.90 devices to
> new names via user-space re-assembly, which are then persistent.
> Had that with one "recovery CD once", that I needed an hour to
> recover from booting.
>
> Yes, the 1.x formats have better capabilities, but what use
> is that when they are fundamentally broken otherwise? "Better
> capabilities" is not a valid reason to break things that worked
> before.
>
> My advice around this whole mess would be to make a simple,
> flat patitioning and RAID schema yourself manually (superblock
> 0.90, type 0xfd, no LVM, no partitioning in RAID, just make
> more RAID arrays) and install into that. That seems the only
> way to get KISS with regard to RAID and partitioning these
> days and KISS is the only thing that gives you reliability
> and ease of debugging.
>
> I think I will add a warning to the FAQ that changing distros
> can make your LUKS containers hard or impossible to access due
> to LVM and new RAID superblock userspace differences and that
> it may mess things up enough that a move back to the old distro
> may not be sufficient to fix things. How these jokers expect
> people to deal with multiple distros on the same machine is
> beyond me.
>
> Arno
>
>
>
>
>
>
> On Thu, Aug 21, 2014 at 22:46:04 CEST, John Wells wrote:
> > Attached below, I've included the output of running these commands from
> > first Ubuntu and then Fedora. Note, apparently they map the raid device
> > numbers differently. Also, you'll note there's a cryptd mounted in
> Ubuntu.
> > That's the one i can successfully mount from Ubuntu.
> >
> > To make things even stranger, I was ***able to luksOpen and then mount
> > /dev/MORE_VG/MORE_LV this time while booted in the Fedora 20*** live cd,
> > while as you recall I was unable to before. I'm also including the
> results
> > of dumping the head of the dev file from Fedora.
> >
> > I'm completely at a loss. At this point I feel I should probably backup
> the
> > entire machine and re-install from scratch, but I'll hold off a bit
> longer.
> > The inconsistent behavior is really disconcerting.
> >
> > Thanks,
> > John
> >
> > ####################### FIRST FROM UBUNTU ########################
> > # lsblk
> > NAME                                  MAJ:MIN RM   SIZE RO TYPE
> MOUNTPOINT
> > sda                                     8:0    0  37.3G  0 disk
> > ├─sda1                                  8:1    0   100M  0 part
> > └─sda2                                  8:2    0  37.2G  0 part
> > sdb                                     8:16   0   1.8T  0 disk
> > └─sdb1                                  8:17   0   1.8T  0 part
> >   └─md0                                 9:0    0   1.8T  0 raid1
> >     └─MORE_VG-MORE_LV (dm-1)          252:1    0   800G  0 lvm
> >       └─cryptd (dm-7)                 252:7    0   800G  0 crypt /home
> > sdc                                     8:32   0   1.8T  0 disk
> > └─sdc1                                  8:33   0   1.8T  0 part
> >   └─md2                                 9:2    0   1.8T  0 raid1
> >     ├─FINALFRONTIER_VG-HOME_LV (dm-2) 252:2    0 698.5G  0 lvm
> >     ├─FINALFRONTIER_VG-SWAP_LV (dm-3) 252:3    0  22.4G  0 lvm   [SWAP]
> >     ├─FINALFRONTIER_VG-OPT_LV (dm-4)  252:4    0 139.7G  0 lvm   /opt
> >     ├─FINALFRONTIER_VG-VMS_LV (dm-5)  252:5    0 102.5G  0 lvm   /vms
> >     └─FINALFRONTIER_VG-DATA_LV (dm-6) 252:6    0 279.4G  0 lvm   /data
> > sdd                                     8:48   0   1.8T  0 disk
> > └─sdd1                                  8:49   0   1.8T  0 part
> >   └─md2                                 9:2    0   1.8T  0 raid1
> >     ├─FINALFRONTIER_VG-HOME_LV (dm-2) 252:2    0 698.5G  0 lvm
> >     ├─FINALFRONTIER_VG-SWAP_LV (dm-3) 252:3    0  22.4G  0 lvm   [SWAP]
> >     ├─FINALFRONTIER_VG-OPT_LV (dm-4)  252:4    0 139.7G  0 lvm   /opt
> >     ├─FINALFRONTIER_VG-VMS_LV (dm-5)  252:5    0 102.5G  0 lvm   /vms
> >     └─FINALFRONTIER_VG-DATA_LV (dm-6) 252:6    0 279.4G  0 lvm   /data
> > sde                                     8:64   0 167.7G  0 disk
> > ├─sde1                                  8:65   0     3G  0 part  /boot
> > ├─sde2                                  8:66   0    12G  0 part
> > └─sde3                                  8:67   0 152.8G  0 part
> >   └─SSDROOT_VG-ROOT_LV (dm-0)         252:0    0  97.7G  0 lvm   /
> > sdf                                     8:80   0   1.8T  0 disk
> > └─sdf1                                  8:81   0   1.8T  0 part
> > sdg                                     8:96   0   1.8T  0 disk
> > └─sdg1                                  8:97   0   1.8T  0 part
> >   └─BACKUPVG-BACKUPLV (dm-8)          252:8    0   1.2T  0 lvm   /backup
> > sr0                                    11:0    1  1024M  0 rom
> > # dmsetup ls
> > FINALFRONTIER_VG-VMS_LV (252:5)
> > BACKUPVG-BACKUPLV (252:8)
> > MORE_VG-MORE_LV (252:1)
> > FINALFRONTIER_VG-OPT_LV (252:4)
> > FINALFRONTIER_VG-SWAP_LV (252:3)
> > SSDROOT_VG-ROOT_LV (252:0)
> > cryptd (252:7)
> > FINALFRONTIER_VG-DATA_LV (252:6)
> > FINALFRONTIER_VG-HOME_LV (252:2)
> > # dmsetup deps
> > FINALFRONTIER_VG-VMS_LV: 1 dependencies : (9, 2)
> > BACKUPVG-BACKUPLV: 1 dependencies : (8, 97)
> > MORE_VG-MORE_LV: 1 dependencies : (9, 0)
> > FINALFRONTIER_VG-OPT_LV: 1 dependencies : (9, 2)
> > FINALFRONTIER_VG-SWAP_LV: 1 dependencies : (9, 2)
> > SSDROOT_VG-ROOT_LV: 1 dependencies : (8, 67)
> > cryptd: 1 dependencies : (252, 1)
> > FINALFRONTIER_VG-DATA_LV: 1 dependencies : (9, 2)
> > FINALFRONTIER_VG-HOME_LV: 1 dependencies : (9, 2)
> >
> > ########################## THEN FROM FEDORA #########################
> >
> > # lsblk
> > NAME                           MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
> > sda                              8:0    0  37.3G  0 disk
> > ├─sda1                           8:1    0   100M  0 part
> > └─sda2                           8:2    0  37.2G  0 part
> > sdb                              8:16   1  14.6G  0 disk
> > ├─sdb1                           8:17   1   953M  0 part
> >  /run/initramfs/live
> > ├─sdb2                           8:18   1   4.9M  0 part
> > └─sdb3                           8:19   1  19.7M  0 part
> > sdc                              8:32   0   1.8T  0 disk
> > └─sdc1                           8:33   0   1.8T  0 part
> >   └─md126                        9:126  0   1.8T  0 raid1
> >     └─MORE_VG-MORE_LV          253:4    0   800G  0 lvm
> > sdd                              8:48   0   1.8T  0 disk
> > └─sdd1                           8:49   0   1.8T  0 part
> >   └─md127                        9:127  0   1.8T  0 raid1
> >     ├─FINALFRONTIER_VG-HOME_LV 253:5    0 698.5G  0 lvm
> >     ├─FINALFRONTIER_VG-SWAP_LV 253:6    0  22.4G  0 lvm
> >     ├─FINALFRONTIER_VG-OPT_LV  253:7    0 139.7G  0 lvm   /opt
> >     ├─FINALFRONTIER_VG-VMS_LV  253:8    0 102.5G  0 lvm
> >     └─FINALFRONTIER_VG-DATA_LV 253:9    0 279.4G  0 lvm
> > sde                              8:64   0   1.8T  0 disk
> > └─sde1                           8:65   0   1.8T  0 part
> >   └─md127                        9:127  0   1.8T  0 raid1
> >     ├─FINALFRONTIER_VG-HOME_LV 253:5    0 698.5G  0 lvm
> >     ├─FINALFRONTIER_VG-SWAP_LV 253:6    0  22.4G  0 lvm
> >     ├─FINALFRONTIER_VG-OPT_LV  253:7    0 139.7G  0 lvm   /opt
> >     ├─FINALFRONTIER_VG-VMS_LV  253:8    0 102.5G  0 lvm
> >     └─FINALFRONTIER_VG-DATA_LV 253:9    0 279.4G  0 lvm
> > sdf                              8:80   0 167.7G  0 disk
> > ├─sdf1                           8:81   0     3G  0 part
> > ├─sdf2                           8:82   0    12G  0 part
> > └─sdf3                           8:83   0 152.8G  0 part
> >   └─SSDROOT_VG-ROOT_LV         253:3    0  97.7G  0 lvm
> > sdg                              8:96   0   1.8T  0 disk
> > └─sdg1                           8:97   0   1.8T  0 part
> > sdh                              8:112  0   1.8T  0 disk
> > └─sdh1                           8:113  0   1.8T  0 part
> >   └─BACKUPVG-BACKUPLV          253:10   0   1.2T  0 lvm   /backup
> > sr0                             11:0    1  1024M  0 rom
> > loop0                            7:0    0     8K  1 loop
> > loop1                            7:1    0   1.2M  1 loop
> > └─live-osimg-min               253:2    0     4G  1 dm
> > loop2                            7:2    0 886.8M  1 loop
> > loop3                            7:3    0     4G  1 loop
> > ├─live-rw                      253:0    0     4G  0 dm    /
> > ├─live-base                    253:1    0     4G  1 dm
> > └─live-osimg-min               253:2    0     4G  1 dm
> > loop4                            7:4    0   512M  0 loop
> > └─live-rw                      253:0    0     4G  0 dm    /
> > # dmsetup ls
> > live-base (253:1)
> > FINALFRONTIER_VG-VMS_LV (253:8)
> > BACKUPVG-BACKUPLV (253:10)
> > MORE_VG-MORE_LV (253:4)
> > FINALFRONTIER_VG-OPT_LV (253:7)
> > live-osimg-min (253:2)
> > FINALFRONTIER_VG-SWAP_LV (253:6)
> > live-rw (253:0)
> > SSDROOT_VG-ROOT_LV (253:3)
> > FINALFRONTIER_VG-DATA_LV (253:9)
> > FINALFRONTIER_VG-HOME_LV (253:5)
> > # dmsetup deps
> > live-base: 1 dependencies : (7, 3)
> > FINALFRONTIER_VG-VMS_LV: 1 dependencies : (9, 127)
> > BACKUPVG-BACKUPLV: 1 dependencies : (8, 113)
> > MORE_VG-MORE_LV: 1 dependencies : (9, 126)
> > FINALFRONTIER_VG-OPT_LV: 1 dependencies : (9, 127)
> > live-osimg-min: 2 dependencies : (7, 1) (7, 3)
> > FINALFRONTIER_VG-SWAP_LV: 1 dependencies : (9, 127)
> > live-rw: 2 dependencies : (7, 4) (7, 3)
> > SSDROOT_VG-ROOT_LV: 1 dependencies : (8, 83)
> > FINALFRONTIER_VG-DATA_LV: 1 dependencies : (9, 127)
> > FINALFRONTIER_VG-HOME_LV: 1 dependencies : (9, 127)
> >
> > ###################### AND NOW THE dumps of both devs from FEDORA
> > ############################
> > # head -c 1024 /dev/FINALFRONTIER_VG/HOME_LV | hexdump -C
> > 00000000  47 4e 55 20 50 61 72 74  65 64 20 4c 6f 6f 70 62  |GNU Parted
> > Loopb|
> > 00000010  61 63 6b 20 30 00 00 00  00 00 00 00 00 00 00 00  |ack
> > 0...........|
> > 00000020  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > *
> > 00000400
> >
> >
> >
> > # head -c 1024 /dev/MORE_VG/MORE_LV | hexdump -C
> > 00000000  4c 55 4b 53 ba be 00 01  61 65 73 00 00 00 00 00
> >  |LUKS....aes.....|
> > 00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000020  00 00 00 00 00 00 00 00  78 74 73 2d 70 6c 61 69
> >  |........xts-plai|
> > 00000030  6e 36 34 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |n64.............|
> > 00000040  00 00 00 00 00 00 00 00  73 68 61 31 00 00 00 00
> >  |........sha1....|
> > 00000050  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000060  00 00 00 00 00 00 00 00  00 00 10 00 00 00 00 40
> >  |...............@|
> > 00000070  4e 94 98 c0 51 a9 25 bb  74 73 88 a4 a7 6e c7 d3
> >  |N...Q.%.ts...n..|
> > 00000080  6f 18 71 fa 94 9a f4 5e  d1 e8 5b 1a 34 3a 9f 9f
> >  |o.q....^..[.4:..|
> > 00000090  1d 79 1f 61 f5 dd 98 09  e1 d6 2e ed c4 29 af 1a
> >  |.y.a.........)..|
> > 000000a0  23 c9 59 da 00 00 77 a1  36 63 63 31 38 38 64 62
> >  |#.Y...w.6cc188db|
> > 000000b0  2d 63 63 64 62 2d 34 63  38 66 2d 39 37 62 32 2d
> >  |-ccdb-4c8f-97b2-|
> > 000000c0  65 34 31 31 39 38 65 63  36 65 34 34 00 00 00 00
> >  |e41198ec6e44....|
> > 000000d0  00 ac 71 f3 00 01 df d7  79 85 dd f0 29 59 98 63
> >  |..q.....y...)Y.c|
> > 000000e0  0b 69 80 fe 48 61 8c 40  5b 3b 57 0f 82 9c ae 90  |.i..Ha.@
> > [;W.....|
> > 000000f0  36 57 45 e2 03 82 26 c5  00 00 00 08 00 00 0f a0
> >  |6WE...&.........|
> > 00000100  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000110  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000120  00 00 00 00 00 00 00 00  00 00 02 00 00 00 0f a0
> >  |................|
> > 00000130  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000140  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000150  00 00 00 00 00 00 00 00  00 00 03 f8 00 00 0f a0
> >  |................|
> > 00000160  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000170  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000180  00 00 00 00 00 00 00 00  00 00 05 f0 00 00 0f a0
> >  |................|
> > 00000190  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 000001a0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 000001b0  00 00 00 00 00 00 00 00  00 00 07 e8 00 00 0f a0
> >  |................|
> > 000001c0  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 000001d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 000001e0  00 00 00 00 00 00 00 00  00 00 09 e0 00 00 0f a0
> >  |................|
> > 000001f0  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000200  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000210  00 00 00 00 00 00 00 00  00 00 0b d8 00 00 0f a0
> >  |................|
> > 00000220  00 00 de ad 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000230  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> > 00000240  00 00 00 00 00 00 00 00  00 00 0d d0 00 00 0f a0
> >  |................|
> > 00000250  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
> >  |................|
> >
> >
> >
> >
> > On Thu, Aug 21, 2014 at 12:13 PM, Jonas Meurer <jonas at freesources.org>
> > wrote:
> >
> > > Hello John,
> > >
> > > Am 21.08.2014 um 17:55 schrieb John Wells:
> > > > Thanks. How would I use this information to track back to a potential
> > > > problem?
> > >
> > > I'm not convinced yet, that your system screwed up partition and/or
> > > device ordering. First you should try to find out which devices are
> > > recognized both in Fedora and Ubuntu and compare these.
> > >
> > > Try to run 'lsblk', 'dmsetup ls' and 'dmsetup deps' on both systems and
> > > compare or post the result here.
> > >
> > > Which Ubuntu version do you run? Do you use an installed Ubuntu or
> > > rather a live image? It would help if you could provide some more
> > > information regarding your system.
> > >
> > > Kind regards,
> > >  jonas
> > >
> > > >
> > > > On Aug 21, 2014 10:41 AM, "Robert Nichols" <
> rnicholsNOSPAM at comcast.net
> > > > <mailto:rnicholsNOSPAM at comcast.net>> wrote:
> > > >
> > > >     On 08/21/2014 09:00 AM, John Wells wrote:
> > > >
> > > >         I will try what you say. To add to the weirdness, I came into
> > > >         work this morning
> > > >         with a unresponsive machine. I hard reset, booted Ubuntu,
> but at
> > > >         that point
> > > >         *Ubuntu wouldn't recognize either partition, and both had the
> > > >         same "GNU Parted
> > > >         Loopback 0" in the output of "head -c 1024 /volume | hd".
> > > >         Neither partition was
> > > >         recognized by luksDump. I panicked.
> > > >
> > > >         Rebooted, and guess what? /dev/MORE_VG/MORE_LV was back to
> > > >         normal and I could
> > > >         mount it. /dev/FINALFRONTIER_VG/HOME_LV was still corrupted
> with
> > > >         the "GNU Parted
> > > >         Loopback 0" output.
> > > >
> > > >         This makes no sense to me. How could the leading bits be
> > > >         different each time I
> > > >         booted up? Could datamapper be assigning the wrong device to
> the
> > > >         logical volume
> > > >         in some way? It just makes no sense.
> > > >
> > > >
> > > >     You can run (as root) "dmsetup deps" on each device in
> /dev/mapper
> > > >     to see
> > > >     the major and minor device numbers needed for each entry.
> > > >
> > > >     --
> > > >     Bob Nichols     "NOSPAM" is really part of my email address.
> > > >                     Do NOT delete it.
> > > >
> > > >     _________________________________________________
> > > >     dm-crypt mailing list
> > > >     dm-crypt at saout.de <mailto:dm-crypt at saout.de>
> > > >     http://www.saout.de/mailman/__listinfo/dm-crypt
> > > >     <http://www.saout.de/mailman/listinfo/dm-crypt>
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > dm-crypt mailing list
> > > > dm-crypt at saout.de
> > > > http://www.saout.de/mailman/listinfo/dm-crypt
> > > >
> > >
> > > _______________________________________________
> > > dm-crypt mailing list
> > > dm-crypt at saout.de
> > > http://www.saout.de/mailman/listinfo/dm-crypt
> > >
>
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt at saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
>
>
> --
> Arno Wagner,     Dr. sc. techn., Dipl. Inform.,    Email: arno at wagner.name
> GnuPG: ID: CB5D9718  FP: 12D6 C03B 1B30 33BB 13CF  B774 E35C 5FA1 CB5D 9718
> ----
> A good decision is based on knowledge and not on numbers. -  Plato
> _______________________________________________
> dm-crypt mailing list
> dm-crypt at saout.de
> http://www.saout.de/mailman/listinfo/dm-crypt
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.saout.de/pipermail/dm-crypt/attachments/20140821/ee1d538a/attachment-0001.html>


More information about the dm-crypt mailing list