[dm-crypt] crypsetup segfaulting during luksFormat

Sven Eschenberg sven at whgl.uni-frankfurt.de
Thu Jul 8 15:37:48 CEST 2010


Just for the record:

The crash happens with other gcc versions as well. As the gentoo bug
report suggests, it seems to be a problem when the executeable is linked
statically on hardened profiles.
And yes, in my case compiling it dynamically resolves the segfault
aswell.

In the src the following variables are used in the handler:

static volatile uint64_t __PBKDF2_global_j = 0;
static volatile uint64_t __PBKDF2_performance = 0;

Since they are used in the sighandler, they would better not just be
volatile but sig_atomic_t, to avoid possible races.

But this should not have any influence on the segfault as far as I can
tell.

Oh, and better use sigaction() instead of signal().

I think I possibly found the problem:

In static int pkcs5_pbkdf2() in pbkdf.c:

size_t tmplen = Slen + 4;
tmp = alloca(tmplen); // allocate Slen+4 bytes on the stack ...

Later:

tmp[Slen + 3] = (i & 0x000000ff) >> 0;

Ohoh, implicit cast to smaller Type, here we trash the stackframe since
tmp was allocated on the stack.

Regards

-Sven


On Thu, 2010-07-08 at 11:30 +0200, Arno Wagner wrote:
> Lets give Milan a few days. I expect he will figure it out.
> I admit, I do not understand what the issue really is either 
> and I do not have gentoo. Mazbe I should take a look at it ...
> 
> Arno
> 
> On Thu, Jul 08, 2010 at 04:22:35AM +0200, Sven Eschenberg wrote:
> > Okay, let's see:
> > 
> > gcc --version 
> > gcc (Gentoo Hardened 4.4.4-r1 p1.0, pie-0.4.5) 4.4.4
> > Copyright (C) 2010 Free Software Foundation, Inc.
> > This is free software; see the source for copying conditions.  There is
> > NO
> > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> > PURPOSE.
> > 
> > I am not really sure it is a problem in the source, maybe some awkward
> > scenario.
> > 
> > If you say, the source did not change, maybe you could recommend some
> > sort of testing scenario. I'd feel happy to contribute here. Currently I
> > feel just stuck...
> > 
> > Regards
> > 
> > -Sven
> > 
> > P.S.: Last time I had to set up a mapping(initially) (as in luksFormat)
> > was like 1 year ago (or somethign amog these figures)... so maybe I just
> > did not hit the bug (before) in the last couple of months ...
> > 
> > 
> > 
> > 
> > 
> > On Wed, 2010-07-07 at 23:22 +0200, Milan Broz wrote:
> > > On 07/07/2010 10:58 PM, Sven Eschenberg wrote:
> > > > For starters, I'll select the easiest case, plain luksFormat without
> > > > special params:
> > > 
> > > thanks.
> > > 
> > > > The segfault comes way before the 10 seconds are over, that much I can
> > > > tell already. For the other info: I'm on gentoo, 10.0, hardened profile,
> > > > on amd64 arch.
> > > 
> > > Which gcc? (gcc --version)
> > > 
> > > I am using gentoo (but 32bit) for upstream testing,
> > > it should fail here too then:)
> > > 
> > > But that code is there since 1.1.0, so I guess something in gcc/glibc
> > > changed probably and uncovers some old bug.
> > > 
> > > > Currently no core file is created, can you give me a fast shot on how I
> > > > force the system to write out a core?
> > > 
> > > try "ulimit -c unlimited" and then run that command
> > > 
> > > you can also define where core files will be stored using
> > > sysctl kernel.core_pattern=/<some dir>/core 
> > > 
> > > Milan
> > > _______________________________________________
> > > dm-crypt mailing list
> > > dm-crypt at saout.de
> > > http://www.saout.de/mailman/listinfo/dm-crypt
> > 
> > 
> > _______________________________________________
> > dm-crypt mailing list
> > dm-crypt at saout.de
> > http://www.saout.de/mailman/listinfo/dm-crypt
> > 
> 




More information about the dm-crypt mailing list