Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
cryptsetup-1.0.5-r1 segfaults on amd64
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
102039
Tux's lil' helper
Tux's lil' helper


Joined: 16 Mar 2005
Posts: 125

PostPosted: Fri Jan 04, 2008 10:35 am    Post subject: cryptsetup-1.0.5-r1 segfaults on amd64 Reply with quote

Hello,

i have a problem creating a encrypted partition using cryptsetup-1.0.5-r1 on an amd64/hardened system.

Quote:

servername ~ # cryptsetup -c aes-lrw-benbi -y -s 384 luksFormat /dev/vg/data

WARNING!
========
This will overwrite data on /dev/vg/data irrevocably.

Are you sure? (Type uppercase yes): YES
Segmentation fault


As you can see, i am trying to encrypt a lvm2 partition, but that shouldn't be a problem, because it works fine on a ~x86 system. All kernel options are correct, i am using hardened extensions, but i also tried by removing all pax/grsec features, which did not help.

Here is my emerge --info
Quote:
Portage 2.1.3.19 (hardened/amd64, gcc-3.4.6, glibc-2.6.1-r0, 2.6.23-hardened-r4 x86_64)
=================================================================
System uname: 2.6.23-hardened-r4 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor 6000+
Timestamp of tree: Fri, 04 Jan 2008 07:00:05 +0000
app-shells/bash: 3.2_p17-r1
dev-java/java-config: 1.3.7, 2.0.33-r1
dev-lang/python: 2.4.4-r6
dev-python/pycrypto: 2.0.1-r6
sys-apps/baselayout: 1.12.10-r5
sys-apps/sandbox: 1.2.18.1-r2
sys-devel/autoconf: 2.61-r1
sys-devel/automake: 1.10
sys-devel/binutils: 2.18-r1
sys-devel/gcc-config: 1.4.0-r4
sys-devel/libtool: 1.5.24
virtual/os-headers: 2.6.23-r2
ACCEPT_KEYWORDS="amd64"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=athlon64 -pipe -O2 -fforce-addr -msse3"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/udev/rules.d"
CXXFLAGS="-march=athlon64 -pipe -O2 -fforce-addr -msse3"
DISTDIR="/usr/portage/distfiles"
FEATURES="ccache distlocks metadata-transfer sandbox sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --filter=H_**/files/digest-*"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="3dnow acl acpi amd64 bcmath berkdb bzip2 calendar cgi cracklib crypt ctype erandom exif fastcgi force-cgi-redirect gd gdbm geoip glibc-omitfp hardened iconv imagemagick imap ipv6 ithreads jpeg justify logrotate maildir memlimit mhash mpm-worker mysql mysqli ncurses nls no-htdocs nptl nptlonly pam pcntl pcre php pic png posix python qmail readline session simplexml slang spamassassin sse sse2 ssl symlink sysvipc tcpd threads tiff tokenizer truetype udev unicode urandom utf8 vhosts xml xml2 xmlrpc xsl zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="mouse keyboard" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="apm ark chips cirrus cyrix dummy fbdev glint i128 i810 mach64 mga neomagic nv r128 radeon rendition s3 s3virge savage siliconmotion sis sisusb tdfx tga trident tseng v4l vesa vga via vmware voodoo"
Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY


Any ideas ?
Back to top
View user's profile Send private message
schachti
Advocate
Advocate


Joined: 28 Jul 2003
Posts: 3765
Location: Gifhorn, Germany

PostPosted: Fri Jan 04, 2008 10:44 am    Post subject: Reply with quote

Does it work if you remove -fforce-addr -msse3 from your CFLAGS and re-emerge cryptsetup?
_________________
Never argue with an idiot. He brings you down to his level, then beats you with experience.

How-To: Daten verschlüsselt auf DVD speichern.
Back to top
View user's profile Send private message
pkerwien
n00b
n00b


Joined: 06 Jul 2005
Posts: 21
Location: Sweden

PostPosted: Fri Jan 04, 2008 11:13 am    Post subject: Reply with quote

Since your are using the LUKS extension, I assume you should install cryptsetup-luks. The latest stable amd64 in portage is cryptsetup-luks-1.0.4-r3.
_________________
If Windows is the answer, you don't understand the question!
Back to top
View user's profile Send private message
102039
Tux's lil' helper
Tux's lil' helper


Joined: 16 Mar 2005
Posts: 125

PostPosted: Fri Jan 04, 2008 11:58 am    Post subject: Reply with quote

schachti wrote:
Does it work if you remove -fforce-addr -msse3 from your CFLAGS and re-emerge cryptsetup?


No, removed those flags and emerge'd cryptsetup again, still the same error.

pkerwien wrote:
Since your are using the LUKS extension, I assume you should install cryptsetup-luks. The latest stable amd64 in portage is cryptsetup-luks-1.0.4-r3.


I also thought that in the first place, but LUKS is included in the cryptsetup package now, since version 1.0.5. I tried anyways with cryptsetup-luks, because you never know... But it didn't help :/
Back to top
View user's profile Send private message
pkerwien
n00b
n00b


Joined: 06 Jul 2005
Posts: 21
Location: Sweden

PostPosted: Fri Jan 04, 2008 3:06 pm    Post subject: Reply with quote

Quote:
I also thought that in the first place, but LUKS is included in the cryptsetup package now, since version 1.0.5. I tried anyways with cryptsetup-luks, because you never know... But it didn't help :/

OK. Thanks for the info. I will now upgrade to cryptsetup-1.0.5...
_________________
If Windows is the answer, you don't understand the question!
Back to top
View user's profile Send private message
pkerwien
n00b
n00b


Joined: 06 Jul 2005
Posts: 21
Location: Sweden

PostPosted: Fri Jan 04, 2008 5:26 pm    Post subject: Reply with quote

FYI: Your command works on my Gentoo amd64 with kernel 2.6.23.12 on a plain IDE drive. But I'm not using hardened. My CFLAGS="-march=nocona -O2 -pipe".

Will now try to create a logical drive and see if I can create some errors...
_________________
If Windows is the answer, you don't understand the question!
Back to top
View user's profile Send private message
Hu
Administrator
Administrator


Joined: 06 Mar 2007
Posts: 22890

PostPosted: Sat Jan 05, 2008 3:34 am    Post subject: Reply with quote

Try building sys-fs/cryptsetup using the -hardenednopiessp variant of gcc. If that produces a working binary, try again with -hardenednossp and -hardenednopie, then report back with which ones produced a working cryptsetup and which ones failed.
Back to top
View user's profile Send private message
102039
Tux's lil' helper
Tux's lil' helper


Joined: 16 Mar 2005
Posts: 125

PostPosted: Sun Jan 06, 2008 3:06 am    Post subject: Reply with quote

Hu wrote:
Try building sys-fs/cryptsetup using the -hardenednopiessp variant of gcc. If that produces a working binary, try again with -hardenednossp and -hardenednopie, then report back with which ones produced a working cryptsetup and which ones failed.


Hi,

have tried that, no change, still segfaulting :/

Quote:
Will now try to create a logical drive and see if I can create some errors...


It works on a x86, i have one machine running, same useflags, same setup...and it worked perfectly. Only difference to this machine is amd64 and the cflags.
Back to top
View user's profile Send private message
pkerwien
n00b
n00b


Joined: 06 Jul 2005
Posts: 21
Location: Sweden

PostPosted: Sun Jan 06, 2008 3:15 am    Post subject: Reply with quote

Forgot to post my results: :oops:

Your cryptsetup command worked for me on my logical volume of 100GB (the volume group consists of 2 x 60GB harddrives). (I'm not sure if the LVM terms are correct. I'm a LVM n00b).
_________________
If Windows is the answer, you don't understand the question!
Back to top
View user's profile Send private message
102039
Tux's lil' helper
Tux's lil' helper


Joined: 16 Mar 2005
Posts: 125

PostPosted: Fri Jan 11, 2008 8:08 am    Post subject: Reply with quote

Hi pkerwien,

thanks for your tests, i guess it must be some kernel setting or cflag then, maybe a library used by cryptsetup must not be emerged with "-msse3" or "-march=athlon64. "-fforce-addr" seems to be ok, because i also use it on the x86 system, which works. I am totally clueless at the moment :/

Anyone else who has an idea ?
Back to top
View user's profile Send private message
pkerwien
n00b
n00b


Joined: 06 Jul 2005
Posts: 21
Location: Sweden

PostPosted: Fri Jan 11, 2008 5:13 pm    Post subject: Reply with quote

I can try to re-compile my whole testsystem first with -msse3 and then -fforce-addr and see what's happen.
_________________
If Windows is the answer, you don't understand the question!
Back to top
View user's profile Send private message
102039
Tux's lil' helper
Tux's lil' helper


Joined: 16 Mar 2005
Posts: 125

PostPosted: Fri Jan 11, 2008 9:06 pm    Post subject: Reply with quote

Would be awesome if you could do this, i could also drop you my kernel config, so you can check that if the cflags make no problem. It would be great if i could get this finally working :)

Big thanks for your help, very much appreciate it !
Back to top
View user's profile Send private message
pkerwien
n00b
n00b


Joined: 06 Jul 2005
Posts: 21
Location: Sweden

PostPosted: Sun Jan 13, 2008 10:59 am    Post subject: Reply with quote

Could not reproduce any problem with the -msse3 flag and then later with the -msse3 + -fforce-addr flag.
_________________
If Windows is the answer, you don't understand the question!
Back to top
View user's profile Send private message
102039
Tux's lil' helper
Tux's lil' helper


Joined: 16 Mar 2005
Posts: 125

PostPosted: Sat Jan 19, 2008 1:13 am    Post subject: Reply with quote

Thanks for testing again! I sent you the my kernel config file, maybe you can try with that again. If that doesn't help i need to give up on the issue :/
Back to top
View user's profile Send private message
pkerwien
n00b
n00b


Joined: 06 Jul 2005
Posts: 21
Location: Sweden

PostPosted: Sat Jan 19, 2008 1:55 pm    Post subject: Reply with quote

Tested with your kernel config + hardened-sources-2.6.23-r4, just changed CPU from Athlon64 to Core 2. Still no problem with the cryptsetup luksFormat command.
_________________
If Windows is the answer, you don't understand the question!
Back to top
View user's profile Send private message
102039
Tux's lil' helper
Tux's lil' helper


Joined: 16 Mar 2005
Posts: 125

PostPosted: Mon Feb 04, 2008 12:24 pm    Post subject: Reply with quote

Some great news...i tested it with the latest SVN version 1.0.6-pre1 of cryptsetup and it works! Thanks to Clemens Fruewirth

Thanks pkerwien again, for running all those tests anyway! I am glad it is running now :)
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum