| View previous topic :: View next topic |
| Author |
Message |
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
 Posted: Sat Oct 28, 2006 11:33 am Post subject: Live USB less than 64MB without catalyst |
 |
|
Hi.
I have a 512MB USB key on which I've created 2 partitions: sda1 - the largest - for storing data to carry between windows machines and sda2, 65MB for the Live USB. Why that configuration? Windows doesn't see more than the first partition ( ), which is very convenient to protect the bootable Live USB code.
I'm installing Gentoo on the small partition (65 MB). I've followed the Howto - Installing Gentoo Live CD/USB from scratch and I created a script that installs Gentoo in a chroot environment, adds/compiles custom packages and creates squashfs and iso images. However I still have some issues that I'd like to solve.
I've used ext2 for the 65 MB partition but df -h shows I only have 61 MB available... Where are these missing 4 MB? BTW:
- Is there a filesystem that fits better than ext2?
- Can ext2 be tweaked to give more space?
My ultimate goal is an image that would be no bigger than Gentoo Minimal LIveCD, i.e. around 50-55MB. For now mine is 61MB, It is quite functional as I can boot with it and use it. I just want to optimize it a bit more.
I made my Live USB using a stage2 and my CFLAGS are "-Os -march=pentium4 -fomit-frame-pointer -pipe". However I still noticed files on my custom installation are bigger in size than those on Gentoo Minimal Live CD, i.e. by taking a look at files in /lib or /usr/lib.
- How did files Gentoo Minimal LiveCD get so much short?
- Should I consider using -march=pentium or the generic -march=i686?
The howto focuses on Grub to boot the live USB instead of syslinux. Meanwhile syslinux doesn't work with ext2.
So my questions - which I still could not answer yet - will be:
- Is ext2 really wasting precious space on the already small partition?
- Is there a better FS alternative with live USB?
- Would syslinux allow me to boot on the second partition instead of the first one?
- How can I reduce files size as much as Gentoo Minimal Live CD?
- Can I tweak CFLAGS or are mine good enough?
- Use Grub or syslinux?
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739!
Last edited by VinzC on Sat Oct 28, 2006 1:19 pm; edited 1 time in total |
|
| Back to top |
|
 |
Dlareh
Advocate
Joined: 06 Aug 2005
Posts: 2102
|
| Posted: Sat Oct 28, 2006 11:35 am Post subject: |
|
|
ext reserves 5% for the superuser; use tune2fs to change
reiser3 (and even more so, 4) are significantly more efficient for small files (both speed-wise and space-wise)
here's an example of a 20% space savings with r4
_________________
"Mr Thomas Edison has been up on the two previous nights discovering 'a bug' in his phonograph." --Pall Mall Gazette (1889)
Are we THERE yet?
Last edited by Dlareh on Sat Oct 28, 2006 12:43 pm; edited 1 time in total |
|
| Back to top |
|
|
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
| Posted: Sat Oct 28, 2006 12:19 pm Post subject: |
|
|
Thanks. But Grub doesn't support reiser4, does it?
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
| Back to top |
|
|
Dlareh
Advocate
Joined: 06 Aug 2005
Posts: 2102
|
| Posted: Sat Oct 28, 2006 12:38 pm Post subject: |
|
|
There's ways to get it working (a google turns up this).
If you don't want to bother, use r3. It's a significant improvement.
_________________
"Mr Thomas Edison has been up on the two previous nights discovering 'a bug' in his phonograph." --Pall Mall Gazette (1889)
Are we THERE yet? |
|
| Back to top |
|
|
hielvc
Advocate
Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca
|
|
| Back to top |
|
|
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
| Posted: Sat Oct 28, 2006 4:32 pm Post subject: |
|
|
| hielvc wrote: | | Remove " -fomit-frame-pointer " it makes the code a little larger. |
But... doesn't that flag remove debugging information (i.e. stack trace)???
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
| Back to top |
|
|
nixnut
Bodhisattva
Joined: 09 Apr 2004
Posts: 10974
Location: the dutch mountains
|
| Posted: Sat Oct 28, 2006 5:42 pm Post subject: |
|
|
Moved from Installing Gentoo to Unsupported Software.
Custom livecds/libusb is not supported stuff, so moved here.
_________________
Please add [solved] to the initial post's subject line if you feel your problem is resolved. Help answer the unanswered
talk is cheap. supply exceeds demand |
|
| Back to top |
|
|
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
| Posted: Sat Oct 28, 2006 6:01 pm Post subject: |
|
|
| nixnut wrote: | Moved from Installing Gentoo to Unsupported Software.
Custom livecds/libusb is not supported stuff, so moved here. |
I've always been puzzled/upset by the term "unsupported" - maybe because of commercial companies' use of that term... Should I expect no answer to my questions, then? Or consider "unsupported" differently in my mind?
BTW: Advocating the Devil - I find the reason a little funny "custom livecd is not supported stuff". But consider every person who potentially can customize Gentoo for his/her usage - it's Gentoo's way. There aren't two Gentoo installations that are the same yet they are "supported stuff", ain't they? In this case, consider it yet another customized Gentoo... My question had nothing to do with Customizing Live CD's or anything like that - although it's the context; it's just about making compiled files smaller and finding a best suited filesystem for USB sticks.
Anyway I accept your reasons. It's no good to fight the mods  .
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
| Back to top |
|
|
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
| Posted: Sat Oct 28, 2006 6:13 pm Post subject: |
|
|
I dropped Grub towards isolinux/syslinux and I got a little more space. The side effect is now the Live USB partition is FAT16, required by syslinux.
EDIT: There is also extlinux for using linux boot with ext2.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739!
Last edited by VinzC on Sun Oct 29, 2006 4:51 pm; edited 1 time in total |
|
| Back to top |
|
|
hielvc
Advocate
Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca
|
|
| Back to top |
|
|
Dlareh
Advocate
Joined: 06 Aug 2005
Posts: 2102
|
| Posted: Sun Oct 29, 2006 4:21 am Post subject: |
|
|
Using -fomit-frame-pointer results in smaller code AFAIK.
_________________
"Mr Thomas Edison has been up on the two previous nights discovering 'a bug' in his phonograph." --Pall Mall Gazette (1889)
Are we THERE yet? |
|
| Back to top |
|
|
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
| Posted: Sun Oct 29, 2006 9:09 am Post subject: |
|
|
Well, I've searched the forums again and again and I read quite contradicting posts: ones that say -fomit-frame-pointer results in significantly smaller binaries and others that say the opposite, i.e. binaries being a little bit (2-3%) larger.
Look, I don't know what to believe anymore  .
EDIT: I think I'll stay with -march=pentium4 -Os -pipe.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
| Back to top |
|
|
Dlareh
Advocate
Joined: 06 Aug 2005
Posts: 2102
|
| Posted: Sun Oct 29, 2006 9:54 am Post subject: |
|
|
Just as a random test I compiled grep /without/ both -fomit-frame-pointer and -momit-leaf-frame-pointer and it was 0.08% smaller. *shrug*
_________________
"Mr Thomas Edison has been up on the two previous nights discovering 'a bug' in his phonograph." --Pall Mall Gazette (1889)
Are we THERE yet? |
|
| Back to top |
|
|
hielvc
Advocate
Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca
|
| Posted: Sun Oct 29, 2006 3:13 pm Post subject: |
|
|
Ive some testing but its wasnt defenitive. Your mini usb gentoo would be a good test sample as oposed random builds of this and that.
_________________
An A-Z Index of the Linux BASH command line |
|
| Back to top |
|
|
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
| Posted: Sun Oct 29, 2006 4:49 pm Post subject: |
|
|
| hielvc wrote: | | Ive some testing but its wasnt defenitive. Your mini usb gentoo would be a good test sample as oposed random builds of this and that. |
Eer... I'm not sure I understand what you mean  . Do you mean you're interrested in my small-footprint LiveUSB Gentoo?
Note for now I've been able to make it run on a CD only; when I come to putting it on a USB stick, it boots but fails when it comes to load the squashfs image (with an error message "Invalid loop location /image.bin", image.bin being my squashfs image).
On the live USB ash shell, there is no /dev/loop/0; only a /dev/loop0 but it might hide a problem with running udev I think. I'm still hunting that problem down. Maybe using Gentoo 2006.1 Live CD kernel config. Let's see.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
| Back to top |
|
|
Dlareh
Advocate
Joined: 06 Aug 2005
Posts: 2102
|
| Posted: Sun Oct 29, 2006 10:38 pm Post subject: |
|
|
He means compile it with and without -fomit-frame-pointer to see which is version smaller.
_________________
"Mr Thomas Edison has been up on the two previous nights discovering 'a bug' in his phonograph." --Pall Mall Gazette (1889)
Are we THERE yet? |
|
| Back to top |
|
|
hielvc
Advocate
Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca
|
| Posted: Mon Oct 30, 2006 12:08 am Post subject: |
|
|
Yes that was what I was thinking. Then it dawn on me that I had a stage1 that I built to a stage3. If I updated it then did a "du -hs of /bin /sbin /lib /usr minus /usr/portage /proc /dev /var would give a base line for CFLAGS="-march=pentium3 -Os -pipe ". After Im thru updateing it I'll add -fomit-frame-pointer and do a --emptytree of it and see what size it is. That should be good enough for a valid comparission.
_________________
An A-Z Index of the Linux BASH command line |
|
| Back to top |
|
|
Archangel1
Veteran
Joined: 21 Apr 2004
Posts: 1212
Location: Work
|
| Posted: Mon Oct 30, 2006 7:55 am Post subject: |
|
|
You could look at using uclibc instead - it's a lot more compact than glibc, although you may have to work a bit harder to get it all going.
_________________
What are you, stupid? |
|
| Back to top |
|
|
gringo
Advocate
Joined: 27 Apr 2003
Posts: 3793
|
| Posted: Mon Oct 30, 2006 10:36 am Post subject: |
|
|
yes, i think so too, uclibc is the way to go, specially if you´re looking for something tiny. Take a look to the embedded stages.
In what the flags refer, i ussually compile for x86 so that it can boot on every pc alike box i can put my hands on 
cheers |
|
| Back to top |
|
|
RobertDavid
Apprentice
Joined: 08 Aug 2005
Posts: 271
Location: Czech Republic
|
| Posted: Mon Oct 30, 2006 11:11 am Post subject: |
|
|
So I do something similar and now I have 66MB livecd with Kdrive, FVWM, MC as fm, rxvt-unicode, elinks, some filesystem utils, ssh etc.
There are a lot of other files I know they are not needed, so I plan to use something like openlog to recognise which files are realy needed.
I have also remove perl! (coz is not needed in my livecd)
I have 1G usb stick and there 3 partitions
1. 5MB fat16 with syslinux (there is also extlinux also but it didnt work for me)
2. 90MB ext2 for livecd and also /root directory to store config (which is mounted during boot)
3.rest, fat16 due to compatibility with windows desktops
I have similar problem as there was writen, during initrd parsing it doesnt recognize /dev/loop/0, but I know the mistake, because it dont mount my /dev/sda3 to /newroot/mnt/cdrom.
I have look into initrd and this is linuxrc script think, while it seachig for file livecd on all the partitions that could be loaded, but I dont know why it didnt find livecd file in my /dev/sda3 (I have ext2 compiled-in kernel)?!?
So I need always type in "mount /dev/sda3 /newroot/mnt/cdrom" and then exit.
DONT use reiser3 or another journaling FS on your stick!! I think that ext2 is good enough.
It is uselles to speak about size optimization if there is one big file livecd.squashfs!!
PS: syslinux now is capable of image background and than is prettyer than grub:) (it can be regular jpeg, 640x480x16  )
_________________
Arch & Fluxbox & 2.6.24-rc-zen!!!!
robertek.brevnov.net |
|
| Back to top |
|
|
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
| Posted: Mon Oct 30, 2006 12:38 pm Post subject: |
|
|
| RobertDavid wrote: | | I have look into initrd and this is linuxrc script think, while it seachig for file livecd on all the partitions that could be loaded, but I dont know why it didnt find livecd file in my /dev/sda3 (I have ext2 compiled-in kernel)?!? |
I've run into the same problem and my USB stick was starting properly until it asked me to select the keyboard mapping (with genkernel --do-keymap-auto argument). Then I had the error message you are talking about.
I've been able to fix the error  . The problem lies in /usr/share/genkernel/generic/initrd.scripts line 75, function findcdmount(). I've commented out the lines that causes a live USB to be mounted with -t iso9660:
| Code: |
good_msg "Attempting to mount media:- ${x}"
#if [ -n "${CDROOT_DEV}" ]
#then
mount -r -t auto ${x} ${NEW_ROOT}/mnt/cdrom \
> /dev/null 2>&1
#else
# mount -r -t iso9660 ${x} ${NEW_ROOT}/mnt/cdrom \
# > /dev/null 2>&1
#fi
if [ "$?" = '0' ]
then
# Check for a LiveCD
if [ -e ${NEW_ROOT}/mnt/cdrom/${SUBDIR}/livecd ]
then
REAL_ROOT="${x}"
break
else
umount ${NEW_ROOT}/mnt/cdrom
fi
fi |
With the current genkernel scripts any Live CD/USB squashfs image is mounted with -t iso9660 instead of -t auto - because variable CDROOT_DEV is not set. I don't know how to circumvent that problem other than commenting out the above lines. Now I've got my custom LiveUSB to boot and work normally.
Here's the command line in syslinux.cfg:
| Code: | label gentoo
kernel vmlinuz
append root=/dev/ram0 init=/linuxrc vga=791 looptype=squashfs loop=/image.squashfs udev nodevfs cdroot dodmraid dobladecenter scandelay=5 real_root=/dev/loop0 initrd=initrd |
No need to say I'm happy for I've been banging my head against the walls for 2 days till I found this. I think this is a bug in file initrd.scripts (unless someone tells me the correct kernel argument to use). I'm planning to make a patch to define a kernel argument "liveusb" that would have the same effect.
Back to my "size" question, I think I've spared about 5 MB over 60 MB by compiling without -fomit-frame-pointer but I'm not 100% sure as I'm comparing only my squashfs image, which I've tweaked since the last time to skip more files.
So I still must redo the same compilation over again twice to get trusted results.
Anyway, this is what I'm using for my LiveUSB:
- Two partitions on the key, LiveUSB on the second one
- syslinux (instead of grub to save more space) and a FAT16 partition
- squashfs image is approx. 55 MB, fits on my 65 MB partition
I'm also planning to use ext2 instead of FAT and extlinux instead of syslinux.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739!
Last edited by VinzC on Mon Oct 30, 2006 8:38 pm; edited 1 time in total |
|
| Back to top |
|
|
hielvc
Advocate
Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca
|
| Posted: Mon Oct 30, 2006 3:49 pm Post subject: |
|
|
Ok here are the results of emerge world -e on a stage3 with and without -fomit-frame-pointer.
CFLAGS="-march=pentium3 -Os -fomit-frame-pointer -pipe" | Quote: | With -fomit-frame-pointer
for i in bin sbin lib usr ;do du -s /$i ;done
5848 /bin
4040 /sbin
5560 /lib
TOTAL 15,448k
cd /usr
tester usr # for i in bin sbin i686-pc-linux-gnu lib libexec local include ;do du
-s $i ;done
57276 bin
12844 sbin
2536 i686-pc-linux-gnu
268116 lib
13040 libexec
36 local
20912 include
TOTAL 374,760k |
CFLAGS="-march=pentium3 -Os -pipe"
Without -fomit-frame-pointer | Quote: | for i in bin sbin lib ;do du -s /$i ;done
5860 /bin
4056 /sbin
5560 /lib
TOTAL 15,476K
cd /usr
for i in bin sbin i686-pc-linux-gnu lib libexec local include ;do du -s $i ;done
57360 bin
12892 sbin
2536 i686-pc-linux-gnu
268204 lib
13040 libexec
36 local
20912 include
TOTAL 374,980K |
390456k WITHOUT
390208k WITH
a 248k difference with -fomit-frame-pointer being smaller.
_________________
An A-Z Index of the Linux BASH command line |
|
| Back to top |
|
|
VinzC
Watchman
Joined: 17 Apr 2004
Posts: 5098
Location: Dark side of the mood
|
| Posted: Mon Oct 30, 2006 8:30 pm Post subject: |
|
|
Hmmm... Not worth all that fuss, as a conclusion... Note it's written in the doc -fomit-frame-pointer is implied by Os whenever it doesn't mess with debugging information. A more significant result would have been, for instance O2 against Os. Both imply -fomit-frame-pointer. I'll check it with my Live USB if I have some time.
_________________
Gentoo addict: tomorrow I quit, I promise!... Just one more emerge...
1739! |
|
| Back to top |
|
|
Dlareh
Advocate
Joined: 06 Aug 2005
Posts: 2102
|
| Posted: Mon Oct 30, 2006 8:38 pm Post subject: |
|
|
-fomit-frame-pointer does mess with debugging information on x86, so it is not implied by either -O2 or -Os.
| Quote: | 390456k WITHOUT
390208k WITH
a 248k difference with -fomit-frame-pointer being smaller. |
So -fomit-frame-pointer was 0.064% smaller.
_________________
"Mr Thomas Edison has been up on the two previous nights discovering 'a bug' in his phonograph." --Pall Mall Gazette (1889)
Are we THERE yet?
Last edited by Dlareh on Mon Oct 30, 2006 8:43 pm; edited 1 time in total |
|
| Back to top |
|
|
RobertDavid
Apprentice
Joined: 08 Aug 2005
Posts: 271
Location: Czech Republic
|
| Posted: Mon Oct 30, 2006 8:42 pm Post subject: |
|
|
So I was not correct my livecd.squashfs is only 53 MB (with that stuff I mentioned above) and when I tryed some cleanings in /usr/share/i18n and /usr/lib/gconv I get 47MB and it is a big difference!!
So the point is not which flag to use and save a 200kB uncompressed but what to erase, what is not necessary at all!!
some of my upgraded scripts:
/root/livecd/source/del
| Code: | #!/bin/bash
rm -f USELESSFILELIST
equery f gcc | grep -v "* Contents of " | grep -v "^/usr/lib/gcc/i686-pc-linux-g
nu/4.1.1" > USELESSFILELIST
for i in `cat progs`
do
equery -q files ${i} >> USELESSFILELIST
done
|
/root/livecd/source/progs
| Code: | sys-apps/less
portage
app-portage/cfg-update
app-misc/ca-certificate
sys-devel/make
app-editors/nano
sys-kernel/genkernel
net-misc/rsync
dev-libs/popt
sys-devel/automake
app-admin/localepurge
sys-apps/slocate
x11-terms/xterm
app-arch/rpm2targz
x11-libs/libdrm
x11-proto/xf86driproto
app-portage/gentoolkit
sys-apps/debianutils
sys-devel/bison
sys-devel/gcc-config
sys-devel/autoconf
sys-devel/gnuconfig
sys-devel/autoconf-wrapper
sys-devel/m4
sys-devel/automake-wrapper
dev-python/pycrypto
sys-apps/sandbox
app-portage/udept
dev-lang/perl
app-admin/perl-cleaner
sys-devel/patch
dev-libs/mpfr
dev-libs/gmp
sys-devel/flex
sys-devel/binutils-config
sys-devel/libtool
sys-kernel/linux-headers
sys-devel/libperl
perl-core/Test-Harness
perl-core/PodParser |
this make a list of files that are unimportant and than use the script that is included in the livecd-from -scratch guide
If someone find out some other program that could be included in this list please reffer, also if someone have or make some script to erase all that cruft in i18n dirs.
_________________
Arch & Fluxbox & 2.6.24-rc-zen!!!!
robertek.brevnov.net |
|
| Back to top |
|
|
|
Unsupported Software
