| View previous topic :: View next topic |
| Author |
Message |
mv
Watchman
Joined: 20 Apr 2005
Posts: 6780
|
 Posted: Thu Sep 13, 2007 12:13 pm Post subject: |
 |
|
New POSIX compatible release 5.2
(For those who downloaded in the previous days/weeks: For some reason there was not 5.1 on the server but an older version).
| synss wrote: | | baselayout-2 is nearing completion and will only use POSIX shell initscripts, i.e. no bashisms. |
I was in the opinion that also the previous version of the script did not use any bashisms, but I had missed some things which were not POSIX compatible. Now the thing has been tested with dash. |
|
| Back to top |
|
 |
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Thu Sep 13, 2007 2:12 pm Post subject: |
|
|
| mv wrote: | | I was in the opinion that also the previous version of the script did not use any bashisms, but I had missed some things which were not POSIX compatible. Now the thing has been tested with dash. |
Woot! Well done mv :D |
|
| Back to top |
|
|
synss
Apprentice
Joined: 08 Mar 2006
Posts: 282
Location: Dijon > Berlin > Tokyo > Nürnberg > München
|
| Posted: Fri Sep 14, 2007 5:09 am Post subject: |
|
|
| mv wrote: | New POSIX compatible release 5.2
(For those who downloaded in the previous days/weeks: For some reason there was not 5.1 on the server but an older version).
| synss wrote: | | baselayout-2 is nearing completion and will only use POSIX shell initscripts, i.e. no bashisms. |
I was in the opinion that also the previous version of the script did not use any bashisms, but I had missed some things which were not POSIX compatible. Now the thing has been tested with dash. |
That was fast, thanks!
EDIT: just to say I have just tried it and it works well with dash. (But my system does not really boot faster.)
_________________
Compress portage tree
Elog viewer
Autodetect swap |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6780
|
| Posted: Tue Jan 01, 2008 9:46 pm Post subject: |
|
|
There is a new version of the initscript available at the initscripts. (The most important bugfix is that an unmount error is now not mistakenly ignored.)
Moreover, also a simple script can be found there which treats several of the corresponding initscripts simultaneously, if you want to recompress/start/stop/view the status of them manually.
Finally, there is also a patch for squashfs-tools-3.2_p2 which redirects the progress bar of mksquashfs to stderr. The advantage is that the init-script will then display the progress bar without printing additional confusing information (but you can also have the latter without the patch with the new version of the script).
If you use the script note that squashfs-tools-3.3 is meanwhile in the portage tree but will create a format which seems to be incompatible with the current gentoo kernel patchset, so you should not upgrade to squashfs-tools-3.3 in the moment... |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6780
|
| Posted: Sun Jan 06, 2008 4:13 pm Post subject: |
|
|
| There is again a new version of the script available. Now it is possible to ignore certain files/dirs resp. ignore "touches" of these files/dirs (in the main directory). This is important if you squash a tex directory and want to avoid unnecessary re-squashing just because one of the system tools updated "ls-R" without actually making any changes to that file. |
|
| Back to top |
|
|
Darknight
Guru
Joined: 26 Jan 2004
Posts: 483
Location: Italy
|
| Posted: Mon Mar 24, 2008 11:32 pm Post subject: |
|
|
| Thanks for this nice script. |
|
| Back to top |
|
|
188562
Apprentice
Joined: 22 Jun 2008
Posts: 186
|
| Posted: Mon Jun 23, 2008 3:54 pm Post subject: Thanks |
|
|
synss Thank you very much for the idea and its embodiment!
My five cents. At the moment I use sys-kernel/zen-sources-2.6.26_rc6-r10 I had a little rewrite your script to aufs all proper correction affected only lines
#mount -t unionfs -o nodev,noexec,dirs=$PORTAGE_RW=rw:$PORTDIR=ro unionfs $PORTDIR
mount -t aufs -o nodev,noexec,dirs=$PORTAGE_RW=rw:$PORTDIR=ro aufs $PORTDIR
After that, everything worked! The truth is now at the time of each file portage.sqfs re-rebooting again this takes some time. |
|
| Back to top |
|
|
synss
Apprentice
Joined: 08 Mar 2006
Posts: 282
Location: Dijon > Berlin > Tokyo > Nürnberg > München
|
| Posted: Tue Jun 24, 2008 1:23 am Post subject: Re: Thanks |
|
|
| init_6 wrote: | synss Thank you very much for the idea and its embodiment!
My five cents. At the moment I use sys-kernel/zen-sources-2.6.26_rc6-r10 I had a little rewrite your script to aufs all proper correction affected only lines
#mount -t unionfs -o nodev,noexec,dirs=$PORTAGE_RW=rw:$PORTDIR=ro unionfs $PORTDIR
mount -t aufs -o nodev,noexec,dirs=$PORTAGE_RW=rw:$PORTDIR=ro aufs $PORTDIR
After that, everything worked! The truth is now at the time of each file portage.sqfs re-rebooting again this takes some time. |
Thank you! now I am not sure I understand your last sentence correctly: you do not need to reboot to update the image, you only need to restart the initscript, which can be done in a monthly cron job, too, BTW.
_________________
Compress portage tree
Elog viewer
Autodetect swap |
|
| Back to top |
|
|
188562
Apprentice
Joined: 22 Jun 2008
Posts: 186
|
| Posted: Tue Jun 24, 2008 3:24 pm Post subject: |
|
|
Every time I do a "reboot" or "halt" or "poweroff" script re-creates file portage.sqfs so if I update portage "emerge --sync"
Most likely this feature aufs but I'm not sure. |
|
| Back to top |
|
|
IvanMajhen
Guru
Joined: 10 Jun 2006
Posts: 392
Location: Croatia
|
| Posted: Tue Jun 24, 2008 11:38 pm Post subject: |
|
|
Add this | Code: | stop() {
ebegin "Updating portage tree"
rm -rf /dev/shm/.portage-rw/.wh..wh.plink
rm /dev/shm/.portage-rw/.wh..wh.aufs
[ ! $PORTAGE_RW ] && PORTAGE_RW="${DEF_RW}"
if [ ! -z `ls -A $PORTAGE_RW | head -n1` ]
then |
|
|
| Back to top |
|
|
188562
Apprentice
Joined: 22 Jun 2008
Posts: 186
|
| Posted: Sat Jun 28, 2008 6:29 am Post subject: |
|
|
So here is the corrected version of the script for squashfs & aufs. All except out in Section stop working as it should work.
| Code: | #!/sbin/runscript
source /etc/make.globals
SQFS_CUR="$SQFS_DIRNAME/portage.sqfs"
SQFS_NEW="$SQFS_DIRNAME/portage-current.sqfs"
SQFS_OLD="$SQFS_DIRNAME/portage-old.sqfs"
DEF_RW="/dev/shm/.portage-rw"
depend() {
need localmount
}
start() {
ebegin "Mounting read-only squashfs image"
mount -rt squashfs -o loop $SQFS_CUR $PORTDIR
retval=$?
eend $retval
[ $retval -ne 0 ] && return $retval
ebegin "Mounting read-write with aufs"
if [ ! $PORTAGE_RW ]
then einfo " mounted in tmpfs (RAM)"
PORTAGE_RW="${DEF_RW}"
fi
[ -d $PORTAGE_RW ] || mkdir -p $PORTAGE_RW
chmod 0750 $PORTAGE_RW
chown portage:portage $PORTAGE_RW
mount -t aufs -o nodev,noexec,dirs=$PORTAGE_RW=rw:$PORTDIR=ro aufs $PORTDIR
eend $?
}
stop() {
# ebegin "Updating portage tree"
# [ ! $PORTAGE_RW ] && PORTAGE_RW="${DEF_RW}"
# if [ ! -z `ls -A $PORTAGE_RW | head -n1` ]
# then
# einfo " Syncing the tree"
# mv -f $SQFS_NEW $SQFS_OLD
# mksquashfs $PORTDIR $SQFS_NEW -no-duplicates 2>/dev/null
# retval=$?
# ln -sf $SQFS_NEW $SQFS_CUR
# else
# einfo " Nothing to do"
# retval=0
# fi
# eend $retval
ebegin "Unmounting the tree"
umount -t aufs $PORTDIR
umount -t squashfs $PORTDIR
rm -rf $PORTAGE_RW
eend 0
} |
Here is this condition (if) why it is always true
| Quote: | # ebegin "Updating portage tree"
# [ ! $PORTAGE_RW ] && PORTAGE_RW="${DEF_RW}"
# if [ ! -z `ls -A $PORTAGE_RW | head -n1` ] |
|
|
| Back to top |
|
|
IvanMajhen
Guru
Joined: 10 Jun 2006
Posts: 392
Location: Croatia
|
| Posted: Sun Jun 29, 2008 12:55 pm Post subject: |
|
|
This is wrong. Sqfs wont be updated when synced.
This is how my stop function looks:
| Code: | stop() {
ebegin "Updating portage tree"
rm -rf /dev/shm/.portage-rw/.wh..wh.plink
rm /dev/shm/.portage-rw/.wh..wh.aufs
[ ! $PORTAGE_RW ] && PORTAGE_RW="${DEF_RW}"
if [ ! -z `ls -A $PORTAGE_RW | head -n1` ]
then
einfo " Syncing the tree"
mv -f $SQFS_NEW $SQFS_OLD
mksquashfs $PORTDIR $SQFS_NEW -no-duplicates 2>/dev/null
retval=$?
ln -sf $SQFS_NEW $SQFS_CUR
else
einfo " Nothing to do"
retval=0
fi
eend $retval
ebegin "Unmounting the tree"
umount -t aufs $PORTDIR
umount -t squashfs $PORTDIR
rm -rf $PORTAGE_RW
eend 0
}
|
|
|
| Back to top |
|
|
NaiL
Apprentice
Joined: 13 Feb 2003
Posts: 228
Location: Spain/BCN
|
| Posted: Sun Dec 14, 2008 11:56 am Post subject: |
|
|
I have using the MV's script for some time, (currently the last version).
I found a problem stopping the init script.
| starting the script wrote: | /etc/init.d/squash_portage start
* Mounting /var/tmp/portage-current.sqfs as /usr/portage ... [ ok ] |
| mount command after the start wrote: | mount
/dev/sda3 on / type reiserfs (rw,noatime,notail)
/proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec)
udev on /dev type tmpfs (rw,nosuid,size=10240k,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,gid=5,mode=620)
shm on /dev/shm type tmpfs (rw,noexec,nosuid,nodev)
usbfs on /proc/bus/usb type usbfs (rw,noexec,nosuid,devmode=0664,devgid=85)
securityfs on /sys/kernel/security type securityfs (rw,noexec,nosuid,nodev)
/dev/loop0 on /tmp/squash_dir.readonly.4TQ2X8 type squashfs (ro)
aufs on /usr/portage type aufs (rw,xino=/dev/shm/portage/.aufs.xino,br:/dev/shm/portage=rw:/tmp/squash_dir.readonly.4TQ2X8=ro)
|
| stopping the script wrote: | # /etc/init.d/squash_portage stop
* Unmounting /usr/portage ...
umount: /var/tmp/portage-current.sqfs: not mounted
* umount -d -- /var/tmp/portage-current.sqfs failed [exit with 1] [ !! ] |
| mount command after the stop wrote: | mount
/dev/sda3 on / type reiserfs (rw,noatime,notail)
/proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec)
udev on /dev type tmpfs (rw,nosuid,size=10240k,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,gid=5,mode=620)
shm on /dev/shm type tmpfs (rw,noexec,nosuid,nodev)
usbfs on /proc/bus/usb type usbfs (rw,noexec,nosuid,devmode=0664,devgid=85)
securityfs on /sys/kernel/security type securityfs (rw,noexec,nosuid,nodev)
/dev/loop0 on /tmp/squash_dir.readonly.4TQ2X8 type squashfs (ro) |
| the config file wrote: | # cat /etc/conf.d/squash_portage
# /etc/conf.d/squash_portage
DIRECTORY="/usr/portage"
DIR_CHANGE="/dev/shm/portage"
FILE_SQFS="/var/tmp/portage-current.sqfs"
FILE_SQFS_OLD="/var/tmp/portage-old.sqfs"
DIR_TMP="/dev/shm" |
Maybe is it a problem of configuration? |
|
| Back to top |
|
|
Darknight
Guru
Joined: 26 Jan 2004
Posts: 483
Location: Italy
|
| Posted: Wed Feb 25, 2009 2:49 pm Post subject: |
|
|
I have the same problem. As far as I know it used to work perfectly until some days ago but I may have missed that error for some time. Unfortunately I can't pinpoint any update, nor change the script to make it work with the new stuff...
EDIT: It has to do with temporary filenames. Set DIR_SQUASH as a workaround. |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6780
|
| Posted: Wed Feb 25, 2009 3:47 pm Post subject: |
|
|
| Darknight wrote: | | I have the same problem |
Sorry, somehow I had missed NaiL's posting.
Anyway, I cannot reproduce the problem on my machine if I use exactly NaiL's config (only with a different DIRECTORY for testing purposes).
Do you use the most current version (7.4) of the script? (The version is in a comment near the beginning).
However, as remarked in the documentation, using the temporary filename feature for DIR_SQUASH is always hackish: It depends on the undocumented /etc/mtab format and thus it might depend on the mount command (i.e. on the util-linux version) whether this will work. During my test just now I had util-linux-2.14.1 installed: The critical part is whether in line 686-688 the correct filename can be read from /etc/mtab by the sed-expression. |
|
| Back to top |
|
|
Darknight
Guru
Joined: 26 Jan 2004
Posts: 483
Location: Italy
|
| Posted: Wed Feb 25, 2009 4:09 pm Post subject: |
|
|
I have the same util-linux version but somehow sed fails (I had nailed it, but I have not bash skillz to modify the script).
Anyway re-reading through the documentation gave me a solution that is good enough for me. |
|
| Back to top |
|
|
NaiL
Apprentice
Joined: 13 Feb 2003
Posts: 228
Location: Spain/BCN
|
| Posted: Sat Feb 28, 2009 2:43 pm Post subject: |
|
|
Hi,
I'm using the 7.4 version of the script and:
sys-apps/util-linux-2.14.1
sys-fs/squashfs-tools-3.3
sys-fs/aufs-20081208-r1
sys-kernel/gentoo-sources-2.6.27-r8
Do you need any other information? |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6780
|
| Posted: Sat Feb 28, 2009 3:58 pm Post subject: |
|
|
NaiL, apparently, the problem is your strange output of
| mount wrote: | | /dev/loop0 on /tmp/squash_dir.readonly.4TQ2X8 type squashfs (ro) |
This shouldn't be so. It should be instead something like
| mount wrote: | | /var/tmp/portage-current.sqfs on /tmp/squash_dir.readonly.4TQ2X8 type squashfs (ro,loop=/dev/loop0) |
I cannot understand why your output is so. What happens after you do manually
| Code: | mkdir /some_directory
mount -t squashfs -o loop,ro /var/tmp/portage-current.sqfs /some_directory |
If this works as expected, the output of mount should then contain
| mount wrote: | | /var/tmp/portage-current.sqfs on /some_directory type squashfs (ro,loop=/dev/loop*) |
and afterwards you should be able to umount by
| Code: | | umount /var/tmp/portage-current.sqfs |
(of course, afterwards, you can remove /some_directory).
Perhaps it makes a change if you upgrade to sys-fs/squashfs-tools-3.4? |
|
| Back to top |
|
|
NaiL
Apprentice
Joined: 13 Feb 2003
Posts: 228
Location: Spain/BCN
|
| Posted: Sun Mar 01, 2009 12:19 pm Post subject: |
|
|
| Quote: | I cannot understand why your output is so. What happens after you do manually
| Code: | mkdir /some_directory
mount -t squashfs -o loop,ro /var/tmp/portage-current.sqfs /some_directory |
If this works as expected, the output of mount should then contain
| mount wrote: | | /var/tmp/portage-current.sqfs on /some_directory type squashfs (ro,loop=/dev/loop*) |
and afterwards you should be able to umount by
| Code: | | umount /var/tmp/portage-current.sqfs |
(of course, afterwards, you can remove /some_directory).
|
The result was:
| Quote: | | /dev/loop1 on /some_directory type squashfs (ro) |
| Quote: | | Perhaps it makes a change if you upgrade to sys-fs/squashfs-tools-3.4? |
Affter the upgrade i get the same output. |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6780
|
| Posted: Sun Mar 01, 2009 12:44 pm Post subject: |
|
|
NaiL, I have no idea what is responsible on your system that the same mount-command (with same versions of mount and squashfs than here) works differently on your system. Do you have perhaps some of these terrible hal things activated or some special udev rules which could cause this behavior? Here are my useflags for util-linux: | Code: | | crypt loop-aes nls slang unicode -old-linux -selinux -uclibc |
|
|
| Back to top |
|
|
NaiL
Apprentice
Joined: 13 Feb 2003
Posts: 228
Location: Spain/BCN
|
| Posted: Mon Mar 02, 2009 6:58 pm Post subject: |
|
|
Tnx! I finally found the problem.
I was some how related to "loop-aes" USE flag.
I had this USE flag disabled. By some reason it has to be activated, may be it will be good to document that some where.
I also noticed that. 
| Quote: | * The loop-aes code has been split out of USE=crypt and into USE=loop-aes.
* If you need support for it, make sure to update your USE accordingly. |
|
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6780
|
| Posted: Mon Mar 02, 2009 11:04 pm Post subject: |
|
|
| In the new release of the script the temporary file name feature was removed for DIR_SQUASH. Instead, now a sane default is chosen for DIR_SQUASH in /var/run. Indeed, since even with the temporary file name feature the thing could only work by using a non-temporary file (in this case /etc/mtab), it is probably more reasonable to force directly to use a non-temporary directory. In this way the implementation is independent of undocumented behavior of the mount command. |
|
| Back to top |
|
|
jowr
n00b
Joined: 27 Dec 2008
Posts: 52
|
| Posted: Tue Mar 03, 2009 7:21 am Post subject: |
|
|
| Where can I get a squashfs utils that makes a SquashFS 4 file system? Apparently 2.6.29 now only has v4, and there are no [known to me..] userland utils for making v4 file systems. |
|
| Back to top |
|
|
Dont Panic
Guru
Joined: 20 Jun 2007
Posts: 322
Location: SouthEast U.S.A.
|
| Posted: Wed Apr 01, 2009 6:50 am Post subject: |
|
|
I've just upgraded to a 2.6.29 kernel, and I am now getting errors like this when trying to mount the squashfs:
| Code: | | SQUASHFS error: Major/Minor mismatch, older Squashfs 3.1 filesystems are unsupported |
I may not be understanding correctly, but the information I'm reading on the Internet indicates that backward compatibility has been broken for squashfs as part of the process for bringing squashfs into the mainline kernel.
Here's a post from the author of squashfs:
http://lkml.indiana.edu/hypermail/linux/kernel/0901.1/02601.html
I'm not finding much information about this on squashfs' homepage on sourceforge
And I also see the entire set of sys-fs/squashfs-tools ebuilds have been hard-masked (as of the time of this post).
This looks like a bumpy ride for my squashfs portage directory. |
|
| Back to top |
|
|
Darknight
Guru
Joined: 26 Jan 2004
Posts: 483
Location: Italy
|
| Posted: Wed Apr 01, 2009 7:46 am Post subject: |
|
|
| When I tried aufs didn't compile against 2.6.29, therefore I'm back to 2.6.27 for now. |
|
| Back to top |
|
|
|
Documentation, Tips & Tricks
