| View previous topic :: View next topic |
| Author |
Message |
msalerno
Veteran
Joined: 17 Dec 2002
Posts: 1338
Location: Sweating in South Florida
|
 Posted: Wed Jul 07, 2010 1:33 am Post subject: |
 |
|
| After I had that issue, I had to restart the sqash_portage service since it had been unmounted. Now it's holding steady at 34 open files. |
|
| Back to top |
|
 |
ernov
n00b
Joined: 28 May 2010
Posts: 55
|
| Posted: Thu Aug 26, 2010 7:30 pm Post subject: |
|
|
| Is this guide up to date? I am not asking because I don't have that few hunderd mb on my hardisk, it's just Im planning to move my system to ssd. Is this howto a good way of treating ssd? |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6780
|
| Posted: Fri Aug 27, 2010 8:24 pm Post subject: |
|
|
| ernov wrote: | | Is this howto a good way of treating ssd? |
You mean concerning reducing of numbers of writes? It depends: It will reduce the numbers of writes if you put TMPDIR and DIR_CHANGE to something else than ssd. To put TMPDIR to a ramdisk is reasonable safe, but putting DIR_CHANGE to ramdisk means that after a power loss all changes are lost. |
|
| Back to top |
|
|
mattst88
Developer
Joined: 28 Oct 2004
Posts: 422
|
| Posted: Sun Jan 09, 2011 5:47 pm Post subject: |
|
|
So I tried this last night, using the ebuilds from mv's overlay.
I'm getting errors during emerge --sync like
rsync: failed to set times on "/usr/portage/app-benchmarks/gtkperf": No such file or directory (2)
which is quite disappointing since I see that these were reported a few months ago. And indeed, I see 623 files open in /proc/`pidof unionfs`/fd.
What am I supposed to do? Use the unionfs-fuse-0.25-9999 ebuild? I'd rather not install mercurial just for a single package. Is there a snapshot ebuild somewhere?
_________________
My Wiki page |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6780
|
| Posted: Sun Jan 09, 2011 6:10 pm Post subject: |
|
|
| mattst88 wrote: | | Use the unionfs-fuse-0.25-9999 ebuild? |
Yes; the bug is not fixed in earlier versions, and untils unionfs-fuse-0.25 is officially released, you will probably not find it in portage.
Another way is to use aufs2 (e.g. with the live-ebuild from the mv overlay). |
|
| Back to top |
|
|
mattst88
Developer
Joined: 28 Oct 2004
Posts: 422
|
| Posted: Tue Jan 11, 2011 10:40 pm Post subject: |
|
|
I hope newer versions fix a lot more than just that.
Ignoring that I can't emerge --sync, every time I do an emerge --prune, it somehow then thinks tons of other software isn't installed anymore.
This simply isn't worth it.
Also, I don't have any idea what's going on here, or what's causing it.
| Code: | Thinkpad pkg # pwd
/var/db.changes/pkg
Thinkpad pkg # ls -lh
total 8.0K
drwxr-xr-x 3 root root 4.0K Jan 11 16:35 app-emulation
crw-rwsr-- 1 32518 32808 1930, 1607 Feb 18 1970 dev-perlArchive-Zip-1.30
crw-rwsr-- 1 32518 32808 2063, 1567 Feb 18 1970 media-libsa52dec-0.7.4-r6
drwxr-xr-x 3 root root 4.0K Jan 11 13:59 media-video
b-ws-ws--- 1 32518 32808 2190, 1583 Feb 18 1970 x11-libscairo-1.8.10 |
_________________
My Wiki page |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6780
|
| Posted: Wed Jan 12, 2011 2:46 pm Post subject: |
|
|
| mattst88 wrote: | | Ignoring that I can't emerge --sync, every time I do an emerge --prune, it somehow then thinks tons of other software isn't installed anymore. |
Did you add least add the line
| Code: | | PORTAGE_RSYNC_EXTRA_OPTS="${PORTAGE_RSYNC_EXTRA_OPTS} --exclude=/.unionfs --exclude=.fuse_hidden*" |
to /etc/make.conf? It is not optimal, but better than nothing, and without it you are screwed.
| Quote: | | Also, I don't have any idea what's going on here, or what's causing it. |
Indeed, something is very broken here - of course, this causes your problems with emerge --prune. I never had such problems (on the other hand, I normally use aufs2). Probably, this broken setting is now already stored in your archive, so I see no chance that we can reproduce how this happened. This is very serious, since /var/db/pkg contains valuable data - you have to reemerge the broken packages to fix it (and probably this re-emerging will cause file collisions and may leave orphaned files). I just can recommend to not use unionfs-fuse for a valuable directory like /var/db/pkg any more, if it does not work reliably on your machine... |
|
| Back to top |
|
|
mattst88
Developer
Joined: 28 Oct 2004
Posts: 422
|
| Posted: Fri Feb 04, 2011 9:29 pm Post subject: |
|
|
I gave up on this, for now at least.
Yeah, I had that line in /etc/make.conf.
You mention this bug being fixed between 0.24 and 0.25, but I don't see any commits after 0.24. http://hg.podgorny.cz/unionfs-fuse/shortlog
So, I don't understand.
_________________
My Wiki page |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6780
|
| Posted: Sat Feb 05, 2011 12:13 pm Post subject: |
|
|
| mattst88 wrote: | | but I don't see any commits after 0.24. |
It seems that unionfs-fuse is now developed in mercurial, only. There is a live ebuild in the mv overlay. |
|
| Back to top |
|
|
mattst88
Developer
Joined: 28 Oct 2004
Posts: 422
|
| Posted: Sun Jul 10, 2011 4:31 am Post subject: |
|
|
So I gave this another try with a unionfs-fuse snapshot from mercurial.
I'm using it on my laptop where it seems to work fine, but I'm also using it on my portage NFS server.
There, I see lots of strange 'Manifest file not found: '/usr/portage/sys-fs/udev/Manifest'' and stale NFS file handle errors. Any ideas what's going on here?
Usually, unmounting and remounting /usr/portage/ on the clients temporarily resolves the problem.
_________________
My Wiki page |
|
| Back to top |
|
|
p04ty
n00b
Joined: 09 Apr 2011
Posts: 55
|
| Posted: Sat Sep 03, 2011 9:11 am Post subject: |
|
|
What about this message:
| Code: | /etc/init.d/squash_portage sync
squash_portage | * Use of the opts variable is deprecated and will be
squash_portage | * removed in the future.
squash_portage | * Please use extra_commands or extra_started_commands.
squash_portage | * Syncing portage tree ... |
|
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6780
|
| Posted: Sat Sep 03, 2011 5:05 pm Post subject: |
|
|
| p04ty wrote: | | What about this message ... |
This should be fixed in squash_dir-11.0. |
|
| Back to top |
|
|
p04ty
n00b
Joined: 09 Apr 2011
Posts: 55
|
| Posted: Sun Sep 04, 2011 6:36 am Post subject: |
|
|
| What's that? How would it compare to in-kernel squashfs? |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6780
|
| Posted: Sun Sep 04, 2011 7:32 am Post subject: |
|
|
| p04ty wrote: | | What's that? How would it compare to in-kernel squashfs? |
I do not understand your question.
squash_dir is essentially only an init-script which in the simplest configuration (when started) uses the in-kernel squashfs (together with aufs or unionfs-fuse or others) to mount a squashed directory rewritable and which (when stopped) recompresses the new directory. Of course, it expects the in-kernel squashfs (and aufs or unionfs-fuse or others) to work, although there are some fallbacks for emergency cases. (But I am not sure whether this was your question...) |
|
| Back to top |
|
|
F1r31c3r
Tux's lil' helper
Joined: 31 Aug 2007
Posts: 107
Location: UK
|
| Posted: Sat Sep 10, 2011 12:02 am Post subject: OpenRC Moving away from opts= to extra_commands= |
|
|
| p04ty wrote: | What about this message:
| Code: | /etc/init.d/squash_portage sync
squash_portage | * Use of the opts variable is deprecated and will be
squash_portage | * removed in the future.
squash_portage | * Please use extra_commands or extra_started_commands.
squash_portage | * Syncing portage tree ... |
|
Hey people this has become global across all init.d scripts for OpenRC. If you use baselayout 2 it is advisable as the warning suggests to change the "opts" to "extra_commands" but in doing this you make backward compatibility a problem. Here is the original comments on this.
http://git.overlays.gentoo.org/gitweb/?p=proj/openrc.git;a=commitdiff;h=df1f02ac848a010092df2d3d40b8828051522b4b
This means that its a good idea to update on one hand but it means this warning is not critical.
Here is how i upgraded the commands:
| Quote: |
# cd /etc/init.d
# grep -r "opts" *
|
This will display all files using the opts as statement. WARNING the opts that dont hove = on eg "$fsck_opts -R" are not to be changed only them that show up at the start of the scripts saying opts=
The above command shows the real opts that need changing.
Here is the quick dirty way to change them to extra_commands NOTE the = sign after the words dont leave it out or it will break your system.
| Quote: |
# grep -lr -e "opts=" * | xargs sed -i "s/opts=/extra_commands=/g"
|
Now double check it change them correctly:
| Quote: |
# grep -r "opts=" *
it should find nothing
# grep -r "extra_commands=" *
All the files should now be listed with the extra_commands and your good to go
|
Pick yourself a init.d script and restart it see if it works i chose iptables and sysklogd:
| Quote: |
# /etc/init.d/iptables restart
# /etc/init.d/sysklogd restart
|
They should all restart without the above named warnings and or any other warning or errors.
You have successfully migrated the opts variable on your baselayout 2 OpenRC gentoo box.
_________________
A WikI, A collection of mass misinformation based on opinion and manipulation by a deception of freedom.
If we know the truth, then we should be free from deception (John 8:42-47 ) |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6780
|
| Posted: Sat Sep 10, 2011 6:35 am Post subject: Re: OpenRC Moving away from opts= to extra_commands= |
|
|
| F1r31c3r wrote: | | but in doing this you make backward compatibility a problem. |
Since baselayout-1 is now gone from the tree and meanwhile even the oldest openrc in the tree supports extra_commands, I would not worry about backward compatibility. However, with patching init-scripts from the tree, I would wait until they are officially bumped (which does not necessarily mean a revision bump of the corresponding package - you have to check manually). |
|
| Back to top |
|
|
kyoshiro
n00b
Joined: 23 Jun 2011
Posts: 1
Location: Beck'sTown
|
| Posted: Sun Sep 11, 2011 11:32 am Post subject: Re: OpenRC Moving away from opts= to extra_commands= |
|
|
| F1r31c3r wrote: | | p04ty wrote: | What about this message:
| Quote: |
# cd /etc/init.d
# grep -r "opts" *
|
This will display all files using the opts as statement. WARNING the opts that dont hove = on eg "$fsck_opts -R" are not to be changed only them that show up at the start of the scripts saying opts=
The above command shows the real opts that need changing.
Here is the quick dirty way to change them to extra_commands NOTE the = sign after the words dont leave it out or it will break your system.
| Quote: |
# grep -lr -e "opts=" * | xargs sed -i "s/opts=/extra_commands=/g"
|
Now double check it change them correctly:
| Quote: |
# grep -r "opts=" *
it should find nothing
# grep -r "extra_commands=" *
All the files should now be listed with the extra_commands and your good to go
|
|
|
Please have a special look at files containing ${opts} references, e.g. /etc/init.d/reboot.sh aso.
You'll have to fix them afterwards:
# grep -lr -e "{opts}" * | xargs sed -i "s/{opts}/{extra_commands}/g" |
|
| Back to top |
|
|
SlashBeast
Retired Dev
Joined: 23 May 2006
Posts: 2922
|
| Posted: Sun Sep 11, 2011 2:28 pm Post subject: |
|
|
| Code: | | sed -e 's:opts=:extra_commands=:g' -e 's:${opts}:${extra_commands}:g' -e 's:$opts:$extra_commands:g' /etc/init.d/* -i |
|
|
| Back to top |
|
|
lpapa
n00b
Joined: 19 Sep 2011
Posts: 1
|
| Posted: Mon Sep 19, 2011 1:48 am Post subject: |
|
|
| SlashBeast wrote: | | Code: | | sed -e 's:opts=:extra_commands=:g' -e 's:${opts}:${extra_commands}:g' -e 's:$opts:$extra_commands:g' /etc/init.d/* -i |
|
Still doesn't work for some scripts, for example, sshd script uses variable "myopts" and then refers to it as ${myopts}.
I fix it adding -e 's:${myopts}:${myextra_options}:g' but I don't know if this may affect other packages which I didn't installed on my system. |
|
| Back to top |
|
|
Stolz
Moderator
Joined: 19 Oct 2003
Posts: 3028
Location: Hong Kong
|
| Posted: Tue Oct 11, 2011 11:00 pm Post subject: |
|
|
| In order to reduce the writes of my future SSD disk I've successfully installed squash_dir via layman and configured it to squash both /urs/portage and /usr/src, saving all the files in my rotational disk. Are there any other directories that are good candidates to be squashed? may be /usr/share/doc?. With "good candidates" I mean they use have a lot of small text files and use to involve lot of writes to the disk but also they are not of vital importance. |
|
| Back to top |
|
|
Dont Panic
Guru
Joined: 20 Jun 2007
Posts: 322
Location: SouthEast U.S.A.
|
| Posted: Wed Oct 12, 2011 3:19 pm Post subject: |
|
|
| Stolz wrote: | | Are there any other directories that are good candidates to be squashed? may be /usr/share/doc?. With "good candidates" I mean they use have a lot of small text files and use to involve lot of writes to the disk but also they are not of vital importance. |
I was going to suggest /var/db/pkg
However, that directory doesn't meet your criteria for "not of vital importance". Bad things will happen if the information in /var/db/pkg gets out of sync or is clobbered.  |
|
| Back to top |
|
|
Stolz
Moderator
Joined: 19 Oct 2003
Posts: 3028
Location: Hong Kong
|
| Posted: Wed Oct 12, 2011 3:54 pm Post subject: |
|
|
| Dont Panic wrote: | I was going to suggest /var/db/pkg
However, that directory doesn't meet your criteria for "not of vital importance". Bad things will happen if the information in /var/db/pkg gets out of sync or is clobbered. |
Thanks Dont Panic, I'm aware of it, the squash_dir README mentions it, but since it's dangerous to play with it and my raid disks already have some S.M.A.R.T errors I prefer to leave it alone |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6780
|
| Posted: Wed Oct 12, 2011 4:23 pm Post subject: |
|
|
| Stolz wrote: | | However, that directory doesn't meet your criteria for "not of vital importance". |
Actually, I have good experience with squashing it with keeping a backup (setting FILE_SQFS_OLD). This backup even saved my installation once when a kernel panic caused damage in /var/db.
/usr/share/doc or even /usr/share is a good candidate for squashing from the viewpoint of compression ratio, but there is an enormous drawback: These directories are huge and change with every package upgrade. So you should put a huge THRESHOLD and then need sufficient disk space to have the compressed directory twice on disk together with the THRESHOLD (since, of course, it will be first compresssed and the old squash removed only afterwards). Also, many things in /usr/share are somewhat vital.
Other good candidates: /usr/share/games /usr/share/texmf-dist /usr/lib/libreoffice (the first two are mentioned in the README).
Note that if you compress a lot of directories you must increase the number of loopback-devices:
Put e.g. | Code: | | options loop max_loop=16 |
into /etc/modprobe.d/loop.conf |
|
| Back to top |
|
|
gringo
Advocate
Joined: 27 Apr 2003
Posts: 3793
|
| Posted: Fri Oct 14, 2011 1:21 pm Post subject: |
|
|
Stolz pointed me to this thread and i have it now running in one of my systems, thanks a lot !
Only one thing though : if you enable the kernel-patch USE in aufs from mv overlay it will also apply the grsec one and if the kernel isnt grsec enabled it will fail to build here.
Maybe a grsec USE is needed or sth. ?
oh, and thanks also for the zram-init stuff, ive been using it for some time already
cheers |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6780
|
| Posted: Sat Oct 15, 2011 3:55 pm Post subject: |
|
|
| gringo wrote: | | if the kernel isnt grsec enabled it will fail to build here. |
The patch should be fixed now to build in all cases. |
|
| Back to top |
|
|
|
Documentation, Tips & Tricks
