| View previous topic :: View next topic |
| Author |
Message |
alsterg
n00b
Joined: 29 Mar 2005
Posts: 1
|
 Posted: Tue Mar 29, 2005 10:07 pm Post subject: Improve linux understanding by categorizing files by package |
 |
|
One of the characteristics of unix-like OSs (contrary to what a user could be used in MS-Windows) is that there are certain directories for certain porposes which all installed programs must share. For instance, there are certain directories where executables must reside (/bin, /sbin, ...). This fact makes it difficult to monitor the files installed and know at any time which package is responsible for which file.
I submit a patch which helps on this matter. It makes sure that all files of a package that is been installed are copied (preserving the directory structure) to a dedicated for this package directory, under /var/tmp/portage/progs/.
The directory is: /var/tmp/portage/progs/<category>/<package_name_version>
For example: /var/tmp/portage/progs/sys-devel/gcc-3.4.1-20050301
It is important to state that those copied files do not take extra space on your hard disk, because they are hardlinks to the actual installed files.
This way you can clearly see the impact on the filesystem of each package, and improve your understanding of your linux box.
TECHNICAL DETAILS:
Nothing really tricky.. This patch just adds an extra step in the emerge proccess, in which hardlinks of files in ${D}/* are created in /var/tmp/portage/${category}/${PF}. Directory structure is preserved. If it's about an update, it removes the hardlinks of the updated package. If it's an unmerge, it removes the hardlinks of the unmerged package. The files that are patched are: portage/bin/ebuild.sh and portage/bin/emerge.
Restrictions:
1. Currently the top directory /var/tmp/portage/progs/ is hardcoded in the patch. I know that it's ugly, but it seems that it's not possible to use something outside /var/tmp/portage/ due to permition restrictions. Of course there must be a way, but I haven't spend much time to find this hack.
2. Since we hardlink to the original files, the /var/tmp/portage/progs/ must be in the same filesystem as all installed files. Virtually, this means that almost all directories like /bin, /sbin , etc.. must be in the same filesystem as /var/tmp/portage/progs/. This is very restrictive for most cases, since having different fs for some top dirs is strongly adviced. You can easily change the patch to create symlinks instead of hardlinks. I just chose hardlinks because when browsing under /var/tmp/portage/progs/, files are properly colored, whereas as symlinks, they are all colored the same.
al_ex
portage-hardlinks.patch
| Quote: |
diff -c -N -r portage-2.0.51.19.old/bin/ebuild.sh portage-2.0.51.19.new/bin/ebuild.sh
*** portage-2.0.51.19.old/bin/ebuild.sh Sat Feb 26 11:22:37 2005
--- portage-2.0.51.19.new/bin/ebuild.sh Tue Mar 29 19:54:39 2005
***************
*** 1115,1120 ****
--- 1115,1137 ----
unset -f stat_perms
+ # Portage creates hardlinks of the to-be-installed files, for convenience.
+ # The default directory is /usr/<name-of-the-package-plus-version>
+ EarlierVersion=`emerge --pretend ${CATEGORY}/${PN} | grep '\[ebuild' | sed 's/^\[ebuild.....U.\] \([^ ]*\) \[\([^ ]*\)\] $/\1 \2/g' | sed 's/^\([^ ]*\)\-r[0-9]* \([^ ]*\)$/\1 \2/g' | sed 's/^\([^ ]*\)\-[^ ]* \([^ ]*\)$/\1-\2/g' | grep -v '\[ebuild'`
+ if [ -n "${EarlierVersion}" ]; then
+ echo -n ">>> Removing hardlinks of ${EarlierVersion} ..."
+ rm -rf "${BUILDDIR}/../progs/${EarlierVersion}" && echo OK
+ fi
+ echo -n ">>> Creating hardlinks for ${CATEGORY}/${PF} ..."
+ if [ ! -d "${BUILDDIR}/../progs/${CATEGORY}/${PF}" ]; then
+ install -d "${BUILDDIR}/../progs/${CATEGORY}/${PF}"
+ else
+ rm -rf "${BUILDDIR}/../progs/${CATEGORY}/${PF}"
+ install -d "${BUILDDIR}/../progs/${CATEGORY}/${PF}"
+ fi
+ (cp -al ${D}/* ${BUILDDIR}/../progs/${CATEGORY}/${PF}/ && echo OK) \
+ || echo ">>> WARNING: Problem encounter while creating hardlinks.."
+
# Portage regenerates this on the installed system.
if [ -f "${D}/usr/share/info/dir.gz" ]; then
rm -f "${D}/usr/share/info/dir.gz"
diff -c -N -r portage-2.0.51.19.old/bin/emerge portage-2.0.51.19.new/bin/emerge
*** portage-2.0.51.19.old/bin/emerge Wed Mar 2 07:21:14 2005
--- portage-2.0.51.19.new/bin/emerge Tue Mar 29 19:54:20 2005
***************
*** 2184,2190 ****
--- 2184,2197 ----
print ">>> Unmerging "+y+"..."
emergelog("=== Unmerging... ("+y+")")
mysplit=string.split(y,"/")
+
#unmerge...
+ for root, dirs, files in os.walk("/var/tmp/portage/progs/"+y, topdown=False):
+ for name in files:
+ os.remove(os.path.join(root, name))
+ for name in dirs:
+ os.rmdir(os.path.join(root, name))
+ os.rmdir("/var/tmp/portage/progs/"+y)
retval=portage.unmerge(mysplit[0],mysplit[1],portage.root,mysettings,unmerge_action not in ["clean","prune"])
if retval:
emergelog(" !!! unmerge FAILURE: "+y)
|
This is patch for portage sources. In order to apply it (taking for granted that you use the emerge facility), you have to edit the portage ebuild and extend it to use this patch. Otherwise you can patch an already installed portage package with the following patches to ebuild.sh and emerge respectively.
ebuild.sh-hardlink.patch
| Quote: |
*** ebuild.sh.old Tue Mar 29 19:49:52 2005
--- ebuild.sh Tue Mar 29 19:43:36 2005
***************
*** 1115,1120 ****
--- 1115,1137 ----
unset -f stat_perms
+ # Portage creates hardlinks of the to-be-installed files, for convenience.
+ # The default directory is /usr/<name-of-the-package-plus-version>
+ EarlierVersion=`emerge --pretend ${CATEGORY}/${PN} | grep '\[ebuild' | sed 's/^\[ebuild.....U.\] \([^ ]*\) \[\([^ ]*\)\] $/\1 \2/g' | sed 's/^\([^ ]*\)\-r[0-9]* \([^ ]*\)$/\1 \2/g' | sed 's/^\([^ ]*\)\-[^ ]* \([^ ]*\)$/\1-\2/g' | grep -v '\[ebuild'`
+ if [ -n "${EarlierVersion}" ]; then
+ echo -n ">>> Removing hardlinks of ${EarlierVersion} ..."
+ rm -rf "${BUILDDIR}/../progs/${EarlierVersion}" && echo OK
+ fi
+ echo -n ">>> Creating hardlinks for ${CATEGORY}/${PF} ..."
+ if [ ! -d "${BUILDDIR}/../progs/${CATEGORY}/${PF}" ]; then
+ install -d "${BUILDDIR}/../progs/${CATEGORY}/${PF}"
+ else
+ rm -rf "${BUILDDIR}/../progs/${CATEGORY}/${PF}"
+ install -d "${BUILDDIR}/../progs/${CATEGORY}/${PF}"
+ fi
+ (cp -al ${D}/* ${BUILDDIR}/../progs/${CATEGORY}/${PF}/ && echo OK) \
+ || echo ">>> WARNING: Problem encounter while creating hardlinks.."
+
# Portage regenerates this on the installed system.
if [ -f "${D}/usr/share/info/dir.gz" ]; then
rm -f "${D}/usr/share/info/dir.gz"
|
emerge-hardlink.patch
| Quote: |
*** emerge.old Tue Mar 29 19:49:44 2005
--- emerge Tue Mar 29 19:43:36 2005
***************
*** 2184,2190 ****
--- 2184,2197 ----
print ">>> Unmerging "+y+"..."
emergelog("=== Unmerging... ("+y+")")
mysplit=string.split(y,"/")
+
#unmerge...
+ for root, dirs, files in os.walk("/var/tmp/portage/progs/"+y, topdown=False):
+ for name in files:
+ os.remove(os.path.join(root, name))
+ for name in dirs:
+ os.rmdir(os.path.join(root, name))
+ os.rmdir("/var/tmp/portage/progs/"+y)
retval=portage.unmerge(mysplit[0],mysplit[1],portage.root,mysettings,unmerge_action not in ["clean","prune"])
if retval:
emergelog(" !!! unmerge FAILURE: "+y)
|
|
|
| Back to top |
|
 |
adaptr
Watchman
Joined: 06 Oct 2002
Posts: 6730
Location: Rotterdam, Netherlands
|
| Posted: Tue Mar 29, 2005 10:59 pm Post subject: |
|
|
Using /var/tmp/portage is useless since it may be cleaned out at any time - as with any tmp directory.
Hardlinks could also introduce potential disasters...
I would sooner go for subdirectories under the ebuilds themselves, since these are static (-ish).
Most package directories already have a subdir called files under them; simply use that if you must.
No, you can't directly link outside of the work dir inside emerge - sandbox, dude.
And a good thing it is, too...
Sandbox, that is, not this particular restriction on your wishes.
_________________
>>> emerge (3 of 7) mcse/70-293 to /
Essential tools: gentoolkit eix profuse screen |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
