| View previous topic :: View next topic |
| Author |
Message |
474
l33t
Joined: 19 Apr 2002
Posts: 714
|
 Posted: Fri May 24, 2002 8:28 pm Post subject: Automatically unemerging stale packages [:-)] |
 |
|
Phew, this one was tricky. I'm sure many of you, like myself, have gotten a bit carried away with the emerge --update command and as a result have found files lying around on the system that belong to older versions of packages on the system. emerge -s does not give you all the gory details, although qpkg -I will. It is necessary to unemerge the specific versions themselves in sequence, but it was a cumbersome procedure - until now!
This script inherits some of the ideas explored in my previous scripts. It determines all installed packages on the system, then determines which ones are stale. It will show which ones are stale and, optionally, delete them by iterating through each one, invoking emerge -C on each specific one. If you run the script without any parameters it gives you a small report and shows you which packages it would unmerge. Re-invoking with --nopretend as a parameter will make it actually unmerge the packages. I have tried hard to make sure it can parse the version formats of all the packages and it seems to work very well. Note: you can expect some of these ideas to go back into my staledistfiles script which I posted before, as the version checking in that script was not perfect!
As ever, suggestions and feedback in general are welcome. I hope you all find this useful! 
| Code: | #!/usr/bin/perl
use strict;
# cleanpkgs.pl - by kerframil
# Report bugs/suggestions to the Tip's'Tricks forum at forums.gentoo.org
# Die if process owner isn't root
if ($< != 0) {
print "You must be root to run this script.\n\n";
exit 0;
}
# Main block
my $pretend;
if ($ARGV[0] && $ARGV[0] eq '--nopretend') { $pretend = 1 } else { $pretend = 0 };
print "Determining installed packages ...\n";
# Grab packages
my @pkgs = get_pkgs();
my $pkgcount = @pkgs;
print "You have $pkgcount packages installed. Determining stale versions ...\n";
# Determine stale packages
my @stalepkgs = get_stale_pkgs(\@pkgs);
my $stalepkgcount = @stalepkgs;
if ($stalepkgcount == 0) {
print "\7\nYou have no stale packages!\n\n";
exit 1;
}
print "$stalepkgcount / $pkgcount packages are stale:\n\n";
# Iterate through packages, deleting if user so requested
foreach my $row (@stalepkgs) {
($pretend == 0) ? print 'Would delete: ' : print 'Deleting: ';
my $pkg = $row->[0] . '/' . $row->[1] . $row->[2] . $row->[3] . $row->[4] . $row->[5] . $row->[6];
print $pkg;
# Does the user want to clean out the stale packages?
if ($pretend == 1) {
(`emerge -C $pkg >> ./cleanpkg.log`) ? die "\n\nProblem unemerging $pkg, check ./cleanpkg.log" : print " -> Success \n";
} else {
print "\n";
}
}
($pretend == 0) ? print "\n\7Re-invoke with the --nopretend parameter to unmerge these packages\n\n" : print "\nSuccess!\n\n";
1;
# Determine all instances of all installed builds!
sub get_pkgs {
# return `qpkg -I' or die "\7Problem during invocation of qpkg, have you installed gentoolkit?";
my @array;
my @pkgs = `qpkg -I`;
foreach (@pkgs) {
# Get component parts of package string, catering for all variations I hope, no it wasn't easy ;-)
/^\x1B.+?m(.+)\/\x1B.+?m(.+?)(-{0,})([\d\.]{1,})([_a-z0-9]{0,})(-{0,1})(r\d+){0,1}/;
# Tip: $1 = dir, $2 = buildname, $3 = possible '-', $4 = version no, $5 = possible '_beta\n' suffix, $6 = possible '-' $7 = possible revision (r\n)
# Push this info into array as anonymous array ref (pseudo-2D array)
push (@array, [$1, $2, $3, $4, $5, $6, $7]);
}
return @array;
}
# Determine stale entries and return a list of packages to be removed, expects a
# ref for an array containing the relevant information (as delivered by &get_pkgs)
sub get_stale_pkgs {
my $pkgs = $_[0]; # Establish the ref for package list
my @array; # This will hold the final list for deletion
my @lastrow;
foreach my $row (@{$pkgs}) {
# Do we have a match?
if ($row->[0] eq $lastrow[0] && $row->[1] eq $lastrow[1]) {
# Yes, let's compare versions to be absolutely safe ...
if (rev_is_gt($row->[3], $lastrow[3])) {
# Yes this version is newer, so the last one to be looked at is stale
push (@array, [$lastrow[0], $lastrow[1], $lastrow[2], $lastrow[3], $lastrow[4], $lastrow[5], $lastrow[6]]);
} else {
# Hmm, the version was apparently the same but we still need to check the
# revision number or beta number if present
if ($row->[4]) {
# Beta suffix test
if (rev_is_gt($row->[4], $lastrow[4])) {
push (@array, [$lastrow[0], $lastrow[1], $lastrow[2], $lastrow[3], $lastrow[4], $lastrow[5], $lastrow[6]]);
}
# Revision suffix test
elsif ($row->[6]) {
if (rev_is_gt($row->[6], $lastrow[6])) {
push (@array, [$lastrow[0], $lastrow[1], $lastrow[2], $lastrow[3], $lastrow[4], $lastrow[5], $lastrow[6]]);
}
}
}
}
}
@lastrow = ($row->[0], $row->[1], $row->[2], $row->[3], $row->[4], $row->[5], $row->[6]);
}
return @array;
}
# Accepts two string arguments containing revision codes, and returns a
# boolean indicating whether the first revision code is newer than the second
#
sub rev_is_gt {
my ($weight1, $weight2);
my ($ver1, $ver2) = @_;
$_ = $_[0];
# Check if revision code uses dotted notation
if (/\./) {
# Yes, establish version "weights" for both codes
$weight1 = rev_weight($ver1);
$weight2 = rev_weight($ver2);
# Now we can accurately compare numerically
return ($weight1 > $weight2);
}
else {
# No, just compare numerically, remembering to filter characters first if there are any
$ver1=~ s/[^0-9]//g;
$ver2=~ s/[^0-9]//g;
return ($_[0] > $_[1]);
}
}
# Establish the "weight" of the dotted revision code, that is, an integer
# that represents its newness (used by &rev_is_gt)
#
sub rev_weight {
my ($weight, $n, @revcodes, $count);
# Split into component numbers
@revcodes = split(/\./, $_[0]);
$count = @revcodes; # How many codes?
# Deal with the least significant code first
$weight = $revcodes[$count - 1];
# Iterate through the more significant codes, accumulating weight suitably
for ($n = $count - 2; $n >= 0; --$n) {
$weight+= $revcodes[$n] * (10 ** ($count - ($n + 1)));
}
return $weight;
}
|
|
|
| Back to top |
|
 |
474
l33t
Joined: 19 Apr 2002
Posts: 714
|
| Posted: Fri May 24, 2002 9:33 pm Post subject: One bug |
|
|
Hmm, found one package that doesn't pick up - at least it's harmless  . Let me know if there are any others it doesn't pick up. You can always validate it did find *all* the stale ones by browsing through the list: qpkg -I | less. There shouldn't be any two lines next to each other featuring the same package ... |
|
| Back to top |
|
|
roTor
n00b
Joined: 21 Apr 2002
Posts: 55
Location: Atlanta, GA
|
| Posted: Sat May 25, 2002 2:29 am Post subject: Problem with your script |
|
|
Gave it a try and got the error::
Missing right curly or square bracket at ./emergeclean.txt line 101, at end of line
syntax error at ./emergeclean line 101, at EOF
Execution of ./emergeclean aborted due to compilation errors.
This isn't a perl script is it?
_________________
Even if you're on the right track, you'll get run over if you just sit there.
-- Will Rogers |
|
| Back to top |
|
|
474
l33t
Joined: 19 Apr 2002
Posts: 714
|
| Posted: Sat May 25, 2002 2:51 am Post subject: Line breaks? |
|
|
| Quote: | | Gave it a try and got the error:: |
Yes, it is a Perl script. You might be getting that error because the forum page inserts line breaks where that shouldn't be any, because the browser window isn't wide enough. I don't know what window manager or text editor you are using but cutting and pasting should preserve the carriage returns properly in most cases. If this is a problem, I'll just have to crop the lines to be narrower. All lines finish with a semicolon ; apart from comment lines beginning with a # and block indicators { } and line breaks musn't be in the wrong places, otherwise you will definitely get errors. It runs fine. |
|
| Back to top |
|
|
heim
n00b
Joined: 31 May 2002
Posts: 5
|
| Posted: Fri May 31, 2002 3:53 pm Post subject: Re: One bug |
|
|
| kerframil wrote: | | .... You can always validate it did find *all* the stale ones by browsing through the list: qpkg -I | less. There shouldn't be any two lines next to each other featuring the same package ... |
or, a much easier way... dpkg -l|uniq -d
| Code: |
[root@pdq /home]# qpkg -l|uniq -d
/usr/share/themes/Xeno
/usr/share/themes/Xeno
/usr/share/themes/Xeno
/usr/share/themes/Xeno
[root@pdq /home]#
|
|
|
| Back to top |
|
|
lx
Veteran
Joined: 28 May 2002
Posts: 1012
Location: Netherlands
|
| Posted: Mon Jun 03, 2002 9:43 am Post subject: USE emerge -p clean |
|
|
 This deletes ALL lower version installed
 Isn't this functionallity provided by the emerge clean option of the latest emerge?
For instance I've installed glib and gtk version 1.2 and 2.0, the lower will get unmerged while these are needed by some packages, whats wrong with using emerge -p clean?????, this takes the SLOT variable into account. Well it ain't perfect yet, but then I use qpkd --dups (gentoolkit) and clean up by hand, only a very few packages.
_________________
"Remember there's a big difference between kneeling down and bending over.", Frank Zappa |
|
| Back to top |
|
|
474
l33t
Joined: 19 Apr 2002
Posts: 714
|
| Posted: Wed Jun 05, 2002 11:26 pm Post subject: Nothing's wrong with emerge clean but ... |
|
|
Bah! I just wrote a detailed reply to your post, lx, and lost my net connection  Anway, yes you're right, it doesn't take into account slots, however I had wanted it to be aggressive which is, of course, why it doesn't delete anything unless told to. See here for why I felt the need to write it, as you say, it ain't perfect yet or at least it wasn't: https://forums.gentoo.org/viewtopic.php?t=3240 |
|
| Back to top |
|
|
lx
Veteran
Joined: 28 May 2002
Posts: 1012
Location: Netherlands
|
| Posted: Thu Jun 06, 2002 12:08 am Post subject: Re: Nothing's wrong with emerge clean but ... |
|
|
| kerframil wrote: | | Anway, yes you're right, it doesn't take into account slots, however I had wanted it to be aggressive which is, of course, why it doesn't delete anything unless told to. See here for why I felt the need to write it, as you say, it ain't perfect yet or at least it wasn't: https://forums.gentoo.org/viewtopic.php?t=3240 |
Well I didn't say it wasn't perfect I just wanted to point out some dangers, I'm the lazy one sitting on my rear ass, what I should build is a remove function which takes the bootstrap / system / world (packages installed by hand) files into account and when deleting calculates all unused dependencies and prints them for your approval, and when you agree deletes them. Think it would be a very very very cool feature, this way you can emerge something for testing and you don't have to remember all the other packages which get installed. This will also clear out stale packages, because newer package will satisfy the dependency and get removed from the remove-list instead of the older versions. But well as I said I sit on my rear ass (I'm becoming quit good at it) and just tell you what I should supposed to be doing, you on the other hand are just doing it (probably also sitting on your ass)...... damn I'm tired,
Cya lX.
Maybe I will put this idea in the suggestion forum, but for now I'm too tired to search for another post stating the same idea.
_________________
"Remember there's a big difference between kneeling down and bending over.", Frank Zappa |
|
| Back to top |
|
|
lx
Veteran
Joined: 28 May 2002
Posts: 1012
Location: Netherlands
|
| Posted: Sat Nov 09, 2002 1:28 pm Post subject: |
|
|
A little script I now use  , in order to use this you need to have a very clean world file, only the necessary packages so always use emerge -u world or clean out the world file.
| Code: | #!/bin/bash
FILES=(`emerge -p -e world | grep -e "^\[" | sed -e "s/\[.*\] //" | sort`)
FILES2=(`/usr/lib/portage/bin/pkglist | cat | sort`)
COUNT=0;
COUNT2=0;
REMOVE=0;
SAME=0;
ADD=0;
while (( $COUNT!=${#FILES[*]} )) && (( $COUNT2!=${#FILES2[*]} )); do
if [ "${FILES[$COUNT]}" = "${FILES2[$COUNT2]}" ]; then
(( COUNT++ )); (( COUNT2++ ));
else
if [[ "${FILES[$COUNT]}" < "${FILES2[$COUNT2]}" ]]; then
echo +${FILES[$COUNT]};
(( COUNT++ ));
else
if [[ "${FILES[$COUNT]}" > "${FILES2[$COUNT2]}" ]]; then
echo -${FILES2[$COUNT2]};
(( COUNT2++ ));
else
exit
fi
fi
fi
done |
Cya lX.
_________________
"Remember there's a big difference between kneeling down and bending over.", Frank Zappa |
|
| Back to top |
|
|
AlterEgo
Veteran
Joined: 25 Apr 2002
Posts: 1619
|
| Posted: Sat Nov 09, 2002 2:27 pm Post subject: |
|
|
lx,
if I run your script, I get a huge output.
It does list "stale" packages for sure, but it also suggests to update "dependency-packages": I mean upgrade to a higher version of packages that have "only" been installed because of dependencies.
I guess this is similar to emerge -p --deep world.
Won't this break things?
Example: I emerge progX, and I need to emerge A and B as dependencies for ProgX to work. If I now update A and B, there is a chance that ProgX will be broken because of these updated dependencies, right?
How should I read the output of your script?
Thanks. |
|
| Back to top |
|
|
lx
Veteran
Joined: 28 May 2002
Posts: 1012
Location: Netherlands
|
| Posted: Sat Nov 09, 2002 2:46 pm Post subject: |
|
|
Well I don't know if it brakes anything but it may very well do this, but I want to have the latest packages installed, and that's what this script shows, (emerge -p -e world (clean install)). Some update can cause other packages to break and these need to be recompiled. A script that takes installed packages into account would be better for most people, but it would be a lot more complicated.
More general it's a problem if you upgrade a package it can brake other packages because the new package has a different library name c.q. different functions. changing your use flags can be a big problem also.
The portage system should take into account when updating / unmerging packages which libraries are removed and if programs depend on them (and if the updated packages replaces them), but then you still have a change of breaking you system because of incompatible version.
Although I can run ldd on al my binaries and libraries quite fast doesn't mean that everybody can. I have one time needed this script, to check which packages needed to be rebuild after updating some packages (using "not found" as argument):
| Code: | #!/bin/bash
FILES=""
resolvedir(){
for FILE in $1; do
if [ -f $FILE ] && [ -x $FILE ] && [ ! -L $FILE ]; then
FOUND=`ldd $FILE | grep -i $2`
if [ -n "${FOUND}" ]; then
FILES="${FILES} $FILE"
fi
fi
done
}
echo Checking /bin /sbin /lib...
resolvedir "/bin/* /sbin/* /lib/*" $1
echo Checking /usr/bin /usr/sbin /usr/lib...
resolvedir "/usr/bin/* /usr/sbin/* /usr/lib/*" $1
echo Checking /usr/X11R6/bin /usr/X11R6/lib...
resolvedir "/usr/X11R6/bin/* /usr/X11R6/lib/*" $1
echo $FILES
for FILE in ${FILES}; do
qpkg -f $FILE
done
exit |
It's very difficult to keep track of all dependencies....
Cya lX.
Ps: ??? emerge -p --deep world ????, what does --deep do?
_________________
"Remember there's a big difference between kneeling down and bending over.", Frank Zappa |
|
| Back to top |
|
|
H-Pi
Apprentice
Joined: 26 Jun 2002
Posts: 175
Location: Delft (NL)
|
| Posted: Mon Nov 11, 2002 4:21 pm Post subject: |
|
|
| lx wrote: |
Ps: ??? emerge -p --deep world ????, what does --deep do? |
seen the results, I think it emerges also dependencies of packages in your world file you have unemerged, or never emerged
for example I unemerged kde-toys and kde-games, and they are listed in my emerge -p --deep world |
|
| Back to top |
|
|
lx
Veteran
Joined: 28 May 2002
Posts: 1012
Location: Netherlands
|
| Posted: Tue Nov 12, 2002 12:51 pm Post subject: |
|
|
| H-Pi wrote: | | lx wrote: |
Ps: ??? emerge -p --deep world ????, what does --deep do? |
seen the results, I think it emerges also dependencies of packages in your world file you have unemerged, or never emerged
for example I unemerged kde-toys and kde-games, and they are listed in my emerge -p --deep world |
Aren't they listen in emerge world then?.
Thkx, I will check next time I've major changes / updates, cause mmm well there are no new packages at the moment needed for my system,........
--update
Checked it and emerge -p --deep world return null packages to upgrade while my script return -lesstif-0.xx +lesstif-0.xx-r1. So it's not the same. Didn't check unemerged packages...
--
Cya lX.
_________________
"Remember there's a big difference between kneeling down and bending over.", Frank Zappa |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
