| View previous topic :: View next topic |
| Author |
Message |
hielvc
Advocate
Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca
|
 Posted: Thu Jan 20, 2005 4:23 am Post subject: An emerge wrapper for breaking emerges into chunks |
 |
|
Filed a bug#97087 about emerge casual handling of TC updates. Bug closed, its not consider a problem at this time.
Update:This script does not do the entire job yet.,at least in one step, It helps with the TC but it does not address the problem yet of the libraries.
If you have a program, prog3, that builds against 2 libraries lib1 and lib2 from other programs, then if they havent been updated
and prog3 is,then you might have problems. If lib1 and/or lib2 get updated by the normal "emerge world -uD" or emwrap.sh prog3
could break.
tcupdate.sh and emwrap.sh prevent unplanned for TC updates. That is probably their best feature. They rebuild the TC in the right
order and only when you want to. This still leaves you open for the libaries problem. Thats why you shouldnt lightly up grade your
TC unless your prepared to do some major emergeing if something hickups. Its an attempt to
build the TC and all the system files in the order such that the libs are built cleanly in one go through as when the system is being built from scratch, after the bootstrap.
This is a compilation of form several threads. The main 2 are GCC 3.4.3 is upon us! pg3
| Quote: | robmoss
Recompilation of glibc is not necessarily required, but it is recommended. It was compiled using a more buggy compiler than the one you have installed; as such, it is an absolute false economy not to recompile it. Personally, after every update of a particularly important package (kernel headers, glibc, gcc, binutils, kernel etc. etc.) I do the following:
| Code: |
emerge -e system
emerge -e system
emerge -e world
emerge -e world |
The first command ensures my toolchain is in order; the second ensures it's been compiled with a clean and current toolchain; the third re compiles all my packages with a clean and current toolchain; the fourth ensures that all my packages are compiled against packages compiled with a clean and current toolchain. |
and as | Quote: | dirtyepic
according to genlop -p, i'll have that completed just around the time gcc 4.0 goes stable.  |
The other thread was rac's A more robust installation method
This works usually,but only if you do all 4 emerges. It waste time and depending on how long its been since you upgraded your linux-headers and glibc it might break your gentoo. Now to some very smart people, who definitlly know more about linux than I, these are fightin words. Its time you rethink about whats happening in the " emerge system ....". | Quote: | emerge system -ep
[ebuild N ] sys-devel/patch-2.5.9-r1
[ebuild N ] sys-libs/zlib-1.2.2
[ebuild N ] dev-python/python-fchksum-1.7.1
[ebuild N ] sys-apps/sandbox-1.2.9
[ebuild N ] sys-libs/gpm-1.20.1-r4
[ebuild N ] sys-libs/ncurses-5.4.20050319
[ebuild N ] app-shells/bash-3.0-r11
[ebuild N ] sys-devel/gnuconfig-20050602
[ebuild N ] sys-libs/readline-5.0-r2
[ebuild N ] sys-apps/diffutils-2.8.7-r1
[ebuild N ] sys-devel/binutils-config-1.8-r3
[ebuild N ] sys-devel/binutils-2.16.1
[ebuild N ] sys-devel/m4-1.4.3
[ebuild N ] sys-devel/bison-2.0
[ebuild N ] sys-apps/sed-4.1.4
[ebuild N ] sys-devel/gcc-config-1.3.11-r3
[ebuild N ] sys-apps/texinfo-4.8
[ebuild N ] sys-devel/gcc-3.4.4
[ebuild N ] sys-libs/libstdc++-v3-3.3.4
[ebuild N ] sys-kernel/linux-headers-2.6.11-r2
[ebuild N ] sys-libs/glibc-2.3.5
[ebuild N ] dev-java/java-config-1.2.11-r1
[ebuild N ] dev-java/blackdown-jre-1.4.2.02
[ebuild N ] app-arch/unzip-5.52
|
the first 24 lines of emerge system -ep and what do we have zlib, ncurses, bash, readline, binutils, gcc, linux-headers and then glibc. Quit a fouled up build order. | Code: | | emerge binutils gcc linux-headers glibc |
It could definitlly break some peoples systems and for those whos systems arent broken its just a waste of time except for building linux-headers and glibc. All the programs and their libararies prior to those two are built against the old TC and everything after is built by a fouled up half assed new TC. The second emerge system finally builds the entire TC once, even if it is built backwards and the last 2/3 to 3/4 of the files are finally rebuilt once with a clean TC. At this point we have built 200+ packages to finally rebuild the complete TC and only a 2/3s of the other important progs and their libs with a once built TC. With the first world build the 9 files that are run time deps for things like glibc ,zlib, ncurses, bash, readline, diffutils. m4, bison, sed and texinfo are finally built against the new TC. Now we are in the first "world" and the TC has started its second build, backwards of course, so it wont be completely built and the rest of system programs are built twice now. It has also started the first go through of world files. Well the last emerge finally rebuilds the TC twice and most of the system files built with it. The world files have been built once with the once built TC and once with the twice built TC. So the TC gets built twice most of the system files get built once with the twice built TC and all the world files are built twice the last time with twice built TC. Not quite what was advertized.
For those who are new to this the 4 basic programs in the ToolChain are linux-headers, glibc, binutils and gcc. There are other programs that should be included because the TC progs use them and their libararies, but these 4 are what a system is built against, linux-headers and glibc and by, gcc and binutils. So a safer less wastefull way to do this would be
| Code: | ]gcc-config -l # Important!! to make sure of which gcc your useing. Switch if necessary
emerge linux-headers glibc binutils-config binutils gcc-config gcc && emerge glibc binutils gcc && \
emerge system -e && emerge world -e && emerge world -e |
This is safer and saves time.
Heres a PPE ( PissPoorExample/Explination ) Linux-headers is like road map to a hugh factory or a small city. It has the locations of where the Hard Drives, keyboard, mouse, memory, NIC, video and all the other stuff your computer has and how to talk to them. Now progarms can be written useing this info directly, but its tedious and makes for bigger programs AND they will only run on a computer that has the same linux-headers or something very very close ( nvida and ATI bin drivers). So once upon a time all the programmers got togather ( Look up Rick Cook for more info ) and had big party meeting. They all agreed that having to write programs directly against linux-headers sucked big time. So they came up with " glibc ". Now glibc is like a detailed phone book that gives you a phone number, Extentsion and exactly what info in what order and type the kernel will take and what info it will return and in what order and type to send something lets say to the disk drives. The application programmers were happy they no longer had to do the work of writing glibc code and apps code and the users liked it because they didnt have to by exact same computer that their favourite programs were written on. Now the dwarfs and drones who maintain linux-headers and glibc occassionally change things, and no, there is no truth to the rumour that they get bored or like to see apps developers jumping up and down, will change something and things that worked dont any longer. They always have a good reason for changing the phone number or putting Miss losseGosse in the video department or having Msssss TightAss answer the phone. So when the kernel developers change things, fix things or add things the map changes. To take advantage of these new toys or fixs and such you have to redo phone book. SO it behoves yourebuild glibc when linux-headers are up graded. Gcc and binutils are the building crews. They take the fancifull flights of imagination that the apps developers come up with and try to make them. Gcc knows that when deweeb mentions Miss losseGosse to put the info in for her. When gcc finishes binutils takes the parts and puts them togather. Now gcc is built against glibc so when then phone book gets upgraded so should gcc. Now gcc makes blocks out the code feed to it like lego blocks. binutils takes these blocks and yup puts togater to the prog thta apps developer dreamed up. Since works so closely with gcc you need to rebuild it also to make sure its the same page as its three buddies.
Now you have seen mentioned here and probably else where the term "libaries". If I didnt mention it above, glibc is the biggest badass library on your system. Now it is not the only lib on your system. Alot of programs make baby phone books so that programs that tie into them, multimedia, sound, video know how to communicate with them and provide facilities for doing so. How many libs are there? Well cut and paste this into a xterm " for i in /lib /usr/lib /usr/X11R6/lib ;do find $i -iname lib* ;done|wc " => 6660 6660 245874 { wc - print the number of newlines, words, and bytes in files } and thats not all of them !
Now back to "emerge system ....". You might be wondering if the TC is nailed down why go through the bother of rebuilding everything a couple times? Well because all the programs and their libraries are built against glibc in one way or the other and it gets worse. Many progarms are built against 3, 4, or more existing programs. If you were to do a single "emerge world -e" to rebuild everything, thats means everything got updated right? Wrong, that would be ideal but think about if progA is built against glibc, progB is built against progA and progC against progB glibc and progA. You emerge world and it updates progC first then progA then glibc and finally progB. Theres a damn good chance that progC will be buggy particularly if there were any sort of major changes in linux-headers and/or glibc. So to cleanly rebuild progC glibc has to come first then progA then progB then progC. But if progB was rebuilt 1st then it isnt cleanly built. ProgB is still useing progA's old libs. But wait I mentioned, I think, that glibc gcc and binutils use other programs and the same appllies to them. The problem has been around ever since you started runing gentoo or any other Linux. Have you ever heard of RPM hell? Or getting packages for Slack from LinuxMafia, they would list what glibc gcc linux-header and other pertinent info that was required for a package to run as advertised. And yes Virginia even everybodies favourite, Windows has the same dang problem. Try applying service and security updates out of order. Now before you decide to switch back to Windows how many "emerge system/world -uD" have you done without noticeing anything being broken?
Ok now we know that there are some problems but hey I havent been bit yet, then again it might be why your here reading my ravings. The one thing that everybody agrees on is if your TC aint broke dont fix it. The problem now is that people are haveing to because of the gcc-3.4.4 switch.
If you look back on my proposed modiified "emerge ... emrge ..." We have "built a full TC, built a partial TC, and built a sys which has another TC build,ass backwards, embeded in it. Now we need that sys rebuild to start building the libs and progs cleanly but we also know that several of the more important progs, because they are run time or compile time dependencies of the TC need rebuilding to really nail down the TC. So rebuilding the TC now is a waste. Here is the deep dark secret of emwrap.sh. (hint you can copy and paste this stuff to play with it) We "emerge system -ep>>sysfile". That takes the output of emerge and puts it in to a file. Now with an editor we can remove the TC commpents and clean the "[ebuild U ]" x11-libs/gtksourceview-1.2.0 "[1.1.1]" out . Or useing some of unix's tools we "cut -f2 -d "] -s" to get rid "[ebui...] and cut -f1 -d "[" to remove the version info. This leaves a list of "category/fullpackageName", x11-libs/gtksourceview-1.2.0. The scripts use sed, awk and grep to find and filter out the TC items. Any way we now have a list that we can feed to emerge with a "for" loop.
| Code: | for i in $(cat sysfile); do
emerge =$i
done |
This simple loop reads one line at a time out of "sysfile" and into the varible "i" and then feeds it to emerge. It has a couple of probs though. 1 it doesnt care if the emerge is successfull or not, it just grabs the next item in the list and builds it. 2 Every program it does emerge successfully, which should be all of them, gets added to your world file. This is not good. So we add --oneshot to keep from adding the files to the world file and --nodeps which makes emerge do one file at a time. But since I have emwrap.sh all I have to do, to "emerge system -e " minus the TC is " emwrap.sh system -e " or since Im real lazy "emwrap.sh -se". If you still think Im selling snake oil add a "p" to -ep or -sep , surprise "p" is --pretend. Ok, we now can change the modified "emerge .. emerge.." but, besides lazy Im a rotten typer and I can misspell anything. So in emwrap.sh if you put a -t in -se => "-set" that means build the entire TC only, when used with --emptytree. To build the TC and the rest of the files use "b". In this case Id use "t" because after building the TC I want to check gcc-config to make sure Im useing gcc-3.4.4
| Code: |
gcc-config -l # make sure of gcc before beginning
emwrap.sh -set or emwrap.sh -St
gcc-config -l #to select new gcc
emwrap.sh -seb or emwrap.sh -Sb # We rebuild the TC and then the rest of the system
emwrap.sh -se or emwrap.sh -S # doing the 2nd system emerge minus the TC |
Now all thats left are the 2 "emerge world -e" which are the eqivilent of "emwrap.sh world -eb or emrwap.sh -web ". It builds the entire TC first and then everything in a regular "emerge world -e" minus the full TC. Now we are at the point where most everything in the system has been rebuild correctly including the programs that are runtime dependencies for glibc, gcc and binutils. So its time for our last TC rebuild plus the first go through of the world files. But before we do this, if you look at the Robs original post we've already built the clean TC and all the system packages against it. Now wouldnt it be nice if we could build just the world files with out the system files. Well " emwrap.sh -W " does just that. It gets the system list and world list and as it did with the TC elements it removes the system files from the world list.
| Code: | emwrap.sh -wet # rebuilds TC against clean system
emwrap.sh -W # first build of the world files against clean TC system
emwrap.sh -W # second rebuild and done |
On my system that 256 system packages and one full TC build shorter and its at least as clean as the original method.
To tie it all to gather | Code: |
gcc-config -l # make sure of gcc before beginning
emwrap.sh -set or -St
gcc-config -l #select the new gcc
emwrap.sh -seb or -Sb # We rebuild the TC and then the rest of the system
emwrap.sh -se or -S # doing the 2nd system emerge minus the TC
emwrap.sh -wet or -St # rebuilds TC against clean system
emwrap.sh -W # first build of the world files against clean TC system
emwrap.sh -W # second world rebuild and done |
Not maintained and broken. A rewrite of emwrap by maguire on page 9
Not maintained, MindEraser's script tcupdate
To get the current version of my script emwrap.sh Points to Seren's emwrap.sh-r4 link.
Version 4.0.8-r4. NOTE gcc_conf will now use either "config" or config-${CHOST} to set gcc-config.
Added a choice for those of you who want a different work directory, see page 24 of this thread.
| Code: |
#!/bin/bash
# Copyright (c) 2004-2007 Hiel Van Campen <hielvc>@{behind the at}<cox.net>
# This program is licensed under the terms of the GPL version 2.
# No blame for this prog should be assinged to Gentoo. Ell they went so far as to stick me in the
# cellar of unsuported software and all I wanted was to be is a dev like them. To drink Mai-tai's,
# chasem wild women's and drivem sporty cars. Well at least
# the mUderators promoted me to a OSER so eat your heart out
#
# Use at yee own risk. It works for me, but then I wrote it.
# Thanks to ecatmur's dep script, gentoo forums, for help_fmt and print_help and clever bashing.
# " MindEraser for her, I think, ideas and script, tcupdate when we first started this beast.
# " maguire's "emwrap" script, I borrowed his gcc-config. This will hopefully stop
# your perfered gcc from being switched, as alot of us have had happen at one time or the
# other. Oh maquire what a dumb script name. ;^)
# Version 3.0.6 , 12-15-05 Added saving of the failed to roots dir.
# Cleaned up and fixed formatting.
# Version 3.0.7 Added build TC once flag such that TC is built only once instead of twice.
# Added build file and failed file detection. If found emwrap.sh will offer to build them
# instead of what you were going to do. Ergo failed is no longer moved to /root
# Removed Portage gcc-config and binutils-config from TC filtering and from TC update
# notification. When they're found they get built
# 1-20-06 Minor updates to help showing -S and -1 usage. Re-added resume filter to diff_emerge
# 2-1-06 The end of a major rewrite and the addition of several new features including adding
# libstdc to the TC Some minor things to do or not. Better input control? Do I want to trust
# a script to switch gcc?
# 2-13-06 removed tc_filter from sys.lst and wrld.lst in to fix filtering error and save an
# unnecessary step. Also fixed TC from running -se in a -sept case. Added dagurasu suggestion for
# a CTRL-C trap.I will no longer be able to build a failed list for when I test :(
# Bumped up Rev to 4.0.1
# Fixed help. Added " exit " to print_help. 2-18-06
# Added version info to "function info" 2-17-06
# Added version to TC list in tc_emerge 3-27-06. Found a package longer than
# "-c17-60".
# Added binutils-config and gcc-config filter to remove them from wrld.lst if TC is being
# built.4-1-06
# 5-5-06 Went back to using just "cut" for file clean up. Its more general purpose than sed. Also
# added "exclude" filtering. Note "exclude or -X" have to be the last option because everything
# after them is put into the exclude_filter. Thanks to dol-sen and Tron_5
# 5-7-06 Added "-q" to flags for more concise pretend output. Also hardwired into all emerges
# for list generation. 5-8-06 moved exclude filter to wrld_lst that way it works for everything
# except the TC. 5-18-06 Changed libstdc to a variable such that if you don't have an stdc I wont
# give you one ! 5-22-06 added clean-up code to clean_up to remove the files created during sys
# file removal from wrld list in World_minus_system_emerge function.
# 5-28-06 put maguire's gcc-config in. bumped version to 4.0.5
# July and early August added removed added gcc-config and eselect joy POCKIN joy
# 8-14-06 Realized it would be nice to jsut get the version info so added version_info and
# added parameter checking to optargs.
# 8-18-06 added the exclude url=http://www.fi3nd.net/linux/emwrap.sh
# 9-18-07 fixed changed of gcc config from /etc/env.d/gcc/config to gcc/config-i686-pc-linux-gnu
# 10-14-07 Makeing use of CHOST for gcc_config -> gcc/config-${CHOST}
# 10-15-07 Added "config" search and use for gcc-config.
PROG="emwrap.sh"
VERSION="4.0.8 with gcc-config-r4"
DATE="Mon Oct 15 21:41:56 PDT 2007"
TAG="The fuzzy tailed rats are winning, Elp"
DESC="A wrapper for emerge so you can break emerges into chunks, TC, system, world
or world minus system"
print_help(){
cat <<-END #|help_fmt
${GR}${PROG} v. ${VERSION} ${Cy}"${TAG}"${NO}
${GR}${DESC} ${DATE}${NO}
Usage: ${PROG} [OPTION]
Date 4-23-05
Use at yee own risk. It works for me, but then I wrote it.
The main idea behind emwrap.sh is if the TC is going to be rebuilt or updated do
it first, then do the rest. The script takes the output of emerge and puts
it into a list, which we can break into chunks, the TC, system, and world.
1. Test if there is a toolchain, TC, update. emwrap.sh will not do anything to your TC
unless you tell it to with the "t" or "b" flags. Unles they're passed, whatever emerge
is going to do will have the TC files removed. So a "emerge system -e" will really be
system - TC, unless you pass the "t" or "b" flag.
2. If there is and you want to, then you can rebuild all or part of your TC and then
using an edited list to build the other items scheduled to be updated.
3. Do the update without updating the TC, toolchain.
Options:
These first flags are the same as what you use with emerge [options] See "man emerge"
-h or ? or help Display this help (default)
-f Fetches files only
-u update
-D deep
-e Does an emerge "--emptytree" world/system
-p Just like pretend in emerge. Works with all other flags.
-N Tells emerge to include installed packages that are affected by the new USE flags
that have changed since the program was built.
-B/b build packages only or build them as you emerge packages
resume, -r Continues emwrap from where it stopped. To use rerun the same command
and add "r" ${Rd}DONT${Yl} USE EMWRAP.SH FOR ANYTHING ELSE UNTIL YOU RUN THE -r${NO}
and emwrap will start from where it stopped without having to rebuild everything
already done.
-K/k use packages. For use if you have --buildpkg in your /etc/make.conf or use -B/b
-G/g use packages from a PORTAGE_BINHOST server.
system, -s "-s" is short for system.The same as "emerge system"
world, -w "-w" is short for world. The same as "emerge world"
For use by the script
-q Quite, it changes the output format in when you run pretend. Try it to see
which output you like.
-t Rebuilds the toolchain components only
-b Rebuilds the toolchain and resumes the emerge.
-1 For use with "t" or "b" only builds the TC once.
-S This is short for a "emerge system --emptytree" build.
-W builds the world files, "emerge world -e", BUT minus the system files.
failed builds the files in the failed list.
nc Turns off color in the script. You still get color in emerge out put.
exclude or -X After either must follow a space separated list of files to be excluded
from being built. EX: emwrap.sh world -uDX gentoo-source openoffice KDE my_program
or emwrap.sh -uDN system exclude (or -X) openoffice KDE my_program
-V, or version Gives short version info.
The flags are chainable and change whether your doing a system or world emerge and
from -uD, -N, to -e. There is also a pretend mode, -p. Like emerge -p it
shows you what will be emerged. I recommend that instead of boldly going forth, add
the -p to the end of the switch's first, to see whats going to be emerged. Then you
can remove it.
Example emwrap.sh -sep ==> emerge system --emptytree -pretend. If you remove the
-p it will do a build of all packages in system except for the TC.
${Yl}NOTE -se or -S do the same thing${NO}
emwrap.sh -set or -wet ==> updates the entire TC. If the -b switch is used instead
of -t then the entire TC is built and then all system/world packages minus
toolchain files.
Examples
${CY}emwrap.sh${NO} prints help.
${CY}emwrap.sh -wuDbp${NO} Checks for updates to the TC and in the world.
${CY}emwrap.sh -wuDt${NO}==> emerge world -uD and only does TC update if any.
${CY}emwrap.sh -wuDb${NO}==> updates, if any, the TC and then the world files.
${CY}emwrap.sh -f{other flags}${NO} will fetch the files for you.
${CY}emwrap.sh system -uDN${NO}==> emerge system -uDN, update deep newuse
${CY}emwrap.sh -Sb1${NO}==> emerge system -e build TC once and then system files
emwrap.sh can "emerge system -e" with or without the TC and remove all packages
built during the system emerge for a following "emerge world -e". This is a major
time savings as 130+ packages wont be rebuilt during the world half.
${RD}NOTE -d -c -r have been changed${Rd}. -d is now -S, -W takes on the old
role of -r. -r is now resume and -c has been removed.${NO}
${CY}emwrap.sh -Sb${NO} ,builds TC and system -e. When it completes run
${CY}emwrap.sh -W${NO} ,this picks where -Sb stopped and builds the rest of
the files in the " world -e ". Why use this, well if you want to break an
" emerge world -e " up into 2 chunks for two different nights this will do it.
Here are the TC build list used in this wrapper when doing updates. The first 3 are
double builds.
If linux-headers TC="linux-headers glibc \$tc_conf binutils gcc glibc binutils gcc"
If glibc TC_glb="glibc \$tc_conf binutils gcc glibc binutils gcc"
If binutils or gcc TCmini="\$tc_conf binutils gcc binutils gcc"
These are the single build TC list
If linux-headers TC1="linux-headers glibc \$tc_conf binutils gcc "
If glibc TC_glb1="glibc \$tc_conf binutils gcc "
If binutils or gcc TCmini1="\$tc_conf binutils gcc"
The TC build scripts are basically fall through. If you have a linux-headers update
then the script will use TC, if its a glibc update then TC_glib and
if binutils and/or gcc then TCmini. To force a full TC build use -set, -St or
-wet or -seb, -Sb or -web.
If you used "exclude" when you last ran emwrap.sh and you dont use "exclude" again
all the packages thet were execulded will be listed and an offer to now build
them is made. if you enter any key + ENTER it will build useing the exclude
list from your last run. If you used exclude during an -uD, update deep, then
you can just re-run useing -uD and anything still haveing an update will be updated.
Where this comes in handy is if you have kde and or gnome. To split out kde and
gnome to build later nights you could do something like 1st night emwrap -wuDX kde
gnome. 2nd night emwrap.sh -wuDX kde, this builds gnome but not kde. 3rd night
night emwrap.sh -wuD. What happens is that exclude.lst is detected
END
exit
}
function color(){
if [[ $nc != "yes" ]]; then
#Set up colors
NO=$'\x1b[0;0m'
BR=$'\x1b[0;01m'
RD=$'\x1b[31;01m' Rd=$'\x1b[00;31m'
GR=$'\x1b[32;01m' Gr=$'\x1b[00;32m'
YL=$'\x1b[33;01m' Yl=$'\x1b[00;33m'
BL=$'\x1b[34;01m' Bl=$'\x1b[00;34m'
FC=$'\x1b[35;01m' Fc=$'\x1b[00;35m'
CY=$'\x1b[36;01m' Cy=$'\x1b[00;36m'
fi
}
source /etc/make.conf
######## Variables ############################
bclass="none";eargs=""; count="0"
nc="" ;do_tc="";tc_once="";do_sys="";do_wrld="";both="";resume="";ftch=""
tc="no"
# Had to give "exclud" a dummy value. If its set to "" everything is filtered out 8^(
exclud="catsisrats"; exclude_list="catsisrats" ; quite="" ; tc_stdc="" ; cat_stdc=""
WORK_DIR=""
TIME=5 # sets the time the script displays failed_build info
function version_info(){
echo "${GR} ${PROG} Version-${RD}${VERSION} "${NO}
echo "${Gr} ${DESC}${NO}"
echo "${Rd} ${DATE}${NO}"
exit 1
}
function info(){
clear
cat <<-END
${GR} Running ${RD}$PROG $VERSION $DATE
${GR} If emwrap.sh is run and it detects a ${CY}"failed' or "build"${GR} file
${GR} then it will offer to build them. You can ignor this by not doing
${GR} anything and what you wanted todo will start after automaticlly.
END
}
function build_info(){
echo
cat <<-END
${Yl}You can now enter ${RD}" r "${Yl} to resume building. If you dont enter anything
then the old build file will be deleted.and ${Rd}emwrap.sh $bclass -$eargs${Yl} will
continue. If you want to stop and scratch your head do a CRTL-C${NO}
END
}
function faild_info(){
echo
cat <<-END
${Yl}You can now enter ${RD}" f "${Yl} to build the faild file list.If you dont enter
anything then the faild file will be deleted and ${Rd}emwrap.sh $bclass -$eargs${Yl}
will continue. You only need to run ${Rd}"emwrap.sh faild"${Yl} after doing a
emptytree system or world. If you want to stop and scratch your head do a CRTL-C${NO}
END
}
function exclude_info(){
echo
cat <<-END
${Yl}You can now enter ${RD}" x "${Yl} to build the exculded file list.If you dont enter
anything then the excluded file will be deleted and ${Rd}emwrap.sh $bclass -$eargs${Yl}
will be run. If you want to stop and scratch your head do a CRTL-C${NO}
END
}
# changing work dir to $WORK_DIR/emwrap
function wrk_dir(){
# change file locations to $WORK_DIR/emwrap/{FILES}
# Changed to explicit directory invoke, because if used to build a system from scratch,
# ergo a new build, $HOME isn't set and every thing for root is written to " / ".
if [ -e ~/.emwrap ]; then
source ~/.emwrap
fi
echo;echo "work_dir is $WORK_DIR"
# If ".emwrap" exist then WORK_DIR will be what you set up, else it defaults to
# "/tmp/emwrap"
WORK_DIR=${WORK_DIR:-/tmp}
if [ ! -d $WORK_DIR/emwrap ];then
mkdir $WORK_DIR/emwrap;chmod 777 $WORK_DIR/emwrap; cd $WORK_DIR/emwrap
else
cd $WORK_DIR/emwrap
fi
if [ "$(pwd)" != "$WORK_DIR/emwrap" ] ;then
echo;echo $RD"Did not change to /tmp/emwrap. Bailing out!"$NO;echo
exit 1
fi
}
# gets "read input" for "chk_failed_build" function following. It un-cluters
# "chk_failed_build".Also used for eselect.
function read_input(){
# if [ "${resume}" != "yes" ] && [ "$failed" != "yes" ] ; then
OUTPUT="no"
for loop_counter in 14 12 10 8 6 4 2 0 ; do
echo -n ${RD} "$loop_counter "${NO}
read -t 2 INPUT
if [ "$?" == "0" ] ; then
OUTPUT="yes"
export OUTPUT
# As soon as that DUMB ASS in front of the screen gets off thier ass
# and does an "f" or an "r" ENTER, I can quit counting backwards !!!
break 1
fi
done
# fi
}
function chk_failed_build(){
if [ "$resume" != "yes" ]; then
if [ "$failed" == "yes" ]; then
mv failed build
resume="yes"
elif [[ -e failed || -e build || -e exclude.lst ]]; then
if [ -s build ]; then
echo $RD "these remain to build"$NO
cat build
build_info
read_input
resume="$OUTPUT"
if [ "${resume}" != "yes" ]; then
echo;echo " $Yl Input was empty I will continue with emwrap.sh $bclass -$eargs"
rm $WORK_DIR/emwrap/build
fi
elif [[ -s failed && ! -e build ]]; then
echo $RD "these failed to build"$NO
cat failed
faild_info
read_input
failed="$OUTPUT"
if [ "${failed}" == "yes" ]; then
mv failed build
resume="yes"
elif [ ${failed} == "no" ]; then
echo;echo " $Yl Input was empty I will continue with emwrap.sh $bclass -$eargs"
rm $WORK_DIR/emwrap/failed
fi
elif [ -s exclude.lst ] ; then
echo ${RD}"These were excluded can now be built"${NO}
cat exclude.lst
exclude_info
read_input
exclude="$OUTPUT"
if [ "${exclude}" == "yes" ]; then
mv exclude.lst build
resume="yes"
elif [ "${exclude}" != "yes" ]; then
echo;echo " $Yl Input was empty I will continue with emwrap.sh $bclass -$eargs"
rm $WORK_DIR/emwrap/exclude.lst
fi
fi
fi
# keeps exclude.lst from becomeing hugh if emwrap.sh -X is run multiple times.
if [ -e excluds.lst ]; then
rm $WORK_DIR/emwrap/eexclude.lst
fi
fi
}
# For cleaning files. if's to keep rm from complaining
function clean_up(){
if [ -s failed ] ; then
echo;echo ${RD} "these failed to build"${NO}
cat failed
elif [[ -e failed && ! -s failed ]]; then
rm failed
fi
if [ -s build ] ; then
echo;echo ${RD} "these remain to build" ${NO}
cat build
elif [[ -e build && ! -s build ]] ; then
rm build
fi
if [ -s exclude.lst ] ; then
echo;echo ${RD} "these were excluded and can be built"${NO}
cat exclude.lst
elif [[ -e exclude && ! -s exclude ]]; then
rm exclude.lst
fi
# clean up
if [ -e sys.lst ] ; then rm sys.lst ; fi
if [ -e wrld.lst ] ; then rm wrld.lst ; fi
if [ -e wrld.tmp ] ; then rm wrld.tmp ; fi
if [ -e sys.lst ] ; then rm ssys.lst ; fi
if [ -e wrld.lst ] ; then rm swrld.lst ; fi
if [ -e sorted.lst ] ; then rm sorted.lst ; fi
}
function getargs(){
if [ $# -eq 0 ]; then
print_help
fi
# LONG OPTIONS
while [[ $1 != "" ]]; do
if echo $1|grep -v - ; then
case $1 in
system ) bclass="system" ;;
world ) bclass="world" ;;
help ) print_help ;;
resume ) resume="yes" ;;
nc ) nc="yes" ;;
failed ) failed="yes";;
ignore-default-opts) largs="ignore-default-opts" ;;
exclude ) shift
while [[ $1 != "" ]]; do
if [ "$exclud" == "catsisrats" ]; then
exclud="$1"
else
exclud="$1|$exclud"
fi
# adding exclude_list for greping of pkg's
exclude_list="$exclude_list $1"
shift
done;;
version ) version_info ;;
* ) echo ${RD}"$OPT is not a valid long flag" ; exit 1 ;;
esac
fi
while getopts "1bDefgGhKkNpqrSWstuwVX*" OPT; do
case $OPT in
s ) bclass="system" ;;
w ) bclass="world" ;;
D ) eargs="${eargs}${OPT}" ;;
e ) eargs="${eargs}${OPT}" ;;
N ) eargs="${eargs}${OPT}" ;;
u ) eargs="${eargs}${OPT}" ;;
g ) eargs="${eargs}${OPT}" ;;
G ) eargs="${eargs}${OPT}" ;;
k ) eargs="${eargs}${OPT}" ;;
K ) eargs="${eargs}${OPT}" ;;
q ) eargs="${eargs}${OPT}" ; quite="q";;
1 ) tc_once="yes" ;;
h ) print_help ;;
p ) eargs="${eargs}${OPT}" ; prtnd="p" ;;
f ) ftch="f" ;;
S ) do_sys="yes" ;bclass="system" ;eargs="e" ;;
W ) do_wrld="yes" ;bclass="system" ;eargs="e";;
b ) both="yes";;
r ) resume="yes" ;;
t ) do_tc="yes" ; Tc="yes";;
V ) version_info ;;
X ) shift
while [[ $1 != "" ]]; do
if [ "$exclud" == "catsisrats" ]; then
exclud="$1"
else
exclud="$1|$exclud"
fi
# adding exclude_list for greping of pkg's
exclude_list="$exclude_list $1"
shift
done;;
? ) echo ${Rd}"$1 is not a valid flag string"; exit 1 ;;
esac
done
shift
done
if [ "$bclass" == "none" ] ; then
echo ${RD}"You didnt select world or system so no, emwrap.sh"${NO}
echo ${RD}"isnt broken,but you are just as dumb as me!"${NO}
exit 1
fi
}
# emerges from build list and keeps count.
function build_count(){
count="0" ; s=$(wc -l build)
for z in $(< build) ;do
count=$(( count + 1 ))
echo -n "${Yl}${count} of ${s} "${NO}
# Added gcc_config which will run only when the TC is being updated.
{
emerge -O1${prtnd}${ftch}${quite} =${z} \
&& gcc_config ${z}
# The $1 is for which failed function, emerge_faild or tc_failed, is to be used.
} || $1
grep -v "${z}" build>tmp;mv tmp build
done
}
# merg_faild records which package fails to emerge buid_count
function emerge_faild(){
echo "${z}">> failed
}
# The next 4 functions are TC functions
# tc_faild bails out of the build_count if emerge error
function tc_faild(){
echo ${RD}"${z} failed to build. Stopping script."${NO}
exit 65
}
function tc_filter(){
awk "!/linux-head|glibc|gcc-[0-9]+|binutils-[0-9]+|libstdc+/"
}
function tc_check(){
# testing for TC components
# 3-27-06 removed awk when I put emerge tc in tc_emerge function to getting
# specific version info also changed gcc and binutils regex. 5-18-06 Changed
# libstdc testing, if you have build it, if not, don't.
if grep -Eq linux-h wrld.lst
then tc="linux-headers"
elif grep -Eq glibc wrld.lst
then tc="glibc"
elif grep -Eq binutils-[0-9].* wrld.lst
then tc="binutils"
elif grep -Eq gcc-[0-9].* wrld.lst
then tc="gcc"
elif grep -Eq libstdc+ wrld.lst
then tc="libstdc" ; tc_stdc="libstdc++-v3" ; cat_stdc="sys-libs/"
fi
#shows what TC items have updates
if [[ $do_tc == "yes" || $both == "yes" ]] ; then
if [ -n "$tc" ] ;then
echo;echo $RD" ToolChain updates found"$NO;echo
echo ${Rd}" $tc"${NO}
echo;echo $BL"======================================================"$NO
echo
else
echo;echo ${RD}" No toolchain update "${NO};echo
fi
# for getting and building config tools in an TC build. Added libstdc so that
# variable tc_stdc could be set, its wasnt being reached if anything above it passed.
tc_conf="" ; tc_conf_regx=""
if grep -Eq binutils-config wrld.lst ; then
tc_conf="sys-devel/binutils-config $tc_conf"
tc_conf_regx="binutils-config $tc_conf_regx" ; fi
if grep -Eq gcc-config wrld.lst ; then
tc_conf="sys-devel/gcc-config $tc_conf"
tc_conf_regx="gcc-config $tc_conf_regx" ; fi
if grep -Eq eselect-compiler wrld.lst ; then
tc_conf="app-admin/eselect-compiler $tc_conf"
tc_conf_regx="eselect-compiler $tc_conf_regx" ; fi
if grep -Eq eselect-[0-9].+ wrld.lst ; then
tc_conf="app-admin/eselect $tc_conf"
tc_conf_regx="eselect-[0-9].* $tc_conf_regx"; fi
if grep -Eq libstdc+ wrld.lst ; then
tc_stdc="sys-libs/libstdc++-v3" ; fi
# added awk filter so not to build these two twice 4-1-06
awk '!/binutils-config|gcc-config|eselect-[0-9].+|eselect-comp/' wrld.lst >tmp
mv tmp wrld.lst
fi
}
function tc_emerge(){
# added tc-once to build TC once only which is what I do. 3-27-06 changed list
# for useing tighter regex in grabing version explict TC items to feed tc_emerge.
# CHanged libstdc to a variable so that if you dont have libstdc I wont give you a stdc ;^P
TC="linux-headers glibc $tc_conf_regx binutils-[0-9].* gcc-[0-9].* glibc binutils-[0-9].* gcc-[0-9].* $tc_stdc"
TC_glb="glibc $tc_conf_regx binutils-[0-9].* gcc-[0-9].* glibc binutils-[0-9].* gcc-[0-9].* $tc_stdc"
TCmini="$tc_conf_regx binutils-[0-9].* gcc-[0-9].* binutils-[0-9].* gcc-[0-9].* $tc_stdc"
TC1="linux-headers glibc $tc_conf_regx binutils-[0-9].* gcc-[0-9].* $tc_stdc"
TC_glb1="glibc $tc_conf_regx binutils-[0-9].* gcc-[0-9].* $tc_stdc"
TCmini1="$tc_conf_regx gcc-[0-9].* binutils-[0-9].* $tc_stdc"
# Added tc build list for getting version info
TC_build="sys-kernel/linux-headers sys-libs/glibc $tc_conf sys-devel/binutils sys-devel/gcc $tc_stdc"
if [[ "${resume}" != "yes" ]];then #prevents filling the build list after a failed TC resume
if [[ $tc_once != "yes" ]];then
case ${tc} in
linux-headers ) tc_regx=$TC;;
glibc ) tc_regx=$TC_glb;;
gcc ) tc_regx=$TCmini;;
binutils ) tc_regx=$TCmini;;
libstdc ) tc_regx=$TCmini;;
esac
else
case ${tc} in
linux-headers ) tc_regx=$TC1;;
glibc ) tc_regx=$TC_glb1;;
gcc ) tc_regx=$TCmini1;;
binutils ) tc_regx=$TCmini1;;
libstdc ) tc_regx=$TCmini1;;
esac
fi
fi
# if for determining if build is empty before refilling it.
if [ ! -e build ] || [[ -e build && ! -s build ]];then
# added emerge to get the version specific info. It also sorts the tc packages.
:>tc.lst
emerge -qp $(echo $TC_build) |grep "/"|cut -c17-65|cut -f1 -d " " >tc.lst
for t in $(echo $tc_regx); do
grep $t tc.lst>>build
done
fi
echo $RD"$(< build)"$NO;echo
build_count tc_faild
echo;echo ${Rd}"End of ToolChain update"${NO};echo
tc="no"
rm tc.lst
}
function gcc_config(){
# The eselect added 7-7-06 A varition of mcquires gcc-config
if [ "$tc" != "no" ] ; then
if echo "$1" | grep "gcc-[0-9].*" ; then
# Adding maguire's gcc_config setup here
gcc_current=""
gcc_new_version_number=""
gcc_for_gcc_config=""
gcc_current_no_dot=""
gcc_version_number_no_dot=""
###############################################################
# Let games begin
###############################################################
#10-14-07 switched to CHOST usage so that i686, x86_64 and whaterver else are picked up.
#10-15-07 added test for "config" or "config-"${CHOST}"
if [[ -e "/etc/env.d/gcc/config" ]] ;then
gcc_current=$(sed 's/^.*\([0-9]\.[0-9]\.[0-9]\)/\1/' /etc/env.d/gcc/config)
elif [ -e "/etc/env.d/gcc/config-"${CHOST}"" ] ; then
gcc_current=$(sed 's/^.*\([0-9]\.[0-9]\.[0-9]\)/\1/' /etc/env.d/gcc/config-"${CHOST}")
fi
# $1 is the "gcc-version" that the function build-count, passes to function gcc_config
gcc_new_version_number=$( echo "$1" |sed 's/^.*\([0-9]\.[0-9]\.[0-9]\).*/\1/')
## echo "${gcc_new_version_number} TEST TEST TesT CHOST"
# For selecting the highest version gcc
if echo "$1" | grep -Eq "gcc-[0-9].*" ; then
# stripping out the dots 4.1.1 ==> 411
gcc_current_no_dot=$(echo ${gcc_current} | sed 's/\.//g')
gcc_version_number_no_dot=$(echo ${gcc_new_version_number} | sed 's/\.//g')
# Now finding which no_dot is larger
if [ "${gcc_current_no_dot}" -eq "${gcc_version_number_no_dot}" ] ; then
high_gcc="${gcc_new_version_number}"
elif [ "${gcc_current_no_dot}" -lt "${gcc_version_number_no_dot}" ] ; then
high_gcc="${gcc_new_version_number}"
else
high_gcc="${gcc_current}"
fi
gcc_for_gcc_config="${CHOST}-${high_gcc}"
# Select the new GCC version for future compilations:
echo
echo "${Rd}BEFORE selecting new GCC:${CY}"
gcc-config -l
echo
# Changed output of pretend to be more readable.Sun May 28 23:20:39 new time stamp.
# For testing. use the first "if"
#if [ "$prtnd" != "p" ] ; then
if [ "$prtnd" == "p" ] ; then
echo "${BL}**********************************************************************${nc}"
echo
echo "${Yl} I would do:${CY} # gcc-config -f ${gcc_for_gcc_config}${nc}"
echo
echo "${BL}**********************************************************************${nc}"
echo
else
echo "${BL}**********************************************************************${nc}"
echo "${Yl} I'm going to execute: # gcc-config ${gcc_for_gcc_config}${nc}"
echo "${BL}**********************************************************************${nc}"
echo "${Yl} Kill me NOW, or forever hold your peace!${nc}"
echo "${Yl} (sleeping for 10 seconds...)${nc}"
for loop_counter in 1 2 3 4 5 6 7 8 9 10;do
echo -n "$loop_counter "
sleep 1
done
echo
echo "${Rd}(continuing...)${nc}"
gcc-config -f "$gcc_for_gcc_config" \
|| {
return_code=$?
echo "${RD}ERROR:${nc} ${Rd}gcc-config $gcc_for_gcc_config failed!${nc}" >&2
exit $return_code
}
echo
echo "${RD}AFTER selecting new GCC:${nc}"
gcc-config -l
echo
echo "${Yl}Now sourcing /etc/profile${nc}"
source /etc/profile
echo
fi
fi
fi
fi
}
function exclude_filter(){
#First run grep and put the excluded files into exclude.lst
for line in "$(echo $exclude_list)"; do
grep "$line" wrld.tmp >>exclude.lst
done
awk "!/$exclud/" wrld.tmp > wrld.lst
}
# generates the the build list, checks for blocked & masked pkgs
function wrld_lst(){
# If for blocking regeneration of list if resume or do_tc is used
# Added exclude_filter to filters here. That way it works "W" 5-8-06
#if [ "$resume" != "yes" ];then
if [[ "$resume" != "yes" && "$failed" != "yes" ]];then
emerge $bclass -${eargs}pq >wrld.lst
# checking for blocked or masked packages
if grep "blocks" wrld.lst ; then
echo
cat <<-END
${Yl}You have packages ${RD}blocking${Yl} each other. To fix run emerge
and take care of the blockage${NO}
END
echo
exit
elif grep "masked" wrld.lst ; then
echo
cat <<-END
${Yl}You have ${RD}masked${Yl} ebuilds. If its keyword masked add it
to /etc/portage/package.keywords. For more info see man emerge.${NO}
END
echo
exit
fi
# Changed slice dice 5-5-06
# wrld.lst includes TC files if any. Which if your doing -W, a world-system emerge means they
# get filtered out of the world list along with the system files
# removed sed -e 's/USE=.*//' -e 's/\[[0-9].*//' for cut. Sed is to specific re modular X's VIDEO_CARDS
#grep "/" wrld.lst | cut -c17-65 | cut -f1 -d " " | exclude_filter >>wrld.tmp
grep "/" wrld.lst | cut -c17-65 | cut -f1 -d " " >>wrld.tmp
if [ "$exclude.lst" != "catsisrats" ]; then
for line in "$(echo $exclude_list)"; do
grep "$line" wrld.tmp >>exclude.lst
done
fi
awk "!/$exclud/" wrld.tmp > wrld.lst
rm wrld.tmp
#mv wrld.tmp wrld.lst
fi
}
# The general purpose emerge function
function emerge_from_list(){
# blocks do_tc from running emerge_from_list with prntd
if [[ "$resume" != "yes" && "$failed" != "yes" ]] ;then
#if [[ "$resume" != "yes" ]] ;then
cat wrld.lst | tc_filter >> build
fi
# Now emerge
echo;echo -n ${YL}"$(wc -l build) to emerge "${NO}
build_count emerge_faild
}
# This is for the generation of the world files minus the system files and building them
function World_minus_system_emerge(){
# stops regenerating the build file if your resuming build
if [ "$resume" != "yes" ]; then
# wrld_list already run for system files.
cat wrld.lst >sys.lst
# filters out sys and TC files and puts the results into into the build list.
if [ "${do_wrld}" == "yes" ] ; then
bclass="world";eargs="eq"
# Calling wrld_lst function. It generates all the list
wrld_lst >wrld.lst
# It seems crazy but using sort comm and then re-ordering the resulting list is
# faster than a for loop and grep -v 8}
sort sys.lst>ssys.lst ; sort wrld.lst>swrld.lst
comm -1 -3 ssys.lst swrld.lst > sorted.lst
for i in $(< wrld.lst);do
grep $i sorted.lst>>build
done
fi
fi
# Now to build what was not removed by removing the system files
sleep 1
echo;echo -n ${YL}"$(wc -l build) to emerge "${NO}
build_count emerge_faild
}
# Main where the functions are called and things get done.
trap "exit 1" 1 2 3 15
color
getargs $@
info
wrk_dir
chk_failed_build
wrld_lst
#for doing resume
if [ "$resume" == "yes" ]; then
emerge_from_list
fi
# for doing the excluded list
if [ "$exclude" == "yes" ]; then
mv exclude build
emerge_from_list
fi
# For building the TC
if [[ "$do_tc" == "yes" || "$both" == "yes" ]];then
tc_check
if [ "$tc" != "no" ];then
tc_emerge
fi
fi
# For most of the emerges inculding -S
if [[ "$do_tc" != "yes" && "$do_wrld" != "yes" ]] && [[ "$both" == "yes" || -n "$bclass" ]]; then
emerge_from_list
fi
# does world --emptytree MINUS the system files
if [ "$do_wrld" == "yes" ]; then
# system files already generated now removal
World_minus_system_emerge
fi
clean_up
exit
# vim:set ts=4 sw=4:
|
# changing work dir to /tmp/emwrap
function wrk_dir(){
# change file locations to /tmp/emwrap/{FILES}
# Changed to explicit directory i
_________________
An A-Z Index of the Linux BASH command line
Last edited by hielvc on Fri Jul 06, 2012 5:53 pm; edited 98 times in total |
|
| Back to top |
|
 |
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3374
Location: USA
|
| Posted: Thu Jan 20, 2005 4:32 am Post subject: |
|
|
| OH YES!!! |
|
| Back to top |
|
|
hielvc
Advocate
Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca
|
| Posted: Thu Jan 20, 2005 5:40 am Post subject: |
|
|
Three things if you got Version 1.8 theres no update with out TC upgrade option  Thats the -N option.
If you want to see the actual ebuild update of the toolchain item remove the " -q " form | Code: |
emerge world -uDp>>wrld.lst
if echo "linux26" | grep -q linux26 wrld.lst
then a=1 lh2=1
fi
if echo "linux-h" | grep -q linux-h wrld.lst
then a=1 lh=1
fi
if echo "glibc" | grep -q glibc wrld.lst
then a=1 glb=1
fi
if echo "binutils" | grep -q binutils wrld.lst
then a=1 bu=1
fi
if echo "gcc" | grep gcc -q wrld.lst
then a=1 gc=1
fi |
from the grep lines.
If you want to remove or modify the list blockers you edit this | Code: | list_filter(){
awk !/DirectFB|source/ {print $1}'} |
in particular "!/DIrectFB|source/ " to block all kernel updates only change to " !/source/ "
Its easier and BETTER to use /etc/portage/package.mask.
_________________
An A-Z Index of the Linux BASH command line |
|
| Back to top |
|
|
drphibes
Guru
Joined: 30 Nov 2004
Posts: 432
|
| Posted: Thu Jan 20, 2005 6:06 pm Post subject: |
|
|
| why emerge the linux26-headers twice? isn't it just a sea of .h files with nothing really to build? |
|
| Back to top |
|
|
MindEraser
n00b
Joined: 06 Jul 2004
Posts: 26
Location: Germany
|
| Posted: Thu Jan 20, 2005 8:06 pm Post subject: |
|
|
Hielvc, sweet idea 'n all : ))
I have some critique about the implementation though 
Ok, I'll start with the minor things:
line 202: typeo: e[m]erge
- around 160: what is this supposed to mean?
Doesn't sth like | Code: | | if grep -q our.tc.keyword wrld.lst ... |
do the same?
- obviously you have given up function toolchain: can be deleted!
Now, the not so trivial stuff:
-If we want to --pretend, isn't it good to exit instead of continuing?
-So we have 16 possible combinations of TC updates availability and u only use 2 possible scenarios for the 2 versions of linux??-headers (which u dont need coz u can just store the exact version in a var and use it later). So if say we just have a gcc update:
| Code: | emerge linux-headers && emerge linux-headers glibc && emerge glibc binutils gcc && \
emerge binutils gcc |
We will totally UNNECESSARY and with almost NO effect emerge linux-headers, then again, then glibc too, then again glibc and binutils and finally come to the point where we upgrade gcc with the very same system we started with. Nice! Sorry, If this is not how your script works and I turn out to be a bad script reader!
The solution I would propose is: Figure out what makes most sence in each of the 15 cases (  This is your job, you re-re-compile the TC maniacs ) and assign the TC components to vars (Par1, Par2...) according to the order we want them re-compiled and pass these vars to emerge. Still we might need several scenarios with different number of emerge calls. Even if we don't act differently to each of the 15 cases a little bit more logic has to be applied than now exists. The least we could do here is start re-emerging the ebuild(s) that did change before the others and that with a predefined precedence if >1 changed.
So, what do u think of my suggestions, I hope they were useful. I will be happy to help with coding if u help me figure out what makes sence in what case. |
|
| Back to top |
|
|
Bob P
Advocate
Joined: 20 Oct 2004
Posts: 3374
Location: USA
|
| Posted: Thu Jan 20, 2005 10:34 pm Post subject: |
|
|
For all of the new users who aren't familiar with the perils of rebuilding your toolkit, it is absolutely essential to perform redundant builds of toolkit after changing a major toolkit element, such as linux26-headers, glibc, or gcc! heil's design to perform these functions redundantly was not an accident -- it was a design objective. this is what you have to do to ensure a stable, reliable, and robust toolkit.
if these steps seem unnecessary to you, then you should read this:
https://forums.gentoo.org/viewtopic.php?t=248245&start=53
it turns out that heil's script is a more efficient method of performing the same redundant operations. but instead of taking a brute force approach, heil's script hits the exact components that need to be hit, in proper order, to perform all of the necessary steps while reducing your compile time. |
|
| Back to top |
|
|
hielvc
Advocate
Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca
|
| Posted: Fri Jan 21, 2005 4:45 am Post subject: |
|
|
MInderser | Quote: |
Code:
if grep -q our.tc.keyword wrld.lst ... |
I'll try it and thanks the typo check.
When you run the -p, pretend, option thats all it does is checks and stops.
As to the TC rebuild, is a double emerge of linux-headers over kill, was sort of what I was leaning towards until last week when I came across a thread about "tk or tcl" and the fix from the bug report was to re-emerge linux-headers and glibc and that fixed it for 4 or 5 people. If you look at the header youll see the original TC thang only did a single linux-headers emerge . For most users of gentoo that is probably adequate. For 2 years I didnt care about the toolchain. Then I went up to linux26-headers and then glibc ~ARCH and in the threads people were haveing weird probs. They were solved by emergeing glibc twice and in some cases linux-headers twice. If your going to do this why not make damn sure and thats the point of this, its LH twice.
As to a gcc or binutils up date no I wouldnt rush out and rebuild my TC. For my use and my thoughts if LH and glibc are nailed down then just rebuilding gcc and binutils should be fine. For Robmoss and someone who is testing everything that is built then this should save them time and they will use it every time they have a TC update. But I am getting to the point where monthly or bi-monthy I am doing either "emerge system or world -e ". After really looking at how portage handles the building the TC I'll be using my wrapper. As a matter of fact the last two options I'll add will be TC with system -e and TC with world -e. The other reason why this is important to Gentoo is that its source based and everything depends on the TC. With fedora. mandrake and such you dont even need gcc.installed. Try that with Gentoo. 
_________________
An A-Z Index of the Linux BASH command line |
|
| Back to top |
|
|
piwacet
Guru
Joined: 30 Dec 2004
Posts: 486
|
| Posted: Fri Jan 21, 2005 8:24 am Post subject: |
|
|
Hi. Thanks for the script. Quick question. My system was build with Bob P's stage 1 NPTL on Stage 3 tarball using gcc3.4.3. I needed to umerge linux-headers and emerge linux26-headers when creating my system. I assume I should edit your script to use the same linux26-headers?
Thanks! |
|
| Back to top |
|
|
hielvc
Advocate
Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca
|
| Posted: Fri Jan 21, 2005 3:34 pm Post subject: |
|
|
No need it already checks for which version of linux-headers that your running. and then uses the tc2_filter or tc_filter.
_________________
An A-Z Index of the Linux BASH command line |
|
| Back to top |
|
|
MindEraser
n00b
Joined: 06 Jul 2004
Posts: 26
Location: Germany
|
| Posted: Fri Jan 21, 2005 3:58 pm Post subject: |
|
|
Hielvc, Bob P,
Now don't get me wrong guys. I'm a noob only in the forums (havn't posted much) but definitely not to gentoo. I do consider emerging and re-emerging linux??-headers a good thing IF there is an update to it (especially that it is done in minutes). My point was to do all the re-emerges in the right order according to what TC updates are available. Now you know what you are doing but suppose a noob uses the script, sees an update of gcc and decides to go for the TC option in the script. This is the scenario I tried to explain in my previous post. If there is an update only to gcc and the latest linux-headers and glibc have already been re-re-emerged, wouldn't it be a redundancy to do it again? So what you are suggesting is basically go for the TC only for linux-headers and glibc updates but don't rush for gcc and binutils. Ok, that's fine and still the script would better do the right thing when someone decides to go for the TC, or?
One more thing: Again if there is no update to linux-headers but to sth else and someone goes for a TC. The script will then do the linux26-headers TC even if the user has linux-headers installed as $lh will be 0!
And a TYPEO at line (about) 220: }} brackets where ]] are needed.
I hope I have better expressed my point now and am waiting for comments. Thank you! |
|
| Back to top |
|
|
hielvc
Advocate
Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca
|
| Posted: Fri Jan 21, 2005 10:11 pm Post subject: |
|
|
No the current listing should be correct. Man version control is driving me nuts. After I tested the "if grep -q tc.word wrld.lst" I edited it posted and started try some other things and script was broken. As you pointed out my " ]]", end brackerts had transformed to "}}", end braces and either at the same time or a little latter hafl of " fi "s were gone. I dont know exactly how I managed that but clever me did it somehow.
As to " to TC or not to TC that is the question" I agree that for most people if linux-heaaders or glibc arnt updated then doing the total TC thang isnt needed. Oh well maybe things will slow down and I can work on it more on Sunday. Lets see instead of 2 more options I'll need at least 3. 1 for binutils and gcc mini-thang and sys/world -e. I thing the same guy who said "to TC or not to TC" also said "permutation, fermentation which do you perfer?"
_________________
An A-Z Index of the Linux BASH command line |
|
| Back to top |
|
|
marder7
n00b
Joined: 15 Nov 2002
Posts: 56
|
| Posted: Sun Jan 23, 2005 12:55 pm Post subject: |
|
|
I just have tried to run the script, but it says:
| Code: |
emrg-wrapper.sh: line 252: syntax error: unexpected end of file |
I double checked if anything went wrong copy&pasting it, but everything seems to be ok. My editor isn't complaining neither. I followed your installation instructions exactly.
Any idea what's wrong in the script?
_________________
This ist just another signature. |
|
| Back to top |
|
|
hielvc
Advocate
Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca
|
| Posted: Sun Jan 23, 2005 3:33 pm Post subject: |
|
|
You were lucky enough to get the original first draft that was accidently posted.
I just put up the lastest revision. It changes structure clarifies help some and should run.
_________________
An A-Z Index of the Linux BASH command line |
|
| Back to top |
|
|
hielvc
Advocate
Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca
|
|
| Back to top |
|
|
marder7
n00b
Joined: 15 Nov 2002
Posts: 56
|
| Posted: Sun Jan 23, 2005 8:03 pm Post subject: |
|
|
the same error still occurs. hm.
I have taken a close look at the code again, but since I am no shell script wizard (far from it), no wonder that I couldn't find a mistake...
_________________
This ist just another signature. |
|
| Back to top |
|
|
hielvc
Advocate
Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca
|
| Posted: Mon Jan 24, 2005 1:36 am Post subject: |
|
|
marder7 Just posted version 1.8.8 dated 1.23.05 I swear its working I just used it  It offers correct filtering for binutils and gcc. If you do a "emerge gcc -s or the same for binutils theres about 4 packages for binutils and 8 for gcc. They are now handled correctly.
_________________
An A-Z Index of the Linux BASH command line |
|
| Back to top |
|
|
MindEraser
n00b
Joined: 06 Jul 2004
Posts: 26
Location: Germany
|
| Posted: Mon Jan 24, 2005 5:18 am Post subject: |
|
|
Ok, Hiel! Here is my version! I hope you all will appreciate that:
Download my version of tcupdate.sh
!!! Attention. Since this version 0.2.1.0 -e option has a NEW meaning. It's been replaced by -x. Please read help!
!!! Please note - I have totally changed the way the options work. Each of -w and -t specifies if the world (rest) and the TC should be build accordingly. They can be supplied together or only one of them. -p with any combination of the other options will tell you the exact consequences of the supplied options.
Enjoy!
EDIT: Ha, and of course now one can specify the target: world or system
EDIT2: Posted again very revised and improved version
EDIT3: New options, some changes and improvements. Uploaded the script in the web
EDIT4: 0.2.2.0
Last edited by MindEraser on Mon Jan 31, 2005 12:52 am; edited 4 times in total |
|
| Back to top |
|
|
Genfoo'
Tux's lil' helper
Joined: 24 Feb 2004
Posts: 112
|
| Posted: Mon Jan 24, 2005 5:43 am Post subject: |
|
|
MindEraser, how did you do that? I mean, how did you update it before it was even written.
 |
|
| Back to top |
|
|
hielvc
Advocate
Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca
|
| Posted: Mon Jan 24, 2005 6:26 pm Post subject: |
|
|
MindEraser I think you forgot to include the " exclude="-sources-2." list. When I run " tcupdate -pw|grep gcc " I get | Code: | ./tcupdate.sh -pwe|grep gcc
[ebuild N ] sys-devel/gcc-3.4.3-r1
[ebuild R ] sys-devel/gcc-config-1.3.9 |
Re-read the help listing and removed "e", but it still throws out the above and I cant find the "exclude" listing besides -sources-2. and its not difined  Haveing become an expert in these sorts oopsise it was the first thing I spoted.
_________________
An A-Z Index of the Linux BASH command line |
|
| Back to top |
|
|
MindEraser
n00b
Joined: 06 Jul 2004
Posts: 26
Location: Germany
|
| Posted: Tue Jan 25, 2005 12:37 am Post subject: |
|
|
Hiel, honestly I did not get at all what you are trying to tell me and what the problem is. The exclude list is by default "-sources-2." It is appleid to the world list when the -e option is used. If a regular expression is supplied after -e it overrides the default. A little bit more about it at the end...
If your consern is about:
| Code: |
[ebuild N ] sys-devel/gcc-3.4.3-r1
[ebuild R ] sys-devel/gcc-config-1.3.9
|
Then for gcc: it doenst mean that it will be build. Look at my output. It lists it as an available tc update but as you see it has no intention of mergeing it. For gcc-config: we havn't yet treated it specially. Should we?
| Code: |
MindEraser ~ # tcemerge -pw -e xorg system
*** Following ToolChain Updates Available:
[ebuild U ] sys-devel/gcc-3.3.5-r1 [3.3.4-r1]
Merging following world/system packages in order:
These are the packages that I would merge, in order:
Calculating dependencies ...done!
[ebuild U ] sys-apps/man-1.5p [1.5o_p2]
[ebuild U ] sys-devel/m4-1.4.2 [1.4.1]
[ebuild U ] sys-apps/help2man-1.35.1 [1.33.1]
............. And so on...
MindEraser ~ #
|
As expected an update to xorg-x11 which would otherwise be included gets filtered out. The TC exclude list if that is what you meant is defined in the awk phrase in the tc_filter() function as it has awlays been.
And now back to the -e option. It is a little bit tricky. This comes from how getopts parses options. As I have declared that -e can accept an argument, if one does not supply one to it and puts it on the wrong place it will brake option parsing:
Good uses of -e:
| Code: |
#tcupdate -wte
#tcupdate -wt -e
#tcupdate -e -wt system
#tcupdate -e xorg -tw system
|
Bad uses of -e:
| Code: |
#tcupdate -epw
#tcupdate -wt -e system
|
In the first case it will parse pw as an argument to -e and in the second - sytem, while the user would expect that system is what is to be updated.
Maybe I should undeclare it as accepting an argument and include parsing code for --exclude=regexpr.
Besides that, do you like what I have done? Can I assume that you accept my changes? I will try to merge the TCmini functionality ASAP.
A beautiful day/night to all!!! |
|
| Back to top |
|
|
hielvc
Advocate
Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca
|
| Posted: Tue Jan 25, 2005 1:57 am Post subject: |
|
|
No I like what youve done. I'm sort of flat tonight so Im enjoying myself by playing with your exclude. It has possibilblities.
_________________
An A-Z Index of the Linux BASH command line |
|
| Back to top |
|
|
fredor
Tux's lil' helper
Joined: 13 Oct 2003
Posts: 84
Location: UK
|
| Posted: Tue Jan 25, 2005 2:23 pm Post subject: |
|
|
I have run the tcupdate program with the '-t' argument and it works fine. If, however, I run 'tcupdate -e -pw system' there is no output.
It's very useful to ba able to update TC without doing the full system.
Many Thanks |
|
| Back to top |
|
|
hielvc
Advocate
Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca
|
| Posted: Tue Jan 25, 2005 3:14 pm Post subject: |
|
|
The "print \$1 should be "print $1 in | Code: |
exclude_filter(){
awk "!/$exclude/ {print \$1}
########should be#########
exclude_filter(){
awk "!/$exclude/ {print $1} |
_________________
An A-Z Index of the Linux BASH command line |
|
| Back to top |
|
|
fredor
Tux's lil' helper
Joined: 13 Oct 2003
Posts: 84
Location: UK
|
| Posted: Tue Jan 25, 2005 3:40 pm Post subject: |
|
|
Adjusted 'print \$1' in exclude_filter but there is still no output with 'tcupdate -e -pw system'
If I run 'tcupdate -pw system' this seems to work OK telling me there is an update for gcc but I thought using '-e' would have listed the system files less TC.
Thanks for your quick response. |
|
| Back to top |
|
|
MindEraser
n00b
Joined: 06 Jul 2004
Posts: 26
Location: Germany
|
| Posted: Wed Jan 26, 2005 1:37 am Post subject: |
|
|
fredor:
You are obviously using my version. Copy the new one, the problem should be fixed. Read the help. Thanks for the bug tip : )))
hielvc:
No you were wrong about the "print \$1". I need to escape it because of the double quotes.
Posted new version.
Fixed the problem with -e (now the short option does not accept an argument). Added --exclude= for that. Read the help!
Put some logic in the tc_emrg(). Needs further development but for now should be ok. If sth in tc fails though the script exits with a warning. There is a serious problem if sth in tc fails and continuing is not a good idea according to me. If you want to test it just override the values of lh, bu, glbc, gcc & gcccfg after the detection and see what happens.
My thoughts about the future:
Could you please write a sed function to put back the original color in piped emerge output. Especially for the use flags when invoked with -v as I am thinking of adding -v option with the same effect.
We could easily extend the script so that not world and system but any package(s) can be updated.
I was thinking also about emerge -e world/system after tc option.
And last but not least these 2 that would require user input (perhaps this behaviour could be everriden with an option):
- if the new gcc installed fills a new slot ask the user to choose a profile
- give the user the option to interactively define the contents and order of the tc build : ))) (just say what you think about that, I will do it). |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Unsupported Software
