An emerge wrapper for breaking emerges into chunks

slycordinator · Advocate Joined: 31 Jan 2004 Posts: 3065 Location: Korea

exclude seems to not work.

hielvc · Posted: Sat May 20, 2006 12:15 am Post subject:

slycordinator Its working for me.

slycordinator · Advocate Joined: 31 Jan 2004 Posts: 3065 Location: Korea

Bob P · Posted: Sun May 21, 2006 10:28 am Post subject:

curmudgeon · Veteran Joined: 08 Aug 2003 Posts: 1741

I still get a lot more output, even when running the exact same command:

dol-sen · Posted: Mon May 22, 2006 4:29 pm Post subject:

You are not using the latest version of emtst.sh. The latest will show the version and time in the header, yours does not, so it is an older version which does not abreviate the output.
_________________
Brian
Porthole, the Portage GUI frontend irc@freenode: #gentoo-guis, #porthole, Blog
layman, gentoolkit, CoreBuilder, esearch...

hielvc · Posted: Mon May 22, 2006 4:33 pm Post subject:

Bob_P Dang electons, sigh, there are days when I could misspell " a ". I keep mine fit by running and testing this script and emerge. :wink:

curmudgeon could you get the latest emtst.sh ? If you look at slycordinator output

curmudgeon · Veteran Joined: 08 Aug 2003 Posts: 1741

Still no go on the brief output. I must have something different in my system, but I can't figure out what:

hielvc · Posted: Tue May 23, 2006 10:57 pm Post subject:

What does " emerge system -uDqp " look like? If its there are no updates needed post the output "emerge system -eqp" . The output differences is strickly the differnce of emerge with and with and without out the "q" flag.
_________________
An A-Z Index of the Linux BASH command line

curmudgeon · Veteran Joined: 08 Aug 2003 Posts: 1741

I don't see any difference. :(

hielvc · Posted: Fri May 26, 2006 4:37 am Post subject:

It must be a portage-2.1 thing. Actually now that I think of it it is. In portage-2.1 when you run emerge it shows USE flags. The "q" just quites the output to show just package names. Well they've been saying that the 2.1 series is gong to turn stabe one these days.
_________________
An A-Z Index of the Linux BASH command line

jsnorman · Tux's lil' helper Joined: 24 Feb 2005 Posts: 131

I have recently upgraded to gcc-4.1.x from gcc-3.6.y and used emtst 4.0.4 to do it (thanks!). However, I have run into a problem that really screwed things up on my last world build.

First, I started out doing purely manual gcc/glibc core emerging as follows: linux-headers, gcc (to 4.1), gcc-config, linux-headers, gcc, glibc, binutils, gcc, glibc in that order. That worked fine.

Second, I double checked gcc-config to confirm my 4.1 profile, and then ran emtst.sh -set1 (not twice since I had alread built gcc/glibc manually once correctly). That worked as well.

Third, I ran emtst.sh -web && emtst -W and let my system do its thing for a few days :-)

However, I do not recall if I checked my gcc-config profile at this stage (probably did but not sure - even so, I ran this command immediately after -set1 finished so gcc should still have been set at 4.1, right?)

I have been monitoring things everyonce in a while, and saw that -web finished early yesterday, and -W had started. By this morning, -W was 400 or so out of 1000 files finished... BUT at some point I restarted my X server and my nvidia module crashed... investigation revealed it had been built with a different version of gcc than the kernel. looking at dmesg, i see that nvidia was built with 3.6!!! gcc-config -l confirms my fear, and I see that the current working gcc version is in fact 3.6!? which means that my emtst.sh has been recompiling everything using gcc 3.6. Ugh.

I shoudl mention this is the second time I have had this happen, so I was very careful doing the manual emerge's of linux-headers, gcc, glibc, binutils, and -set1 to make sure I had the correct gcc. Once -set1 finished I thought I was home free but apparantly something is causing gcc to flip backwards to 3.6.

How can I prevent this in the future?

hielvc · Posted: Mon May 29, 2006 5:37 am Post subject:

jsnorman I dont know the cause of the problem. I delebertly installed gcc-3.4.6 knowing that there is a "r1" update. and I have upgraded to gcc-4.1.1 but Ive been unable to cause the problem. So I put in maguires auto gcc-config. Appearently it works becauce it there havint been any complaints about it.

emtst.sh new version 4.0.5 Sun May 28 21:49:26 PDT 2006
_________________
An A-Z Index of the Linux BASH command line

jsnorman · Tux's lil' helper Joined: 24 Feb 2005 Posts: 131

Just finished rebuilding 4.1.1 without a hitch. I think it may have been the fact that when I sstarted last week, 4.1 was still masked/unreleased ... somehhow this must have caused reversion at some point to 3,6 of gcc? anyway, 3d time;s the charm iguess, and seems ok with 4.1.1

slycordinator · Advocate Joined: 31 Jan 2004 Posts: 3065 Location: Korea

hielvc · Posted: Mon May 29, 2006 5:41 pm Post subject:

slycordinator i agree that thats whats happening but I've never been able to see this behavuior. Like you I remove the older versions of gcc. The problem is that most peeps dont. I deliberately reinstalled gcc-3.4.6 last night so that I have gcc-4.1.1 and gcc-3.4.6 . Now there is an update to gcc-3.4.6 to gcc-3.4.6-r1. I then ran emtst.hs, emwrap.sh emerge system & world -ep and does the "r1" update show up ? No. Will it yes but when and why have me stumped. Right now I'm favouring the blue moon theroy but a close second is the earth sucks and its time to remind you of it :twisted:

So the problem with the script or portage is that Murphy strikes and gcc-3.4.6-r1 finally pops up and the script will happily change you to gcc-3.4.6 . Now I can change the gcc_config function to always select the highest version of gcc. But there are people who want the harden and such. versions of gcc. Well I think I should put a feature request in for gcc-config/portage to have something like GCC_PRFERNCE="hardenedpiessp" in /etc/make.conf. My thoughts for emwrap.sh is to have it select the highest plain version of gcc.

Slightly newer emtst.sh mainly color and formating changes to gcc_config and added credits and GPL back into top comments.
Timestamp May 29 10:46
_________________
An A-Z Index of the Linux BASH command line

hielvc · Posted: Tue May 30, 2006 1:09 am Post subject:

The newest emtst.sh which will always select the highest gcc version when it runs gcc-config after updateing gcc.

TIMESTAMP .Mon May 29 18:10:20 PDT 2006
_________________
An A-Z Index of the Linux BASH command line

hielvc · Posted: Tue May 30, 2006 2:31 am Post subject:

Submitted a feature request in regaurd to a GCC_PREFERNCE variable to be added to /etc/make.conf. If yourt interested its Bug# 134856
_________________
An A-Z Index of the Linux BASH command line

jsnorman · Tux's lil' helper Joined: 24 Feb 2005 Posts: 131

Wow, that was quick. New emtst.sh in my sbin.

Thanks!

Jeff

jsnorman · Tux's lil' helper Joined: 24 Feb 2005 Posts: 131

hielvc · Posted: Tue May 30, 2006 2:48 am Post subject:

Tommorow or the day after my brain is fried.
_________________
An A-Z Index of the Linux BASH command line

zigver · Tux's lil' helper Joined: 09 May 2003 Posts: 87

Outstanding work, hielvc. I've spent a couple of hours reading through all the posts and would like to clarify a couple of things.

1) For emwrap.sh v. 4.0.2, the help says the "-S" is short for a "emerge system --emptytree" build. But it does not build the TC so it's really an "emerge system --emptytree minus the TC" build. Do I understand correctly?

2) From your post on Tue Apr 04, 2006 5:13 pm, the current (most current I could find) correct way to do a full rebuild using emwrap.sh v4.0.2 is:

hielvc · Posted: Wed May 31, 2006 10:00 pm Post subject:

FIRST moved emstst.sh to emwrap.sh. This is version 4.0.5 TIMESTAMP or Date " Mon May 29 18:10:20 PDT 2006 " . It has the new "exclude and/or -X" flag for excludeing things for things from being built. It also has the gcc-config form maguire's script that I moded so that will automaticly always set gcc-config to highest gcc-version.

Today or tomrorow I'm going to change the "exclude" function. What it does now is to remove anything that matches a simple regex. I was looking at my emerge system --emptytree which now numbers 261 files. One of the main purposes of this script was so that you break builds into smaller chunks and do them over night. What with system and world --emptytree becomeing so big it dawned on me that if the excluded files were put into exclude.lst and I modified function chk_failed_build to detect it then it could be used subdivide these large builds for slower systems, boxin that is (slow cpu's ok). On my box most of the added files to emptytree system are media-fonts and x11. So if I ran "emwrap.sh -se -X or emwrap.sh system --emptytree exclude x11 media-fonts " that reduces my list to 171 packages. Then the next time you run emwrap.sh or if you couldnt resist and ran it a bunch of times for whatever reason you could rerun "emwrap.sh -sep -X x11 media-fonts " to regenerate the exclude.lst and build it. Things to do on thsi figure out what to call this flag and write the code.

zigver For question 1 yes if you dont give the "t or b" flag then the TC is not built so its system-TC.
For question 2 if you get the latest emwrap.sh it will set gcc-config for your highest version of gcc. If this is not the behaviour you want then youll have build the TC on the command line and set gcc-config. After that you can use emwrap.sh again , just dont rebuild your TC with it. Ok back to 2 that code listing is the safe way to rebuild your system and world.

If your upgradeing to gcc-4.1.1 then

jsnorman · Tux's lil' helper Joined: 24 Feb 2005 Posts: 131

zigver · Tux's lil' helper Joined: 09 May 2003 Posts: 87