An emerge wrapper for breaking emerges into chunks

[fredor]

MindEraser: Yes it was your script I used but I seem to have misunderstood. I thought the objective was to emerge the toolchain seperately from 'emerge -e system/world'
With your script as it stands it only picks up the updated ebuilds so if there are none available the build.lst is empty.
If you change the code as follows:-

[hielvc]

fredor Thanks for the summury. What I actually had in mind was

1 Mini Toolchain
2. Toolchain
3. emerge -e system less Toolchain
4 emerge -e world less System less Toolchain
5 emerge -e world less Toolchain
6. emerge -uD sytem less Toolchain
7. emerge -uD world less System less Toolchain


Which would be very easy to add or compelet now that MindEraser has gotten the code so far. I not sure about MindEraser but once I started, getting all the oopies and "oh I forgot to have it do that like TCfiltering here or buid there" and getting the right working revision overrode "the lets add that feature".

The only one of the above items that migh be trickier than first appearances would be #4. The nice feature that MindEraser added and diffently has posibilities is the "-e" taking multipule agr's, particularly with what I think MindEraser wants to do with added use of emerge -v. And I'll take a look at color output MindEraser.
_________________
An A-Z Index of the Linux BASH command line

[hielvc]

MindEraser if the filters are run and then the "emerge for loop" youll get color output. Will this do what you need?
_________________
An A-Z Index of the Linux BASH command line

[fredor]

heilvc What you suggest would be great. As you say item #4 is the hardest. I'm not really a programmer and the only way I can think of is to use 'diff' but I'm sure there is a better way.
I was wondering if initially a build list could be created for each of the system and world sections. Then the rest of the programmng would be relatively easy. Anyway, I'll leave it to you experts.
Many thanks.

[hielvc]

While driveing home I was thinking about #4 "emerge -e world less system less TC" A simple " TCupdate -system && TCupdate -world." takes care of it. If we start caching files of a system update to remove from a world update its only going to be good till your next sync. I see nothing but a can of worms trying to do that and to keep track times and dates when portage already does the data base tracking for us. ARRRR taht only works for -uD and not --emtytree.
_________________
An A-Z Index of the Linux BASH command line

[hielvc]

MindEraser Ok I'm sorry for takeing so long to respond. But work has become quite
busy which is good I make money but getting home after 8 and 9 leaves me with a fried
brain. As to new gcc install I would want to use the newer gcc. As to haveing the user
select it, well over the last 4 months sometimes you need to it and sometimes portage does it
for you and sometimes you have to create the profile yourself. Currently libaries are
being broken, be it gcc, ncurses X but alot post are about them Which segs in to your other
question YES IF THE TC FAILS WE NEED TO STOP. I actually had that in one of my versions.
I dont think that we should let, invite someone to try and define the order of their TC
build. Those who are that advanced can hack the script easily, those who arent will have a
harder time blowing their own feet off.
_________________
An A-Z Index of the Linux BASH command line

[MindEraser]

Hi, guys, I've been busy lately too. If I find a couple of hours free I will try to implement some of the features.
First about the colour. What I meant is I want to grep (or otherwise pipe the output of emerge) but retain the color. Since emerge detects that and spills colorless output the only thing I can think of is putting it back in with sed (like you hielvc color your help). Then I can go on implementing -v. I would be happy to get rid of the "These are the packages that ... in order.." and "Calculating dependencies" messages by emerge. The problem is relevant mainly for TC, since for world when pretending (for speed and color reasons) I don't actually run the emerge loop as the real run does but instead do a emerge -p the_whole_build_list. Perhaps I should do the same for TC: first let portage state the exact versions (when -v the use flags too) for TC and then just print the exact order of merging.
About the 7 points.
1 done
2 done
6 done
3 & 5 I dont see the point of that if one nails down the toolchain first then no problem rebuilding it again with system.
4 & 7 Very interesting. Worth the try. I think we can figure out dynamically which packages portage considers system at the moment (perhaps with emerge -e --nodeps system or if not i will think of sth) and filter them out from emerge -e/-uD world.
If we r going to implement the 4 & 7 options then the current -e is perhaps misleading. Better change it to -x (eXclude), uh??? And then use -e for empty-tree...
hielvc, can I then assume that you tested the way my script builds the TC and you agree? Any corrections?
I'll be around when I find some time.

[dbaigrie]

The froced TC update requires the variables to be set or it won't merge anything!
_________________
--------------------------------------

The sign of a truly advanced intelligence is the ability to hold two opposing thoughts and still be able to function rationally.

[MindEraser]

[fredor]

MindEraser The reason for #3 and #5 is time.
For me it take 3 to 4 hours for the Toolchain (with multilib)
ca. 5 hours for 'emerge -e system
ca. 16+ hours for 'emerge -e world'
So it is an advantage if the Toolchain is removed from 'emerge -e system.
I have just updated a Gnome system to the 2005.0 profile and afterwards used your script to re-compile the Toolchain. I am now using the script with 'emerge -e system' replacing the 'emerge system -uD' to compile the system packages less Toolchain. So far it's running OK.
One thing I noticed, however, it effectively compiles one package at a time and the heading shows 'emerge (1 of 1) so unless you have a file list you don't know how many packages have been compiled at any one time. It may be difficult to alter this.

[MindEraser]

[fredor]

The Systems less Toolchain compilation has just finished taking just about 2 hours which is not bad since it includes 'xorg'
So I am glad you agree on #3 and #5.

[hielvc]

MindEraser I was testing your TC build. There is an extra build of gcc if you have an gcc update with either a glibc and/or linux-headers update. The rest work well.

[MindEraser]

hielvc,
I know that. As I said before there are 15 combinations of at least one tc update. I just tried to give my best figuring out what is best in which case. I pushed one gcc more so that we start using the new compiler as soon as possible. Your help will be very appreciated here. What I know tells me this:
The priority for merging is as follows:
os-headers, glibc, gcc, binutils
(... or, perhaps better os-headers, gcc, glibc, binutils - I really don't know)
Furthermore, since os-heasers don't get compiled then just emerge them twice in the beginning.
I did a little bit of thinking and I am convinced now that I was overcomplicating things. As I see it rebuilding the TC should be in 2 steps:
STEP 1: Update all TC packages to the newest version.
- This part should vary according to available updates
STEP 2: Rebuild the TC. Fixed. I would go for glibc, bu, gcc, glibc, bu, gcc.

My question is, shouldn't we take care of portage too? I've seen it appear in the middle of the list for world updates. And although it is mainly bash and python there are a few binaries for sandboxing and stuff that get compiled.

I'm getting to work. I'll fix the toolchain and all the options I've promised. I'll post when I am ready.
_________________
Portage updates your ToolChain (gcc, glibc...) in a very stupid way. Here is the solution: an emerge wrapper script

[MindEraser]

Posted new version.
Addressed all requests except the thing with emerge (1 of 1).
Don't forget to read the help, options tend to change!
Use the link below to go to my post with the link.
_________________
Portage updates your ToolChain (gcc, glibc...) in a very stupid way. Here is the solution: an emerge wrapper script

[rhill]

wow, this thing really took off. amazing work guys.

just as a thought on the priority, i was looking into the toolchain's build order myself a little while ago and i settled on following the LFS suggestion of headers->glibc->binutils->gcc. they actually build binutils first during the bootstrap.

[MindEraser]

Thanks for the tip, dirtyepic
I don't know if it really matters in which order I update the packages since I am re-emerging everything twice afterwards. I was trying to push gcc in front as I don't see much sense in compiling things with the old gcc.
Anyways, if one doesn't like my order she can edit the script & redefine it

_________________
Portage updates your ToolChain (gcc, glibc...) in a very stupid way. Here is the solution: an emerge wrapper script

[MindEraser]

New Version again: 0.2.2.0
Cool new features and other improvements
As usual, (can't say that enough) READ HELP!
Follow, the link in my sig to go to the post with the actual link.

TODO:
(I've already laid down the bases for that so expect soon!)
- Status check. The script will write current progress info in a file and calling it from another shell with --status option will give you the info.
Can't think of any way of overriding what emerge puts on xterm titles.
- Control over which dirs are used for tmp and log
_________________
Portage updates your ToolChain (gcc, glibc...) in a very stupid way. Here is the solution: an emerge wrapper script

[landon]

Hmm, came across this particular thread after clicking on 2 different signatured-urls. Which script is currently being developed? Is it now a joint development?

[MindEraser]

Kinda joint development, yeah
Use mine : )) It's the latest version
_________________
Portage updates your ToolChain (gcc, glibc...) in a very stupid way. Here is the solution: an emerge wrapper script

[hielvc]

1st added a link above my script to MindErasers.

I did manage to look around some considering the build order. Dirtyepic is right. The question I have is should the order be " glibc binutils gcc && glibc binutils gcc " or should it be "glibc glibc && binutils gcc && binutils gcc". Ive seen both and and done both. I wonder which is more correct or if either is more correct. Any all knowing peeps out there?

Minderaser as to portage I agree I like to emerge it first then the rest of whatever.
_________________
An A-Z Index of the Linux BASH command line

[rhill]

[MindEraser]

Hey, I got it right then
At least the rebuilding part
What about the order of initial updating/building?
Do you agree it should be
portage - 2x os-headers - glibc - binutils-cfg - binutils - gcc-cfg - gcc

hielvc, thanks for the link

How do you guys find the new features?
Any bugs, corrections, wishes?
_________________
Portage updates your ToolChain (gcc, glibc...) in a very stupid way. Here is the solution: an emerge wrapper script

[fredor]

TCupdate is coming along fine now - Well Done !!!
'tcupdate.sh -tep' gives the following :-

[fredor]

Running tcupdate.sh with the '-e' option picks up up all the toolchain packages and adds them to 'tc_list'.
Then for the forced 'tc' update the 'tc_build' is added to this list produced the list in the previous post.
I have modified my copy adding a 'do_tc_force' flag and using this so the 'tc_list' is made to equal only 'tc_build'.
It works fine now