View previous topic :: View next topic |
Author |
Message |
Lews_Therin l33t
Joined: 03 Oct 2003 Posts: 657 Location: Banned
|
Posted: Tue Feb 08, 2005 6:47 am Post subject: Re: A solution for game cd's and other .iso's |
|
|
truekaiser wrote: | Lews_Therin wrote: | tsawyer wrote: | As Gentoo becomes more popular, especially with the introduction Gentoo Games, I would guess that serving ISOs generates much of the load on the mirrors. Seems to me that if the Gentoo community would adopt an open p2p solution, such as Gnutella, we could make the ISOs more readily available. This would hopefully reduce load on the mirrors for rsync purposes. Gnutella has the additional benefit of being widely used by Windorks who are likely to download the game CDs.
We could finally use p2p for legitimate purposes.
Thoughts? Ideas? |
There are already official torrents for the ISOs. When I used them last, I got a steady 300 kb/s |
he means for the larger packages in portage. |
I was responding to tsawyer, not the original poster, and tsawyer is most certainly talking abou the installation ISOs. |
|
Back to top |
|
|
mikenerone n00b
Joined: 11 Feb 2004 Posts: 22 Location: San Antonio, TX
|
Posted: Tue Feb 08, 2005 1:21 pm Post subject: |
|
|
Relevant discussion at Bug 78407. |
|
Back to top |
|
|
Arkham Tux's lil' helper
Joined: 30 Aug 2004 Posts: 111 Location: Quilpue, Chile
|
Posted: Tue Feb 08, 2005 2:02 pm Post subject: |
|
|
I think the "Gentoo way" is the distcc aproach... sharing binary packages with "popular" flags will led people to standarize theyr flags and you will be punished for selecting EXTREME tweaks
My thought only..
Can someone post numbers here??
Assuming there're enough IDLE machines, and those machines have the tarballs, how long would it take to compile KDE FULL (with split ebuilds of course )
2 hours? 1? 30 min??
Anyway, I think the benefits for the comunity are huuuuuge. It may worth a try
? |
|
Back to top |
|
|
LucaSpiller Apprentice
Joined: 10 Sep 2004 Posts: 188 Location: Censorship Land (aka England)
|
Posted: Tue Feb 08, 2005 3:07 pm Post subject: |
|
|
Since I saw this topic last night I decided to do a bit of research into this to see what has been discussed before. Rather than discussing just sharing specific packages most of these focused on sharing the whole of whatever you have in /usr/portage/distfiles/.
There was also some discussion of sharing the ebuilds themselves, but some sort of security signature would have to be added by the developers so fake ebuilds cannot be made that could damage your system - patches and whatever would also have to have this. The list of the public keys would have to be hosted on the Gentoo site or mirrors so that it is known to be secure. Have a look at this post for more info (the whole topic is actually worth a read).
I really don't think a BT type system would work, it is designed for people to share single (and usually) large files easily without having to do much work. You would have to set up a webserver to share the .torrents, ok so that wouldn't be that much but it will take up bandwidth, and if the current mirrors themselves are seeding the files they will take most of the brunt. Ok so every kb saved helps them but it won't make a huge difference.
I believe a serverless p2p service such as Kad(emlia) would be much better suited to the job because there is no single point of redudancy. Also rather than sending the whole file if somebody already has an older version a Deltup type system (see my previous post in this topic) could be used to send out just the changes. I believe one of the devs (see the topic I linked to above) was working on a p2p Portage but nothing new has really been announced so I don't know if it is still going. _________________ :: Luca :: Mac Fag :: Original Macbook, 2g RAM :: Closet Linux user (seasoned with salt and pepper) :: C2D E4400 @ 2ghz, 4g RAM (only 3.2g detected under 64bit...), Nvidia 9600GSO :: |
|
Back to top |
|
|
Lews_Therin l33t
Joined: 03 Oct 2003 Posts: 657 Location: Banned
|
Posted: Tue Feb 08, 2005 5:11 pm Post subject: |
|
|
Arkham wrote: | I think the "Gentoo way" is the distcc aproach... sharing binary packages with "popular" flags will led people to standarize theyr flags and you will be punished for selecting EXTREME tweaks |
The only way you could be "punished" for your CFLAGS is if they broke binary compatability with other packages. And that's bad, mmkay?
Arkham wrote: | My thought only..
Can someone post numbers here?? |
9 24 6 42
Arkham wrote: | Assuming there're enough IDLE machines, and those machines have the tarballs, how long would it take to compile KDE FULL (with split ebuilds of course ) |
It seems more efficient to simply maintain a package server with some simple opts (-O2 -mcpu=i686 -pipe -fomit-frame-pointer -funit-at-a-time, equivilent for other archs)
Arkham wrote: | 2 hours? 1? 30 min?? |
Take whatever the compilation speed for a package is on something like a P4 or Amd64, and use that. It's pointless to share compilations across the internet unless the "main" computer is absurdly slow.
Arkham wrote: | Anyway, I think the benefits for the comunity are huuuuuge. It may worth a try
? |
There has been no "distrubuted-distcc" thing tried that I'm aware of, but I believe there are one or two packages servers floating around somewhere. |
|
Back to top |
|
|
TazG Guru
Joined: 22 May 2004 Posts: 320 Location: Canada
|
Posted: Tue Feb 08, 2005 8:45 pm Post subject: |
|
|
Lews_Therin wrote: | Take whatever the compilation speed for a package is on something like a P4 or Amd64, and use that. It's pointless to share compilations across the internet unless the "main" computer is absurdly slow. | I don't understand that. Wouldn't it always be faster the more computers there are? |
|
Back to top |
|
|
LucaSpiller Apprentice
Joined: 10 Sep 2004 Posts: 188 Location: Censorship Land (aka England)
|
Posted: Tue Feb 08, 2005 9:50 pm Post subject: |
|
|
A dictcc thing over the internet would be extremely slow, let alone insecure. I guess what distcc does is send out different source files to be compiled to different computers which then send back the .o file. Then when compilation of all of them has been done the linker on the main computer puts them all together.
The reason why I say this would be extremely slow is because just having a quick look the .o files are around 10x bigger than the corresponding .c and .h files put together. Although it may not take as long to compile these files you still have to get them back to the main system to be linked. Here in Britiain the norm upload speed for broadband is 256k and download is 512k, so if you are compiling something such as OOo and lets say the source compiled into .o files is 200mb (just a guess) you would be waiting a hell of a long time for it to get to you. _________________ :: Luca :: Mac Fag :: Original Macbook, 2g RAM :: Closet Linux user (seasoned with salt and pepper) :: C2D E4400 @ 2ghz, 4g RAM (only 3.2g detected under 64bit...), Nvidia 9600GSO :: |
|
Back to top |
|
|
southsider Guru
Joined: 05 Jul 2004 Posts: 358
|
Posted: Tue Feb 08, 2005 10:10 pm Post subject: |
|
|
Also you could have people sending you infected .o's.
No thanks. |
|
Back to top |
|
|
Arkham Tux's lil' helper
Joined: 30 Aug 2004 Posts: 111 Location: Quilpue, Chile
|
Posted: Wed Feb 09, 2005 12:15 pm Post subject: |
|
|
Lews_Therin wrote: | Arkham wrote: | I think the "Gentoo way" is the distcc aproach... sharing binary packages with "popular" flags will led people to standarize theyr flags and you will be punished for selecting EXTREME tweaks |
The only way you could be "punished" for your CFLAGS is if they broke binary compatability with other packages. And that's bad, mmkay?
|
My point is, if you can benefit for using popular CFLAGS, then Gentoo will become a Binary Distro instead of a Source one..., everybody should be able to use "da system"
If you want to put "-O7 -frame-cigar-whatever", then so be it
?
P.D.: Nice numbers BTW |
|
Back to top |
|
|
|