| View previous topic :: View next topic |
| Author |
Message |
carmen
n00b
Joined: 27 May 2005
Posts: 9
|
 Posted: Tue Nov 01, 2005 1:08 am Post subject: |
 |
|
to have a 'decentralized' build environment - any client that builds something instantly making the binary and source pkg available to everyone else..you need to do this (i think):
- run http-replicator on each client, and set its proxy to itself (and/or run repcacheman every time) to ensure any downloaded packages end up in cache
- on each client machine, add all the other machines as source/binhosts to make.conf (and edit all machines whenever you add one)
- youre still wasting disk space by having the copies eventually exist on almost all clients, sometimes 2x in distfiles and cache dir
FEATURES=replication could solve all this (one centralized server, any clients upload binary-pkg via http. no more client-cache needed or server on every machine or crazily-bloating client lists pointing to eachother, and also eliminate the need for thinking about repcacheman). or is there already a way to do a distributed binhost type situation? I THINK this thing is great and portage should be aware of it in other words  |
|
| Back to top |
|
 |
carpman
Advocate
Joined: 20 Jun 2002
Posts: 2202
Location: London - UK
|
| Posted: Thu Nov 10, 2005 3:17 pm Post subject: |
|
|
Hello what script would recommend for cleaning out duplicate files in httprep dir?
have seen a few scripts that either are no longer maintained or read installed package files to decide what to delete, which of course is no good for httprep as it holds apps downloaded for many machines.
cheers
_________________
Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb |
|
| Back to top |
|
|
flybynite
l33t
Joined: 06 Dec 2002
Posts: 620
|
| Posted: Fri Nov 11, 2005 1:49 am Post subject: |
|
|
| carpman wrote: |
have seen a few scripts that either are no longer maintained or read installed package files to decide what to delete |
This is a tough problem for every gentoo user, not just http-replicator users.
Do you want to remove:
1. packages not in portage (might still be installed as a dependant)
2. packages not installed?
3. packages not accessed in a certain period of time?
4. superceeded packages ( see #1)
5 etc, etc.
My advice is pick one that you like based on your prefs and disk space.
Most scripts can be easily modified to clean the cache dir instead of the distfile dir.
If you don't want to modify the scripts, just move the cached files to the distfile dir (mv /var/cache/http-replicator/* /usr/portage/distfiles/ ) and then run your script and then repcacheman. repcacheman will check the files and then move them back to the cache. mv'ing files in the same partition are just a rename so it is fast.
Here are a couple places to look for scripts:
First the FAQ:
https://forums.gentoo.org/viewtopic-t-30547.html
Some Gentoo Experimental scripts:
http://gentooexperimental.org/script/repo/search?categories_mod%5B0%5D=is&category%5B0%5D=cleaners&srch=true
Tmpwatch is in portage and looks like it will work:
* app-admin/tmpwatch
Available versions: 2.9.2.2 2.9.4.1
Installed: none
Homepage: http://download.fedora.redhat.com/pub/fedora/linux/core/development/SRPMS/
Description: Utility recursively searches through specified directories and removes files which have not been accessed in a specified period of time.
Of course there are numerous scripts that can delete based on last access time, like this which delete files not accessed in 30 days:
find /var/cache/http-replicator -type f -atime +30 -exec rm -f "{}" \;
See man find if you want to delete files based on other criteria.
Hope this helps.... |
|
| Back to top |
|
|
carpman
Advocate
Joined: 20 Jun 2002
Posts: 2202
Location: London - UK
|
| Posted: Fri Nov 11, 2005 8:19 am Post subject: |
|
|
suppose what am looking for is superceeded files, the only problem here is if i am running ~z86 system on lan, which i am for testing GCC4 reiser4.
Will have a look links, cheers
_________________
Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb |
|
| Back to top |
|
|
dalek
Veteran
Joined: 19 Sep 2003
Posts: 1353
Location: Mississippi USA
|
| Posted: Fri Nov 11, 2005 9:05 am Post subject: |
|
|
Well, I cleaned mine out this way. I have http-repl* set up here. I made a back-up copy of /usr/portage/distfiles just to be safe. I'm on dial-up and wouldn't want to loose to much. I already had run repcacheman to get it up to date. Then I rm -rfv /usr/portage/distfiles to remove them all. I then did a emerge -efv world to reload all the ones it would need to rebuild all packages installed. It took it a bit but it did. There was about 1.5GBs that went missing.
After I knew I had all the files that I would ever need, I then deleted the ones I had copied for back-up. That should work. If the distfiles directory is empty, it will download only the ones it has to have then.
Don't forget to put the proxy line in your make.conf. Point it back at yourself and it works well.
I'm not going to tell you how many times I hit the tab key to type in those commands and locations above.   It's a habit because I type pretty bad.  
That make sense?
_________________
My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
60TBs of hard drive space using LVM
Cooler Master HAF-932 Case |
|
| Back to top |
|
|
carpman
Advocate
Joined: 20 Jun 2002
Posts: 2202
Location: London - UK
|
| Posted: Fri Nov 11, 2005 11:59 am Post subject: |
|
|
| dalek wrote: | Well, I cleaned mine out this way. I have http-repl* set up here. I made a back-up copy of /usr/portage/distfiles just to be safe. I'm on dial-up and wouldn't want to loose to much. I already had run repcacheman to get it up to date. Then I rm -rfv /usr/portage/distfiles to remove them all. I then did a emerge -efv world to reload all the ones it would need to rebuild all packages installed. It took it a bit but it did. There was about 1.5GBs that went missing.
After I knew I had all the files that I would ever need, I then deleted the ones I had copied for back-up. That should work. If the distfiles directory is empty, it will download only the ones it has to have then.
Don't forget to put the proxy line in your make.conf. Point it back at yourself and it works well.
I'm not going to tell you how many times I hit the tab key to type in those commands and locations above. It's a habit because I type pretty bad.
That make sense?
|
Take you need to do emerge -efv world on every machine on network?
This is the problem with many existing scripts, they assume that you only want to keep files in that machines world/pkg list, but as i have httprep on headless server all the gui apps would be deleted.
I could do via versions but again i if i am running different spec boxs, amd64, x86, ~x86, then i would need different versions of apps.
All a bit of a pain really, looks like i will best be served by date of file, so if not used or changed in 6 months delete, but this again would cause problems for apps not update often.
What would be nice is for script to read world file on all machines network and then clean out httprep dir.
_________________
Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb |
|
| Back to top |
|
|
dalek
Veteran
Joined: 19 Sep 2003
Posts: 1353
Location: Mississippi USA
|
| Posted: Fri Nov 11, 2005 3:18 pm Post subject: |
|
|
If you do emerge -efv world, from what I have read and what I have done, it downloads everything needed to reinstall everything on the system, not just the packages in your world file. That includes the dependancies of packages in your world file. Mine fetches 515 files for my install.
Maybe you have something unique but I am doing my second install doing this and I have yet to have to download anything since I am installing what I have currently on this system. I have 4 computers here, some run as servers some as desktops, emerge -efv woirld works just fine for me. After I cleaned everything out, it copied everything I needed back to /usr/portage/distfiles and is working fine.
Like I said, maybe you have something way unique, but it works fine for me. If you have something that unique, back-ups may be a good idea unless you can redownload pretty easy. If I had DSL or something I would have just deleted the files and let it redownload it all that way. It would take me a week on dial-up though.
It was a idea.
_________________
My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
60TBs of hard drive space using LVM
Cooler Master HAF-932 Case |
|
| Back to top |
|
|
todw1fd
Tux's lil' helper
Joined: 21 Mar 2003
Posts: 93
|
| Posted: Fri Nov 25, 2005 9:20 pm Post subject: replicator shows started but isn't |
|
|
Had this problem before some time ago. But just did a fresh install and now seeing same thing again. During startup, Http Replicator show as started with and OK, and there will be a pid listed in the pid file. Replicator log shows service as started. But clients cannot connect. When you run "/etc/init.d/http-replicator status" response shows service started. When you run "/etc/init.d/http-replicator stop" you're informed there is not an instance running:
| Quote: | gentoo-1 log # /etc/init.d/http-replicator status
* Caching service dependencies ... [ ok ] * status: started
gentoo-1 log # /etc/init.d/http-replicator stop
* Stopping Http-Replicator ...
No http-replicator found running; none killed. [ ok ]
gentoo-1 log # /etc/init.d/http-replicator start
* Starting Http-Replicator .. |
After either doing a start or a zap and start, the service then functions and shows as running with a ps. Clients can then connect. Tried starting from local.start also with same results. Any ideas? |
|
| Back to top |
|
|
todw1fd
Tux's lil' helper
Joined: 21 Mar 2003
Posts: 93
|
| Posted: Mon Nov 28, 2005 1:39 pm Post subject: |
|
|
| Seems to start and stop fine manually as long as I don't have it trying to start automatically first in the default runlevel or from local.start. Not quite understanding this behavior. |
|
| Back to top |
|
|
flybynite
l33t
Joined: 06 Dec 2002
Posts: 620
|
| Posted: Tue Nov 29, 2005 6:09 pm Post subject: |
|
|
| todw1fd wrote: | | Seems to start and stop fine manually as long as I don't have it trying to start automatically first in the default runlevel or from local.start |
Are you running ~x86?
The only time I've seen this is when replicator is unable to open the port. Maybe problem with startup scripts or config file?
You can simulate the auto startup by
| Code: |
source /etc/conf.d/http-replicator
/usr/bin/http-replicator
|
Which may give an error such as:
http-replicator: error: port 8080 is not available
If it is a config file problem you can also check what options are set by sourcing the config file and echo'ing the opts which should look similiar to this.:
| Code: |
source /etc/conf.d/http-replicator
echo $GENERAL_OPTS
|
which should give:
| Code: |
--dir /var/cache/http-replicator --user portage
|
Then try :
Which should give:
| Code: |
--dir /var/cache/http-replicator --user portage --alias /usr/portage/packages/All:All --alias /usr/src/:src --log /var/log/http-replicator.log --debug --ip 192.168.*.* --ip 10.*.*.* --port 8080
|
|
|
| Back to top |
|
|
todw1fd
Tux's lil' helper
Joined: 21 Mar 2003
Posts: 93
|
| Posted: Tue Nov 29, 2005 9:00 pm Post subject: |
|
|
| Thanks Flyby. I'll give it a shot when I get back to the system at home. |
|
| Back to top |
|
|
todw1fd
Tux's lil' helper
Joined: 21 Mar 2003
Posts: 93
|
| Posted: Thu Dec 01, 2005 12:36 am Post subject: |
|
|
| flybynite wrote: | | todw1fd wrote: | | Seems to start and stop fine manually as long as I don't have it trying to start automatically first in the default runlevel or from local.start |
Are you running ~x86?
No, running x86
The only time I've seen this is when replicator is unable to open the port. Maybe problem with startup scripts or config file?
You can simulate the auto startup by
| Code: |
source /etc/conf.d/http-replicator
/usr/bin/http-replic
|
Which may give an error such as:
http-replicator: error: port 8080 is not available
Returns "INFO: HttpReplicator started"
If it is a config file problem you can also check what options are set by sourcing the config file and echo'ing the opts which should look similiar to this.:
| Code: |
source /etc/conf.d/http-replicator
echo $GENERAL_OPTS
|
which should give:
| Code: |
--dir /var/cache/http-replicator --user portage[b]
|
Yup
Then try :
Which should give:
| Code: |
--dir /var/cache/http-replicator --user portage --alias /usr/portage/packages/All:All --alias /usr/src/:src --log /var/log/http-replicator.log --debug --ip 192.168.*.* --ip 10.*.*.* --port 8080
|
|
Yup, except missing your second alias /usr/src/:src
Doesn't want to complain after I'm up and running. Just doesn't want to cooperate using the startup scripts for some reason. |
|
| Back to top |
|
|
NightMonkey
Guru
Joined: 21 Mar 2003
Posts: 356
Location: Philadelphia, PA
|
| Posted: Thu Dec 01, 2005 2:31 am Post subject: repcacheman error |
|
|
Apologies if this has been dealt with earlier in this thread, but Search doesn't really work too well within long threads. Today, I ran repcacheman, and got this nice note:
| Code: | # repcacheman
Replicator's cache directory: /opt/http-replicator/
Portage's DISTDIR: /usr/portage/distfiles/
Comparing directories....
Done!
New files in DISTDIR:
jdk-1_5_0_05-linux-i586.bin
jce_policy-1_5_0.zip
Checking authenticity and integrity of new files...
Searching for ebuilds's ....
Done!
Found 21866 ebuilds.
Extracting the checksums....
Traceback (most recent call last):
File "/usr/bin/repcacheman.py", line 162, in ?
digestpath = os.path.dirname(digestpath)+"/files/digest-"+pv
File "/usr/lib/python2.4/posixpath.py", line 119, in dirname
return split(p)[0]
File "/usr/lib/python2.4/posixpath.py", line 77, in split
i = p.rfind('/') + 1
AttributeError: 'NoneType' object has no attribute 'rfind' |
Any pointers on where to find the source of this problem? Let me know if you need more information. http-replicatior has worked solidly (and silently) for over a year for me, so its record isn't bad! Thanks in advance.
_________________
|
|
| Back to top |
|
|
dahoste
Tux's lil' helper
Joined: 01 Dec 2005
Posts: 138
Location: Maryland, USA
|
| Posted: Thu Dec 01, 2005 6:15 am Post subject: |
|
|
Ditto NightMonkey's error. Saw it for the first time today (11/30) -- but only on one of my gentoo servers. The other one got through 'emerge --update --deep world' and 'repcacheman' just fine. Haven't chased the problem yet - hoping that someone familiar with http-replicator's guts has an easy solution. Also, as with most things gentoo, this might go away in 24 hours.
Note: the only files in my distfiles dir (when attempting to run 'repcacheman') are the following:
pciutils-2.2.0.tar.gz
nfs-utils-1.0.6.tar.gz
pci.ids-20051015.bz2 |
|
| Back to top |
|
|
dahoste
Tux's lil' helper
Joined: 01 Dec 2005
Posts: 138
Location: Maryland, USA
|
| Posted: Fri Dec 02, 2005 4:31 am Post subject: |
|
|
| Nevermind. Problem magically went away (as hoped). No problem with repcacheman today. Odd, but life's too short to worry about it. |
|
| Back to top |
|
|
carpman
Advocate
Joined: 20 Jun 2002
Posts: 2202
Location: London - UK
|
| Posted: Tue Dec 13, 2005 1:22 pm Post subject: |
|
|
Hello, ok i am having a persistant problem with emerging 3 apps this has been going on for about a week with an emerge sync once a day.
The problem is that the source files cannot be found on servers, apps concerned
| Code: |
media-libs/jbigkit-1.4
media-libs/jasper-1.701.0
media-libs/netpbm-10.30-r1
|
If i by pass my httprep server they download and emerge fine, any ideas
_________________
Work Station - 64bit
Gigabyte GA X48-DQ6 Core2duo E8400
8GB GSkill DDR2-1066
SATA Areca 1210 Raid
BFG OC2 8800 GTS 640mb
--------------------------------
Notebook
Samsung Q45 7100 4gb |
|
| Back to top |
|
|
flybynite
l33t
Joined: 06 Dec 2002
Posts: 620
|
| Posted: Thu Dec 15, 2005 3:56 am Post subject: Re: repcacheman error |
|
|
| NightMonkey wrote: | Apologies if this has been dealt with earlier in this thread, but Search doesn't really work too well within long threads. Today, I ran repcacheman, and got this nice note:
|
This usually happens because a gentoo developer doesn't input all the correct info into an ebuild. repcacheman doesn't deal well with this.
I've considered beefing up the error correction but put it off, because like you said, it goes away when the developer fixes their mistake and the problem is actually hard to fix.
Well, the error isn't hard to fix, but if you can't match all files to their md5sum in portage, how can you know what files are bad and what is caused by the portage error? It may be better just to wait for the portage update than to try and guess???
I have to do some more research on the different type of portage errors to know for sure..... |
|
| Back to top |
|
|
flybynite
l33t
Joined: 06 Dec 2002
Posts: 620
|
| Posted: Thu Dec 15, 2005 4:15 am Post subject: |
|
|
| carpman wrote: | Hello, ok i am having a persistant problem with emerging 3 apps this has been going on for about a week with an emerge sync once a day.
If i by pass my httprep server they download and emerge fine, any ideas |
Yes! This is most likely caused by a rare but possible remote server error. The file in replicator cache is corrupted. This shouldn't happen but it does in a few rare cirumstance that I have NEVER been able to reproduce!
repcacheman is the fix. Move all files from replicator cache into the distfile dir and then run repcacheman. All files are checksummed and then moved back to the cache. Any files left behind are the corrupted versions and will be deleted by portage or delete them yourself.
This post has a full explanation and the fix:
https://forums.gentoo.org/viewtopic-t-173226-postdays-0-postorder-asc-start-105.html |
|
| Back to top |
|
|
dalek
Veteran
Joined: 19 Sep 2003
Posts: 1353
Location: Mississippi USA
|
| Posted: Thu Dec 15, 2005 6:22 am Post subject: |
|
|
Is there any way to tell it not to delete my java tarball that I download from Sun? It would be nice if it would just move it to the cache and then copy it back if it needs it but it seems determined to delete the thing every time I run repcacheman. Then I get a error and can not use the && thing since it errors out and won't continue with the next command.
Bug, feature, me nuts?????
It only does it on this one file though.
Later
_________________
My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
60TBs of hard drive space using LVM
Cooler Master HAF-932 Case |
|
| Back to top |
|
|
flybynite
l33t
Joined: 06 Dec 2002
Posts: 620
|
| Posted: Thu Dec 15, 2005 8:14 am Post subject: |
|
|
| dalek wrote: | | Is there any way to tell it not to delete my java tarball that I download from Sun? |
I guess I'm not exactly sure what the problem is?
If I understand you than it's portage that's the problem, not replicator or repcacheman.
| Code: |
gate1 ~ # mv /home/tom/j2sdk-1_4_2_10-linux-i586.bin /usr/portage/distfiles/
gate1 ~ # repcacheman
Replicator's cache directory: /var/cache/http-replicator/
Portage's DISTDIR: /usr/portage/distfiles/
Comparing directories....
Done!
New files in DISTDIR:
j2sdk-1_4_2_10-linux-i586.bin
Checking authenticity and integrity of new files...
Searching for ebuilds's ....
Done!
Found 21857 ebuilds.
Extracting the checksums....
Done!
Verifying checksum's....
/usr/portage/distfiles/j2sdk-1_4_2_10-linux-i586.bin
MD5 OK
SUMMARY:
Found 0 duplicate file(s).
Deleted 0 dupe(s).
Found 1 new file(s).
Added 1 of those file(s) to the cache.
Rejected 0 corrupt or incomplete file(s).
Done!
|
replicator can verify the md5sum and add the file to the cache. That's it job. So far so good....
Now the problem is portage will NEVER, EVER do it's job and try to download the file again!!! The file is ready and waiting in the cache, but the call never comes.....
Look up RESTRICT="fetch" in the gentoo docs for more info on this problem.
I"ve asked gentoo dev's to change this or make an exception possible. It hasn't happened yet.... Please feel free to file a bug also, the more the merrier....
So gentoo says you gotta download the file yourself, repcacheman doesn't stop you. By moving the file into the cache, all boxes on your net can download the file from the cache.
You have to outsmart portage if you want to sidestep the gentoo restriction. You can copy the ebuild to the overlay dir and comment out the "fetch" restriction and all will be fine. If you find yourself having to reinstall the binary this is the best solution. |
|
| Back to top |
|
|
dalek
Veteran
Joined: 19 Sep 2003
Posts: 1353
Location: Mississippi USA
|
| Posted: Thu Dec 15, 2005 9:27 am Post subject: |
|
|
I see what you mean now. That is nasty of portage ain't it? I do keep a seperate copy laying around though. That way I know I don't have to spend 3 hours or so downloading it again.
In one sense, it is actually Suns fault. They want you to accept their stupid licence thing to get it. If the others would work correctly, I would use them instead of Suns java. I have tried them but they were causing me to lock up on some sites. So I am stuck with Suns silly restrictions. Since you got me straight on that, may as well move the blame on up the food chain. 
Thanks
_________________
My rig: Gigabyte GA-970A-UD3P mobo, AMD FX-8350 Eight-Core CPU, ZALMAN CNPS10X Performa CPU cooler,
G.SKILL 32GB DDR3 PC3 12800 Memory Nvidia GTX-650 video card LG W2253 Monitor
60TBs of hard drive space using LVM
Cooler Master HAF-932 Case |
|
| Back to top |
|
|
lannocc
n00b
Joined: 01 Dec 2004
Posts: 14
Location: Billings, MT
|
| Posted: Wed Dec 21, 2005 1:08 am Post subject: "Deleting invalid distfile" |
|
|
I occassionally run into the following problem when doing an emerge with my http_proxy pointing at a server running http-replicator. Portage tries to download the file a few times and then gives up, reporting the following after each attempt:
| Code: | | >>> Deleting invalid distfile. (Improper 404 redirect from server.) |
Previously my solution was to temporarily comment out the http_proxy variable in make.conf and run the emerge again on the client machine. Today however I was browsing through this thread and ran into the situation again so I decided to try something different.
On the server running http-replicator I moved the affected file (in this case linux-2.6.14.tar.bz2, from gentoo-sources) from /var/cache/http-replicator back into /usr/portage/distfiles. I then ran repcacheman and 'lo and behold it tells me the file is corrupt:
| Code: | Replicator's cache directory: /var/cache/http-replicator/
Portage's DISTDIR: /usr/portage/distfiles/
Comparing directories....
Done!
New files in DISTDIR:
linux-2.6.14.tar.bz2
Checking authenticity and integrity of new files...
Searching for ebuilds...
Done!
Found 22291 ebuilds.
Extracting the checksums....
Done!
Verifying checksum's....
/usr/portage/distfiles/linux-2.6.14.tar.bz2
CORRUPT or INCOMPLETE
SUMMARY:
Found 0 duplicate file(s).
Deleted 0 dupe(s).
Found 1 new file(s).
Added 0 of those file(s) to the cache.
Rejected 1 corrupt or incomplete file(s).
Done! |
That's good news, because at least now I know why my emerge was failing (corrupted file). So my first question is this: how did the corrupted file end up in http-replicator's cache in the first place?. From what I understand reading this thread, there are safeguards to prevent that, no?
Secondly (multi-part), is it http-replicator that is returning the 404 redirect? I know what a 404 is and what a redirect is, but what is a 404 redirect? Is there a different http error status that http-replicator could return in this instance that might make more sense? |
|
| Back to top |
|
|
flybynite
l33t
Joined: 06 Dec 2002
Posts: 620
|
| Posted: Fri Dec 23, 2005 9:05 am Post subject: Re: "Deleting invalid distfile" |
|
|
| lannocc wrote: | So my first question is this: how did the corrupted file end up in http-replicator's cache in the first place?
|
See the answer about half a page up... If anyone can reproduce this error, I'm still looking...
https://forums.gentoo.org/viewtopic-t-173226-start-392.html
| lannocc wrote: |
Secondly (multi-part), is it http-replicator that is returning the 404 redirect? |
No |
|
| Back to top |
|
|
lannocc
n00b
Joined: 01 Dec 2004
Posts: 14
Location: Billings, MT
|
| Posted: Fri Dec 23, 2005 10:14 pm Post subject: |
|
|
| Quote: | | If anyone can reproduce this error, I'm still looking... |
I wonder, how much work would it be to add the checksum code from repcacheman to http-replicator, so that when a file is requested it does an integrity check on it first? If integrity fails, it should remove the file and download it again as though it were never there. Would performing the integrity check significantly slowdown local (cached) downloads? |
|
| Back to top |
|
|
flybynite
l33t
Joined: 06 Dec 2002
Posts: 620
|
| Posted: Sat Dec 24, 2005 8:33 am Post subject: |
|
|
| lannocc wrote: |
I wonder, how much work would it be to add the checksum code from repcacheman to http-replicator, so that when a file is requested it does an integrity check on it first? |
The problem with this is that http-replicator would then be gentoo specific. replicator started out as and still is a debian (.deb) cache and will work on many platforms.
Keeping the gentoo specific code in repcacheman makes sense. To solve the problem you've had, I planned on adding code to repcacheman to check replicators cache, not just the distfile dir.
The same thing can be done just by running repcacheman like this:
| Code: |
mv /var/cache/http-replicator/* /usr/portage/distfiles/
repcacheman
|
This is simple, cheap if both dir's are on the same filesystem, and a whole lot more flexible.
For example, I and probably others use replicator to cache other files besides portage files. I can't have repcacheman just start deleting files because there not in portage.
But in typical linux fashion, there are more ways to do this, maybe just some more detail in the howto would work too..... |
|
| Back to top |
|
|
|
Documentation, Tips & Tricks
