View previous topic :: View next topic |
Author |
Message |
jagdpanther l33t
Joined: 22 Nov 2003 Posts: 764
|
Posted: Sun Nov 24, 2024 5:52 pm Post subject: mirrorselect not outputing servers |
|
|
I wanted to update the "GENTOO_MIRRORS=" line in my /etc/portage/make.conf because of slow down-load speeds.
Here is today's execution of the same command I ran in June 2023 to get my current GENTOO_MIRRORS servers.
Code: | $ mirrorselect -R "North America" -4 -D -s 5 -b 10 -S -o
* Using url: https://api.gentoo.org/mirrors/distfiles.xml
* Limiting test to "region=North America" hosts.
* Limiting test to https hosts.
* Downloading a list of mirrors...
Got 238 mirrors.
* Downloading 100k files from each mirror... [7 of 7]
No search results found. Check your filter settings and re-run mirrorselect |
So I tried some less restrictive options:
Code: | $ mirrorselect -R "North America" -4 -s 5 -b 5 -S -o
* Using url: https://api.gentoo.org/mirrors/distfiles.xml
* Limiting test to "region=North America" hosts.
* Limiting test to https hosts.
* Downloading a list of mirrors...
Got 238 mirrors.
Using netselect to choose the top5 hosts, in blocks of 5. 2 of 2 blocks complete.te.
ERROR: Netselect failed to return any mirrors. Try again using block mode.
$ mirrorselect -4 -s 5 -b 5 -S -o
* Using url: https://api.gentoo.org/mirrors/distfiles.xml
* Limiting test to https hosts.
* Downloading a list of mirrors...
Got 238 mirrors.
Using netselect to choose the top5 hosts, in blocks of 5. 15 of 15 blocks complete.te.
ERROR: Netselect failed to return any mirrors. Try again using block mode. |
I was using '-b' which is that block mode right?
Ideas? |
|
Back to top |
|
|
Banana Moderator
Joined: 21 May 2004 Posts: 1783 Location: Germany
|
|
Back to top |
|
|
Genone Retired Dev
Joined: 14 Mar 2003 Posts: 9615 Location: beyond the rim
|
Posted: Mon Nov 25, 2024 11:14 am Post subject: |
|
|
Increase the verbosity level (-d option) to see what is going on, in particular anything netselect might print to stderr. |
|
Back to top |
|
|
freke Veteran
Joined: 23 Jan 2003 Posts: 1033 Location: Somewhere in Denmark
|
Posted: Mon Nov 25, 2024 6:06 pm Post subject: |
|
|
-S is https-mirrors only Code: | ns ~ # mirrorselect
Usage: mirrorselect [options]
examples:
automatic:
# mirrorselect -s5
# mirrorselect -s3 -b10 -o >> /mnt/gentoo/etc/portage/make.conf
# mirrorselect -D -s4
interactive:
# mirrorselect -i -r
Options:
--version show program's version number and exit
-h, --help show this help message and exit
Main modes:
-a, --all_mirrors This will present a list of all filtered search
results to make it possible to select mirrors you wish
to use. For the -r, --rsync option, it will select
the rotation server only. As multiple rsync URL's are
not supported.
-D, --deep Deep mode. This is used to give a more accurate speed
test. It will download a 100k file from each server.
Because of this you should only use this option if you
have a good connection.
-i, --interactive Interactive Mode, this will present a list to make it
possible to select mirrors you wish to use.
Server type selection (choose at most one):
-c COUNTRY, -country COUNTRY
only use mirrors from the specified country NOTE:
Names with a space must be quoted eg.: -c 'South
Korea'
-F, --ftp ftp only mode. Will not consider hosts of other types.
-H, --http http only mode. Will not consider hosts of other types
-S, --https https only mode. Will not consider hosts of other
types
-r, --rsync rsync mode. Allows you to interactively select your
rsync mirror. Requires -i or -a to be used.
-R REGION, -region REGION
only use mirrors from the specified region NOTE: Names
with a space must be quoted eg.: -R 'North America'
-4, --ipv4 only use IPv4
-6, --ipv6 only use IPv6
Other options:
-b BLOCKSIZE, -blocksize BLOCKSIZE
This is to be used in automatic mode and will split
the hosts into blocks of BLOCKSIZE for use with
netselect. This is required for certain routers which
block 40+ requests at any given time. Recommended
parameters to pass are: -s 3 -b 10
-d VERBOSITY, -debug VERBOSITY
debug mode, pass in the debug level [1-9]
-f FILE, -file FILE
An alternate file to download for deep testing. Please
choose the file carefully as to not abuse the system
by selecting an overly large size file. You must also
use the -m, --md5 option.
-m MD5, --md5=MD5 An alternate file md5sum value used to compare the
downloaded file against for deep testing.
-o, --output Output Only Mode, this is especially useful when being
used during installation, to redirect output to a file
other than /etc/portage/make.conf
-P PROXY, -proxy PROXY
Proxy server to use if not the default proxy in the
environment
-q, --quiet Quiet mode
-s SERVERS, -servers SERVERS
Specify Number of servers for Automatic Mode to
select. this is only valid for download mirrors. If
this is not specified, a default of 1 is used.
-t TIMEOUT, -timeout TIMEOUT
Timeout for deep mode. Defaults to 10 seconds.
-e EXCLUDE, -exclude EXCLUDE
Exclude host from mirrors list. | For me adding -D seems to result in Code: | No search results found. Check your filter settings and re-run mirrorselect | Haven't done any real testing nor debugging (-d) to follow up on it though. Code: | ns ~ # mirrorselect -R 'Europe' -D -s 5 -b 5 -S -o
* Using url: https://api.gentoo.org/mirrors/distfiles.xml
* Limiting test to "region=Europe" hosts.
* Limiting test to https hosts.
* Downloading a list of mirrors...
Got 238 mirrors.
* Downloading 100k files from each mirror... [41 of 41]
No search results found. Check your filter settings and re-run mirrorselect
ns ~ # mirrorselect -R 'Europe' -s 5 -b 5 -S -o
* Using url: https://api.gentoo.org/mirrors/distfiles.xml
* Limiting test to "region=Europe" hosts.
* Limiting test to https hosts.
* Downloading a list of mirrors...
Got 238 mirrors.
Using netselect to choose the top5 hosts, in blocks of 5. 9 of 9 blocks complete.te.
GENTOO_MIRRORS="https://135.181.111.198/gentoo/gentoo-distfiles/ \
https://130.236.254.253/gentoo/ \
https://mirrors.ircam.fr/pub/gentoo-distfiles/ \
https://129.102.1.37/pub/gentoo-distfiles/ \
https://130.236.254.251/gentoo/" | Also it ends quite a bit quicker with the -D option, so I wonder if it really works... Code: | ns ~ # time mirrorselect -D -6 -R 'Europe' -s5 -b5 -S -o
* Using url: https://api.gentoo.org/mirrors/distfiles.xml
* Limiting test to "region=Europe" hosts.
* Limiting test to https hosts.
* Downloading a list of mirrors...
Got 238 mirrors.
* Downloading 100k files from each mirror... [41 of 41]
No search results found. Check your filter settings and re-run mirrorselect
real 0m19.697s
user 0m1.122s
sys 0m0.175s
ns ~ # time mirrorselect -6 -R 'Europe' -s5 -b5 -S -o
* Using url: https://api.gentoo.org/mirrors/distfiles.xml
* Limiting test to "region=Europe" hosts.
* Limiting test to https hosts.
* Downloading a list of mirrors...
Got 238 mirrors.
Using netselect to choose the top5 hosts, in blocks of 5. 9 of 9 blocks complete.te.
GENTOO_MIRRORS="https://mirrors.ircam.fr/pub/gentoo-distfiles/ \
https://mirror.netcologne.de/gentoo/ \
https://ftp-stud.hs-esslingen.de/pub/Mirrors/gentoo/ \
https://[2a0d:f302:136:1d62::1]/gentoo/ \
https://mirror.kumi.systems/gentoo/"
real 2m40.018s
user 0m2.427s
sys 0m3.377s | Actually adding -d2 shows that the test-file it's supposed to fetch seems to be non-existant on the mirrors? Code: | ns ~ # time mirrorselect -d2 -D -R 'Europe' -s5 -b5 -S -o
main(); config_path = /etc/portage/make.conf
get_filesystem_mirrors(): config_path = /etc/portage/make.conf
get_filesystem_mirrors(): mirrorlist = ['https://mirrors.ircam.fr/pub/gentoo-distfiles/', '\\', 'https://ftp.fau.de/gentoo', '\\', 'https://mirror.netcologne.de/gentoo/', '\\', 'https://[2a0d:f302:111:83ae::1]/gentoo/', '\\', 'https://[2a0d:f302:136:1d62::1]/gentoo/']
get_filesystem_mirrors(): ignoring non-accessible mirror = \
get_filesystem_mirrors(): ignoring non-accessible mirror = \
get_filesystem_mirrors(): ignoring non-accessible mirror = \
get_filesystem_mirrors(): ignoring non-accessible mirror = \
get_filesystem_mirrors(): fsmirrors = []
using url: https://api.gentoo.org/mirrors/distfiles.xml
* Using url: https://api.gentoo.org/mirrors/distfiles.xml
* Limiting test to "region=Europe" hosts.
* Limiting test to https hosts.
getlist(): fetching https://api.gentoo.org/mirrors/distfiles.xml
* Downloading a list of mirrors...
Got 238 mirrors.
Extractor(): fetched mirrors, 41 hosts after filtering
* Downloading 100k files from each mirror... [1 of 41]
_deeptime(): maxtime is 10
deeptime(): ip's for host gentoo.mirrors.ovh.net: ['213.32.5.7']
deeptime(): testing url: https://213.32.5.7/gentoo-distfiles/distfiles/mirrorselect-test
deeptime(): connection to host gentoo.mirrors.ovh.net
returned HTTPError: HTTP Error 404: Not Found for ip 213.32.5.7
Switching back to original url
deeptime(): connection to host gentoo.mirrors.ovh.net
returned HTTPError: HTTP Error 404: Not Found for ip 213.32.5.7
Switching back to original url
deeptime(): unable to connect to host gentoo.mirrors.ovh.net
* Downloading 100k files from each mirror... [2 of 41]
_deeptime(): maxtime is 10
deeptime(): ip's for host mirror.leaseweb.com: ['37.58.58.140']
deeptime(): testing url: https://37.58.58.140/gentoo/distfiles/mirrorselect-test
deeptime(): connection to host mirror.leaseweb.com
returned HTTPError: HTTP Error 404: Not Found for ip 37.58.58.140
Switching back to original url
deeptime(): connection to host mirror.leaseweb.com
returned HTTPError: HTTP Error 404: Not Found for ip 37.58.58.140
Switching back to original url
deeptime(): unable to connect to host mirror.leaseweb.com
* Downloading 100k files from each mirror... [3 of 41] | Which is why it doesn't return any mirrors when using -D I assume.
EDIT:
Found this when I was about to bug.... https://bugs.gentoo.org/921759 |
|
Back to top |
|
|
jagdpanther l33t
Joined: 22 Nov 2003 Posts: 764
|
Posted: Tue Nov 26, 2024 12:23 am Post subject: |
|
|
Thanks for the replies.
Banana:
I may try to use https://mirrorstats.gentoo.org/rsync/ to pick sites by hand.
Here are my runs from your two suggestions. I added a '-o' because I didn't want to update /etc/portage/make.conf without seeing the update first.
Code: | $ mirrorselect -s3 -b10 -D -R "North America" -o
* Using url: https://api.gentoo.org/mirrors/distfiles.xml
* Limiting test to "region=North America" hosts.
* Downloading a list of mirrors...
Got 238 mirrors.
* Downloading 100k files from each mirror... [26 of 26]
No search results found. Check your filter settings and re-run mirrorselect |
Code: | $ mirrorselect -s3 -b10 -D -c USA -o
* Using url: https://api.gentoo.org/mirrors/distfiles.xml
* Limiting test to "country=USA" hosts.
* Downloading a list of mirrors...
Got 238 mirrors.
* Downloading 100k files from each mirror... [17 of 17]
No search results found. Check your filter settings and re-run mirrorselect |
Genone:
I tried my original mirrorselect command adding '-d 2'. It appears that something called deeptime() is having issues:
Code: | $ mirrorselect -R "North America" -4 -D -s 5 -b 10 -S -d 2 -o
main(); config_path = /etc/portage/make.conf
get_filesystem_mirrors(): config_path = /etc/portage/make.conf
get_filesystem_mirrors(): mirrorlist = ['https://mirrors.rit.edu/gentoo/', '\\', 'https://mirror.csclub.uwaterloo.ca/gentoo-distfiles/', '\\', 'https://mirror.servaxnet.com/gentoo/', '\\', 'https://mirror.clarkson.edu/gentoo/', '\\', 'https://gentoo.osuosl.org/']
get_filesystem_mirrors(): ignoring non-accessible mirror = \
get_filesystem_mirrors(): ignoring non-accessible mirror = \
get_filesystem_mirrors(): ignoring non-accessible mirror = \
get_filesystem_mirrors(): ignoring non-accessible mirror = \
get_filesystem_mirrors(): fsmirrors = []
using url: https://api.gentoo.org/mirrors/distfiles.xml
* Using url: https://api.gentoo.org/mirrors/distfiles.xml
* Limiting test to "region=North America" hosts.
* Limiting test to https hosts.
getlist(): fetching https://api.gentoo.org/mirrors/distfiles.xml
* Downloading a list of mirrors...
Got 238 mirrors.
Extractor(): fetched mirrors, 7 hosts after filtering
* Downloading 100k files from each mirror... [1 of 7]
_deeptime(): maxtime is 10
deeptime(): ip's for host mirror.csclub.uwaterloo.ca: ['129.97.134.71']
deeptime(): testing url: https://129.97.134.71/gentoo-distfiles/distfiles/mirrorselect-test
deeptime(): connection to host mirror.csclub.uwaterloo.ca
returned HTTPError: HTTP Error 404: Not Found for ip 129.97.134.71
Switching back to original url
deeptime(): connection to host mirror.csclub.uwaterloo.ca
returned HTTPError: HTTP Error 404: Not Found for ip 129.97.134.71
Switching back to original url
deeptime(): unable to connect to host mirror.csclub.uwaterloo.ca
* Downloading 100k files from each mirror... [2 of 7]
_deeptime(): maxtime is 10
deeptime(): ip's for host mirror.reenigne.net: ['172.83.105.10']
deeptime(): testing url: https://172.83.105.10/gentoo/distfiles/mirrorselect-test
deeptime(): connection to host mirror.reenigne.net
returned HTTPError: HTTP Error 404: Not Found for ip 172.83.105.10
Switching back to original url
deeptime(): connection to host mirror.reenigne.net
returned HTTPError: HTTP Error 404: Not Found for ip 172.83.105.10
Switching back to original url
deeptime(): unable to connect to host mirror.reenigne.net
* Downloading 100k files from each mirror... [3 of 7]
_deeptime(): maxtime is 10
deeptime(): ip's for host gentoo.osuosl.org: ['64.50.233.100', '64.50.236.52', '140.211.166.134']
deeptime(): testing url: https://64.50.233.100/distfiles/mirrorselect-test
deeptime(): connection to host gentoo.osuosl.org
returned HTTPError: HTTP Error 404: Not Found for ip 64.50.233.100
Switching back to original url
deeptime(): testing url: https://64.50.236.52/distfiles/mirrorselect-test
deeptime(): connection to host gentoo.osuosl.org
returned HTTPError: HTTP Error 404: Not Found for ip 64.50.236.52
Switching back to original url
deeptime(): testing url: https://140.211.166.134/distfiles/mirrorselect-test
deeptime(): connection to host gentoo.osuosl.org
returned HTTPError: HTTP Error 404: Not Found for ip 140.211.166.134
Switching back to original url
deeptime(): connection to host gentoo.osuosl.org
returned HTTPError: HTTP Error 404: Not Found for ip 140.211.166.134
Switching back to original url
deeptime(): unable to connect to host gentoo.osuosl.org
* Downloading 100k files from each mirror... [4 of 7]
_deeptime(): maxtime is 10
deeptime(): ip's for host mirrors.mit.edu: ['18.7.29.125']
deeptime(): testing url: https://18.7.29.125/gentoo-distfiles/distfiles/mirrorselect-test
deeptime(): connection to host mirrors.mit.edu
returned HTTPError: HTTP Error 404: Not Found for ip 18.7.29.125
Switching back to original url
deeptime(): connection to host mirrors.mit.edu
returned HTTPError: HTTP Error 404: Not Found for ip 18.7.29.125
Switching back to original url
deeptime(): unable to connect to host mirrors.mit.edu
* Downloading 100k files from each mirror... [5 of 7]
_deeptime(): maxtime is 10
deeptime(): ip's for host mirrors.rit.edu: ['129.21.171.72']
deeptime(): testing url: https://129.21.171.72/gentoo/distfiles/mirrorselect-test
deeptime(): connection to host mirrors.rit.edu
returned HTTPError: HTTP Error 404: Not Found for ip 129.21.171.72
Switching back to original url
deeptime(): connection to host mirrors.rit.edu
returned HTTPError: HTTP Error 404: Not Found for ip 129.21.171.72
Switching back to original url
deeptime(): unable to connect to host mirrors.rit.edu
* Downloading 100k files from each mirror... [6 of 7]
_deeptime(): maxtime is 10
deeptime(): ip's for host mirror.clarkson.edu: ['128.153.145.19']
deeptime(): testing url: https://128.153.145.19/gentoo/distfiles/mirrorselect-test
deeptime(): connection to host mirror.clarkson.edu
returned HTTPError: HTTP Error 404: Not Found for ip 128.153.145.19
Switching back to original url
deeptime(): connection to host mirror.clarkson.edu
returned HTTPError: HTTP Error 404: Not Found for ip 128.153.145.19
Switching back to original url
deeptime(): unable to connect to host mirror.clarkson.edu
* Downloading 100k files from each mirror... [7 of 7]
_deeptime(): maxtime is 10
deeptime(): ip's for host mirror.servaxnet.com: ['162.255.226.75']
deeptime(): testing url: https://162.255.226.75/gentoo/distfiles/mirrorselect-test
deeptime(): connection to host mirror.servaxnet.com
returned HTTPError: HTTP Error 404: Not Found for ip 162.255.226.75
Switching back to original url
deeptime(): connection to host mirror.servaxnet.com
returned HTTPError: HTTP Error 404: Not Found for ip 162.255.226.75
Switching back to original url
deeptime(): unable to connect to host mirror.servaxnet.com
deeptest(): got 7 hosts, and returned []
deeptest(): final rethost []
deeptest(): final md5 failures 0 of 7
No search results found. Check your filter settings and re-run mirrorselect |
Using the same -R 'Europe' as you I still get no results after a min. or two.
Code: | $ mirrorselect -R 'Europe' -D -s 5 -b 5 -S -o
* Using url: https://api.gentoo.org/mirrors/distfiles.xml
* Limiting test to "region=Europe" hosts.
* Limiting test to https hosts.
* Downloading a list of mirrors...
Got 238 mirrors.
* Downloading 100k files from each mirror... [41 of 41]
No search results found. Check your filter settings and re-run mirrorselect |
Without the '-D' I receive an instant error
Code: | $ mirrorselect -R 'Europe' -s 5 -b 5 -S -o
* Using url: https://api.gentoo.org/mirrors/distfiles.xml
* Limiting test to "region=Europe" hosts.
* Limiting test to https hosts.
* Downloading a list of mirrors...
Got 238 mirrors.
Using netselect to choose the top5 hosts, in blocks of 5. 9 of 9 blocks complete.te.
ERROR: Netselect failed to return any mirrors. Try again using block mode. |
|
|
Back to top |
|
|
Genone Retired Dev
Joined: 14 Mar 2003 Posts: 9615 Location: beyond the rim
|
Posted: Tue Nov 26, 2024 10:55 am Post subject: |
|
|
deeptime() is just the name of the function within mirrorselect that does most of the testing.
As you've already figured out, the core issue is apparently that mirrorselect hasn't been updated to match the current mirror directory structure. Might be worth to ask on the bug report if there are any status updates. |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|