Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Portage rsync tree verification
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4  
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
Hu
Administrator
Administrator


Joined: 06 Mar 2007
Posts: 22938

PostPosted: Fri Jul 27, 2018 11:59 pm    Post subject: Reply with quote

Verification is relatively RAM-inexpensive, and if it did fail due to low memory, it would not fail in the manner you reported. As mv noted, you have a one bit flip. ` is 0x60; a is 0x61. That is very typical of bad RAM.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6780

PostPosted: Sat Jul 28, 2018 6:23 am    Post subject: Reply with quote

kajzer wrote:
I'm sure memory is fine

How can you be sure? Did you run memtest recently? It might be that one of your RAM banks has become unstable.
Since the "expected value" differs only in one bit, making it even an invalid hex string, I consider network and software errors unlikely (I assume that this string is calculated from gentoo-release.asc and wrong data there would result in a completely different but valid hex string).
Back to top
View user's profile Send private message
The Main Man
Veteran
Veteran


Joined: 27 Nov 2014
Posts: 1172
Location: /run/user/1000

PostPosted: Sat Jul 28, 2018 6:44 am    Post subject: Reply with quote

Well, I guess if memory is bad there would be more problems and I don't see any.
I'll run memtest later to be sure.

btw I reported what happened because several days ago I asked what happens if verification fails, otherwise I don't think I would.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6780

PostPosted: Sat Jul 28, 2018 6:58 am    Post subject: Reply with quote

I don't use rysnc for syncing. I just realized that it might be that the hex string is directly contained in one of the transferred files. You should check this (e.g. with grep -R ...)
Back to top
View user's profile Send private message
Goverp
Advocate
Advocate


Joined: 07 Mar 2007
Posts: 2191

PostPosted: Sat Jul 28, 2018 8:09 am    Post subject: Reply with quote

kajzer wrote:
Well, I guess if memory is bad there would be more problems and I don't see any.
I'll run memtest later to be sure. ...

FWIW, my box started throwing random errors last weekend. After a few false starts, I ran memtest, and found there was a bad chip on one of the memory cards. We've had a record-breaking high temperature spell here for the last month or so; I wonder if that's what triggered the chip to go bad.
_________________
Greybeard
Back to top
View user's profile Send private message
The Main Man
Veteran
Veteran


Joined: 27 Nov 2014
Posts: 1172
Location: /run/user/1000

PostPosted: Sat Jul 28, 2018 8:53 am    Post subject: Reply with quote

mv wrote:
I don't use rysnc for syncing. I just realized that it might be that the hex string is directly contained in one of the transferred files. You should check this (e.g. with grep -R ...)


I just realized something, this was in the error :

Quote:
Manifest mismatch for app-portage/elt-patches/elt-patches-20170422.ebuild
SHA512: expected: 8c1f168c3fc9088d6d3e24be3584c44068562d59ac7e9240ba0dcabc616355efcaa22bde1bc5ba689a12ef22b669731`78236c02b4bb4adc660d31979ba6e06d, have: 8c1f168c3fc9088d6d3e24be3584c44068562d59ac7e9240ba0dcabc616355efcaa22bde1bc5ba689a12ef22b669731a78236c02b4bb4adc660d31979ba6e06d


If I'm not mistaken expected SHA value was wrong, indicating that there's error somewhere else.
I just ran
Code:
sha512sum /usr/portage/app-portage/elt-patches/elt-patches-20170422.ebuild

and got this :
Code:
8c1f168c3fc9088d6d3e24be3584c44068562d59ac7e9240ba0dcabc616355efcaa22bde1bc5ba689a12ef22b669731a78236c02b4bb4adc660d31979ba6e06d  /usr/portage/app-portage/elt-patches/elt-patches-20170422.ebuild


Anyway, mismatch is because expected SHA512 was wrong, for some reason, and I don't expect that value to be related to user's machine.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6780

PostPosted: Sat Jul 28, 2018 9:56 am    Post subject: Reply with quote

kajzer wrote:
Anyway, mismatch is because expected SHA512 was wrong

This was exactly my argument.
Quote:
and I don't expect that value to be related to user's machine.

I do, because I assume that it is calculated from some other data.
As mentioned, you should grep the portage directory for such checksums to verify whether my assumption is true.
But even then it would be strange, because the checksum file should be signed, so a corruption in the checksum file itself should have triggered an earlier failure of the signature check.
Chances are really high that the bit toggled within you machine's RAM. Whether this was a one-shot event or happens more regularly can be verified only with a RAM-test.
Back to top
View user's profile Send private message
The Main Man
Veteran
Veteran


Joined: 27 Nov 2014
Posts: 1172
Location: /run/user/1000

PostPosted: Sat Jul 28, 2018 10:59 am    Post subject: Reply with quote

Just finished memtest86+ with no errors.

As for portage directory grep, I did another sync after the failure so whatever was corrupted in the first sync was fixed, as I understand this process, could be wrong though.
But here it is :
Code:
$ grep -R '8c1f168c3fc9088d6d3e24be3584c440' /usr/portage/                                                                                                                                                                   
/usr/portage/app-portage/elt-patches/Manifest:EBUILD elt-patches-20170422.ebuild 825 BLAKE2B d652e8bea87970f26c4db1e26d525c4cd9700f6069e5e48f89ef2478b005cb0e3a3142a4ce0799b19a202ab4e18b5453b7d61d731095ca89f04f971b16801db7 SHA512 8c1f168c3fc9088d6d3e24be3584c44068562d59ac7e9240ba0dcabc616355efcaa22bde1bc5ba689a12ef22b669731a78236c02b4bb4adc660d31979ba6e06d


Maybe one of the rsync servers I got on that error sync was in some problem, next time I got some other server and it went fine.
I don't have mirror uri specified.
if everything is logged somewhere I can help further with providing more information.

To conclude, I don't know what happened but it did, if mistake on the server end is completely out of the question and impossible then it's something on my end, for some unknown reason.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6780

PostPosted: Sat Jul 28, 2018 12:29 pm    Post subject: Reply with quote

kajzer wrote:
Maybe one of the rsync servers I got on that error sync was in some problem

But the manifest file itself should have been checked by some checksum+signature (probably from metamanifest). The strange thing is that apparently that check went fine and after that you got the broken data.
Of course, it might be that the file was broken (and the wrong file signed) on all gentoo servers. But then other people would have experienced the same problem. Moreover, the "damaged" file was not changed in git since the last 8 months, so it would really have to be gentoo's main rsync server which has the hardware problem if it was not local on your system.
Back to top
View user's profile Send private message
The Main Man
Veteran
Veteran


Joined: 27 Nov 2014
Posts: 1172
Location: /run/user/1000

PostPosted: Sat Jul 28, 2018 1:48 pm    Post subject: Reply with quote

mv wrote:
Moreover, the "damaged" file was not changed in git since the last 8 months, so it would really have to be gentoo's main rsync server which has the hardware problem if it was not local on your system.


Yeah, strange. Well I don't know.
I know I would notice something weird going on with this machine, it never goes offline though and reboots only when updates are done, uptime prior to this memtest was 10 days.

Next time if it happens hopefully I'll be able to debug it more, especially if /usr/portage/.tmp-unverified-download-quarantine file is still alive after the process is finished, but I doubt it will be.
I never used rsync before, instead I was using git, but was curious and wanted to try this new feature.
I picked one server now for syncing and specified it in the config file.
Back to top
View user's profile Send private message
Trovalds
n00b
n00b


Joined: 16 Jun 2011
Posts: 13
Location: Cuiaba/MT - Brazil

PostPosted: Fri Apr 05, 2019 8:27 pm    Post subject: Reply with quote

Sorry, old topic, etc.

I got this error on installation of my gentoo box.

Code:

INFO:root:Refreshing keys from keyserver...
ERROR:root:OpenPGP keyring refresh failed:
Unable to refresh keys: gpg: refreshing 4 keys from hkps://hkps.pool.sks-keyservers.net
gpg: keyserver refresh failed: Address family not supported by protocol


Researching the internet, I arrive here in this topic. The only way I got things to work is setting verify signature to false.

BUT I figured out this: instead of hkps://hkps.pool.sks-keyservers.net the right patch is http://hkps.pool.sks-keyservers.net ?

No clue how do I change this on Gentoo, anyways.

PS: sorry my English, not my native language.
Back to top
View user's profile Send private message
dylanmc
Apprentice
Apprentice


Joined: 18 Apr 2014
Posts: 157
Location: Modena, Italy

PostPosted: Mon Aug 12, 2019 6:35 pm    Post subject: Reply with quote

Code:
snoopx /home/luca # eix-sync
 * Running emerge --sync
OpenPGP keyring refresh failed:
gpg: refreshing 4 keys from hkps://keys.gentoo.org
gpg: keyserver refresh failed: No keyserver available



I have this problem, I can't resync my portage tree (and I can't update)

Any solution?
Back to top
View user's profile Send private message
mike155
Advocate
Advocate


Joined: 17 Sep 2010
Posts: 4438
Location: Frankfurt, Germany

PostPosted: Mon Aug 12, 2019 7:31 pm    Post subject: Reply with quote

Developers recently added a feature called 'tree verification'. It doesn't work well and many users report problems.

My suggestion: disable that feature: https://forums.gentoo.org/viewtopic-p-8358476.html#8358476

Other users propose other solutions, see: https://forums.gentoo.org/viewtopic-t-1100202-start-0.html
Back to top
View user's profile Send private message
The Main Man
Veteran
Veteran


Joined: 27 Nov 2014
Posts: 1172
Location: /run/user/1000

PostPosted: Mon Aug 12, 2019 8:01 pm    Post subject: Reply with quote

What worked for me is this line in gentoo.conf
Code:
sync-uri = rsync://rsync.europe.gentoo.org/gentoo-portage


that way when error happens you just sync it again and most likely it will work, some other mirror will be used, if there's only one mirror then syncing it again wouldn't help, at least I think that's the case, I remember someone told me several months ago that my memory is bad and that I should check it, and that's why I was actually getting that error, fun times :)
Back to top
View user's profile Send private message
dylanmc
Apprentice
Apprentice


Joined: 18 Apr 2014
Posts: 157
Location: Modena, Italy

PostPosted: Mon Aug 12, 2019 8:40 pm    Post subject: Reply with quote

mike155 wrote:
Developers recently added a feature called 'tree verification'. It doesn't work well and many users report problems.

My suggestion: disable that feature: https://forums.gentoo.org/viewtopic-p-8358476.html#8358476

Other users propose other solutions, see: https://forums.gentoo.org/viewtopic-t-1100202-start-0.html



Thanks, your suggestion did it, the sync now works
I hope developers found a fix, system update is important :)
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Goto page Previous  1, 2, 3, 4
Page 4 of 4

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum