| View previous topic :: View next topic |
| Author |
Message |
nagmat84
Apprentice
Joined: 27 Mar 2007
Posts: 260
|
 Posted: Sat Jun 15, 2024 9:07 am Post subject: [Solved] Sync problems with git-based, cached ebuild repo |
 |
|
"emerge --sync" fails to sync the overlay repository "booboo" and reports "fatal: detected dubious ownership in repository at '/var/db/repos/booboo/.git'". How do I fix that problem? The ownership seems to be correct.
More background:
I am running a build server and binary host for three clients. The build server also serves as a cache/proxy for all ebuild repositories. First the build server syncs the ebuild repositories with upstream, then the clients sync with the build server. The clients report the error message when pulling from the build server; when the build server pulls the git repo from upstream everything is fine.
Build server
FEATURES="userfetch userpriv usersandbox usersync"
server ~ # cat /etc/portage/repos.conf/booboo.conf
[booboo]
location = /var/db/repos/booboo
sync-type = git
sync-uri = https://github.com/gentoo-mirror/booboo.git
server ~ # ls -lhd /var/db/repos/booboo{/.git,}
drwxr-xr-x 62 portage portage 4.0K Apr 4 17:49 /var/db/repos/booboo
drwxr-xr-x 7 portage portage 197 Jun 15 08:53 /var/db/repos/booboo/.git
Client(s)
FEATURES="userfetch userpriv usersandbox usersync"
client-1 ~ # cat /etc/portage/repos.conf/booboo.conf
[booboo]
location = /var/db/repos/booboo
sync-type = git
sync-uri = ssh://gentoo@server.my-domain.tld/var/db/repos/booboo/
client-1 ~ # ls -lhd /var/db/repos/booboo{/.git,}
drwxr-xr-x 62 portage portage 4.0K Apr 4 20:18 /var/db/repos/booboo
drwxr-xr-x 7 portage portage 4.0K May 13 17:31 /var/db/repos/booboo/.git
Last edited by nagmat84 on Sat Jul 13, 2024 1:00 pm; edited 1 time in total |
|
| Back to top |
|
 |
eeckwrk99
Apprentice
Joined: 14 Mar 2021
Posts: 231
Location: Gentoo forums
|
| Posted: Sat Jun 15, 2024 9:14 am Post subject: |
|
|
All directories in /var/db/repos have root:root ownership as far as I'm concerned, not portage:portage.
What about other directories ownership in /var/db/repos?
Maybe try removing the repository and add it again with
| Code: | # eselect repository remove booboo
# eselect repository add booboo |
|
|
| Back to top |
|
|
nagmat84
Apprentice
Joined: 27 Mar 2007
Posts: 260
|
| Posted: Sat Jun 15, 2024 9:20 am Post subject: |
|
|
| Quote: | | What about other directories ownership in /var/db/repos? |
All other directory are also owned by portage. I believe that this has something to do with the features "userfetch userpriv usersandbox usersync". As I understand, emerge drops privileges to the user portage during sync if "usersync" is enabled.
| Quote: | | aybe try removing the repository and add it again |
I have already tried that.
Also, I have already tried to set different combinations of the feature "usersync" and directory ownership both on the build server and client, i.e. "usersync" enabled files owned by root, "usersync" enabled files owned by portage, "usersync disabled files owned by root, and so on. No luck so far with any combination. |
|
| Back to top |
|
|
eeckwrk99
Apprentice
Joined: 14 Mar 2021
Posts: 231
Location: Gentoo forums
|
| Posted: Sat Jun 15, 2024 9:34 am Post subject: |
|
|
| nagmat84 wrote: | | Quote: | | What about other directories ownership in /var/db/repos? |
All other directory are also owned by portage. I believe that this has something to do with the features "userfetch userpriv usersandbox usersync". As I understand, emerge drops privileges to the user portage during sync if "usersync" is enabled. |
Indeed, I didn't pay attention to these options.
| nagmat84 wrote: | | Also, I have already tried to set different combinations of the feature "usersync" and directory ownership both on the build server and client, i.e. "usersync" enabled files owned by root, "usersync" enabled files owned by portage, "usersync disabled files owned by root, and so on. No luck so far with any combination. |
Maybe a stupid question, just trying to help as I've never used these options:
| man make.conf wrote: |
usersync
Drop privileges to the owner of ${repository_location} for emerge(1) --sync operations. Note that this feature assumes that all sub‐directories of ${repository_location} have the same ownership as ${repository_location} itself. It is the user’s responsibility to ensure correct ownership, since otherwise Portage would have to waste time validating ownership for each and every sync operation. |
Are sub-directories also owned by portage:portage on all the machines? |
|
| Back to top |
|
|
nagmat84
Apprentice
Joined: 27 Mar 2007
Posts: 260
|
| Posted: Sat Jun 15, 2024 10:16 am Post subject: |
|
|
| Quote: | | Maybe a stupid question, just trying to help as I've never used these options: |
Are you sure? I haven't enabled those features in my make.conf explicitly, those features seem to have become the profile default.
| Quote: | | Are sub-directories also owned by portage:portage on all the machines? |
Yes. I have even run a "chown -R" just to be sure. |
|
| Back to top |
|
|
eeckwrk99
Apprentice
Joined: 14 Mar 2021
Posts: 231
Location: Gentoo forums
|
| Posted: Sat Jun 15, 2024 11:01 am Post subject: |
|
|
| nagmat84 wrote: | | Quote: | | Maybe a stupid question, just trying to help as I've never used these options: |
Are you sure? I haven't enabled those features in my make.conf explicitly, those features seem to have become the profile default.
|
You're right, they're used by default here too:
| Code: | #emerge --info
[..]
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg-live candy config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync merge-wait multilib-strict network-sandbox news parallel-fetch parallel-install pid-sandbox pkgdir-index-trusted preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
[..] |
It seems that the error "fatal: detected dubious ownership in repository at '/var/db/repos/booboo/.git'" is caused by Git itself, I can trigger it too:
| Code: | % cd /var/db/repos/gentoo/.git
% git log
fatal: detected dubious ownership in repository at '/var/db/repos/gentoo/.git'
To add an exception for this directory, call:
git config --global --add safe.directory /var/db/repos/gentoo/.git |
| Code: | % su -
# cd /var/db/repos/gentoo/.git
# git log
commit ee64bad62f51820fd8f5b68d42a7e5ae59a976a4 (grafted, HEAD -> stable, origin/stable, origin/HEAD)
Author: Repository mirror & CI <repomirrorci@gentoo.org>
Date: Fri Jun 14 18:34:09 2024 +0000
2024-06-14 18:34:08 UTC |
I guess the error is printed if the current user running the command isn't the owner of the Git repository.
I wonder if you'd still get the error with:
| Code: | | # chown -R root:root /var/db/repos |
|
|
| Back to top |
|
|
nagmat84
Apprentice
Joined: 27 Mar 2007
Posts: 260
|
| Posted: Sat Jun 15, 2024 11:17 am Post subject: |
|
|
| Quote: | | It seems that the error "fatal: detected dubious ownership in repository at '/var/db/repos/booboo/.git'" is caused by Git itself, I can trigger it too: |
Yes, Git itself triggers the error. That was not the question. This is a security feature built into Git. One can read about the background of this feature in the Git documentation. The question is rather why does Git print it. "Portage" is the owner of the process running "emerge --sync" and and of the repositories. Hence, this error should not appear. Unless, Emerge does not some crazy user-joggling in between the error shouldn't be there. Another interesting question is why does the error only occurs when the clients sync with the built server, but why does the error not occur when the built server syncs with upstream? Where is the difference?
| Quote: | I wonder if you'd still get the error with: | Code: | | # chown -R root:root /var/db/repos |
|
Yes, I do. |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22619
|
| Posted: Sat Jun 15, 2024 2:14 pm Post subject: |
|
|
As I read the git code, there are only two places that can raise this error, and both are because ensure_valid_ownership failed. In turn, that should be because lstat found a st_uid that is not acceptable, on one of three possible files. I suggest using strace to identify which of these three files is being rejected.
Also, as a personal opinion, I dislike programs that give vague errors like this. I could see an argument for an enhancement to git that it should be more specific than just "dubious ownership" and instead tell you the specific uid it wanted to see and name the specific file that had the wrong uid. |
|
| Back to top |
|
|
nagmat84
Apprentice
Joined: 27 Mar 2007
Posts: 260
|
| Posted: Sat Jul 13, 2024 11:02 am Post subject: |
|
|
I am lost. I get | Code: | portage@pc /var/db/repos/booboo $ strace -f --trace='%lstat' -v git fetch origin --depth 1
strace: Process 6536 attached
fatal: detected dubious ownership in repository at '/var/db/repos/booboo/.git'
To add an exception for this directory, call:
git config --global --add safe.directory /var/db/repos/booboo/.git |
Note: Here, I did not call "emerge --sync", but invoked the problematic git command as user portage directly in order to avoid unnecessary output from strace due to all the things "emerge --sync" does.
From the output above, it seems that git never calls "lstat". However, from looking at the git code, it should. The method "die_upon_dubious_ownership" (in git source code) calls "ensure_valid_ownership" which calls "is_path_owned_by_current_user" three times and the letter is defined to be an alias for "is_path_owned_by_current_uid". That again clearly calls "lstat".
When I remove the filter "--trace?'%lstat'" from the command above, I get a lot of output (as git reads all the configuration settings upon start-up), but nothing caught my eye. |
|
| Back to top |
|
|
nagmat84
Apprentice
Joined: 27 Mar 2007
Posts: 260
|
| Posted: Sat Jul 13, 2024 12:59 pm Post subject: |
|
|
I solved the problem. The error message was completely misleading. The actual reason was that the SSH-user who logged into the remote server to sync the git repo was not the owner of the (remote) repository.
As recommend by wiki articles, which I followed to setup my build sever, I created a special user "gentoo" on my build server to create, sign and host Genoo Binary packages. My clients also used that user to fetch binary packages from the build server, i.e. on my clients I had | Code: | client ~ # cat /etc/portage/binrepos.conf/my-build-server.conf
[mhnnet]
priority = 1
sync-uri = ssh://gentoo@my-build-server.my-domain.tld/srv/gentoo-build/<client-chroot>/var/cache/binpkgs |
As I already had a special user "gentoo" to pull binary packages from my build server, I also used that user to sync the ebuild repository. I didn't see a point in making the user "portage" accessible through SSH. So I had | Code: | client ~ # cat /etc/portage/repos.conf/gentoo.conf
[DEFAULT]
main-repo = gentoo
[gentoo]
location = /var/db/repos/gentoo
sync-type = rsync
sync-uri = ssh://gentoo@my-build-server.my-domain.tld/var/db/repos/gentoo/
client ~ # cat /etc/portage/repos.conf/booboo.conf
[booboo]
location = /var/db/repos/booboo
sync-type = git
sync-uri = ssh://gentoo@my-build-server.my-domain.tld/var/db/repos/booboo/ |
The remote ebuild repositories (on my build server) are owned by "portage" (as all ebuild repositories are). Nonetheless, they were world readable. Rsync over ssh (for the first repositories) worked without problems. But Git over SSH didn't liked that. So the situation was like that: local (i.e. client-side) repository Booboo was owned by "portage", remote (i.e. server-side) repository Booboo was owned by "portage", but SSH used the user "gentoo". The error message I saw was generated on the server side by the GIT command which was executed within the SSH shell.
Solution: I enabled PPK-login for the portage user on my build server and now I am using the user portage to sync the ebuild repositories. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
