View previous topic :: View next topic |
Author |
Message |
John R. Graham Administrator
Joined: 08 Mar 2005 Posts: 10666 Location: Somewhere over Atlanta, Georgia
|
Posted: Mon Dec 02, 2024 7:46 pm Post subject: Unexpected issue fetching a local git repo from an ebuild |
|
|
I have a workflow I've been using for years where I create a bespoke ebuild that fetches source code from a "feature branch" (that I create) of a local git repository that is itself a clone of an upstream repository. Currently I'm working on upstream samba so I've cloned the upstream samba repo into /home/jgraham/Projects/Gentoo/samba, checked out the tag I'm interested in and created the feature branch, in this case named nss_winbind_experiments, and switched to that branch. This structure allows me to make changes in the local repo and, as soon as I commit those changes, they can be built and installed with Portage. I use versions like 9998, 9997 for my local experimental branches.
This has been working for years but in my latest attempt git is no longer happy: Code: | terra ~ # emerge -1v =samba-9998
These are the packages that would be merged, in reverse order:
Calculating dependencies... done!
Dependency resolution took 9.84 s (backtrack: 0/30).
[ebuild U ~] net-fs/samba-9998:0/2.10.0::JRG [4.21.1_p1:0/2.10.0::JRG] USE="acl -addc ads -ceph client -cluster cups -debug (-fam) -glusterfs -gpg -iprint -json ldap -llvm-libunwind -lmdb pam -profiling-data python -quota regedit (-selinux) -snapper -spotlight syslog (-system-heimdal) system-mitkrb5 -systemd (-test) -unwind winbind -zeroconf" ABI_X86="-32 (64) (-x32)" PYTHON_SINGLE_TARGET="-python3_10 -python3_11 python3_12 -python3_13" 0 KiB
Total: 1 package (1 upgrade), Size of downloads: 0 KiB
>>> Verifying ebuild manifests
>>> Emerging (1 of 1) net-fs/samba-9998::JRG
* Using python3.12 to build
>>> Unpacking source...
* Repository id: home_jgraham_Projects_Gentoo_samba.git
* To override fetched repository properties, use:
* EGIT_OVERRIDE_REPO_HOME_JGRAHAM_PROJECTS_GENTOO_SAMBA
* EGIT_OVERRIDE_BRANCH_HOME_JGRAHAM_PROJECTS_GENTOO_SAMBA
* EGIT_OVERRIDE_COMMIT_HOME_JGRAHAM_PROJECTS_GENTOO_SAMBA
* EGIT_OVERRIDE_COMMIT_DATE_HOME_JGRAHAM_PROJECTS_GENTOO_SAMBA
*
* Fetching file:///home/jgraham/Projects/Gentoo/samba ...
git fetch file:///home/jgraham/Projects/Gentoo/samba +refs/heads/nss_winbind_experiments:refs/heads/nss_winbind_experiments
fatal: detected dubious ownership in repository at '/home/jgraham/Projects/Gentoo/samba/.git'
To add an exception for this directory, call:
git config --global --add safe.directory /home/jgraham/Projects/Gentoo/samba/.git
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists. | Now that seems like good advice, so checking existence and access rights of my local repo I get: Code: | terra ~ # stat /home/jgraham/Projects/Gentoo/samba/.git
File: /home/jgraham/Projects/Gentoo/samba/.git
Size: 12 Blocks: 17 IO Block: 131072 directory
Device: 0,32 Inode: 129304 Links: 7
Access: (0755/drwxr-xr-x) Uid: (10000/HOME\jgraham) Gid: (11001/HOME\domain users)
Access: 2024-12-02 13:47:42.665658596 -0500
Modify: 2024-12-02 13:46:23.286102287 -0500
Change: 2024-12-02 13:47:42.665658596 -0500
Birth: 2024-12-02 13:44:18.226801144 -0500
terra ~ # namei -m /home/jgraham/Projects/Gentoo/samba/.git
f: /home/jgraham/Projects/Gentoo/samba/.git
drwxr-xr-x /
drwxr-xr-x home
drwxr-xr-x jgraham
drwxr-xr-x Projects
drwxr-xr-x Gentoo
drwxr-xr-x samba
drwxr-xr-x .git | That looks good to me (or at least doesn't look like an impediment). Do you all see an issue?
Next I followed the other advice, creating a safe.directory exception: Code: | terra ~ # sudo -u portage git config --global --add safe.directory /home/jgraham/Projects/Gentoo/samba/.git | but this does not appear to have any effect on the issue. I get the same error message.
The full ebuild is wgetpasted here. It's essentially the standard samba ebuild with two additions:- Lines 13 - 41 were added to establish custom repo or source tarball URIs depending on the version number, and
- Lines 215 - 225 are a new src_unpack() phase function that uses either the git-r3 eclass or the default src_unpack() depending on the version number.
Any advice? (Other than to stop spitting into the wind and just fork the darned upstream repo and create my feature branches there?)
- John _________________ I can confirm that I have received between 0 and 499 National Security Letters. |
|
Back to top |
|
|
CaptainBlood Advocate
Joined: 24 Jan 2010 Posts: 3954
|
Posted: Mon Dec 02, 2024 8:51 pm Post subject: |
|
|
Unexperienced here...
Did you had previous success with EAPI 8?
Thks 4 ur attention, interest & support. _________________ USE="-* ..." in /etc/portage/make.conf here, i.e. a countermeasure to portage implicit braces, belt & diaper paradigm
LT: "I've been doing a passable imitation of the Fontana di Trevi, except my medium is mucus. Sooo much mucus. " |
|
Back to top |
|
|
logrusx Advocate
Joined: 22 Feb 2018 Posts: 2475
|
|
Back to top |
|
|
John R. Graham Administrator
Joined: 08 Mar 2005 Posts: 10666 Location: Somewhere over Atlanta, Georgia
|
Posted: Mon Dec 02, 2024 10:32 pm Post subject: |
|
|
CaptainBlood wrote: | ...Did you had previous success with EAPI 8? | It was before EAPI 8 came out when I last created one of these beasts, but that turns out not to be it. See below.
That link gave me a clue. I was blindly applying git's recommendation, without researching the options. I was sudoing the recommended command with user "portage", because I'd verified that that was the user under which git was running, but that must've not been exactly correct. The link showed a version of the command with system-wide scope: Code: | git config --system --add safe.directory /home/jgraham/Projects/Gentoo/samba/.git | which may not result in the least escalation of privilege, but did work. I had lazily not researched this well enough. Thanks so much for the help.
- John _________________ I can confirm that I have received between 0 and 499 National Security Letters. |
|
Back to top |
|
|
CaptainBlood Advocate
Joined: 24 Jan 2010 Posts: 3954
|
Posted: Tue Dec 03, 2024 12:13 am Post subject: |
|
|
Glad you made it with support from others.
Nice sharing of kinda uncommon ebuild architecture,
Bookmarked.
Thks 4 ur attention, interest & support. _________________ USE="-* ..." in /etc/portage/make.conf here, i.e. a countermeasure to portage implicit braces, belt & diaper paradigm
LT: "I've been doing a passable imitation of the Fontana di Trevi, except my medium is mucus. Sooo much mucus. " |
|
Back to top |
|
|
Hu Administrator
Joined: 06 Mar 2007 Posts: 22771
|
Posted: Tue Dec 03, 2024 12:16 am Post subject: |
|
|
Yes, using --system is unusual here, since that writes it to the system-wide configuration, but you only needed it for the Portage user. I think I recall that Portage sets $HOME to a value other than the one in /etc/passwd, so your sudo'd command probably wrote the file in a directory that would work if you ran su - portage and ran git commands, but does not work when emerge runs git as part of an ebuild. I do not recall how Portage sets $HOME, or why I think that it is special. Perhaps the environment file from a failed run would help. |
|
Back to top |
|
|
John R. Graham Administrator
Joined: 08 Mar 2005 Posts: 10666 Location: Somewhere over Atlanta, Georgia
|
Posted: Tue Dec 03, 2024 4:23 am Post subject: |
|
|
I had gone so far as to look at portage's home directory as recorded in /etc/passwd and confirmed that the initially recommended command (--global; not --system) indeed wrote a .gitconfig file in that directory, but you make a good point. Checking the environment file produced by Code: | ebuild `equery which samba` prepare | shows no HOME variable defined. But instrumenting the eclass with additional einfo statements shows that the home directory is uniquely created for each run in "/var/tmp/portage/${CATEGORY}/${P}/homedir", which is empty, so no way would setting a user-specific global safe directory from the command line work.
- John _________________ I can confirm that I have received between 0 and 499 National Security Letters. |
|
Back to top |
|
|
John R. Graham Administrator
Joined: 08 Mar 2005 Posts: 10666 Location: Somewhere over Atlanta, Georgia
|
Posted: Tue Dec 03, 2024 4:26 am Post subject: |
|
|
CaptainBlood wrote: | Glad you made it with support from others.
Nice sharing of kinda uncommon ebuild architecture,
Bookmarked. | Yes, it's part of what makes Gentoo awesome. By the way, I initially described the workflow, in somewhat greater detail, back in 2017. Hu helped me refine the idea.
- John _________________ I can confirm that I have received between 0 and 499 National Security Letters. |
|
Back to top |
|
|
logrusx Advocate
Joined: 22 Feb 2018 Posts: 2475
|
Posted: Tue Dec 03, 2024 5:23 am Post subject: |
|
|
John R. Graham wrote: | I was blindly applying git's recommendation, without researching the options. I was sudoing... |
Sudo has been the culprit of so many problems here on the forums. I recommend against using it.
Best Regards,
Georgi |
|
Back to top |
|
|
Hu Administrator
Joined: 06 Mar 2007 Posts: 22771
|
Posted: Tue Dec 03, 2024 1:29 pm Post subject: |
|
|
Although sudo is misused with shocking frequency, it is innocent in this case. John was misled by the assumption that the command printed by git (not sudo) would work as written. If Portage ran all ebuilds with $HOME set to the value from /etc/passwd, the command probably would have worked. Since it does not, neither sudo -u portage git ... nor su - -c 'git ...' will do the right thing. |
|
Back to top |
|
|
John R. Graham Administrator
Joined: 08 Mar 2005 Posts: 10666 Location: Somewhere over Atlanta, Georgia
|
Posted: Tue Dec 03, 2024 10:23 pm Post subject: |
|
|
You know, when I first crafted this workflow back in 2017, the running ebuild environment contained ${HOME} defined to what was specified in the /etc/passwd file for the portage user (call that the "normal home directory" for the sake of discussion). That directory was, of course, writable to the portage user. That means that ebuilds could alter the state of the system in a way that would affect future ebuilds. If I may speculate (as I have know first-hand knowledge for the reason for the change), it appears that what may have happened is, in a laudable attempt to create a completely consistent and normalized environment for all ebuilds, the new home directory location was ephemerally created in /var/tmp/portage/${CATEGORY}/${P}/homedir as part of the setup for each run of an ebuild, created empty. The ebuild is free (PMS confirms) to populate that directory if some program involved in the build needs something to be there.
But what just occurred to me is that it could be useful if, instead of creating an empty homedir there, portage were to populate it with the contents of the normal portage user home directory. This would still allow each new ebuild to have a consistent environment—because the contents of the homedir would be guaranteed to be consistent from run to run. The system owner could intentionally populate the normal portage home directory but the ebuild would be forbidden (at least by best practice) from writing there. It'd also make the git command that failed for me work in an unsurprising way—a better way than being forced to change the system-wide safe directory list.
Can anyone think of a downside to this approach? I might create a bug report / feature request—and work on an implementation.
- John _________________ I can confirm that I have received between 0 and 499 National Security Letters. |
|
Back to top |
|
|
|