| View previous topic :: View next topic |
| Author |
Message |
proteusx
Guru
Joined: 21 Jan 2008
Posts: 340
|
 Posted: Thu Apr 08, 2021 3:02 pm Post subject: 300 Rebuilds due to USE=" (-abi_riscv_lp64%) ..." |
 |
|
After today's sync emerge wants to needlessly rebuild about 300 packages because of some riscv related ABI use flags. Here are a couple of typical cases:
| Code: | emerge -auDNv world
.
.
.
[ebuild R ] dev-libs/mpfr-4.1.0 USE="-static-libs (-abi_riscv_lp64%) (-abi_riscv_lp64d%)" ABI_X86="32 (64) (-x32)"
[ebuild R ] media-libs/ladspa-sdk-1.15-r1 USE="(-abi_riscv_lp64%) (-abi_riscv_lp64d%)" ABI_X86="32 (64) (-x32)"
.
.
. |
My systems are x86_64 (amd64) and I have nothing to do with riscv.
Does anyone know where in the profile (or wherever) these abi_riscv_lp* things come from? I looked through some of the ebuilds, the eclasses and the profiles and found nothing.
Thank you.
Stickied. -- Zucca |
|
| Back to top |
|
 |
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 6145
Location: Dallas area
|
| Posted: Thu Apr 08, 2021 3:07 pm Post subject: |
|
|
Don't use the -N flag, that's pulling the riscv stuff in.
It's all due to the fact that some one that put the flags in, doesn't know how to back them out now, without causing grief for all the users.
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
| Back to top |
|
|
proteusx
Guru
Joined: 21 Jan 2008
Posts: 340
|
| Posted: Thu Apr 08, 2021 3:16 pm Post subject: |
|
|
Thank you for the reply.
I am just curious as to where in the emerge system these flags are set. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54577
Location: 56N 3W
|
| Posted: Thu Apr 08, 2021 3:26 pm Post subject: |
|
|
proteusx,
In the profiles.
riscv is a newish arch. The ::gentoo repo is being prepared to support it. riscv hardware is becoming available at prices with reach of hobbyists.
It has a long way to go to get to Raspberry Pi like prices,
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22634
|
| Posted: Thu Apr 08, 2021 4:28 pm Post subject: |
|
|
| Anon-E-moose wrote: | | It's all due to the fact that some one that put the flags in, doesn't know how to back them out now, without causing grief for all the users. |
What is the proper way to make the required change without triggering rebuilds for people who use --newuse? |
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 6145
Location: Dallas area
|
| Posted: Thu Apr 08, 2021 4:28 pm Post subject: |
|
|
But being on amd64, one should never see the riscv flags, no more than one sees any other architecture.
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 6145
Location: Dallas area
|
| Posted: Thu Apr 08, 2021 4:38 pm Post subject: |
|
|
| Hu wrote: | | Anon-E-moose wrote: | | It's all due to the fact that some one that put the flags in, doesn't know how to back them out now, without causing grief for all the users. |
What is the proper way to make the required change without triggering rebuilds for people who use --newuse? |
I don't know off the top of my head, I didn't create the mess in the first place. I'd have to play with it.
Edit to add: I don't think there's a quick and easy fix for the rebuild if using new use flag, if there was one, they would likely have found it.
Having said that, avoid using -N (new use flags), use -U (changed use flags).
A regular package emerge will always grab all changed and new flags, if one is sure that they need new flags, ie emerge -1<rest of flags> <pkg name>
And in time, they will roll off with normal emerges. OR bite the bullet and recompile all packages (waste of resources, IMO)
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22634
|
| Posted: Thu Apr 08, 2021 6:03 pm Post subject: |
|
|
| I agree, --changed-use is better than --newuse here. From your prior comment ("doesn't know how to back them out now"), I thought there was a way, even if not straightforward for the maintainers, to get the desired result without requiring users to do anything special. |
|
| Back to top |
|
|
proteusx
Guru
Joined: 21 Jan 2008
Posts: 340
|
| Posted: Thu Apr 08, 2021 7:04 pm Post subject: |
|
|
In case anyone has the same problem, here is how I prevented the pointless rebuilding.
Backing up the /var/db/pkg directory may be a good move.
1. Emerge everything except the USE="(-abi_riscv_lp*)" crap.
2. Pretend emerge the packages with --newuse and send the output to a file
| Code: | | emerge -DuNpv world | tee emerge-list |
3. Inspect the file 'emerge-list' to make sure that it lists only packages with riscv flags This step is not really necessary.
4. Run this one liner to delete the offending use flags from the package database.
| Code: | | for d in $(cat emerge-list | cut -d\ -f8); do sed -i 's/abi_riscv_lp64 //;s/abi_riscv_lp64d //' /var/db/pkg/$d/*USE*; done |
This will make Al Gore and Greta Thunberg very happy and the poor polar bears will not drown. |
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 6145
Location: Dallas area
|
| Posted: Thu Apr 08, 2021 7:07 pm Post subject: |
|
|
There is a way to fix it without re-compile/emerge and that is to directly edit the relevant files under /var/db/pkg.
I'll look at what it would take to do it programmatically, but should be doable.
Lets look at mpfr from the first post
| Code: | /var/db/pkg/dev-libs/mpfr-4.1.0 $ grep _riscv *
IUSE:static-libs abi_x86_32 abi_x86_64 abi_x86_x32 abi_mips_n32 abi_mips_n64 abi_mips_o32 abi_riscv_lp64d abi_riscv_lp64 abi_s390_32 abi_s390_64
IUSE_EFFECTIVE:abi_mips_n32 abi_mips_n64 abi_mips_o32 abi_riscv_lp64 abi_riscv_lp64d abi_s390_32 abi_s390_64 abi_x86_32 abi_x86_64 abi_x86_x32 alpha amd64 amd64-fbsd amd64-linux arm arm64 elibc_AIX elibc_Cygwin elibc_Darwin elibc_DragonFly elibc_FreeBSD elibc_HPUX elibc_Interix elibc_NetBSD elibc_OpenBSD elibc_SunOS elibc_Winnt elibc_bionic elibc_glibc elibc_mingw elibc_mintlib elibc_musl elibc_uclibc hppa ia64 kernel_AIX kernel_Darwin kernel_FreeBSD kernel_HPUX kernel_NetBSD kernel_OpenBSD kernel_SunOS kernel_Winnt kernel_freemint kernel_linux m68k m68k-mint mips ppc ppc-aix ppc-macos ppc64 ppc64-linux prefix prefix-guest prefix-stack riscv s390 sparc sparc-solaris sparc64-solaris static-libs userland_BSD userland_GNU x64-cygwin x64-macos x64-solaris x64-winnt x86 x86-cygwin x86-fbsd x86-linux x86-macos x86-solaris x86-winnt |
This shows two files "_riscv" shows up in, the IUSE and IUSE_EFFECTIVE file.
I'll have to play with it and see but I think just editing, using sed, whatever on IUSE to remove those would suffice to fix the new keyword re-emerge problem.
As always backup beforehand. Wise to do the whole /var/db/pkg dirs into a compressed tarball, that saves everything with minimal space used.
Just tried, hand edit of IUSE
Before edit of IUSE
| Code: | | dev-libs/mpfr-4.1.0:0/6::gentoo USE="-static-libs (-abi_riscv_lp64%) (-abi_riscv_lp64d%)" 0 KiB |
After edit to remove abi_riscv_* flags
| Code: | | dev-libs/mpfr-4.1.0:0/6::gentoo USE="-static-libs" 0 KiB |
A script to walk the directory structure, with a sed command for the edit would probably work.
Edit to add: I see proteusx thought of the same fix 
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
| Back to top |
|
|
GDH-gentoo
Veteran
Joined: 20 Jul 2019
Posts: 1693
Location: South America
|
| Posted: Thu Apr 08, 2021 7:22 pm Post subject: |
|
|
| Never updating with --newuse looks waaaay easier. Just saying... |
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 6145
Location: Dallas area
|
| Posted: Thu Apr 08, 2021 7:52 pm Post subject: |
|
|
| GDH-gentoo wrote: | | Never updating with --newuse looks waaaay easier. Just saying... |
Is easier. I haven't used the new use flag in ... forever it seems like.
But I do edit files in /var/db/pkg, typically the repository file, never had a problem.
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
| Back to top |
|
|
koonat
n00b
Joined: 06 Aug 2006
Posts: 6
Location: back alley oregon
|
| Posted: Sat Apr 10, 2021 5:44 am Post subject: |
|
|
I've always used --newuse
Glad I found this thread, I was a little confused at what I was seeing.
So, what's the long-term plan, here, then? Do I have to edit files? Never use --newuse again? Is this getting cleaned up?
_________________
Double click here for unlimited hapiness. |
|
| Back to top |
|
|
Ionen
Developer
Joined: 06 Dec 2018
Posts: 2852
|
| Posted: Sat Apr 10, 2021 5:59 am Post subject: |
|
|
| koonat wrote: | | So, what's the long-term plan, here, then? Do I have to edit files? Never use --newuse again? Is this getting cleaned up? |
This was caused by the cleanup if anything, the flags (and workaround) were removed and --newuse thinks it needs to rebuild.
e.g. "disabled flag" -> "removed flag" is a rebuild with --newuse, but isn't it meaningless? --changed-use on the other hand wouldn't but would still rebuild if it was "enabled flag" -> "removed flag" given that one is on->off rather than off->off and an actual change.
--newuse "could" make sense if "disabled" -> "removed" actually _enabled_ the feature by default but in these cases there should be a version/revbump that will force rebuild. Personally I'd say just don't use --newuse anymore.
Anyhow, removal of these flags was ~1.5 years ago, so you should have rebuilt major things already (aka large slow packages that update often, rest should be trivial even if numerous), which led to the permanent removal of the workaround and now brought this up again. |
|
| Back to top |
|
|
krumpf
Apprentice
Joined: 15 Jul 2018
Posts: 187
|
| Posted: Sat Apr 10, 2021 6:56 am Post subject: |
|
|
Reading this thread, I see many people recommending -U instead of -N with emerge.
Then should the emerge -c text be modified, since it advises to use the -N before depcleaning ?
I've always favored -N during updates because of this
| Code: | # emerge -pc
* Always study the list of packages to be cleaned for any obvious
* mistakes. Packages that are part of the world set will always
* be kept. They can be manually added to this set with
* `emerge --noreplace <atom>`. Packages that are listed in
* package.provided (see portage(5)) will be removed by
* depclean, even if they are part of the world set.
*
* As a safety measure, depclean will not remove any packages
* unless *all* required dependencies have been resolved. As a
* consequence of this, it often becomes necessary to run
* `emerge --update --newuse --deep @world` prior to depclean.
|
|
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 6145
Location: Dallas area
|
| Posted: Thu Jul 15, 2021 6:41 pm Post subject: |
|
|
| Hu wrote: | | Anon-E-moose wrote: | | It's all due to the fact that some one that put the flags in, doesn't know how to back them out now, without causing grief for all the users. |
What is the proper way to make the required change without triggering rebuilds for people who use --newuse? |
Run this 
| Code: | #!/usr/bin/python3
import os
bad_flags = {"abi_riscv_lp64", "abi_riscv_lp64d"}
for dp, dn, fn in os.walk("/var/db/pkg"):
for targ_fn in [ "IUSE", "IUSE_EFFECTIVE" ]:
if targ_fn not in fn:
continue
usep = os.path.join(dp, targ_fn)
with open(usep, "r") as usef:
old_dat = usef.read()
old_flags = set(old_dat.split())
new_flags = old_flags - bad_flags
if old_flags == new_flags:
continue
with open(usep + ".bak", "w") as usef:
usef.write(old_dat)
with open(usep, "w") as usef:
usef.write(" ".join(sorted(list(new_flags)))) |
| Code: | pworld -N |grep riscv
[ebuild R ] dev-games/physfs-3.0.2::gentoo USE="zip -7zip -doc -grp -hog -iso -mvl -qpak -slb -static-libs -vdf -wad (-abi_riscv_ilp32%) (-abi_riscv_ilp32d%)" 0 KiB |
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
| Back to top |
|
|
proteusx
Guru
Joined: 21 Jan 2008
Posts: 340
|
| Posted: Thu Jul 15, 2021 7:27 pm Post subject: |
|
|
For some of us who are not masters of the python, it would be nice if there was some explanation as to what the script does.
Also, what is 'pworld'?
Thank you. |
|
| Back to top |
|
|
Anon-E-moose
Watchman
Joined: 23 May 2008
Posts: 6145
Location: Dallas area
|
| Posted: Thu Jul 15, 2021 7:44 pm Post subject: |
|
|
| proteusx wrote: | For some of us who are not masters of the python, it would be nice if there was some explanation as to what the script does.
Also, what is 'pworld'?
Thank you. |
alias pworld='emerge -pvuD @world' 
default I don't use the -N flag, but wanted to show that indeed the abi_riscv_lp64* stuff was removed
It walk all directories in /var/db/pkg, looking at IUSE* files for any reference to abi_riscv_lp64*
if it finds those flags, it makes a backup copy ".bak" of each IUSE file and then removes the flags.
To be safe, I did tar up all /var/db/pkg before I did anything but it wasn't necessary, in my case.
Edit to add: and it wasn't written by me, you can thank D.Robbins for it
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
