View previous topic :: View next topic |
Author |
Message |
Xywa Veteran
Joined: 23 Jul 2005 Posts: 1631 Location: /mnt/Gentoo/Europe
|
Posted: Thu Jan 29, 2015 8:51 pm Post subject: Cpu_flags_x86 |
|
|
Hi,
I have just read a news about new: CPU_FLAGS_X86
Quote: | In order not to lose CPU-specific optimizations, users will be required
to update their make.conf (and package.use) file. For example, if
the following USE flags were present:
USE="mmx mmxext sse sse2 sse3"
Those flags need to be copied into:
CPU_FLAGS_X86="mmx mmxext sse sse2 sse3"
|
I have used yours pythond script and I have differnet flags than before (eg. sse4_2 instead sse42 or few new eg. mmxext or aes ). What should I do?
Code: | USE="uxa xvmc vdpau ruby_targets_ruby21 ois soprano tools abi_x86_64 opus gles2 sqlite \
mysql kdenlive script ebook djvu gde ios lm_sensors scanner \
libkms xmp demosaic vaapi sna sse41 sse42 sse3 cddb java \
reports nsplugin smp threads embedded ssse3 vcd dv frei0r icu \
consolekit dbus policykit udev ffmpeg mlt melt minizip mmx sse \
sse2 alsa"
CPU_FLAGS_X86="aes avx mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3" |
|
|
Back to top |
|
|
Martux Veteran
Joined: 04 Feb 2005 Posts: 1917
|
Posted: Thu Jan 29, 2015 9:00 pm Post subject: |
|
|
Use the flags from the script, if something changes in a packages use flags, you will see it like this:
Code: |
(-3dnow%) (-mmx%*) (-sse%*) (-sse2%*)" CPU_FLAGS_X86="mmx%* sse%* sse2%* -3dnow%
|
_________________ "Coincidence is God's way of remaining anonymous."
Albert Einstein
"The road to success is always under construction" |
|
Back to top |
|
|
229566 Tux's lil' helper
Joined: 16 Aug 2010 Posts: 127
|
Posted: Fri Jan 30, 2015 1:29 am Post subject: |
|
|
ffmpeg loses cpu-specific flags because they're removed from USE, but doesn't appear to have CPU_FLAGS_X86. Is this a bug? Or not all ebuilds are compliant at this stage? |
|
Back to top |
|
|
khayyam Watchman
Joined: 07 Jun 2012 Posts: 6227 Location: Room 101
|
Posted: Fri Jan 30, 2015 1:41 am Post subject: |
|
|
GrueXYZ wrote: | ffmpeg loses cpu-specific flags because they're removed from USE, but doesn't appear to have CPU_FLAGS_X86. Is this a bug? Or not all ebuilds are compliant at this stage? |
GrueXYZ ... you're not supposed to remove them from USE ...
eselect news wrote: | In order to ensure safe migration and maintain compatibility with external repositories, it is recommended to preserve the old USE settings for a period of one year or until no package of interest is still using them. |
best ... khay |
|
Back to top |
|
|
ct85711 Veteran
Joined: 27 Sep 2005 Posts: 1791
|
Posted: Fri Jan 30, 2015 1:53 am Post subject: |
|
|
Oh fun, another shift that most likely is going to be half pushed out, breaking stuff because it's not pushed fully out to everything, and properly supported... |
|
Back to top |
|
|
229566 Tux's lil' helper
Joined: 16 Aug 2010 Posts: 127
|
Posted: Fri Jan 30, 2015 3:06 am Post subject: |
|
|
khayyam wrote: | ... you're not supposed to remove them from USE ... |
Ah dammit I somehow skipped that last paragraph. Thanks! |
|
Back to top |
|
|
Maitreya Guru
Joined: 11 Jan 2006 Posts: 445
|
Posted: Fri Jan 30, 2015 2:23 pm Post subject: |
|
|
I just moved them from USE to CPU_FLAGS_X86 and then add ${CPU_FLAGS_X86} to USE |
|
Back to top |
|
|
limn l33t
Joined: 13 May 2005 Posts: 997
|
Posted: Fri Jan 30, 2015 9:39 pm Post subject: |
|
|
Theirs not to reason why |
|
Back to top |
|
|
lost+found Guru
Joined: 15 Nov 2004 Posts: 509 Location: North~Sea~Coa~s~~t~~~
|
Posted: Fri Jan 30, 2015 10:40 pm Post subject: |
|
|
The script doesn't mention the `sse4' flag on a Core i3, although there are packages offering it...
Code: | # cpuinfo2cpuflags-x86
CPU_FLAGS_X86="mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 ssse3"
# grep ':sse4 ' /usr/portage/profiles/use.local.desc
dev-libs/marisa:sse4 - Enable sse4 instruction support
media-libs/freeverb3:sse4 - Enable SSE4 support
media-video/ffmpeg:sse4 - Enables SSE4 optimizations: Intel Core microarchitecture and AMD K10 or later.
net-misc/bfgminer:sse4 - Enable support for CPU SSE4 mining algorithm |
I was surprised of the `popcnt' flag... Didn't know it |
|
Back to top |
|
|
iamben Apprentice
Joined: 10 May 2004 Posts: 278
|
Posted: Fri Jan 30, 2015 11:32 pm Post subject: |
|
|
Once all packages have been converted, you will no longer need CPU_FLAGS_X86 in USE. ffmpeg was converted 6 hrs ago, I believe all packages in portage have been converted, but all bets are off with overlays |
|
Back to top |
|
|
eccerr0r Watchman
Joined: 01 Jul 2004 Posts: 9855 Location: almost Mile High in the USA
|
Posted: Sat Jan 31, 2015 12:08 am Post subject: |
|
|
Is it possible to now do a per-package CPU_FLAGS_X86?
Like if I want a generic install but allow some apps to be runnable only on those machines that have the capability? (namely want a generic install that will run on all CPUs but allow adobe-flash to run only on those capable of SSE2)? _________________ Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
Back to top |
|
|
khayyam Watchman
Joined: 07 Jun 2012 Posts: 6227 Location: Room 101
|
Posted: Sat Jan 31, 2015 12:35 am Post subject: |
|
|
hello ... I honestly don't see what benefit this change brings, I mean, does it matter if its in USE or some other var if they are all essentially useflags. I can see negatives in the forum of package rebuilds due to --changed-use || --newuse but what exactly is the problem its intended to solve? Is the USE namespace cluttered, and if so how does adding a sub-set improve this?
With the addition of PYTHON_TARGETS we saw no end of rebuilds (for users here) because one or other target had changed, and because they were using --newuse rather than --changed-use ... the former is practically unusable because useflags change constantly as python versions are bumped, or made obsolete, and/or packages incur a use change. While its not likely that this will be the case with CPU_FLAGS_X86 I question the wisdom of causing further rebuilds for no apparent reason ... other than having a new subset of useflags what has been achieved?
best ... khay |
|
Back to top |
|
|
khayyam Watchman
Joined: 07 Jun 2012 Posts: 6227 Location: Room 101
|
Posted: Sat Jan 31, 2015 12:40 am Post subject: |
|
|
eccerr0r wrote: | Is it possible to now do a per-package CPU_FLAGS_X86? |
eccerr0r ... yes, you'd use 'cpu_flags_x86_mmx', 'cpu_flags_x86_sse', etc ... I haven't tested, but I imagine it'll work exactly the same as 'abi_x86_32', 'python_targets_python2_7', etc.
HTH & best ... khay |
|
Back to top |
|
|
iamben Apprentice
Joined: 10 May 2004 Posts: 278
|
Posted: Sat Jan 31, 2015 1:14 am Post subject: |
|
|
khayyam wrote: | hello ... I honestly don't see what benefit this change brings, I mean, does it matter if its in USE or some other var if they are all essentially useflags. I can see negatives in the forum of package rebuilds due to --changed-use || --newuse but what exactly is the problem its intended to solve? Is the USE namespace cluttered, and if so how does adding a sub-set improve this?
With the addition of PYTHON_TARGETS we saw no end of rebuilds (for users here) because one or other target had changed, and because they were using --newuse rather than --changed-use ... the former is practically unusable because useflags change constantly as python versions are bumped, or made obsolete, and/or packages incur a use change. While its not likely that this will be the case with CPU_FLAGS_X86 I question the wisdom of causing further rebuilds for no apparent reason ... other than having a new subset of useflags what has been achieved?
best ... khay | The pros & cons of this are discussed in this thread: https://marc.info/?l=gentoo-dev&m=142123671715691&w=2
I'd highly recommend subscribing to this mailing list if you want be involved in such decisions, even as "just a user" your input there is taken into consideration |
|
Back to top |
|
|
limn l33t
Joined: 13 May 2005 Posts: 997
|
Posted: Sat Jan 31, 2015 1:56 am Post subject: |
|
|
Thank you for the link iamben.
The change then is to have a separate use variable that will have all the optimization flags for a given CPU that are or are ever likely to be included in an ebuild.
In this context running the script is preferred, rather than simply copying the currently existing USE flags. |
|
Back to top |
|
|
khayyam Watchman
Joined: 07 Jun 2012 Posts: 6227 Location: Room 101
|
Posted: Sat Jan 31, 2015 3:05 am Post subject: |
|
|
iamben ... that doesn't read like a "pros and cons" discussion to me, its a "yes, and you know something about hardware feature flags [...]" discussion. If that thread amounts to the argument for its adoption then I'd say its a poor one.
iamben wrote: | I'd highly recommend subscribing to this mailing list if you want be involved in such decisions, even as "just a user" your input there is taken into consideration |
Honestly, I've been thoroughly discouraged of the truth of such consideration by the number of times I've been presented with "if you don't like it ... show us an alternate implementation" (aka the "show us the code" argument). Its that same logic that seems to situate me as "just a user" ... implying, as it does, that gentoo proper is in fact the developers ... and not the community.
best ... khay |
|
Back to top |
|
|
The_Great_Sephiroth Veteran
Joined: 03 Oct 2014 Posts: 1602 Location: Fayetteville, NC, USA
|
Posted: Sat Jan 31, 2015 3:50 am Post subject: |
|
|
I just saw this on my laptop (and an Acer I am building) and set them. I did not realize I had sse3 (I had set ssse3 because I saw it) until running the script. I did the same as the other fellow. I appended CPU_FLAGS_X86 to USE and removed my CPU stuff from USE. This way once the shift is done, I simply stop appending the one variable to the other. I did have to rebuild ffmpeg to gain sse3 support. I do not mind this change and actually like it due to havign my CPU-specific stuff in one var and my system-wide flags in USE for my programs to build with. _________________ Ever picture systemd as what runs "The Borg"? |
|
Back to top |
|
|
iamben Apprentice
Joined: 10 May 2004 Posts: 278
|
Posted: Sat Jan 31, 2015 4:02 am Post subject: |
|
|
khayyam wrote: |
iamben ... that doesn't read like a "pros and cons" discussion to me, its a "yes, and you know something about hardware feature flags [...]" discussion. If that thread amounts to the argument for its adoption then I'd say its a poor one.
iamben wrote: | I'd highly recommend subscribing to this mailing list if you want be involved in such decisions, even as "just a user" your input there is taken into consideration |
Honestly, I've been thoroughly discouraged of the truth of such consideration by the number of times I've been presented with "if you don't like it ... show us an alternate implementation" (aka the "show us the code" argument). Its that same logic that seems to situate me as "just a user" ... implying, as it does, that gentoo proper is in fact the developers ... and not the community.
best ... khay | Please get involved as much as you can. Gentoo can't really be "for the community" if you aren't making your opinion known. I think this change makes good sense, everybody else seems to be pretty happy about it, besides the grumbles about any change being bad. |
|
Back to top |
|
|
mv Watchman
Joined: 20 Apr 2005 Posts: 6780
|
Posted: Sat Jan 31, 2015 9:24 am Post subject: |
|
|
khayyam wrote: | hello ... I honestly don't see what benefit this change brings, I mean, does it matter if its in USE or some other var if they are all essentially useflags. |
I can only speak for me. For me, it has several advantages:
First, I can now clearly see what are CPU-optimization flags, and what are flags which just add optional functionality: There were several flags like popcnt for which I even did not know about or was not sure what they really did; also some ambiguity is resolved (e.g. aes vs. aes-ni was very strange since one was processor optimization, the other functionality).
Second advantage: I build quite often packages for weak laptops on other machines - depending on where I am currently (my machines are 1000 miles away from each other). Now I can do this easily by just exporting CPU_FLAGS_X86 which will now just override the defaults in /etc/portage/make.conf.
Previously, I had to export USE, but this was not enough, since it would not override but just add to the other USE flags; so I had some wrapper script which exported (depending on the architecture where I am and for which I compile) things like USE='-sse3 -sse4_1 -ssse4_2 sse sse2'; it was quite some work to maintain this script if new flags were added or if I had to add a new machine (as source or destination). The main part of this script is now obsolete, finally.
Third advantages: It adds consistency. Now all system-specific things (CPU_FLAGS_X86, ALSA_CARDS, VIDEO_CARDS) have their own expand-USE which can be set more easily. |
|
Back to top |
|
|
krinn Watchman
Joined: 02 May 2003 Posts: 7470
|
Posted: Sat Jan 31, 2015 12:28 pm Post subject: |
|
|
Before:
To disable sse in a package: <CFLAGS="-nosse" USE="-sse" emerge package>
Now: <CFLAGS="-nosse" CPU_FLAGS_X86="" emerge package>
I don't see any benefits there, just that i should remember to watch out CPU_FLAGS_X86 now instead of USE
It would had been an improvement if CPU_FLAGS_X86 was autobuild by parsing current CFLAGS: to disable sse, a simple <CFLAGS="-nosse" emerge package">
It looks like what i'm calling a "wanking feature", because renaming a feature to endup doing the same as before is not some shiny new feature, just some mental masturbation. |
|
Back to top |
|
|
limn l33t
Joined: 13 May 2005 Posts: 997
|
Posted: Sat Jan 31, 2015 12:34 pm Post subject: |
|
|
I would have expected the news item to have the reason for the change.
Reading the item alone, it seemed just shuffling deck chairs. |
|
Back to top |
|
|
khayyam Watchman
Joined: 07 Jun 2012 Posts: 6227 Location: Room 101
|
Posted: Sat Jan 31, 2015 1:54 pm Post subject: |
|
|
iamben wrote: | Please get involved as much as you can. Gentoo can't really be "for the community" if you aren't making your opinion known. I think this change makes good sense, everybody else seems to be pretty happy about it, besides the grumbles about any change being bad. |
iamben ... yeah, grumbles.
khayyam: what exactly is the problem its intended to solve?
iamben: the pros & cons of this are discussed [here]
khayyam: that doesn't read like a "pros and cons" discussion to me [...] if that amounts to the argument for its adoption then I'd say its a poor one.
iamben: everybody else seems to be pretty happy about it ... oh, and please get involved as much as you can ... and stop your grumbling about (unqualified) change being bad.
You seem the illustrate exactly why "getting involved" is a no win proposition.
mv wrote: | I can only speak for me. For me, it has several advantages: |
@mv ... your use case seems to me marginal, and though it adds some benefit for you it came at the cost of package rebuilds for those who gain no particular benefit. Like the runscript => openrc-run rename, its a change whose justification seems to be entirely paltry and cosmetic but whose effects are disproportionate to the benefits brought by implementing them.
best ... khay |
|
Back to top |
|
|
iamben Apprentice
Joined: 10 May 2004 Posts: 278
|
Posted: Sat Jan 31, 2015 2:21 pm Post subject: |
|
|
You seem to be convinced that gentoo devs are out to ruin your life. I think they're doing a great job and I'm willing to give them the benefit of the doubt on changes like this.
The trick with package rebuilds is to let your CPU do most of the work... you don't have to interpret every package rebuild like a slap in the face from the maintainer. On my desktop install with 1400 packages, around 15 (fifteen!) needed a rebuild for some CPU flag change... not a big deal. |
|
Back to top |
|
|
229566 Tux's lil' helper
Joined: 16 Aug 2010 Posts: 127
|
Posted: Sat Jan 31, 2015 2:23 pm Post subject: |
|
|
krinn wrote: | It would had been an improvement if CPU_FLAGS_X86 was autobuild by parsing current CFLAGS |
+1 for that.
How come this wasn't done to begin with? Taking a look at what cpuinfo2cpuflags does, it looks like a straightforward task, the CPU specific USE flags are already separated and known to portage. |
|
Back to top |
|
|
iamben Apprentice
Joined: 10 May 2004 Posts: 278
|
Posted: Sat Jan 31, 2015 2:29 pm Post subject: |
|
|
CFLAGS to USE flag mapping isn't nearly as straightforward as cpuinfo to USE flag mapping |
|
Back to top |
|
|
|