| View previous topic :: View next topic |
| Author |
Message |
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3150
|
 Posted: Wed Aug 10, 2022 2:08 pm Post subject: LE got RUSTy... How is this a thing? |
 |
|
Apparently, I can't update my VPS because I don't have 11GB of free disk space for rust (hard-masked rust to find the culprit)
| Code: | # emerge -avuDN @world
These are the packages that would be merged, in order:
Calculating dependencies... done!
!!! All ebuilds that could satisfy "virtual/rust" have been masked.
!!! One of the following masked packages is required to complete your request:
- virtual/rust-1.62.1::gentoo (masked by: package.mask)
/etc/portage/package.mask/manual:
#dev-lang/rust
- virtual/rust-1.62.0::gentoo (masked by: package.mask, ~amd64 keyword)
- virtual/rust-1.61.0::gentoo (masked by: package.mask, ~amd64 keyword)
- virtual/rust-1.60.0::gentoo (masked by: package.mask)
- virtual/rust-1.59.0::gentoo (masked by: package.mask)
(dependency required by "dev-python/setuptools-rust-1.4.1::gentoo[-test]" [ebuild])
(dependency required by "dev-python/cryptography-37.0.4::gentoo" [ebuild])
(dependency required by "app-crypt/acme-1.29.0::gentoo" [ebuild])
(dependency required by "app-crypt/certbot-1.29.0::gentoo" [ebuild])
(dependency required by "@selected" [set])
(dependency required by "@world" [argument])
For more information, see the MASKED PACKAGES section in the emerge
man page or refer to the Gentoo Handbook.
|
I know, I know, bin-rust, local overlay, blah, blah, blah... Whatever. I'll handle it in _some_ way.
Just felt like complaining about something as simple as certbot inviting rust to my system. Pun absolutely intended. |
|
| Back to top |
|
 |
Hu
Administrator
Joined: 06 Mar 2007
Posts: 21709
|
| Posted: Wed Aug 10, 2022 2:49 pm Post subject: |
|
|
| This is not specifically the fault of Let's Encrypt / certbot. certbot depends on acme, which depends on cryptography. There was a big blow-up last year when the dev-python/cryptography project added a hard dependency on rust. If I recall correctly, some consuming projects made noise about moving off cryptography in response. The cryptography project "relented" and discussed putting off requiring Rust for most of a year, asserting that such would allow all the consumers to adjust to it. This seems excessively optimistic, since there are multiple architectures on which pre-Rust cryptography ran fine, but where Rust itself does not work and had no obvious path to changing that. See https://lwn.net/Articles/845535/ and https://lwn.net/Articles/847736/ for some places where this was discussed. There were likely others. |
|
| Back to top |
|
|
tld
Veteran
Joined: 09 Dec 2003
Posts: 1816
|
| Posted: Wed Aug 10, 2022 4:33 pm Post subject: |
|
|
Holy crap. I totally missed that one. So looking in that first link, is that to say that eventually portage itself will have to require at least rust-bin? Gentoo having to drop a whole list of architectures around that one is horrific frankly.
So how long until the rust folks decide to drop support for 32-bin x86, and we loose that as well? You know, I've NEVER been a fan of python in general, and much less so when it comes to rust. This crap confirms why to both of those.
Tom |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 21709
|
| Posted: Wed Aug 10, 2022 5:09 pm Post subject: |
|
|
Since Rust is a huge fan of static linking, Gentoo could have a cryptography-bin package, where someone with rust-bin prebuilds dev-python/cryptography and uploads it for Portage to consume. That's fairly nasty, but it would avoid the bootstrap loop on systems where Rust works. Alternately, we could just make the Rust curlpipesh a mandatory prerequisite to installing Portage, after which you can use Portage to manage future installations of rustc.  None of that helps people who use architectures where Rust doesn't work, though.
Is it really fair to say that Rust supports 32-bit x86 now? Stock Rust doesn't work on pre-SSE2 x86 chips, and from what I've read of the adventures of people trying to make it work, it's quite painful to get a working rustc for those systems.
I suggest that you not tar the Python project and ecosystem over the decisions of one popular Python library. It's possible that dev-lang/python will eventually have a dev-lang/rust-bin dependency, but so far, that is not the case, and I am not aware of any plans for it to become so in the definite future. |
|
| Back to top |
|
|
pa4wdh
l33t
Joined: 16 Dec 2005
Posts: 815
|
| Posted: Wed Aug 10, 2022 5:23 pm Post subject: |
|
|
I noticed something similar with a small python project i was working on which depends on cryptography and is intended to be run on an old raspberry pi.
Even though i use a binary distro so installing rust itself is not a problem there, there isn't enough memory to compile cryptography (which is done when installing it via pip in a virtual environment) because of the memory requirements for running the rust compiler. The solution? Get a newer pi ... good luck these days  .
I'm actually surprised that i've never heard about cryptography ports without the rust dependency, or did i miss something?
_________________
The gentoo way of bringing peace to the world:
USE="-war" emerge --newuse @world
My shared code repository: https://code.pa4wdh.nl.eu.org
Music, Free as in Freedom: https://www.jamendo.com |
|
| Back to top |
|
|
tld
Veteran
Joined: 09 Dec 2003
Posts: 1816
|
| Posted: Wed Aug 10, 2022 5:38 pm Post subject: |
|
|
| Hu wrote: | | Is it really fair to say that Rust supports 32-bit x86 now? Stock Rust doesn't work on pre-SSE2 x86 chips, and from what I've read of the adventures of people trying to make it work, it's quite painful to get a working rustc for those systems. |
Interesting. My two x86 systems are post-SSE2 at least, and there's certainly a rust-bin available for them now I see. | Hu wrote: | | I suggest that you not tar the Python project and ecosystem over the decisions of one popular Python library. |
Yea, I know. I think part of that comes from the fact that I'm one of those in the camp that has never, and will never warm up to the use of significant white space . | Hu wrote: | | It's possible that dev-lang/python will eventually have a dev-lang/rust-bin dependency, but so far, that is not the case, and I am not aware of any plans for it to become so in the definite future. |
That's good to know. I'm just starting to really really hate the way everything seems to be going in the tech world in general. I have updates running right now. Because of my work I need to use Thunderbird which I probably wouldn't be using otherwise, and seeing the portage update it's absurd requirements like nodejs just plain pisses me off. A growing amount of that BS seems to come from the Mozilla direction too...like rust itself.
Tom |
|
| Back to top |
|
|
pa4wdh
l33t
Joined: 16 Dec 2005
Posts: 815
|
| Posted: Wed Aug 10, 2022 6:11 pm Post subject: |
|
|
@szatox:
I personally use acme.sh (app-crypt/acme-sh) to update my LE certs, it might be worth a try. Since it's just a bunch of shell scripts i actually haven't installed it via portage but did a manual install under /opt.
_________________
The gentoo way of bringing peace to the world:
USE="-war" emerge --newuse @world
My shared code repository: https://code.pa4wdh.nl.eu.org
Music, Free as in Freedom: https://www.jamendo.com |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3150
|
| Posted: Wed Aug 10, 2022 6:26 pm Post subject: |
|
|
Well, those links were an interesting read for sure.
| Quote: | Yea, I know. I think part of that comes from the fact that I'm one of those in the camp that has never, and will never warm up to the use of significant white space .
|
lol, yeah, I dislike this one too.
But the thing that really pissed me off was when python tried to "help" me by doing things I didn't ask for but only when I wasn't looking.
Literally... Dumping the output to a file would magically enable UTF conversion to ANSI or whatever was considered "safe" and then crash the whole thing.
It was a simple program I wrote. And no, this behaviour was NOT what I unwittingly told it to do. I had to essentially bug that python's feature to make the whole program work.
A bad practice, poorly implemented, for no reason. As far as I'm concerned, kill it with fire. |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20067
|
| Posted: Wed Aug 10, 2022 9:52 pm Post subject: |
|
|
| Hu wrote: | | I suggest that you not tar the Python project and ecosystem over the decisions of one popular Python library. |
It is unfortunate, but when one popular library becomes the proverbial tail wagging the dog, the ecosystem seems to be the problem. If the ecosystem is unable or unwilling to extricate itself from the "means well bad actor," then the ecosystem isn't really able to throw up its hands and say "don't blame us."
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
Ionen
Developer
Joined: 06 Dec 2018
Posts: 2727
|
| Posted: Thu Aug 11, 2022 4:26 am Post subject: |
|
|
Gentoo been removing (or making optional) the cryptography dep wherever it's possible because of that (may still be some unhandled cases, but generally), portage itself used to indirectly depend on cryptography too but been done away and "most" systems are okay without it (until you need something that does anyway, personally don't have it installed albeit I still need rust for other things even on my headless server anyway).
The old non-rust cryptography finally got removed recently (an unmaintained crypto package is not such a great thing to have on top of things slowly starting to require the newer one to work), and there was a short dev ML post to warn about it:
https://archives.gentoo.org/gentoo-dev/message/19bd6c2f413cf21cb25b67aaf3aa8107
Trying to restore that would just be a short term solution, if want to avoid it (notably on arches without rust support at all), look into alternatives or porting packages to not need it upstream. |
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 3357
Location: Rasi, Finland
|
| Posted: Thu Aug 11, 2022 7:07 am Post subject: |
|
|
I want to make sure I'm understanding the situation here correctly and the problem with rust on other platforms.
So...Rust takes huge amounts of resources and the compiler, rustc, needs to be compiled with rustc. The big problem is that you cannot run rustc on any "obscure" architectures, like MIPS or Risc-v. Precompiled rust programs aren't that problematic. In fact those are almost always statically linked.The problem with dev-python/cryptography is that it requires rust instead of X nowdays because of performance gains or security or both. Fill in and/or correct the text above, thank you.
Lastly... I'd imagine doing cross compiling of rust programs is a PITA. Doing it on a VM might be impossible for certain architectures?
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3150
|
| Posted: Thu Aug 11, 2022 10:24 am Post subject: |
|
|
| Quote: | | I personally use acme.sh (app-crypt/acme-sh) to update my LE certs, it might be worth a try. Since it's just a bunch of shell scripts i actually haven't installed it via portage but did a manual install under /opt. |
I will. Thanks.
Need a few more hours to complete the big, 3-month update first though.
| Quote: | | Gentoo been removing (or making optional) the cryptography dep wherever it's possible because of that (may still be some unhandled cases, but generally), portage itself used to indirectly depend on cryptography too but been done away and "most" systems are okay without it |
That's a good news
| Quote: | | Trying to restore that would just be a short term solution |
Yeah, well, the short-term solution I went for was changing python targets an single target to 3_9 (effectively blocking the default 3_10). This made emerge update everything except for
dev-python/cryptography:0
dev-python/pyopenssl:0
Good enough for this week.
| Quote: | So...
Rust takes huge amounts of resources |
Yes. And THE THING can be just as well be done with openssl instead, so...
| Quote: | | and the compiler, rustc, needs to be compiled with rustc |
I don't think this one is a bad thing. I mean, it does make sense for a compiler to be self-hosted, just like C compilers tend to be written in C, so stage3 tarball comes with gcc binaries.
Yes, it is a problem for "obscure" AKA unsupported architectures. This is a second example of chicken and egg problem within 2 lines, unfortunately this one seems to be a bit more difficult to solve.
Still, rust does take a lot of resources.... And at this point I'd rather do without it. |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 21709
|
| Posted: Thu Aug 11, 2022 11:34 am Post subject: |
|
|
| Zucca wrote: | Rust takes huge amounts of resources and the compiler, rustc, needs to be compiled with rustc. |
Yes, and to make it worse, generally you need the most previous recently released rustc to compile the current one, so you cannot skip versions. With many other languages, you can skip a version or sometimes even several versions, when you decide to update. That lets you put off or entirely skip some builds, which can be desirable on resource-limited systems. Rust is still changing so rapidly that rustc officially requires the most previous release to build it. Sometimes something older will work, but Rust upstream does not consider it a bug if rustc 1.X cannot compile rustc 1.(X+2). Even this might not be that bad if they had a slower release cadence. However, X increments about every 6 weeks:I have not monitored rust-using packages closely, but I would not be surprised if many of them exhibit the same problem: a package released in July requires a rustc from June or later, rather than "any rustc from the last 12 months." | Zucca wrote: | The big problem is that you cannot run rustc on any "obscure" architectures, like MIPS or Risc-v. |
Yes, but it's slightly worse than that. It is not that you cannot run rustc on those architectures. It is that you cannot run rustc for those architectures. Therefore, as I understand it, you simply cannot build a current release of dev-python/cryptography for the "obscure" architectures because to do so requires a rustc that does not exist, and for which no one has announced serious plans to bring into existence. | Zucca wrote: | Precompiled rust programs aren't that problematic. In fact those are almost always statically linked. |
Yes. I would argue that this form of static linking is itself problematic, but that problem is outside the scope of the issues raised in this thread with this Python package. | Zucca wrote: | The problem with dev-python/cryptography is that it requires rust instead of X nowdays because of performance gains or security or both. |
I believe the justification used was that Rust's "memory safety" features made it so much harder to make security mistakes that they needed to use Rust instead of trusting the programmers not to make those mistakes in a memory unsafe language like C. In principle, memory safety is a great thing. I just wish the language was more approachable. | Zucca wrote: | | Lastly... I'd imagine doing cross compiling of rust programs is a PITA. Doing it on a VM might be impossible for certain architectures? |
I believe I've read that rust is actually quite good at cross-compiling - provided you stay within the limited subset of architectures on which rust works at all. |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3150
|
| Posted: Thu Aug 11, 2022 11:45 am Post subject: |
|
|
| Quote: | | I believe the justification used was that Rust's "memory safety" features made it so much harder to make security mistakes that they needed to use Rust instead of trusting the programmers not to make those mistakes in a memory unsafe language like C. |
It was the justification...
But since you mentioned it, it comes with an implication that they trust the team behind Rust more than they tust the team behind cryptography. |
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 3357
Location: Rasi, Finland
|
| Posted: Thu Aug 11, 2022 8:17 pm Post subject: |
|
|
Thanks Hu.
| Hu wrote: | | In principle, memory safety is a great thing. I just wish the language was more approachable. |
Exactly my words too.
I have thought to start learning rust many times, but after reminding myself about the "crate hell" when creating ebuilds, I simply start looking elswere... Like go... oh no. Maybe not that either... ;) Fortran! Nope.
One language that got me interested is D. Just hadn't had time to try it yet. Also Zig. Oh well...
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3150
|
| Posted: Mon Aug 22, 2022 2:00 pm Post subject: |
|
|
Alright, time to wrap it up. Does anyone know if it's possible to create a certificate with acme.sh without email address?
Certbot understands --register-unsafely-without-email, is there some undocumented acme.sh counterpart to this?
The script it almost 8k lines long, so I'd rather avoid fully analyzing it... |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 21709
|
| Posted: Mon Aug 22, 2022 2:58 pm Post subject: |
|
|
I have never used that script before, but I looked at it now. It looks like you can omit the e-mail parameter, and it will work except for this block: | Code: | if [ "$ACME_DIRECTORY" = "$CA_ZEROSSL" ]; then
if [ -z "$_eab_id" ] || [ -z "$_eab_hmac_key" ]; then
_info "No EAB credentials found for ZeroSSL, let's get one"
if [ -z "$_email" ]; then
_info "$(__green "$PROJECT_NAME is using ZeroSSL as default CA now.")"
_info "$(__green "Please update your account with an email address first.")" |
Presumably, if you avoid using ZeroSSL, then you can avoid providing an e-mail address. |
|
| Back to top |
|
|
pa4wdh
l33t
Joined: 16 Dec 2005
Posts: 815
|
| Posted: Mon Aug 22, 2022 4:53 pm Post subject: |
|
|
I'm using acme.sh and i don't remember giving an e-mail address to set up my letsencrypt account. I have to admit that was ~4 years ago, so things could have changed since then.
_________________
The gentoo way of bringing peace to the world:
USE="-war" emerge --newuse @world
My shared code repository: https://code.pa4wdh.nl.eu.org
Music, Free as in Freedom: https://www.jamendo.com |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3150
|
| Posted: Mon Aug 22, 2022 8:04 pm Post subject: |
|
|
Huh... Yes, indeed, the default CA, zerossl, was the thing that asked for email.
Thanks
| Quote: | | acme.sh --issue --standalone --no-cron --httpport 81 --local-address 127.0.0.1 --server "https://acme-v02.api.letsencrypt.org/directory" -d <domain name> |
simply created a certificate, no questions asked.
Standalone and non-standard port because I have a reverse proxy in front of several web apps; adding another backend selected by path is super easy and makes webroot look like a hackish workaround it is.
Gotta add some deploy hook and I can bid certbot (and cryptography) adieu. Cool! |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3150
|
| Posted: Sun Oct 02, 2022 9:23 pm Post subject: |
|
|
Well... After a "Short Break"™ I came back trying to actually migrate to acme.sh and man, I kinda feel like I should rewrite this thing.
Why? Because my setup relies on hooks to reload the affected services.
Hooks in acme.sh are horribly inconsistent, that's one annoying thing. Options naming hooks sometimes expect a full path to an executable, and sometimes the base name of a function library file in some predefined location.
Names of the hooks are surprising and invoked in a bizarre order. Like, why the hell is it post→renew→deploy rather than renew→deploy→repeat_for_every_domain→post_only_ran_once?
One would think that since post runs so early in the chain, the notify hook would run at the end. Nope, it wasn't invoked even once, meaning it probably serves some other purpose.
Just my little rant.
I wonder though, if I actually opted to rewrite it, would anyone be interested in it?
I imagine only http challenge in standalone mode, because of how easy it is to use with _any_ service that might want to use ssl. (If the port is used by a http server, just proxy the request on path match; there is no reason to temporarily shut it down)
Also, I'm not doing a dozen of fallbacks at every single step just to make sure it works on a system with weirdosh, wtfsed and awkD that doesn't have more common tools installed. |
|
| Back to top |
|
|
pa4wdh
l33t
Joined: 16 Dec 2005
Posts: 815
|
| Posted: Mon Oct 03, 2022 3:38 pm Post subject: |
|
|
I'm sorry to hear you had that much trouble with the hooks. I'm not using them so i didn't know.
As for your re-write idea, it wouldn't be that interesting for me. First because i don't experience the problems you ran in to, and secondly i use DNS to verify with letsencrypt, so if that isn't supported it doesn't fit my use case.
_________________
The gentoo way of bringing peace to the world:
USE="-war" emerge --newuse @world
My shared code repository: https://code.pa4wdh.nl.eu.org
Music, Free as in Freedom: https://www.jamendo.com |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3150
|
| Posted: Mon Oct 10, 2022 10:16 pm Post subject: |
|
|
I found another bash implementation of acme client: dehydrated.
It is in portage, and it triggers a whole lot of hooks from a single library script. (no standalone mode, but there is a hook to prepare a challenge and another one to clean it up, so ddns goes brrr)
The only problem is it does not seem to store domains' configs anywhere, which forces quite a bit of unnecessary complexity into deploy hook, since the same script may need to behave differently depending on the domain it is invoked for.
Well, I think I can live with that (for now) so I'll call it good enough. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Gentoo Chat
