| View previous topic :: View next topic |
| Author |
Message |
lars_the_bear
Tux's lil' helper
Joined: 05 Jun 2024
Posts: 136
|
 Posted: Thu Jun 20, 2024 9:00 pm Post subject: Java and ebuilds |
 |
|
Hi folks
I'm in the process of writing .ebuild files for software that I maintain. In general, I think I'm kind-of on top of this for C/C++, although there are still many things I don't understand.
For Java, though, it's a bit different.
I've been looking at the documentation concerning Java and Portage, but most of it looks a bit old (4-5 years). I was struck by two statements, however:
1. Packagers are encouraged to have the Java built from source
2. There's no built-in support for Maven or Gradle
These two statements seem pretty contradictory to me, since my experience is that Maven and Gradle pretty-much dominate the Java world now. I can't remember the last time I saw anything else used. To be fair, some long-standing projects like Tomcat still use Ant (I think), but they are the exception, rather than the rule. I use Maven; I don't much like it, but that's the way things are going.
So it seems to me that if I want to package my Java applications, I'm faced with this choice:
1. Build from source with Maven, using build scripts and yadda yadda yadda that I write myself. I can do this, so long as there actually is an implementation of Maven that I can specify as a build-time dependency (which I think there is), but it's a bit tedious. Or:
2. Don't build from source at all. Just download the compiled JARs from their repositories, and copy them into sensible places, along with the scripts, etc., needed to run them.
I'm curious about how other people are tackling this problem (if they are). Downloading compiled JARs seems hugely easier to me, and the arguments given for building Java from source don't seem particularly compelling to me.
Comments welcome.
BR, Lars. |
|
| Back to top |
|
 |
fedeliallalinea
Administrator
Joined: 08 Mar 2003
Posts: 31048
Location: here
|
| Posted: Fri Jun 21, 2024 4:54 am Post subject: |
|
|
The problem with new build systems is that, most of the time, they are not well thought out for offline use.
And even if they are you have to either write an ebuild for each dependency (see python packages) or have a long list of dependencies in SRC_URI and have an eclass compile them (see go packages).
There is an eclass written by Sakaki, a brilliant gentoo user who is no longer active, several years ago:
_________________
Questions are guaranteed in life; Answers aren't. |
|
| Back to top |
|
|
logrusx
Veteran
Joined: 22 Feb 2018
Posts: 1831
|
| Posted: Fri Jun 21, 2024 4:57 am Post subject: Re: Java and ebuilds |
|
|
Hello Lars,
| lars_the_bear wrote: | Hi folks
I'm in the process of writing .ebuild files for software that I maintain. In general, I think I'm kind-of on top of this for C/C++, although there are still many things I don't understand.
For Java, though, it's a bit different.
I've been looking at the documentation concerning Java and Portage, but most of it looks a bit old (4-5 years). I was struck by two statements, however:
1. Packagers are encouraged to have the Java built from source |
To me it doesn't make much sense particularly for Java, but it's in the spirit of Gentoo for all else.
| lars_the_bear wrote: | 2. There's no built-in support for Maven or Gradle
These two statements seem pretty contradictory to me, since my experience is that Maven and Gradle pretty-much dominate the Java world now. I can't remember the last time I saw anything else used. To be fair, some long-standing projects like Tomcat still use Ant (I think), but they are the exception, rather than the rule. I use Maven; I don't much like it, but that's the way things are going. |
Build tools for Java download random stuff and break network sandbox. With other build systems like the ones used by Go and Rust, those files are pre-downloaded and hosted somewhere, so ebuilds can download them and build offline. There's no such support for Maven or Gradle. If you can make them work offline, you can use them, but it's your responsibility.
| lars_the_bear wrote: | | So it seems to me that if I want to package my Java applications, I'm faced with this choice: |
First, let me ask you, are you sure you really need to package your application? Are you going to distribute it?
| lars_the_bear wrote: | | 1. Build from source with Maven, using build scripts and yadda yadda yadda that I write myself. I can do this, so long as there actually is an implementation of Maven that I can specify as a build-time dependency (which I think there is), but it's a bit tedious. |
As I already said, those things break network sandbox and I haven't found a way to make them work offline, though there may be.
| lars_the_bear wrote: | Or:
2. Don't build from source at all. Just download the compiled JARs from their repositories, and copy them into sensible places, along with the scripts, etc., needed to run them. |
That's what I do with Briss which I'm running it from the command line from time to time. I also have an .apps dir in my home dir where I keep things not packages like Eclipse, IntelliJ et.c.
And here's what I do with the one Java application I'm maintaining an ebuild for: https://github.com/logrusx/gentoo/tree/master/app-misc/freeplane-bin
Best Regards,
Georgi
Last edited by logrusx on Fri Jun 21, 2024 5:54 am; edited 2 times in total |
|
| Back to top |
|
|
charles17
Advocate
Joined: 02 Mar 2008
Posts: 3675
|
|
| Back to top |
|
|
logrusx
Veteran
Joined: 22 Feb 2018
Posts: 1831
|
|
| Back to top |
|
|
lars_the_bear
Tux's lil' helper
Joined: 05 Jun 2024
Posts: 136
|
| Posted: Fri Jun 21, 2024 1:14 pm Post subject: Re: Java and ebuilds |
|
|
| logrusx wrote: |
First, let me ask you, are you sure you really need to package your application? Are you going to distribute it?
|
That's a good question. I"m doing this for my own convenience, not for distribution. So, in the end, I'll do what works best for me; but I'm just curious whether there's a nice way to do it that I just overlooked. The answer seems to be that there isn't, particularly if you use Maven.
This isn't a problem specific to Gentoo. So far as I know, no Linux distribution has a joined-up way of dealing with Java applications. For my commercial Java work, the distribution is just a zipfile full of JAR files and scripts -- that's what users expect to get. There doesn't seem to be any user expectation, that Java applications will share JARs. In practice, that means that every application is about 70% a duplicate of every other Java application, because many JARs are ubiquitous.
Fortunately, storage is cheap these days.
BR, Lars. |
|
| Back to top |
|
|
Goverp
Advocate
Joined: 07 Mar 2007
Posts: 2043
|
| Posted: Fri Jun 21, 2024 5:59 pm Post subject: |
|
|
Perhaps think of your bunch of jar files as a Snap. It may lead to duplication, but it does provide a known Java environment for the application.
_________________
Greybeard |
|
| Back to top |
|
|
lars_the_bear
Tux's lil' helper
Joined: 05 Jun 2024
Posts: 136
|
| Posted: Fri Jun 21, 2024 6:24 pm Post subject: |
|
|
| Goverp wrote: | | Perhaps think of your bunch of jar files as a Snap. It may lead to duplication, but it does provide a known Java environment for the application. |
Yeah, exactly. If users can tolerate the profligacy of snap/flatpak, they should be able to cope with a bit of JAR duplication.
To be honest, I tend to think that the simplest way to distribute open-source Java applications is just to provide users with a Maven pom.xml and a script that executes `mvn exec:java`. Maven takes care of all the dependency management and downloading, at least on a per-user level. And everything gets cached locally, so this approach gets more effective the more you use it.
It seems to me that trying to run Maven within Portage amounts to putting one dependency management system inside another. In a way, Maven is Portage for Java.
It turns out that it's pretty easy, even for me, to write an ebuild file that downloads some JARs from GitHub and generates a script to run them, so that's what I'll use for my own purposes.
BR, Lars. |
|
| Back to top |
|
|
charles17
Advocate
Joined: 02 Mar 2008
Posts: 3675
|
| Posted: Sat Jun 22, 2024 6:15 am Post subject: |
|
|
| lars_the_bear wrote: | | Goverp wrote: | | Perhaps think of your bunch of jar files as a Snap. It may lead to duplication, but it does provide a known Java environment for the application. |
[...]
To be honest, I tend to think that the simplest way to distribute open-source Java applications is just to provide users with a Maven pom.xml [...] |
In many if not most cases pom.xml can be easily "translated" into ebuild. |
|
| Back to top |
|
|
lars_the_bear
Tux's lil' helper
Joined: 05 Jun 2024
Posts: 136
|
| Posted: Sat Jun 22, 2024 7:46 am Post subject: |
|
|
| charles17 wrote: | | In many if not most cases pom.xml can be easily "translated" into ebuild. |
If there's a systematic way to do that, I'd be interested in seeing it. Specifically, I'd be interested in knowing how the dependency-fetching part of Maven can be implemented easily in some other way.
I mean, I guess I could do 'mvn dependency:tree' to enumerate all the transitive dependencies and then use a script to parse the list and fetch them all one-by-one and store them. Then another script to build another script to launch Java with all the relevant class search path arguments. But I'd still be getting binary JARs for most of the final result, even if compiled part of it.
In the end, though, I'm not sure what the point would be. I appreciate that Gentoo favours building from source, but I just can't see anybody building all the Java library JARs from source. Certainly that wouldn't happen if you used Maven as your build platform. It assumes that all the dependencies are binary, and don't need to be built. So even if Maven could be integrated with Portage, you wouldn't really be 'building from source' in a meaningful way.
Nobody distributes Java applications in source form, so far as I know. With C/C++, sometimes you have to. I maintain a small number of C++ libraries that have to go to customers as source, because we don't have the resources to build and maintain binaries for every platform out there. But Java? Never. There's just no point.
BR, Lars. |
|
| Back to top |
|
|
charles17
Advocate
Joined: 02 Mar 2008
Posts: 3675
|
| Posted: Sat Jun 22, 2024 8:08 am Post subject: |
|
|
| lars_the_bear wrote: | | charles17 wrote: | | In many if not most cases pom.xml can be easily "translated" into ebuild. |
[...] a script to parse the list and fetch them all one-by-one and store them. [...] |
Not store them. The usual way is to package them, as done e.g. for dev-java/bnd-ant.
Some random shit packages however have ugly unpackageable dpendencies, see https://github.com/gentoo/gentoo/pull/37119 |
|
| Back to top |
|
|
lars_the_bear
Tux's lil' helper
Joined: 05 Jun 2024
Posts: 136
|
| Posted: Sat Jun 22, 2024 9:26 am Post subject: |
|
|
| charles17 wrote: |
Not store them. The usual way is to package them, as done e.g. for dev-java/bnd-ant.
|
OK; but the whole point -- at least a significant point -- of Maven is that it does store dependencies. Of course you can package them as well, but my $HOME/.m2 has 20Gb of JARs in it. I suspect that it's Maven's ability to download and store dependencies that killed off Ant.
BR, Lars. |
|
| Back to top |
|
|
logrusx
Veteran
Joined: 22 Feb 2018
Posts: 1831
|
| Posted: Sat Jun 22, 2024 9:55 am Post subject: |
|
|
| lars_the_bear wrote: | | charles17 wrote: |
Not store them. The usual way is to package them, as done e.g. for dev-java/bnd-ant.
|
OK; but the whole point -- at least a significant point -- of Maven is that it does store dependencies. Of course you can package them as well, but my $HOME/.m2 has 20Gb of JARs in it. I suspect that it's Maven's ability to download and store dependencies that killed off Ant.
BR, Lars. |
It's not a good idea to not package the dependencies and a very bad one to count on Maven. Nowadays there is jlink. I'm only saying this because we're already well off topic.
Best Regards,
Georgi |
|
| Back to top |
|
|
lars_the_bear
Tux's lil' helper
Joined: 05 Jun 2024
Posts: 136
|
| Posted: Sat Jun 22, 2024 12:47 pm Post subject: |
|
|
| logrusx wrote: |
It's not a good idea to not package the dependencies and a very bad one to count on Maven. Nowadays there is jlink.
|
Can you name an up-to-date, substantial, commercial-grade Java project that is maintained using Maven or Gradle? Tomcat is the only one I can think of.
That's not an endorsement -- I hate Maven. But I use it every day, because that's where Java is today.
From a Gentoo perspective, the maintainers could just quietly ignore it, and hope it gets replaced by something more Gentoo-friendly. Nothing has a very long lifespan in the Java world.
BR, Lars. |
|
| Back to top |
|
|
logrusx
Veteran
Joined: 22 Feb 2018
Posts: 1831
|
| Posted: Sat Jun 22, 2024 6:36 pm Post subject: |
|
|
I meant not to count on maven to deliver the client whatever is necessary. That's a recipe for disaster. The deliverables should be complete by themselves.
Best Regards,
Georgi |
|
| Back to top |
|
|
lars_the_bear
Tux's lil' helper
Joined: 05 Jun 2024
Posts: 136
|
| Posted: Sun Jun 23, 2024 7:15 am Post subject: |
|
|
| logrusx wrote: | I meant not to count on maven to deliver the client whatever is necessary. That's a recipe for disaster. The deliverables should be complete by themselves.
|
That's certainly the attitude taken by most organizations that distribute Java applications, including my own. It's just very inefficient, both in the amount of duplication between applications, and the amount of work that has to be done to distribute a trivial patch. The patching is a particular problem -- I have to oversee the distribution of patches to hundreds of customers, and make sure they know how to install them without breaking their systems. And then fix their systems when they don't read the instructions.
I don't think anybody is worried about the inefficiency these days, given how cheap storage is. But maintenance of Java-based applications in production continues to be a drag.
I only mention this because the difficulties that the Gentoo maintainers raise about handling Java are just a reflection of the mess in the IT industry as a whole.
BR, Lars. |
|
| Back to top |
|
|
Goverp
Advocate
Joined: 07 Mar 2007
Posts: 2043
|
| Posted: Sun Jun 23, 2024 11:42 am Post subject: |
|
|
Does this imply a need for a shared jar library (if that's not too recursive a concept)? And similar for Rust crates, whatever packages Go uses, etc. sort of /lib/java/foo for each component foo? TeX has a similar problem, and part of the solution is the kpathsea tool which uses config files to search a vast library tree built according to the TeX Directory Standard (TDS). Or does such a shared library for Java already exist, and I just haven't noticed. I guess emerge (or maybe jmerge) would download a binary jar or try to compile from source depending on some config files in /etc/portage.
_________________
Greybeard |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
