| View previous topic :: View next topic |
| Author |
Message |
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20407
|
 Posted: Mon Sep 23, 2024 6:22 am Post subject: improving Makefile object handling |
 |
|
First the general question with some details, then the specific question and the rest of the details.
What is a better way to compile with all of the object files given the current structure and Makefile's handling of foobarbaz and fib objects?
The releveant Makefile snippet (whole file at the end): | Code: | # EXE is the target 'main'.
# FBB_OBJ and FIB_OBJ generate their respective object files.
${EXE}: ${FBB_OBJ} ${FIB_OBJ} ${SRC}
${CC} ${CFLAGS} ${INC} -o ${EXE} ${SRC} ${FBB_OBJ} ${FIB_OBJ}
$(FBB_OBJ): obj/%.o: foobarbaz/%.c
${CC} ${CFLAGS} ${INC} -c $^ -o $@
$(FIB_OBJ): obj/%.o: fib/%.c
${CC} ${CFLAGS} ${INC} -c $^ -o $@ |
The specific question is in handling target EXE compilation when adding more *_OBJ dependencies. Yes I can add A_OBJ, ... ZZ_OBJ, but at some point that seems like the wrong solution.
The real project is for learning, so I don't know how many I'll have. The purpose of breaking them out into their own directories is for code organization and readability.
This currently works. Here is the output for clarity: | Code: | $ ./main
[foobarbaz]
Fibonacci:
1 2 3 5 8 13 21 34 55 89 |
Given this layout: | Code: | .
├── fib Next number in fibonacci series, from 1.
│ └── fib.c Static variables retain previous results.
├── foobarbaz Combines multiple strings, returns a struct string.
│ ├── bar.c Returns "bar".
│ ├── bar.h Declares function bar() for foobarbaz().
│ ├── baz.c Returns "baz".
│ ├── baz.h Declares function baz() for foobarbaz().
│ └── foobarbaz.c Combines foo, bar, baz, into a struct string.
├── inc "Public" api.
│ ├── fib.h Declares function next_in_fib() for main().
│ └── foobarbaz.h Declares function foobarbaz() for main().
├── main Executable
├── Makefile
├── obj
│ ├── bar.o
│ ├── baz.o
│ ├── fib.o
│ └── foobarbaz.o
└── src
└── main.c Calls foobarbaz() and loops next_in_fib(). |
The makefile: | Code: | CC := /usr/lib/llvm/18/bin/clang
C_STANDARD := c11
CFLAGS := -Wall -Wextra -std=$(C_STANDARD) -pedantic
INC_PATHS := ./inc
INC := $(addprefix -I,${INC_PATHS})
EXE := main
SRC := src/${EXE}.c
FBB_SRC := $(wildcard foobarbaz/*.c)
FBB_OBJ := $(patsubst foobarbaz/%.c, obj/%.o, $(FBB_SRC))
FIB_SRC := $(wildcard fib/*.c)
FIB_OBJ := $(patsubst fib/%.c, obj/%.o, $(FIB_SRC))
${EXE}: ${FBB_OBJ} ${FIB_OBJ} ${SRC}
${CC} ${CFLAGS} ${INC} -o ${EXE} ${SRC} ${FBB_OBJ} ${FIB_OBJ}
$(FBB_OBJ): obj/%.o: foobarbaz/%.c
${CC} ${CFLAGS} ${INC} -c $^ -o $@
$(FIB_OBJ): obj/%.o: fib/%.c
${CC} ${CFLAGS} ${INC} -c $^ -o $@
.PHONY: clean showvar
clean:
ifneq (,$(wildcard obj/*.o ${EXE}))
rm $(wildcard obj/*.o ${EXE})
endif |
I've spent about a week to get this far, pretty much from close to zero knowledge of how to use a Makefile. Around half of that time was getting the Makefile to work with something in it's own directory separate from src/, so I'm a bit slow on thoughts about it right now.
My first thought is to simply find a way to combine all of the *_OBJ variables into one used for the compilation. Something like: | Code: | | ALL_OBJ := ${A_OBJ} ... ${ZZ_OBJ} |
But before I try that, I'm wondering if I'm going down a wrong path and should be doing something very different (Makefile, directory structure, something else).
Hopefully that makes sense and I didn't omit something important.
Thanks!
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
 |
eschwartz
Developer
Joined: 29 Oct 2023
Posts: 166
|
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 3634
Location: Rasi, Finland
|
| Posted: Mon Sep 23, 2024 8:10 pm Post subject: |
|
|
Very informative site. Thanks!
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
eschwartz
Developer
Joined: 29 Oct 2023
Posts: 166
|
| Posted: Mon Sep 23, 2024 8:13 pm Post subject: |
|
|
Yeah, it's by the author of GNU Make so as you can imagine it knows all the tricks.  |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20407
|
| Posted: Tue Sep 24, 2024 5:29 am Post subject: |
|
|
Thanks. I'm skimming it now and will read it more closely tomorrow.
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
Zucca
Moderator
Joined: 14 Jun 2007
Posts: 3634
Location: Rasi, Finland
|
| Posted: Tue Sep 24, 2024 6:45 am Post subject: |
|
|
| eschwartz wrote: | | Yeah, it's by the author of GNU Make |
... explains...
I tend to plan my little projects so that users don't need to install too much dependencies. That being said... I haven't tested or ran any other makes. So the question is: Is it fair to assume all Linuxes use gmake as /usr/bin/make? And how about BSDs?
_________________
..: Zucca :..
Gentoo IRC channels reside on Libera.Chat.
--
| Quote: | | I am NaN! I am a man! |
|
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20407
|
| Posted: Tue Sep 24, 2024 3:44 pm Post subject: |
|
|
I chuckled at that one because it seems like MS saying "use MS".
Regarding vpath, that's a lot to absorb. It seems I may be doing some things "I shouldn't be doing" in the Makefile. And to add, I don't want to build object files in the same directory as the source files. I'm not sure why that's the recommended approach. When I compile on the command line, there are zero issues, so maybe it's a problem with make.
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22442
|
| Posted: Tue Sep 24, 2024 4:19 pm Post subject: |
|
|
In my opinion, the Makefile is easier to write, and the consequences easier to understand, if you use matched directory structure. As I read the Makefile earlier in this thread, all objects end up in obj/, regardless of their source directory. I suggest instead having:
Sources:- src/a.c
- fib/b.c
- foobarbaz/c.c
Outputs:- obj/src/a.o
- obj/fib/b.o
- obj/foobarbaz/c.o
This has two advantages. First, if you had src/a.c and fib/a.c, then the resulting a.o will not overwrite each other. As I read the existing Makefile, they would collide. Second, it becomes easier to wipe out all object files corresponding to a given source directory: rm -r obj/fib/.
As regards BSDs, my understanding is that /usr/bin/make is not necessarily GNU make on BSD systems, but that if /usr/bin/gmake exists, then that will be GNU make. I could be wrong. I have not looked in a long time. |
|
| Back to top |
|
|
dmpogo
Advocate
Joined: 02 Sep 2004
Posts: 3388
Location: Canada
|
| Posted: Tue Sep 24, 2024 4:23 pm Post subject: |
|
|
| Do you need EXE to be dependent on SRC ? I would write EXE line to just assembling objects and necessary libraries into executable. Though you will have to make src/main.o somewhere else |
|
| Back to top |
|
|
eschwartz
Developer
Joined: 29 Oct 2023
Posts: 166
|
| Posted: Tue Sep 24, 2024 4:35 pm Post subject: |
|
|
| Zucca wrote: | | eschwartz wrote: | | Yeah, it's by the author of GNU Make |
... explains...
I tend to plan my little projects so that users don't need to install too much dependencies. That being said... I haven't tested or ran any other makes. So the question is: Is it fair to assume all Linuxes use gmake as /usr/bin/make? And how about BSDs? |
POSIX make describes very little, although Issue 8 is adding some more badly-needed functionality that is commonly implemented by Make editions.
Linux in general always tends to use GNU Make as /usr/bin/make.
BSD uses "BSD make". You can install it as dev-build/bmake. It is regularly imported from NetBSD's "make" and the version of bmake reflects the date of the import from NetBSD. Then bmake is also exported into FreeBSD. I didn't look to see what OpenBSD does since no one should pay attention to OpenBSD...
GNU Make and bmake have wildly different featuresets outside of the common POSIX make stuff (including Issue 8 improvements). For example, $(shell ...) is specific to GNU, and bmake has != which GNU Make added support for as part of standardizing it for POSIX. GNU Make has ifeq/ifdef, bmake does not -- and instead has .if / .ifdef / .ifmake and a specialized operator grammar for that context. GNU make generalizes $(function ...) and provides many such functions such as patsubst, filter, strip.... bmake supports $(variable:M<pattern>) or $(variable:O) and other "modifiers".
If you want to write portable Makefiles, give up. If you want to write non-portable Makefiles, then either you force BSD users to install gmake (trivial via the package manager) or you force linux users to install bmake (trivial via the package manager).
Since your goal was that users don't need to install too much dependencies, consider this: the entire point of GNU autoconf is that users don't need to install ANY dependencies, since autoconf is a developer tool run by people named "Zucca" for creating portable configure scripts that generate portable Makefiles.
 |
|
| Back to top |
|
|
eschwartz
Developer
Joined: 29 Oct 2023
Posts: 166
|
| Posted: Tue Sep 24, 2024 4:40 pm Post subject: |
|
|
| pjp wrote: | I chuckled at that one because it seems like MS saying "use MS".
Regarding vpath, that's a lot to absorb. It seems I may be doing some things "I shouldn't be doing" in the Makefile. And to add, I don't want to build object files in the same directory as the source files. I'm not sure why that's the recommended approach. When I compile on the command line, there are zero issues, so maybe it's a problem with make. |
Some might argue that one shouldn't be doing things in a Makefile at all, but rather do it in configure.ac and Makefile.am, but if your goal is to be doing things in a Makefile then I don't think what you're doing is intrinsically wrong overall.
It is *easy* to build object files in the same directory as the source files, but every configurable build system (autoconf + automake, meson, cmake, etc.) support building "out of source" for very good reason: it makes it easy to clean up or run parallel builds with different options if you produce object files in a different directory.
Implementing this in raw Make is fiddly. VPATH is a tool you can use for this. You can also just write out each object file and each source file as separate commands, but pattern rules (which you want to use, presumably) make this much more annoying to do reliably. |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20407
|
| Posted: Tue Sep 24, 2024 5:49 pm Post subject: |
|
|
| Hu wrote: | | As I read the existing Makefile, they would collide. Second, it becomes easier to wipe out all object files corresponding to a given source directory: rm -r obj/fib/. |
Yes, they could collide. I only used this approach for simplicity while trying to get compiling to work. I originally considered using fib/obj/, but didn't like that. It didn't occur to me to use sub directories in obj. That was a simple change and now works (I haven't fixed "clean" yet).
| dmpogo wrote: | | Do you need EXE to be dependent on SRC ? I would write EXE line to just assembling objects and necessary libraries into executable. Though you will have to make src/main.o somewhere else |
I honestly don't know :).
There is no main.o, only the executable main which is put into the project directory (mainly because I got tired of using ./bin/main). I think what depending on SRC achieves is recompiling when a .c file changes. And as I have it written, I don't know that it is sufficient. I think if I change a .h file, make doesn't recompile. At any rate, I believe that is why I put it there.
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
pjp
Administrator
Joined: 16 Apr 2002
Posts: 20407
|
| Posted: Tue Sep 24, 2024 5:49 pm Post subject: |
|
|
| eschwartz wrote: | | Some might argue that one shouldn't be doing things in a Makefile at all, but rather do it in configure.ac and Makefile.am, but if your goal is to be doing things in a Makefile then I don't think what you're doing is intrinsically wrong overall. |
Something in one of the vpath links made me think that. I'd have to go back and find it.
I'm not attached to using a Makefile, it was the first solution I knew about, and it seemed like understanding at least at a basic level was beneficial. Maybe now is a good time to learn configure.ac. Is that part of Autotools? I only ask because of some "bad press" Autotools received after the xz problem.
| eschwartz wrote: | | It is *easy* to build object files in the same directory as the source files, but every configurable build system (autoconf + automake, meson, cmake, etc.) support building "out of source" for very good reason: it makes it easy to clean up or run parallel builds with different options if you produce object files in a different directory. |
I may have misunderstood the point being made in the vpath information. It's a bit confusing so far.
| eschwartz wrote: | | Implementing this in raw Make is fiddly. VPATH is a tool you can use for this. You can also just write out each object file and each source file as separate commands, but pattern rules (which you want to use, presumably) make this much more annoying to do reliably. |
I'm not attached to using pattern rules, it just seemed better than having to hard code file names and paths and subsequently having to update them. Maybe that's where configure.ac helps? I'll look into it (and continue reading about vpath).
This works too :) (yes, I get that my small project makes it not impractical). | Code: | $ time ./build.sh
real 0m0.651s
user 0m0.094s
sys 0m0.045s
$ cat build.sh
#!/bin/sh
gcc -c fib/fib.c -o obj/fib/fib.o
gcc -c foobarbaz/bar.c -o obj/foobarbaz/bar.o
gcc -c foobarbaz/baz.c -o obj/foobarbaz/baz.o
gcc -c foobarbaz/foobarbaz.c -o obj/foobarbaz/foobarbaz.o
gcc src/main.c -o main obj/fib/fib.o obj/foobarbaz/*.o -I./inc |
_________________
Quis separabit? Quo animo? |
|
| Back to top |
|
|
Hu
Administrator
Joined: 06 Mar 2007
Posts: 22442
|
| Posted: Tue Sep 24, 2024 6:30 pm Post subject: |
|
|
| pjp wrote: | | And as I have it written, I don't know that it is sufficient. I think if I change a .h file, make doesn't recompile. At any rate, I believe that is why I put it there. |
Make can be made to recompile on header changes, but it is your responsibility to inform it which headers matter for which source files. This is tedious in larger projects, and is one of the reasons people often decide they don't want to maintain that information manually.
configure.ac is the canonical name for the input to autoconf, which yes, is a part of Autotools. Autotools mainly gets bad press for how readily innocent scripts can become unreadable. As a corollary to that, it is comparatively easier to hide malicious code in an Autotools input, because few people will read that input for fun, and so much legitimate autoconf input is difficult to read that obfuscated code will not jump out. Compare that to something like Python, where "good style" usually produces code that is not particularly dense, so obfuscated code can be presumed to be suspicious. |
|
| Back to top |
|
|
eschwartz
Developer
Joined: 29 Oct 2023
Posts: 166
|
| Posted: Tue Sep 24, 2024 6:55 pm Post subject: |
|
|
| pjp wrote: | Something in one of the vpath links made me think that. I'd have to go back and find it.
I'm not attached to using a Makefile, it was the first solution I knew about, and it seemed like understanding at least at a basic level was beneficial. Maybe now is a good time to learn configure.ac. Is that part of Autotools? I only ask because of some "bad press" Autotools received after the xz problem.
|
What's the difference between "bad" and "there is a more elegant way to do it that takes advantage of obscurely powerful tricks"?
Yes, configure.ac is part of autotools. The bad press is more of a meme than anything else -- the problem was really, if you think about it, the use of the GNU Gettext infrastructure as a plugin added on top of configure.ac.
configure.ac can be very short and simple, as can Makefile.am -- when running the autoconf command, it generates an immensely long shell script that is distinctly unreadable, but it's also trivial to recreate by re-running autoconf from a trusted source and making sure that the results compare the same. The xz problem was because a *maintainer* of xz turned out to be evil, and edited the resulting configure logic by hand.
| pjp wrote: |
| eschwartz wrote: | | Implementing this in raw Make is fiddly. VPATH is a tool you can use for this. You can also just write out each object file and each source file as separate commands, but pattern rules (which you want to use, presumably) make this much more annoying to do reliably. |
I'm not attached to using pattern rules, it just seemed better than having to hard code file names and paths and subsequently having to update them. Maybe that's where configure.ac helps? I'll look into it (and continue reading about vpath). |
That is actually where Makefile.am, the other half of autotools, comes in handy.
| Code: | $ cat configure.ac
AC_INIT([pjp-project], [1.0.0])
AM_INIT_AUTOMAKE([foreign subdir-objects])
AC_PROG_CC
if test "$ac_cv_prog_cc_c11" = no; then
AC_MSG_ERROR([We require -std=c11!])
fi
AC_CONFIG_FILES([Makefile])
AC_OUTPUT
|
| Code: | $ cat Makefile.am
AM_CPPFLAGS = -I$(srcdir)/inc
AM_CFLAGS = -Wall -Wextra -pedantic
bin_PROGRAMS = main
main_SOURCES = src/main.c \
fib/fib.c \
foobarbaz/bar.c \
foobarbaz/bar.h \
foobarbaz/baz.c \
foobarbaz/baz.h \
foobarbaz/foobarbaz.c
|
A few notes:
- this is readable because it is simple, autotools is written in a language unique to it called "m4sh", which mixes POSIX sh and GNU m4 into one file.
- initing automake passes two options: foreign, meaning we are not a GNU project and don't mandate stuff like an AUTHORS / NEWS file, and subdir-objects, which means we put all the object files in directories such as foobarbaz/ or fib/ as appropriate.
- AM_CPPFLAGS and AM_CFLAGS are for project-specific arguments as opposed to the user's CFLAGS="-march=native -flto -O3 -ggdb"
- you do not strictly need to list headers in main_SOURCES. as automake creates "header dependencies" for you automatically, but if you do not list them in SOURCES then running "make dist" will create a SRC_URI distfile tarball that does not include the .h files, which is a bit of an oopsie.
This handles a lot of things for you:
- generates rules to compile each source
- generates rule to link
- by default places all objects next to sources
- automatically handles changes in the headers for you!!!
- supports "mkdir obj; cd obj; ../configure; make" which places the Makefile in obj/Makefile and objects there as well
- creates a "make clean" rule for you
- works with POSIX make, GNU Make, BSD Make, any Make you like
- supports the user's overridden CFLAGS, LDFLAGS
- respects the user's $CC
- allows you to run "make dist" and create pjp-project-1.0.0.tar.gz which you can upload somewhere, containing the ./configure script and everything needed to build
After you run autoreconf -fi to generate the portable Makefile.in and configure script:
| Code: | $ wc -l configure Makefile.in
5421 configure
809 Makefile.in
6230 total
|
But, you can rerun autoreconf if you don't trust the author, and verify that you get the exact same byte-identical copy of the configure script and Makefile.in |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
