| View previous topic :: View next topic |
| Author |
Message |
apurkrt
Tux's lil' helper
Joined: 26 Feb 2011
Posts: 116
Location: Czechia, Europe
|
 Posted: Sat Feb 08, 2020 2:21 pm Post subject: sys-process/cronie and /etc/cron.hourly,daily,weekly,monthly |
 |
|
Hi!
I would just like to sum up my recent encounter with cronie and its' default config on gentoo, and raise a few questions.
Thanks to people on irc and irl who helped me with this.
I was interested in system-wide cron jobs (not user crontabs).
vixie-cron is now removed from portage, so let's begin with installing cronie:
| Code: | gentoo /etc # emerge -av sys-process/cronie
These are the packages that would be merged, in order:
Calculating dependencies... done!
[ebuild N ] sys-process/cronbase-0.3.7-r6::gentoo 0 KiB
[ebuild N ] sys-process/cronie-1.5.5::gentoo USE="anacron inotify pam (-selinux)" 0 KiB
Total: 2 packages (2 new), Size of downloads: 0 KiB
...
|
Ok, installed.
note: by default, "anacron" USE flag is enabled, so /usr/sbin/anacron will get installed.
| Code: | gentoo ~ # qlist -e cronie | grep bin
/usr/bin/crontab
/usr/bin/cronnext
/usr/sbin/crond
/usr/sbin/anacron
gentoo /etc # qlist -e cronbase | grep bin
/usr/sbin/run-crons
|
To start cronie, you can now:
| Code: | gentoo ~ # /etc/init.d/cronie start
|
note: the daemon is named "crond".
My main question was: How exactly does the /etc/cron.{hourly,daily,weekly,monthly} work?
Let's find out.
The starting point for me was /etc/crontab (I knew about this file before)
| Code: | gentoo ~ # cat /etc/crontab
# Global variables
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/
# For details see man 5 crontab
# Example of job definition:
# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * * * * * user-name command to be executed
|
Some variables, explanatory comments, but no job set up here! Let's head to man:
| Code: | gentoo ~ # man 5 crontab
...
Jobs in /etc/cron.d/
The jobs in cron.d and /etc/crontab are system jobs, which are used
usually for more than one user, thus, additionally the username is
needed. MAILTO on the first line is optional.
...
|
Ok, so let's examine /etc/cron.d/ directory as well
| Code: | gentoo ~ # ls /etc/cron.d
0hourly
gentoo ~ # cat /etc/cron.d/0hourly
# Run the hourly jobs
SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
01 * * * * root run-parts /etc/cron.hourly
|
Finally something - the last line means: execute "run-parts /etc/cron.hourly" as root on every hour, 01 minute (i.e. "<hour>:01").
That would explain how jobs in /etc/cron.hourly would get run (hourly).
Fine, but what about /etc/cron.{daily,weekly,monthly}? Let's investigate further.
Where does run-parts come from and what does it do:
| Code: | gentoo ~ # which run-parts
/bin/run-parts
gentoo ~ # qfile /bin/run-parts
sys-apps/debianutils: /bin/run-parts
gentoo ~ # man run-parts
...
run-parts runs all the executable files named within constraints described
below, found in directory directory . Other files and directories are silently ignored.
...
Files are run in the lexical sort order
...
|
Ok so let's examine /etc/cron.hourly/
| Code: | gentoo ~ # ls -l /etc/cron.hourly/
total 4
-rwxr-x--- 1 root root 580 Feb 8 11:29 0anacron
|
What does this 0anacron do?
| Code: | gentoo ~ # cat /etc/cron.hourly/0anacron
#!/bin/sh
# Check whether 0anacron was run today already
if test -r /var/spool/anacron/cron.daily; then
day=`cat /var/spool/anacron/cron.daily`
fi
if [ `date +%Y%m%d` = "$day" ]; then
exit 0
fi
# Do not run jobs when on battery power
online=1
for psupply in AC ADP{0..9} ; do
sysfile="/sys/class/power_supply/$psupply/online"
if [ -f $sysfile ] ; then
if [ `cat $sysfile 2>/dev/null`x = 1x ]; then
online=1
break
else
online=0
fi
fi
done
if [ $online = 0 ]; then
exit 0
fi
/usr/sbin/anacron -s
|
Huh. So once a day, and if not on battery power, it does run "/usr/sbin/anacron -s".
So cron is (by default) running anacron. Seems weird? It did to me, but I later found on gentoo wiki
"Anacron is not a cron daemon, it is something that usually works in conjunction with one.".
Only thing contraintuitive is that there is also a solo package sys-process/anacron in portage (that's different anacron implementation).
Anyway, back to the anacron which came with cronie
| Code: | gentoo ~ # man anacron
...
Anacron is used to execute commands periodically, with a frequency
specified in days.
...
Anacron reads a list of jobs from the /etc/anacrontab configuration
file (see anacrontab(5))
...
|
So what is in /etc/anacrontab?
| Code: | gentoo ~ # cat /etc/anacrontab
# /etc/anacrontab: configuration file for anacron
# See anacron(8) and anacrontab(5) for details.
SHELL=/bin/sh
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
# the maximal random delay added to the base delay of the jobs
RANDOM_DELAY=45
# the jobs will be started during the following hours only
START_HOURS_RANGE=3-22
#period in days delay in minutes job-identifier command
1 5 cron.daily nice run-parts /etc/cron.daily
7 25 cron.weekly nice run-parts /etc/cron.weekly
@monthly 45 cron.monthly nice run-parts /etc/cron.monthly
|
Finally! That is what runs /etc/cron.{daily,weekly,monthly} in the end.
The whole line is like follows:
"/etc/init.d/cronie start" -> crond started, reads "/etc/cron.d/0hourly"
-> "run-parts /etc/cron.hourly" -> runs "/etc/cron.hourly/0anacron" on every <hour>:01
-> runs "/usr/sbin/anacron -s" once a day -> interprets /etc/anacrontab -> runs /etc/cron.{daily,weekly,monthly}
So /etc/cron.{hourly,daily,weekly,monthly} is a mixed bag, wrt how they are executed: /etc/cron.hourly is run directly by crond, whereas /etc/cron.{daily,weekly,monthly} is run indirectly by anacron, which is run by crond
note: anacron cannot start hourly jobs
Question: starting times So when exactly do the cron jobs start?
For jobs in /etc/cron.hourly, the answer is - they will start once 0anacron is done (since 0anacron is usually alphabetically first).
0anacron has a check for running once a day only, "anacron -s" becomes a sleeping process, so mostly the hourly jobs will be run at <hour>:01
When does anacron run the jobs i.e. when do the /etc/cron.{daily,weekly,monthly} start is (afaict) random - there is delay+random delay,
there is START_HOURS_RANGE. Tbh, I had not investigated this that much.
Question: reentrancy (overlapping hourly jobs)
The most important question for me was: will crond start a new (hourly)
job even if the previous instance is still running?
The answer for /etc/cron.hourly/ is a definite yes, happily (tested it experimentally with job containing sleep 65m)
reason: in my case, there is a shell script in /etc/cron.hourly, that synchronizes files from other server, using rsync, and I would not like two rsynces running at the same time (if the sync takes more than an hour).
note: I was advised, that it is a good thing to use flock inside the sync script, something like (a fully atomic way of preventing the race condition)
| Code: | ...
exec 9>/var/run/myscript.lock; flock -n 9 || exit
...
|
For /etc/cron.daily (weekly, monthly) and jobs running for more than one day (week, month), i.e. those that are run by anacron, I do not know.
Perhaps anacron in itself prevents the concurrent running? Really not sure
| Code: | gentoo ~ # man anacron
...
-s Serializes execution of jobs. Anacron does not start a new job
before the previous one finished.
...
|
But "0anacron" is started hourly by cron.d/0hourly, it checks for date, runs "anacron -s" daily.
So perhaps more than one instance of "anacron -s" could finally be running (if the job takes more than 24h). Again, here I am unsure.
Final note
Previously, with vixie-cron (now removed from the tree), the running of /etc/cron.{hourly,daily,weekly,monthly} was accomplished in a different way - /etc/crontab contained (by default) the following:
| Code: | 59 * * * * root rm -f /var/spool/cron/lastrun/cron.hourly
9 3 * * * root rm -f /var/spool/cron/lastrun/cron.daily
19 4 * * 6 root rm -f /var/spool/cron/lastrun/cron.weekly
29 5 1 * * root rm -f /var/spool/cron/lastrun/cron.monthly
*/10 * * * * root test -x /usr/sbin/run-crons && /usr/sbin/run-crons
|
The above config can still be used but first you have to comment out the following line in /etc/cron.d/0hourly
| Code: | | #01 * * * * root run-parts /etc/cron.hourly |
and also do the following
| Code: | | gentoo ~ # chmod -x /etc/cron.hourly/0anacron |
effectively disabling anacron (otherwise be prepared for https://bugs.gentoo.org/621706).
EDIT: beware - upon reinstalling or updating sys-process/cronie, the executable bit on /etc/cron.hourly/0anacron gets silently enabled again, switching on anacron. So rather than "chmod -x", it is probably better to edit 0anacron and comment out the line with "anacron -s".
The advantage of using the above config, i.e. relying on run-crons script over anacron is probably better control on when exactly the daily,weekly,monthly scripts will get run (though you can setup random delay to zero and have that control with anacron too). Every ten minutes /usr/sbin/run-crons would be run (if it was executable). It would check the /var/spool/cron/lastrun/cron.* files, and run executables in /etc/cron.hourly/* on every <hour>:00, /etc/cron.daily/* on 3:10am, /etc/cron.weekly/* on every saturday 4:20am, /etc/cron.monthly/* on 5:30am every first day of the month.
I have verified experimentally, that the run-crons script (still provided nowadays with sys-process/cronbase) prevents the reentrant execution of the /etc/cron.hourly/ scripts.
To that end, I have created a script
| Code: | gentoo ~ # cat /etc/cron.hourly/test65m
#!/bin/sh
ST=`date +%s`
touch /root/crontest/${ST}start
sleep 65m
touch /root/crontest/${ST}end
|
let it run overnight with the following result (reformatted)
| Code: | gentoo ~ # ls -l /root/crontest/
-rw-r--r-- 1 root root 0 Feb 8 02:00 1581123601start
-rw-r--r-- 1 root root 0 Feb 8 03:05 1581123601end
-rw-r--r-- 1 root root 0 Feb 8 03:10 1581127801start
-rw-r--r-- 1 root root 0 Feb 8 04:15 1581127801end
-rw-r--r-- 1 root root 0 Feb 8 04:20 1581132002start
-rw-r--r-- 1 root root 0 Feb 8 05:25 1581132002end
-rw-r--r-- 1 root root 0 Feb 8 05:30 1581136201start
-rw-r--r-- 1 root root 0 Feb 8 06:35 1581136201end
-rw-r--r-- 1 root root 0 Feb 8 06:40 1581140401start
-rw-r--r-- 1 root root 0 Feb 8 07:45 1581140401end
-rw-r--r-- 1 root root 0 Feb 8 07:50 1581144601start
-rw-r--r-- 1 root root 0 Feb 8 08:55 1581144601end
|
Here are some related commits for those interested (also you can use "git log -p -- sys-process/cronie/files/cronie-1.3-crontab" from within the checked out gentoo tree)
https://gitweb.gentoo.org/repo/gentoo.git/commit/sys-process/cronie/files?id=331b15d5fd648fa6b2a43d07757b365e19d749f8
https://gitweb.gentoo.org/repo/gentoo.git/commit/sys-process/cronie/files?id=c474dda61c900197ef05e64d1e5af35785cbb7c1
https://gitweb.gentoo.org/repo/gentoo.git/commit/sys-process/cronie/files?id=f79f8ac5d95fa3f88301f0bad8b03e9d5f497a59
Last edited by apurkrt on Wed Feb 12, 2020 8:32 pm; edited 6 times in total |
|
| Back to top |
|
 |
gtwrek
Tux's lil' helper
Joined: 10 Mar 2017
Posts: 110
Location: San Jose, CA
|
| Posted: Sat Feb 08, 2020 3:44 pm Post subject: Thanks |
|
|
I've nothing to add other than expressing thanks. With the recent (deprecation?) of vixie-cron, I was looking closer at the details of cron(ish) solutions. - I never paid too much attention.
The whole cron vs anacron and how things worked (under the covers) was a bit confusing too me. Apparently, I'm not alone.
I started the exact exploration you've done, but only got about a third of the way you did.
Your note should be a sticky of some sort. I know I'm bookmarking it. |
|
| Back to top |
|
|
apurkrt
Tux's lil' helper
Joined: 26 Feb 2011
Posts: 116
Location: Czechia, Europe
|
| Posted: Sun Feb 09, 2020 8:53 am Post subject: |
|
|
Some more info on anacron
I used the default setup, did remove the locks
| Code: | | gentoo ~ # rm /var/spool/anacron/cron.{daily,weekly,monthly} |
to clean up the environment beforehand and then
| Code: | | gentoo ~ # /etc/init.d/cronie start |
on 22:40:47, let it run overnight.
Here is an extract from /var/log/syslog
| Code: | Feb 8 22:40:47 gentoo crond[10273]: (CRON) STARTUP (1.5.5)
Feb 8 22:40:47 gentoo crond[10273]: (CRON) INFO (Syslog will be used instead of sendmail.)
Feb 8 22:40:47 gentoo crond[10273]: (CRON) INFO (RANDOM_DELAY will be scaled with factor 15% if used.)
Feb 8 22:40:47 gentoo crond[10273]: (CRON) INFO (running with inotify support)
Feb 8 23:01:01 gentoo CROND[11924]: (root) CMD (run-parts /etc/cron.hourly)
Feb 8 23:01:01 gentoo anacron[11931]: Anacron started on 2020-02-08
Feb 8 23:01:01 gentoo anacron[11931]: Will run job `cron.daily' in 7 min.
Feb 8 23:01:01 gentoo anacron[11931]: Will run job `cron.weekly' in 27 min.
Feb 8 23:01:01 gentoo anacron[11931]: Will run job `cron.monthly' in 47 min.
Feb 8 23:01:01 gentoo anacron[11931]: Jobs will be executed sequentially
Feb 8 23:08:01 gentoo anacron[11931]: Job `cron.daily' started
Feb 8 23:08:01 gentoo anacron[11931]: Job `cron.daily' terminated
Feb 8 23:28:01 gentoo anacron[11931]: Job `cron.weekly' started
Feb 8 23:28:01 gentoo anacron[11931]: Job `cron.weekly' terminated
Feb 8 23:48:01 gentoo anacron[11931]: Job `cron.monthly' started
Feb 8 23:48:01 gentoo anacron[11931]: Job `cron.monthly' terminated
Feb 8 23:48:01 gentoo anacron[11931]: Normal exit (3 jobs run)
Feb 9 00:01:01 gentoo CROND[15730]: (root) CMD (run-parts /etc/cron.hourly)
Feb 9 00:01:01 gentoo anacron[15737]: Anacron started on 2020-02-09
Feb 9 00:01:01 gentoo anacron[15737]: Normal exit (0 jobs run)
Feb 9 01:01:01 gentoo CROND[19447]: (root) CMD (run-parts /etc/cron.hourly)
Feb 9 01:01:01 gentoo anacron[19454]: Anacron started on 2020-02-09
Feb 9 01:01:01 gentoo anacron[19454]: Normal exit (0 jobs run)
Feb 9 02:01:01 gentoo CROND[23240]: (root) CMD (run-parts /etc/cron.hourly)
Feb 9 02:01:01 gentoo anacron[23247]: Anacron started on 2020-02-09
Feb 9 02:01:01 gentoo anacron[23247]: Normal exit (0 jobs run)
Feb 9 03:01:01 gentoo CROND[27158]: (root) CMD (run-parts /etc/cron.hourly)
Feb 9 03:01:01 gentoo anacron[27165]: Anacron started on 2020-02-09
Feb 9 03:01:01 gentoo anacron[27165]: Will run job `cron.daily' in 48 min.
Feb 9 03:01:01 gentoo anacron[27165]: Jobs will be executed sequentially
Feb 9 03:49:01 gentoo anacron[27165]: Job `cron.daily' started
Feb 9 03:49:01 gentoo anacron[27165]: Job `cron.daily' terminated
Feb 9 03:49:01 gentoo anacron[27165]: Normal exit (1 job run)
Feb 9 04:01:01 gentoo CROND[31110]: (root) CMD (run-parts /etc/cron.hourly)
Feb 9 05:01:01 gentoo CROND[2467]: (root) CMD (run-parts /etc/cron.hourly)
Feb 9 06:01:01 gentoo CROND[6387]: (root) CMD (run-parts /etc/cron.hourly)
Feb 9 07:01:01 gentoo CROND[10424]: (root) CMD (run-parts /etc/cron.hourly)
Feb 9 08:01:01 gentoo CROND[14513]: (root) CMD (run-parts /etc/cron.hourly)
Feb 9 09:01:01 gentoo CROND[19041]: (root) CMD (run-parts /etc/cron.hourly)
|
State of the lock files now
| Code: | gentoo ~ # cat /var/spool/anacron/cron.daily
20200209
gentoo ~ # cat /var/spool/anacron/cron.weekly
20200208
gentoo ~ # cat /var/spool/anacron/cron.monthly
20200208
|
note: Every "Anacron started on ..." corresponds to "anacron -s" command, started by /etc/cron.hourly/0anacron. So "anacron -s" is not run only once in a day as I have written before. It is instead run every hour (see 00:01:01, 01:01:01, 02:01:01, 03:01:01), until /var/spool/anacron/cron.daily gets updated. After that, the check at the beginning of /etc/cron.hourly/0anacron kicks in and no more "anacron -s" starts for the day.
/var/spool/anacron/cron.daily gets updated only after 3am - that comes from "START_HOURS_RANGE=3-22" in /etc/anacrontab.
note: The randomness of start times is also well seen. On 23:01:01 there is "Will run job `cron.daily' in 7 min." (weekly in 27 min, monthly in 47 min).
7 comes from 5+2 (delay 5 + RANDOM_DELAY 2 (from the 0-45 range)). 27=25+2, 47=45+2. The random delay is the same for all jobs within one instance of "anacron -s".
On 03:01:01, there is "Will run job `cron.daily' in 48 min.". Here it is 5+43 (delay 5 + random delay 43). |
|
| Back to top |
|
|
jmanko
Tux's lil' helper
Joined: 11 Sep 2005
Posts: 139
Location: NEPA, USA
|
| Posted: Fri Jun 05, 2020 6:21 pm Post subject: |
|
|
Can't you just emerge cronie without the anacron USE flag? Shouldn't that remove anacron and leave cronie to just run /etc/crontab as normal?
_________________
"What stands in the way becomes the way." -- Marcus Aurelius |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Gentoo Chat
