{"id":311,"date":"2016-12-23T14:41:02","date_gmt":"2016-12-23T06:41:02","guid":{"rendered":"https:\/\/www.zhuyanbin.com\/?p=311"},"modified":"2016-12-23T14:57:16","modified_gmt":"2016-12-23T06:57:16","slug":"mac-os-x-launchd-is-cool","status":"publish","type":"post","link":"https:\/\/www.yanbin888.com\/?p=311","title":{"rendered":"Mac OS X: Launchd Is Cool"},"content":{"rendered":"

One of the core components of Mac OS X is launchd<\/b><\/a>, and it turns out it can do some cool things.<\/p>\n

I particularly like the idea of using QueueDirectories<\/b> to monitor and act upon files dropped into a directory, without having to run any extra daemons. The files could be uploaded to S3, transcoded to a different video format, gzipped\u2026 anything.<\/p>\n

Anyway, I recently fell into the launchd<\/b> documentation, and came out with this write-up. Let me know<\/a> if you find it useful.<\/p>\n

Overview<\/h2>\n

The first thing that the Mac OS kernel runs on boot is launchd, which bootstraps the rest of the system by loading and managing various daemons, agents, scripts and other processes. The launchd man page<\/a> clarifies the difference between a daemon and an agent:<\/p>\n

In the launchd lexicon, a \u201cdaemon\u201d is, by definition, a system-wide service of which there is one instance for all clients. An \u201cagent\u201d is a service that runs on a per-user basis. Daemons should not attempt to display UI or interact directly with a user\u2019s login session. Any and all work that involves interacting with a user should be done through agents.<\/p><\/blockquote>\n

Daemons and agents are declared and configured by creating .plist<\/b> files in various locations of the system:<\/p>\n

~\/Library\/LaunchAgents         Per-user agents provided by the user.\r\n\/Library\/LaunchAgents          Per-user agents provided by the administrator.\r\n\/Library\/LaunchDaemons         System-wide daemons provided by the administrator.\r\n\/System\/Library\/LaunchAgents   Per-user agents provided by OS X.\r\n\/System\/Library\/LaunchDaemons  System-wide daemons provided by OS X.\r\n<\/pre>\n
Perhaps best of all, launchd is open source under the Apache License 2.0<\/a>. You can currently find the latest source code on the Apple Open Source site<\/a>.<\/p>\n
launchd as cron<\/h2>\n
The Mac OS crontab<\/a> man page says:<\/p>\n
Although cron(8) and crontab(5) are officially supported under Darwin,\r\ntheir functionality has been absorbed into launchd(8), which provides a\r\nmore flexible way of automatically executing commands.\r\n<\/pre>\n
Turns out launchd<\/b> has a simple StartInterval <integer><\/b> property, which starts the job every N seconds. However the true cron-like power lies in StartCalendarInterval<\/b>:<\/p>\n
StartCalendarInterval <dictionary of integers or array of dictionary of integers>\r\n\r\nThis optional key causes the job to be started every calendar interval as\r\nspecified. Missing arguments are considered to be wildcard. The semantics\r\nare much like crontab(5).  Unlike cron which skips job invocations when the\r\ncomputer is asleep, launchd will start the job the next time the computer\r\nwakes up.  If multiple intervals transpire before the computer is woken,\r\nthose events will be coalesced into one event upon wake from sleep.\r\n\r\n     Minute <integer>\r\n     The minute on which this job will be run.\r\n\r\n     Hour <integer>\r\n     The hour on which this job will be run.\r\n\r\n     Day <integer>\r\n     The day on which this job will be run.\r\n\r\n     Weekday <integer>\r\n     The weekday on which this job will be run (0 and 7 are Sunday).\r\n\r\n     Month <integer>\r\n     The month on which this job will be run.\r\n<\/pre>\n
Lets find the shortest example of this in action:<\/p>\n
pda@paulbook ~ > grep -rl StartCalendarInterval \\\r\n                   \/Library\/Launch* \/System\/Library\/Launch* | \\\r\n                   xargs wc -l | sort -n | head -n1 | awk '{print $2}' | xargs cat\r\n\r\n<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n<!DOCTYPE plist PUBLIC \"-\/\/Apple\/\/DTD PLIST 1.0\/\/EN\" \"http:\/\/www.apple.com\/DTDs\/PropertyList-1.0.dtd\">\r\n<plist version=\"1.0\">\r\n<dict>\r\n        <key>Label<\/key>\r\n        <string>com.apple.gkreport<\/string>\r\n        <key>ProgramArguments<\/key>\r\n        <array>\r\n                <string>\/usr\/libexec\/gkreport<\/string>\r\n        <\/array>\r\n        <key>StartCalendarInterval<\/key>\r\n        <dict>\r\n                <key>Minute<\/key><integer>52<\/integer>\r\n                <key>Hour<\/key><integer>3<\/integer>\r\n                <key>WeekDay<\/key><integer>5<\/integer>\r\n        <\/dict>\r\n<\/dict>\r\n<\/plist>\r\n<\/pre>\n
Better than cron? Apart from better handling of skipped jobs after system wake, it also supports per-job environment variables, which can save writing wrapper scripts around your cron jobs:<\/p>\n
EnvironmentVariables <dictionary of strings>\r\n\r\nThis optional key is used to specify additional environmental variables to\r\nbe set before running the job.\r\n<\/pre>\n
So, anything XML is obviously worse than 0 52 3 * 5 \/path\/to\/command<\/b>, but launchd<\/b> is packing more features than cron, so it can pull it off.<\/p>\n
launchd as a filesystem watcher<\/h2>\n
Apart from having an awesome daemon\/agent manager, Mac OS X also has an excellent Mail Transport Agent called postfix<\/a>. There\u2019s a good chance your ISP runs the same software to handle millions of emails every day. We\u2019ll be using it as an example of how launchd<\/b> can start jobs based on filesystem changes.<\/p>\n
Because your laptop isn\u2019t, and shouldn\u2019t be, a mail server, you don\u2019t want postfix running all the time. But when messages are injected into it, e.g. by a script shelling out to \/usr\/sbin\/sendmail<\/b> or \/usr\/bin\/mail<\/b>, you want them to be delivered straight away.<\/p>\n
Here\u2019s how Mac OS X does it (\/System\/Library\/LaunchDaemons\/org.postfix.master.plist<\/b>):<\/p>\n
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n<!DOCTYPE plist PUBLIC \"-\/\/Apple Computer\/\/DTD PLIST 1.0\/\/EN\" \"http:\/\/www.apple.com\/DTDs\/PropertyList-1.0.dtd\">\r\n<plist version=\"1.0\">\r\n<dict>\r\n    <key>Label<\/key>\r\n    <string>org.postfix.master<\/string>\r\n    <key>Program<\/key>\r\n    <string>\/usr\/libexec\/postfix\/master<\/string>\r\n    <key>ProgramArguments<\/key>\r\n    <array>\r\n        <string>master<\/string>\r\n        <string>-e<\/string>\r\n        <string>60<\/string>\r\n    <\/array>\r\n    <key>QueueDirectories<\/key>\r\n    <array>\r\n        <string>\/var\/spool\/postfix\/maildrop<\/string>\r\n    <\/array>\r\n    <key>AbandonProcessGroup<\/key>\r\n    <true\/>\r\n<\/dict>\r\n<\/plist>\r\n<\/pre>\n
We\u2019ll start with the simple part. ProgramArguments<\/b> passes -e 60<\/b> to postfix, described thusly<\/a>:<\/p>\n
-e exit_time\r\n              Terminate the master process after exit_time seconds.\r\n              Child processes terminate at their convenience.\r\n<\/pre>\n
So postfix is told to exit after running for 60 seconds. The mystery (to me, earlier today, at least) is how it gets started. It could be on a cron-like schedule, but (a) it isn\u2019t, (b) that would suck, and (c) it would result in delayed mail delivery. It turns out the magic lies in QueueDirectory<\/b>, which I initially overlooked thinking it was a postfix option. The launchd.plist<\/a> man page says:<\/p>\n
WatchPaths <array of strings>\r\nThis optional key causes the job to be started if any one of the listed\r\npaths are modified.\r\n\r\nQueueDirectories <array of strings>\r\nMuch like the WatchPaths option, this key will watch the paths for\r\nmodifications. The difference being that the job will only be started if\r\nthe path is a directory and the directory is not empty.<\/pre>\n
The Launchd Wikipedia page<\/a> actually goes into more detail:<\/p>\n
QueueDirectories\r\nWatch a directory for new files. The directory must be empty to begin with,\r\nand must be returned to an empty state before QueueDirectories will launch\r\nits task again.\r\n<\/pre>\n
So launchd<\/b> can monitor a directory for new files, and then trigger an agent\/daemon to consume them. In this case, the postfix sendmail(1) man page<\/a> tells us that \u201cPostfix sendmail(1) relies on the postdrop(1) command to create a queue file in the maildrop directory\u201d, and the man page for postdrop(1)<\/a> tells us that \/var\/spool\/postfix\/maildrop<\/b> is the maildrop queue. launchd<\/b> sees new mail there, fires up postfix, and then stops it after 60 seconds. This might cause deferred mail to stay deferred for quite some time, but again; your laptop isn\u2019t a mail server.<\/p>\n
launchd as inetd<\/h2>\n
Tranditionally the inetd<\/a> and later xinetd<\/a> \u201csuper-server daemon\u201d were used to listen on various ports (e.g. FTP, telnet, \u2026) and launch daemons on-demand to handle in-bound connection, keeping them out of memory at other times. Sounds like something launchd<\/b> could do\u2026<\/p>\n
Lets create a simple inetd-style server at ~\/Library\/LaunchAgents\/my.greeter.plist<\/b>:<\/p>\n
<plist version=\"1.0\">\r\n<dict>\r\n  <key>Label<\/key><string>my.greeter<\/string>\r\n  <key>ProgramArguments<\/key>\r\n  <array>\r\n    <string>\/usr\/bin\/ruby<\/string>\r\n    <string>-e<\/string>\r\n    <string>puts \"Hi #{gets.match(\/(\\w+)\\W*\\z\/)[1]}, happy #{Time.now.strftime(\"%A\")}!\"<\/string>\r\n  <\/array>\r\n  <key>inetdCompatibility<\/key><dict><key>Wait<\/key><false\/><\/dict>\r\n  <key>Sockets<\/key>\r\n  <dict>\r\n    <key>Listeners<\/key>\r\n    <dict>\r\n      <key>SockServiceName<\/key><string>13117<\/string>\r\n    <\/dict>\r\n  <\/dict>\r\n<\/dict>\r\n<\/plist>\r\n<\/pre>\n
Load it up and give it a shot:<\/p>\n
pda@paulbook ~ > launchctl load ~\/Library\/LaunchAgents\/my.greeter.plist\r\npda@paulbook ~ > echo \"My name is Paul.\" | nc localhost 13117\r\nHi Paul, happy Friday!\r\n<\/pre>\n
launchd as god<\/a>!<\/h2>\n
You can use launchd to ensure a process stays alive forever using <key>KeepAlive<\/key><true\/>, or stays alive under the following conditions.<\/p>\n