Readme

Read me

Xhooks

http://www.macos.utah.edu/xhooks

Xhooks is a collection of OS modifications, scripts, and utilities that make it easy to configure and automate maintenance on Mac OS X computers.  It is intended to work with and automate radmind.  If you are not using radmind, you still may find Xhooks useful because it contains things that aren't available elsewhere (or aren't easily available like hping2, SetDisplay, and Screen Preserver) so you may want to borrow pieces.

The main author and user of Xhooks is the University of Utah Student Computing Labs Mac Group, "us".  In that respect, it works great for us and what we distribute is tailored for our computers.  We are deploying this right now even though it is technically alpha or beta or whatever.  We are aware there are parts that do not work (2009.04.27).  We think those parts aren't vitally important.  But the point is, this is beta.  But in production.

We ***still*** haven't updated much documentation other than this file and Installing.txt.  We are very sorry for this.  We are more than willing to email or iChat people who want to try to use this, so please don't hesitate to contact us if you have questions.

################################################################

THE FOUR PARTS OF XHOOKS


HOOKING INTO THE OS
-------------------
The first major feature is hooking into Mac OS X.  This functionality allows the administrator to run scripts or utilities at startup before the loginwindow appears, login, logout, and right after radmind runs.  This part of Xhooks is actually very small but rather critical to the whole thing.  For more details, see the section "System hooks details" below.


MODULES
-------
The second major feature is the scripts and utilities, or as we call it, the "modules".  Here is a description of what some of the major modules do.
- Automate running Radmind and logging (including many radmind tools to help manage overloads)
- Screen Saver over the loginwindow
- Manage home folders for users in a lab or guest environment (this is easily turned off for staff)
- Customize the loginwindow
- Set display resolution based on machine model, set desktop picture based on resolution
- Turn Mac OS X computers into fully automated and self-maintaining Safari or PowerPoint kiosks
- Manage prefs not easily managed by Radmind

There really isn't documentation on the included modules right now other than the scripts themselves and a file named "Readme Modules.txt" that was written by someone who didn't know what all the scripts did and it is incomplete anyway.



PREFERENCE MANAGEMENT
---------------------
The third major feature is really the feature "Manage prefs not easily managed by Radmind".  Radmind manages files all or nothing.  Xhooks includes scripts that can change parts of a pref file.  Xhooks includes some modules that configure specific preferences and it also includes the ability to tweak with any plist file generically like the defaults command but with more power (using the super_defaults tool).

Here is a scenario that demonstrates a problem that often occurs when using Radmind on Mac OS X.

If you install Xcode it will modify /etc/authorization.  If you modify that file yourself (like if you want to allow a non-admin user to set the DVD region code the first time a DVD is inserted) and you distribute that modification in a different overload, you now have 2 overloads that are fighting each other.  If the Xcode overload is listed after your overload, Xcode wins and your non-admin users will not be able to set the DVD region code the first time.  If Xcode is listed before your overload, your overload wins and parts of Xcode may not work correctly because the changes the installer made to /etc/authorization are not on your computers.

Another example is Safari preferences.  On some computers you may want to block pop-ups.  On others you may want to allow them.  Instead of creating 2 overloads, you can turn that feature on and off with this line included in a command file for a computer:

# SUPER_DEFAULTS += /Users/template/Library/Preferences/com.apple.Safari.plist WebKitJavaScriptCanOpenWindowsAutomatically -b 0

Xhooks is designed to remove this headache by modifying the files after they are distributed to the computers.

Here are some of the files or features that can be managed with Xhooks
- Any plist file (using the superDefaults tool)
- The firewall (ipfw, not the 10.5 application firewall)
- /etc/cups/printers.conf (which printers are installed)
- Certificates (on 10.5 the /Library/Keychains/System.keychain file, on 10.4 /System/Library/Keychains/X509Anchors and X509Certificates)
- Global or user Library/Preferences/loginwindow.plist (specifies what runs at login as set by the System Preferences Accounts pane)
- /Library/Preferences/com.apple.SystemLoginItems.plist (a 2nd way of specifying what launches at login, used by the Wacom tablet driver and Timbuktu)
- /Library/Preferences/SystemConfiguration/preferences.plist (AppleTalk and network proxy settings)
- ~/Library/Preferences/com.apple.systemuiserver.plist (what is shown in the system menu bar, like time, volume, displays, etc)
- The LaunchDaemon plist that specifies when Xhooks runs radmind at night
- /etc/crontab
- /etc/hostconfig (on 10.4 only)

Basically, if you create an overload of some software and you notice it modifies any of these files, instead of distributing that file in an overload, you find the modification that is made and you distribute the modification with Xhooks instead.


XHOOKS COMMAND FILE CONFIGURATION
---------------------------------
The last major feature of Xhooks is how the above configurations are distributed and how you manage Xhooks in general.  Xhooks is configured 3 ways.  First, by using a transcript of links that point to scripts that should be executed ("l ./Library/Xhooks/Modules/xhooks/hooks/SEE_run_me /path/to/SEE_run_me").  Second, by using "commented out command files" that contain Xhooks settings.  Third, by putting variables in machine command files that are made available to any script or took that reads the Xhooks prefs, basically giving bash, perl, and any tool that can read a plist file the ability to read the variables.

The first configuration part is how the Xhooks "hooks" decides what to run at startup, login, logout, etc.  This is managed with "hook" transcripts.  Without hook transcripts, Xhooks will do *nothing*.  The scaffold will run, but they will do nothing.  A hook transcript puts links in a specific folder and those links point to scripts to run.  That is how Xhooks decides what to run and when to run it.  For more information see the section "The hook launcher script details" below.

The second part is "commented out command files".  These are technically radmind command files but they are commented out so radmind downloads them to clients but does nothing with them.  Xhooks uncomments them and moves to a specified location.  This is really just a round about way to distribute a file outside of an overload.  Thus there is no checksum and it is downloaded with ktcheck instead of lapply.  And you can edit them freely on the Radmind server because they have no checksum associated with them.  Xhooks will uncomment them and copy them to some other location by including this line:

## Begin uncomment and save to /Library/Preferences/Xhooks/edu.purdue.ics.tjohnson.printerSetup/10.5/printerDefinitions.txt


This way you can use this file as a "config" file on the radmind server.  You can change it and not have to update a checksum (which is required if you use overloads for configuration).  Printers, /etc/crontab, when radmind runs at night are examples of these files.

Finally we have our favorite configuration feature.  After radmind runs, the machine's command file and all included command files are searched for Xhooks variables.  Inside of a command file, you can create variables using this syntax:

# SOME_VARIABLE = value

Or you can create arrays with:

# SOME_ARRAY += value1
# SOME_ARRAY += value2

"SOME_VARIABLE" becomes a variable that is instantly usable in any script that includes the Xhooks config file.  For more information see the section "Modules" below.

Another form is to include the IP or IP range in parenthesis before the actual variable name.  For example  this will assign different values different computers based on their IP. (This particular variable can actually be stored in /var/radmind/config--see the very end of the Installing.txt file.)

# (10.0.1.1-32) NIGHTLY_RADMIND_REBOOT = 30    1    U    -
# (10.0.1.33-64) NIGHTLY_RADMIND_REBOOT = 0    2    U    -

Nearly all of the Xhook module scripts use this mechanism for all configuration.  This makes it easy to set variables only on the machines you want using Radmind's command file in a command file feature (kinks) or using IP's.  I have included sample kinks so you can see how this works.

The SUPER_DEFAULTS example above is an example of this type of configuration.

################################################################

LICENSING

See the included License file.

For third party software, please see the Licenses folder.

################################################################

INSTALLING

Xhooks is intended to work with the Radmind file system software and it is intended to be "installed" by including the Xhooks overloads on your computers.  It is not intended to be installed with Installer.app or copied by the Finder or cp or a script.  Radmind isn't "required", but you are on your own if you try to install it without radmind.  The files in this download DO NOT HAVE CORRECT PERMISSIONS.  The radmind transcripts list the correct permissions.  It is possible to download it out of context with the script /usr/local/bin/download_overload.pl (part of the xhooks_core.T overload).  Or if you really want you can pester the Xhooks author and he might try to accommodate you.

Detailed instructions on how to install are in the "Installing.txt" file.  It discusses what to do with everything inside of "xhooks (radmind server)" except for some sample stuff that I'll discuss now. The xhooks stuff is inside of folders named "xhooks".  The sample stuff is not, which inludes "config", "command/all.K", "command/nosync", "command/stable", and "command/tech".  Everything else is xhooks.

Because Xhooks now uses Radmind command files for configuration, we are including these sample command files.  We are also using our own radmind directory structure in the examples, so be aware that it doesn't have to be this complex.  But if your setup gets complex, using a directory scheme actually will make your life so much easier.

I should give a warning.  We experiment and change our command folder and kink scheme periodically, and making Xhooks publicly available forces us to try to make it make sense.  So we make a lot of changes and the public package we give out is not being used in production (you hardly want to see every command file or overload we deploy) so it hasn't been tested and there might be typos or changes that haven't been updated.

Anyway, inside of "xhooks (radmind server)/command/" you will see:

all.K
nosync
stable
tech
xhooks

"xhooks (radmind server)/command/xhooks" contains configuration command files used by Xhooks.  They are basically commented out files with this as the first line:

## Begin uncomment and save to /some/path

With that as the first line, Xhooks basically uncomments everything in the file and saves it to the path specified.  If you include "## End uncomment and save" before the end of the file, then it will stop saving.  "cron_settings.conf", "printerDefinitions_10.5.conf", and the files in "local_mcx" all use this mechanism.

This folder "xhooks (radmind server)/command/xhooks" contains one file that is a convenience, "download_overload.pl".  This is basically the script that is part of the xhooks_core.T overload, but here it is commented out.  I include it as a command file because it is the easiest way to get the script to a box that does not have Xhooks.  Install Radmind, include this in the command file for the machine, run ktcheck and you have the script on the box.  The second line of the script is a command that will uncomment and move the file to /usr/local/bin.  You can then run that script to download the rest of Xhooks or whatever you want.  Kinda complex, but it has made my life so much easier when dealing with computers that doesn't have Xhooks installed.  The Installing.txt file actually discribes using this method to download Xhooks to a client.

Another "convience" file in "xhooks (radmind server)/command/xhooks" is "radmind_prerun_tasks.conf".  This is a commented out script.  Basically, the Xhooks run_radmind.pl script uncomments this script and then runs it *before* it tries to run radmind.  This is very useful if you want to do something very important before radmind runs next.  Because the run_radmind.pl script is already on a client, you can't really update it.  So this allows you to push out a script instantly and have it run the very next time computers run radmind.  In it is mostly commented out code examples or things that we have used to save our butts, like case changes (not sure if that code works) or path changes.  Note, "commented out" lines in a "commented out" file have 2 pounds: "##".  Note, this script doesn't have to be perl either.  It just has to be a text file that is commented out.  If it fails to run because of errors, it wont stop the radmind run.

Finally, there is the issue of the "nosync", "stable" and "tech/james" folders.  When we upgraded from 10.3 to 10.4 we learned many hard lessons.  First, we needed to have a way to have multiple sets of command files so that we could have production machines and test machines.  That is essentially what "stable" and "tech/james" are.  Within each of those 2 locations, there is a "sync" folder.  That "sync" folder is how we actually manage everything.  Individual command files just point to overloads in one or the other sync folder.  We keep the 2 "sync" folders synced up using BBEdit's "Find Differences" feature.  We actually have about 8 different locations that we sync in our environment, stable, guineapigs (our staff is forgiving of errors, so they are actually our final test bed before making it totally public), and 6 tech/test locations.  The 6 tech locations are usually a mess as our tech staff make lots of changes to their command files.  Anyway, this really isn't a feature of Xhooks, just a suggestion how to have multiple stages of deployment and how to scale Radmind.  The contents of the sync folder really should be organized but we haven't decided yet how we want to organize ours so we are just giving you a non-organized sync folder.

################################################################

System hooks details

There are several system files that allow administrators to run things at specific times.  This is important because this should help you understand what gets executed and why.  It is very similar to tracing the in and out cables on a very complex TV/DVD/VCR/stereo/game console system with multiple audio and video ins and outs.

These files are part of the xhooks_on_10.5.T and xhooks_on_10.4.T transcripts (with one exception described below).

================
Login and Logout:

/var/root/Library/Preferences/com.apple.loginwindow.plist

The modifications to this file will run these scripts:

/Library/Xhooks/Modules/xhooks/bin/login.hook
/Library/Xhooks/Modules/xhooks/bin/logout.hook

=============
Startup Early (before the loginwindow appears)

10.4
/private/etc/rc.local

10.5
/System/Library/LaunchDaemons/com.apple.loginwindow.plist

We change this file to run our script instead of the loginwindow, at the end of our script we start the loginwindow...  Yes we know it is a major hack and could lead to a blue screened Mac, but it works well.

The modification will run this script:

/Library/Xhooks/Modules/xhooks/bin/startupearly.hook

============
Startup Late (after the loginwindow appears)

/Library/LaunchDaemons/edu.utah.scl.startuplate.plist

Runs this script:

/Library/Xhooks/Modules/xhooks/bin/startuplate.hook

It should be noted that startuplate.hook *waits* for startupearly.hook to finish using a trigger file (called the baton hahaha)

================
Post Maintenance:
This is not a part of either xhooks_on_10.5.T or xhooks_on_10.4.T but is hook created in a hook transcript:

l ./Library/Xhooks/Modules/xhooks/hooks/SEE_2_run_postmaintenance.pl ../../edu.utah.scl.radmind/SEE_2_run_postmaintenance.pl

It runs this script:

/Library/Xhooks/Modules/xhooks/bin/postmaintenance.hook

===============
Home idlescript:

/Library/LaunchAgents/edu.utah.scl.homes_idlescript.plist

This runs when the machine is logged out and runs this script:

/Library/Xhooks/Modules/edu.utah.scl.home_folder/homes_idlescript.pl

===========
Screensaver:

/Library/LaunchAgents/edu.utah.scl.logoutsaver.plist

This runs when the machine is logged out and runs this script:

/Library/Xhooks/Modules/edu.utah.scl.logoutsaver/logoutsaver.pl

===========
Idle script:

/Library/LaunchDaemons/edu.utah.scl.idlescript.plist

Runs this script 100% of the time:

/Library/Xhooks/Modules/edu.utah.scl.idle/IS_idlescript.pl

I am trying to get rid of this file and right now (2008.08.05) the idlescript doesn't actually do anything *at all*.

===========
Watchfolder:

/Library/LaunchDaemons/edu.utah.scl.watchfolder.plist

Runs this script when files in /private changes (should never happen unless the admin does it...):

/Library/Xhooks/Modules/edu.utah.scl.idle/watchFolder.pl

=======
Nightly:

/Library/LaunchDaemons/edu.utah.scl.nightly.plist

This file is created by /Library/Xhooks/Modules/edu.utah.scl.nightly/PMD_create_nightly_launchd.pl (which is launched by the post maintenance hook)

This file launches:

/Library/Xhooks/Modules/edu.utah.scl.nightly/nightly.pl

################################################################

The hook launcher script details

These are the hook launcher scripts:

/Library/Xhooks/Modules/xhooks/bin/login.hook
/Library/Xhooks/Modules/xhooks/bin/logout.hook
/Library/Xhooks/Modules/xhooks/bin/postmaintenance.hook
/Library/Xhooks/Modules/xhooks/bin/startupearly.hook
/Library/Xhooks/Modules/xhooks/bin/startuplate.hook

They have some functionality that I'm not sure was ever important to add.  Mainly, they use these scripts:

/Library/Xhooks/Modules/xhooks/bin/lazy_ihook.sh
/Library/Xhooks/Modules/xhooks/bin/pidcheck.sh
/Library/Xhooks/Modules/xhooks/bin/runhooks.sh

But don't worry about them.  Essentially, the hook launcher scripts' purpose is to execute something in the following folder.

/Library/Xhooks/Modules/xhooks/hooks

(I really should have put that in /Library/PreferencesXhooks/hooks but I didn't.  Maybe I'll change it at some later date, I don't know.)

These files can really be anything, executables, scripts, but we use a "hooks" transcript to create links that point to the scripts we really want to execute.  Either way, the filenames must begin with some combination of what is below in order to be executed.

First 2 chars:
SE - Startup Early
SL - Startup Late
LO - Logout
LI - Login
PM - Post Maintenance

The 3rd char:
E - Early
D - Detach
L - Late

So you can have any of these combinations:

SEE
SED
SEL
SLE
SLD
SLL

...etc.

The early scripts will run in order of the name.  So SEE_1 runs before SEE_2.  The detached scripts will run all at once after the last early runs.  So SED_1 and SED_2 run at the same time (after all SEE* scripts).  The late scripts will run in order of the name after all detach scripts finish. So SEL_1 runs before SEL_2.

The login and logout hooks pass the username of the user logging in our out.  So all LO* and LI* scripts can tell who is logging in or out in the first parameter (ARGV[0] in Perl, $1 in shell).

I know, I know, yet another launch mechanism.  This is actually a modified scheme from an old Unix OS.  Launchd is great, but it just doesn't give us the granularity we need.  Also, login and logout hooks have nothing to do with Launchd.

I am including an old pdf in the "docs" folder named "Entman.pdf".  It shows hook launcher scripts are executed and what they execute.  Besides the paths being incorrect, it is still correct for 10.4 (10.5 is slightly different).

You should now know enough to be able to trace when and how stuff gets executed.  Just look in /Library/Xhooks/Modules/xhooks/hooks. I have included the hook transcripts I am using right now.  They all create links from /Library/Xhooks/Modules/xhooks/hooks to /Library/Xhooks/Modules/somename/somescript.  So at that point the module is responsible for its own thing.

################################################################

Modules

Modules are located in /Library/Xhooks/Modules/ and as is described above, you "hook" a module up by creating a link in /Library/Xhooks/Modules/xhooks/hooks that points to the script or executable you want to run.

After that point the module can pretty much do whatever it wants.  All scripts run as root, so it really can do whatever it wants.

Preferences or files that need to be customized for each institution should be placed in /Library/Preferences/Xhooks/module_name.  The files in /Library/Xhooks/Modules/ should be global for any admin.

There are 2 things that Xhooks provides as a convenience to modules.

First is the command file variables.  3 files are created with the variables, one for Perl, one for shell, and finally a plist file for Ruby, Python, or Cocoa. These variables are accessed by including the correct conf file in a Perl or Shell script or by reading the plist file in Ruby, Python, or Cocoa.  A very detailed description how to do this is in the file:

/Library/Xhooks/Modules/xhooks/bin/radmind_xhooks_conf.pl

The second module assistant is the library files for Perl and shell located in /Library/Xhooks/Modules/xhooks/lib.  There are many functions that are used over and over (like find the machines IP).  The perl library also includes the command file variables so you do not need to.  There is no longer a bunch of strange Perl includes.

For examples of all of this, just check out the included modules.

There is some interdependency between the included modules, but I have tried to eliminate it all.  Each module should be self contained and can just be thrown away if you don't want to use it.  You can create your own as well and we welcome any additions.

There really isn't documentation on the included modules right now other than the scripts themselves and a file named "Xhooks Modules.txt" that was written by someone who didn't know what all the scripts did and it is incomplete anyway.

################################################################

History

Version 18 (2009.06.04)
- Lots of changes.  If I try to list them I'll never get this posted.

Version 7 (2008.08.05)
- Updated Screen Preserver to .52 (somehow the old version was in use)

Version 6 (2008.08.04)
- Changes to prefs only (not visible to public)

Version 5 (2008.08.01)
- Enabled Post Maintenance iHook so that post maintenance now has a GUI
- Changes to prefs (not visible to public)

Version 4 (2008.07.22)
- edu.utah.scl.home_folder: added debugging to assimilateUser.pl and got it to work.  Also fixed create_cached_home_folders.pl and create_lost_and_found.pl.  Added little fixes to all the other home folder scripts.  Created homes_idlescript.pl and a launchd that runs when logged out.  It creates the homes (thus removing the functionality from the idlescript finally)
- Includes iHook 1.2.0
- logoutsaver.pl - changed the sleep timer to check to see how long until the idle time comes
- PMD_buildKeychains.sh - updated for 10.5
- PMD_manage_system_menubar_prefs.pl: switched it to use command file variable SYSTEM_MENUBAR_ITEMS
- printerSetup.pl: switched it to use my get_ip() because the original version didn't work with machines with airport
- Split PMD_network.pl into PMD_set_appletalk.pl and PMD_set_proxy.pl

Version 3 (2008.07.11)
- common.pl, config.pl, config.sh - fixed paths, cleaned up
- display_conf.pl - Added iMac7,1
- launch_maintenance (called by iHook) - It now runs ktcheck and radmind_xhooks_conf.pl before fully running radmind.  Because of this it is now possible to make changes to command files and the changes will be picked up on the next radmind run (they are instant)
- edu.utah.scl.display - Added launchagent edu.utah.scl.display_resolution.plist and scripts load_launchagent.pl and set_display.pl so that the display resolution can be set at startup
- Added /usr/local/bin/xktcheck to auto run ktcheck using the Xhooks settings
- Added /usr/local/bin/single_user_radmind, which sets up single user mode so radmind can be run

Version 2 (2008.07.03)
- Replaced idlescript with a stub file that does nothing (but does not quit and thus does not fill system.log with launchd spawn messages)
- SED_find_errors.pl - changed radmind error log reporting; added emailing radmind error to RADMIND_ERROR_EMAIL
- SEL_9_set_computer_name.pl - if a computer has radmind error, the computer name will have an "_" after it.
- Added file_emailer.pl (used by SED_find_errors.pl)

Version 1 (2008.06.06)
- Converted Entman to Xhooks.  Changed all the paths.  Entman already had tons of changes from the last publicly released version...  I doubt I'll ever list them all.