diff --git a/Changelog b/Changelog index 5f75484..bd491bc 100644 --- a/Changelog +++ b/Changelog @@ -1,4 +1,10 @@ -02-17-2006 - metche 1.0 +1.1 (2006 09 16) + + * ververs: added Linux VServers support. + * EXCLUDES: added a few everchanging Postfix files, and AlternC's + local.sh + * documentation: clear up the way backups are made (Closes: #365694) + +1.0 (2006 02 17) * Initial release. - diff --git a/FAQ b/FAQ new file mode 100644 index 0000000..12a1b4e --- /dev/null +++ b/FAQ @@ -0,0 +1,86 @@ +,------------------------------------------------------------------------------- +| Frequently asked0 questions about metche +`------------------------------------------------------------------------------- + +1. How are the monitored Changelog files ($CHANGELOG_FILE or + $CHANGELOG_DIR/*/Changelog) generated? + + With Emacs or Vim. They are written by *you*! They are an important + part of the collective sysadmin process metche is supposed to simplify. + +2. How is metche working, and what are the underlying concepts I have to + understand? + + When called with the cron command line switch, metche verifies changes in + the system state during the last $TESTING_TIME minutes. If any changes + took place an "unstable state" is saved. Otherwise, a "testing state" is + saved and a report is emailed to you. + + A mechanism exists to automatically turn a "testing state" into + a "stable" one. See metche(8) for explanations. + +3. How do I see the saved states list? + + Run "metche list". + +4. I've broken my system, how can I see a report against a previous, system + state, known as working? + + Run "metche report [{stable,testing,unstable}-YYYYMMDDHHMM]". + If no saved state is specified, the most recently saved "testing state" + is used as reference. + +5. How do I create a "stable state" manually? + + Run "metche stabilize [testing-YYYYMMDDHHMM]". + This turns the given "testing state", if specified, otherwise the one most + recently saved, into a "stable state". + +6. Why is metche filling my /var partition? + + First, metche needs to make backups of your "system state" in order to be + able to provide you with changes reports. Whatever you do, metche *will* use + some disk space (a few dozens megabytes on a typical server). Please note + metche performs full, and not incremental, backups. + + But there are a few ways to limit the disk space metche uses. Here is + the cookbook. + + a) Exclude some big files from the set monitored by metche. + - Have a look to your metche backup files: + # ls -l /var/lib/metche/*.tar.bz2 + - If the average size for these files is greater than a few megabytes, + you should investigate which file or directory (in $WATCHED_DIR) is + guilty and use the EXCLUDES option to keep it out of metche's hands. + + b) Speed up the mechanism that automatically turns an "unstable state" into + a "testing state", and a "testing state into a "stable" one. + - First, let metche run for a while with its default TESTING_TIME and + STABLE_TIME configuration values (the "for a while" meaning depends on + how much nervous you get when you run 'df /var' every five minute; + a few days is a bare minimum, a few weeks is better). + - Then, have a look to the files metche has kept in $BACKUP_DIR; a simple + 'metche list' is enough if you're not monitoring any VServer. + - If there is a huge list of "unstable" but only few "testing" states, + you can try lowering TESTING_TIME. If there are many more "testing" + states than "stable" state, you can try lowering STABLE_TIME. + It's better to only tweak one of these two parameters at a time. + - Let metche run "for a while" again... and iterate. + + c) Still despaired? + If this does not work for you, you might also have found a weird case that + metche does not handle well, call it a bug or whatever you want; you + should actually e-mail us (metche AT lists DOT riseup DOT net) the output + of 'metche list', your configuration file (stripped from private + information), and some information about the metche version you're running. + +7. metche does not send us any report + + It could be an e-mail problem, but, more probably, one of your installed + programs violates the FHS and writes one or more ever-changing files in /etc + instead of /var. This prevents metche of turning any "unstable" state into + a "testing" state, and therefore to send you any report. + You can diagnose this by asking metche to generate a report against a recent + unstable state, such as '# metche report unstable-latest'. If the output + contains changed files you don't want to monitor, such as status files, add + them to your EXCLUDES option in metche.conf, and wait. diff --git a/INSTALL b/INSTALL index 8dbb1cc..0eda5a5 100644 --- a/INSTALL +++ b/INSTALL @@ -2,9 +2,8 @@ | INSTALLATION `------------------------------------------------------------------------------- -1. Have a look to the REQUIREMENTS section in the README file. +1. Have a look at the REQUIREMENTS section in the README file. 2. Copy the metche executable to /usr/local/sbin/ -3. As root, mkdir /var/lib/metche -4. Copy the manpage (metche.8) to /usr/local/man/man8/ -5. Read the README file and the manpage +3. Copy the manpage (metche.8) to /usr/local/man/man8/ +4. Read the README file, the FAQ and the manpage diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..b324467 --- /dev/null +++ b/Makefile @@ -0,0 +1,17 @@ +all: doc + +metche-manpage.xml: metche-manpage.sgml + sgml2xml -xlower -xid $< > $@ + +metche.8: metche-manpage.xml + db2x_xsltproc -s /usr/share/sgml/docbook/stylesheet/xsl/nwalsh/manpages/docbook.xsl $< + +doc: metche.8 + +clean: + rm -f metche-manpage.xml + +distclean: clean + rm -f metche.8 + +.PHONY: all doc clean distclean diff --git a/NEWS b/NEWS new file mode 100644 index 0000000..7e2794d --- /dev/null +++ b/NEWS @@ -0,0 +1,6 @@ +metche 1.1 + + * configuration: report e-mails' subject is now automatically prepended + by the hostname of the monitored system; you may have to update your + _MAIL_SUBJECT configuration in /etc/metche.conf, if you have + customized it. diff --git a/README b/README index ed8df8a..f01ad7d 100644 --- a/README +++ b/README @@ -1,5 +1,6 @@ metche - reducing root bus factor -homepage : https://poivron.org/dev/metche/ +homepage: https://poivron.org/dev/metche/ +dev team: metche AT lists DOT riseup DOT net ,------------------------------------------------------------------------------- | Copyright (C) 2004-2006 boum.org collective - property is theft ! @@ -22,46 +23,49 @@ | OVERVIEW `------------------------------------------------------------------------------- -metche is a tool meant to facilitate collective sysadmin ; basically, it +metche is a tool meant to ease collective system administration. It does periodically : - - saves the "system state" to $BACKUP_DIR (default /var/lib/metche), i.e. : + - save the "system state" of . $WATCHED_DIR (default: /etc) . $CHANGELOG_FILE (default /root/Changelog) or $CHANGELOG_DIR/*/Changelog (default: /root/changelogs/*/Changelog) . Debian packages states and versions (using apt-show-versions) - - sends you, in a nicely formated email, the last changes to the "system - state"; see https://poivron.org/dev/metche/ for an example. + to $BACKUP_DIR (default: /var/lib/metche). + - send a nicely formatted email to a defined email address, listing the last + changes that have been made to the system. + See https://poivron.org/dev/metche/ for an example. ,------------------------------------------------------------------------------- | BASIC USAGE `------------------------------------------------------------------------------- -When installed and configured, metche is run by a cronjob, and you just have to -read the report emails. Of course, it's not useful at all if you don't : - - set $EMAIL_ADDRESS config variable to your sysadmin collective mailing-list - address ; +When installed and configured, metche is run by a cron job. You just have to +read the report emails. Of course it's not useful at all if you don't: + - set the $EMAIL_ADDRESS config variable to your sysadmins' collective + mailing-list or email address; - use the Changelog files in a rigorous way. -Note: it is dangerous to use metche without before reading the SECURITY section -of the manpage. +Note: It is dangerous to use metche without reading the SECURITY section +of the manpage before. -For a deeper explanation of the way metche works, read the metche(8) manpage. +For further explanation of the way metche works, read the metche(8) manpage and +the FAQ file. ,------------------------------------------------------------------------------- | REQUIREMENTS `------------------------------------------------------------------------------- - * Debian GNU/Linux and apt-show-versions (if Debian packages monitoring is - enabled) + * Debian GNU/Linux and apt-show-versions (if Debian package monitoring is + enabled) * mutt * bzip2 - * If you want metche to encrypt the email it sends you : gnupg - * If you want metche to monitor your vservers as well : util-vserver tools + * If you want metche to encrypt the email it sends: gnupg + * If you want metche to monitor your vservers as well: util-vserver tools * If you want metche to monitor one manually written Changelog file, it must be located at $CHANGELOG_FILE. _or_ If you want metche to monitor multiple manually written Changelog - files, they must be located in $CHANGELOG_DIR/*/Changelog ; - for example, you can have : + files, they must be located in $CHANGELOG_DIR/*/Changelog; + for example this way: /root/ changelogs/ common/Changelog @@ -72,7 +76,7 @@ | INSTALLATION `------------------------------------------------------------------------------- -See the included INSTALL file. +Please see the included INSTALL file. ,------------------------------------------------------------------------------- | CONFIGURATION @@ -81,50 +85,42 @@ 1. Copy metche.conf.default to /etc/metche.conf and edit it so that it suits your needs. 2. Read the next sections of this document and the metche(8) manpage. -3. Add to /etc/cron.d/metche something like : +3. Add to /etc/cron.d/metche something like: 0-59/5 * * * * root test -x /usr/local/sbin/metche && \ /usr/local/sbin/metche cron + See the cron (8) manpage for further explanations on how to create a cron + job. ,------------------------------------------------------------------------------- -| SECURITY : BIG FAT WARNING +| VSERVERS SUPPORT +`------------------------------------------------------------------------------- + +Just add one (or more) VServer name to the VSNAMES configuration variable, and +metche will monitor it in addition to the host system: + - the WATCHED_DIR, CHANGELOG_DIR and CHANGELOG_FILE paths are also used, + relatively to your VServer's root directory; + - the "system state" backups will be saved in sub-directory in BACKUP_DIR's, + called the same as the VServer; + - EMAIL_ADDRESS is not used for the VServers: the report messages will be + sent to the your VServer's root email address; if you're using + ENCRYPT_EMAIL to encrypt the report messages, make sure their public key is + in the host system root's keyring, trusted enough to be used blindly by + metche; + - all other configuration variables are used as it. + +Most of metche commands (namely: list, report and stabilize) support a +'-h ' option: + - if -h is specified, metche only operates on the VServer provided as an + argument; + - else, metche only operates on the host system. + +Troubleshooting: in case you have a not-purely-Debianish VServer setup, you may +have to customize some of the VServers-related configuration variables. See the +comments in metche.conf for more details. + +,------------------------------------------------------------------------------- +| SECURITY WARNING `------------------------------------------------------------------------------- Read the SECURITY section of metche(8). Really. -,------------------------------------------------------------------------------- -| FAQ -`------------------------------------------------------------------------------- - -1. How are the monitored Changelog files ($CHANGELOG_FILE or - $CHANGELOG_DIR/*/Changelog) generated ? - - With Emacs or Vim. Ah ah. They are written by *you* ! They are an important - part of the collective sysadmin process metche is supposed to facilitate. - -2. Hey, how is metche working, and what are the underlying concepts I have to - know ? - - When called with the "cron" command line switch, metche looks if the "system - state" has changed in the last $TESTING_TIME minutes. If it is the case, a - "unstable" state is saved. Otherwise, a "testing state" is saved, and a - report is emailed to you. - - A similar mechanism to automatically turn a "testing" state into a "stable" - one ; see metche(8) for explainations. - -3. How do I see the saved states list ? - - Run "metche list". - -4. I've broken my system, how can I see a report against a previous, known - working, system state ? - - Run "metche report [{stable,testing,unstable}-YYYYMMDDHHMM]". - If no saved state is specified, the latest "testing state" is used as - reference. - -5. How do I create a "stable state" by hand ? - - Run "metche stabilize [testing-YYYYMMDDHHMM]". - This turns the given testing state, if specified, or the latest one, - otherwise, into a "stable state". diff --git a/metche b/metche index 971f0f5..f5f92c0 100755 --- a/metche +++ b/metche @@ -1,5 +1,8 @@ #! /bin/bash # -*- mode: sh; sh-basic-offset: 4; indent-tabs-mode: nil; -*- +# +# SVN version: $Id: metche 176 2006-09-15 15:31:11Z intrigeri $ +# $URL: http://poivron.org/dev/svn/metche/upstream/tags/metche-1.1/metche $ # # metche: reducing root bus factor # Copyright (C) 2004-2006 boum.org collective - property is theft ! @@ -27,35 +30,52 @@ ### display_usage() { - ( echo "Usage: `basename $0` list" - echo " `basename $0` report" \ + ( echo "Usage:" + echo " `basename $0` [-h VSERVER] list" + echo " `basename $0` [-h VSERVER] report" \ "[{stable|testing|unstable}-YYYYMMDDHHMM]" + echo " `basename $0` [-h VSERVER] stabilize [testing-YYYYMMDDHHMM]" + echo "" + echo " With -h, the VServer VSERVER is operated on instead of the host system." + echo "" + echo "Non-interactive usage:" echo " `basename $0` cron" - echo " `basename $0` stabilize [testing-YYYYMMDDHHMM]" echo "" ) >&2 } fatal() { - echo -e "$@" >&2 + echo -e "Fatal: $@" >&2 exit 2 +} + +warning() { + echo -e "Warning: $@" >&2 +} + +debug() { + [ "$DEBUG" != yes ] || echo -e "Debug: $@" >&2 } executable_not_found() { local executable="$1" local software="$2" - local option="$3" - - fatal "$executable not found. Please install $software or turn $option off." -} - -debug() { - [ "$DEBUG" != yes ] || echo -e "debug: $@" >&2 + local dependant_option="$3" + local solution_option="$4" + + if [ -z "$solution_option" ]; then + fatal "$executable not found on `current_system`." \ + "Please install $software or turn $dependant_option off." + else + fatal "$executable not found on `current_system`." \ + "Please install $software, customize $solution_option" \ + "or turn $dependant_option off." + fi } email() { - debug "email $@" - local subject="$_MAIL_SUBJECT : $1" + debug "* email $@ to $EMAIL_ADDRESS" + local subject="`current_system` - $_MAIL_SUBJECT : $1" if [ $ENCRYPT_EMAIL = "yes" ]; then LC_ALL="$LOCALE" gpg --batch --armor --encrypt \ --recipient "$EMAIL_ADDRESS" | @@ -63,6 +83,10 @@ else LC_ALL="$LOCALE" mutt -s "$subject" "$EMAIL_ADDRESS" fi +} + +current_system() { + $VSERVER_EXEC_PREFIX hostname -f } ### @@ -82,12 +106,18 @@ STABLE_TIME="3" EMAIL_ADDRESS="root@`hostname -f`" ENCRYPT_EMAIL="no" -EXCLUDES="*.swp #* *~ *.key ifstate adjtime ld.so.cache shadow* \ - blkid.tab* aumixrc net.enable mtab \ - vdirbase run.rev vdir run.rev" +EXCLUDES="*.swp #* *~ *.gpg *.key ifstate adjtime ld.so.cache shadow* \ + .gnupg blkid.tab* aumixrc net.enable mtab backup.d \ + vdirbase run.rev vdir run.rev \ + prng_exch smtp_scache.pag smtpd_scache.pag \ + smtp_scache.dir smtpd_scache.dir local.sh" LOCALE="C" -_MAIL_SUBJECT="`hostname -f` - changes report" +VSNAMES="" +VSERVERINFO=/usr/sbin/vserver-info +VSERVER=/usr/sbin/vserver + +_MAIL_SUBJECT="changes report" _NO_DEBIAN_PACKAGES_CHANGE="No change in Debian packages state." _NO_CHANGE="No change." @@ -126,18 +156,16 @@ " if [ "$1" = "-h" ]; then - if [ -f /etc/metche/$2.conf ]; then - . /etc/metche/$2.conf - CMD="$3" - MILESTONE="$4" - else - display_usage - fatal "Config file /etc/metche/$2.conf does not exist." - fi -elif [ -f /etc/metche.conf ]; then - . /etc/metche.conf + VSNAME="$2" + CMD="$3" + MILESTONE="$4" +else CMD="$1" MILESTONE="$2" +fi + +if [ -f /etc/metche.conf ]; then + . /etc/metche.conf else display_usage fatal "Config file not found." @@ -150,27 +178,52 @@ unset LANG umask 077 -test -d "$WATCHED_DIR" || fatal "WATCHED_DIR ($WATCHED_DIR) does not exist." -test -d "$BACKUP_DIR" || fatal "BACKUP_DIR ($BACKUP_DIR) does not exist." -test -z "$TAR_OPTS" || fatal "TAR_OPTS is deprecated, use EXCLUDES instead." - -if [ "$DO_PACKAGES" = "yes" ]; then - which apt-show-versions > /dev/null || - executable_not_found "apt-show-versions" "it" "DO_PACKAGES" +# Manage deprecated configuration files and options +test ! -d /etc/metche || fatal "An old configuration directory (/etc/metche/)" \ + "was found, please upgrade your configuration." +test -z "$TAR_OPTS" || fatal "TAR_OPTS is deprecated, use EXCLUDES instead." + +# Backup various configuration values: these non-underscored variable +# names will be re-initialized in context_config() and re-used everywhere. +_WATCHED_DIR="$WATCHED_DIR" +_BACKUP_DIR="$BACKUP_DIR" +_CHANGELOG_DIR="$CHANGELOG_DIR" +_CHANGELOG_FILE="$CHANGELOG_FILE" +_DO_PACKAGES="$DO_PACKAGES" +_EMAIL_ADDRESS="$EMAIL_ADDRESS" + +if [ -n "$VSNAMES" ]; then + # check VSERVERINFO and VSERVER availability + test -x "$VSERVERINFO" || \ + executable_not_found "vserver-info" "util-vserver" "VSNAMES" "VSERVERINFO" + test -x "$VSERVER" || \ + executable_not_found "vserver" "util-vserver" "VSNAMES" "VSERVER" + # check VROOTDIR availability + test -n "$VROOTDIR" || \ + VROOTDIR="`$VSERVERINFO info SYSINFO \ + | grep '^ *vserver-Rootdir' | awk '{print $2}'`" + test -n "$VROOTDIR" || \ + fatal "VSNAMES is not empty, but VROOTDIR could not be guessed." \ + "Please set VROOTDIR in /etc/metche.conf." + test -d "$VROOTDIR" || \ + fatal "VSNAMES is not empty, but VROOTDIR ($VROOTDIR) does not exist." \ + "Please set VROOTDIR in /etc/metche.conf." + # expand VSNAMES if it is set to "all" + if [ "$VSNAMES" = all ]; then + VSNAMES=`ls $VROOTDIR | grep -E -v "lost\+found|ARCHIVES" | tr "\n" " "` + if [ -z "$VSNAMES" ]; then + warning "VSNAMES is set to \"all\", but no VServer could be found" \ + "in VROOTDIR ($VROOTDIR)." + fi + fi fi if [ "$ENCRYPT_EMAIL" = "yes" ]; then which gpg > /dev/null || - executable_not_found "gpg" "GnuPG" "ENCRPYT_EMAIL" - gpg --batch --list-public-keys $EMAIL_ADDRESS >/dev/null 2>&1 || - fatal "GnuPG public key for $EMAIL_ADDRESS not found." + executable_not_found "gpg" "GnuPG" "ENCRYPT_EMAIL" fi DATE=`date "+%Y%m%d%H%M"` -WATCHED_PARENT=`dirname $WATCHED_DIR` -if [ "$WATCHED_PARENT" != '/' ]; then - WATCHED_PARENT="$WATCHED_PARENT/" -fi # How to use $TAR_OPTS: # - $TAR_OPTS should be used unquoted @@ -197,27 +250,157 @@ done set +o noglob -### -### Modules enabling/disabling -### - -DO_CHANGELOGS="no" -if [ "$CHANGELOG_DIR" ]; then - if [ -d "$CHANGELOG_DIR" ]; then - DO_CHANGELOGS="dir" - fi -elif [ -f "$CHANGELOG_FILE" ]; then - DO_CHANGELOGS="file" -fi - -# Debian packages -# Enabled/disabled by $DO_PACKAGES, initialized to "yes", can be -# overriden by the sourced conf file. - ### ### A few functions to do the real work ### + +# Check and mangle the context-dependant configuration variables, +# i.e. the parameters specific to the host system or to a given VServer. +# If $1 is empty, operate on the host system. +# Else, operate on a VServer and return with exit-code: +# - 2 if $1 is not an existing VServer +# - 3 if $1 is not a VServer listed in VSNAMES +# - 4 if $1 is not a running VServer +# Anyway, return with exit-code: +# - 5 if the GnuPG public key can not be found +context_config() { + local vsname="$1" + local res + debug "-------- Operating on" \ + "`if [ -z $vsname ]; then echo 'the host system'; else echo VServer $vsname; fi`" + debug "* context_config" + + # + # Variables & VServer + # + + if [ -z "$vsname" ]; then + WATCHED_DIR="$_WATCHED_DIR" + BACKUP_DIR="$_BACKUP_DIR" + VSERVER_EXEC_PREFIX="" + EMAIL_ADDRESS="$_EMAIL_ADDRESS" + else + WATCHED_DIR="$VROOTDIR/$vsname/$_WATCHED_DIR" + BACKUP_DIR="$_BACKUP_DIR/$vsname" + VSERVER_EXEC_PREFIX="$VSERVER $vsname exec" + EMAIL_ADDRESS="root@`current_system`" + # does the current VServer exist ? + if [ ! -d "$VROOTDIR/$vsname" ]; then + warning " VServer $vsname does not exist (error 2)." + return 2 + fi + # is the current VServer listed in VSNAMES ? + local found="no"; + for i in $VSNAMES; do + if [ "$vsname" = "$i" ]; then + found=yes + break + fi + done + if [ $found = no ]; then + warning " VServer $vsname is not listed in VSNAMES (error 3)." + return 3 + fi + # is the current VServer running ? + res="" + $VSERVERINFO -q "$vsname" RUNNING || res=failed + if [ "$res" = failed ]; then + warning " VServer $vsname is not running (error 4)." + return 4 + fi + fi + + # E-mail encryption + if [ $ENCRYPT_EMAIL = "yes" ]; then + gpg --batch --list-public-keys $EMAIL_ADDRESS >/dev/null 2>&1 + res=$? + if [ $res -ne 0 ]; then + warning " GnuPG public key for $EMAIL_ADDRESS not found." + return 5 + fi + fi + + # + # Files and directories + # + + # Check the existence of WATCHED_DIR + test -d "$WATCHED_DIR" || \ + fatal "$WATCHED_DIR directory (built from WATCHED_DIR) does not exist." + + # Initialize WATCHED_PARENT + WATCHED_PARENT=`dirname $WATCHED_DIR` + if [ "$WATCHED_PARENT" != '/' ]; then + WATCHED_PARENT="$WATCHED_PARENT/" + fi + + # Check the existence of the resulting BACKUP_DIR, creating it if needed. + if [ ! -d "$BACKUP_DIR" ]; then + debug " Creating $BACKUP_DIR directory for `current_system`." + if mkdir -p "$BACKUP_DIR"; then + debug " Successfully created $BACKUP_DIR directory." + else + fatal " Failed to create $BACKUP_DIR directory." + fi + fi + + # + # Modules enabling/disabling + # + + # DO_CHANGELOGS + DO_CHANGELOGS="no" + if [ -n "$_CHANGELOG_DIR" ]; then + if [ -z "$vsname" ]; then + CHANGELOG_DIR="$_CHANGELOG_DIR" + else + CHANGELOG_DIR="$VROOTDIR/$vsname$_CHANGELOG_DIR" + fi + if [ -d "$CHANGELOG_DIR" ]; then + DO_CHANGELOGS="dir" + else + warning " The directory $CHANGELOG_DIR (built from CHANGELOG_DIR)" \ + " does not exist. Changelogs file monitoring thereferore" \ + " cannot be performed this time for `current_system`." + fi + elif [ -n "$CHANGELOG_FILE" ]; then + if [ -z "$vsname" ]; then + CHANGELOG_FILE="$_CHANGELOG_FILE" + else + CHANGELOG_FILE="$VROOTDIR/$vsname$_CHANGELOG_FILE" + fi + if [ -f "$CHANGELOG_FILE" ]; then + DO_CHANGELOGS="file" + else + warning " The file $CHANGELOG_FILE (built from CHANGELOG_FILE)" + warning " does not exist. Changelog file monitoring thereferore" + warning " cannot be performed this time for `current_system`." + fi + fi + + # DO_PACKAGES + DO_PACKAGES="$_DO_PACKAGES" + if [ "$DO_PACKAGES" = "yes" ]; then + $VSERVER_EXEC_PREFIX which apt-show-versions > /dev/null + res=$? + if [ $res -ne 0 ]; then + warning " apt-show-versions not found on `current_system`." + warning " Please install it or turn DO_PACKAGES off." + warning " DO_PACKAGES therefore cannot be performed this time" + warning " for `current_system`." + DO_PACKAGES=no + fi + fi + + # + # Final steps to get a coherent initial status + # + + # Make sure we've got at least one testing and one stable + milestone_exists testing-latest || save_state "testing" + milestone_exists stable-latest || stabilize_state "testing-latest" +} # Returns 0 if, and only if, specified milestone exists. milestone_exists() { @@ -276,7 +459,7 @@ # This will save an archive of the watched directory with the given prefix save_files() { - debug " - save_files $@" + debug " - save_files $@" set -o noglob tar jcf "$BACKUP_DIR/$1-$DATE".tar.bz2 \ -C "$WATCHED_PARENT" $TAR_OPTS `basename "$WATCHED_DIR"` @@ -286,16 +469,16 @@ # This will save packages list with the given prefix save_packages() { - debug " - save_packages $@" - apt-show-versions -i - apt-show-versions | + debug " - save_packages $@" + $VSERVER_EXEC_PREFIX apt-show-versions -i + $VSERVER_EXEC_PREFIX apt-show-versions | sort > "$BACKUP_DIR/$1-$DATE".packages ln -sf "$1-$DATE".packages "$BACKUP_DIR/$1"-latest.packages } # This will save Changelogs with the given prefix save_changelogs() { - debug " - save_changelogs $@" + debug " - save_changelogs $@" local changelog domain file if [ "$DO_CHANGELOGS" = "dir" ]; then @@ -314,7 +497,7 @@ # Save whatever reflect the current state with the given prefix save_state() { - debug "save_state $@" + debug "* save_state $@" save_files "$1" [ $DO_PACKAGES = "no" ] || save_packages "$1" [ $DO_CHANGELOGS = "no" ] || save_changelogs "$1" @@ -322,7 +505,7 @@ # Report changes against given version to standard output report_changes() { - debug "report_changes $@" + debug "* report_changes $@" local tmp tmpdir changelog domain diff tar_diff diff_diff local files old new tmp_packages file @@ -397,9 +580,9 @@ echo "$DEBIAN_PACKAGES_HEADER" >> "$tmp" tmp_packages=`mktemp -q` - apt-show-versions -i - apt-show-versions | sort > "$tmp_packages" - if diff -wEbB "$BACKUP_DIR/$1".packages "$tmp_packages"; then + $VSERVER_EXEC_PREFIX apt-show-versions -i + $VSERVER_EXEC_PREFIX apt-show-versions | sort > "$tmp_packages" + if diff -wEbBN "$BACKUP_DIR/$1".packages "$tmp_packages"; then echo "$_NO_DEBIAN_PACKAGES_CHANGE" fi | grep -v '^[0-9-]' >> "$tmp" fi @@ -429,7 +612,7 @@ # Turns into stable the given testing. # NB: argument validity is supposed to have been already checked. stabilize_state() { - debug "stabilize_state $@" + debug "* stabilize_state $@" local testing stable file dst testing="$1" @@ -506,13 +689,10 @@ ### Main ### -# make sure we've got at least one testing and one stable -milestone_exists testing-latest || save_state "testing" -milestone_exists stable-latest || stabilize_state "testing-latest" - case "$CMD" in report) + context_config "$VSNAME" || fatal "Aborting (error $?)." DO_DETAILS="yes" if [ -z "$MILESTONE" ]; then report_changes "testing-latest" @@ -525,6 +705,7 @@ ;; list) + context_config "$VSNAME" || fatal "Aborting (error $?)." for file in "$BACKUP_DIR"/*.tar.bz2; do echo `basename ${file%%.tar.bz2}` done @@ -532,57 +713,82 @@ cron) STABLE_TIME_MIN=`expr 24 '*' 60 '*' "$STABLE_TIME"` - - ### Algorithm - # - # if (no change happened for TESTING_TIME) then - # if (something has changed since the last testing) then - # send a report against last testing - # save a new testing state - # delete all saved unstable states - # elif (no change happened for STABLE_TIME) then - # if (something has changed since the last stable) then - # save a new stable state and notify EMAIL_ADDRESS - # delete all saved testing states older than STABLE_TIME - # fi - # fi - # elif (last unstable exists) then - # if (something has changed since the last unstable) then - # save a new unstable state - # fi - # else - # save a new unstable state - # fi - if no_change_since "$TESTING_TIME"; then - debug "no change since TESTING_TIME" - if changed_from "$BACKUP_DIR"/testing-latest.tar.bz2; then - debug "changed from testing-latest" - report_changes "testing-latest" | email "testing-$DATE" - save_state "testing" - debug "removing all saved unstable states." - find "$BACKUP_DIR" -name 'unstable-*' -exec rm "{}" \; - elif no_change_since "$STABLE_TIME_MIN"; then - if changed_from "$BACKUP_DIR"/stable-latest.tar.bz2; then - save_state "stable" - echo "metche saved a new stable state : stable-${DATE}." | + if [ -n "$VSNAME" ]; then + display_usage + fatal "-h option not available for 'metche cron'" + fi + + for i in "" $VSNAMES; do + res=0 + context_config "$i" || res=$? + if [ $res -ne 0 ]; then + warning "-------- Ignoring" \ + "`if [ -z $i ]; then \ + echo 'the host system'; \ + else \ + echo VServer $i; fi` (error $res)" + continue + fi + + ### Algorithm + # + # if (no change happened for TESTING_TIME) then + # if (something has changed since the last testing) then + # send a report against last testing + # save a new testing state + # delete all saved unstable states + # elif (no change happened for STABLE_TIME) then + # if (something has changed since the last stable) then + # save a new stable state and notify EMAIL_ADDRESS + # delete all saved testing states older than STABLE_TIME + # fi + # fi + # elif (last unstable exists) then + # if (something has changed since the last unstable) then + # save a new unstable state + # fi + # else + # save a new unstable state + # fi + + debug "* main algorithm" + + if no_change_since "$TESTING_TIME"; then + debug " no change since TESTING_TIME" + if changed_from "$BACKUP_DIR"/testing-latest.tar.bz2; then + debug " changed from testing-latest" + report_changes "testing-latest" | email "testing-$DATE" + save_state "testing" + debug " removing all saved unstable states." + find "$BACKUP_DIR" -name 'unstable-*' -exec rm "{}" \; + elif no_change_since "$STABLE_TIME_MIN"; then + if changed_from "$BACKUP_DIR"/stable-latest.tar.bz2; then + save_state "stable" + echo "metche saved a new stable state: stable-${DATE}." | email "stable-$DATE" - debug "removing all saved testing states older " \ - "than STABLE_TIME ($STABLE_TIME)." - find "$BACKUP_DIR" -name 'testing-*' \ - -ctime +"$STABLE_TIME" -exec rm "{}" \; + debug " removing all saved testing states older" \ + "than STABLE_TIME ($STABLE_TIME)." + find "$BACKUP_DIR" -name 'testing-*' \ + -ctime +"$STABLE_TIME" -exec rm "{}" \; + fi fi - fi - elif milestone_exists unstable-latest; then - if changed_from "$BACKUP_DIR"/unstable-latest.tar.bz2; then - debug "changed from unstable-latest" + elif milestone_exists unstable-latest; then + if changed_from "$BACKUP_DIR"/unstable-latest.tar.bz2; then + debug " changed from unstable-latest" + save_state "unstable" + else + debug " not changed from unstable-latest" + fi + else save_state "unstable" fi - else - save_state "unstable" - fi + + done + ;; stabilize) + context_config "$VSNAME" || fatal "Aborting (error $?)." if [ -z "$MILESTONE" ]; then stabilize_state "testing-latest" elif [ "`milestone_version $MILESTONE`" = "testing" -a \ @@ -595,18 +801,22 @@ ;; test) - milestone_version "stable-200507040202" - milestone_version "testing-latest" - milestone_version "testing-200507030047" - milestone_version "testing-200507030047qsfd" - milestone_date "stable-200507040202" - milestone_date "testing-latest" - milestone_date "testing-200507030047" - milestone_date "testing-200507030047qsfd" - (is_latest testing-latest && echo oui) || echo non - (is_latest testing-200507031821 && echo oui) || echo non - (is_latest stable-200507031831 && echo oui) || echo non - (is_latest stable-200507040202 && echo oui) || echo non + for i in "" $VSNAMES; do + res=0 + context_config "$i" || res=$? + if [ $res -ne 0 ]; then + warning "-------- Ignoring" \ + "`if [ -z $i ]; then \ + echo 'the host system'; \ + else \ + echo VServer $i; fi` (error $res)" + continue + fi + milestone_version "stable-200507040202" + milestone_version "testing-latest" + milestone_date "testing-latest" + #report_changes "testing-latest" | email "testing-$DATE" + done ;; *) diff --git a/metche-manpage.sgml b/metche-manpage.sgml new file mode 100644 index 0000000..f6f721b --- /dev/null +++ b/metche-manpage.sgml @@ -0,0 +1,196 @@ + + + + + + metche + 8 + + + + metche + reducing root bus factor + + + + + metche + -h vserver + + report + + + stable + testing + unstable + -YYYYMMDDHHMM + + + list + stabilize testing-YYYYMMDDHHMM + + + + + + + DESCRIPTION + + metche is a tool meant to ease collective system administration by + monitoring changes in the system configuration. + + metche basic usage is to monitor changes in a directory, usually + /etc ; optionally, metche can also + monitor: + + + one or more user maintained changelog files, + + + the state of Debian packages and versions. + + + + metche should be installed with a cronjob that regularly runs to + automatically save the system state as needed. These states are saved in a + way similar to the Debian development model: + + + unstable states are saved as soon as + a change is detected. They are kept until a new + testing state appears. + + + testing states is created from the last + unstable state that has not been changed + after a short amount of time (by default, one hour). Old + unstable states are deleted afterwards. + + + stable states are created from the last + testing state, either manually, or after a + long amount of time (by default, 3 days). Old testing states are deleted afterwards. + + + + When a new testing state is saved, an email is + sent to a configurable address, giving an overwiew of + the differences with the previous testing. + A notification is also sent when a new stable state is + saved. + + metche's configuration is read from + /etc/metche.conf. Various settings like changelog + monitoring or time between system state switches are described + there. + + + + + OPTIONS + + If -h VSERVER is specified, + the VServer VSERVER is operated on instead of + the host system. This, along with the VSNAMES option, + allows to monitor several VServers running on the system. + + One of the following commands must be specified on the + command line: + + + + report + + When run with the report command, metche + displays a report against the specified saved state, or if unspecified, + against the latest testing state. This is useful when you + have broken your system and want to know which changes have been made + since a given, known working, system state. + + + + list + + When run with the list command, metche + displays a list of all the saved states. + + + + stabilize + + When run with the stabilize command, metche + turns a "testing state" into a "stable state". By default, it will + use the last "testing state", but this can be overriden by giving + a specific state as argument. + + + + cron + + This command should not be called manually, but used from + a cronjob. When called, it can perform various operations like: + saving "unstable", "testing" or "stable" states as needed and + sending reports and notification if configured to do so. This command + does not support the -h option. + + + + + + + + FILES + /etc/metche.conf contains metche configuration. + + + When configured to monitor one changelog, + CHANGELOG_FILE (default: + /root/Changelog). + + + When configured to monitor multiple changelogs, + CHANGELOG_DIR/*/Changelog + (default: /root/changelogs). + + System states are saved in + BACKUP_DIR (default: + /var/lib/metche). + + + SECURITY + metche is able to use GnuPG to encrypt the email it sends, but does + not by default; just enable the ENCRYPT_EMAIL configuration + option, and make sure EMAIL_ADDRESS' public key is in root's + keyring, trusted enough to be used blindly by metche. + + When DO_DETAILS is enabled and + ENCRYPT_EMAIL is disabled, metche sends in clear + text email the changes made to the watched directory... either + make sure that the EXCLUDES configuration variable prevents + it to send sensitive information, or triple check that secure connections + will be used end-to-end on the email path. If unsure, set + EMAIL_ADDRESS configuration variable to a local mailbox. + Please note that EMAIL_ADDRESS is not used for VServers: + a VServer's report messages are sent to its root email address. + + metche stores, in BACKUP_DIR (default : + /var/lib/metche), various backups of + WATCHED_DIR. Make sure that this backup place is at least as + secured as the source. + + + BUGS + See metche's ticket + system for known bugs, missing features, and the development + road-map. + + + AUTHOR + metche and this manual page were written by the boum.org collective + metche@lists.riseup.net. + + + + diff --git a/metche.8 b/metche.8 index a7a6a03..acd6802 100644 --- a/metche.8 +++ b/metche.8 @@ -1,122 +1,158 @@ -.\"Generated by db2man.xsl. Don't modify this, modify the source. -.de Sh \" Subsection -.br -.if t .Sp -.ne 5 -.PP -\fB\\$1\fR -.PP -.. -.de Sp \" Vertical space (when we can't use .PP) -.if t .sp .5v -.if n .sp -.. -.de Ip \" List item -.br -.ie \\n(.$>=3 .ne \\$3 -.el .ne 3 -.IP "\\$1" \\$2 -.. -.TH "METCHE" 8 "" "" "" -.SH NAME +.\" Title: metche +.\" Author: +.\" Generator: DocBook XSL Stylesheets v1.70.1 +.\" Date: 09/16/2006 +.\" Manual: +.\" Source: +.\" +.TH "METCHE" "8" "09/16/2006" "" "" +.\" disable hyphenation +.nh +.\" disable justification (adjust text to left margin only) +.ad l +.SH "NAME" metche \- reducing root bus factor .SH "SYNOPSIS" -.ad l -.hy 0 .HP 7 -\fBmetche\fR {\fBcron\fR | \fBreport\ [{\fBstable\fR\ |\ \fBtesting\fR\ |\ \fBunstable\fR}\-\fIYYYYMMDDHHMM\fR]\fR | \fBlist\fR | \fBstabilize\ [testing\-\fIYYYYMMDDHHMM\fR]\fR} -.ad -.hy - +\fBmetche\fR [\-h\ \fIvserver\fR] {[report\ [\ {[stable]\ |\ [testing]\ |\ [unstable]}\-\fIYYYYMMDDHHMM\fR\ ]] | [list] | [stabilize\ [testing\-\fIYYYYMMDDHHMM\fR]]} .SH "DESCRIPTION" - .PP -metche is a tool meant to facilitate collective sysadmin by monitoring changes in the system configuration\&. - +metche is a tool meant to ease collective system administration by monitoring changes in the system configuration. .PP -metche basic usage is to monitor changes in a directory, usually \fI/etc\fR ; optionally, metche can also monitor: - -.TP 3 +metche basic usage is to monitor changes in a directory, usually +\fI/etc\fR +; optionally, metche can also monitor: +.TP 3n \(bu one or more user maintained changelog files, -.TP +.TP 3n \(bu -the state of Debian packages and versions\&. -.LP - +the state of Debian packages and versions. +.sp +.RE .PP -metche should be installed with a cronjob that regularly runs to automatically save the system state as needed\&. These states are saved in a way similar to the Debian development model: - -.TP 3 +metche should be installed with a cronjob that regularly runs to automatically save the system state as needed. These states are saved in a way similar to the Debian development model: +.TP 3n \(bu -\fIunstable\fR states are saved as soon as a change is detected\&. They are kept until a new \fItesting\fR state appears\&. -.TP +\fIunstable\fR +states are saved as soon as a change is detected. They are kept until a new +\fItesting\fR +state appears. +.TP 3n \(bu -\fItesting\fR states is created from the last \fIunstable\fR state that has not been changed after a short amount of time (by default, one hour)\&. Old \fIunstable\fR states are deleted afterwards\&. -.TP +\fItesting\fR +states is created from the last +\fIunstable\fR +state that has not been changed after a short amount of time (by default, one hour). Old +\fIunstable\fR +states are deleted afterwards. +.TP 3n \(bu -\fIstable\fR states are created from the last \fItesting\fR state, either manually, or after a long amount of time (by default, 3 days)\&. Old \fItesting\fR states are deleted afterwards\&. -.LP - +\fIstable\fR +states are created from the last +\fItesting\fR +state, either manually, or after a long amount of time (by default, 3 days). Old +\fItesting\fR +states are deleted afterwards. +.sp +.RE .PP -When a new \fItesting\fR state is saved, an email is sent to a configurable address, giving an overwiew of the differences with the previous \fItesting\fR\&. A notification is also sent when a new \fIstable\fR state is saved\&. - +When a new +\fItesting\fR +state is saved, an email is sent to a configurable address, giving an overwiew of the differences with the previous +\fItesting\fR. A notification is also sent when a new +\fIstable\fR +state is saved. .PP -metche's configuration is read from \fI/etc/metche\&.conf\fR\&. Various settings like changelog monitoring or time between system state switches are described there\&. - +metche's configuration is read from +\fI/etc/metche.conf\fR. Various settings like changelog monitoring or time between system state switches are described there. .SH "OPTIONS" - +.PP +If +\fB\-h\fR +\fIVSERVER\fR +is specified, the VServer +\fIVSERVER\fR +is operated on instead of the host system. This, along with the +\fBVSNAMES\fR +option, allows to monitor several VServers running on the system. .PP One of the following commands must be specified on the command line: - -.TP +.TP 3n \fBreport\fR -When run with the \fBreport\fR command, metche displays a report against the specified saved state, or if unspecified, against the latest testing state\&. This is useful when you have broken your system and want to know which changes have been made since a given, known working, system state\&. - -.TP +When run with the +\fBreport\fR +command, metche displays a report against the specified saved state, or if unspecified, against the latest testing state. This is useful when you have broken your system and want to know which changes have been made since a given, known working, system state. +.TP 3n \fBlist\fR -When run with the \fBlist\fR command, metche displays a list of all the saved states\&. - -.TP +When run with the +\fBlist\fR +command, metche displays a list of all the saved states. +.TP 3n \fBstabilize\fR -When run with the \fBstabilize\fR command, metche turns a "testing state" into a "stable state"\&. By default, it will use the last "testing state", but this can be overriden by giving a specific state as argument\&. - -.TP +When run with the +\fBstabilize\fR +command, metche turns a "testing state" into a "stable state". By default, it will use the last "testing state", but this can be overriden by giving a specific state as argument. +.TP 3n \fBcron\fR -This command should not be called manually, but used from a cronjob\&. When called, it can perform various operations like: saving "unstable", "testing" or "stable" states as needed and sending reports and notification if configured to do so\&. - +This command should not be called manually, but used from a cronjob. When called, it can perform various operations like: saving "unstable", "testing" or "stable" states as needed and sending reports and notification if configured to do so. This command does not support the +\fB\-h\fR +option. .SH "FILES" - .PP -\fI/etc/metche\&.conf\fR contains metche configuration\&. - +\fI/etc/metche.conf\fR +contains metche configuration. .PP -When configured to monitor one changelog, \fI\fBCHANGELOG_FILE\fR\fR (default \fI/root/Changelog\fR)\&. - +When configured to monitor one changelog, +\fBCHANGELOG_FILE\fR +(default: +\fI/root/Changelog\fR). .PP -When configured to monitor multiple changelogs, \fI\fBCHANGELOG_DIR\fR/*/Changelog\fR (default : \fI/root/changelogs\fR)\&. - +When configured to monitor multiple changelogs, +\fICHANGELOG_DIR/*/Changelog\fR +(default: +\fI/root/changelogs\fR). .PP -System states are saved in \fI\fBBACKUP_DIR\fR\fR (default \fI/var/lib/metche)\fR\&. - +System states are saved in +\fBBACKUP_DIR\fR +(default: +\fI/var/lib/metche)\fR. .SH "SECURITY" - .PP -metche is able to use GnuPG to encrypt the email it sends, but does not by default; just enable the \fBENCRYPT_EMAIL\fR configuration option, and make sure \fBEMAIL_ADDRESS\fR' public key is in root's keyring, trusted enough to be used blindly by metche\&. - +metche is able to use GnuPG to encrypt the email it sends, but does not by default; just enable the +\fBENCRYPT_EMAIL\fR +configuration option, and make sure +\fBEMAIL_ADDRESS\fR' public key is in root's keyring, trusted enough to be used blindly by metche. .PP -In its default setup (\fBENCRYPT_EMAIL\fR configuration option disabled) metche sends in \fIclear text email\fR the changes made to the watched directory\&.\&.\&. either make sure that the \fBTAR_OPTS\fR configuration variable prevents it to send sensitive information, or triple check that secure connections will be used end\-to\-end on the email path\&. If unsure, set \fBEMAIL_ADDRESS\fR configuration variable to a local mailbox\&. - +When +\fBDO_DETAILS\fR +is enabled and +\fBENCRYPT_EMAIL\fR +is disabled, metche sends in +\fIclear text email\fR +the changes made to the watched directory... either make sure that the +\fBEXCLUDES\fR +configuration variable prevents it to send sensitive information, or triple check that secure connections will be used end\-to\-end on the email path. If unsure, set +\fBEMAIL_ADDRESS\fR +configuration variable to a local mailbox. Please note that +\fBEMAIL_ADDRESS\fR +is not used for VServers: a VServer's report messages are sent to its root email address. .PP -metche stores, in \fBBACKUP_DIR\fR (default : \fI/var/lib/metche\fR), various backups of \fBWATCHED_DIR\fR\&. Make sure that this backup place is at least as secured as the source\&. - +metche stores, in +\fBBACKUP_DIR\fR +(default : +\fI/var/lib/metche\fR), various backups of +\fBWATCHED_DIR\fR. Make sure that this backup place is at least as secured as the source. .SH "BUGS" - .PP -See metche's ticket system: \fIhttps://poivron.org/dev/metche/\fR for known bugs, missing features, and the development road\-map\&. - +See +[1]\&\fImetche's ticket system\fR +for known bugs, missing features, and the development road\-map. .SH "AUTHOR" - .PP -This manual page was written by the boum collective \&. - +metche and this manual page were written by the boum.org collective +. +.SH "REFERENCES" +.TP 3 +1.\ metche's ticket system +\%https://poivron.org/dev/metche/ diff --git a/metche.conf.default b/metche.conf.default index 60abcd2..80a5ab4 100644 --- a/metche.conf.default +++ b/metche.conf.default @@ -43,9 +43,11 @@ # - Periods (`.') or forward slashes (`/') are not considered special for # wildcard matches. # Example (default value): -#EXCLUDES="*.swp #* *~ *.key ifstate adjtime ld.so.cache shadow* \ -# blkid.tab* aumixrc net.enable mtab \ -# vdirbase run.rev vdir run.rev" +#EXCLUDES="*.swp #* *~ *.gpg *.key ifstate adjtime ld.so.cache shadow* \ +# .gnupg blkid.tab* aumixrc net.enable mtab backup.d \ +# vdirbase run.rev vdir run.rev \ +# prng_exch smtp_scache.pag smtpd_scache.pag \ +# smtp_scache.dir smtpd_scache.dir local.sh" # Locale (will be used to feed LC_ALL) # Warning: values different from "C" are untested. @@ -56,12 +58,35 @@ ####################################################################### # +# Linux VServers configuration +# + +# VServers to monitor (space-separated list) +# The host system is monitored, whatever VSERVERS is set to. +# Set VSNAMES to "all" in order to monitor all VServers. +#VSNAMES="" +#VSNAMES="name1 name2" +#VSNAMES="all" + +# vserver-info command location (default: /usr/sbin/vserver-info) +#VSERVERINFO=/usr/sbin/vserver-info + +# vserver command location (default: /usr/sbin/vserver) +#VSERVER=/usr/sbin/vserver + +# Directory where the VServers live; example: a VServer called "twiki" +# will have its root directory in $VROOTDIR/twiki +# (default: /var/lib/vservers) +#VROOTDIR=/var/lib/vservers + +####################################################################### +# # Localizable strings # # Add your custom headers below -#_MAIL_SUBJECT="`hostname -f` - changes report" +#_MAIL_SUBJECT="changes report" #_NO_DEBIAN_PACKAGES_CHANGE="No change in Debian packages state." #_NO_CHANGE="No change." #MAIN_HEADER=""