URL: http://www.switch.ch/aai/docs/shibboleth/SWITCH/1.3/idp/refresh-metadata-idp.html
Author: Patrik Schnellmann - SWITCH
Author: Valery Tschopp - SWITCH
Author: Lukas Haemmerle - SWITCH
$Date: 2007/08/09 08:30:57 $
$Revision: 1.8 $

Automatically Refresh SWITCHaai Federation Metadata for Shibboleth Identity Provider 1.3

Table of Contents

Introduction
SWITCHaai Metadata Signer Certificate
Installation for Unix Systems
Installation for Windows
Links

Introduction

For the proper function of an Identity Provider in the SWITCHaai federation, the SWITCHaai Federation Metadata (metadata.xml) has to be automatically retrieved and checked. Therefore an automatic update has be configured to keep the file up-to-date.

SWITCHaai Metadata Signer Certificate

All available SWITCHaai metadata files are digitally signed with the self-signed SWITCHaai Metadata Signer certificate.

Subject: C=CH, O=SWITCH - Teleinformatikdienste fuer Lehre und Forschung, OU=AAI, CN=SWITCHaai Metadata Signer, emailAddress=aai@switch.ch
MD5 Fingerprint: 1F:E6:A3:83:D8:24:FF:1E:8A:61:7D:73:F6:D7:EF:1A

The X.509 certificate can be downloaded at http://www.switch.ch/aai/downloads/metadata.crt.

Java Keystore for Identity Provider (Home Organization)

The Identity Provider metadatatool tool uses the SWITCHaai Metadata Signer Java keystore to verify the signature and integrity of the metadata file.

A Java keystore containing the Metadata Signer certificate can be downloaded on: http://www.switch.ch/aai/downloads/metadata.jks. The password for the keystore is "metadata"; it is only necessary for write access to the keystore.

Installation of the metadatarefresh Shell Script

For Identity Providers, the metadatatool tool is installed with the Shibboleth IdP 1.3. The wrapper script metadatarefresh.sh/metadatarefresh.bat extends it to be used as a cron job and automates the metadata refreshing.

The script will first download and verify the signature of the XML metadata file. It will then compare the local metadata file currently in use with the newly downloaded one.
If the downloaded metadata file is newer than the local one, the script backups the existing one and replaces it with the up-to-date version.

Note: Due to a bug in OpenSAML 1.1 the metadatatool provided by Shibboleth IdP (1.3, 1.3a, 1.3b) may cause problems downloading and verifying the metadata. Therefore, it is strongly recommended to download the OpenSAML 1.1b JAR and copy it to the /opt/shibboleth-idp/lib/ or /usr/local/shibboleth-idp/lib/ directory. The old version of the OpenSAML library, opensaml-1.1.jar, should be deleted.
  This issue does not affect installations of Shibboleth IdP 1.3c (or later) as OpenSAML 1.1b is already included.

Installation for Unix Systems

  1. Download the metadatarefresh.sh shell script and the Java keystore with the signer certificate: metadatarefresh.tar.gz. 
    $ cd /opt/shibboleth-idp/
$ wget http://www.switch.ch/aai/downloads/metadatarefresh.tar.gz
  2. Extract the .tar.gz package: 
    $ tar -xzf metadatarefresh.tar.gz
  3. Install the script and java keystore. Place metadatarefresh.sh in /opt/shibboleth-idp/bin/ and the java keystore file in /etc/shibboleth-idp/ (or the respective places on your system). 
    $ mv metadatarefresh.sh /opt/shibboleth-idp/bin/
$ mv metadata.jks /etc/shibboleth/metadata.jks

  4. Configure the variables highlighted in red in the script according to your own installation:

    IDP_HOME
    The home directory of Shibboleth IdP 1.3.
    Typically: /opt/shibboleth-idp or /usr/local/shibboleth-idp
    IDP_ETC
    The directory which contains the Shibboleth IdP configuration files.
    Typically: /etc/shibboleth or /opt/shibboleth-idp/etc
    JAVA_HOME
    Your Java installation directory.
    METADATA_URL
    The download URL of the SWITCHaai Federation Metadata file.
    METADATA_FILE
    Local filename of the SWITCHaai Federation Metadata file.
    SIGNER_KEYSTORE
    Location of the SWITCHaai Metadata Signer java keystore.
    #! /bin/sh
#
# Copyright (c) 2005 SWITCH - The Swiss Education & Research Network
#
# metadatarefresh: automatically download and verify the federation metadata
#
# SWITCHaai Federation Metadata (1.3)
#  http://www.switch.ch/aai/metadata/
# 
# SWITCHaai Metadata Signer Java keystore:
#  http://www.switch.ch/aai/downloads/metadata.jks
#
# Put this script in $IDP_HOME/bin/metadatarefresh.sh
# and a add a crontab entry like this:
# 30 4 * * * /opt/shibboleth-idp/bin/metadatarefresh.sh
# 
# Valery Tschopp - SWITCH - 20050909
# Dominique Petitpierre - University of Geneva - 20050810
#


##
# Configure directories and path
##
IDP_HOME=/opt/shibboleth-idp
IDP_ETC=/etc/shibboleth
LOG_FILE=$IDP_HOME/logs/metadatarefresh.log
JAVA_HOME=/opt/java

##
# Configure metadata download URL and local name
##
METADATA_URL=http://www.switch.ch/aai/federation/SWITCHaai/metadata.switchaai_signed.xml
METADATA_FILE=$IDP_ETC/metadata.switchaai.xml

# export path variable
export PATH="/bin:/usr/bin"

##
# Configure signer certificate java keystore
##
SIGNER_KEYSTORE=$IDP_ETC/metadata.jks
SIGNER_ALIAS=switchaaimetadatasigner

if [ ! -r "$SIGNER_KEYSTORE" ] ; then
    echo "ERROR: Metadata Signer java keystore $SIGNER_KEYSTORE not found"
    exit 1
fi


############################################
# Environment variables
export IDP_HOME
export JAVA_HOME

############################################
# Refresh metadata
NOW=`date +%Y%m%d%H%M`
# logging
echo "$NOW: $0 starts" >> $LOG_FILE

if [ -f "$METADATA_FILE" ] ; then
    # echo "refresh: $METADATA_URL -> $METADATA_FILE"
    $IDP_HOME/bin/metadatatool \
        --in $METADATA_URL \
        --out $METADATA_FILE.$NOW \
        --keystore "$SIGNER_KEYSTORE" \
        --alias "$SIGNER_ALIAS" \
        >> $LOG_FILE 2>&1
    ret=$?
    if [ "$ret" -ne "0" ] || [ ! -f "$METADATA_FILE.$NOW" ] ; then
        sed -n -e "/^$NOW/"',$p' $LOG_FILE
        echo "ERROR: refresh of $METADATA_URL failed ($ret)"
        echo "$NOW: ERROR: refresh from $METADATA_URL failed ($ret)" >> $LOG_FILE
        exit $ret
    fi
fi

############################################
# Make backups
[ ! -d "$IDP_ETC/backup" ] && mkdir $IDP_ETC/backup

# Compare new to existing.  If same, delete new, if not replace existing
# and backup new.
if [ -r $METADATA_FILE.$NOW ] ; then
    if cmp -s $METADATA_FILE.$NOW $METADATA_FILE ; then
        rm -f $METADATA_FILE.$NOW
        echo "$NOW: unmodified $METADATA_FILE" >> $LOG_FILE
    else
        echo "$NOW: new $METADATA_FILE"
        METADATA_BASENAME=`basename $METADATA_FILE`
        cp -p $METADATA_FILE $IDP_ETC/backup/$METADATA_BASENAME.$NOW
        mv -f $METADATA_FILE.$NOW $METADATA_FILE
        echo "$NOW: new $METADATA_FILE installed" >> $LOG_FILE
    fi
fi

# logging
echo "$NOW: $0 done." >> $LOG_FILE
  5. Test the script manually: 
    $ /opt/shibboleth-idp/bin/metadatarefresh.sh
    Check the output of the script and its logs in $IDP_HOME/logs/metadatarefresh.log.
  6. Let cron periodically call the metadatarefresh.sh script i.e. install the script as a cron job. Place a symbolic link to the script in /etc/cron.daily/ and cron will call the metadatarefresh script each day. 
    $ ln -s /opt/shibboleth-idp/bin/metadatarefresh.sh /etc/cron.daily/

Installation for Windows Systems

  1. Download the metadatarefresh.bat batch script and the Java keystore with the signer certificate: metadatarefresh.zip.
  2. Extract the zip archive
  3. Install the script and java keystore. Place metadatarefresh.bat in shibboleth-idp/bin/ and the java keystore file in shibboleth-idp/etc (or the respective places on your system).

  4. Configure the variables highlighted in red in the script according to your own installation:

    IDP_HOME
    The home directory of Shibboleth IdP 1.3.
    Typically: c:\shibboleth-idp
    IDP_ETC
    The directory which contains the Shibboleth IdP configuration files.
    Typically: c:\shibboleth-id\etc
    JAVA_HOME
    Your Java installation directory.
    METADATA_URL
    The download URL of the SWITCHaai Federation Metadata file.
    METADATA_FILE
    Local filename of the SWITCHaai Federation Metadata file.
    SIGNER_KEYSTORE
    Location of the SWITCHaai Metadata Signer java keystore.
    @echo off
SETLOCAL
REM Made by Philip Brusten 2005-09-20
REM Adapted by Lukas Haemmerle 2007-05-30
REM ########################################################################
REM 
REM  Welcome to the automated metadata update tool for Shibboleth IdP software on Windows.
REM  This tool helps you to keep your SWITCHaai metadata up-to-date
REM 
REM  Please send any remarks or updates/improvements to aai@switch.ch
REM 
REM ########################################################################

REM ############################SETTING DATE#################################

For /f "tokens=1-7 delims=:/-, " %%i in ('echo exit^|cmd /q /k"prompt $D $T"') do (
	For /f "tokens=2-4 delims=/-,() skip=1" %%a in ('echo.^|date') do (
		set dow=%%i
		set %%a=%%j
		set %%b=%%k
		set %%c=%%l
		set hh=%%m
		set min=%%n
		set ss=%%o
	)
)
REM #########################################################################

REM ############################START CONFIG#################################

REM Set the variables to the right directory or filename
SET IDP_HOME=c:\shibboleth-idp
SET IDP_ETC=%IDP_HOME%\etc
SET JAVA_HOME=c:\jdk
SET LOG_FILE=%IDP_HOME%\logs\metadatatool.log

REM Provide the metadata url and file

SET METADATA_URL=http://www.switch.ch/aai/federation/SWITCHaai/metadata.switchaai_signed.xml
SET METADATA_FILE=metadata.switchaai.xml
SET TMP_METADATA_FILE=metadata.switchaai.tmp.xml
SET METADATA_BACKUP_FILE=%yy%%mm%%dd%_%hh%h%min%m%ss%s-%METADATA_FILE%

REM Provide the location of the java keystore used for verification of the signed metadata
SET SIGNER_KEYSTORE=c:\pki\metadata.jks
SET SIGNER_ALIAS=switchaaimetadatasigner
SET SIGNER_PASSWORD=metadata
REM #############################END CONFIG##################################

echo ########## Start wrapper script %yy%%mm%%dd%_%hh%h%min%m%ss%s ########### >> %LOG_FILE%

IF EXIST %SIGNER_KEYSTORE% GOTO logging
IF NOT EXIST %SIGNER_KEYSTORE% GOTO nokeystore

:nokeystore
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - Error: No Metadata Signer Java keystore found. >> %LOG_FILE%
GOTO end

:logging
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - Succeed: Metadata Signer Java keystore found. >> %LOG_FILE%

:synchronise
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - Attempting to synchronise with metadata (%METADATA_URL% to %IDP_ETC%\%METADATA_FILE%) >> %LOG_FILE%

REM CALL %IDP_HOME%\bin\metadatatool.bat --in %METADATA_URL% --out %IDP_ETC%\%TMP_METADATA_FILE% --noverify 2>> %LOG_FILE%
CALL %IDP_HOME%\bin\metadatatool.bat --in %METADATA_URL% --out %IDP_ETC%\%TMP_METADATA_FILE% --keystore %SIGNER_KEYSTORE% --alias %SIGNER_ALIAS% --password %SIGNER_PASSWORD% 2>> %LOG_FILE%
IF ERRORLEVEL 1 GOTO error
IF ERRORLEVEL 0 GOTO succes

:error
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - ERROR: Couldn't synchronise with %METADATA_URL% >> %LOG_FILE%
GOTO end

:succes
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - Metadata succesfully synchronised to %IDP_ETC%\%TMP_METADATA_FILE% >> %LOG_FILE%

IF EXIST %IDP_ETC%\%METADATA_FILE% GOTO compare
:nometadata
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - No metadata file (%IDP_ETC%\%METADATA_FILE%) present >> %LOG_FILE%
goto movemetadata

:compare
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - Comparing old (%METADATA_FILE%) with new metadata (%TMP_METADATA_FILE%) >> %LOG_FILE%
echo n|comp %IDP_ETC%\%METADATA_FILE% %IDP_ETC%\%TMP_METADATA_FILE% 
IF ERRORLEVEL 1 GOTO differentsize
IF ERRORLEVEL 0 GOTO samesize

:samesize
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - Metadata is the same as previously backupped>> %LOG_FILE%
goto movemetadata

:differentsize
IF EXIST %IDP_ETC%\backup GOTO dobackup
mkdir %IDP_ETC%\backup
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - Folder %IDP_ETC%\backup created >> %LOG_FILE%

:dobackup
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - Metadata is newer and the old one will be backupped>> %LOG_FILE%
copy %IDP_ETC%\%METADATA_FILE% %IDP_ETC%\backup\%METADATA_BACKUP_FILE% 1>> %LOG_FILE%
IF ERRORLEVEL 1 GOTO copyerror
IF ERRORLEVEL 0 GOTO copysucces

:copyerror
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - ERROR: Couldn't backup %IDP_ETC%\%TMP_METADATA_FILE% >> %LOG_FILE%
GOTO movemetadata

:copysucces
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - %IDP_ETC%\%METADATA_FILE% copied to %IDP_ETC%\backup\%METADATA_BACKUP_FILE% >> %LOG_FILE%

:movemetadata
echo %dd%/%mm%/%yy% %hh%:%min%:%ss% - Saving the new metadata %IDP_ETC%\%TMP_METADATA_FILE% to %IDP_ETC%\%METADATA_FILE% >> %LOG_FILE%
move /Y %IDP_ETC%\%TMP_METADATA_FILE% %IDP_ETC%\%METADATA_FILE% 

:end
  5. Test the script manually: 
    $ c:\shibboleth-idp\bin\metadatarefresh.bat
    Check the output of the script and its logs in $IDP_HOME\logs\metadatarefresh.log.
  6. Let the Windows "Scheduled tasks" service call the metadatarefresh.sh script at least once a day in order to keep the metadata up-to-date.

Links

SWITCH AAI project website
http://www.switch.ch/aai/
SWITCHaai Federation metadata
http://www.switch.ch/aai/metadata/
Metadataupdate script and signer certificate in tar.gz archive
http://www.switch.ch/aai/downloads/metadatarefresh.sh.tar.gz
Metadataupdate script and signer certificate in zip archive
http://www.switch.ch/aai/downloads/metadatarefresh.zip
--
$Id: refresh-metadata-idp.html,v 1.8 2007/08/09 08:30:57 haemmer Exp $