IdP migration from 1.3 to Shibboleth 2

Most important changes from IdP 1.3 to Shibboleth 2

  • CAS is no longer required for ArpViewer (new: uApprove).
  • External authentication (i.e. CAS, Pubcookie, ...) is no longer needed as the IdP features its own form based Username/Password login handler.
  • IdP automatically downloads metadata and attribute filter (formerly known as ARP) files.
  • Database required to support persistent identifiers (instructions for a MySQL DB are part of the deployment guides).
  • New entityID (Provider ID) scheme for IdPs: urn:mace:switch.ch:SWITCHaai:example.org changes to https://aai-logon.example.org/idp/shibboleth.
  • IdP installer generates self-signed certificate. While we recommend the use of a self-signed certificate for Shibboleth, it is also possible to use a certificate issued by an accepted Root CA (CA that is either included with the Mozilla or Microsoft program). Such a certificate (e.g. issued by a SWITCHpki) should be used anyway to secure the login page.
  • The Shibboleth 2 IdP is interoperable with 1.3 Service Providers, but not with the outdated SP version 1.2.
    Note: EZproxy version 5.1a and newer are interoperable with the Shib 2 IdP.

Possible migration strategies

It is possible to keep the downtime of the IdP service low, if the migration has been well prepared and thorough interoperability testing has been done. A short downtime will allow to bring the new production IdP online and, if needed, to migrate the ArpViewer (uApprove) database.

The current best practice to put a new system into production is to first install it in a test environment and document the installation, do testing and finally install the production service analogous to the test system.

It is advised to keep the DNS name which is used for the 1.3 IdP on the new IdP. Otherwise users might be confused about a new hostname in the login URL. As a consequence, there are two options to put the new IdP online on the old hostname:
Change DNS entries to point on the new IdP's IP address(es).
Be aware that clients may cache the IP addresses on their own strategy regardless of TTL settings. Consequently, some requests may go to the old address for a while.
Re-use the IP address(es) of the old IdP on the new IdP.
The router/switch may cache the ARP address of the IdP. This is usually not a problem as the network components detect the change automatically.

If the Shibboleth 2 IdP will get a new hostname, the old and new IdP can run in parallel for the migration phase. This will allow to test the interoperability of the new IdP before putting it into production. The new IdP can also be put in the list of IdP's in the Discovery Service WAYF) in the section of "Upcoming" IdPs.

Steps towards a successful migration

Testing

  1. Install the IdP in the test federation and test against Service Providers in the test federation before going for the production federation.
  2. For test with SPs, use the following page to construct login links that point to the new IdP: Service Provider Login Link Composer.
  3. If uApprove (ArpViewer) is used on the 1.3 IdP, migrate the database and configuration to the Shib 2 IdP as a dry run for the migration of the production system.

Coordination with Service Providers and SWITCH

  1. Coordinate date of migration with the AAI team. The AAI team will change the WAYF entry and Resource Registry specific settings on the day of the migration. According to statistics from the central WAYF show that the number of AAI users is the lowest on Saturday morning. That may be the best time for a maintenance window to do the upgrade.
  2. To prepare the IdP for the production federation, do an installation with the entityID that will finally be used in the production federation, e.g. https://example.org/idp/shibboleth.
  3. Send the IdP Metadata file (/opt/shibboleth-idp/metadata/idp-metadata.xml) to the AAI team on aai@switch.ch for inclusion in federation metadata. It is advised to do this at least 2-3 working days before the date of the migration. This will ensure that Service Providers will be able get the metadata with the new IdP's entry before the actual migration.
  4. Inform resources of the your institution about the migration. In case they use their own WAYF (or redirect directly to your IdP's login page), make sure they will adapt their configuration. See the recipes for a way to find resources that do not use the central WAYF. Administrators of these resources should be informed separately. Their contact information is listed in the Resource Registry.
  5. Inform the SP admins of the SWITCHaai federation about the migration via aai-operations@switch.ch. Include exact time of migration, old and new entityID of IdP in the note. You may also ask the AAI team to post the note on the mailing list.
  6. On the login page of the old IdP inform users about migration. In particular they should be made aware of host name/path changes because their browser may not complete the login form with username and passwort autoamtically anymore.
  7. Check whether digital content providers (Elsevier, ScienceDirect, Metapress, ...) or other resources rely on the IdP's entityID. A reconfiguration on their Shibboleth Service Provider may be necessary.

Migration

  • Put the Shibboleth 2 IdP in production on the date agreed on. Keep in contact with the SWITCH AAI team for changes on the central WAYF and in the Resource Registry.
  • If needed, migrate the ArpViewer (uApprove) database to the Shib 2 IdP and adapt the configuration for uApprove.
  • If you were using the updateARP tool with Shibboleth 1.3 and if you had defined custom rules to be applied to the downloaded arp.site.xml file, make sure you apply these rules also for your new Identity Provider's attribute-filter.xml. You can do this by going to the Resource Registry and define "Specific Attribute Release Policy rules" in your Home Organization Description. By defining such excpetion rules you should be able to get the same results like with the updateARP rules.

Post-Migration

  • You still should let run the old Identity Provider for at least some days in order to find out which Service Provider or users still are using the old site because of out-dated login links or bookmarks. Just have a look at the shib-access.log file to out more.
  • Another good idea is also to inform users that still are redirected to the old Identity Provider to update bookmarks or links. This can be done by removing the old Identity Provider and replacing the login pages with set of PHP files (provided by UniL) or a simple web page at the old location of the Identity Provider (usually /shibboleth-idp/SSO or /cas/login).

How to contact the AAI team

The AAI team can be contacted via e-mail on aai@switch.ch or telephone on +41 (0) 44  268 1505.


Recipes

How to get a list of SP's accessed from the IdP 1.3

 cat /var/log/shibboleth/shib-access.2010*log | \
 sed -n -e 's/.*Authen.*(\(http[^)]*\)).*/\1/p' | sort | \
 uniq -c | sort --reverse --numeric

The commands above produce an ordered list of SP entityIDs with the number of authentication assertions issued.

How to find Service Providers that do not use the central WAYF

Service Providers that do not use the central WAYF most probably use their own WAYF or do a redirect to the old IdPs login page. In order to find these SPs, two lists can be compared:

  • The list of all SPs that users have accessed in the past few weeks (see above)
  • The list of SPs that were accessed via the SWITCHaai WAYF
The SPs that appear your IdP's list but not on the SWITCHaai WAYF list (or just with low numbers) are of interest. They most probably do not use the central WAYF and will be affected by the migration.

To get the SWITCHaai WAYF list, write an email to aai@switch.ch. Please specify the time range the list shall cover.