SWITCHaai Certificate Migration

This page describes how certificates used by a Service Provider (SP) or Identity Provider (IdP) can be replaced without causing any service disruptions.

Table of Contents

Background Information

When certificates are embedded in federation metadata, one is very flexible when it comes to the creation of such a certificate. It's no longer required to get a certificate of a specific root certification authority like SWITCHpki or Verisign. The certificate only have to meet the SWITCHaai certificate requirements. In addition, embedding certificates allows the use of the new SAML2 feature to encrypt attribute assertions with the Service Provider's public key that is part of an embedded certificate.

A minor drawback, however, is the slightly more complicated procedure to replace expiring or compromised certificates. Proper timing plays now an important role for a seamless migration. Just replacing the certificate on the IdP/SP side and embedding the new certificate in metadata without proper planning, can result in service disruptions of up to 24 hours.
The reason: The other IdPs and SPs in the federation first must know about this new certificate through a metadata update. Because they download metadata at least once per day, this also defines the maximum propagation delay that can occur until every SP and IdP knows about the new certificate.

Therefore, it's crucial for a seamless certificate migration to obey the timing recommendations provided below.

To top

Service Provider Certificate Migration

Replacing a Service Provider's certificate requires more steps than for an IdP because the SP certificate is used not only for signing but also for encrypting content (e.g. attribute assertions). Therefore, five steps are needed in total. However, step 5 is optional and steps 1 and 2 as well as 3 and 4 can be carried out without timing restrictions.

Service Provider recommended migration timeline
Step 0: Create new certificate
It is assumed the new certificate to be included in the federation metadata has been generated. This may be a self-signed certificate or a certificate issued by a well-known and accepted CA according to the requirements for certificates embedded in SWITCHaai metadata.
Step 1: Configure new certificate as standby
Before adding the new certificate (see deployement guide for a description how to generate a new certificate or consult your CA provider's guides and make sure to read the SWITCHaai certificate requirements) to metadata, one should first add it to the SP configuration (shibboleth2.xml). This way the SP knows about the new certificate before it receives the first assertion encrypted with the new certificate's public key. The SP however must not yet actively use the new certificate. Instead it shall be used as standby certificate.
Configuring the new certificate as standby certificate can be done by defining a chaining credential resolver like this: 
<CredentialResolver type="Chaining">
    <!--
    Certificate/Private key pairs are read in sequence
    First one is used for attribute requests. 
    -->
     

     <CredentialResolver type="File">
          <Key>
                <Name>Active</Name>
                <Path>/etc/shibboleth/old-crt.key</Path>
          </Key>
          <Certificate>
                <Path>/etc/shibboleth/old-crt.crt</Path>
          </Certificate>
     </CredentialResolver>
     

     

     <CredentialResolver type="File">
          <Key>
                <Name>Standby</Name>
                <Path>/etc/shibboleth/new-crt.key</Path>
          </Key>
          <Certificate>
                <Path>/etc/shibboleth/new-crt.crt</Path>
          </Certificate>
     </CredentialResolver>
     

</CredentialResolver>
It's possible that your current CredentialResolver looks different, e.g. like this:
<CredentialResolver 
  type="File" 
  key="/etc/shibboleth/old-crt.key"
  certificate="/etc/shibboleth/old-crt.crt"/>
Don't be confused by that. Both of the above forms are valid but the first form with the CredentialResolver of type Chaining is required to configuring multiple certificates.
So, after adding the new certificate as standby certificate, it is recommended to check the configuration with the command:
shibd -tc /etc/shibboleth/shibboleth2.xml
It should say that the overall configuration is loadable. Then restart the Shibboleth daemon, e.g. with:
/etc/init.d/shibd restart
As result of this configuration change, the SP can decrypt assertions encrypted by either the old or the new certificate. However, the SP still only use the old certificate for attribute requests to the IdPs. So far, they don't know yet about the new certificate since it's not yet included in metadata.
Step 2: Add new certificate to metadata
This step can be done right after or at the same time as step 1. Now that the SP has the new certificate configured as standby, it's time to let the IdPs in the federation know about it too. Therefore, the new certificate has to be included in metadata.
In order to accomplish this, go to the Resource Registry and add the new certificate to the corresponding Resource Description of the SP. The order of the certificates doesn't matter. Then submit the change for approval and wait until a Resource Registration Authority (RRA) administrator of your organization has approved the change.

Service Provider recommended migration timeline
After approval of the Resource Description containing the new certificate, the federation metadata file will contain the new certificate.
It then takes at most 24 hours until every IdP will have downloaded this new metadata file. To be on the safe side, one should not proceed with the next step before this time has elapsed.
Now with two certificates available in metadata, the IdP may choose randomly which one to use for encrypting assertions for this SP. That's why the SP by this time already must have the new certificate configured in order to be able to decrypt assertions with both, the old and the new certificate.
Step 3: Configure SP to use new certificate
After at least 24 hours, every IdP should have downloaded the latest metada file which includs the new certificate. Subsequently, the Service Provider can now be configured to make use of the new certificate, but now keeing the old one as standby certificate. This is accomplished by swapping the two certificates: 
<CredentialResolver type="Chaining">
    <!--
    Certificate/Private key pairs are read in sequence
    First one is used for attribute requests. 
    -->
     

     <CredentialResolver type="File">
          <Key>
                <Name>Active</Name>
                <Path>/etc/shibboleth/new-crt.key</Path>
          </Key>
          <Certificate>
                <Path>/etc/shibboleth/new-crt.crt</Path>
          </Certificate>
     </CredentialResolver>
     

     

     <CredentialResolver type="File">
          <Key>
                <Name>Standby</Name>
                <Path>/etc/shibboleth/old-crt.key</Path>
          </Key>
          <Certificate>
                <Path>/etc/shibboleth/old-crt.crt</Path>
          </Certificate>
     </CredentialResolver>
     

</CredentialResolver>
Again, check the configuration and restart the Shibboleth daemon after applying this change.
Step 4 (optional): Remove old certificate from metadata
This step can be done at the same time as step 3 or omitted completely in case the old certificate expires within a couple of days since the Resource Registry drops expired certificates automatically.
Should you however want to remove the certificate from metadata for other reasons (e.g. because there is a risk that the key got compromised), just to go the Resource Registry and remove the old certificate from the corresponding Resource Description of the SP. After this change was approved by an RRA, it will take at most 24 hours until it has propagated to all Identity Providers.
Step 5 (optional): Remove old certificate from Service Provider configuration
In this optional step one can remove or comment out the old (Standby) certificate from the Service Provider configuration. It is however important to obey a 24 hours propagation delay after the certificate has been removed from federation metadata in which case you should have received an automated notification email.
Otherwise, the SP still could receive assertions encrypted with the old certificate which was removed from configuration. Therefore, decryption would fail and the user would see an error message.
Uncommenting the old certificate in the shibboleth2.xml will look like this: 
<CredentialResolver type="Chaining">
    <!-- Read in sequence -->
     

     <CredentialResolver type="File">
          <Key>
                <Name>Active</Name>
                <Path>/etc/shibboleth/new-crt.key</Path>
          </Key>
          <Certificate>
                <Path>/etc/shibboleth/new-crt.crt</Path>
          </Certificate>
     </CredentialResolver>
     

     <!--
     <CredentialResolver type="File">
          <Key>
                <Name>Standby</Name>
                <Path>/etc/shibboleth/old-crt.key</Path>
          </Key>
          <Certificate>
                <Path>/etc/shibboleth/old-crt.crt</Path>
          </Certificate>
     </CredentialResolver>
     -->
</CredentialResolver>
To top

Identity Provider Certificate Migration

The procedure to replace a certificate for an Identity Provider is slightly simpler than for a SP. Since the IdP does not need a standby certificate, the migration procedure has only two required steps.

Identity Provider recommended migration timeline
Step 0: Create new certificate
It is assumed the new certificate to be included in the federation metadata has been generated. This may be a self-signed certificate or a certificate issued by a well-known and accepted CA according to the requirements for certificates embedded in SWITCHaai metadata.
Step 1: Add new certificate to metadata
Before the Identity Provider can actually use the new certificate to sign assertions, the Service Providers in the federation first must know about this new certificate.
In order to add the new certificate to the IdP description, go to the Resource Registry and add the new certificate to the corresponding Home Organization Description of the IdP. The order of the certificates doesn't matter.

Service Provider recommended migration timeline
After the Home Organization Description with the new certificate was saved, the federation metadata file will contain the new certificate. Now, it takes at most 24 hours until every SP should have refreshed this new metadata file.
Step 2: Configure IdP to use new certificate
After at least 24 hours, every SP should have downloaded the latest metadata file including the new certificate, provided they follow the recommendation. Subsequently, the Identity Provider can now be configured to use the new certificate. One usually has to configure the new certificate in two places: The Apache or Tomcat configuration and the Shibboleth configuration.
For Shibboleth, one just has to specify the path to the new certificate/key pair in the file relying-party.xml like this:
<!-- ========================================== -->
<!--     Security Configurations                -->
<!-- ========================================== -->


<security:Credential 
 id="IdPCredential"
 xsi:type="security:X509Filesystem">
    <security:PrivateKey>/opt/shibboleth-idp/credentials/new-crt.key</security:PrivateKey>
    <security:Certificate>/opt/shibboleth-idp/credentials/new-crt.crt</security:Certificate>
</security:Credential>
The locally maintained metadata which the IdP has about itself has to be updated, too.
  1. Backup the file /opt/shibboleth-idp/metadata/idp-metadata.xml.
  2. In the idp-metadata.xml, replace the two PEM-formatted blocks in the <ds:X509Certificate> elements with the new certificate in PEM format. Omit the -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- lines. Do this two times, once in the IDPSSODescriptor and once in the AttributeAuthorityDescriptor element. 
           

    
<!-- IDPSSODescriptor / AttributeAuthorityDescriptor -->
<!-- ... -->
<KeyDescriptor>
     <ds:KeyInfo>
         <ds:X509Data>
             <ds:X509Certificate>
MIIDFDCCAfygAwIBAgIJAOVA31cn9tCxMA0GCSqGSIb3DQEBBQUAMBkxFzAVBgNV
BAMTDnNwLmV4YW1wbGUub3JnMB4XDTA4MDMwNzE2MTAxNVoXDTExMDMwODE2MTAx
NVowGTEXMBUGA1UEAxMOc3AuZXhhbXBsZS5vcmcwggEiMA0GCSqGSIb3DQEBAQUA
A4IBDwAwggEKAoIBAQDNhGHf6eeZF14sJduvDBG1MyMxjHWN5UorfLXPDAYZhr5W
D9biVzyfq9FCbP4ndDkBSVdu5rgXKLxyzod1MkDeN+4idr9EZiAVbhJaLDir5YwB
agStiCC65QLsrAGrTKbGn7HoXOfO5o4+l4a62mMgMFMAUp9W/JQ0k4fWmbnWcXHo
UzEeN/XukiD4jgUFnf9/dB/9P93ViAnkSUoEHeHIz7Oiw1c/9pCJktiU2ns9exUw
wRGaagLXdPBxAdvUx1o1MG+Uajfg85EwhVzlO2kBClq3k9koJi01eTam4XxsBGSQ
TmvsKf8aPES0nrhSJjOtceGOuyxAQ9UXSRi+0rrXAgMBAAGjXzBdMDwGA1UdEQQ1
MDOCDnNwLmV4YW1wbGUub3JnhiFodHRwczovL3NwLmV4YW1wbGUub3JnL3NoaWJi
b2xldGgwHQYDVR0OBBYEFEJItQ8/IYfolzHeMZ3aNEzisC44MA0GCSqGSIb3DQEB
BQUAA4IBAQBPJi0wFTrqfATuhuA1h91efgYqE6WIC99Go4i6UFnsjwl3k9ZCF2xs
9T1uZ22nOFmAnAPbUS3cMiqFWGwlpC4w0KaXIeeUuh51g5+nKnqRE0XkCmm+5515
66L6PR+RMN9kx2h4HCuwvyFN02Y+Iz49oSOBW9exV1BmP0mhWF1j6lPoDkRDLZnG
eLtJhzirb37BEk0fJkIg8OM1SPZ1mdB/5qhkWj4RRp47e03zeVE2YQMNhn6sE9hj
xTdeeVyguekdSeLaF89jXvW6XBJR9NsZ+NswXIFN6nIYIRQha8+w2p1Saq0fvf8g
wf2JYThoxNmfL8cHfoPqJAikP1gwuRjt
             </ds:X509Certificate>
         </ds:X509Data>
     </ds:KeyInfo>
</KeyDescriptor>
<!-- ... -->
If Apache is used in front of Tomcat (as recommended in our deployment guides), the configuration for the VirtualHost serving port 8443 should be adapted like this:
<VirtualHost idp.example.org:8443>
    [...]
        

    SSLCertificateFile      /opt/shibboleth-idp/credentials/new-cert.crt
    # In case of a certificate issued by a well-known CA
    # make sure to configure the certificate chain
    # This is not necessary for self-signed certificates
    #SSLCertificateChainFile /etc/ssl/certs/qvsslica.crt.pem
    SSLCertificateKeyFile   /opt/shibboleth-idp/credentials/new-cert.key
    

    [...]
</VirtualHost>
After applying this change to the configuration, restart Apache (a reload only might not be sufficient) e.g. with:
/etc/init.d/apache2 restart

In case you are using the Tomcat web server only, without an Apache web server in front of it, the new certificate must be imported in the Tomcat keystore for port 8443, configured in Tomcat's server.xml. Afterwards, restart Tomcat.
Step 3 (optional): Remove old certificate from metadata
This step can be done at the same time as step 2 or ommitted completely in case the old certificate expires within a couple of days because the Resource Registry drops expired certificates automatically.
However, should you want to remove the certificate from metadata for other reasons (e.g. because there is a risk that the key got compromised), just to go the Resource Registry and remove the old certificate from the corresponding Home Organization Description of the IdP. After applying this change, it will take at most 24 hours until it has propagated to all properly configured Service Providers.
To top

References

Additional information on certificates in Metadata