Planning for the End of 2016: A Leap Second and the End-of-Support for SHA-1 TLS Certificates
New Year's Eve is typically in the depth of end-of-year change freezes for most IT organizations. At the end of 2016, however, two major events will be occurring right at the end of the year: a leap second and the final end of browser support for SHA-1 TLS certificates. Both of these changes have the potential to break software systems and applications. Significant preparation, planning, and testing ahead of time can significantly reduce the risk for both.
The leap second
At the stroke of midnight UTC on New Year's Eve, an extra second will be introduced, effectively causing the last second of the day to last for two seconds. Time will advance from: 2016-12-31 23:59:58 UTC to 23:59:59 to 23:59:60 before reaching 2017-01-01 00:00:00 UTC. This will apply earlier in other time zones. For example, the US East Coast will observe 6:59:60 PM. Since this change has resulted in significant software bugs across the world during the past two leap seconds in both 2012 and 2015, either significant testing or some other mitigation strategy is required to shield software systems and devices from leap second bugs.
As we did during the leap second in 2015 and as Google pioneered in 2008 and 2012, Akamai will again be implementing a "leap smear" where we apply the leap second over the course of 24 hours rather than all at once. We believe this is significantly safer overall, but during the peak of the leap smear it does mean that Akamai edge server clocks may appear up to a second off relative to origin server clocks on origin servers not implementing a leap smear or applying a different leap smear curve.
This difference has some risk of causing issues by itself (although relying on sub-second time synchronization between Akamai servers and origins is generally discouraged as it may result in problems during normal operations), but the overall risks are significantly lower than introducing an additional whole second on New Year's Eve.
SHA-1 TLS certificates
When the TLS/SSL certificates used to identify websites are signed by certificate authorities (CAs), the signatures use a cryptographic "signature algorithm" that includes a cryptographic hash function. The end of 2016 marks the end of support for the SHA-1 hash function in most major browsers as a signature algorithm for leaf and intermediate certificates. This is because cryptographic weaknesses in the SHA-1 algorithm, combined with increases in computational power, make it viable for well-funded attackers to generate multiple certificates with the same SHA-1 hash but with different contents. When a CA applies a SHA-1 RSA signature over what they have validated and what appears benign, they are thus also applying a signature to the alternate certificate that the attacker has prepared, which may impersonate a well-known website.
To address this risk, CAs were prohibited from issuing any SHA-1 certificates after January 1, 2016, as well as any expiring after December 31, 2016. Many browsers (Chrome, Mozilla Firefox, Microsoft IE/Edge, and others) will also start rejecting certificates signed with SHA-1 early in 2017. While some have exceptions for locally installed CA roots, Chrome plans to also present fatal network errors even in these cases.
The big risk of the retirement with SHA-1 certificates is that some clients out there (including some Symbian phones, Windows XP SP2, some old versions of Android, and many custom applications and embedded devices) do not know how to handle modern RSA SHA-256 certificates, and return validation errors when encountering them.
Much of the web has already switched over to RSA SHA-256 certificates. Since Akamai started offering SHA-256 certificates in early 2015, more than 95% of customer certificates served on Akamai's secure CDN have already switched over to using only RSA SHA-256 certificates. However, we do still periodically encounter customers who have a custom client or application break when their SHA-1 certificate rotates into a SHA-256 certificate. This has typically involved a rollback of the cert rotation, a scramble to make and release changes to the application, and then rolling forward with the rotation. As SHA-1 certificates are no longer possible to provision, options here quickly become limited.
Some large websites, as well as Akamai's shared certificate platform (with *.akamaihd.net and *.akamaized.net and a248.e.akamai.net), have been trying to stretch out SHA-1 support as far as safely possible. On the Akamai shared certificate platform, we return an RSA SHA-256 certificate for clients using TLS 1.2 and indicating support with the signature_algorithms extension. Even excluding clients that also send a TLS SNI extension (indirectly correlated with SHA-256 support), we still see significant numbers of handshakes completing and using SHA-1. At the end of 2016, handing out SHA-1 will no longer be possible (as this would mean serving an expired or invalid certificate to clients that may or may not support SHA-256, but would very likely error on any expired certificate).
To avoid making the change to our shared certificate on New Year's Eve, we will be shutting off the SHA-1 certificate and will hand out an RSA SHA-256 or ECDSA SHA-256 certificate on or around December 27, 2016.
Note that if your company has local CA root signing certificates for internal sites, you should still make sure that SHA-1 certs are no longer in use, as Chrome (for one) has announced that they will start returning errors on these early next year.
A common question is whether SHA-1 root certificates will continue to work: The answer is yes, but only for signatures they perform with SHA-256. This is because of the risk exposure is associated with performing signatures over a hash function where two certificate inputs can be readily found that hash to the same value.
Takeaways
To have a more relaxing New Year's Eve, we suggest you:
Make sure you have a leap second plan: either by performing complete testing on all software, systems, and infrastructure, or by implementing a leap smear
Make sure you have all sites on Akamai internally, and elsewhere rotated over to using SHA-256 certificates before getting any further into end-of-year lockdowns
Make sure applications or devices that rely on Akamai's shared certificate can handle SHA-256 certificates when Akamai drops SHA-1 support
It may be a very Western-centric perspective, but I'd also plead with folks to pick days for leap seconds and other major infrastructure change milestones that aren't on a New Year's Eve.
Erik Nygren is a fellow and chief Architect in Akamai's Platform Engineering organization. He spent New Year's Eve on December 31, 1999, in the Akamai Network Operations Command Center just in case we'd missed something in our Y2K planning.
