Jamf Security Cloud - secure DNS services are not working due to a wrong certificate

Incident Report for Jamf

Postmortem

June 15, 2025 - Jamf SecureDNS Outage RCA

Summary and Timeline
On June 15, 2025 (00:00 UTC), devices using SecureDNS lost internet access due to an outage affecting Web Threat Prevention and Content Filtering capabilities within Jamf Safe Internet, Jamf Protect, and Jamf Trust services.The root cause of the incident was an expired SSL certificate, leading to loss of security features or loss of internet connectivity for end user devices. Although the certificate in question was renewed on May 22nd, 2025, the process used to deploy this certificate had an error, which resulted in incomplete distribution across our global cloud environment.After receiving customer reports of the outage, we investigated, identified the issue and impacted services, and then manually deployed the new certificate to restore the service.
We are currently conducting a thorough internal review of the outage and will work to improve the certificate management process to minimize the risk of a repeat occurrence.

Incident Timeline (UTC)

  • June 15, 2025 at 00:00: The SSL certificate expires
  • June 15, 2025 at ~00:25 : Troubleshooting efforts begin
  • June 15, 2025 at 01:45: Root cause identified
  • June 15, 2025 at 02:30: Teams locate all services requiring updated certificate
  • June 15, 2025 at 04:15: New certificate is deployed to US regions
  • June 15, 2025 at 04:30: Jamf verifies that the issue has been resolved across all regions

Service disruption duration: 270 minutes

Impact Assessment
Users Affected: All end user devices with Encrypted DNS Gateway configured, lost internet access.
Services Impacted: SecureDNS

Remediation Items & Next Steps
We are currently conducting a thorough internal investigation of the events that led to the outage and identifying any gaps that we will need to address to minimize the risk of such occurrences. Our key areas of focus include:

1. Certificate management reviews – Ensure certificate documentation is up to date and include explicit verification steps for each service component. Enhance existing certificate expiration monitoring.

2. Monitoring – review and enhance monitoring and alerting of anomalies in traffic patterns. Add in synthetic monitoring.

3. Communications – Refine internal and customer-facing workflows to ensure rapid response and clarity during incidents.

4. Evaluate adding additional critical domains to on-device content filtering safe list.

For additional questions and needs, please reach out to your customer success manager or log a support ticket at support.jamf.com.

Posted Jun 16, 2025 - 21:30 UTC

Resolved

This incident has been resolved.
Posted Jun 15, 2025 - 04:31 UTC

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Jun 15, 2025 - 04:18 UTC

Identified

The issue has been identified and a fix is being implemented.
Posted Jun 15, 2025 - 03:17 UTC

Investigating

We are currently investigating this issue.
Posted Jun 15, 2025 - 03:02 UTC
This incident affected: Jamf Threat Defense, Jamf Private Access and Jamf Data Policy (Private Access).