DigitalOcean

原因：Beginning at 09:36 UTC on August 3, some customers may experience intermittent errors when accessing the DigitalOcean Cloud Control Panel, including an "mTLS verification failed" message. Our engineering team is actively investigating. We apologise for the inconvenience this may have caused. We will provide another u…

修复：Between 09:36 and 13:30 UTC on August 3, some customers experienced intermittent errors while accessing the DigitalOcean Cloud Control Panel. We renewed the affected internal certificates and restored access. The issue is resolved, and the Cloud Control Panel is operating normally. We apologize for the disruption.

原因：Our Engineering team has resolved the issue causing empty or broken responses from Kimi-K3. If you continue to experience any problems, please open a ticket with our Support team. We apologize for any inconvenience this may have caused.

修复：Our Engineering team has resolved the issue causing empty or broken responses from Kimi-K3. If you continue to experience any problems, please open a ticket with our Support team. We apologize for any inconvenience this may have caused.

修复：This incident has been resolved.

原因：Between 07:46  UTC & 10:24 UTC today, some users in India accessing resources in the NYC region may have experienced connectivity issues. The impact was limited to users using Airtel as their ISP. Our investigation determined that the issue was caused by an internal network issue within Airtel and was not related to D…

修复：Between 07:46  UTC & 10:24 UTC today, some users in India accessing resources in the NYC region may have experienced connectivity issues. The impact was limited to users using Airtel as their ISP. Our investigation determined that the issue was caused by an internal network issue within Airtel and was not related to D…

修复：This incident has been resolved.

修复：This incident has been resolved.

原因：Our Engineering team has confirmed that the issue regarding volume attachment to Droplets in the NYC1, NYC3, SGP1, SYD1, and BLR1 regions has been fully resolved, and all services are now operating normally. If you continue to experience any issues, please contact our Support team by opening a ticket. We apologize for…

修复：Our Engineering team has confirmed that the issue regarding volume attachment to Droplets in the NYC1, NYC3, SGP1, SYD1, and BLR1 regions has been fully resolved, and all services are now operating normally. If you continue to experience any issues, please contact our Support team by opening a ticket. We apologize for…

修复：Our engineering team has resolved the outbound network connectivity issue in the BLR1 region. If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.

原因：Our Engineering team has confirmed that the underlying issue affecting Reserved IP routing in the TOR1 region has been fully resolved, and services are now operating normally. If you continue to experience any issues, please contact our Support team by opening a ticket. We apologize for any inconvenience caused.

修复：Our Engineering team has confirmed that the underlying issue affecting Reserved IP routing in the TOR1 region has been fully resolved, and services are now operating normally. If you continue to experience any issues, please contact our Support team by opening a ticket. We apologize for any inconvenience caused.

原因：We are currently experiencing an issue affecting customers using the Gemma 4 model on our Serverless Inference platform. Customers may experience significantly increased latency or request timeouts. Our Engineering team has identified a backend configuration issue as the root cause, which is temporarily impacting mode…

修复：The deployed fix has successfully restored full functionality, and our monitoring shows that system performance has completely stabilized. Response times for all Gemma 4 inference workflows have returned to normal baseline levels. We will continue to track platform stability moving forward to ensure long-term reliabili…

原因：The issue affecting Kubernetes deployments in NYC1 has subsided. Workloads should now be functioning normally. Our Engineering team is continuing to monitor the affected systems to confirm full resolution. We'll update this page if any further action is needed. We apologize for the inconvenience this may have caused.

修复：The issue affecting Kubernetes deployments in the NYC1 region has been resolved. Our investigation found intermittent DNS timeouts affecting a small number of DOKS clusters, with affected worker nodes running on shared-CPU Droplets. This is a documented limitation for latency-sensitive cluster DNS workloads such as Cor…

原因：Our Engineering team has implemented a fix for the issue with the Deepseek V4 Pro model in Serverless Inference and Agent Platform. The model should now be operational, and users should no longer receive error 429 messages when attempting to use it. We are currently monitoring the situation to ensure the fix is success…

修复：Our Engineering team has confirmed that the issue with the Deepseek V4 Pro model in Serverless Inference and Agent Platform has been fully resolved. The model is now operational, and users should be able to use it without experiencing any errors. If you continue to experience any problems, please open a ticket with ou…

原因：Our Engineering team has implemented a fix to address the issue affecting Droplets in the NYC3 region. The issue, which began at 12:38 UTC, caused disruptions to customers attempting to perform actions on their Droplets in the NYC3 region. During this time, customers may also have experienced errors when trying to cre…

修复：Our Engineering team has confirmed that the issue impacting Droplets in the NYC3 region has been fully resolved. Users can now perform actions on their Droplets or create new Droplets in this region without any issues. We appreciate your patience while we worked to resolve this issue. If you continue to experience any…

修复：Our engineering team has resolved the issue with Managed Database Clusters. All database systems should now be operating normally. If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience.

修复：The intermittent DNS lookup failures between Managed Kubernetes and Managed Database hostnames in the FRA1 region have been resolved for now but please let us know if you see the issue again. Connectivity has remained stable, and all systems are operating normally. If you continue to experience any problems, please ope…

修复：Our Engineering team has confirmed that the issue impacting Droplet resizes using the "Downscale Anytime" option has been fully resolved. Users can now resize their Droplets using the "Downscale Anytime" option without any issues. We appreciate your patience while we worked to resolve this issue. If you continue to ex…

原因：Our Engineering team is currently investigating an issue where Agents are experiencing timeouts while retrieving data from Knowledge Bases. As a result, affected Agents may fail to retrieve data and return the following error: "Failed to retrieve data from Knowledge base(s) - timeout" Please be assured that we are tr…

修复：This incident has been resolved, and our teams continue to monitor the results. If you experience any further issues, please contact support.

原因：Between 00:00 UTC & 16:00 UTC today, our Engineering team identified an issue affecting backup operations on Droplets. During this period, backups scheduled within this window may not have been created and may appear as missing. Our team has taken necessary measures to resolve the issue, and we can confirm that the b…

修复：Between 00:00 UTC & 16:00 UTC today, our Engineering team identified an issue affecting backup operations on Droplets. During this period, backups scheduled within this window may not have been created and may appear as missing. Our team has taken necessary measures to resolve the issue, and we can confirm that the b…

原因：Our Engineering team has mitigated an issue with Anthropic models. Previously, users may have encountered 400 errors when attempting to use any Anthropic model. Although the root cause of the issue is still being addressed by Anthropic, users should now be able to access and use Anthropic models again. We will continu…

修复：Our Engineering team has resolved the issue that was causing HTTP 400 errors for requests to Anthropic models. Users should now be able to access Anthropic models without any issues. If you continue to experience problems, please open a ticket with our Support team so we can investigate further. We apologize for any in…

原因：Our Engineering team is currently investigating an issue affecting the visibility of monitoring graphs within the Cloud Panel. During this period, users may notice missing or unavailable monitoring graphs for services such as Droplets(with DO agent installed), Load Balancers, Databases, etc. We apologize for the incon…

修复：Between 13:22 & 14:26 UTC today, users have experienced missing monitoring graphs for Droplets (with DO agent installed), Load Balancers, Databases, and other services within the Cloud Control Panel. Our engineering team has identified the root cause of the issue and has taken appropriate steps to restore functionalit…

修复：The connectivity issues affecting our Serverless/GenAI Inference API have been fully resolved. Our engineering teams successfully completed the connectivity restoration. All systems should be functioning normally, and the endpoint should be fully operational.

修复：This incident has been resolved.

原因：Between 09:41 and 11:24 UTC, our Engineering team identified an issue impacting the DNS API service. During this period, users may have experienced issues performing domain and DNS record management operations through the Control Panel and API. Services dependent on DNS API operations, including Let's Encrypt certifica…

修复：Between 09:41 and 11:24 UTC, our Engineering team identified an issue impacting the DNS API service. During this period, users may have experienced issues performing domain and DNS record management operations through the Control Panel and API. Services dependent on DNS API operations, including Let's Encrypt certifica…

修复：Our Engineering team has confirmed that the issue impacting build failures on App Platform has been resolved at 07:00 am UTC. All App Platform builds are now succeeding as expected. Customers who previously encountered build failures should now be able to deploy their applications without further issues. If you contin…

原因：Between 06:35 UTC & 12:36 UTC today, our engineering team identified an issue impacting the Cloud Firewall. During this period, users might have encountered issues while updating their firewall rules. Our team has taken appropriate measures to address the issue. We can confirm that service has been restored and is now…

修复：Between 06:35 UTC & 12:36 UTC today, our engineering team identified an issue impacting the Cloud Firewall. During this period, users might have encountered issues while updating their firewall rules. Our team has taken appropriate measures to address the issue. We can confirm that service has been restored and is now…

修复：Our Engineering team has verified that the issue causing incorrect Cloud Firewall rules to display on the cloud panel is fully resolved. The user interface continues to function as expected, and all systems have returned to normal operating conditions. If you continue to experience problems, please open a ticket with…

原因：From 00:00 UTC to 01:40 UTC, users may have experienced degraded write performance and intermittent impacts to workloads dependent on Block Storage Volumes in the NYC3 region. Our Engineering team has confirmed that the underlying infrastructure issue affecting Block Storage Volumes at the storage layer has been fully…

修复：From 00:00 UTC to 01:40 UTC, users may have experienced degraded write performance and intermittent impacts to workloads dependent on Block Storage Volumes in the NYC3 region. Our Engineering team has confirmed that the underlying infrastructure issue affecting Block Storage Volumes at the storage layer has been fully…

原因：As of the current time, our Engineering team has confirmed that the issue with Serverless Inference has been resolved. The root cause of the issue was an outage with our provider, Anthropic, which affected users of Sonnet 4.6 and Opus 4.7 models. According to Anthropic's status page (https://status.claude.com/incidents…

修复：As of the current time, our Engineering team has confirmed that the issue with Serverless Inference has been resolved. The root cause of the issue was an outage with our provider, Anthropic, which affected users of Sonnet 4.6 and Opus 4.7 models. According to Anthropic's status page (https://status.claude.com/incidents…

原因：Our engineering team continues to investigate an issue affecting our DNS service. At this time, DNS resolution remains functional; however, new changes to DNS records are currently not being reflected at the edge. This issue is also impacting related services. Specifically, customers may be unable to create new Let's…

修复：As of 19:50 UTC, the issue affecting the DNS service has been fully resolved, and all related services, including Let's Encrypt certificate issuance and Managed MongoDB provisioning, are now operating normally. The backlog of delayed requests has been successfully processed, and all DNS record updates, pending certifi…

修复：The remediation for the issue affecting the GradientAI Agent Platform Playground has been fully implemented, and the service is operating normally. We are no longer observing customer impact related to this issue, and the incident has been resolved.

原因：Between 5:35 and 14:25 UTC today, our Engineering team identified an issue that prevents enabling Two-Factor Authentication (2FA) and Google/GitHub authentication through the Control Panel. During this period, users might have encountered issues while enabling authentication methods and accessing teams with secure sign…

修复：Between 5:35 and 14:25 UTC today, our Engineering team identified an issue that prevents enabling Two-Factor Authentication (2FA) and Google/GitHub authentication through the Control Panel. During this period, users might have encountered issues while enabling authentication methods and accessing teams with secure sign…

修复：The upstream outage with Let's Encrypt has been resolved. Customers should now be able to issue Let's Encrypt certificates for Spaces, Load Balancers, and App Platform Custom Domains. Our Engineering team has also confirmed that stuck or delayed actions with Mongo, Advanced PG, and Advanced MySQL databases should compl…

修复：Our engineering team has resolved the issue with multiple services in NYC2 region, and all services should now be operating normally. If you continue to experience problems, please open a ticket with our support team. We apologize for any inconvenience and thank you for your patience.

修复：Service for Serverless Inference has been restored. We’ve implemented tighter rate limits to help prevent recurrence and are closely monitoring system performance. Some users may still experience intermittent latency as we complete final stabilization efforts. Our team remains actively engaged to ensure full recovery.…

修复：The issue is resolved, and service is operating normally.

修复：This incident has been resolved.

修复：Our Engineering team has confirmed that the issue impacting App Platform deployments and Kubernetes (DOKS) nodes has been fully resolved at 09:22 UTC. Users may already notice improvements while deploying apps and DOKS nodes. All App Platform deployments are now succeeding as expected. Customers who previously encount…

原因：Our Engineering team has resolved the issue that was impacting the DigitalOcean API and the Cloud Control Panel. The system is now operating normally. During the incident, users may have experienced difficulties accessing droplets, viewing the Droplets page, or performing CRUD (Create, Read, Update, Delete) operations…

修复：Our Engineering team has resolved the issue that was impacting the DigitalOcean API and the Cloud Control Panel. The system is now operating normally. During the incident, users may have experienced difficulties accessing droplets, viewing the Droplets page, or performing CRUD (Create, Read, Update, Delete) operations…

修复：Our Engineering team has confirmed that the issue affecting the DNS resolution of .co top-level domain (TLD) has been resolved. The DNS resolution for .co domains is now working as expected. We apologize for the inconvenience. If you continue to face any issues, please open a support ticket from within your account.

原因：From 17:22 to 17:46 UTC, our Engineering team observed an issue impacting Spaces availability in the NYC3 region. During this time, customers may have encountered 500 errors and degraded performance while accessing Spaces buckets. The issue has now been fully resolved. If you continue to experience problems, please ope…

修复：From 17:22 to 17:46 UTC, our Engineering team observed an issue impacting Spaces availability in the NYC3 region. During this time, customers may have encountered 500 errors and degraded performance while accessing Spaces buckets. The issue has now been fully resolved. If you continue to experience problems, please ope…

原因：Our Engineering team is investigating an issue causing elevated error rates due to service instability and terminating open connections which causes some 500s. Some requests may fail while we work to resolve it. We apologize for the inconvenience and will share an update once we have more information.

修复：Our Engineering team has implemented a fix for the issue causing elevated error rates due to service instability. We are currently monitoring the situation to ensure stability and confirm that error rates, including HTTP 500 responses, have returned to normal levels. We will provide a further update once we confirm th…

原因：From 16:07 to 16:50 UTC, Our Engineering team observed an issue with App Platform Deployments in all regions. During this time, App deployments of both new and existing apps would have been affected. Our team has fully resolved the issues as of 16:50 UTC. All new and existing App deployments should now be functioning…

修复：From 16:07 to 16:50 UTC, Our Engineering team observed an issue with App Platform Deployments in all regions. During this time, App deployments of both new and existing apps would have been affected. Our team has fully resolved the issues as of 16:50 UTC. All new and existing App deployments should now be functioning…

修复：Our Engineering team has resolved the issue with resize operations for Managed Databases and should now be operating normally. If you continue to experience problems, please open a ticket with our support team from within your Cloud Control Panel. We apologize for any inconvenience.

原因：Subject: Droplet Availability in All Regions Our Engineering team has identifed an issue with Droplet creates in all regions. A root cause has been found, a fix has been put in place and we are currently monitoring the situation to ensure full resolution. Users should be able to create new Droplets at this time. We…

修复：Subject: Droplet Availability in All Regions Our Engineering team has identifed an issue with Droplet creates in all regions. A root cause has been found, a fix has been put in place and we are currently monitoring the situation to ensure full resolution. Users should be able to create new Droplets at this time. We…

原因：Our Engineering team has resolved the control plane disruption that occurred from 17:06 to 17:18 UTC. During this time, users may have experienced intermittent issues with managing their resources through the Cloud Control Panel or DigitalOcean API. The root cause of the disruption was identified and addressed, and all…

修复：Our Engineering team has resolved the control plane disruption that occurred from 17:06 to 17:18 UTC. During this time, users may have experienced intermittent issues with managing their resources through the Cloud Control Panel or DigitalOcean API. The root cause of the disruption was identified and addressed, and all…

修复：Service has been fully restored, and the model is now operating normally. We have implemented improvements to enhance stability and reduce the likelihood of similar issues in the future.

修复：This incident has been resolved.

原因：The issue impacting delays in App Platform deployments has been confirmed to be resolved. Between approximately 00:08am UTC & 11:46am UTC, users may have noticed delays while creating or updating apps, or may have encountered failed deployments. For failed deployments, please trigger a redeploy, which should successful…

修复：The issue impacting delays in App Platform deployments has been confirmed to be resolved. Between approximately 00:08am UTC & 11:46am UTC, users may have noticed delays while creating or updating apps, or may have encountered failed deployments. For failed deployments, please trigger a redeploy, which should successful…

修复：Our Engineering team has implemented a fix, the issues impacting Gradient AI Platform have been resolved. All agents are back up and healthy. Service has been fully restored.