LunaNode

修复：the maintenance has completed. Services should be back online at this time. If your VM is still offline, please reach out to support@lunanode.com.

原因：We are investigating disk issue on this hypervisor. The hypervisor is currently offline due to the issue.

修复：issue resolved.

原因：There will be up to 6 hours of network downtime (although much shorter downtime is expected) due to maintenance being performed by our datacenter (Cogent Toronto). They say "The purpose of this work is to apply and activate software module upgrades on code running on the edge routers per the manufacturer's recommendati…

原因：We are currently aware of a networking issue causing packet loss in our Toronto datacenter. Our upstream provider has identified the root cause and is actively working on a resolution. We will provide further updates as more i nformation becomes available.

修复：issue resolved.

原因：We are currently aware of a networking issue causing packet loss in our Toronto datacenter. Our upstream provider has identified the root cause and is actively working on a resolution. We will provide further updates as more information becomes available.

修复：issue resolved.

原因：There will be up to 3 hours of network downtime (although much shorter downtime is expected) due to maintenance being performed by our datacenter (Cogent Toronto). They say "The purpose of this work is to apply and activate software module upgrades on code running on the edge routers per the manufacturer's recommendati…

修复：services have been restored, while we continue to monitor the situation we do not anticipate more service interruptions.

原因：the filesystem repair is complete and virtual machines have been restored. Data on four virtual machines was lost due to disk corruption. We will compensate affected users with three months of credit for downtime and twelve months of credit for data loss -- please open a ticket to request the credit be added to your ac…

修复：the filesystem repair is complete and virtual machines have been restored. Data on four virtual machines was lost due to disk corruption. We will compensate affected users with three months of credit for downtime and twelve months of credit for data loss -- please open a ticket to request the credit be added to your ac…

修复：the maintenance is completed.

原因：services are back online at this time. The downtime occurred due to a hardware issue with one of the SSDs on the hypevisor; the SSD needed to be replaced but due to the failure mode (where it did not immediately fail) it also caused data corruption to the filesystem. The filesystem is now repaired but at least five vir…

修复：services are back online at this time. The downtime occurred due to a hardware issue with one of the SSDs on the hypevisor; the SSD needed to be replaced but due to the failure mode (where it did not immediately fail) it also caused data corruption to the filesystem. The filesystem is now repaired but at least five vir…

修复：services on this hypervisor are back online.

原因：Cogent continues to work on the power issue, no additional updates

修复：It looks like service has been restored, we will continue to monitor.

原因：We rebooted this hypervisor due to abnormal system errors. We will continue monitoring the stability of the hypervisor in case the reboot does not address the problems.

原因：There will be up to one hour of network downtime due to maintenance being performed by our datacenter (Cogent Toronto). They say "the purpose of this maintenance is Network Maintenance hardware and software upgrades reboot required."

修复：the maintenance is completed.

修复：(Completed)

修复：the server is back online.

原因：We rebooted this hypervisor due to abnormal system errors. We will continue monitoring the stability of the hypervisor in case the reboot does not address the problems.

原因：Hypervisor down due to disk errors

修复：The hypervisor is back online after raid controller replacement.

原因：NameSilo has re-activated lndyn.com. Services should be restored at this time. We have not received additional information about the original reason for the suspension yet, just notification that it was re-activated. We are in the process of migrating all of our domains off of NameSilo due to the 24 hours of downtime t…

修复：NameSilo has re-activated lndyn.com. Services should be restored at this time. We have not received additional information about the original reason for the suspension yet, just notification that it was re-activated. We are in the process of migrating all of our domains off of NameSilo due to the 24 hours of downtime t…

原因：One hypervisor in Toronto (cbec3404b692) was down for approximately 10 minutes due to disk issue. The hypervisor is restored now.

修复：One hypervisor in Toronto (cbec3404b692) was down for approximately 10 minutes due to disk issue. The hypervisor is restored now.

原因：a prolonged outage occurred today due to incorrect identification of affected hypervisor while diagnosing the issue. Once we identified the correct hypervisor, the hypervisor was brought back online by replacing a faulty disk. Network connectivity to VMs that were using this hypervisor as network node was affected as w…

修复：a prolonged outage occurred today due to incorrect identification of affected hypervisor while diagnosing the issue. Once we identified the correct hypervisor, the hypervisor was brought back online by replacing a faulty disk. Network connectivity to VMs that were using this hypervisor as network node was affected as w…

原因：we have heard from the datacenter that the power issue was due to breaker tripping incorrectly. Cogent says they have swapped the breaker with a new breaker and we should not experience any further issues with that unit.

修复：most services are restored at this time. Remaining services should be online shortly.

原因：packet loss has come up again. It is due to DDoS attack targeting multiple IPs making it difficult to null route. We are working again to restore connectivity.

修复：services should be restored now.

原因：One Toronto hypervisor (a4120f5bedfa) offline for 20-30 minutes due to hardware failure (disk plane). Services are restored after swapping the disks to a spare physical server.

修复：One Toronto hypervisor (a4120f5bedfa) offline for 20-30 minutes due to hardware failure (disk plane). Services are restored after swapping the disks to a spare physical server.

原因：One Toronto hypervisor (1ee830f71342) has one power supply failed. Due to mismatch in firmware we were unable to swap the failed power supply with one in stock.

修复：services have been restored.

修复：services have been restored.

修复：Our upstream provider in Toronto is performing a software update on the core router resulting in intermitten network issues. We expect the update to be completed within the next 30 minutes..

原因：The network issues due to router malfunction similar to "19 October 2022" issue have recurred. The latest network downtime was for two minutes at 11:00 EST. We upgraded router firmware last night as change list indicated it may solve the problem but it did not. We are continuing to look for alternative solutions as pre…

修复：services are back online.

原因：b4fa814b28f1 is online now. The outage started during routine maintenance and checkup of power bar in one rack. However, power utilization surged seemingly due to faulty network switch and tripped two breakers. After power was restored, several hypervisors had issues booting, most of them booted after removing disks no…

修复：b4fa814b28f1 is online now. The outage started during routine maintenance and checkup of power bar in one rack. However, power utilization surged seemingly due to faulty network switch and tripped two breakers. After power was restored, several hypervisors had issues booting, most of them booted after removing disks no…

原因：One SSD hypervisor in Montreal is offline, it is due to datacenter internal network issue and we are waiting for response.

修复：controller node is back online, controller panel functions restored

原因：One hypervisor in Toronto has failed due to hardware issue. We are investigating.

修复：raid array repair complete and services have been restored

原因：there was another brief outage due to router issue today morning, tonight at around 21:00 EDT we will swap sfp modules to see if it is issue with an sfp module.

修复：we have not seen more outages, so it appears that the sfp module replacement on 21 October 2022 has resolved the issue despite the one final outage after the replacement.

原因：Our upstream provider, Cogent Communications, will be performing network maintenance in the Toronto datacenter starting midnight of 29 July 2022. The maintenance will cause network downtime in our Toronto region for up to 60 minutes between 30 July 2022 00:00 EST and 05:00 EST.

原因：This appears due to datacenter outage of rack 46-D06 ( http://vms.status-ovhcloud.com/index_rbx6.html ). We are waiting for reply from the datacenter (OVH).

修复：all services back online again.

原因：the issues in Toronto today are caused by failure in the datacenter HVAC system due to a recent storm, meaning that a wide range of equipment is overheating. We will post updates as we receive more information from the datacenter (Cogent Toronto).

修复：HVAC units are being restored and temperature alerts on the servers are clearing

修复：The networking upgrade has been completed, virtual machines had to be rebooted in the region for the new configuration to take effect. We apologize for the inconvinience.

原因：There is downtime on some services in Montreal due to network maintenance in the datacenter. See OVH webpage for details.

修复：Update: services are back online.

原因：The same Montreal SSD hypervisor (55d26960353e) was offline again due to an unrelated issue (OVH vRack outage), until approximately 19:10 EST.

修复：services are back online. We believe the recurring kernel panic is resolved.

修复：services are back online.

修复：services are back online.

修复：We have resolved downtime on one Toronto SSD hypervisor (b3700f1d6136). Services are back online at this time (06:03 EST).

修复：services are back online.

原因：the second disk that was giving read errors has now also been replaced, and the RAID1 building has finished. At this time both disks have been replaced and the RAID1 array is in good status.

修复：services should be restored at this time. Please open ticket if your VM is still offline.

修复：services are back online.

原因：We are investigating network downtime in Roubaix. Services appear to be offline due to issue with vrack infrastructure from the datacenter that we use in Roubaix (OVH).

修复：services have been restored.

修复：Services have been restored. It is unclear at this time whether outage is caused by datacenter infrastructure issue or our network node server having kernel lockup. We will continue to investigate.

修复：Services have been restored.

原因：Due to a detected potential disk issue on Roubaix hypervisor e3ba7defacb5, we will perform emergency maintenance involving migrating all virtual machines off the hypervisor to other hypervisors. This will involve a reboot of each VM with 2-10 minutes downtime depending on the disk size of the VM. The maintenance will b…

原因：the maintenance was performed at 23:45 instead due to some issues. But it is done now with up to 2 minute network interruption to most services.

修复：services are back online.

原因：One Montreal SSD hypervisor (de6e4fa83fdc) is offline for emergency maintenance due to disk issue. We expect 10-20 minutes downtime.

修复：RFO: our monitoring system detected that one disk in RAID10 group failed today morning. We use service from OVH in Montreal and Roubaix and requested disk replacement. It seems they needed to turn server off to replace the disk instead of hot-swapping it. But now the RAID10 group is restored to normal. Note that in our…

修复：services are back online.

原因：the downtime was due to kernel panic. We will check further to determine if the current kernel is sufficient to avoid future recurrence of this downtime incident.

修复：services are back online.

原因：We perform emergency maintenance on hypervisor 7fd1830faf11 in Toronto due to memory bugs associated with old kernel. Downtime ten minutes needed to update to new kernel to resolve these issues.

修复：services are back online.

原因：Services were offline from 21:10 EDT to 21:20 EDT due to upstream issue. See http://travaux.ovh.net/?do=details&id=46759 for details.

原因：Again network is down due to upstream (OVH) vrack network issue.

修复：services are back online.

原因：Network is down due to upstream (OVH) vrack issue. http://travaux.ovh.net/?do=details&id=45816

修复：services are back online.

原因：Hypervisor 1ee830f71342 is offline due to power cable tension issue and technician error during installation/migration of equipment for/to new 20A circuit. Services should be back online in five to ten minutes.

修复：services are back online.

原因：RFO Update (06 July 2020 21:00 EDT) : Outage summary: at 06 July 2020 10:05 EDT there was power surge at Cogent's Toronto 245 Consumers Rd datacenter which led to server restarts and one PDU failure in one of our racks. (Other Cogent customers were also impacted and throughout the day the datacenter was quite crowded.)…

修复：RFO Update (06 July 2020 21:00 EDT) : Outage summary: at 06 July 2020 10:05 EDT there was power surge at Cogent's Toronto 245 Consumers Rd datacenter which led to server restarts and one PDU failure in one of our racks. (Other Cogent customers were also impacted and throughout the day the datacenter was quite crowded.)…

修复：services are back online after replacing hardware component.

修复：the reboot last night did not solve the latency problem, but we apply additional tuning steps today and confirm the issue is resolved and latency is low and stable.

原因：finally the datacenter has issue open about this incident: http://travaux.ovh.net/?do=details&id=45319& . Also at this time the network connectivity is fully offline. We observe heavy packet loss in Roubaix starting at 14:40 EDT due to datacenter issue. We are contacting the datacenter to investigate this problem.

修复：the network is back online now but it may still be very unstable. We have received absolutely no update from the datacenter.

原因：Roubaix services are offline due to datacenter (OVH) issue.

修复：services are back online.

修复：services are back online after five minutes.

原因：services are back online after eight minutes. We need to verify the errors are gone shortly.

修复：services are back online after eight minutes. We need to verify the errors are gone shortly.

修复：services are back online.

原因：One Montreal SSD hypervisor has network downtime due to datacenter vrack issue (OVH).

修复：they fix it, back online.

原因：One Roubaix SSD hypervisor is offline for ten minutes from 16:45 to 16:55 EDT due to emergency maintenance to correct kernel issue causing high latency and dropped packets.

修复：services are back online.

原因：we still see 1% packet loss in Toronto. It does not appear to be datacenter-wide issue. Our team is on-site and still investigating. We may switch to backup router or perform other similar actions that may cause brief network disruptions (less than one minute).

修复：we do not see any further issues. Update 3 (09 April 2020 13:25 EDT) : services are back online at this time. We continue to monitor the hypervisor, but we believe the issues should be resolved.

原因：We see network downtime due to datacenter issue at this time.

修复：services are back online at this time. Here is OVH issue.

原因：There may be 30-60 minutes of network downtime between 00:00 EDT and 03:00 EDT in Toronto due to Cogent planned network maintenance.

修复：services are back online at this time.

原因：Internal and external network for VMs on one Montreal hypervisor is offline due to datacenter issue. We are in communication with OVH to resolve the problem.

修复：services are back online at this time. Here is OVH issue.

修复：services are back online at this time.

原因：Due to substantial cost increases imposed by the datacenter we use in Montreal and Roubaix, we are sunsetting those locations on 31 January 2023.

补偿/善后：Make sure to migrate your VMs before 31 January 2023. You will receive three months of free credit for VMs migrated before 31 January 2023.