Due to cbtec Video being a small, independent project, we use third party sources for most of our servers, including those hosting cbtec Video. One of our hosting providers is German server hosting company Contabo. One of the drawbacks of hosting externally is the fact that we do not have physical control over the servers, meaning that in the case of a natural disaster, we are unable to migrate the data over to another server. Today, 2nd September 2024, we were made aware of an outage on our core server EU-001 that is responsible for the main site and the backend services behind cbtec Video. We began investigating the cause and found that the issue related to our hosting provider, Contabo, and that there was nothing we could do to recover our service.
Post-outage, we received more information from Contabo about the details of the outage:
‘On early morning 02/09, a storm formed in the area around the Contabo Nuremberg data centre and resulted in the ventilation system being struck by lightning. This resulted in damage to the cooling systems for the servers at the data centre. Due to the danger of having many servers running simultaneously without correct cooling, the many servers were shut down as a safety measure until the ventilation system was restored to full capacity.’ This affected cbtec Video, as our servers were hosted in the affected data centre.
We are now putting new measures in place on our other servers, hosted outside of Contabo, to add redundancy to our network, so that, in the event that this happens again, we are able to quickly switch over to backup systems.