The global Siemens.com-website is a regular target of cyberattacks. Most are standard and barely cause a blip. In the last week of July though we experienced a targeted Distributed-Denial-of-Service-attack that caught Siemens’s cybersecurity experts’ attention – and showed how it’s possible to thwart larger attacks with the right safeguards and, of course, a good team in place.

Screenshot of www.siemens.com start page

Starting in the morning on July 25, 2022, Siemens faced a series of distributed-denial-of-service (DDoS) attacks. These attacks use more than one unique IP address, often from thousands of hosts infected with malware – so-called botnets.

The incident didn’t come as a complete surprise, as the number of DDoS attacks is steadily increasing. They are simple and inexpensive to execute, and their main goal is to sabotage the target. They are used by various groups, among them “hacktivists” who pay in bitcoin and rent these services in the Dark Web. In this case, the attacks made it impossible for Siemens.com to offer its online services in some regions within a short time frame.

How did it happen? Generally speaking, Siemens faces thousands of attacks daily, but most are weeded out thanks to customizable services offered by e.g. Amazon Web Services (AWS). The DDoS attacks in July using a botnet of several thousand IP addresses managed to bypass these safeguards, causing a significant increase in traffic. It was a massive, targeted attack, and it could have been even bigger. Thankfully, the team at Siemens took the decisive steps required.

For a global company such as Siemens, a state-of-the art cybersecurity setup is essential, it allows us to react as fast as possible.
Fabian Fischer, Software Architect Communications Technology Delivery

Thousands of entry points

But what helped the attackers to be partially successful? The main reason is that repeated requests from a single IP-address may initially appear to be benign – it is only with unusually high traffic, deployed using thousands of such addresses at the same time, that one can identify a botnet attack.

Yet, the way the system is designed helps to defend against such attacks. If there was only a single entry-point for accessing the Siemens.com-website, the system quickly would come crashing down. But there are literally thousands of entry points around the world offered by service providers such as AWS. It’s a consequence of what is called “serverless computing”. The infrastructure provider does all the heavy lifting: maintaining hardware, patching operating systems, providing network connectivity, and other infrastructure legwork – while the client cares about the computing power.

The great benefit of the serverless approach is that it makes computing power available virtually anywhere. It enables the best website customer experience. For example, providers will accelerate delivery by executing part of the application code on these edge servers. Siemens builds its applications on top of this Content Delivery Network (CDN) feature. Its main goal is to speed up the delivery of website content by bringing it closer to the end user. That works great – until it doesn’t because of a cyberattack.

Characteristics of a DDoS-attack

A large DDoS attack is, arguably, the worst-case scenario for organizations that rely on global websites to connect with their customers. To detect an attack, the first simple step is to know what’s normal. At Siemens.com, for instance, traffic starts off low before a bump in the morning (European time). A lull at lunchtime is followed by a second bump in the afternoon when US-based users also start to go online.

The second step is to have close monitoring of deviations of the normal state. Hence, we know that during DDoS attacks, traffic patterns quickly jump out compared to normal business days.

As traffic rapidly increased end of July, we received the first alerts concerning intermittent availability problems. After a quick evaluation, we found evidence that the site was hitting request limits in some regions. Further investigation showed that a DDoS attack was ongoing, peaking at about 85,000 requests per second (a normal scenario would be 500-1,000 requests per second during European morning hours).

In response to the first attack wave, which lasted 24 minutes, we increased the limits in all affected regions to provide more headroom for the sheer amount of traffic. But that’s a double-edged sword, as this also allowed more requests to come through. And that’s why, as a second step and in response to two more attack waves that followed (peaking at about 116,000 requests per second), the limit of requests was lowered. But of course, not blindly.

Certain rules were applied, such as limiting the number of allowed repeated requests from a single IP address to 500 within five minutes, and enforcing CAPTCHAs to determine whether a human or a machine is involved. Also, a whitelist for Siemens-proxies, specifically prepared for such cases, was applied. While this may have limited some legitimate requests, knowing that an attack was happening, it was a price worth paying. We were sure that even if legitimate users would have been blocked by the stricter rules, they could have solved the CAPTCHA and enter the website without any issues.

Resilient and self-healing

The result: during the final – and largest – stage of the attack, the site was available to most users without issue. Meaning: The biggest impact happened during the first attack, even though it was smaller compared to the following attack waves. But by then the team had put appropriate countermeasures in place.

The bottom line is that our infrastructure proved to be resilient the way it was designed to be. The “under the radar” attack pattern was designed to sneak through the automated threat detection of the AWS Shield. Yet, with alarm bells going off immediately, and countermeasures being put in place, the team managed to keep the website online with the appropriate measures. Nevertheless, as these attacks don’t cost attackers much, they are expected to continue.

And that’s why our cybersecurity experts and the siemens.com communications technology delivery team need to remain vigilant.