Keep your eyes open: How to avoid exposing your internal proxy

As you know, DDoS attacks are pretty popular nowadays. Generally, we use services like Cloudflare to protect against them as these services absorb malicious traffic. But even additional protection mechanisms (e.g., “I’m Under Attack” mode combined with a white list in Cloudflare) will not help, if you disclose your IP address to the outside world (without even noticing it). In this article, we will talk about one of our real-life cases and discuss various ways to prevent them (and resulting DDoS attacks).

Preface. A few words about the scanners

As a starting point for discovering security breaches in your infrastructure, you can use some widely available services.

Here are examples of well-known systems that will help you to find vulnerable or incorrectly configured devices:

• Shodan is a search engine that allows you to use various filters to find Internet-connected devices;
• Censys.io is a novel, IoT-based search engine that, like Shodan, scans all publicly available IP addresses and saves their responses. You can use the resulting list of devices (map) to search for various vulnerabilities or monitor the current state of the network infrastructure;
• CloudFail is a tactical reconnaissance tool which aims to gather all available information about a service protected by Cloudflare. It uses this information to discover the location of the server;
• SecurityTrails is a set of tools for gathering information on a domain name, IP address, and WHOIS data. Part of the data is provided free of charge on the web platform with the same name. Additional information is available via the commercial API;
• Project Sonar “conducts internet-wide surveys across more than 70 different services and protocols to gain insights into global exposure to common vulnerabilities.”

UPDATE (March’21): Thanks to Roman Romanchenko who has shared with us another useful tool:

• Spyse — advertised as “the most complete Internet assets registry for every cybersecurity professional” — helps you to make a quick target overview in the way similar to Shodan.

We have used one of the above tools — Censys — in our real-life case discussed below.

Our case

Here is what we’ve had:

• test.com domain;
• Cloudflare as a DNS service;
• an NGINX load balancer containing certificates for 1.test.com, 2.test.com, *.test.com.

What’s been the problem? Well, NGINX serves a certificate (containing our test.com domain) when there is a request to port 443. Thus, an attacker can map an IP address of the balancer with the domain name and run a DDoS attack.

For example, our domain (test.com) uses GCORE servers. You can simply enter this data into the search bar of one of the widely known scanners and get a complete mapping of IP addresses with the domain names specified in the certificate. But should not these addresses be hidden from public access?..

Here is the related Censys screenshot:

As the above picture shows, you can get the IP address of the server by matching the public IP address with the domain contained in the certificate provided by NGINX. An attacker can get it this way and take advantage of it.

How the problem has been solved

In our case, we solved the problem by creating a self-signed certificate for the default_server containing example.com as a domain. We added the following lines to the NGINX config on the load balancer:

server {
  listen 443 default_server ssl http2;  ssl_certificate /etc/nginx/ssl/examplecom/tls.crt;
  ssl_certificate_key /etc/nginx/ssl/examplecom/tls.key;
}

Thus, with all future IP scans, NGINX will serve a certificate that would not be associated with the real domain (test.com). The example.com domain will be served in place of the real domain, so the attacker won’t be able to associate the domain with the load balancer’s IP address.

At the same time, you will need to replace the exposed IP address of the server with the new one.

General recommendations

Here goes a more general look at the problem accompanied by a list of advices that you should pay attention to if you want to prevent such situations from happening:

1. After the site is protected by an external service (such as the aforementioned Cloudflare), change the IP address of the server. In this case, DNS would serve an address of the proxy server, and the address of the original server would remain confidential.
2. Using whitelist, grant the direct access to your site from IP addresses of the proxying service (Cloudflare) only. As a result, scanners (and all the outside world) will not be able to see what content is being sent to the specific IP address/host.
3. To preserve the anonymity of the web server, you should set a default vhost that is not related to your company/website, and use a self-signed certificate specifying some fake domain such as youshallnotpass.com (or example.com).
4. A good idea is to replace the default Apache or NGINX page with something like “It works!” or similar. At the same time, you should hide the version of the web server (nginx how-to, Apache how-to). Thus, you would protect yourselves from scanners gathering IP addresses, making it more difficult for attackers to discover the real address of your web service.
5. Using an SSL certificate is a good and elegant solution. However, keep in mind that information about the issued certificate is recorded in the Certificate Transparency log (the certificate issuance registration and monitoring system), which is publicly available. In other words, if you issue a new certificate to your domain, it will be published. The attacker can easily find it and discover the hostname.Here is an example output of crt.sh certificate search engine for the example.com domain:You can see when the certificate was updated as well as view the contents of all previous certificates
6. Delete all DNS records that are not used. As you know, once something ends up on the Internet, it stays there forever. This is also true for DNS records. In addition to online services that you can use to find old DNS records for a specific domain (such as SecurityTrails), historical data can also be found in the Project Sonar database.
7. You can also put a cloud load balancer (e.g., Amazon ELB) in front of the server for higher safety. It will provide an additional layer of protection. However, this recommendation is only applicable if you (or your end-users, to be more precise) can tolerate an increase in latency.

Conclusion

SSL/TLS is the basis of a secure and safe Internet. As far as web services are concerned, it is not only suitable for obvious areas, such as processing credit card data. It ensures confidentiality, critical-level security, and integrity of the data for both the sites and their users.

There is a wide variety of services that provide additional protection to websites by fighting DDoS attacks, facilitating the issuance of certificates, and so on. However, all these tools will not help if your infrastructure is incorrectly configured in the first place. You can start searching for potential security challenges using scanners mentioned at the beginning of this article.