Enable HTTPS in .NET APIs with Manual Let's Encrypt

September 04, 2022 (~13 Minute Read)
HTTPS TLS SSL WEB API LETS ENCRYPT DNS WILDCARD

Introduction

So, we have created our Web API, and it works as expected 🎉. However, all transferred data (e.g., in the URL, POST, etc.) are open to external access without a security protocol. Therefore, the transferred data could be accessed or altered before the Web API server receives them. Furthermore, let’s not forget that Web API requests and responses (all network packages) may travel in multiple networks to reach their destination.

The good news is that we can improve our Web API communication with integrity and security against unauthorized access by encrypting the transferred data. This is where SSL (Secure Socket Layer) and TLS (Transport Layer Security) cryptographic protocols come to the rescue.

SSL and TLS are protocols for encrypting, securing, and authenticating web communications (client and server, emails, VoIP, etc.). The SSL protocol was replaced by the TLS protocol (in 1999). However, we commonly use the SSL term for both protocols. Secure connections are indicated by the HTTPS (Hyper Text Transfer Protocol Secure) in the URL instead of HTTP.

In this article, we will learn the basics of the TLS protocol and the Let’s Encrypt authority. In addition, we will use a tool to manually generate a Wildcard certificate using the DNS challenge and install it in NGINX and Kestrel.

The TLS Protocol

The current version of TLS is 1.3 (defined in August 2018). However, TLS 1.2 is still used. You can read here about the difference between TLS 1.3 and TLS 1.2 and the advantages of using TLS 1.3.

In short, the TLS communication sessions begin with a TLS handshake, which is based on asymmetric encryption. In asymmetric encryption (aka public-key cryptography), a pair of related keys are used, a public key and private key. The public key can be used to encrypt a message so that it can only be decrypted with the private key. Thus, the public key is available publicly and the private key (aka secret key) is kept secret and only used on the server side.

During the TLS handshake, the client and server use the public and private keys respectively to exchange some random generated data. This random data is used to create new keys for encryption, called session keys. After the TLS handshake, both sides use the same session keys for encryption. The session keys are temporal, meaning that they are used until the session is terminated.

Let’s Encrypt Certificate Authority

We will need a certificate from a Certificate Authority (CA) to enable HTTPS on our Web API. The certificate (public key) is a file that the CA provides with a private key file and a certificate chain file.

The Certificate Authority is responsible for validating (challenge) the domain name ownership, generating the certificate, and making it publicly available. Let’s Encrypt is a free, automated, and open certificate authority, run for the public’s benefit.

In the following table, we can see the most common challenges (HTTP and DNS). A wildcard certificate protects a root domain name (e.g. example.com) and all its subdomains (e.g. www.example.com, staging.example.com, blog.example.com, etc.).

Manual Certificate Generation using Certbot

Certbot is a client application that fetches a certificate from Let’s Encrypt. Usually, we run it directly on our production server and not on our personal computer. However, there are cases in which we would like to control this process to make our automation based on our system requirements.

In this section, we will learn how to use the Certbot client to generate a wildcard certificate manually (on our personal computer), which we could upload to our production server based on our release process.

Step 0: Install Prerequisites

In this tutorial, we will not install Certbot on our personal computer, but we will use its official Docker image (certbot/certbot). If you do not have Docker installed, you can follow these instructions to download and install it.

Step 1: Start a Let’s Encrypt Challenge

We will use the DNS Challenge to generate a Wildcard certificate by running the following command in Windows PowerShell. It is essential to replace the example.com and the [email protected] with yours before running the command. In addition, we can see how to set the path to store the certificate files. This path is vital for your Web APIs, and I recommend having backups. Figure 3 shows that this command will start the DNS challenge.

cd <A local path to store the certificates, e.g. E:\SSL>

docker run -it --rm --name certbot `
--volume "${PWD}/etc-letsencrypt:/etc/letsencrypt" `
--volume "${PWD}/var-lib-letsencrypt:/var/lib/letsencrypt" `
certbot/certbot certonly `
--manual --manual-public-ip-logging-ok `
--preferred-challenges dns `
--email [email protected] --agree-tos `
--domain *.example.com `
--domain example.com `
--rsa-key-size 2048

Step 2: Add and Verify Challenge Proof

In this step, we must prove to Let’s Encrypt that we control the domain name. We have selected the DNS challenge. So, we need to add the specified DNS TXT record(s) to our DNS provider and Verify that the record(s) is available before continuing.

To verify that our TXT records are available, we can use the digwebinterface.com online dig tool to query the DNS records. Figure 5 shows how this tool can query the required TXT records in some main DNS servers.

After we verify both TXT records, we can press Enter to continue. If everything goes well, we will see the following result. In case of a timeout error, etc., we can start a new challenge (Step 1). It’s essential to notice that the Let’s Encrypt certificates are valid for 90 days. So, we can either repeat this process manually before its expiration or automate it.

Step 3: Install the Certificate

Hooray! 🎉We have generated our certificate. Our last step is to install the certificate, which depends on how we serve our Web API. In the following sections, we will see examples of installing our certificate in NGINX and Kestrel.

However, let’s start by finding the generated keys and issued certificates in the \etc-letsencrypt\live\{Our domain name}\ folder. The following table shows the generated files.

NGINX HTTPS Using PEM Certificate

We are serving our Web API with NGINX as a reverse proxy server in this example. First, we have placed the privkey.pem and fullchain.pem files in an ssl folder in NGINX. Finally, we configured the NGINX (nginx.conf) to serve contents using our server certificate (as shown below). This is an example that you should modify based on your use case.

http {
  #... other stuff
  
  # Use SSL/TLS on all sites.
  
  # Alternatively, use the following SSL configurations per "server".

  ssl_session_cache         shared:SSL:10m;
  ssl_session_timeout       10m;
 
  ssl_certificate           ssl/fullchain.pem;
  ssl_certificate_key       ssl/privkey.pem;
  
  ssl_protocols             TLSv1.2;
  ssl_prefer_server_ciphers on;
  ssl_ciphers               HIGH:!aNULL:!MD5;
  ssl_ecdh_curve            secp384r1;
  ssl_session_cache         shared:SSL:10m;
  ssl_session_tickets       off;
  ssl_stapling              on; #ensure your cert is capable

  ssl_stapling_verify       on; #ensure your cert is capable

  
  # Redirect (301) All Sites to HTTPS

  server {
​    listen 80 default_server;
​    listen [::]:80 default_server;
​    server_name _;
​    return 301 https://$host$request_uri;
  }
  
  # API Server Exampl

  server {
​    listen 443 ssl;
​    listen [::]:443 ssl;
​    
​    # TODO: Set your server name

​    server_name api.example.com;  
​    
​    # TODO: Set your API location and internal path (proxy_pass).

​    # FYI: If the longest matching prefix location has the ^~ modifier, then NGINX will immediately end its search and select this location to serve the request.

​    location ^~ /api/ {
​      proxy_pass               http://example.web.api/api/;
 
​      # Configure Nginx as a reverse proxy to forward requests to your ASP.NET Core app:

​      proxy_http_version       1.1;
​      proxy_set_header         Upgrade $http_upgrade;
​      proxy_set_header         Connection keep-alive;
​      proxy_set_header         Host $host;
​      proxy_cache_bypass       $http_upgrade;
​      proxy_set_header         X-Forwarded-For $proxy_add_x_forwarded_for;
​      proxy_set_header         X-Forwarded-Proto $scheme;
​      
​      client_max_body_size     5M;
​      client_body_buffer_size  128k;
​      proxy_connect_timeout    90;
​      proxy_send_timeout       90;
​      proxy_read_timeout       90;
​      proxy_buffers            32 4k;
 
​      break;
​    }
  }
}

Kestrel HTTPS Using PEM Certificate

We are serving Web API directly from Kestrel server in this example. Therefore, to serve the Web API contents using our server certificates, we should configure Kestrel by adding the following (Kestrel section) in the appsetting.json file.

{
    "Kestrel": {
        "Certificates": {
            "Default": {
                "Path": "your_path/fullchain.pem",
                "KeyPath": "your_path/privkey.pem"
            }
        }
    }
}

Summary

Our Web APIs may provide or receive sensitive data that can be accessed or altered without using a security protocol. TLS (commonly used with the SSL term) is a protocol for encrypting, securing, and authenticating web communications (client and server, emails, VoIP, etc.).

Let’s Encrypt is a free, automated, and open certificate authority run for the public’s benefit. So, we can use Let’s Encrypt to generate certificates per domain or/and a wildcard (all subdomains). HTTP and DNS are the most common ways to verify (challenge) domain name ownership.

In this article, we used the Certbot client application to generate a Wildcard certificate using the DNS challenge manually. Then, we learned how to install our certificate in NGINX and Kestrel.

With Let’s Encrypt, we have no excuses! We can secure our Web APIs with a Free Let’s Encrypt certificate and, based on our needs, decide to use a paid certificate.