A bastion castle

Using a Bastion Host to access your AWS EC2 Instances

Following on from our article on running a static website in S3 , this time out we’re looking at deploying a Bastion host in the AWS cloud. This is not a new idea but I only recently came across this architecture design so I’m guessing other folks may not know about it too. Many companies use a VPN or direct connect to connect with the AWS cloud. However, if you don’t have either of these, a bastion host is the way to go.

The idea is to have two subnets, a public and a private one. The public is effectively a DMZ and provides access to and from the internet. The private subnet is where you run your applications. Using security groups, routing tables and tools like masquerading, this architecture provides a secure way of accessing your applications from the Internet.

Bastiom host design

The diagram shows the architecture of this solution. (I must give a quick plug to draw.io which I used to create this work of art. Not only is it easier to use than Visio, it’s free!). If any of the terms used in this article are unfamiliar to you, you can always check out our AWS glossary .

As you can hopefully see, the VPC contains the two subnets: the public subnet and the private subnet. The public subnet will be set up with a routing table that connects it to an Internet gateway. This means that any EC2 instances launched in that subnet will be available via a public IP address, i.e. you’ll be able to connect to them from the Internet.

The private subnet however, will be configured with a local routing table that restricts it to contacting devices within either the private subnet or the public subnet. This means only authorised users can access this subnet.

In this design we have allowed VMs running in the private subnet to be able to connect to the internet to facilitate server patching, etc. To ensure this is done in a safe way, a NAT (Network Address Translation) server is deployed public subnet which VMs in the private subnet use to access the internet. As part of this procedure, a specific route is configured between the private subnet and the NAT server to allow this.

In order for you to be able to access this environment, a Bastion server is set up in the public internet. This provides a jump box, you log into this from the internet and from there, you can log into the private subnet VMs. Access to the Bastion server will be restricted via a Security Group and it will be configured to allow secure access of the internal VMs.

Enough chat, let’s get started. First off you need to create the VPC.

Log into the AWS Console and under Network & Content Delivery, select VPC, click on create VPC and in the dialog box enter:

  • Name tag: My Test VPC
  • CIDR Block: 10.10.0.0/16
  • Leave the Tenancy field set to Default so the VPC will use shared hardware (rather than dedicated for your use)

The CIDR block specified means that you can use IP addresses ranging from 10.10.0.0 to 10.10.255.255 in the VPC created (NOTE: This is in the private CIDR block range 10.0.0.0 – 10.255.255.255 which are for internal private networks). If you’re not familiar with subnets and masks, check out our subnetting article .

Next we need to create the public and private subnets

Whilst you’re still in the VPC Dashboard screen, select Subnets and click the Create Subnet button. In the Create Subnet dialog box enter:

  • Name tag: Public Subnet
  • VPC: Select the VPC just created (My Test VPC).
  • Availability Zone: Select any that’s valid for you (usually your nearest one)
  • CIDR Block: 10.10.0.0/24. (IP addresses 10.10.0.0 through 10.10.0.255 will be available in this subnet).

To create the private subnet, repeat the previous steps but use the values:

  • Name tag: Private Subnet
  • CIDR Block: 10.10.1.0/24
  • Availability Zone: Same as used for the Public Subnet

From the subnets listed, select Public Subnet and click Modify Auto-Assign Public IP and select the Enable auto-assign Public IP check box.

Usually you’ll need instances in the public subnet to be assigned public IP addresses by default i.e. an internet facing IP address. By setting the Auto-Assign Public IP attribute on the public subnet, this will be the default behavior for new instances created in this subnet.

Next we need to create the Internet gateway

As the name suggests, an Internet gateway allows communication between instances in a VPC and the Internet. It has two purposes: to provide a target in the VPC route tables for Internet bound traffic, and to perform network address translation (NAT) for instances that have been assigned public IPv4 addresses as their private IP addresses cannot be exposed to the internet.

Still in the VPC Dashboard, select Internet Gateways and click Create Internet Gateway. In the dialog box, in the Name tag box enter: Public Subnet Gateway and click Yes, Create.

Next, select the gateway that you just created and click Attach to VPC. In the Attach to VPC dialog box, in the VPC drop-down list, select My Test VP and click Yes, Attach. It should end up looking something like this (available and attached).

Internet Gateway

 

 

 

 

 

 

Now we need to create the route for the public subnet that enables traffic through the Internet gateway just created.

Still in the VPC Dashboard, click Create Route Table and in the dialog box, enter:

  • Name tag: Public Route Table
  • VPC: select My Test VPC.

Click Yes, Create.

Select the new route table, and then click the Routes tab in the lower pane. Add a new route by clicking Edit. For the new route, enter the following values:

  • Destination: 0.0.0.0/0
  • Target: Move the cursor in the textbox and select Public Subnet Gateway.

Click Save.

The default route in IPv4 is designated as the zero-address 0.0.0.0/0 in CIDR notation. The subnet mask is given as /0, which effectively specifies all networks. A route lookup that does not match any other route will use this route.

Still in the lower pane click the Subnet Associations tab and click Edit. Both the public and private subnet will be displayed. Select the check box next to the Public Subnet to associate it with your route table and click Save.

 

 

 

 

 

Now we need to create the private route table which will allow the private subnet to connect to the NAT server.

Click Create Route Table again. In the Create Route Table dialog box, enter:

  • Name tag: Private Route Table
  • VPC: select My Test VPC.

Click Yes, Create.

Next we’ll connect the private route table with the private subnet. We’ll connect the private subnet to the NAT server instance after the NAT instance is created.

On the Route Tables page, click the Subnet Associations tab in the bottom pane, click Edit and this time select the Private Subnet.

Follow the steps in above to associate the subnet named Private Subnet with the route table named Private Route Table.

Click Save.

Our next task is to create the NAT Server Instance Security Group. This will restrict the NAT to receiving traffic only from the private subnet.

On the Services menu, select EC2. Select Network & Security and click Security Groups. Click Create Security Group. In the Create Security Group dialog box, configure the security group as follows:

  • Security group name: NAT SG
  • Description: Security for the NAT server instance
  • VPC: Select My Test VPC.
  • Under Security Group Rules on the Inbound tab, click Add Rule.

For your new rule, use the following values:

  • Type: Select All Traffic. (If All Traffic is not available, select All TCP.)
  • Source: Select Custom
  • In the text box next to Custom IP, type: 10.10.1.0/24 (This is the CIDR notation for the Private Subnet.)

Click Create.

 

 

 

 

 

Now we can create NAT EC2 instance

Still in the EC2 Dashboard, click Instances, click Launch Instance.

To launch an instance, you must select an AMI (Amazon Machine Image), which is a preconfigured image. You may have your preference but in this example we’ll use the Amazon Linux AMI.

From the Quick Start menu, select the Amazon Linux AMI,and click Select.

Choose an Instance Type page. We’ll use t2.micro as this is on the free tier. You can select other types depending on your memory CPU requirements.

So for now accept the default (t2.micro), click Next: Configure Instance Details.

On the Configure Instance Details page, set the following parameters and accept the other default values:

  • Network: My Test VPC
  • Subnet: Public Subnet
  • Auto-Assign Public IP: Because you enabled Auto- Assign IP Addresses for the public subnet, this should already default to the correct setting, Enable.

Expand the Advanced Details section. Copy the contents of the user data script below  and it into the User data box:

#!/bin/sh
echo 1 > /proc/sys/net/ipv4/ip_forward
echo 0 > /proc/sys/net/ipv4/conf/eth0/send_redirects
/sbin/iptables -t nat -A POSTROUTING -o eth0 -s 0.0.0.0/0 -j MASQUERADE
/sbin/iptables-save > /etc/sysconfig/iptables
mkdir -p /etc/sysctl.d/
cat <<EOF > /etc/sysctl.d/nat.conf
net.ipv4.ip_forward = 1
net.ipv4.conf.eth0.send_redirects = 0
EOF

This Linux shell script configures your server as a NAT server by enabling IP forwarding on the machine and by enabling IP masquerading so that the NAT server can make external requests on behalf of internal servers. It is executed when the instance launches.

Stepping through this script, first IP forwarding is enabled, echo 1 > /proc/sys/net/ipv4/ip_forward (it is disabled by default)

Next “send_redirects’ is disabled , echo 0 > /proc/sys/net/ipv4/conf/eth0/send_redirects.’ This stops a host should sending ICMP Redirect messages. It is used by routers to notify the host about a better routing path that is available for a particular destination. Following this, the host updates the route cache entry and forwards the subsequent packets directly over the optimal path/route suggested via ICMP redirect message. However, this mechanism of routing information updation is risky and is a concern for security community as ICMP redirects can be tampered/faked by malicious software/attacker for redirection to their desired path.

/sbin/iptables -t nat -A POSTROUTING -o eth0 -s 0.0.0.0/0 -j MASQUERADE creates a rule that uses the NAT packet matching table (-t nat) and specifies the built-in POSTROUTING chain for NAT (-A POSTROUTING) on the instances external NIC (-o eth0) from the source default route (-s 0.0.0.0/0) . POSTROUTING allows packets to be altered as they are leaving the instance’s external device. The -j MASQUERADE target is specified to mask the private IP address of a node with the external IP address of the gateway. Then the rule is saved, /sbin/iptables-save > /etc/sysconfig/iptables.

The next lines create a file /etc/sysctl.d/nat.conf containing the ip_forward and send_redirect kernel settings so that they’re set whenever the instance is rebooted.

Getting back to our instance launch, accept the Add Storage defaults and skip forward to the tagging page and enter NAT Server in the Value box.

Click Next: Configure Security Group, click the Select an existing security group option, and select the group NAT Security Group, i.e. the security group that was just set up..

Click Review and Launch.

After reviewing your work, click Launch.

In the key pair dialog box, select Create Key Pair, call it Nat Key Pair and Download the Key Pair

Select the acknowledgement check box, and then click

Launch instances.

NOTE: It has been assigned a public IP address.

Now we need to disable source/destination checking on the NAT server. This is necessary to prevent AWS from rejecting IP packets that are not directly addressed to the NAT server instance’s IP address.

On the EC2 dashboard, click Instances then right-click the instance labeled NAT Server. Under the Networking option click Change Source/Dest. Check and click Yes, Disable.

Now we can finish linking the private subnet to the NAT server instance by creating a private route in the private route table that routes traffic from the private subnet to the NAT server instance.

On the Services menu, select VPC and click Route Tables and select the one labeled Private Route Table.

On the Routes tab in the lower pane, click Edit to add a new route. Fill out the fields as follows:

  • Destination: 0.0.0.0/0
  • Target: Click in the text box, and then click NAT Server.

Click Save.

So the default route in the private subnet is to the NAT server which, being in the public subnet, picks up the default route there which is to the Internet Gateway.

Our next task is to deploy our Bastion Server jump host. However, first we need to create a security group for the instance.

On the Services menu, click EC2, select Network & Security, click Security Groups and click Create Security Group.

In the Create Security Group dialog box, configure the security group as follows:

  • Security group name: Bastion Security Group
  • Description: Security for the bastion server instance
  • VPC: Select My Test VPC.

Under Security Group Rules on the Inbound tab, click Add Rule. Use the following values for your rule:

  • Type: Select SSH.
  • Source: Custom and the IP address or range you will access it from. To keep this secure, keep the range as small as possble.

Click Create.

Now launch Bastion host instance

In the navigation pane, click Instances and then click Launch Instance. From the Quick Start menu, in the row for the first Amazon Linux AMI, click Select.

As before, accept the default (t2.micro), click Next: Configure Instance Details.

On the Configure Instance Details page, set the following parameters and accept the other default values:

  • Network: Select My Test VPC.
  • Subnet: Select Public Subnet.
  • Auto-Assign Public IP: default (Enable).

Expand the Advanced Details section and copy the script below into the User data box:

#!/bin/sh
yum update -y

This Linux shell script will ensure that the server contains all of the latest security updates.

Accept the storage defaults and on the tagging page and In the Value box, type Bastion Server

Click Next: Configure Security Group.

Here, click the Select an existing security group option and then select Bastion Security Group just created

Click Review and Launch.

After reviewing your work, click Launch.

In the key pair dialog box, select the key pair NAT Key Pair . Select the acknowledgement check box, and then click

Launch instances.

We should now have 2 instances running:

Ensure you can SSH to the Bastion host. You will need to use the key specified earlier.

Finally we can deploy an instance into the private subnet. For security purposes, we need to ensure that:

  • This instance uses a different key pair than the key pair used to access the bastion host instance
  • The security group of the private subnet instance accepts only SSH connections from the bastion host.
  • To log in (SSH) to the instance you will need to connect to the bastion host first and then jump to the private instance form there

In the navigation pane of the EC2 Management Console, click Key Pairs and click Create Key Pair. In the Create Key Pair dialog box, in the Key pair name box, type BastionKeyPair.

Click Create.

Save the file BastionKeyPair to your hard drive.

Note If you are using PuTTY as your SSH client, you will need to convert this file to PPK format in order to use it to log in to your private instance later

  1. Start puttygen and select “Load”
  2. Select your .PEM file.
  3. Putty will convert the .PEM format to .PPK format.
  4. Select “Save Private Key”

In the navigation pane of the EC2 Management Console, click Security Groups. In the list of existing security groups, find the group named Bastion Security Group. Make a note of  its Group ID value (or save it to your Clipboard).

Click Create Security Group, then on the Create Security Group page, enter:

  • Security group name: Internal Instance
  • Security Description: Security group for private instances
  • VPC: Select My Test VPC.

To add a rule to your security group, click Add Rule.

For your new security rule, specify the following values:

  • Type: Select SSH.
  • Source: Select Custom.
  • In the text box after the Source drop-down list, put the Group ID value of Bastion Security Group

This will ensure that only instances that are part of the bastion security group can connect to instances that are part of the internal instance security group.

Launch an EC2 instance , assign a security group, click the Select an existing security group option and then select the group Internal Instance Security. In the key pair dialog box, select the key pair BastionKeyPair that you previously

In the EC2 Management Console, in the navigation pane, click Instances. Select the instance Internal Server and make a note of the private IP address.

Return to the connection to your bastion host previously opened Use the SSH command to connect to your private instance:

ssh ec2-user@<private-ip-address> Note: You will see a message like this when you run the SSH command, you will see a message aling the lines of:

The authenticity of host [host] can’t be established.

RSA key fingerprint is 97:8c:1b:…

Are you sure you want to continue connecting (yes/no)?

This warning tells you that this is the first time you are connecting to this host. Type yes, and then press ENTER to continue.

Rub yum-update to confirm you can connect to the Internet

This example is non-resilient. In practice, you may decide to use an Elastic IP and spread out your subnets across several Availability Zones to ensure high availability in case of an outage or disaster. You may also want to harden your Bastion and NAT instances to ensure they are as secure as possible.

That’s it, a secure way to connect to your EC2 instances from the Internet. Don’t forget you can follow us on Twitter at @itsjustsomestuf or sign up to our Newsletter (see below). Bye!


Leave a Reply