EC2 Deployment Part 1:
I am interested in hosting my personal site on AWS (Amazon Web Services). Previously I have set it up on an elastic beanstalk instance through AWS and that worked great. I am interested now in trying to set up the server and software myself. To do this I want to set up a EC2 instance and install the server (nginx), web server gateway interface (uWSGI) and Django (v. 2.0) myself. I have never done this before and am trying to transition into web development, this seems like a good way to learn a lot of the background quickly. I am following a number of other reference and tutorials to do this and will try to cite them as they come up. The primary resource is another tutorial at oliverelliot.com. My tutorial is very similar to his but is updated with all of the little changes that occur over a few years and a few other things will vary.
Set up EC2 instance on AWS
The first step is to setup an AWS account if you don't already have one AWS. Once you have setup an account you can access the online gui AWS console aws.amazon.com/console/. This allows you to interact with your AWS account and setup new cloud computing resources. An example of what my console landing page looks like is presented below.
AWS provides a wide variety of cloud computing options. Here we are only interested in creating an EC2 instance. In my case there is a link to "Launch a virutal machine with EC2". Following this link leads to the first step (of 7 in total) in setting up the instance, selecting an AMI (Amazon Machine Image). As described on the page an AMI is a software template that AWS will use to set up your virtual machine.
There is a long list of potential operating systems to select from. Note that in this tutorial we will only be using basic and popular packages (e.g. nginx and vanilla Django) if you need a more esoteric package that may not be as easy to install on the Amazon Linux you should go with an Ubuntu or RHEL OS AMI instead. Beyond the operating system different AMIs also come with different software installed. Since we are interested in doing it ourselves we will use the first/top option, Amazon Linux 2 AMI.
Once you select your AMI you will be moved to step 2 on the next page "Choose Instance Type". I went with the default t2.micro. There is really not much going on with the website we will be using so why not save some money here?
On step 3, "Configure Instance" I left all defaults in place.
Step 4 asks you to add a storage device to your EC2 instance. The EC2 instance type that we just selected in step two is a functioning virtual machine and therefore has its own storage. For the purposes of this example that would probably be sufficient. The entire idea behind the EC2 instances though is that they can be transient, they can be created and destroyed. If your website has a database then it is a good idea to create add a separate storage volume that will persist even if the EC2 instance crashes and is restarted.
Just for the fun of it I am adding a Elastic Block Storage (EBS) volume for this instance. There is a really good description of the different storage options, when they should be used and why it is important to separate storage from your server here CloudBerryLab.
The particular AMI we selected is actually 'backed' by an EBS. That means that the AMI OS and software is installed with that more stable storage so that even if the EC2 instance crashes or is stopped the data will be retained.
I skipped step 5 "Add Tags".
In step 6 you first set up your security group. Among other things this details which connections are allowed on your instance (like a firewall). The default connection allowed is ssh on port 22. We will use this later to access our EC2 instance and set up our website. By default connections are allowed via ssh from any IP address. This is an unnecessary security issue so use the handy source drop-down menu to only permit ssh connections from "My IP". It will find the IP that you are currently using and add that to the allowed list. Alternatively if you plan to use other computers you can add custom IP addresses to the list too.
Since we know that we will be using this instance as a web server we should allow HTTP connections on the standard port 80. Here we will allow connections from any IP address.
Finally step 7 asks you to review your settings and if you are happy with everything you can finally launch the instance. When you do launch you will be asked to provide or create a new cryptographic key pair. This key pair will allow you to securely connect via ssh to your instance from your local computer terminal. Select "Create a new key pair" from the drop down, provide a name and then click download key pair.
Upon its creation you will download the private key and AWS will have no further knowledge of it. Like it says, make sure you store that private key in a secure location and don't lose it; otherwise you will be unable to access your instance. Since I use Ubuntu Linux locally and already have ssh installed I have a hidden directory
~/.ssh where I store all of my key pairs. If you don't have ssh installed on your local machine you will need to for the next steps. Depending on your OS you may just need to search a little for a good tutorial.
Once you have downloaded and stored the key file (*.pem file) somewhere you can again click "Launch" and the instance will actually launch this time. Click on the "Services" drop down on the top navbar once you reach the successful instance launch page and go to "EC2" and select "INSTANCES/instances" from the side bar.
You should then be able to see the instance page like mine.
The IP address and public domain name that AWS provides for the instance are listed here (highlighted in red). For fun you can copy one of those into your browser and see what happens when you send a request to the instance (on port 80) from your browser. I get a "site can't be reached" error, which is to be expected since we have yet to set up a server to handle any HTTP requests.
Setting up an nginx server on our EC2 instance
Lets take the next step in launching our website and set up the server software we want to use on our EC2 instance. The first step is to ssh into our instance. I am using Ubuntu locally so ssh is already set up, for people using other operating systems, particularly Windows some additional steps may be required. To connect via ssh using our newly created key pair we first need to change the permissions of the key pair .pem file to read only by the owner.
$ chmod 400 <path to key pair>
<path to key pair> is the path to the .pem file you downloaded earlier (e.g. in my case it is
~/.ssh/tutorial_key_pair.pem). Now we can use ssh and our key to access the instance
$ ssh -i <path to key pair> ec2-user@<public domain name>
-i argument indicates which key to use and the
<public domain name> is the same that we just used to test the website, i.e. the "Public DNS (IPv4)" or "IPv4 Public IP" listing in the instance page. It will probably ask you when you first run this command if you trust the site you are connecting to and since we do tell it yes and it will add the site to the list of trusted hosts for future connections. If this didn't work checkout the AWS tutorial.
One problem with this that really hung me up for a while was forgetting to add the user name to the address I was trying to access. It will not work unless you prepend your EC2 user-name and "@", i.e.
ec2-user@, to the public domain name. It makes sense, since you are logging in as a remote user you must specify which user you are. When I neglected to add the user-name I got the following error on my local terminal.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
Once logged in to the instance we can install nginx. When you first connect there is some ascii art and a suggestion to update the yum package manager, do that.
In my initial attempt to install nginx, following oliverelliot.com, I used
$ sudo yum install nginx
When you run that you are informed that it is unavailable and you should instead use the Amazon Linux extra packages. Run the command they provide
$ sudo amazon-linux-extras install nginx1.12
and nginx will be installed. Before we start the nginx service lets take a look at what processes are currently watching port 80 for incoming traffic
$ sudo netstat -tlpn
This should show that although there are several processes watching several ports (e.g. sshd is on port 22) there is no server program listening to port 80 yet. We can start nginx by first adding it to the list of services and then starting the service with the
service command (taken from this github gist)
$ sudo chkconfig nginx on $ sudo systemctl start nginx.service
Now if we use
netstat -tlpn again to check for processes listening to port 80 we can see that nginx is listening for both IPv4 and IPv6 incoming IP addresses. If we again use our browser to access the public IP address (or domain name) indicated in the instance we should now be able to see the nginx splash page.
In the next installment of this tutorial we will set up Python3 within a virtualenv, uWSGI and Django on our EC2 instance and perform some testing to make sure that they all work together.