Welcome to Day 3 of 21 Days of AWS using Terraform, Let continue our journey, yesterday I discussed how to build AWS VPC using terraform.
In order to deploy EC2 instance we need a bunch of resources
- AMI
- Key Pair
- EBS Volumes Creation
- User data
The first step in deploying EC2 instance is choosing correct AMI and in terraform, there are various ways to do that
- We can hardcore the value of AMI
- We can use data resource(similar to what we used for Availability Zone in VPC section) to query and filter AWS and get the latest AMI based on the region, as the AMI id is different in a different region.
data "aws_ami" "centos" {
owners = ["679593333241"]
most_recent = true
filter {
name = "name"
values = ["CentOS Linux 7 x86_64 HVM EBS *"]
}
filter {
name = "architecture"
values = ["x86_64"]
}
filter {
name = "root-device-type"
values = ["ebs"]
}
}NOTE: Use of data resource is not ideal and each and every used case, eg: In the case of Production we might want to use a specific version of CentOS.
- The above code will help us to get the latest Centos AMI, the code is self-explanatory but one important parameter we used is owners
owners– Limit search to specific AMI owners. Valid items are the numeric account ID,amazon, orself.most_recent– If more than one result is returned, use the most recent AMI.This is to get the latest Centos AMI as per our use case.
https://www.terraform.io/docs/providers/aws/d/ami.html
- Other ways to find the AMI ID
Go to https://us-west-2.console.aws.amazon.com/ec2 --> Instances --> Launch Instances --> Search for centos
- Same thing you can do using AWS CLI
$ aws --region us-west-2 ec2 describe-images --owners aws-marketplace --filters Name=product-code,Values=aw0evgkw8e5c1q413zgy5pjce
{
"Images": [
{
"Architecture": "x86_64",
"CreationDate": "2019-01-30T23:43:37.000Z",
"ImageId": "ami-01ed306a12b7d1c96",
"ImageLocation": "aws-marketplace/CentOS Linux 7 x86_64 HVM EBS ENA 1901_01-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-05713873c6794f575.4",
"ImageType": "machine",
"Public": true,
"OwnerId": "679593333241",
"ProductCodes": [
{
"ProductCodeId": "aw0evgkw8e5c1q413zgy5pjce",
"ProductCodeType": "marketplace"
}
],
"State": "available",
"BlockDeviceMappings": [
{
"DeviceName": "/dev/sda1",
"Ebs": {
"Encrypted": false,
"DeleteOnTermination": false,
"SnapshotId": "snap-040d21883a90fad29",
"VolumeSize": 8,
"VolumeType": "gp2"
}
}
],
"Description": "CentOS Linux 7 x86_64 HVM EBS ENA 1901_01",
"EnaSupport": true,
"Hypervisor": "xen",
"ImageOwnerAlias": "aws-marketplace",
"Name": "CentOS Linux 7 x86_64 HVM EBS ENA 1901_01-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-05713873c6794f575.4",
"RootDeviceName": "/dev/sda1",
"RootDeviceType": "ebs",
"SriovNetSupport": "simple",
"VirtualizationType": "hvm"
},
{
"Architecture": "x86_64",
"CreationDate": "2018-04-04T00:11:39.000Z",
"ImageId": "ami-0ebdd976",
"ImageLocation": "aws-marketplace/CentOS Linux 7 x86_64 HVM EBS ENA 1803_01-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-8274d6ff.4",
"ImageType": "machine",
"Public": true,
"OwnerId": "679593333241",
"ProductCodes": [
{
"ProductCodeId": "aw0evgkw8e5c1q413zgy5pjce",
"ProductCodeType": "marketplace"
}
],
"State": "available",
"BlockDeviceMappings": [
{
"DeviceName": "/dev/sda1",
"Ebs": {
"Encrypted": false,
"DeleteOnTermination": false,
"SnapshotId": "snap-0b665edcc96bbb410",
"VolumeSize": 8,
"VolumeType": "gp2"
}
}
],
"Description": "CentOS Linux 7 x86_64 HVM EBS ENA 1803_01",
"EnaSupport": true,
"Hypervisor": "xen",
"ImageOwnerAlias": "aws-marketplace",
"Name": "CentOS Linux 7 x86_64 HVM EBS ENA 1803_01-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-8274d6ff.4",
"RootDeviceName": "/dev/sda1",
"RootDeviceType": "ebs",
"SriovNetSupport": "simple",
"VirtualizationType": "hvm"
},
{
"Architecture": "x86_64",
"CreationDate": "2018-06-13T15:58:14.000Z",
"ImageId": "ami-3ecc8f46",
"ImageLocation": "aws-marketplace/CentOS Linux 7 x86_64 HVM EBS ENA 1805_01-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-77ec9308.4",
"ImageType": "machine",
"Public": true,
"OwnerId": "679593333241",
"ProductCodes": [
{
"ProductCodeId": "aw0evgkw8e5c1q413zgy5pjce",
"ProductCodeType": "marketplace"
}
],
"State": "available",
"BlockDeviceMappings": [
{
"DeviceName": "/dev/sda1",
"Ebs": {
"Encrypted": false,
"DeleteOnTermination": false,
"SnapshotId": "snap-0313e2ec7fa27f2e9",
"VolumeSize": 8,
"VolumeType": "gp2"
}
}
],
"Description": "CentOS Linux 7 x86_64 HVM EBS ENA 1805_01",
"EnaSupport": true,
"Hypervisor": "xen",
"ImageOwnerAlias": "aws-marketplace",
"Name": "CentOS Linux 7 x86_64 HVM EBS ENA 1805_01-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-77ec9308.4",
"RootDeviceName": "/dev/sda1",
"RootDeviceType": "ebs",
"SriovNetSupport": "simple",
"VirtualizationType": "hvm"
},
{
"Architecture": "x86_64",
"CreationDate": "2018-05-17T09:30:44.000Z",
"ImageId": "ami-5490ed2c",
"ImageLocation": "aws-marketplace/CentOS Linux 7 x86_64 HVM EBS ENA 1804_2-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-55a2322a.4",
"ImageType": "machine",
"Public": true,
"OwnerId": "679593333241",
"ProductCodes": [
{
"ProductCodeId": "aw0evgkw8e5c1q413zgy5pjce",
"ProductCodeType": "marketplace"
}
],
"State": "available",
"BlockDeviceMappings": [
{
"DeviceName": "/dev/sda1",
"Ebs": {
"Encrypted": false,
"DeleteOnTermination": false,
"SnapshotId": "snap-012ad984270e9fede",
"VolumeSize": 8,
"VolumeType": "gp2"
}
}
],
"Description": "CentOS Linux 7 x86_64 HVM EBS ENA 1804_2",
"EnaSupport": true,
"Hypervisor": "xen",
"ImageOwnerAlias": "aws-marketplace",
"Name": "CentOS Linux 7 x86_64 HVM EBS ENA 1804_2-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-55a2322a.4",
"RootDeviceName": "/dev/sda1",
"RootDeviceType": "ebs",
"SriovNetSupport": "simple",
"VirtualizationType": "hvm"
},
{
"Architecture": "x86_64",
"CreationDate": "2017-12-05T14:49:18.000Z",
"ImageId": "ami-b63ae0ce",
"ImageLocation": "aws-marketplace/CentOS Linux 7 x86_64 HVM EBS 1708_11.01-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-95096eef.4",
"ImageType": "machine",
"Public": true,
"OwnerId": "679593333241",
"ProductCodes": [
{
"ProductCodeId": "aw0evgkw8e5c1q413zgy5pjce",
"ProductCodeType": "marketplace"
}
],
"State": "available",
"BlockDeviceMappings": [
{
"DeviceName": "/dev/sda1",
"Ebs": {
"Encrypted": false,
"DeleteOnTermination": false,
"SnapshotId": "snap-045714dfd4f364480",
"VolumeSize": 8,
"VolumeType": "standard"
}
}
],
"Description": "CentOS Linux 7 x86_64 HVM EBS 1708_11.01",
"EnaSupport": true,
"Hypervisor": "xen",
"ImageOwnerAlias": "aws-marketplace",
"Name": "CentOS Linux 7 x86_64 HVM EBS 1708_11.01-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-95096eef.4",
"RootDeviceName": "/dev/sda1",
"RootDeviceType": "ebs",
"SriovNetSupport": "simple",
"VirtualizationType": "hvm"
}
]
}
- OR if you want output in tabular format
$ aws ec2 describe-images \
> --owners aws-marketplace \
> --filters Name=product-code,Values=aw0evgkw8e5c1q413zgy5pjce \
> --query 'Images[*].[CreationDate,Name,ImageId]' \
> --filters "Name=name,Values=CentOS Linux 7*" \
> --region us-west-2 \
> --output table \
> | sort -r
| 2019-01-30T23:43:37.000Z| CentOS Linux 7 x86_64 HVM EBS ENA 1901_01-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-05713873c6794f575.4 | ami-01ed306a12b7d1c96 |
| 2018-06-13T15:58:14.000Z| CentOS Linux 7 x86_64 HVM EBS ENA 1805_01-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-77ec9308.4 | ami-3ecc8f46 |
| 2018-05-17T09:30:44.000Z| CentOS Linux 7 x86_64 HVM EBS ENA 1804_2-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-55a2322a.4 | ami-5490ed2c |
| 2018-04-04T00:11:39.000Z| CentOS Linux 7 x86_64 HVM EBS ENA 1803_01-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-8274d6ff.4 | ami-0ebdd976 |
| 2017-12-05T14:49:18.000Z| CentOS Linux 7 x86_64 HVM EBS 1708_11.01-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-95096eef.4 | ami-b63ae0ce |
| DescribeImages |
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
+--------------------------+----------------------------------------------------------------------------------------------------------+-------------------------+
+--------------------------+----------------------------------------------------------------------------------------------------------+-------------------------+- If you look at the Description field
"Description": "CentOS Linux 7 x86_64 HVM EBS ENA 1901_01",
and then check the Terraform code in of the filter we use “CentOS Linux 7 x86_64 HVM EBS *” and that is one of the reasons of using that
filter {
name = "name"
values = ["CentOS Linux 7 x86_64 HVM EBS *"]
}
https://wiki.centos.org/Cloud/AWS
- As we are able to figure out the AMI part, the next step is to create and use the key pair
- Either we can hardcode the value of key pair or generate a new key via command line and then refer to this file.
$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/prashant/.ssh/id_rsa): /tmp/id_rsa
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /tmp/id_rsa.
Your public key has been saved in /tmp/id_rsa.pub.
The key fingerprint is:
SHA256:A7lYe+KDAVoUFEDsC14wnb+NePzCntjZruJdJf0+apM prashant@prashant-ThinkPad-T440s
The key's randomart image is:
+---[RSA 2048]----+
|++*o. |
| = o . |
|. = . + |
|.+ o + +. |
|+ o = B.So |
| o . O +o.. |
| + +. .. |
| .+o=o E.. |
| .ooB+o..o.. |
+----[SHA256]-----+
resource "aws_key_pair" "mytest-key" {
key_name = "my-test-terraform-key"
public_key = "${file(var.my_public_key)}"
}
* In var.my_public_key set the location as /tmp/id_rsa.pub * To refer to this file, we need to use the file function
https://www.terraform.io/docs/configuration/functions/file.html
- There is one more resource, I want to use here called template_file. The
template_filedata source renders a template from a template string, which is usually loaded from an external file. This you can use with user_data resource to execute any script during instance boot time
data "template_file" "init" {
template = "${file("${path.module}/userdata.tpl")}"
}- userdata.tpl file will be like
#!/bin/bash
yum -y install httpd
echo "hello from terraform" >> /var/www/html/index.html
service httpd start
chkconfig httpd on- After AMI and Keys out of our way, let start building EC2 instance
resource "aws_instance" "my-test-instance" {
count = 2
ami = "${data.aws_ami.centos.id}"
instance_type = "${var.instance_type}"
key_name = "${aws_key_pair.mytest-key.id}"
vpc_security_group_ids = ["${var.security_group}"]
subnet_id = "${element(var.subnets, count.index )}"
user_data = "${data.template_file.init.rendered}"
tags = {
Name = "my-instance-${count.index + 1}"
}
}Most of these parameters I already discussed in the first section, but let's quickly review it and check the new one * count: The number of instance, we want to create * ami: This time we are pulling ami using data resource * instance_type: Important parameter in AWS Realm, the type of instance we want to create * key_name: Resource we create earlier and we are just referencing it here * tags: Tags are always helpful to assign label to your resources.
Below ones are special(vpc_security_group_ids & subnet_id), because both of these resource we created during the vpc section, so now what we need to do is to output it during VPC module and use there output as the input to this module.
- If you notice the above code, one thing which is interesting here is vpc_security_group_ids and subnet_id
- The interesting part, we already created these as a part of VPC code, so we just need to call in our EC2 terraform and the way to do it using outputs.tf.
output "public_subnets" {
value = "${aws_subnet.public_subnet.*.id}"
}
output "security_group" {
value = "${aws_security_group.test_sg.id}"
}- After calling these values here, we just need to define as the part of main module and the syntax of doing that is
security_group = "${module.vpc.security_group}"
subnets = "${module.vpc.public_subnets}"
- Final module code for EC2 instance look like this
module "ec2" {
source = "./ec2"
my_public_key = "/tmp/id_rsa.pub"
instance_type = "t2.micro"
security_group = "${module.vpc.security_group}"
subnets = "${module.vpc.public_subnets}"
}
- Let’s create two EBS volumes and attach it to two EC2 instances we created earlier
resource "aws_ebs_volume" "my-test-ebs" {
count = 2
availability_zone = "${data.aws_availability_zones.available.names[count.index]}"
size = 1
type = "gp2"
}
resource "aws_volume_attachment" "my-vol-attach" {
count = 2
device_name = "/dev/xvdh"
instance_id = "${aws_instance.my-test-instance.*.id[count.index]}"
volume_id = "${aws_ebs_volume.my-test-ebs.*.id[count.index]}"
}* To create EBS Volumes, I am using ebs_volume resource and to attach it use aws_volume_attachment * We are creating two Volumes here * As Volume is specific to Availibility Zone, I am using aws_availibility_zone data resource * Size of the Volume is 1GB * Type is gp2(other available options "standard", "gp2", "io1", "sc1" or "st1" (Default: "standard"))
https://www.terraform.io/docs/providers/aws/r/ebs_volume.html
https://www.terraform.io/docs/providers/aws/r/volume_attachment.html
- As this time we are creating ebs volume, let’s modify our userdata.tpl script and format the partition
#!/bin/bash
mkfs.ext4 /dev/xvdh
mount /dev/xvdh /mnt
echo /dev/xvdh /mnt defaults,nofail 0 2 >> /etc/fstabGithub Link
https://github.com/100daysofdevops/21_days_of_aws_using_terraform
- Looking forward for you guys to join this journey
- Website: https://100daysofdevops.com/
- Twitter: @100daysofdevops OR @lakhera2015
- Facebook: https://www.facebook.com/groups/795382630808645/
- Medium: https://medium.com/@devopslearning
- GitHub: https://github.com/100daysofdevops/100daysofdevops
- Slack: https://join.slack.com/t/100daysofdevops/shared_invite/enQtNzg1MjUzMzQzMzgxLWM4Yjk0ZWJiMjY4ZWE3ODBjZjgyYTllZmUxNzFkNTgxZjQ4NDlmZjkzODAwNDczOTYwOTM2MzlhZDNkM2FkMDA
- YouTube Channel: https://www.youtube.com/user/laprashant/videos?view_as=subscriber
In addition to that, I am going to host 5 meetups whose aim is to build the below architecture.
- Meetup: https://www.meetup.com/100daysofdevops
- Day1(Nov 10): Introduction to Terraform
- Day 2(Nov 16): Building VPC using Terraform
- Day 3(Nov 17): Creating EC2 Instance inside this VPC using Terraform
- Day 4(Nov 23): Adding Application Load Balancer and Auto-Scaling to the EC2 instance created on Day 3
- Day5(Nov 24): Add Backend MySQL Database and CloudWatch Alarm using Terraform
I am a little lost on Module and outputfile
could please elaborate at what point should we use module or what is module
Thanks, Mark for reading my blog, I would say always use the module, as that will increase your code re-usability. Please feel free to join the below meetup, I am going to talk more about these topics.
21 Days of AWS using Terraform(Online) -Building AWS VPC using Terraform
https://www.meetup.com/100daysofdevops/events/266400556/
21 Days of AWS using Terraform(Online)-Creating AWS EC2 Instance using Terraform
https://www.meetup.com/100daysofdevops/events/266400580/