There are tons of instructions out there on how to create an EBS backed images. Most of them don’t work for me because they are either out dated or the person’s instructions for copying the file structure doesn’t work. I finally found instructions though that do work.
So here is how I did it:
1) Start Up an Instance
The first thing I do is start up an instance. Everyone has instructions on how to do this from the command line. But I’m lazy and find that Amazon’s API tools change regularly which is annoying and makes it difficult to follow the instructions other people have written. I recommend doing it in the AWS Console.
Go to EC2 in the AWS Console. Choose AMI and find an image that you like (I tend to lean toward Alestic’s Ubuntu images, but that’s just me). Once I found an AMI I like I highlight it and then choose “Launch Instance”. I follow the instructions, making sure that I choose the correct security group (aka firewall) and the correct key pair (you access ec2 servers using a key pair rather than a password).
Once the server is up and running, I update everything, configure it, transfer data onto it, blah blah blah. The usual things that you do when you are prepping a server. Once I’m all set I’m ready to make it an EBS backed AMI.
2) Create a New EBS Volume
The next thing I do is create an EBS Volume, again doing it in the AWS Console. Go to the EC2 section of the AWS Console. Choose “Volumes” and then “Create Volume”. You’ll want to make sure of two things:
First that the volume is large enough to hold everything. I forgot to do that the first time around and *thought* I had to do everything all over again (see next post for how to really do it). The easiest thing would be to just size it right the first time around.
Second that the volume is in the same availability zone as the instance you started up in step 1. If you aren’t sure what the availability zone is for your instance, go back to the instance, highlight it, and in the bottom panel you’ll see what zone its in.
3) Attach Volume to Instance
When you are done you’ll want to attach the volume to the instance that you created in step 1. Once the volume is available, highlight it and choose “Attach Volume”. You’ll be stepped through attaching it to your server. If you have only one instance running its pretty straightforward. If you have multiple ones running, you’ll want to make sure that you’re attaching it to the right place. You’ll be asked to indicate where you want the attached, I usually do /dev/sdf. Oh and this process won’t work if the volume and the instance aren’t in the same availability zone, hence my warning in the previous step.
4) Copy the Data from the Server to the Volume
This part of the instructions I found over at Winners Don’t Lose. This helps you format the volume and copy the data on the current instance over to the volume:
In a root shell on the instance: # mkfs.ext3 /dev/sdf # mkdir /mnt/target && mount /dev/sdf /mnt/target # rsync -avHx / /mnt/target # rsync -avHx /dev /mnt/target # sync;sync;sync;sync && umount /mnt/target The above commands did the following: formatted the entire volume /dev/sdf as an extended 3 filesystem created directory /mnt/target and mounted /dev/sdf at /mnt/target rsync’d the root instance-store filesystem to the ebs volume synchronized the /dev directory from the instance-store filesystem flush all pending write ops, and unmount the EBS volume
To be honest these are the only instructions I’ve ever been able to follow that doesn’t produce some error that I have to debug.
5) Detach the Volume
The next step is to detach the volume that you just created from the running instance. Again you can do this in the AWS console by finding the volume, highlighting it, and choosing detach. Make sure the you’re detaching the right volume though!
6) Create a Snapshot of the Volume
You’ll want to create a snapshot of the volume that you just created. You can again do that in the AWS Management console but finding the volume, right clicking on it, and choosing create snapshot.
7) Register your AMI
This is the part that took me forever to figure out. Mostly because the documentation for the ec2-register command didn’t jive with the tools that i had access too. The command I used was:
ec2-register --block-device-mapping /dev/sda1=<snapshot id>::true --name <ami name> --kernel <kernel id> --ramdisk <ram disk id> -K <your pk file>.pem -C <your cert file>.pem
(note i updated this: if you want the volume to persist after termination then you’ll want to change the true in the –block-device-mapping flag to false)
You’ll want to make sure that you specify and kernel id and a ram disk id, I usually use the same one that my instance I’m creating this new AMI from was using. You don’t have to do it, but that would mean that if your AMI users don’t choose one during start up, then the instance won’t work.
Note that you may have your key and cert file set as part of your environment variable. I have multiple accounts so I don’t, and instead just specify them when I’m running any commands (annoying but I don’t do much from the commandline anyway).
Jorge on said:
Hi there,
I’m not sure if I’m talking about the same thing, but just from the AWS Console, right clicking over the instance and then choosing “Create Image (EBS AMI)” will do the same and after a few minutes I’m getting my AMI registered.
Please let me know if I’m wrong and you are talking about another thing. I’ve just started with AWS 2 days ago so I’m not completely sure about it.
Best Regards :)
Rosalyn Metz on said:
Hi Jorge,
You can register your AMI if you’re already using an existing running AMI. In the example I’m explaining, I’m creating an EBS backed AMI from scratch, not from an existing one. So yes your explanation will work if you’re creating an EBS backed AMI from an EBS backed AMI, but if you’re creating an EBS backed AMI from an instance store AMI, you would want to follow directions like this.
Rosalyn
Jorge on said:
Ok, I should imagine that it was more complex than I thought… I guess I need to study more. We, the human race, have managed to create this amazing complex world (lol).
Thanks for your answer, it will help :)
Jorge on said:
Hi Rosalyn
Just to tell you that today I had to use your procedure. Now I understand what you told me the first day ;)
Thanks a lot
Dan on said:
Hey great guide. I was able to create the AMI, however when I launch an instance of it, the console shows that its running but it doesn’t seem to work.
heres the system log:
Xen Minimal OS!
start_info: 0xb10000(VA)
nr_pages: 0x6a400
shared_inf: 0xaf117000(MA)
pt_base: 0xb13000(VA)
nr_pt_frames: 0×9
mfn_list: 0×967000(VA)
mod_start: 0×0(VA)
mod_len: 0
flags: 0×0
cmd_line: root=/dev/sda1 ro 4
stack: 0×946780-0×966780
MM: Init
_text: 0×0(VA)
_etext: 0x621f5(VA)
_erodata: 0×76000(VA)
_edata: 0x7b6d4(VA)
stack start: 0×946780(VA)
_end: 0x966d34(VA)
start_pfn: b1f
max_pfn: 6a400
Mapping memory range 0xc00000 – 0x6a400000
setting 0×0-0×76000 readonly
skipped 0×1000
MM: Initialise page allocator for e6c000(e6c000)-0(6a400000)
MM: done
Demand map pfns at 6a401000-7a401000.
Heap resides at 7a402000-ba402000.
Initialising timer interface
Initialising console … done.
gnttab_table mapped at 0x6a401000.
Initialising scheduler
Thread “Idle”: pointer: 0x7a402008, stack: 0x6a030000
Initialising xenbus
Thread “xenstore”: pointer: 0x7a402478, stack: 0x6a040000
Dummy main: start_info=0×966880
Thread “main”: pointer: 0x7a4028e8, stack: 0x6a050000
“main” “root=/dev/sda1″ “ro” “4″
vbd 2049 is hd0
******************* BLKFRONT for device/vbd/2049 **********
backend at /local/domain/0/backend/vbd/194/2049
Failed to read /local/domain/0/backend/vbd/194/2049/feature-barrier.
Failed to read /local/domain/0/backend/vbd/194/2049/feature-flush-cache.
73400320 sectors of 0 bytes
**************************
vbd 2051 is hd1
******************* BLKFRONT for device/vbd/2051 **********
backend at /local/domain/0/backend/vbd/194/2051
Failed to read /local/domain/0/backend/vbd/194/2051/feature-barrier.
Failed to read /local/domain/0/backend/vbd/194/2051/feature-flush-cache.
1835008 sectors of 0 bytes
**************************
[H
[J
GNU GRUB version 0.97 (1740800K lower / 0K upper memory)
[m
[4;2H+————————————————————————-+
[5;2H|
[5;76H|
[6;2H|
[6;76H|
[7;2H|
[7;76H|
[8;2H|
[8;76H|
[9;2H|
[9;76H|
[10;2H|
[10;76H|
[11;2H|
[11;76H|
[12;2H|
[12;76H|
[13;2H|
[13;76H|
[14;2H|
[14;76H|
[15;2H|
[15;76H|
[16;2H|
[16;76H|
[17;2H+————————————————————————-+
[m
Use the ^ and v keys to select which entry is highlighted.
Press enter to boot the selected OS, ‘e’ to edit the
commands before booting, or ‘c’ for a command-line.
[5;78H
[m
[5;3H Xenpae — SUSE Linux Enterprise Server 10 SP2 – 2.6.16.60-0.29 (xenpa>
[5;75H
[m
[m
[6;3H Xenpae — SUSE Linux Enterprise Server 10 SP2 – 2.6.16.60-0.29
[6;75H
[m
[m
[7;3H SUSE Linux Enterprise Server 10 SP2 – 2.6.16.60-0.31 (default)
[7;75H
[m
[m
[8;3H Failsafe — SUSE Linux Enterprise Server 10 SP2 – 2.6.16.60-0.31 (def>
[8;75H
[m
[m
[7m
[9;3H SUSE Linux Enterprise Server 10 SP2 – 2.6.16.60-0.31 (smp)
[9;75H
[m
[m
[10;3H Failsafe — SUSE Linux Enterprise Server 10 SP2 – 2.6.16.60-0.31 (smp>
[10;75H
[m
[m
[11;3H Floppy
[11;75H
[m
[m
[12;3H
[12;75H
[m
[m
[13;3H
[13;75H
[m
[m
[14;3H
[14;75H
[m
[m
[15;3H
[15;75H
[m
[m
[16;3H
[16;75H
[m
[16;78H
[9;75H
[23;4H The highlighted entry will be booted automatically in 8 seconds.
[9;75H
[23;4H The highlighted entry will be booted automatically in 7 seconds.
[9;75H
[23;4H The highlighted entry will be booted automatically in 6 seconds.
[9;75H
[23;4H The highlighted entry will be booted automatically in 5 seconds.
[9;75H
[23;4H The highlighted entry will be booted automatically in 4 seconds.
[9;75H
[23;4H The highlighted entry will be booted automatically in 3 seconds.
[9;75H
[23;4H The highlighted entry will be booted automatically in 2 seconds.
[9;75H
[23;4H The highlighted entry will be booted automatically in 1 seconds.
[9;75H
[H
[J Booting ‘SUSE Linux Enterprise Server 10 SP2 – 2.6.16.60-0.31 (smp)’
root (hd0,1)
Error 5: Partition table invalid or corrupt
Press any key to continue…
Amjed on said:
Hi Rosalyn,
Am new to AWS, please help me understand.
This is what I udnerstood from your post:
To start an instance from a snapshot(which i created from a running instance’s EBS volume).
I should first register an AMI, for this I should have the
kernel Id
RAM disk Id
pk file
cert file
am I right ?
now a couple of questions:
1. I know the pk file, what is the cert file ? the public key file ?
2. I want to do this, to backup my instance ( system, pkgs and my data). Is this the right way ? I know you can create an AMI and always create an instance out of it. But I didnt feel creating AMI for everytime I back an elegant solution.
thanks,
Rosalyn Metz on said:
hi amjed,
the cert file is part of your security credentials with amazon web services. you can find more information about it on their website: http://bit.ly/pcGd3c
you can get the cert file under my account > access credentials on the aws page.
hope that helps!
rosalyn
deepti on said:
HI,
thanks for the great post. I tried to create a ebs backed ami from instance store ami following your post, everything goes through well and my ebs backed ami is created. but when i launch an instance from the newly created ami, i am unable to ssh and connect to the instance.
i am using xvdf in following commands instead of the sdf
# mkfs.ext3 /dev/sdf
# mkdir /mnt/target && mount /dev/sdf /mnt/target
# rsync -avHx / /mnt/target
# rsync -avHx /dev /mnt/target
# sync;sync;sync;sync && umount /mnt/target
but while registering the ami, i am specifiying sda as the block-device-mapping. would that be the reason for the problem? also i cannot use xvdf while registering the ami, i get following error – Client.InvalidBlockDeviceMapping: Invalid device name /dev/xvda1. please let me know.
Zed on said:
Hey Rosalyn,
Thanks for the great post. I have been trying to clone and EBS backed AMi using the following but have been unsuccessful.
* Choose an EBS backed AMI (I have done Ubuntu ami-fd589594 and Debian ami-1212ef7b) and launch as micro instances
* SSH successfully using my security group and my key
* Go to EBS tab and right click and create a snapshot of the EBS
* Create an Image out of the snapshot
* Launch an image using the same security group and key (as micro instance)
* Try to ssh to it and
Am I missing any steps?
Thanks,
Zed
Alex on said:
When I try to execute this line:
mount /dev/sdf /mnt/target
I get:
mount: you must specify the filesystem type
Evgeni Stavinov on said:
@deepti
I was also getting this error: Client.InvalidBlockDeviceMapping
I’m not sure why, but after I tried different device options, it worked with /dev/sdb
I was using Amazon 64-bit AMI instance