Hackers and Slackers

AWS Adventures: Why Can't I SSH Into My Damn EC2 Instance?

Watch as I struggle with the cloud! My pain, your gain!

Sometimes I find myself responsible for setting up and maintaining my own infrastructure for doing data stuff. In light of this, I've taken it upon myself to learn more about AWS in order for this experience to be less terrifying. I've been following a course on Udemy for the AWS Solutions Architect Certification. So far, so good.

Cut to the EC2 section. Has you spin up a little EC2 instance and SSH into it. I would find myself SSHing in, then after a few minutes my connection would be reset and then any further attempts would be met with a Resource temporarily unavailable.

I googled around. Lots of stuff that didn't work. I logged into a client's console that I had access to, tried to see if there were any settings that were different from mine. What on Earth did I mess with? I first made an AWS account in 2012, and go into phases of messing with my personal account and forgetting it exists. I must have hit some kind of "Make it so every server you launch is unreachable and useless" button at some point, then forgotten where this button was.

It's a tough little jungle. There's numerous different versions of security on AWS. Then, finally, I noticed an angel on StackOverflow posted that you should check to see whether you have a public DNS address. Turns out I didn't! https://stackoverflow.com/questions/20941704/ec2-instance-has-no-public-dns

  1. Go to console.aws.amazon.com
  2. Go To Services -> VPC
  3. Open Your VPCs
  4. select your VPC connected to your EC2 and
  5. select Actions => Edit DNS Hostnames ---> Change DNS hostnames: to YES

Aaand it worked!

How did it get like this? I deleted and re-made the VPC numerous times, so it was off by default for some reason. When did I mess with this? Why did I mess with this? Why did it work at all instead of blocking me from the beginning? Who knows!

P.S. Digging the "code journal" style of tech talk, among other reasons because I know I'm not going to accidentally plagiarize some tutorial I've read.

UPDATE:
Just tried to log back in with my settings from before, and wouldnchaknowit, it suddenly started being wacky again! Exact same problems too - can log in for a bit shortly after the instance starts, then connection reset, then temporarily unavailable.

I decided to look at the System Logs (accessible from Actions -> Instance Settings -> Get System Log). I noticed that it was blank. And when logs showed up, coincidentally my connection was reset again! The last line of the system log was ip-172-30-0-53 login:. A clue, mayhaps! Maybe it has something to do with the little intranet it was logged into!

Frantic googling. Eventually, I discover that when I created my account, there was a system called "EC2 Classic" in place (it presumably was not called this at the time). They eventually added the VPC feature (short and sweet: VPC stands for either "Volatile Psionic Cloak" or "Virtual Private Cloud", and is one of the several terms that mean "a place where your VMs run" (each one has its own subtle idiosyncracies. I will be bitching about them again, I promise you.) Why doesn't this text editor do paren-highlighting? (print-str "I love lisp") Eh, let's say this is enough.)

Sooo, VPCs put a spanner in the works of how EC2 classic worked. If you made your AWS account after VPC was introduced, you had to do stuff with VPC. However, if your account was old enough that you had access to EC2 Classic, they let you use both systems so as not to nuke old infrastructure you might have had. Note that whether you were "VPC Only" or "VPC And EC2 Classic" is something that's flagged at the account level.

I think it MIGHT have been okay, but while trying to log in, I deleted the VPC that was present cuz I read that it'd spawn a new one that was set to default options. Now, this did happen, except that the one I deleted was my "Default VPC". I do not fully understand the signifigance of this, but it seems to be what was getting in my way. So, simple - just make a new Default VPC! However, if you have an "EC2 And VPC" account, you cannot make a new Default VPC (for reasons that are mysterious to me).

Remember when I said "VPC and EC2 Classic" status was at the account level? Unfortunately, that means you need to contact AWS support in order to change it. If you've ever used normal Amazon customer support, you're probably accustomed to it being pretty good - you just open a chat window, say your stuff didn't arrive, then they overnight the missing delivery. AWS support (if you're not paying extra money) is not like that. I put in the request on Wednesday night, and I've had several rounds of confirming "Yes, I want you to do this", each with at least a day-long turn-around. It is now Sunday, and I got a message from yesterday saying that my account is finally in the process of being moved over. Hopefully it all goes off without a hitch, but we'll see!

Author image
Center of the Universe Website
Super villain in somebody's action hero movie. Experienced a radioactive freak accident at a young age, which rendered him part-snake and strangely adept at Python.
Author image
Center of the Universe

Super villain in somebody's action hero movie. Experienced a radioactive freak accident at a young age, which rendered him part-snake and strangely adept at Python.