On Hi-Tech Travel

I am by no means a road warrior, however, I have spent quite a bit of time on the road over the last few years and found a few things to make life a bit easier.

Hardware:

  1. Light laptop & Tablet combo – If you’re at a conference, you will be walking, a lot. The lighter your gear combo, the better. Personally I travel with an iPad mini and 13” retina MacBook
  2. Good bag – For a while this was an old (2009ish) conference bag. Now-a-days, its a much lighter Colombia bag.
  3. Backup Batteries – This one: http://www.newtrent.com/store/external-battery/icarrier-imp120d.html It can charge whilst being charged, and will charge my iDevices about 6 times.
  4. USB Firewall – https://lockedusb.com/ <- These just started shipping. It’s a little USB ‘condom’ or firewall of sorts. It will allow you to charge at an optimal rate while keeping the data lines disconnected. Basically, it’s a $14 device that will protect your $500 phone and it’s data.

Software

  1. Little Snitch – This:  http://www.obdev.at/products/littlesnitch/index.html < It’s a harder-core OSX firewall. Worth every penny and then some.
  2. Sidestep – This: http://chetansurpur.com/projects/sidestep/ < This will redirect all your system traffic over an SSH tunnel. Useful when on an unsecure wireless connection.
  3. Blogsy – This: http://blogsyapp.com/ < This has been the most useful iOS app for live blogging during an event.
  4. Windows Live Writer – For the longer posts
  5. FaceTime / Skype – For seeing the family

Suggestions? Things I missed? Drop a line in the comments.

Basic Hardening Part 2: Using Heat Templates

Remember the basic hardening we did? No? Well, go forth and read here. I’ll wait.

Done? Ok, so you are with us on the promise and premise of cloud and short lived application servers. Cattle all the things! You are also with me on the idea that even cattle need to be vaccinated from some common issues, right? Thought so. Finally, you like the promise of automation via OpenStack Orchestration (Heat), right?

So specifying that as user-data every time is going to get cumbersome, as will managing your build scripts if you are using CLI tools and the like. A better way, is to build it into a Heat Template. This allows for some flexibility in both version controlling it as well as layering HOT templates to a desired end state (HOT allows for template nesting.

Our Template!

This will be the ‘hardening’ template. Go ahead and drop this into a text file and save it as “hardening.yaml”. If you read the above linked ‘hardening’ post, it’s just a translation of the same.

There isn’t anything too particularly interesting in here. In fact it is more or less an amalgamation of the spiderfoot bits from here & the hardening bits from before.

Enjoy!

Basic Hardening with User Data / Cloud-Init

The ‘cloud’ presents new and interesting issues around hardening an instance. If you buy into the cattle vs. puppies or cloudy application building mentality, your instances or VMs will be very short lived (Hours, Weeks, maybe Months). The internet however, doesn’t much care for how short lived the things are, and the attacks will begin as soon as you attach to the internet. Thus, the need to harden at build time. What follows is a quick guide on how to do this as part of the ‘cloud-init’ process.

Note: Yes, you can use pre-hardened images. However, keeping them up to date and deploying them everywhere presents it’s own set of challenges.

Note the second: The things I do below only harden some of the things, and only do an ‘ok’ job at it. You will want to build this out as needed per your policies / regulatory requirements.

What is User Data / Cloud-Init?

A full discussion of cloud init is a bit beyond the scope of what we’re doing here. That said, there are some details that will help you understand what we’re doing and how it works. From the OpenStack User Guide:

“User data is the mechanism by which a user can pass information contained in a local file to an instance at launch time. The typical use case is to pass something like a shell script or a configuration file as user data.”

and:

“To do something useful with the user data, you must configure the virtual machine image to run a service on boot that gets user data from the metadata service and takes some action based on the contents of the data. The cloud-init package does exactly this. This package is compatible with the Compute metadata service and the Compute configuration drive.”

Using User Data / Cloud-Init to Harden an Instance

Now that we’ve an understanding of the mechanism to run boot time scripts in your images, next up, we need to do just that. For this example we’ll be using an Ubuntu 12.04 image. First create a file with the following contents:

Now that you have that in a file, you’ll need to fire off a command similar to:

nova boot --user-data ./myfile.txt --image myimage myinstance

Resources

  1. http://docs.openstack.org/user-guide/content/user-data.html
  2. http://plusbryan.com/my-first-5-minutes-on-a-server-or-essential-security-for-linux-servers
  3. http://www.thefanclub.co.za/how-to/how-secure-ubuntu-1204-lts-server-part-1-basics

How to Make A LOT of Devstack with Vagrant

Sometimes you need to make A LOT of devstack instances, and on different providers. That is, some in the cloud, others locally, and then some. At least, I keep finding myself in situations where this happens, so, I made a little Vagrantfile to handle the details for me. Here is the Vagrantfile I use to make that happen. After you look the file over, I’ll explain a few key spots and show you how to use it:

Now let’s talk about the interesting spots:

  • Line 5: This defines the host name and number of devstacks you need (2 in this case). It also defines the starting IP if you are using this locally.
  • Lines 8 – 71: These are the bit that gets you ready to do the devstacks.
  • Lines 88 – 93: To make this work on the Rackspace cloud, modify these lines as needed.
  • Lines 97 – 118: These are some VMware Fusion performance hacks. Note 116 sets it to 6gb. Change that if doing this locally.

If you intend to use the Rackspace provider, you’ll need it installed, you can get that from here
That’s about it for the interesting parts. Now to use this, you can bring up all the machines at once using:

vagrant up --provider=vmware_fusion or vagrant up --provider=rackspace

To bring them up one at a time:

vagrant up devstack-01 --provider=rackspace

Once things are online, you still need to access and run stack.sh. To do that:

vagrant ssh devstack-01
su stack -
cd ~/devstack
./stack.sh

OpenStack Heat Concepts, Architecture, and Workflow

It’s not often (at least in the OpenStack world) that you get 70% done with a blog post explaining a thing and then find a resource that does it better than you could have. In this case, while it’s a bit dated, is a presentation from Mirantis on Heat, it’s Concepts, Architecture, and Workflow:

Autoscaling with Heat on Devstack

Autoscaling is one of the more interesting (and outstanding) features of OpenStack Heat. In this post, you’ll build a Devstack environment in a VM with Heat, Neutron, and Ceilometer. This will enable you to run the Autoscaling examples.

 

Getting Started

To get started, copy the following into a Vagrantfile and vagrant-up. This generally takes 3-5 minutes. Now that you have the VM powered on we need to get Devstack running. To do that, run the following commands:

vagrant ssh
sudo su stack
cd ~/devstack/
./stack.sh

Note: Now is the time to top off your coffee and batch process your emails. It’s going to be a while. On my box: stack.sh completed in 2679 seconds.

Basic WordPress

Now that Devstack is up and running, let’s dive right into the examples. First we’ll build a single WordPress install just to get a feel for things. To do that, run the following commands:

source ~/devstack/openrc
cd ~/heat-templates/hot/
ssh-keygen -t rsa -N ""
nova keypair-add --pub-key ~/.ssh/id_rsa.pub testkey
heat stack-create WordPress -f F20/WordPress_Native.yaml -P key_name=testkey

What we did, was set some environment variables for our credentials (as who wants to type those out EVERY time?). Next we moved into where we cloned the heat templates, generated an SSH key, and added it to Nova as ‘testkey’. Finally we launched our heat command.

Let’s break that command down some:

  • heat – The CLI tool we’re calling stack-create – This follows the noun-verb nomenclature of OpenStack. For heat, the ‘thing’ on which you operate is a stack. In this case we’re instructing heat to create a stack.
  • WordPress – This provides the name of the stack. These should be unique within a tenant.
  • -f F20/WordPress_Native.yaml – This specifies the template file to use. In this case, the Fedora 20 WordPress install.
  • -P key_name=testkey – -P tells the heat client that we are providing parameters that are requested within the template. In this case, key_name is a required parameter. So we tell it to use the testkey we created.

Ok, it should be done building now, let’s use another heat command to check:

heat stack-list
+--------------------------------------+------------+-----------------+----------------------+
| id                                   | stack_name | stack_status    | creation_time        |
+--------------------------------------+------------+-----------------+----------------------+
| 5fad5da6-75e9-40b2-a32f-86345b0263f3 | WordPress  | CREATE_COMPLETE | 2014-04-04T17:33:08Z |
+--------------------------------------+------------+-----------------+----------------------+

There we go! Let’s find out a bit more about it, we do that with heat stack-show:

 heat stack-show WordPress
+----------------------+----------------------------------------------------------
| Property             | Value
+----------------------+----------------------------------------------------------
| capabilities         | []
| creation_time        | 2014-04-04T17:33:08Z
| description          | Heat WordPress template to support F20, using only Heat
|                      | OpenStack-native resource types, and without the
|                      | requirement for heat-cfntools in the image. WordPress

Note, this can be enough of the name to be unique, or the UUID. In most cases, you’ll want to use the UUID. Finally, let’s clean up our environment to make room for the autoscale example:

heat stack-delete 5fad5da6-75e9-40b2-a32f-86345b0263f3
+--------------------------------------+------------+--------------------+----------------------+
| id                                   | stack_name | stack_status       | creation_time        |
+--------------------------------------+------------+--------------------+----------------------+
| 5fad5da6-75e9-40b2-a32f-86345b0263f3 | WordPress  | DELETE_IN_PROGRESS | 2014-04-04T17:33:08Z |
+--------------------------------------+------------+--------------------+----------------------+
heat stack-list
+----+------------+--------------+---------------+
| id | stack_name | stack_status | creation_time |
+----+------------+--------------+---------------+
+----+------------+--------------+---------------+

Autoscaling WordPress with Heat

Sorry for the delay in getting here, however, covering the basics will let us skip a lot of that now. The Autoscale environment is a bit more complex. That is, we have the following:

Load Balancer (LBaaS) –> Web Tier (Autoscaling Group) –> Database Tier

To build the stack, we need to find out some things about our environment. Specifically a subnet ID to place the load balancer and nodes on and an Image ID for the base of our webserver & db servers. Let’s do that:

neutron subnet-list
+--------------------------------------+----------------+-------------+--------------------------------------------+
| id                                   | name           | cidr        | allocation_pools                           |
+--------------------------------------+----------------+-------------+--------------------------------------------+
| 82596713-60ff-42ee-b766-6c501991cf55 | private-subnet | 10.0.0.0/24 | {"start": "10.0.0.2", "end": "10.0.0.254"} |
+--------------------------------------+----------------+-------------+--------------------------------------------+
$ glance image-list
+--------------------------------------+---------------------------------+-------------+------------------+-----------+--------+
| ID                                   | Name                            | Disk Format | Container Format | Size      | Status |
+--------------------------------------+---------------------------------+-------------+------------------+-----------+--------+
| 7b3bf53c-3875-4f7c-bc6d-eff11290078c | F19-i386-cfntools               | qcow2       | bare             | 469383680 | active |
| 134c4d81-5c00-45d5-b089-9fbd2d5b9afc | F19-x86_64-cfntools             | qcow2       | bare             | 528297984 | active |
| 64ade0c6-9e10-4257-9884-7406d68d49de | Fedora-x86_64-20-20131211.1-sda | qcow2       | bare             | 214106112 | active |
+--------------------------------------+---------------------------------+-------------+------------------+-----------+--------+

Now that we have those, let’s fire up the build:

heat stack-create WP-Autoscale -f autoscaling.yaml -P database_flavor=m1.small -P subnet_id=82596713-60ff-42ee-b766-6c501991cf55 -P key="testkey" -P flavor=m1.small -P image=7b3bf53c-3875-4f7c-bc6d-eff11290078c
+--------------------------------------+--------------+--------------------+----------------------+
| id                                   | stack_name   | stack_status       | creation_time        |
+--------------------------------------+--------------+--------------------+----------------------+
| f736fd6f-0348-4381-b4e5-7a4da4c3282e | WP-Autoscale | CREATE_IN_PROGRESS | 2014-04-04T17:45:25Z |
+--------------------------------------+--------------+--------------------+----------------------+

On my box, this takes A WHILE to complete. Once it’s done, you can use the same commands to view the environment, or, you can log into the dashboard and check it out:

Stack Detail - OpenStack Dashboard 2014-04-04 12-52-30 2014-04-04 12-52-33

OpenStack Heat Link-Dump

Link dump! That is, I’ve been looking into OpenStack Heat quite a bit lately. That generally means I’ve got way too many browser tabs open at the moment and need to shut some down. Thus, here comes a huge list of OpenStack links and resources.

Heat Websites

Heat related videos

#vBrownBag at OpenStack Summit, first Sponsor

I’m delighted to welcome HP to the vBrownBag sponsor list. HP are the first sponsor to signup to support the vBrownBag TechTalks at the OpenStack Summit in Atlanta.

The TechTalks are taking the place of the lightning talks that happened at previous summits. We are mainly adding recording/streaming as well as the structure. The aim is still to get as many people as possible to present the core of a topic for the OpenStack community.

We already have 25 sessions proposed, and plan on accepting every session we can fit into our schedule. If you would like to present simply fill in this form. I’m really impressed with the depth of the topics being proposed, it should be a great set of TechTalks.

We need a few more sponsors, if your organization would like to be aligned with this community effort then please drop me an email, alastair@vbrownbag.com

With only six weeks to go it’s getting exciting, looking forward to meeting some new people.

My First OpenStack Heat Template

If you have followed some of my past work, you know I have a thing for automation and orchestration. Needless, I was super excited when I discovered OpenStack Orchestration (aka Heat).

Don’t know what Heat is? The OpenStack Wiki page has a great explanation:

Heat is the main project in the OpenStack Orchestration program. It implements an orchestration engine to launch multiple composite cloud applications based on templates in the form of text files that can be treated like code. A native Heat template format is evolving, but Heat also endeavours to provide compatibility with the AWS CloudFormation template format, so that many existing CloudFormation templates can be launched on OpenStack. Heat provides both an OpenStack-native ReST API and a CloudFormation-compatible Query API.

In fact the Heat documentation is ok in most areas: http://docs.openstack.org/developer/heat/getting_started/index.html

Note: The example below uses Rackspace Cloud. In order to adjust this to work with Devstack some changes will need to be made (changing the flavors, etc).

Building a Simple OpenStack Heat Template

Heat Templates use a format conveniently called the Heat Orchestration Template or HOT. It’s important to note that at this time, HOT is still under active development. For the template, I decided to draw on my earlier work with SpiderFoot here. To get started, open your favorite text editor, and paste this YAML in. Once you have it in, I’ll describe it a bit:

 

  • 1: specifies the HOT template version. When reviewing third party templates, you can use this to see if said template is compatible with the version of Heat you have deployed.
  • 3 – 4: The description. Not required but a really good practice. the | character allows it to be split onto multiple lines
  • 6 – 39: Parameters. These lines specify various bits of input that are required when spinning a thing up from a HOT template.
  • 6: Opens the parameters clause
  • 8 – 15: Opens the “server_name” parameter. The name here is arbitrary, but should be self descriptive and consistent throughout
    • 9: Provides the variable type for ‘server_name’, in this case, a string
    • 10: A description
    • 11: default – allows us to specify a default value. In this case, we require this value to come from the CLI (get_param) (Edited per comments)
    • 12 – 15: This is where we set up a string replacement. The specifies a string in which we perform replacements. The “params” specifies what to replace with.
  • 16 – 25: Image – Specifies which image to use. A lot of this follows the same setup as lines 8-15. That said, note 22 – 25. This provides constraints on inputs. In this case defining images that can be used.
  • 26-39 – Flavor – Specifies the flavors to use. Note: These flavors are rather Rackspace Cloud centric. You can modify to fit your OpenStack deployment.
  • 41 – 65: Resources – Resources is a required section. It specifies what resources will be deployed and how they will be configured when you launch the Heat template. There are a number of resource types available, a relatively complete list can be found here: http://docs.openstack.org/developer/heat/template_guide/openstack.html
    • 42 – 65: “spiderfoot” This specifies the one resource we will be deploying as such:

Note: Don’t actually use these, especially 48 – 65. I do some terrible things like setting the root password to vagrant and running an InfoSec service that has no auth control (basically anyone, anywhere, can use the box to scan /anything/). I’ve only provided this as an example of how to assemble a Heat template

  • 66 – End of the file: These specify outputs available. These are what becomes available using the heat stack-list and heat stack-show commands.

Summary

Ok, in what turned out to be a much longer post than I’d anticipated, we built a very very simple heat template. Hope this has been helpful. In some future posts, I’ll show how to deploy Heat itself and from there, how to use this template to deploy against it.