Shaving The Dev/Working Environment Yak

So one of the more interesting things about starting to work at a new company is getting the development environment right. For me, what gets really interesting is understanding all of the things that this involves. I am going to skip discussing things like iTerm2 and my tmux config and really focus in on what I have been doing to work with Chef and adapt to the way we do things as we deal with Cookbooks.

Starting with Chef DK

I lucked out that Chef DK was released right before I started at Chef. As you can probably guess, that is what I decided to use as my starting point. I have to say, one of the things that has impressed me the most since moving to doing Chef regularly is all the various extension points. As I started work on things with Supermarket, the ability to quickly build something like knife-supermarket was a huge win.

So to that end, one of the first things I learned how to use as part of Chef DK was chef gem install. In general, I have installed things as I have needed them. The biggest thing I have started with is installing the stove gem by Seth Vargo. This gem gives me an easy way to setup all the things around publishing new versions of cookbooks to the community site. In addition to that, I am a huge fan of guard. I use guard to watch for changes to files and run my tests automatically. To that end, I have installed the following guard plugins:

  • guard-foodcritic
  • guard-rubocop
  • guard-rspec

All of these were installed by running chef gem install to make sure they were installed into the same world that my Chef DK environment uses.

To The Cloud

So one of the more interesting parts of my job at this point is that multi-platform support is not only a nice to have, it is a basic necessity for many of the cookbooks we support. While this is insanely cool, it does make for an interesting adventure trying to get setup to test across the gamut of operating systems. For starters, there isn’t really a single cloud provider that you can test all the OS variants against so you have to use more than one provider. Second, it is actually beneficial for us to use all of the major cloud providers to make sure we see when things change and break for our users.

What that translates to is a lot of stuff to install and setup to get going. In many, hopefully most very soon, of the Chef supported cookbooks you will find a .kitchen.cloud.yml that is setup to do testing across cloud platforms and OS distributions. If you take a look at a typical .kitchen.cloud.yml file like the one found in the opscode-cookbooks/mysql repo on GitHub, you will see a myriad of different providers and environment variables. For now, I am going to gloss over the environment variables. It is enough to say that there are a lot of them and not always intuitive so please hit me up on freenode, if you have any questions about a particular env variable. To enable all the different providers, I have installed a number of test-kitchen plugins. You can’t even run a kitchen list until they exist when using the cloud yaml file. The test-kitchen plugins I have installed are:

As mentioned before, I installed the above using the chef gem install command to ensure that all of those kitchen plugins work with the test-kitchen install that comes with Chef DK. The icing on the cake that makes it all work is that I set KITCHEN_YAML=.kitchen.cloud.yml so that test-kitchen takes advantage of the awesome config that we have setup.

Rub a Little RVM on It

So… I have a little confession to make. While I still don’t consider myself a developer, I have found that over the years writing ruby has been a lot of fun for those pesky web interfaces and complicated scripts that I write. Most of my ruby code would make a good dev cry, but it is still an easy language to get work done in. To that end, having a sane ruby dev environment has also been important for various reasons. Even in the time I have worked at Chef I have found myself writing code to automate things. (For example, I wrote a tool to grab all the neat things going on with Supermarket. That code is on GitHub as cwebberOps/supermarket-report.) While I get that bundler is awesome, I still like to be able to separate out my ruby and my gemsets.

So I looked at chruby and chgems and a little at rbenv. Both left me wanting to go back to rvm, so that is what I did. The interesting thing is that if you have a ruby selected via rvm you lose out on the awesome that is Chef DK. So to get things the way I want them, I have set my default ruby to the system ruby in rvm by running rvm use system –default. For whatever reason, I then have to load a new shell for my prompt to not complain about rvm-prompt being missing. This setup has worked well. My general workflow is to open a new session in tmux and when I switch into a project have the .rvmrc change me to the right ruby. An alternative approach can be found over on Joshua Timberman’s blog about how he uses Chef DK.

All things equal, the setup of the software has been smooth. The only real places I have run into issues is getting all the environment variables setup and accounts created. While having two workstations I use regularly has added to the confusion at times, as I have gotten things squared away, things just tend to work. A huge shout out to the folks working hard on Chef DK, it has for sure made my experience delightful.

Finding the Edge

As part of some of the work I am doing with Supermarket, Chef’s new community site, we needed to make some updates to the way knife worked, specifically giving it the ability to talk to more than one site. So… I set about contacting the folks that do dev work on knife to figure out how the functionality would get added.

Just as soon as I asked in the Dev room on HipChat who I should contact from the Dev team, my boss hit me up. A few lines of chat later and he suggested I open a pull request with the code that adds the needed functionality. Holy crap. What did I just get myself into? My gut reaction was something along the lines of, “Really, me, write Ruby? Have you seen my code?” Well, a few minutes later I had snapped out of it and realized I just needed to level up and, as Opbeat would put it, “Fuck it, Ship it”. After a few more lines of discussion in HipChat, we decided that a knife plugin was the easiest way to get things out and usable quickly.

Digging In

So off I went to create my first knife plugin. Having dabbled in Ruby a bit over the years, I recognized pretty quickly that I could just inherit the functionality from the current knife cookbook site commands and make the few changes that were needed to get a new option setup. Between the following two docs, I was able to get the download command extended to do what I wanted:

So cool, I had code. For testing I just symlinked each file into ~/.chef/plugins/knife/. By doing this I was able to just run knife and test that everything worked. Interesting note for those that go look at the code, to verify things worked, I pointed at the staging site for Supermarket which is located at http://supermarket-staging.getchef.com. Once I did that, I pointed at the current place knife is set to look, http://cookbooks.opscode.com. From there, It didn’t take much to override each of the various methods that mentioned the current cookbooks site. I was off to the races pretty quickly.

Interestingly enough, the most confusing part of the whole adventure was dealing with copyright. More specifically, who to assign authorship to was a bit confusing. The license was already chosen for me which made it easy, but deciding how to handle who to list as authors was kind of weird. In reality, most of the code was copy-pasta, so did that mean I should do a git blame and figure out who was responsible for the lines of code that came from the other subcommands? If I was doing that, should I update their email addresses since we were now Chef and not Opscode? In the end, I decided that I would remove the other authors because of the amount of code and really, I didn’t want them to get random questions about a project they had likely never heard of.

Not only was this my first knife plugin, this was my first Ruby Gem. After some boilerplate borrowed from the knife-openstack plugin, and creating an account on rubygems.org, it was super simple to get a gem uploaded and working. A quick chef gem install knife-supermarket and I was able to find my first bug. The gem process was super smooth and pretty quickly I was able to get a few fixes in place and out the door.

The real take away from all of this was the reminder that I enjoy life the most when living out on the edge of my comfort zone. As I approach that edge, it means that I am learning more and able to experience new things. As my time at Chef begins, I am looking forward to more opportunities to get pushed to the edge and over the edge of my comfort zone.

The Results

If you are running Chef DK you can install the plugin by running:

$ chef gem install knife-supermarket

Otherwise, you can install the gem but running:

$ gem install knife-supermarket

The code can be found at:

Finally, the gem can be found at: