Fork me on GitHub

Drew's World

Rants, News, Etc on my Life and Projects

Shrinking Vagrant Linux Boxes

by Andrew De Ponte (@cyphactor)

So, lately I have been using Virtual Machines via Vagrant a siginificant amount as we are now using them to host our development environments and dependencies for all the development work we do. We have been using Puppet to automate the configuration of our environments and to make building our boxes easier.

Once you have provisioned a VM and you have it all dialed, you will most likely want to create a Vagrant Box from that VM and host it up on S3 so that new developers can get started simply by installing Vagrant, cloning your repository and running:

$ vagrant up

This actually works great. However, I found that the VM disks both for the Vagrant VMware Provider and the VirtualBox Provider grow dynamically, but they never shrink. So, when I was going through the process of building the VM and installing all the dependency software, etc. it was bloating the base machine. This was the case even though I removed the temporary cruft from the filesystem on the guest.

With a little research and some trial and error I found that you can shrink these disks as long as they are ext3 or ext4 file systems with the help of the following command:

$ vagrant ssh
$ sudo dd if=/dev/zero of=wipefile bs=1024x1024; rm wipefile

What the above command is doing is simply writing zero bytes to the wipefile in chunks of 1024 bytes until there is no disk space left in your VM’s disk. Then it is removing the wipefile. This basically leaves all those excess bytes zero’d out.

This is necessary because the shrink/compaction tools provided by either VMWare or VirtualBox both have no way of identifying space they can free up in the disks unless they are zero’d out.

Before shrinking/compacting you should always make sure to exit the guest machine using the following commands:

$ exit
$ vagrant halt

With VMware you can shrink the vmdk disk by doing the following:

$ vmware-vdiskmanager -d /path/to/main.vmdk
$ vmware-vdiskmanager -k /path/to/main.vmdk

I discovered the above commands in the Vagrant VMware documentation, http://docs.vagrantup.com/v2/vmware/boxes.html

With VirtualBox the only way I was able to shrink the disk image was to clone it to a smaller copy using the following command:

$ VboxManage clonehd name-of-original-vm.vdi name-of-clone-vm.vdi

Once you have cloned the vdi you can then import it into the VM through VirtualBox and get rid of the original vdi.

That is how you “Shrink Vagrant Linux Boxes!”

RVM Installed Ruby (SSL certificate verify failed)

by Andrew De Ponte (@cyphactor)

Recently a few of our developers were having problems getting SSL certificate verify failures in Ruby. The error was the following:

OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed
  from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/net/http.rb:799:in `connect'
  from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/net/http.rb:799:in `block in connect'
  from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/timeout.rb:54:in `timeout'
  from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/timeout.rb:99:in `timeout'
  from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/net/http.rb:799:in `connect'
  from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/net/http.rb:755:in `do_start'
  from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/net/http.rb:744:in `start'
  from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:306:in `open_http'
  from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:775:in `buffer_open'
  from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:203:in `block in open_loop'
  from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:201:in `catch'
  from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:201:in `open_loop'
  from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:146:in `open_uri'
  from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:677:in `open'
  from /Users/adeponte/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/open-uri.rb:33:in `open'

The error was reproducable by running the following Ruby snippet in an irb instance of the RVM installed Ruby version.

require 'open-uri'
open 'https://google.com'

I spent a good amount of time digging into things and trying to figure out exactly what the issue was. I came to the conclusion that they either had an out of date Root CA Certificate Bundle or SSL couldn’t find it for some reason. So, I went down the path of trying to import cacert.pem from http://curl.haxx.se/ca/cacert.pem (cURL’s website).

I went down this path because I vaguely remember doing this at some point in the past to solve the Twitter API SSL Verify fail issue when they changed their certs. I ran into a huge amount of difficulty trying to get the thing imported or converted into a format that Mac OS X Keychain Access would be happy with importing. I tried pem, crt, pck12, etc. Nothing I did seemed to make a difference.

This is when I saw a post of someone talking about SSL being installed inside of RVM and I looked at the latest output from rvm requirements. I quickly noticed that it now says to install SSL via the following command:

rvm pkg install openssl

Come to find out if you have run the above command before installing your Ruby version it will be using the RVM packaged openssl instead of the system openssl. That also means that the Root CA Certificate Bundle is different, or in this case, missing completely.

After discovering this and doing a little googling I found the solution to be as simple as the following command:

curl http://curl.haxx.se/ca/cacert.pem -o ~/.rvm/usr/ssl/cert.pem

I found this solution on the following blog post http://fredwu.me/post/28834446907/fix-openssl-error-on-mountain-lion-and-rvm. Hopefully, writing another blog post on the subject and linking to this one will help it gain its ranks in google search so that other people can find it quicker.

After running the above command you should now be able to restart your irb instance and re-run the test snippet described at the beginning of this post. This time you should get output looking similar to the following:

 => #<File:/var/folders/j_/9nxyjmd10js8g1sjx446vrsm0000gn/T/open-uri20120907-36935-4i4y64> 

Just like that your SSL woes are gone.

Sublime Text 2 Guard Plugin Updates

by Andrew De Ponte (@cyphactor)

I woke up early this morning and thought it was time for a little sublime_guard love. I went through and tackled all of the syntax highlighting issues. This included the following issues (#27, #30, #31, #38, #43, #29).

I also cleaned up a number of tickets that were lying around and had either been completed already or were tickets that were invalid.

I also started a branch for adding python3 support to the plugin. The branch is named python3_support. I would love it if anyone out there in the ether would grab that branch, test it out, and give me some feedback.

Anyways, I hope you guys enjoy the updates. As usual if you installed from Package Control just restart Sublime Text 2. If you installed manually then just pull the latest version down with Git.

See you all next time.

Ruby and Rails Mountain Lion Hiccups

by Andrew De Ponte (@cyphactor)

So when Mac OS X Mountain Lion was released a few days ago I jumped on upgrading and got switched over as soon as I could. My mentality for upgrading is that I would much rather deal with the pain of upgrading in smaller chunks when needed rather than waiting. Plus, I just like to be on the latest and greatest in terms of the tools I use for my job.

Anyways, below is going to be a listing of some of the hiccups that either I or other developers on my team have run into in terms of their Ruby and Rails development environment setups.

Git

Some of the devs on my team had Git installed via the git-osx-installer. These developers seemed to lose their Git installs when they upgraded. I was using a Homebrew install of Git and it worked just fine for me after upgrading. So be aware of what your situation is and resort to brew because it is awesome.

XCode 4.4

In order to build gems with the native exensions and just to have the basic development tools around I installed the latest XCode 4.4. Once installed I went to the preferences menu, in the download tab, and chose to install the Command Line Tools because that is really what I was looking for more than anything. I have read on other posts online that you can download the Command Line Tools alone and install just them. I personally chose to install XCode as I do use XCode and the native tools to work on iPhone/iPad app development as well.

gcc-4.2

The latest XCode now comes with clang/llvm which is a new C/C++/Objective-C compiler. The problem is that the Ruby language isn’t fully compatible with clang and llvm. Therefore, you need to install GCC 4.2 using Homebrew. GCC 4.2 is the latest version of the GCC compiler that apple provided before switching to clang/llvm. Anyways, you can install GCC 4.2 by doing the following with brew:

brew tap homebrew/dupes
brew install apple-gcc42

The above will install the apple gcc-4.2 in a side by side fashion with XCode. This means that it won’t mess up your existing XCode install like other solutions.

Once you install gcc-4.2 then you can install/reinstall your ruby version inside of RVM as needed.

Apache 2 (Web Sharing)

In the upgrade it seems that they have revamped the Sharing preference pane. In the process of this it seems that Apple has decided to remove the Web sharing component of this preference pane. Don’t worry though. Apache is still there and running. You just can’t start it and stop it from the preferences pane anymore. You can use the following commands instead.

sudo apachectl stop
sudo apachectl start
sudo apachectl restart

Note: When the upgrade happened it nuked all my /etc/apache2/users/ configs and all my other apache configs. So, if you have any custom apache configs as I did you should probably back them up before upgrading.

X11

It seems that another thing that was changed with the upgrade to Mac OS X Mountain Lion was that they removed the X11 that I had previously installed and seemed to no longer provide it. Thats ok though because XQuartz 2.7.2 is what you want and it is available freely at http://xquartz.macosforge.org/landing/. Just install that bad boy and follow the on screen instructions to reboot when it is done and you should be good to go in terms of X11.

ImageMagick

Before upgrading I had ImageMagick installed via Homebrew. After, the upgrade was finished my ImageMagick was exiting with failures about some dylib. To solve this problem I simply ran the following and I was back in running order with respect to ImageMagick:

brew upgrade imagemagick

Pygments

I use Pygments to handle syntax highlighting in my jekyll sites and it seemed to not be working after the upgrade. I believe this is because it is now using Python 2.7.2. To resolve this issue I just re-installed Pygments using the following:

sudo easy_install Pygments

Sublime Text 2 Guard Plugin Updates

by Andrew De Ponte (@cyphactor)

It has been a while since I have provided an update on my Sublime Text 2 Guard Plugin and I just made some changes so I figured I would make a quick post. The project has recently had a number of contributions which I will list in the changes below.

I have been completely stoked with the growing usage of this plugin as well as the abundance of contributions people have been making to the plugin. In this update there are a number of people that need to be called out with special thanks.

@diogomafra provided a Clear output when predefined text found feature. I am sure many users will find this feature a nice addition to smoothing out their workflow. Thank you for your contribution @diogomafra.

@chrislerum provided rbenv support. I know that this feature will be huge for a number of the users as this was a highly requestd feature. Thank you so much for your contribution @chrislerum.

@brandonblack provided the ability to function without bundler. This will allow users who aren’t using bundler to be able to use this plugin. I think this is a great addition as the more people we can get using this plugin the more feedback we can get. Hence, the better it will become. Thanks for your contribution @brandonblack.

@whitequark provided the basis for the Run all Tests & Show Guard Output feature. I know a number of developers who have been waiting for something like this to fit their personal workflow. Thank you very much for your contribution @whitequark.

A big thanks goes out to all the above listed contributors and all of our prior contributors. Thank you all for helping make this a solid very useful plugin for the development community. As usual please make issues for any bugs or feature requsets you have at Sublime Text 2 Guard Plugin.