Generate SQL from your migrations

On one of the projects we're working on, we needed to occassionaly generate SQL from our migrations. Déjà vu, I thought. A few minutes of Googling later, I remembered why: Jay and I had been on a project a couple of years ago whence we had the same need. Jay's code didn't quite work any more due to some ActiveRecord changes, and a search for an alternative implementation turned up nothing.

I took his code and modified it to our needs. A short time later, I had the code pulled out into a rails plugin, migration_sql_generator. Install it (script/plugin install git://github.com/muness/migration_sql_generator.git) and then run the rake task:

rake db:generate:migration_sql

Running this task generates two sql files per migration in db/migration_sql in the form 20090216224354_add_users.sql and 20090216224354_add_users_down.sql.

I've used the plugin with success using the sqlserver adapter, less luck with the mysql adapter (change_column and rename_column blow up because the mysql adapter checks for the presence of a column first) and no luck with the sqlite adapter. Haven't tried it with the postgres adapter.

Easily switch between Ruby 1.8.6 and 1.9.1

NOTE: I don't keep Ruby Switcher updated anymore. Use RVM instead.

Chad mentioned that he'd gotten ruby 1.9.1 and 1.8.6 side by side on his workstation (his code for this is in his awesome spicy-config repo).

I took some time this morning to get a similar setup working. Now, on my prompt, I type, use_ruby_186 or use_ruby_191 to go back and forth between the Ruby 1.8.6 shipped with Leopard and a self-compiled install of ruby 1.9.1.

Steps for you to get there:

1. Compile and install ruby 1.9.1:

   mkdir -p ~/tmp
   cd ~/tmp
   curl -O ftp://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.1-p0.tar.gz
   tar xzf ruby-1.9.1-p0.tar.gz
   cd ruby-1.9.1-p0
   ./configure --prefix=$HOME/.ruby_versions/ruby_191
   make
   make install
   

2. Install the ruby switcher:

   curl -L http://github.com/relevance/etc/tree/master%2Fbash%2Fruby_switcher.sh?raw=true?raw=true > ~/ruby_switcher.sh
   echo "source ~/ruby_switcher.sh" >> ~/.bash_profile
   

Note that I use ~/.gem for my gems. If you don't do that, you'll have to modify the switcher to specify your GEM_HOME. Type gem env and look for GEM PATHS to figure out what you should set it to.

Maatkit installation: the concise guide

If you use MySQL, odds are you need Maatkit whether you know it or not. It's the Swiss army knife: I mostly use it to parallelize backup and restore of huge databases on multi-core machines, but it's also handy for fake splitting large files, executing sql on multiple tables and a whole lot more. (You also want mytop if you're wondering what's up with MySQL connections.)

Here's the install script if you too want Maatkit at your fingertips (and you do. trust me.):

#!/bin/sh
# tested on Leopard with MySQL installed using the package installer

sudo perl -MCPAN -e 'install DBI::Bundle'
sudo perl -MCPAN -e 'install DBD::mysql'
# you may need to force the install as follows:
# sudo perl -MCPAN -e 'force install DBD::mysql'

mkdir -p ~/tmp
cd ~/tmp
curl -O http://maatkit.googlecode.com/files/maatkit-2725.tar.gz
tar xzvf maatkit-2725.tar.gz
cd maatkit-2725
perl Makefile.PL
sudo make install

No network after copying an Ubuntu VMWare image

After you copy an Ubuntu image, you'll probably lose your network connectivity. After a little bit of digging, it turns out that Ubuntu persists the MAC address of the network device in /etc/udev/rules.d/*net.rules . The fix:

  sudo rm /etc/udev/rules.d/*net.rules
  sudo shutdown -r now #to reboot

Git: things I love and manipulating remote branches

I've been happily using Git for months now. The primary benefit for me has been the ability to work offline without messing with SVK, not having to use the svn commands for renames and moves, the ability to use multiple remote servers and better tools to look at branch level history (think gitk).

Branching is much improved, besides. In Subversion, to branch, you copy the current tree elsewhere and then you work in this branch and finally, merge your changes manually back into trunk (to wit, you have to keep track of what was already merged and not reapply those commits). In git, though, that's all handled for you. Another difference is best illustrated by example: you've started a new feature. Several file modifications later, you look at your diff and realize, "I should have started a new branch for this. doh." With Subversion, there's not much you can do automatically to get these changes into a branch. You'd typically now:

1. generate a diff
2. undo your changes
3. create a branch
4. switch to the branch
5. apply the diff
6. back to work

In git on the other hand, you type git branch feature_22.

Sweet, huh?

Well, not altogether. Pushing this local branch to a remote server is altogether too complicated:

git push origin feature_22:refs/heads/feature_22
git fetch origin
git config branch.feature_22.remote origin
git config branch.feature_22.merge refs/heads/feature_22
git checkout feature_22

Sure, you could have created the branch on the remote server first and saved yourself some hassle:

git push origin master:refs/heads/feature_22
git fetch origin
git branch --track feature_22 origin/feature_22
git checkout feature_22

Not exactly straight forward either. Neither are other standard branch related activities. Here's how you delete that remote branch:

git push origin :refs/heads/feature_22 # yeah, that's how you delete a remote branch.  :(
git branch -d feature_22 # delete it locally too

Fortunately, Git has a vibrant community that work to abstract away some of these complications. One of my favorite Git tools is git_remote_branch. By using it, working with remote git branches becomes a breeze:

grb publish feature_22 # publish a local branch
grb create feature_22 # create a remote branch and local branch to track it
grb delete feature_22 # delete the remote and local tracking branch

Ah, that's better!

Mingle, meet Git

Several weeks back, Adam Monago, Mingle's product manager was visiting our office in Chapel Hill. One of the topics that came up was Git integration with Mingle. Alas, it sounded like it was a ways out. But he explained that the SCM integration was pluggable and that we could write the code and drop it in place, echoing things I'd heard from other ThoughtWorkers.

A couple of days later, Don and I were curious as to just what it'd take to implement the integration. Reverse engineering the interface we had to implement from the Subversion and Perforce plug-ins was anything but fun, but before the day was over, we had rudimentary integration working: we could see commit messages and the list of files modified per check-in. Since then, Don implemented the rest of it including Mingle-based source code browsing.

We've been using the mingle_git plugin for a little while with no problems. For performance reasons, we're not using the source code browsing from inside Mingle. Instead, we use GitHub's source browsing: see the README for instructions on wiring things that way too.

Installation is documented in the README. We've only used this on Mac OS, FreeBSD and Linux. Windows users, you may have luck by using msysgit.

Caveat emptor: if you browse around the code, you'll quickly conclude that it is a spike: we started with the subversion plugin and evolved it to this. Since the SCM integration API is not documented, we can't even be positive that we implemented it correctly (though the evidence suggests that we have). Also note that Mingle makes the assumption that check-in numbers are sequential, and that's an assumption that is not Git-friendly and it shows in the code.

Enjoy, but keep in mind that this is unsupported, use-at-your-own-risk software. For me, that's better than no Git integration. ;)

Note: Mingle's APIs changed from 2.0 to 2.1 and we haven't had a chance to update this plugin yet. If you installed this plugin, and are ready to upgrade to 2.1, you'll want to remove the plugin and then execute this SQL on your Mingle database: delete from plugin_schema_info where plugin_name = 'mingle_git';

Encrypt your client data in 53 minutes

Thanks to Aaron's continued efforts to make us ever more security conscious, I've been encrypting client data on my laptop. This came up in conversation during erubycon (thanks EdgeCase for a fun conference with engaging evening events). A couple of people asked me to get them started on encrypting their data too.

Here are instructions. They're written for Mac users, but they'd be nearly identical for Linux and Windows users since TrueCrypt is available there too.

1. Install TrueCrypt.
2. Create a container based encrypted file. Pick FAT as the file system. I use both keyfiles and a password.
3. Erase the FAT partition and then format as HFS/ext2/[fs of choice] using Disk Utility/[partition manager of choice]. Name it client_data.
4. Mount the newly created partition.
5. Move your sensitive data to /Volumes/client_data. This is by far the slowest part of the process. While you wait, watch The Enemies of Reason (48 minutes -- everything else should take 5 minutes or less).
6. Modify TrueCrypt preferences to suit your needs. Some suggestions: leave encrypted volumes mounted when quitting TrueCrypt, set Auto-dismount volume after 30 minutes of inactivity in the partition.

As mentioned in my previous blog entry, I use a script to automatically create aliases to allow easy switching to project directories. All I had to do was switch that entry to point to /Volumes/client_data instead of the old ~/work/client_data. Now, when I want to work on client projects or contracts, I launch TrueCrypt, mount the container and open a new terminal window. All my old aliases work just like they used to.

Mac users: if you don't care about the cross-platform compatibility that TrueCrypt offers, and trust Apple to encrypt your data, you could use Disk Utility to create encrypted partitions instead. Some people insist it's more convenient (You know who you are!).

Lazy bash cd aliases

Those of you who've found value in my last couple of bash specific posts may also like the latest addition to my ~/.bash_profile.

This one iterates through the one or more directories and creates aliases to the subdirectories so I don't have to. Here's the scenario: I've got a directory ~/work/ where I keep work projects, a ~/writings/ where I keep all the writing projects and so on. I used to have aliases to each subdirectory. e.g. alias project1="cd /Users/muness/work/project1". With shame, I admit that I maintained each of these manually. No more!

Install instructions:

curl -L http://github.com/relevance/etc/tree/master%2Fbash%2Fproject_aliases.sh?raw=true?raw=true > ~/.project_aliases.sh
echo "source ~/. project_aliases.sh" >> ~/.bash_profile

Usage instructions:

• Move your work projects to ~/work.
• Above the source ~/. project_aliases.sh add PROJECT_PARENT_DIRS[0]="$HOME/work".

You may be interested in my blog post over at PragMactic OS-Xer where I describe my motivation for these recent shell scripts.

bash: don't make me think

Last week I figured out a way to make my life a little bit easier by abstracting the scm I was using and having the prompt indicate whether I was in a Subversion or Git. Thanks to Mike Hommey whose script I tweaked for my needs.

For a long time, I've wanted my iTerm tab title to be more useful. I started by having it display the last process I executed in that tab (this also shows the full path in iTerm's window title bar):

PS1='\[\e]2;\h::\]${PWD/$HOME/~}\[\a\]\[\e]1;\]$(history 1 | sed -e "s/^[ ]*[0-9]*[ ]*//g")\a\]\$ '

Next up I wanted to show the currently running command. Google showed me how:

trap 'echo -e "\e]1;$BASH_COMMAND\007\c"' DEBUG

Instead of trying to explain this code I'll quote the Bash man page:

BASH_COMMAND
The command currently being executed or about to be executed, unless the shell is executing a command as the result of a trap, in which case it is the command executing at the time of the trap.

Some more tweaks followed:

• Distinguish between a previously executed command and a currently executing command by decorating them (I chose surrounding the former with braces and the latter with >/<. e.g. [ls] and >vi<).
• Show the context of the executing command in the tab. This is just the repository where I executed the command.
• Make this coexist with TextMate.

And a screenshot to summarize:

The code is in github/relevance/etc/tree/master/bash/bash_vcs.sh.

Install instructions:

curl -L http://github.com/relevance/etc/tree/master%2Fbash%2Fbash_vcs.sh?raw=true > ~/.bash_dont_think.sh
echo "source ~/.bash_dont_think.sh" >> ~/.bash_profile

Enjoy! Tested with iTerm on Leopard.

Stop the presses: bash said to embrace subversion and git

Update: I've since updated and moved this script. See my new blog post.

I had a chance to pair with Rob today. On his console window, I noticed something I wanted: his bash prompt had an indication that he was in a Git repository along with the branch he was on. Later, after he'd left, I found myself wondering how that worked and that I simply had to have it, and I wasn't willing to wait a whole day to figure out how he'd done it. Patience is not a virtue. (In that spirit, click here for the code if you're too impatient to read the rest of this blog entry which I spent countless hours writing. Uphill both ways. In the snow.)

A few minutes of googling turned up a blog entry with relevant bash fu. The code there was for changing the prompt such that it indicated whether you were using svn, git, svk or Mercurial along with some repository metadata. A couple of minutes later (it took a while to copy and paste the code since it was interweaved with comments), I found out it didn't work on Leopard. readlink turned out to be the culprit. Instead of figuring out why it wasn't working, I replaced it with something simpler (specifically I used: base_dir=`cd $base_dir; pwd`). Voila, things worked: I now had a hella cool prompt that showed me if I was in a Subversion or a Git repository - I never used Mercurial, and left SVK for Git a couple of months ago. Yay.

Being who I am I had to do something more with this new ability to distinguish between Subversion and Git. And I knew exactly which itch to scratch: 90% of the time I use the same scm commands. I even had shell aliases for them so I wouldn't have to type them. A few minutes later I'd converted my previously Subversion specific aliases to generic ones that worked with Git too. Screenshot says it all:

If you want the same, download the script as ~/.bash_vcs and add source ~/.bash_vcs at the end of your ~/.bash_profile.