Ruby on Rails Planet

Making CRUD less "Cruddy", one step at a time »

Created at: 27.04.2010 20:42, source: OnRails.org, tagged: Rails Tips Ruby on Rails

One of the great “new” features of Rails (as of 2.3) is accepts_nested_attributes_for, allowing you to build cross-model CRUD forms without “cruddying” your controller. There are some great examples out there about how to do this, but I’d like to walk thorough a particular use case—managing the “join” records in a has_many :through relationship.

Consider the following database schema:

class Villain < ActiveRecord::Base
  has_many :gifts
  has_many :super_powers, :through => :gifts
end

class Gift < ActiveRecord::Base
  belongs_to :villain
  belongs_to :super_power

  validates_uniqueness_of :super_power, :scope => :villain_id
end

class SuperPower < ActiveRecord::Base
  has_many :gifts
  has_many :villains, :through => :gifts
end

In our dataset, there are a relatively small number of super powers which we wish to present as a list of checkboxes on the villain management form. Checking/unchecking the boxes will manage the gift records for that villain, effectively managing the list of super powers available to the baddy.

To get started, we need to add accepts_nested_attributes_for :gifts to the Villain class—piece of cake. To complete the implementation, we need to change the params hash that our form generates. Let’s review the cases that we need to support and the associated params hash format needed to implement the correct functionality.

The first case is a super power record that is not currently associated with the villain. Here, the UI should display an unchecked checkbox. If we check it and submit the form, a gift record should be created linking the villain with the super power, making this bad guy that much badder. Here is an example of the params hash we should be sending to accomplish this:

  {
    'villain' => {
      'name' => 'Lex Luthor',
      ...
      'gifts_attributes' => {
        1 => { 'super_power_id' => 5 },
        2 => { 'super_power_id' => 7 },
        ...
      }
    }
  }

The alternate case is a super power this villain already possesses. In this instance, the UI should display a checked checkbox, and if we uncheck it, the existing gift record should be deleted, diminishing the villain’s capacity for evil. And our params hash needs to look like:

  {
    'villain' => {
      'name' => 'Two-Face',
      ...
      'gifts_attributes' => {
        1 => { 'id' => 101, '_delete' => true },
        ...
      }
    }
  }

Note that the keys for the gifts_attributes hash are arbitrary; we can use any scheme to generate unique keys for the hash.

So how can we craft a form that sends the params hash that Rails wants to see? Here’s my implementation:

  <%- SuperPower.all.each_with_index do |super_power, index| -%>
    <label>
      <%- if gift = @villain.gifts.find_by_super_power_id(super_power.id) -%>
        <%= hidden_field_tag "villain[gifts_attributes][#{ index }][id]", gift.id %>
        <%= check_box_tag "villain[gifts_attributes][#{ index }][_delete]", false, true %>
        <%= hidden_field_tag "villain[gifts_attributes][#{ index }][_delete]", true %>
      <%- else -%>
        <%= check_box_tag "villain[gifts_attributes][#{ index }][super_power_id]", super_power.id %>
      <%- end -%>

      <%= super_power.name %>
    </label><br />
  <%- end -%>

If the gift is detected, the villain has the super power, and we handle our second case from above, using the checkbox / hidden field hack Rails employs in the check_box helper method to make sure a value is sent whether or not the checkbox is checked. The else block handles the other case, setting up our params hash to create the gift if the checkbox is checked.

This works, but we probably don’t want to copy and paste that code everywhere we use this pattern. How can we reuse this in a DRY fashion? Here’s a helper method that encapsulates the logic:

  def has_join_relationship(model, join_collection_name, related_item, collection_index, options={})
    returning "" do |output|
      relationship_name = options[:relationship_name] || related_item.class.table_name.singularize + "_id"
      tag_prefix = "#{ model.class.class_name.underscore }[#{ join_collection_name }_attributes][#{ collection_index }]"

      if join_item = model.send(join_collection_name).find(:first, :conditions => { relationship_name => related_item.id })
        output << hidden_field_tag("#{ tag_prefix }[id]", related_item.id)
        output << check_box_tag("#{ tag_prefix }[_delete]", false, true)
        output << hidden_field_tag("#{ tag_prefix }[_delete]", true)
      else
        output << check_box_tag("#{ tag_prefix }[#{ relationship_name }]", related_item.id, false)
      end
    end
  end

Drop that in a helper, and then your form code becomes:

  <%- SuperPower.all.each_with_index do |super_power, index| -%>
    <label>
      <%= has_join_relationship(@villain, :gifts, super_power, index) %>
      <%= super_power.name %>
    </label><br />
  <%- end -%>

Much nicer … although I’m not sold on the name has_join_relationship. Any suggestions?

Non-blocking ActiveRecord & Rails »

Created at: 15.04.2010 23:39, source: igvita.com, tagged: ruby Ruby on Rails activerecord eventmachine rails

Rails and MySQL go hand in hand. ActiveRecord is perfectly capable of using a number of different databases but MySQL is by far the most popular choice for production deployments. And therein lies a dirty secret: when it comes to performance and 'scalability' of the framework, the Ruby MySQL gem is a serious offender. The presence of the GIL means that concurrency is already somewhat of a myth in the Ruby VM, but the architecture of the driver makes the problem even worse. Let's take a look under the hood.

Dissecting Ruby MySQL drivers

The native mysql gem many of us use in production was designed to expose a blocking API: you issue a SQL query, and the library blocks until the server returns a response. So far so good, but unfortunately it also introduces a nasty side effect. Because it blocks inside of the native code (inside mysql_real_query() C function), the entire Ruby VM is frozen while we wait for the response. So, if you query happens to have taken several seconds, it means that no other block, fiber, or thread will be executed by the Ruby VM. Ever wondered why your threaded Mongrel server never really flexed its threaded muscle? Well, now you know.

Fortunately, the little known mysqlplus gem addresses the immediate problem. Instead of using a single blocking call, it forwards the query to the server, and then starts polling for the response. For the curious, there are also two implementations, one in pure Ruby with a select loop, and a native (C) one which uses rb_thread_select. The benefit? Well, now you can have multiple threads execute database queries without blocking the entire VM! In fact, with a little extra work, we can even get some concurrency out of ActiveRecord.

However, we could even drop threads in our quest for concurrency! Instead of making every thread poll on a socket, we could pass each of those sockets to a single event loop (EventMachine) library, and let it handle all the IO scheduling for us: gem install em-mysqlplus. Same API, in fact, it uses mysqlplus under the covers, but now every query has a callback for true non-blocking database access. Take a look at a few examples in the slides:

Non-blocking Rails with MySQL

Now we come around full circle. The downside of a true asynchronous library is that it requires callbacks, spaghetti code and a fully asynchronous stack. Thankfully, we already have Thin for our async app server, and with the introduction of Fibers in Ruby 1.9, we can wrap our asynchronous driver to behave just as if it had a blocking API.

So, we install em-mysqlplus, require em-synchrony to emulate the 'blocking api', implement an activerecord adapter, and we finally have a fully non-blocking ActiveRecord driver which we can drop into our Rails app! Well, almost, a few other modifications: Rails provides a threaded ConnectionPool, which we need to replace with a Fiber aware one, and finally, we need to disable the built in Mutex (hap tip to Mike Perham for doing all the dirty work for us). Now let's give it a try:

> widgets_controller.rb

class WidgetsController < ApplicationController
  def index
    Widget.find_by_sql("select sleep(1)")
    render :text => "Oh hai"
  end
end

thin -D start
ab -c 5 -n 10 http://127.0.0.1/widgets/

Server Software: thin
Server Hostname: 127.0.0.1
Server Port: 3000

Concurrency Level: 5
Time taken for tests: 2.210 seconds
Complete requests: 10
Requests per second: 4.53 [#/sec] (mean)

Our widget action simulates a blocking one-second query, we start up a single thin server, and run an ab test against it: 10 requests, with a max concurrency of 5. And as you can see, the test finishes in just slightly over 2 seconds!

Rails 3, Ruby 1.9 and Drizzle

By mid summer we will see production releases of Rails 3, Ruby 1.9, and Drizzle, and that convergence is worth paying attention to. Both Rails 3 and Ruby 1.9 offer raw performance improvements across the board. In the meantime, Drizzle already provides a fully async libdrizzle driver (talks to MySQL & Drizzle) which we could adopt to future proof our applications. Combine all three with a fibered ActiveRecord driver, an async application server such as Thin, and we could make some serious steps forward when it comes to performance of Rails: significantly lower memory footprint and much better performance across the board.

Announcing EncryptedCookieStore plugin for Rails 2.3 »

Created at: 14.04.2010 00:46, source: Phusion Corporate Blog, tagged: Ruby on Rails Software

EncryptedCookieStore is similar to Ruby on Rails’s CookieStore (it saves session data in a cookie), but it uses encryption so that people can’t read what’s in the session data. This makes it possible to store sensitive data in the session.

EncryptedCookieStore is written for Rails 2.3. Other versions of Rails have not been tested.

Note: This is not ThinkRelevance’s EncryptedCookieStore. In the Rails 2.0 days they wrote an EncryptedCookieStore, but it seems their repository had gone defunct and their source code lost. This EncryptedCookieStore is written from scratch by Phusion.

Source code at http://github.com/FooBarWidget/encrypted_cookie_store

Installation and usage

First, install it:

./script/plugin install git://github.com/FooBarWidget/encrypted_cookie_store.git

Then edit config/initializers/session_store.rb and set your session store to EncryptedCookieStore:

ActionController::Base.session_store = EncryptedCookieStore

You need to set a few session options before EncryptedCookieStore is usable. You must set all options that CookieStore needs, plus an encryption key that EncryptedCookieStore needs. In session_store.rb:

ActionController::Base.session = {
        # CookieStore options...
        :key            => '_session',     # Name of the cookie which contains the session data.
        :secret         => 'b4589cc9...',  # A secret string used to generate the checksum for
                                           # the session data. Must be longer than 64 characters
                                           # and be completely random.

        # EncryptedCookieStore options...
        :encryption_key => 'c306779f3...', # The encryption key. See below for notes.
}

The encryption key must be a hexadecimal string of exactly 32 bytes. It should be entirely random, because otherwise it can make the encryption weak.

You can generate a new encryption key by running rake secret:encryption_key. This command will output a random encryption key that you can then copy and paste into your environment.rb.

Operational details

Upon generating cookie data, EncryptedCookieStore generates a new, random initialization vector for encrypting the session data. This initialization vector is then encrypted with 128-bit AES in ECB mode. The session data is first protected with an HMAC to prevent tampering. The session data, along with the HMAC, are then encrypted using 256-bit AES in CFB mode with the generated initialization vector. This encrypted session data + HMAC are then stored, along with the encrypted initialization vector, into the cookie.

Upon unmarshalling the cookie data, EncryptedCookieStore decrypts the encrypted initialization vector and use that to decrypt the encrypted session data + HMAC. The decrypted session data is then verified against the HMAC.

The reason why HMAC verification occurs after decryption instead of before decryption is because we want to be able to detect changes to the encryption key and changes to the HMAC secret key, as well as migrations from CookieStore. Verifying after decryption allows us to automatically invalidate such old session cookies.

EncryptedCookieStore is quite fast: it is able to marshal and unmarshal a simple session object 5000 times in 8.7 seconds on a MacBook Pro with a 2.4 Ghz Intel Core 2 Duo (in battery mode). This is about 0.174 ms per marshal+unmarshal action. See rake benchmark in the EncryptedCookieStore sources for details.

EncryptedCookieStore vs other session stores

EncryptedCookieStore inherits all the benefits of CookieStore:

It works out of the box without the need to setup a seperate data store (e.g. database table, daemon, etc).
It does not require any maintenance. Old, stale sessions do not need to be manually cleaned up, as is the case with PStore and ActiveRecordStore.
Compared to MemCacheStore, EncryptedCookieStore can “hold” an infinite number of sessions at any time.
It can be scaled across multiple servers without any additional setup.
It is fast.
It is more secure than CookieStore because it allows you to store sensitive data in the session.

There are of course drawbacks as well:

It is prone to session replay attacks. These kind of attacks are explained in the Ruby on Rails Security Guide. Therefore you should never store anything along the lines of is_admin in the session.
You can store at most a little less than 4 KB of data in the session because that’s the size limit of a cookie. “A little less” because EncryptedCookieStore also stores a small amount of bookkeeping data in the cookie.
Although encryption makes it more secure than CookieStore, there’s still a chance that a bug in EncryptedCookieStore renders it insecure. We welcome everyone to audit this code. There’s also a chance that weaknesses in AES are found in the near future which render it insecure. If you are storing *really* sensitive information in the session, e.g. social security numbers, or plans for world domination, then you should consider using ActiveRecordStore or some other server-side store.

JRuby: Illegal Key Size error

If you get this error (and your code works with MRI)…

    Illegal key size

    [...]/vendor/plugins/encrypted_cookie_store/lib/encrypted_cookie_store.rb:62:in `marshal'

…then it probably means you don’t have the “unlimited strength” policy files installed for your JVM. Download and install them. You probably have the “strong” version if they are already there.

As a workaround, you can change the cipher type from 256-bit AES to 128-bit by
inserting the following in config/initializer/session_store.rb:

EncryptedCookieStore.data_cipher_type = 'aes-128-cfb'.freeze  # was 256

Please note that after changing to 128-bit AES, EncryptedCookieStore still requires a 32 bytes hexadecimal encryption key, although only half of the key is actually used.

Cucumber, meet Routes »

Created at: 07.04.2010 07:13, source: OnRails.org, tagged: Ruby on Rails Rails Tips cucumber

I’ve been loving Rails BDD with Cucumber for the past year or so — it helps me focus on the next required step to build a feature in my application, and better focus equals better development velocity. However, one thing I found tedious in starting with Cucumber was defining route matchers in paths.rb.

The stock path_to method is implemented as a case statement, allowing you to add a case for each path you want to recognize. This works, but leaves you feeling a bit un-DRY since you’re basically duplicating information in your routes.rb file.

Here’s a quick hack to paths.rb that lets you leverage your existing routes:

change

    else
      raise "Can't find mapping from \"#{page_name}\" to a path.\n" +
        "Now, go and add a mapping in #{__FILE__}"
    end

    else
      begin
        page_name =~ /the (.*) page/
        path_components = $1.split(/\s+/)
        self.send(path_components.push('path').join('_').to_sym)
      rescue Object => e
        raise "Can't find mapping from \"#{page_name}\" to a path.\n" +
          "Now, go and add a mapping in #{__FILE__}"
      end
    end

In your Cucumber steps, you can now use any named route that does not require a parameter. For example, users_path would become “the users page”, and new_product_path would become “the new product page”. As you add new resources, at least the index and new options should work out of the box — no further edits to paths.rb required!

UPDATE:

w00t!

This is now baked into cucumber-rails!

MWRC 2010 - Day 1 Live Video »

Created at: 11.03.2010 17:53, source: OnRails.org, tagged: Ruby on Rails ruby

The conference is about to start in 30 minutes, the room starts to buzz. The confreaks guys have their camera and video recording equipment all setup. So you will be able to catch up the conference online soon. Somehow I really like single track conferences and the sessions seem really great and will be fast passed, 30 to 45 minutes. Check out the schedule. So I will sit back and enjoy the show.

Follow it live on Justin TV!!

Watch live video from Mountain West Ruby Conference on Justin.tv

Salt Lake City is definitively a beautiful city, surrounded by it’s mountains…