bgror

Modularized Association Methods in Rails 3.2 »

Created at: 20.01.2012 21:03, source: has_many :through, tagged: activerecord associations rails

Happy Friday! It's Rails 3.2 day! The official release announcement mentions a few of the big changes, but I'd like to take a moment to highlight a relatively small change I was responsible for, one that I hope may make your life a little easier.

From the ActiveRecord CHANGELOG:

Generated association methods are created within a separate module to allow overriding and
composition using `super`. For a class named `MyModel`, the module is named
`MyModel::GeneratedFeatureMethods`. It is included into the model class immediately after
the `generated_attributes_methods` module defined in ActiveModel, so association methods
override attribute methods of the same name. *Josh Susser*

The point of this change is to allow more flexibility in working with associations in your model classes. When you define an association, ActiveRecord automagically generates some methods for you to work with the association. For example, a has_many :patches association generates the methods patches and patches= (and a few others).

Previously, those association methods were inserted directly into your model class. This change moves those methods into their own module which is then included in your model class. Your model gets the same methods through inheritance, but also gets to override those methods and still call them using super. Let's take a look at two ways this makes things easier for you.

Sometimes you want to replace the standard generated association methods. That's always been easy to do simply by defining new methods in your model class. The only wrinkle was that you had to make sure you defined your method after you set up the association, or calling has_many would overwrite your method, since last writer wins. That was usually not a problem, but sometimes plugins or other monkey patching extensions could add an association after your model's class was defined, which wouldn't give you a chance to add your method afterwards. With this change, you don't have to worry about those order dependencies anymore. Since those methods are generated in their own module, the order doesn't matter. This is a pretty small issue all told and I doubt it affected many people, but it's still worth mentioning.

The real reason for this change is being able to compose your own methods with the standard generated methods. Before this change, you'd have to use alias_method_chain or some other fancy footwork to layer your own logic on top of the standard association functionality. Either that or you'd have to somehow duplicate the standard behavior in your own method. Ick. Now you can compose methods using inheritance and super, the way Alan Kay intended you to. Here's the example from the docs:

class Car < ActiveRecord::Base
  belongs_to :owner
  belongs_to :old_owner

  def owner=(new_owner)
    self.old_owner = self.owner
    super
  end
end

If you're familiar with ActiveRecord it's probably fairly obvious what's going on there, but I'll spell it out for the new kids. When you define the belongs_to :owner association, that generates a standard owner= method, and puts it in the module named Car::GeneratedFeatureMethods, which is the closest ancestor of class Car. If you're curious what this looks like, fire up the rails console and type Car.ancestors to see the class's inheritance chain. (Or use your own app and model, since that will be much easier than making up a new app just to see that one thing.)

In this Car class, you can see that changing owners keeps track of the old owner, so the new owner knows who to call when he can't figure out how to open the trunk. The generated owner= method does a fair amount of stuff including managing counter caches, running callbacks, setting inverse associations, etc. Skipping that could break a number of things, so after saving the old owner, you also want to run the generated method. Since it's in a module that Car inherits from, you only have to call super to get that to run. No muss, no fuss!

One more step towards simpler OOP in Rails! Thanks to my fellow Ruby Rogues Avdi Grimm and James Edward Gray II for complaining about the old state of things enough to motivate me to finally go fix this.

Non-blocking ActiveRecord & Rails »

Created at: 15.04.2010 23:39, source: igvita.com, tagged: ruby Ruby on Rails activerecord eventmachine rails

Rails and MySQL go hand in hand. ActiveRecord is perfectly capable of using a number of different databases but MySQL is by far the most popular choice for production deployments. And therein lies a dirty secret: when it comes to performance and 'scalability' of the framework, the Ruby MySQL gem is a serious offender. The presence of the GIL means that concurrency is already somewhat of a myth in the Ruby VM, but the architecture of the driver makes the problem even worse. Let's take a look under the hood.

Dissecting Ruby MySQL drivers

The native mysql gem many of us use in production was designed to expose a blocking API: you issue a SQL query, and the library blocks until the server returns a response. So far so good, but unfortunately it also introduces a nasty side effect. Because it blocks inside of the native code (inside mysql_real_query() C function), the entire Ruby VM is frozen while we wait for the response. So, if you query happens to have taken several seconds, it means that no other block, fiber, or thread will be executed by the Ruby VM. Ever wondered why your threaded Mongrel server never really flexed its threaded muscle? Well, now you know.

Fortunately, the little known mysqlplus gem addresses the immediate problem. Instead of using a single blocking call, it forwards the query to the server, and then starts polling for the response. For the curious, there are also two implementations, one in pure Ruby with a select loop, and a native (C) one which uses rb_thread_select. The benefit? Well, now you can have multiple threads execute database queries without blocking the entire VM! In fact, with a little extra work, we can even get some concurrency out of ActiveRecord.

However, we could even drop threads in our quest for concurrency! Instead of making every thread poll on a socket, we could pass each of those sockets to a single event loop (EventMachine) library, and let it handle all the IO scheduling for us: gem install em-mysqlplus. Same API, in fact, it uses mysqlplus under the covers, but now every query has a callback for true non-blocking database access. Take a look at a few examples in the slides:

Non-blocking Rails with MySQL

Now we come around full circle. The downside of a true asynchronous library is that it requires callbacks, spaghetti code and a fully asynchronous stack. Thankfully, we already have Thin for our async app server, and with the introduction of Fibers in Ruby 1.9, we can wrap our asynchronous driver to behave just as if it had a blocking API.

So, we install em-mysqlplus, require em-synchrony to emulate the 'blocking api', implement an activerecord adapter, and we finally have a fully non-blocking ActiveRecord driver which we can drop into our Rails app! Well, almost, a few other modifications: Rails provides a threaded ConnectionPool, which we need to replace with a Fiber aware one, and finally, we need to disable the built in Mutex (hap tip to Mike Perham for doing all the dirty work for us). Now let's give it a try:

> widgets_controller.rb

class WidgetsController < ApplicationController
  def index
    Widget.find_by_sql("select sleep(1)")
    render :text => "Oh hai"
  end
end
 

thin -D start
ab -c 5 -n 10 http://127.0.0.1/widgets/

Server Software: thin
Server Hostname: 127.0.0.1
Server Port: 3000

Concurrency Level: 5
Time taken for tests: 2.210 seconds
Complete requests: 10
Requests per second: 4.53 [#/sec] (mean)

Our widget action simulates a blocking one-second query, we start up a single thin server, and run an ab test against it: 10 requests, with a max concurrency of 5. And as you can see, the test finishes in just slightly over 2 seconds!

Rails 3, Ruby 1.9 and Drizzle

By mid summer we will see production releases of Rails 3, Ruby 1.9, and Drizzle, and that convergence is worth paying attention to. Both Rails 3 and Ruby 1.9 offer raw performance improvements across the board. In the meantime, Drizzle already provides a fully async libdrizzle driver (talks to MySQL & Drizzle) which we could adopt to future proof our applications. Combine all three with a fibered ActiveRecord driver, an async application server such as Thin, and we could make some serious steps forward when it comes to performance of Rails: significantly lower memory footprint and much better performance across the board.

Rails and Merb Merge: ORM Agnosticism (Part 5 of 6) »

Created at: 23.02.2010 20:15, source: Engine Yard Blog, tagged: Technology ActionPack ActiveModel activerecord api datamapper haml Rails 3 Railties

It’s been a little while since I’ve last posted in this series. During that time, we released Rails 3.0 beta, and announced the launch of RailsPlugins.org. Plugin authors have registered almost 150 plugins, with 40 already boasting compatibility with Rails 3. Over the next few weeks, we’re going to roll out some more features to help users find projects to help get updated, so keep an eye out.

Today I’m going to talk about the features we added to Rails 3 to provide ORM agnosticism. When we first started, the idea of such agnosticism was pretty fuzzy. As we approached the beta, we became convinced that we would shoot for the moon and give DataMapper, Sequel and other ORMs first-class access to all of the same parts of the framework that ActiveRecord had. Again, this sounds pretty fuzzy, so let me lay it out for you.

ActiveRecord is a Framework Extension

You read that right. While Rails itself ships with ActiveRecord as a default, the Railties gem, responsible for bootstrapping Rails, knows nothing about ActiveRecord. In order to achieve this, we had to make that bootstrapping process much more pluggable. While this is amazing for extensions, like DataMapper, that want to replace a built-in framework, it also exposes more functionality to any Rails extension distributed as a gem.

Rails::Railtie

In short, Railties now coordinates the initialization process, which involves the activation of a number of individual Railtie subclasses. Cute, no? Every Rails framework has its own Railtie subclass, which provides a number of useful pieces of functionality:

• The ability to add a key (like config.active_record) to an Application’s configuration, and assign it defaults that will exist before Application configuration. For instance, a plugin could use this functionality to set a configuration key as an Array, so the Application could simply do config.action_view.view_paths << "my_path". While this isn’t exactly earth-shattering, it levels the playing field between Rails components, like ActiveRecord, and third-party extensions, like DataMapper or Haml
• The ability to create additional generators that hook into Rails’ default generators. For instance, Rails’ scaffold generator invokes sub-generators for stylesheets, template engine, test framework, helpers, and ORM. For instance, when a user installs Haml as a gem, it can register itself as the handler for template engine generators, and provide replacements for each case where Rails provides a default implementation. In this case, Haml would replace the template engine generators provide by ActionController’s Railtie
• The ability to supply Rake tasks to load when the user invokes the Rake command. If DataMapper supplies the same named Rake tasks as those supplied by the ActiveRecord Railtie, the user can completely remove ActiveRecord and replace it with DataMapper, retaining all of the named Rake tasks. This would allow other tasks, such as those used in testing, to automatically prepare the database and other tasks it performs by invoking tasks with certain names (like db:create)
• Supply a log subscriber to integrate seamlessly with the uniform request logging. This allows extensions like DataMapper to add their timings into the log output. In this case, DataMapper simply replaces the “Model” timing that ActiveRecord provides, allowing a level of very tight integration with the rest of the framework
• Specify initializers that should run at specific points in the initialization process. This allows extensions to set things up very early in the boot process, but then defer some setup until after the user has configured their application, or after specific parts of the initialization process have occurred. Each initializer also receives an instance of the Application object, giving it full access to the user’s configuration

In fact, if you take a look at ActiveRecord’s deep integration with the rest of the framework (which is roughly equivalent to its highly coupled implementation in Rails 2.3), you’ll find that it looks extremely similar to DataMapper’s equally deep integration. In fact, a major goal of the work we put into improving modularity in Rails 3 was to maintain the same level of stack integration we’ve historically had, while exposing the same toolchain for those who wanted to replace only certain elements of the stack.

The ActiveModel API

Giving DataMapper access to the same level of integration with Railties as ActiveRecord is one of two major pillars in making Rails truly ORM agnostic. The second part is decoupling Rails’ historic connection between ActiveRecord, its ORM, and ActionPack, its controller and view layer. In this area, Rails takes a holistic, very conventional approach to linking the model and the view using REST principles.

In particular, domain objects, persisted via the ORM, can have canonical URLs that are used pervasively throughout the controller and view layer. The specific URLs can be configured via the router, and Rails 3’s router is particularly powerful, but once configured, objects do have canonical URLs.

@post = Post.first
 
redirect_to @post           #=> Location: /posts/orm-agnositicism
redirect_to Post            #=> Location: /posts
 
form_for @post              #=> <form action="/posts/orm-agnosticism" method="PUT">
form_for Post.new           #=>
 
error_messages_for @post    #=> a representation of the validation errors that exist on the object
 
link_to @post.title, @post  #=> <a href="/posts/orm-agnosticism">
                            #     Rails and Merb Merge: ORM Agnosticism (Part 5 of 6)
                            #   </a>
 
posts_path                  #=> "/posts"
post_path(@post)            #=> "/posts/orm-agnosticism"
 
@posts = Post.where(:author => "wycats")
 
render @post                #=> renders "_post.html.erb", passing in @post as a local
render @posts               #=> renders a collection of "_post.html.erb", passing in each
                            #   Post instance as a local in turn
 
respond_with @post          #=> performs content negotiation between the formats
                            #   the client is willing to Accept (xml, json, html),
                            #   and the formats the @post object is willing to
                            #   provide. The available providable formats are
                            #   transparently determined by introspecting the @post
                            #   object. Additionally, if the object has not been persisted,
                            #   redirect it back to the editing page for further modification.
                            #   If it has been persisted, redirect it back to the index (or
                            #   some other appropriate path).</form>

In short, Rails 2.3 provided a fair bit of integrated functionality between ActiveRecord and ActionPack. For instance, to determine what an object’s canonical URL was, Rails called an internal .class.naming method on it, which provided the information needed to form the canonical URLs. And while other ORMs could attempt to emulate these internals, there was no guarantee that they’d be used in the same way from release to release (and, in fact, they evolved quite a bit, making them a moving target at best).

In Rails 3, we opened them up, exposing an explicit public API via ActiveModel. The ActiveModel API requires that objects respond to to_model, returning an Object that complies with the larger ActiveModel API. This allows objects that already implement methods with the same names as those required by the API to create a facade that presents itself to ActionPack as a fully compliant model.

The API itself is fairly small, with a few methods around validation, the ability to determine whether an object has persisted or not (which can be safely stubbed by objects without persistence), and a number of methods that tell ActionPack how to convert the object into a canonical URL or template name. By doing this, Rails has completely decoupled ActionPack from ActiveRecord directly, and ActiveRecord becomes just one of many ORMs to implement the ActiveModel API.

The DataMapper ORM has already released a Rails extension (called dm-rails) that implements both the Railtie functionality and ActiveModel, making it a full drop-in replacement for ActiveRecord. The Sequel ORM is also hard at work on their Railtie and ActiveModel adapter.

The really great thing about all of this is that we didn’t need to sacrifice one bit of the (some have said overly) tight integration between various parts of Rails, to provide the ability to swap in entirely different components with the same apparent “coupling” that the Rails frameworks have with each other.

As always, questions and comments are welcome!

Let Them Code Cake! »

Created at: 09.02.2010 20:00, source: Engine Yard Blog, tagged: Technology activerecord cake datamapper refactoring

In TDD and BDD we write small, focused, technical tests, sometimes called micro-tests. One of the core ideas is that these tests should run fast, really fast—each one measured in milliseconds. If you’re writing plain Ruby code, that’s pretty easy to accomplish. However, when you’re using something like Rails or Merb and DataMapper or ActiveRecord, it can get a bit more challenging.

Why do we end up with model tests/specs that run slowly? In looking at how we write and test our code, let’s ignore controllers and views, and focus on models. After all, models are where you put your business logic.

There can be several reasons for a slow test suite: database access, unnecessary objects, a massive setup that takes a while to load—just to name a few. This time though, I want us to focus on a new reason, one that’s a result of operating in the context of ActiveRecord or DataMapper. One that tricks us into thinking we’re doing well. One that exposes a few somewhat major flaws in the design of AR and DM.

With ActiveRecord and DataMapper, each model class is responsible for its own persistence, hence DHH using the name “Active Record.” It was inspired by Fowler’s writing on the pattern in Patterns of Enterprise Application Architecture and is the enabler of our model addiction.

OverActive Record

Used properly, the active record pattern is great for persisting data. These objects are great at handling their own persistence. Accessors, associations, and validation make a good wrapper around the transactional nature of a database. This is fine when the model is a simple data object, but we all run into problems when we start adding other behavior to the model class.

Weighted Companion Cake: The cube was the cake!

(Compliments to flickr user Mandrake88)

Recall the Single Responsibility Principle: “There should never be more than one reason for a class to change.” As we commonly use them, ActiveRecord and DataMapper¹ classes almost always violate this principle. There are usually at least two responsibilities handled by every AR/DM model: persistence and business logic. Carrying around that persistence behavior, and all the dependencies that go along with it, is what bloats and slows down our specs.

There have been various attempts at dealing with this: in memory databases, stubbing parts of the DM/AR frameworks, etc. None of these are ideal. Either they don’t speed things up as much as we’d like, they’re awkward, or they bulk up the specs an objectionable amount.

Is There a Better Way? Why, Yes! Yes There Is.

DM/AR lets you define a model’s properties, its persistent parts, and generates the associated accessors and mutators. All access to the persistent properties is handled through these accessors and mutators. This gives us a perfect seam along which to split the class. All of the persistence-dependent functionality is on one side of this seam, and the business logic of the model is on the other.

We need a good saving throw against this frosting.

Courtesy of Mike’s Amazing Cakes

Pretend… It’s a Cake.

Yes, you read that correctly. Keep on reading—it makes sense!

Each of our models is like a frosted cake. We have the nice fluffy cake batter, baked with our properties, validations, and persistence. Then we frost the outside with our business logic. We can take a simple ActiveRecord cake and make it into almost anything, with a good amount of frosting.

Now herein lies the problem: the frosting starts to take on a life of its own. Before we know it we’re baking a few 100 ActiveRecord cakes each time we need to test the frosting behavior… There are two really good ways to get around this: cupcakes and cardboard.

Strategy 1: Oh Look! Cupcakes!

When we build a web application with ActiveRecord or DataMapper, it’s really hard to think outside the persistence box. If something is persisted on that model, or if it uses something on that model, we automatically add the code to the model. This leads quickly to a gigantic cake that does everything.

This is especially bad with primary models like User. The poor bloated User gets tons of code stuffed into it because our app is usually focused around users. Users do things in our app. Users get affected by things. They control the app and most things belong to the user in some way. But this doesn’t mean our User class should be 1000 lines.

This cake does too much (yes that’s cake)

The solution is to break out chunks of code into their own classes. There are many common patterns that we can find in our classes: Factory, Provider, Policy, Strategy, and plenty of types that are custom to each domain (Cake Decorator?). Let’s lump these all into something that we’ll call Cupcake Classes.

At Engine Yard, one of the places we found an opportunity for a Cupcake was our Deployment initialize. The method had to do a lot of data mutating before it could create an instance. Blank strings needed to become nil, string representations of a type needed to be converted to the actual type, and defaults needed to be applied to fill out the incoming data. The initialize method was a ridiculous 50 lines long. This needed to become its own class.

The result is a really clean initialize:

  def initialize(params={})
    merger = Deployment::DefaultsMerger.new(params)
    merger.each_attribute { |att, val| send("#{att}=", val) }
  end

…and a class that knows all about how to merge incoming params and defaults. It has a clear API, and it’s easily testable. We know what comes in and what comes out and we don’t have to create a database record every time we test it.

class Deployment
  class DefaultsMerger

    def initialize(params)
      @attributes = merge_with_defaults(params)
      scrub_blank 0, :data_volume_size, :app_server_count
      scrub_blank nil, :db_volume_id, :data_volume_id
      coerce_value(:instance_size, :db_instance_size) do |val|
        Instance::Size.coerce(val)
      end
      # ...
    end

    def each_attribute
      @attributes.each do |k,v|
        yield k, safe_to_i(v)
      end
    end

  private

    def defaults
      Mash.new({
        :data_volume_size => Volume::DEFAULT_SIZE,
        :app_server_count => 1,
        # ...
      })
    end
    # ...
  end
end

Cupcake Coding Rule of Thumb: Messy Code Makes for Good Cupcakes!

When you want to clean up your models, look for messy code. Long methods or sets of utility methods (methods that barely, if at all, touch instance state) are just begging to be cut up into smaller classes.

What tipped us off was that the initialize method was long, nasty, and really had nothing to do with the class it was in. Mostly, it dealt with params, a hash. Almost every line in the old method did hash operations, not operations dealing with the actual model.

We clearly needed a class that would know how to work with merging our defaults into an incoming hash. The resulting class would also improve our real test coverage. It’s much easier to test a bunch of branches when they don’t have to pass through the model life cycle.

describe Deployment::DefaultsMerger do
  describe_attribute :data_volume_size do
    it_defaults_to Volume::DEFAULT_SIZE
    it_converts ["", nil, 0, "0"], :to => 0
    it_converts [123, "123"],      :to => 123
  end

  describe_attribute :app_server_count do
    it_defaults_to 1
    it_converts ["", nil, 0, "0"], :to => 0
    it_converts [2, "2"],          :to => 2
  end
  # ...
end

With a focused class, it’s easy to write focused specs. With a few custom methods in your specs, you have beautifully concise descriptions of behavior to match your simple code.

Looks yummy!

Thanks to flickr user hazellterry

Strategy 2: Cardboard Cake

The next way to improve your models and your testing is to extract the model’s behavior from the persistence. The idea is to test the frosting without having to bake a cake every time. Our example class after the behavior has been extracted:

class Oven
  include DataMapper::Resource
  include Oven::Behavior

  ## Properties

  property :id,           Serial
  property :oven_type,    String
  property :temp_setting, String

  has 1, :heating_element
  has 1, :latch
  ## ...

end

Here’s the frosting, or behavior, that has been extracted from it:

class Oven
  module Behavior

    def check_lock
      latch.engage! if dangerous_temperature?
    end

    def dangerous_temperature?
      Oven::Temperatures.dangerous? temp_setting
    end

    def turn_on(temp = 350) # ...
    def turn_off # ...
    def preheating? # ...
  end
end

Once this is done, you can write specs that focus on the logic in isolation. Instead of testing every aspect of behavior on top of the persistence framework (slow), you can isolate the behavior (fast). Sort of like making the cake out of cardboard so you can test our frosting without the baking time (See? Told you it would make sense!).

How Do We Achieve This Isolation? Mock Object Trickery, But of Course!

Let’s create a mock to stand in for the model object being tested. Let’s call these Cake Mocks. Then, we’ll extend the mock object with with the Behavior we’re testing.

  @oven = mock("Oven").extend(Oven::Behavior)

It still acts like an oven, but instead we’ve frosted the outside of a cardboard cake with the same business logic that we would add to a real ActiveRecord object. The trick is to have the mock object extend the module that contains the business logic. The result is an object containing the behavior of the model but not its persistence.

Now we can stub accessors on the mock (but only those involved in the example) and set expectations on the mutators. Then the example can call methods on the mock oven as if it was a real Oven instance. Those calls invoke the actual business logic methods that we want to test. When those methods access the internal behavior of our persistence layer, our mock answers with the methods we stubbed. The mock merely fills in the center of the cake.

describe Oven::Behavior do
  describe "preheating" do
    before(:each) do
      @latch = mock("latch")
      @oven = mock("Oven", :latch => @latch).extend(Oven::Behavior)
    end

    it "isn't locked when temperature setting is less than 500" do
      @oven.stub!(:temp_setting).and_return(350)
      @latch.should_receive(:engage!).never
      @oven.check_lock
    end

    it "isn't locked when temperature setting is 500" do
      @oven.stub!(:temp_setting).and_return(500)
      @latch.should_receive(:engage!).never
      @oven.check_lock
    end

    it "is locked when temperature setting is greater than 500" do
      @oven.stub!(:temp_setting).and_return(505)
      @latch.should_receive(:engage!).once
      @oven.check_lock
    end
  end
end

This cake mock style encourages another good testing practice: integration testing. When the model is completely mixed up with behavior, the need for integration testing is easier to ignore, but no less important. With separated behavior and persistence we can write simple integration tests that make sure the seam between the cake and the frosting is still intact.

Give Your Models Some Attention

When looking at models this way, whether you’re breaking them up in to cupcakes or separating the frosting, it’s easier to see the difference between behavior and persistence. If the logic can stand on its own, make a new class. If it’s more like an extension of the model, make a module and mix it back in. Allow ActiveRecord and DataMapper models to take care of what they do best without all the extra weight.

Take a few minutes and look at the models in your project. If the classes weren’t backed by the database, would you still have all that stuff jammed into one place? Are your models more complex than they need to be? If so… do something about it!

Rails and Merb Merge: Rails Core (Part 4 of 6) »

Created at: 22.01.2010 22:00, source: Engine Yard Blog, tagged: Uncategorized activerecord plugins rails Rails 3

After working to make Rails faster and more modular, we took a look at the glue code holding Rails together: Railties. Railties started life as a relatively modest piece of code in the early days of Rails, but it eventually grew to encompass quite a few different areas. For instance, Rails 2.3 included quite a bit of code for managing plugins, and some additional code for managing gems. Adding in explicit support for engines meant yet more code.

In isolation, each of these pieces of code made a lot of sense as they were added, but as the pieces accumulated, plugin authors, and even Rails itself, had to often resort to contortions to get the job done. Our goal in Rails 3 was to solve these problems.

An Application Object

The first thing we addressed was that a Rails 2.3 application had pieces spread out in a number of places. For example, it stored the configuration on a global configuration object, stored the routes on a global router object, and booted up the server and console from this collection of global state. It also looked up plugins using PluginLocators and PluginLoaders. While all of this worked, we found it difficult to reason about and it was easy to trip over when implementing new features. We wanted to unify these pieces.

Second, we wanted to make it possible to run multiple applications in a single process. This meant we had to remove as much global state as possible. We will likely be continuing to work toward this goal over the next several releases, but the nice thing is that removing global state has already resulted in improvements in code quality and we have found that it makes code easier to understand.

If you take a look at a newly generated Rails 3 app, you now find a named application.

module YourApp
  class Application < Rails::Application
    # Settings in config/environments/* take precedence over those specified here.
    # Application configuration should go into files in config/initializers
    # -- all .rb files in that directory are automatically loaded.
 
    # Add additional load paths for your own custom dirs
    # config.load_paths += %W( #{config.root}/extras )
 
    # Only load the plugins named here, in the order given (default is alphabetical).
    # :all can be used as a placeholder for all plugins not explicitly named
    # config.plugins = [ :exception_notification, :ssl_requirement, :all ]
 
    # Activate observers that should always be running
    # config.active_record.observers = :cacher, :garbage_collector, :forum_observer
 
    # Set Time.zone default to the specified zone and make Active Record auto-convert to this zone.
    # Run "rake -D time" for a list of tasks for finding time zone names. Default is UTC.
    # config.time_zone = 'Central Time (US & Canada)'
 
    # The default locale is :en and all translations from config/locales/*.rb,yml are auto loaded.
    # config.i18n.load_path += Dir[Rails.root.join('my', 'locales', '*.{rb,yml}')]
    # config.i18n.default_locale = :de
 
    # Configure generators values. Many other options are available, be sure to check the documentation.
    # config.generators do |g|
    #   g.orm             :active_record
    #   g.template_engine :erb
    #   g.test_framework  :test_unit, :fixture => true
    # end
  end
end

This looks a whole lot like the initializer block in Rails 2.3, but the Application object, not a global configuration object, now has the configuration information. Similarly, routes.rb now begins YourApp::Application.routes.draw. The application object lives in the center of Railties in Rails 3, providing a core object that the rest of the frameworks can build on.

Improving Initialization

Rails 2.3 initialized a Rails application by calling a number of methods in sequence. Correct operation relied heavily on the order of those methods, and Rails ran some methods multiple times (apparently to hack around long-forgotten issues). As a next step towards getting a grip on Railties, we refactored the initializers so that Rails declared each initializer separately in a way that users could easily hook into.

This idea originated in Merb, where the framework defined each initializer as a separate class. Merb also provided a mechanism for declaring in the class body, that the framework should run a particular initializer before or after another one. By breaking the initializers up into reusable chunks, we provided a convenient API for extensions to hook in at whatever part of the process they wanted to.

In Rails 3, we have defined a slightly different API, inspired by the same ideas.

initializer :load_all_active_support do
  require "active_support/all" unless config.active_support.bare
end
 
# Set the <tt>$LOAD_PATH</tt> based on the value of
# Configuration#load_paths. Duplicates are removed.
initializer :set_load_path do
  config.paths.add_to_load_path
  $LOAD_PATH.uniq!
end

These first initializers in the Rails bootup process require active_support/all and set the load paths. Each initializer has a name, which allows other initializers (in plugins) to hook in at the appropriate time. Initializers also have access to the configuration object.

Making initializers their own isolated chunks definitely improves the code, and exposed a number of issues in the existing initializers. With initializers now living on the brand new application object, we then took a hard look at plugins.

Unifying Plugins

We wanted to unify plugins into one coherent system. Rails 2.3 supported several different kinds of plugins: vendor/plugins, gem plugins, vendor/plugins engines, and gem engines.

Rails 2.3 had a number of objects to find and load each kind of plugin, that resulted in a certain amount of magical behavior in each case. For instance, it automatically ran init.rb for plugins in vendor/plugins at the appropriate time. For Rails 3, we have created a new class, called Rails::Plugin, that can itself set configuration and have initializers. Rails 3 now merges in those initializers, giving plugins the ability to directly participate in the initialization process.

Each plugin in vendor/plugins automatically become a Rails::Plugin, but we also allow gems to explicitly create a subclass and add their own initialization. This makes plugins more powerful, giving them the ability to hook in at more precise parts of the process.

initializer :load_init_rb, :before => :load_application_initializers do |app|
  file   = "#{@path}/init.rb"
  config = app.config
  eval File.read(file), binding, file if File.file?(file)
end

Here, vendor/plugins defines an initializer called :load_init_rb, which runs before the application initializers. The block receives the application object, so it can take a look at the configuration, or other parts of the application in question.

Railties (Or, How Frameworks Became Plugins)

After making all of this progress, we revisited an earlier mission set out at the beginning of the merge: make all frameworks, like ActiveRecord and ActionController, behave like regular plugins. This would allow other plugins, like DataMapper, to use exactly the same APIs used by ActiveRecord. By definition, other projects could then replace all the functionality in ActiveRecord in a self-contained gem.

After moving each initializer into its own block, we noticed that a large number of the initializers were wrapped in if config.frameworks.include?(:active_record) and the like. If we fixed this, we could make good on our promise to make each framework a plugin. So now, instead of having Railties inspect the list of available frameworks and run appropriate initializers, the frameworks themselves add their own initializers, but only when loaded. When the user loads ActiveRecord, its plugin adds its initializers; when the user loads DataMapper, its plugin adds its initializers.

The monolithic Railties framework is now a set of individual Railties, each inheriting from a new class, Rails::Railtie. Railties has more power than simple plugins. For instance, a Railtie’s name also serves as its key in the configuration object. And a Railtie can also specify rake tasks. For instance, if the user chooses DataMapper instead of ActiveRecord, he only sees DataMapper’s rake tasks.

Because Railties merely specifies how Rails initializes them, requiring them has no side-effects. Taking a look at the ActiveRecord Railtie, you can see that the file first requires “rails” and “action_controller/railtie”, ensuring that Rails runs the ActionController initializers before the ActiveRecord ones. It also allows ActiveRecord to define initializers that run before or after ActionController initializers.

Exposing an API to allow ActiveRecord to run decoupled from Railties forced us to enable functionality for third parties. For instance, because Rails loads Railties before initialization, this allows third-party plugins to hook into the very earliest stages of the process. By making the keys on config more general, we now can have config.data_mapper as well as config.active_record. By moving rake tasks and generators to the frameworks, and triggering them through the Railtie, Rails guarantees that the user can remove all artifacts of these frameworks by installing an alternative plugin. It also means that any plugin can add these artifacts.

Most usefully, it moves everything about how Rails should initialize ActiveRecord in a different context, such as booting up an app, running rake tasks, or running generators, into a single location that we can more easily maintain and that others can more easily copy. We think this will result in people being able to easily write Rails extensions that behave similarly to builtin Rails frameworks.

In short, Rails frameworks are now plugins to Rails.