bgror

Sidekiq – simple, efficient messaging for Rails »

Created at: 07.02.2012 09:10, source: Mike Perham, tagged: ruby

I introduced my latest project, Sidekiq, this morning. Sidekiq is the culmination of several years of research and development into message processing with Ruby. I first wrote a single-threaded mysql-backed queue called QBert for FiveRuns in 2008 before delayed_job was introduced. Next I wrote a single-threaded SQS-based processor called Jobber and then a Fiber-based, RabbitMQ-backed processor called Qanat for OneSpot. Most recently I wrote girl_friday, an Actor-based, redis-backed processor and a custom multi-threaded version of Resque for a Carbon Five client.

So yeah, messaging and I have a history but I think that’s because Ruby still doesn’t have a simple, scalable option for message processing. Sidekiq is my attempt to solve that need.

Simple

Sidekiq integrates tightly with Rails 3 so your application’s worker code is nearby and easy for Sidekiq to find and load. I’d like to support plain Ruby applications but I need a replacement for the nice integration Rails 3 gives me.

Efficient

Sidekiq uses multiple threads so you can process hundreds of messages in parallel if you desire without the overhead of hundreds of processes. Internally Sidekiq uses the Celluloid actor library to make threading easier and safer to manage.

Developer Friendly

I’ve tried to make Sidekiq developer friendly in two ways:

• Resque-compatible – I love conventions, why reinvent the wheel when you can stick closely to what someone else has done and leverage that work? I aim to make Sidekiq easy to integrate into an existing Resque project and monitorable via resque-ui.
• Middleware – Sidekiq pulls in a middleware API similar to Rack for running code before/after/around a processed message and ships with middleware that helps with ActiveRecord and Airbrake.

Licensing

Sidekiq is an experiment in one regard: licensing. I’ve decided to license Sidekiq under the GPLv3 for free or a commercial license for a $50 pledge at my Pledgie page. During my time as a Ruby consultant, $50 would get you less than 20 minutes of my time. I hope you think this is a reasonable price.

Please let me know how you like Sidekiq – I love hearing stories about how people are using it.

JRuby 1.6.6 is Released »

Created at: 30.01.2012 23:29, source: Engine Yard Blog, tagged: ruby

The JRuby community is pleased to announce the release of JRuby 1.6.6.

• Homepage: http://www.jruby.org/
• Download: http://www.jruby.org/download

The primary goal of the 1.6.x series is to round out our 1.9 support by fixing any reported incompatibilities. Of course, as with any JRuby release, we will continue fixing any found incompatibilities and also improve performance. All users of 1.6.5.1 (and lower) are encouraged to upgrade to 1.6.6.

Because master keeps getting further and further away from our 1.6 branch we have decided to make this our last 1.6 release. We largely fulfilled our goal of having reasonable 1.9 support. Follow up fixes for 1.9 support will only be fixed on master from this point forward. JRuby 1.7.0 will be the next release of JRuby.

Notable Changes:

• Updated stdlib to match Ruby 1.8.7p357 and 1.9.2p312
• Updated RubyGems to 1.8.15
• Multiple 1.9-mode yield/splat bugs fixed (pp, rspec 2.8 working again)
• Multiple 1.9-mode encoding bugs fixed
• Critical fixes in Random and Fiber
• Map Scala operator methods to symbolic names ($plus, etc)

1.6.5 Issues Resolved:

• JRUBY-6386 time.localtime not taking any arguments
• JRUBY-6384 yaml broken for last 1.6.6 build?
• JRUBY-6383 Scala integration breaks with 1.6.6
• JRUBY-6382 1.9: Padrino can’t generate an app
• JRUBY-6381 java.util.Collection#each dose not respect to_ary defined by objects that are iteratered
• JRUBY-6380 Original array is overwritten when select! is called on a copy
• JRUBY-6377 rspec .should include() fails in –1.9 mode
• JRUBY-6375 Uninformative YAML parser error
• JRUBY-6373 ThreadError: Mutex is not owned by calling thread, when interrupting thread using a Ruby Mutex
• JRUBY-6370 Regression in 1.6.6 in –1.9 mode
• JRUBY-6367 –pre command line switch not working in 1.9 runtime
• JRUBY-6366 More array splatting bugs in 1.9 mode
• JRUBY-6361 RbConfig reports wrong OS type on Solaris
• JRUBY-6359 Can’t convert nil to String building ActiveSupport RDoc in 1.9 mode
• JRUBY-6354 SyntaxError: (RegexpError) invalid multibyte escape in 1.9 mode in the 50th iteration
• JRUBY-6338 JRuby does not look for .jrubyrc in home directory on Windows
• JRUBY-6324 random seed for srand is not initialized properly
• JRUBY-6323 JRuby does not pay attention to either -U or LANG in determining encoding for ARGV (it is always ASCII-8BIT)
• JRUBY-6319 ‘binding’ returns wrong binding
• JRUBY-6318 Tempfile#open does not return the value of the block given to it
• JRUBY-6307 Powering operation of Integer sometimes gets a wrong calculation when 1.9 mode.
• JRUBY-6303 Cannot gem install from a remote repository in 1.9 mode
• JRUBY-6295 Dir.chdir, $HOME and $LOGDIR behavior
• JRUBY-6284 Calls to Kernel#exit result in an exception printed on stderr
• JRUBY-6282 Colon is not allowed in a file name on Windows
• JRUBY-6281 1.9: Applet does not work in the 1.9 mode
• JRUBY-6272 Encoding exception running JRuby 1.6.5 (1.8 mode)
• JRUBY-6233 jruby-complete-1.6.5.jar!/META-INF/jruby.home/bin/rake missing
• JRUBY-6227 1.9: Struct#members and Struct::members should return an Array of Symbols in 1.9
• JRUBY-6224 In MRI 1.9 the flag for Module#const_get also controls lookup of toplevel constants but not in JRuby
• JRUBY-6217 Coverage module not working with Rails ActiveRecord associations
• JRUBY-6214 Dir#rmdir raises improper exception if directory is not empty.
• JRUBY-6212 IO#inspect in 1.9 could be prettier
• JRUBY-6209 Hash#rehash does not work under some condition
• JRUBY-6208 bad gem file creation using mode –1.9
• JRUBY-6206 Incorrect SHA1 on two required packages in Maven Central
• JRUBY-6205 ‘Bad file descriptor’ when using IO.popen4 with OpenJDK 7
• JRUBY-6204 UTF-8 char in XML hangs in Joni
• JRUBY-6202 JIT-ed class names only use method names, causing collisions
• JRUBY-6201 File reading performance regression
• JRUBY-6200 1.9: Loading some Unicode characters results in non-printable characters on Windows
• JRUBY-6199 JRuby is hardcoded to use ‘-mmacos-version-min=10.4’ which is not compatible with ‘-rpath’ being used
• JRUBY-6198 When calling dup on file open in binmode the new object does not respect binmode
• JRUBY-6192 jruby::Handle declarations use ‘extern “C”’, causing linker symbol mismatches
• JRUBY-6182 Marshal.dump yields different value after adding/removing instance variables (and disagrees with MRI)
• JRUBY-6176 SecureRandom.uuid is not implemented
• JRUBY-6173 pp is broken in –1.9 mode
• JRUBY-6172 Requiring a file from a JAR that has a path inside the JAR that coincides with a path on the file system that includes a symlink fails
• JRUBY-6171 Enumerable does not splat
• JRUBY-6170 Fibers are broken in JRuby 1.6.5

And We’re Live! »

Created at: 26.01.2012 21:31, source: Engine Yard Blog, tagged: community ruby

The 2012 JRubyConf site is live, and we're really excited about it!

We’ll be using the site as a way to keep you up to date with conference developments and details as they unfold. This will be the fourth annual JRubyConf and definitely the biggest and best yet! Join us in talking about and celebrating the union of the Java and Ruby communities. Follow the news section of the site to find out more about sponsorships, speakers, after-events and a soon-to-follow call-for-proposals. Also, stay tuned for more information about Workshop Day (Monday the 21st) and the conference schedule (Tuesday & Wednesday the 22nd & 23rd).

This year’s JRubyConf will take place in Minneapolis, Minnesota, home to some of JRuby’s main contributors and biggest cheerleaders, at the Guthrie Theater on the banks of the mighty Mississippi.

Want to support our cause? We’re currently looking for sponsors for everything from snacks to after-parties, so check out our prospectus for more information, or email us at jrubyconf@engineyard.com if you have other cool ideas. We’d love to see your participation.

Want to join us? Buy your tickets now. Can’t wait to see you all there!

RVM Stable and More »

Created at: 24.01.2012 22:13, source: Engine Yard Blog, tagged: ruby Technology

$ bash -s stable < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer)

$ rvm get stable # the same as:
$ rvm get branch wayneeseguin/stable

$ rvm get branch mpapis/master

$ rvm info rvm
rvm:
version:      "rvm 1.10.0 by Wayne E. Seguin <wayneeseguin@gmail.com>, Michal Papis <mpapis@gmail.com> [https://rvm.beginrescueend.com/]"
updated:      "2 hours 52 minutes 49 seconds ago"

$ rvm get head
...
Upgrade Notes:
* If you see the following error message: Unknown alias name: 'default'
re-set your default ruby, this is due to a change in how default works.
...

$ rvm get head
...
Upgrade Notes:
* No new notes to display.
...

$ rvm get head
...
RVM reloaded!

Important Changes

$ command rvm use 1.9.3
RVM is not a function, selecting rubies with 'rvm use ...' will not work.
$ rvm use 1.9.3
Using /home/mpapis/.rvm/gems/ruby-1.9.3-p0

$ rvm alias create default <version>

$ rvm install 1.9.3

$ rvm reinstall 1.9.3

$ rvm install 1.9.3 --force

$ rvm get head --auto

$ rvm user [gemsets/all]

$ sudo rvm user [gemsets/all] --skel

$ rvm install rbx --19   # will install Rubunius with default mode set to 1.9
$ rvm install jruby --18 # will install JRuby with default mode set to 1.8

$ rvm install 1.9.3 --universal # to build fat binary including both 32 and 64 bit binaries
$ rvm install 1.8.7 --32 # to build only 32 bit ruby
$ rvm install 1.8.7 --with-arch=i386 # is equivalent to the 32 bit one, but is available only via RVM, ruby 1.8.7 sources do not support it.

$ rvm install 1.8.7 --32 -n 32 # will fail
$ rvm install 1.8.7 --32 -n n32 # will work
$ rvm install 1.8.7-n32 --32 # equivalent of the above

$ rvm list         # will show the colored list by default in terminal
$ rvm --color=force list | less -R         # will show the colored list in less
$ rvm_pretty_print_flag=auto rvm list | tee my-rubies.list         # will automatically disable colors
$ rvm --color=no list         # will always disable colors

Bundler and public applications »

Created at: 19.01.2012 23:24, source: Phusion Corporate Blog, tagged: ruby Ruby on Rails

I think Bundler is a great tool. Its strength lies not in its ability to install all the gems that you’ve specified, but in automatically figuring out a correct dependency graph so that nothing conflicts with each other, and in the fact that it gives you rock-solid guarantees that whatever gems you’re using in development is exactly what you get in production. No more weird gem version conflict errors.

This is awesome for most Ruby web apps that are meant to be used internally, e.g. things like Twitter, Basecamp, Union Station. Unfortunately, this strength also turns in a kind of weakness when it comes to public apps like Redmine and Juvia. These apps typically allow the user to choose their database driver through config/database.yml. However the driver must also be specified inside Gemfile, otherwise the app cannot load it. The result is that the user has to edit both database.yml and Gemfile, which introduces the following problems:

• The user may not necessarily be a Ruby programmer. The Gemfile will confuse him.
• The user is not able to use the Gemfile.lock that the developer has provided. This makes installing in deployment mode with the developer-provided Gemfile.lock impossible.

This can be worked around in a very messy form with groups. For example:

group :driver_sqlite do
  gem 'sqlite3'
end

group :driver_mysql do
  gem 'msyql'
end

group :driver_postgresql do
  gem 'pg'
end

And then, if the user chose to use MySQL:

bundle install --without='driver_postgresql driver_sqlite'

This is messy because you have to exclude all the things you don’t want. If the app supports 10 database drivers then the user has to put 9 drivers on the exclusion list.

How can we make this better? I propose supporting conditionals in the Gemfile language. For example:

condition :driver => 'sqlite' do
  gem 'sqlite3'
end

condition :driver => 'mysql' do
  gem 'mysql'
end

condition :driver => 'postgresql' do
  gem 'pg'
end

condition :driver => ['mysql', 'sqlite'] do
  gem 'foobar'
end

The following command would install the mysql and the foobar gems:

bundle install --condition driver=mysql

Bundler should enforce that the driver condition is set: if it’s not set then it should raise an error. To allow for the driver condition to not be set, the developer must explicitly define that the condition may be nil:

condition :driver => nil do
  gem 'null-database-driver'
end

Here, bundle install will install null-database-driver.

With this proposal, user installation instructions can be reduced to these steps:

1. Edit database.yml and specify a driver.
2. Run bundle install --condition driver=(driver name)

I’ve opened a ticket for this proposal. What do you think?