Ruby on Rails Planet

Ruby & WebSockets: TCP for the Browser »

Created at: 22.12.2009 18:39, source: igvita.com, tagged: Architecture html5 realtime websocket

WebSockets are one of the most underappreciated innovations in HTML5. Unlike local storage, canvas, web workers, or even video playback, the benefits of the WebSocket API are not immediately apparent to the end user. In fact, over the course of the past decade we have invented a dozen technologies to solve the problem of asynchronous and bi-directional communication between the browser and the server: AJAX, Comet & HTTP Streaming, BOSH, ReverseHTTP, WebHooks & PubSubHubbub, and Flash sockets amongst many others. Having said that, it does not take much experience with any of the above to realize that each has a weak spot and none solve the fundamental problem: web-browsers of yesterday were not designed for bi-directional communication.

WebSockets in HTML5 change all of that as they were designed from the ground up to be data agnostic (binary or text) with support for full-duplex communication. WebSockets are TCP for the web-browser. Unlike BOSH or equivalents, they require only a single connection, which translates into much better resource utilization for both the server and the client. Likewise, WebSockets are proxy and firewall aware, can operate over SSL and leverage the HTTP channel to accomplish all of the above - your existing load balancers, proxies and routers will work just fine.

WebSockets in the Browser: Chrome, Firefox & Safari

The WebSocket API is still a draft, but the developers of our favorite browsers have already implemented much of the functionality. Chrome’s developer build (4.0.249.0) now officially supports the API and has it enabled by default. Webkit nightly builds also support WebSockets, and Firefox has an outstanding patch under review. In other words, while mainstream adoption is still on the horizon, as developers we can start thinking about much improved architectures that WebSockets enable. A minimal example with the help of jQuery:

> websocket.html

<html>
  <head>
    <script src='http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js'></script>
    <script>
      $(document).ready(function(){
        function debug(str){ $("#debug").append("<p>"+str+"</p>"); };
 
        ws = new WebSocket("ws://yourservice.com/websocket");
        ws.onmessage = function(evt) { $("#msg").append("<p>"+evt.data+"</p>"); };
        ws.onclose = function() { debug("socket closed"); };
        ws.onopen = function() {
          debug("connected...");
          ws.send("hello server");
        };
      });
    </script>
  </head>
  <body>
    <div id="debug"></div>
    <div id="msg"></div>
  </body>
</html>

The above example showcases the bi-directional nature of WebSockets: send pushes data to the server, and onmessage callback is invoked anytime the server pushes data to the client. No need for long-polling, HTTP header overhead, or juggling multiple connections. In fact, you could even deploy the WebSocket API today without waiting for the browser adoption by using a Flash socket as an intermediate step: web-socket-js.

Streaming Data to WebSocket Clients

WebSockets are not the same as raw TCP sockets and for a good reason. While it may seem tempting to be able to open a raw TCP connections from within the browser, the security of the browser would be immediately compromised: any website could then access the network on behalf of the user, within the same security context as the user. For example, a website could open a connection to a remote SMTP server and start delivering spam - a scary thought. Instead, WebSockets extend the HTTP protocol by defining a special handshake in order for the browser to establish a connection. In other words, it is an opt-in protocol which requires a standalone server.

Nothing stops you from talking to an SMTP, AMQP, or any other server via the raw protocol, but you will have to introduce a WebSocket server in between to mediate the connection. Kaazing Gateway already provides adapters for STOMP and Apache ActiveMQ, and you could also implement your own JavaScript wrappers for others. And if a Java based WebSocket server is not for you, Ruby EventMachine also allows us to build a very simple event-driven WebSocket server in just a few lines of code:

> websocket.rb

require 'em-websocket'
 
EventMachine::WebSocket.start(:host => "0.0.0.0", :port => 8080) do |ws|
  ws.onopen    { ws.send "Hello Client!"}
  ws.onmessage { |msg| ws.send "Pong: #{msg}" }
  ws.onclose   { puts "WebSocket closed" }
end

em-websocket (Ruby EventMachine WebSocket Server)

Downloads: 75 File Size: 0.0 KB

Consuming WebSocket Services

Support for WebSockets in Chrome and Safari also means that our mobile devices will soon support bi-directional push, which is both easier on the battery, and much more efficient for bandwidth consumption. However, WebSockets can also be utilized outside of the browser (ex: real-time data firehose), which means that a regular Ruby HTTP client should be able to handle WebSockets as well:

> em-http-websocket.rb

require 'eventmachine'
 
EventMachine.run {
  http = EventMachine::HttpRequest.new("ws://yourservice.com/websocket").get :timeout => 0
 
  http.errback { puts "oops" }
  http.callback {
    puts "WebSocket connected!"
    http.send("Hello client")
  }
 
  http.stream { |msg|
    puts "Recieved: #{msg}"
    http.send "Pong: #{msg}"
  }
}

em-http-request (Asynchronous HTTP Client)

Downloads: 27 File Size: 0.0 KB

WebSocket support is still an experimental branch within em-http-request, but the aim is to provide a consistent and fully transparent API: simply specify a WebSocket resource and it will do the rest, just as if you were using a streaming HTTP connection! Best of all, HTTP & OAuth authentication, proxies and existing load balancers will all work and play nicely with this new delivery model.

WebHooks, PubSubHubbub, WebSockets, ...

Of course, WebSockets are not the panacea to every problem. WebHooks and PubSubHubbub are great protocols for intermittent push updates where a long-lived TCP connection may prove to be inefficient. Likewise, if you require non-trivial routing then AMQP is a powerful tool, and there is little reason to reinvent the powerful presence model built into XMPP. Right tool for the right job, but WebSockets are without a doubt a much-needed addition to every developers toolkit.

Consuming XMPP PubSub in Ruby »

Created at: 10.11.2009 19:22, source: igvita.com, tagged: Architecture pubsub realtime xmpp

XMPP is a very versatile protocol with well over several hundred proposed and working extensions, which has also proven itself in production (ex: Google Talk). Presence, roster management, federated and server to server (S2S) messaging are all examples of features that you get for free, which make it a very appealing platform for messaging applications. Combine it with extensions such as XEP-0060 (PubSub), and we have all the relevant buzzwords: pubsub, real-time, federated, and presence.

The PubSub specification within XMPP, as defined in XEP-0060, is definitely not as flexible as that of AMQP, but it is often times enough to cover the most popular use cases. However, technical merits aside, one of the key missing components, especially in Ruby, has been the historical lack of functioning libraries - xmpp4r claims to support it, but examples are lacking. Thankfully, after test driving the latest batch of gems, it looks like we're finally there.

Getting off the ground with XMPP

Without a good toolkit XMPP can be a gnarly protocol to get started with - Pidgin IM client has some great tools for spying on the exchange, but monitoring pages of XML scroll by can only get you so far. Thankfully, Seth Fitzsimmons has built switchboard ("curl for XMPP"), which offers a powerful command line tool to greatly simplify the process. Make sure to read the full tutorial, or jump right into it by testing it with the Wordpress XMPP stream:

# list available options, subscribe to a blog, list subscriptions and then open the stream
switchboard disco --target pubsub.im.wordpress.com info
switchboard pubsub --server pubsub.im.wordpress.com --node /blog/icanhazcheesburger.com subscribe
switchboard pubsub --server pubsub.im.wordpress.com subscriptions
switchboard pubsub --server pubsub.im.wordpress.com listen

Based on xmpp4r, switchboard is also a toolkit for assembling your own XMPP clients, which means that it can be easily customized to power a PubSub consumer. From start to finish, and since examples are still hard to come by:

> switchboard-pubsub.rb

require 'rubygems'
require 'switchboard'
 
class WordpressJack
  def self.connect(switchboard, settings)
    switchboard.plug!(PubSubJack)
    switchboard.hook(:post)
 
    switchboard.on_pubsub_event do |event|
      event.payload.each do |payload|
        payload.elements.each do |item|
          on(:post, item)
        end
      end
    end
  end
end
 
settings = Switchboard::Settings.new
settings['pubsub.server'] = 'pubsub.im.wordpress.com'
settings['jid'] = 'user@im.wordpress.com'
settings['password'] = 'password'
 
switchboard = Switchboard::Client.new(settings)
switchboard.plug!(WordpressJack)
 
switchboard.on_post do |post|
  puts "A new post was received:"
  puts post.methods.sort.uniq
  exit
end
 
switchboard.run!

XMPP with EventMachine and Nokogiri

If you have an EventMachine stack, or looking for a high performance library, Jeff Smick's blather is definitely a gem to investigate. The combination of the asynchronous nature of EventMachine, a SAX parser within Nokogiri, and a great DSL make it very fast and a pleasure to work with:

> blather-pubsub.rb

require 'rubygems'
require 'blather/client/client'
require 'blather/client/dsl/pubsub'
require 'blather'
 
EventMachine.run {
  host = 'pubsub.im.wordpress.com'
  node = 'blog/icanhazcheesburger.com'
  user = 'user@im.wordpress.com'
  pass = 'pass'
 
  jid = Blather::JID.new(user)
  client = Blather::Client.setup(jid, pass)
  client.register_handler(:ready) {
    puts "Connected. Send messages to #{client.jid.inspect}."
    pub = Blather::DSL::PubSub.new(client, host)
  }
 
  client.register_handler(:pubsub_event) { |event|
    puts event
  }
 
  client.connect
}

PubSub & Event-Driven Architecture

Having personally struggled in the past with XMPP PubSub and Ruby, it's been great to revisit the use case and find a new set of fully functional libraries. The event driven architecture which is enabled by technologies such as XMPP, AMQP, Comet, Webhooks and PubsubHubbub are increasingly becoming the staple of many web applications, and for a good reason. If you haven't already, grab switchboard or blather and take XMPP for a test drive.