bgror

The Curious Tale of the Humble Micro »

Created at: 28.09.2010 14:45, source: Engine Yard Blog, tagged: Technology aws ec2 fog micro instance

Amazon recently announced micro instances for Elastic Compute Cloud, or EC2. Historically new instance types were larger than existing ones, but this time we got a smaller, more affordable option. It was an exciting announcement for me, as a hobbyist on the look out for servers on the cheap. At 613MB of RAM and two cents per hour, micro instances have roughly a third of the ram, at a quarter of the price of small instances. Small instances are 32 bit, while micro instances (unlike any other types) match the 32 or 64 bit value from their image. Beyond this simple comparison, however, things get a bit pear shaped. Micro instances have no ephemeral storage, which also means they do not support instance-store images. Instance-store images are stored in snapshots and copied to the ephemeral root device when instances are booted. The alternative is to use volume-store images, which mount the image snapshot on an Elastic Block Store (EBS) volume. Making a choice between instance and volume store involves some trade offs. Instance-store boots slower but costs less. Volume-store boots faster, runs slower at first while data finishes copying and costs more (the price difference comes from EBS transfer/storage costs). Publicly available, supported AMIs do not necessarily provide volume-store versions. A conversion process will allow the transformation of an instance-store image into a volume-store image, but this effectively forks away from the supported version, and it is not clear that it's worth the trouble if your only goal is running micros. The characteristic of micro instances that probably sticks out the most is the allotted CPU. EC2 describes its machines in terms of Elastic Compute Units, or ECU, each unit equivalent to a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor. Instances prior to micro were listed with a number of allotted ECUs, with small providing a reference point at 1 ECU. Micro instances break the trend though, and are listed as providing "Up to 2 EC2 Compute Units (for short periodic bursts)". The fine print provides some guidelines but no hard numbers. Bursting is everything above some background level much lower than 1 ECU, and is allowed for some percentage of each time period. Unfortunately the specifics are not revealed and bursting out of turn will both be throttled and increase the time before the next burst. To provide context, micro instances are are described as serving tens of requests per minute, with the fine print containing a number of graphs showing recommended CPU usage patterns. So, at the end of the day it turns out that micro really is not the new small. They are smaller and have fewer, weirder resources which are allotted in a non-specific way that seems counter to the way many people might use them. Load balancing, proxies, cron or monitoring systems might be workable, but running web or database servers would be pushing your luck. If you need something cheap but consistent, your best bet probably remains small instances (if not a vps or different cloud). If you need affordable, bursty processing, you're probably better served using non-micro spot instances. Based on these considerations, we are not currently rushing to add micro support to Engine Yard AppCloud, where it could easily provide a poor user experience for many customers. However, we will keep these things in mind as they become more tried and as we develop new features. Anecdotal evidence aside, I welcome you to try them out for yourself:

  require 'rubygems'
  require 'fog'

  # NOTE: You should replace ALL_CAPS things, manually or by defining constants
  compute = Fog::AWS::Compute.new(
    :aws_access_key_id => AWS_ID,
    :aws_secret_access_key => AWS_KEY
  )

  # bootstrap:
  # uses or creates fog_default keypair from private key to boot server with
  # opens port 22 of the first security specified or 'default' group.
  # an 'EBS boot' ubuntu image list is available: http://alestic.com/
  # waits for the server to be ssh-able, which can take a minute or two
  server = compute.servers.bootstrap(
    :flavor_id => 't1.micro',
    :image_id => 'ami-6c06f305',
    :private_key_path => PRIVATE_KEY_PATH,
    :public_key_path => PUBLIC_KEY_PATH,
    :username => 'ubuntu'
  )

  # run some commands over ssh and return results
  server.ssh(ARRAY_OF_COMMANDS_TO_RUN)

  # shutdown the server
  server.destroy

Double Shot #659 »

Created at: 02.03.2010 13:00, source: A Fresh Cup, tagged: Double Shot aws erector flowdock github mongo ujs

I could use at least a small slowing-down of the treadmill for a day or two.

• How I reduced translation costs of 200 articles from $9000 to $46 - Second Life as an alternative to Mechanical Turk.
• Notes from a production MongoDB deployment - Some detailed stories from the trenches.
• Rapid prototyping with HAML, SASS and Ruby - A workflow for coming up with web site designs.
• Erector - Builder-like view framework for object-oriented HTML composition.
• ambitious_query_indexer - Another plugin for testing whether your Rails application needs more indexes.
• Flowdock Public Beta to launch - Our team has been using this tool, and it's been working well for us.
• Introducing GitHub Compare View - More analytic power from GitHub, who are clearly concentrating on feature differentiation these days.
• Unobtrusive, yet explicit - Jamis Buck tries to work out a useful pattern for UJS in Rails 3.
• S3Hub - Amazon S3 client for OS X.

Double Shot #645 »

Created at: 10.02.2010 12:56, source: A Fresh Cup, tagged: Double Shot aws bundler firebug jbreadcrumb jquery rails

It's time to buckle down and deliver some code.

• Plugin Authors: Toward a Better Future - I personally do not count breaking every plugin in the Rails ecosystem as "better." But if you've got a plugin that you want or need to make Rails 3 compatible, this is essential information.
• Using Bundler in Real Life - Scenario-based workflows for the new Bundler gem.
• jQuery with Rails 3 - How to get hooked up to the new unobtrusive javascript bits.
• New Feature: Amazon S3 now supports Object Versioning - It'll be interesting to see how long it takes someone to build a blob-based SCM on top of this.
• jBreadCrumb - Somewhat busy display aimed at handling deeply nested breadcrumb navigation.
• FireQuery - Firebug extension for jQuery development.

Double Shot #644 »

Created at: 09.02.2010 13:10, source: A Fresh Cup, tagged: auto_timeout aws css devise haml mapreduce rails riak ruby savon sinatra textmate

There's certainly no shortage of goings-on to link to lately.

• Devise 1.0.0 - Major milestone for this Rack-based authentication solution for Rails.
• Cloudfront: no-brainer CDN Support for S3 - I've got a couple of apps out there where I really should implement this.
• Save file and reload Safari from TextMate - Passed on by Alex Heaton.
• Bye Bye Github and GitHub post - Afterthoughts - Elad Meidar discusses expectations and git hosting.
• SendEmail 1.56 - Command line email sending that can handle TLS (and thus Google Mail servers). Requires some perl modules.
• sinatra_more - Gem to boost Sinatra to handling more complex applications.
• Auto timeout sessions in Rails - Useful plugin from Matthew Bass, for the times when you need to keep people from just idling. I ended up forking it to make it use jQuery instead of Prototype.
• MR.Flow - Web-based designer for MapReduce operations.
• EdgeRails.info - Ryan Daigle is moving all his edge content to a fresh domain.
• RubyInstaller Release Candidate 2 - 1.8.6, 1.8.7 and 1.9.1 - Useful tool for people who want to run Ruby on Windows without resorting to a virtual machine.
• Savon - Ruby SOAP client that includes WSSE authentication.
• Wrap your SQL head around Riak's Map-Reduce - Personally my SQL head says "no! enough already!" but I expect I'll have to learn this stuff sooner or later.
• Haml Sucks for Content - Actually a good post on some advanced HAML techniques rather than another entry in the holy war.
• Less.app - Autocompiler for your LESS CSS files. Well, if you have any.

Double Shot #628 »

Created at: 18.01.2010 14:13, source: A Fresh Cup, tagged: Double Shot aws cacheable deadweight github has_price helium maglev popthis rails ruby

The inital round of the Developer Wellwishes campaign is drawing to an end on Thursday, when we'll be combining donations with a few other campaigns to make a contribution to charity:water. If you've done your share for earthquake relief and still want to help more people out, do drop by.

• Distributed Ruby with the MagLev VM - A nice look at what's going on with this alternate Ruby, which to my mind is more significant than Rubinius or MacRuby. Being able to store distributed objects with no ORM is likely to be a killer feature.
• Deadweight and Helium - CSS coverage tools that can report which selectors are unused in your application.
• popthis - Turn a folder of email files into a POP3 server for testing purposes.
• git-googles - Git utilities for branch management and code review.
• The least you should know about S3 with Rails - This stuff is all out there, but here's a one page overview.
• Yaml Cookbook - Turns out Ruby's YAML support is a good deal more flexible than I had realized.
• Ruby Fibonacci Shootout - MacRuby is turning out some impressive performance numbers. Too bad it can't run anything I care about yet.
• has_price - DSL for organizing things like taxes, discounts, and base prices. Wish I'd had this a year ago.
• cacheable - Gem that adds cacheify, which is similar to memoize but uses memcached.