bgror

Introducing Node.js and Engine Yard Labs »

Created at: 18.11.2011 00:55, source: Engine Yard Blog, tagged: Engine Yard Cloud ruby Technology

This is an exciting day for us. Engine Yard, the leading platform-as-a-service for Ruby on Rails and PHP—has learned a new trick. You can now run your Node.js apps on Engine Yard Cloud!

Node.js

Starting today, Engine Yard is enabling early access support for Node.js applications on Engine Yard Cloud. Node.js is a popular event-driven framework written in JavaScript that is ideal for low-latency, real-time applications.

Node.js has a growing, active and self-motivated community that reminds us of the time when we started with Ruby. We love that energy, and so do our customers.

Every Engine Yard Cloud customer—including Free Trial customers—will now be able to take advantage of Node.js features. This feature is being delivered to you through a new program called Engine Yard Labs.

Engine Yard Labs

Engine Yard Labs allows users to try out experimental new features and capabilities on the Engine Yard platform. Engine Yard Labs will accelerate the pace of innovation by involving you, the user, in the vetting of new ideas. By actively taking in your feedback, we will be able to rapidly decide and act on the most promising features for Engine Yard products.

The features released through Engine Yard Labs are not officially supported and may or may not become supported as a part of Engine Yard products in the future. Our goal is to innovate jointly with you, harvest the most compelling features and make them generally available as quickly as possible. We hope you enjoy the process and benefit from the results!

How to access Node.js

Go to the Early Access Docs Page, and request access to Node.js.

Node.js in a few minutes

If you’re new to the Node.js world, let us explain the minimum required pieces that you’ll need to assemble a new application.

1. An application source file called “app.js”. or “server.js” A simple example using also the web framework Express looks like this:

   var express = require('express');
   var app = express.createServer();
   var port = process.env.PORT || 3000;
   
   app.get('/', function(request, response) {
      response.send('Hello Engine Yard Cloud!');
   });
   
   app.listen(port);

   We pass through the internal port number via `process.env.PORT` for your application to listen on.

2. An application and dependency descriptor called “package.json”. We use it internally to install your Node.js dependencies via the great npm packaging system. A simple application descriptor that has the web framework Express as a dependency looks like this:

   {
      "name": "nodeApplication",
      "version": "0.0.1",
      "dependencies": {
         "express": "2.5.0"
      }
   }

3. Store the application in a hosted git repository, just like you’re already doing with your current Engine Yard Cloud applications. Feeling a little lazy? Try a demo Node.js app we made earlier for you. With this you’re ready to deploy your first Node.js application on Engine Yard Cloud. Go to your dashboard and create a new app. This time, select Node.js in the application stack drop-down menu. Fill out the form with the information about your application, create a new environment, and deploy your app.

Perhaps now try an example that uses WebSockets.

You can read more about Node.js on Engine Yard Cloud in our documentation.

Pssst

If you're feeling more adventurous, follow this screencast and give other experimental features a try.

Groupon Makes History in More Ways Than One »

Created at: 15.11.2011 01:30, source: Engine Yard Blog, tagged: Customers ruby

Changing the World with Open Data »

Created at: 11.11.2011 23:13, source: Engine Yard Blog, tagged: Open Source php ruby

A few weeks ago, I attended a conference called Brooklyn Beta. In its second year, the conference is basically a collection of designers, developers, entrepreneurs, and crazy ne'er-do-wells. This conference usually leaves me feeling inspired, excited, intimidated, and ready to conquer the world, and this year was no different.

While there were a few underlying themes running through the conference, the main theme was "Make Something You Love." An intriguing and simple statement, but how many of us really are making something we love? How many of us are using our powers for Awesome versus just Meh? Creating something different, listening to your heart when all others think you're doing it wrong, helping society and others around you, and changing the lives of others, were all common mentions at this conference. Whatever tools you choose or language you code in, it doesn't matter. What matters is that at the end of the day, you made a difference. You left your mark on society. You made the world a better place to be in. You created something that wasn't there the day before.

Admittedly, I'm a bit of a data geek so when I saw a wonderful speaker named Todd Park give a talk on healthdata.gov, it really got my attention. (As well, Todd is a superbly dynamic speaker who can make even the topic of healthcare in America interesting and engaging.) Healthdata.gov is an open initiative that takes data from Medicaid/Medicare, hospitals, pharmacies, doctors offices and other related agencies, and releases it out into the public. To clarify, of course they aren't releasing personal, private information about you and your medical history. That is illegal. But they have been able to release aggregated information such as:

• HIV/AIDS facilities and research
• Foster care and adoption directory and statistics
• Fertility statistics
• Eldercare locator data
• Foodborne illness data
• Organ transplant data
• Designated Health Professional Shortage Areas
• FDA Drug Label data
• FDA recall data
• National Health Expenditures data
• Clinical trials for patients in need
• Toxic chemicals and environment data
• Medicare costs
• Cancer research
• Health and Well-Being statistical data

And this is only a sampling. Currently, there are 248 sources of data at this one site, just waiting for you to have a look. The best part is, they're working hard to encourage people to do cool things with the data they provide. Check out the Apps Expo on their site for some amazing apps that have been written so far, and their list of developer challenges and Code-A-Thons.

You might think some of this data sounds kind of boring. But what if you took it one step further and cross-matched it with other, unrelated data? Are there trends, connections, or correlations? For example, could a hospital prepare in advance for flu cases based on weather reports and trends? Could you track cost of living expenses with where there is a shortage of health professionals to help nurses in school figure out where they should live? What other kinds of seemingly unrelated data could we cross-match? For that matter, what other data is out there?

When we think of open data, we usually think about commercially released APIs from sites like Twitter and Facebook. But, there are hundreds of terabytes of other data just waiting for us to twist and turn and mold into something that wasn't there before. Like a sculptor creating a masterpiece from a block of stone or clay, we have a block of raw numbers that we can put together in new ways.

For instance, did you know that besides the government's health data, we also have at our disposal data on the following?

• NOAA weather and climate patterns
• Current NASA Space Mission Instrument and science data
• Consumer Broadband speed test statistics
• US Per Capita Food Availability data
• Demographic statistics (age/sex/race/geography) and historical data
• Air Quality data
• Disaster management for emergency networks
• Airport service and status data in realtime
• Railroad inspection, accident, and incident data
• Automobile safety ratings data
• Patent applications and assignments
• Vegetation and Drought data
• Threatened and Endangered Species data
• Historical climate, precipitation, freeze/frost data

There are over 100 XML feeds, APIs, and other data sources at http://www.usgovxml.com/, including other topics such as court findings, treasury information, social security data, etc. The list above is just a sampling.

Besides data available at the government level, let's not also forget public records data at the state and county levels. Publicly available information includes:

• Abandoned and Unclaimed Properties
• Grants
• Criminal records and data
• Real estate sales, transfers, and deeds
• Marriage/divorce/childbirth records
• Most Wanted and Fugitive data
• Obituaries
• Stolen vehicles/boats data
• Historical Newspaper Headline data
• Missing persons
• Economical data (exports, demographics, property values, salaries, etc.)
• Tourism data

A link to each individual state government website can be found on USA.gov.

What happens if we cross reference traffic patterns with age of people driving, or motorcycle licenses with crash reports, so we can better educate our motorcycle riding population and reduce the number of injuries? What kinds of motorcycle crashes are the most common? What part of the country has the lowest motorcycle fatality rate? Does average age of riders or weather have an effect on this? What makes them successful? What can we learn from that?

What happens if we overlap sports statistics with education? Or what if we look at health records and divorce records versus marriage licenses? Does a population with a high divorce rate also die earlier? Does it go to the hospital more? Are there diseases that are more common in married people versus single people? What if we referenced dog ownership with divorce rates? Or use health records as an indicator of where dog therapy programs should be?

What if we wanted to write an app that would help veterans or homeless advocates find abandoned and unclaimed properties, and match them with available grants? Or what if we wanted to help photographers get that perfect shot by matching up sunset times and beach locations with weather patterns, so they know when and where they can find a perfect cloud-free beach sunset?

What things can we predict? What correlations can be made? How can we help people do their jobs better or live better lives? How can we use our skills to make data available to others? What other data should we have access to, and who can we talk to about that?

It should also be known that there are hundreds of other data sources for other countries, not just the US. Some other resources for you, if you're interested in looking at international data:

• EuroStat
• United Nations Economic Commission for Europe
• International Data Resource Center

Knowledge is power, especially when it can be backed by looking at data in a new and exciting way. It's open and waiting for all of us. What will you do with it?

Deploying Rails 3.1 Applications to Engine Yard Cloud »

Created at: 11.11.2011 00:35, source: Engine Yard Blog, tagged: Engine Yard Cloud ruby

New Projects

If you are creating a new project starting with Rails 3.1, there is not much to worry about that is new. The main questions are the same, but the considerations are a bit new:

• Which test framework?
• Which database?
• Which js framework?
• Coffescript or not?
• Sass or not?

Existing Projects

You have already made all the above decisions for existing projects, but you may want to consider changing some based on the same considerations, so read on.

Asset Pipeline

Many of the updates in Rails 3.1 work together to provide a faster and more effective application. The core of this is the asset pipeline. And it has one of the largest impacts on operations and deployment of Rails 3.1 apps.

If you were an early adopter of Rails 3.1 on Engine Yard Cloud and added deploy hooks to precompile your assets, you will need to remove them as the Engine Yard recipes now do this for you and use a shared directory to hold current and previous deployment assets. This is to allow live deployment with Unicorn to survive the transition. There are new deploy hooks to allow you to synchronize any custom logic with the asset deployment process.

The new recipes place all compiled assets in the shared/assets directory and symlinks them to the project directory. This allows the symlink to be changed to 'last_assets' during the deployment process to keep the prior deployment live in the case of overlapping Unicorn deployment. The new hooks are 'before_compile_assets' and 'after_compile_assets'.

Assets are compiled into their final form during deployment and their file names reference the hash computed at this time. See the following section on CoffeeScript, jQuery, Sass, and ERB for some comments on the changes you need to deal with in referencing your assets in this new world.

Query Optimizations and PostgreSQL

A large performance enhancement is part of Rails 3.1 in the use of prepared queries. In MySQL unfortunately, this does not generate much benefit as the implementation in MySQL is sub-par. In PostgreSQL it is a significant improvement. If this is a new project you may want to use
--database=postgresql; you will need PostgreSQL installed on your development machine.

Streaming

When using Unicorn in Engine Yard you can take advantage of the http streaming support in Rails 3.1. This allows the response headers and body to be deliver before the response is fully formed. In particular for assets such as images this can be a significant response time difference, and you can see images load top to bottom in Safari which is a bit cool on its own. if you are delivering even larger assets like video or large documents this is a must have feature, and will influence your choice of app server.

Migrations

There are really no deployment impacts from the change in migration syntax, the main advantage is to developers during development.

Coffescript, jQuery, Sass, and ERB

With Rails 3.1 the asset pipeline defaults to supporting CoffeeScript, jQuery and Sass in generated assets and in the Asset
Pipeline. You do not need to use CoffeeScript, you can rename the files to .js and stay with raw javascript, and similarly
use .css for raw css files.

One of the more critical aspects of using the Asset Pipeline is that references to assets have changed. You can no longer just use the file name of the asset as in 'image_tag "test.png"' you need to account for the compiled asset name which is computed using a hash at deployment time. There is a new helper method 'asset_path' which will generate the correct path when building the request. But, since this is a Rails helper method you need to pass the assets like Sass or CSS or js or coffee files through ERB to do the path creation at request time. The asset compile process supports this as well so it is not being processed on every request (except in development). Add .erb to the file name extensions and the asset pipeline will do the right thing passing the source file through ERB then to the next pre-processor and the next as needed. See the Rails 3.1 Release Notes for more. While ordinary file references do work in the Engine Yard environment (because the files are copied to the assets directory as well as being compiled), their use is less desirable as you do not get browser cache activity based on content like you do with hash based names.

If you choose to use the new pre-processors you will get compiled assets for the generated files (css, js). One of the large benefits of this approach is that you end up with one compressed asset file for all your styles and js rather than a bunch of separate asset files. This provides more efficient caching and fewer browser round trips to load assets, affording faster page load times when combined with request streaming.

The following css example places the rails icon to the left of the element with .header class. The path to the asset is computed using ERB to reference the hash based file name created at deploy time.

   .header {
       background-image: url('< %= asset_path 'rails.png' %>');
       background-repeat: no-repeat;
       padding-left: 60px;
   }

Deploying on Engine Yard Cloud

As of this writing PostgreSQL support on Engine Yard is in beta so you will need to sign up for that feature if you wish to use that database for your Rails 3.1 application and the prepared statement support. If you enabled PostgreSQL support you will have the option under create or edit environment to select the database stack you wish for your environment. Note that it is not the application but the environment that selects the database stack. Your rails application does not need modification to change databases, because Engine Yard generates the database.yml at deploy time. You will need the appropriate gem in your gemfile, and can in fact have both MySQL and PostgreSQL gems referenced (mysql2 and pg).

You create an application as usual with the Engine Yard Cloud web interface indicating it is a Rails 3 application. Then create the desired environment options (Unicorn if you wish to use streaming). If you use Unicorn be sure to have the Unicorn gem in your gemfile as Unicorn will only use gems from the gemfile not from the system. Then boot your desired instance or cluster.

Deploy Something

In summary, Rails 3.1 is here and relatively painless to use with Engine Yard Cloud. The main features of 3.1 yield better performance, better structure, and more efficient applications. Moving to 3.1 is highly recommended. Even if you are not ready to deploy your production app to Rails 3.1 get to know it, kick the tires, and deploy something.

Engine Yard Winter Updates! »

Created at: 08.11.2011 23:59, source: Engine Yard Blog, tagged: Product ruby

Greetings! As the weather begins to cool, the team here is keeping warm by continuing to knock out more amazing features for all of you. I wanted to let you all know about some of the great things we have cooking over here at Engine Yard that will make life more awesome than it already is.

Highly Available, Highly Addictive

In talking to many of our customers, partners, and developers, one of the issues that frequently comes up is the ability to stay running when data centers inevitably experience an outage. Last April, one such outage impacted a variety of customers on Amazon Web Services. While we are always striving to improve our service, with our multi-region support, we were able to effectively move many of our impacted customers to Amazon’s US West region while minimizing downtime.

We plan to continue adding new features to further mitigate the effects of outages and disruptions. Currently, our development team is in the process of rolling out multiple availability zone deployments. When you create an application, Engine Yard Cloud will automatically deploy your instances across different availability zones in an Amazon Web Services region.  We balance the instances efficiently and ensure that master and slave instances are in separate zones. To further improve reliability, each tier (app server, database, and utility) will be balanced independently. The team is doing some final tests and should have it rolled out in a week or so. We’ll release more information on this soon and will announce availability via our @engineyard twitter account.

For customers with specific high availability requirements, Engine Yard Professional Services can customize an environment to meet your needs. For example, Engine Yard Professional Services recently deployed a high availability solution across two data centers for one our customers and completed a regional failover test with success. Be on the lookout soon for a case study on this initiative.  If you would like to learn more, please get in touch with our Professional Services team.

Need More Data!

A couple months ago, our data engineering team laid out a plan to strengthen Engine Yard’s data offerings. In an effort to make more database options available to our customers, the data engineering team released PostgreSQL 9.0 into alpha. After collecting some feedback, the team will be releasing PostreSQL 9.1 as a public beta over the next couple of weeks for all of you to use. PostgreSQL has earned a solid reputation for reliability, data integrity, and performance. As we noted earlier, those using Rails 3.1 can take advantage of significant performance gains using PostgreSQL.

The team also recently released MySQL 5.1 and 5.5 into alpha and will be releasing them as public beta shortly, as well. Please check back soon for news regarding MongoDB, Redis, and more!