bgror

Ludum Dare for Rubyists: An Online 48 Hour Game Coding Competition »

Created at: 13.12.2011 16:55, source: Ruby Inside, tagged: events News

Ludum Dare is an online accelerated game development event that focuses on regular 48 hour competitions. Think Rails Rumble but for games! It's been around since 2002 but has had a big publicity boost recently due to the participation of Notch, the creator of the mind-bogglingly popular indie game Minecraft.

The next Ludum Dare contest is taking place this coming weekend between December 16-19, 2011 and I want to encourage Rubyists to take part. The competition tends to be dominated by Java, Flash, Microsoft XNA developers, and HTML5 developers, so it'd be great to see more Ruby entries (of which there have only been a couple so far).

During August's event, I, along with hundreds of others, was glued to Notch's livestream watching him code his game, Prelude of the Chambered (a 6 minute version is on YouTube). I was inspired enough to port his Java code into Ruby using JRuby, producing
potc-jruby (sadly far slower than the original Java version). This time, I plan to enter for real and build my own original game.

How to Take Part in Ludum Dare

Go to the Ludum Dare homepage, read the rules and guide, register on their WordPress blog, wait until the 'theme' has been decided, and start coding once the countdown is finished.

During the 48 hours (or 72 if you do the 'jam' version), you can post blog entries directly to the main Ludum Dare site (if you want) and submit your entry via a special link at the end. Entrants play and judge each other's entries for a period of three weeks before the winners are announced. Having more Rubyists involved would be useful since our games may be less likely to work cross platform or without Ruby installed.. (more on this shortly)

A quick summary of the rules:

• You have to work alone. (If you want to do a team effort, you need to enter the less restrictive 'jam' contest.)
• All content and code must be created within the 48 hours (except for libraries, legally licensed fonts and drum/instrument samples).
• Your game has to be based on the theme given out before the contest.
• You must share the source code with the other participants at the end of the contest though you do not have to give it an open source license.

The contest has a popular IRC channel (which is already quite active) at #ludumdare on irc.afternet.org. I'm petercooper on there - say hi! I'll be lurking in there a lot over the next week. Also, follow @ludumdare on IRC for more updates and info.

To get a feel for the contest, check out this "keynote" from the last Ludum Dare. There'll be a new one for this year soon:

Building a Game in Ruby?

Building games in Ruby isn't popular but it's not frontier country either. Rubystein, a Wolfenstein 3D pastiche by the Phusion guys, remains a favorite of mine and it even runs on 1.9 with only a few tweaks.

There's a great series by Andrea Wright that dates from 2007 but still has some handy pointers. We also have Ray, RubyGame, and Gosu which all have their fans (Ray is the most recent Ruby game library I'm aware of).

Or.. JRuby!

My choice for the contest is none of the above. Instead, it's JRuby. As part of the 'warmup process' for the contest, I've been playing with JRuby and the popular Slick2D Java library. The performance is amazing and the development process pretty straightforward.

Being a popular library in the Java world, I can use a lot of the Java-based tutorials and code samples for Slick2D to get a feel for how it all works. And.. I'll be writing an article for Ruby Inside in the next day or two showing you how to get started with it for yourself :-)

ZendCon After-Party: PHree Beer and Good Company »

Created at: 18.10.2011 23:19, source: Engine Yard Blog, tagged: community events

To celebrate the fact that Orchestra has joined the Engine Yard family, we are throwing our first ever PHP Conference after-party tomorrow! If you will be in Santa Clara for ZendCon, come on out to the Faultline Brewing Company.

After-Party: PHree Beer and Good Company
Wednesday, October 19, 2011 from 7:00pm - 9:30pm
Faultline Brewing Company, 1235 Oakmead Parkway, Sunnyvale, CA 94085
Full details here.

Shuttles will be picking up attendees from the Santa Clara Convention Center beginning at 6:45pm and spiriting folks away to our epic soiree. Grab some tasty appetizers and one of Faultline's specialty brews (or a few--shuttles will be running throughout the night).

Do you have a cool PHP app you want to deploy on our platform? This is also a great opportunity for you to get some face time with the experts on the Orchestra team (@davidcoallier, @EamonLeonard, @gwoo, @h) to discuss how they can make your project "sing."

See you there!

It’s All About JRuby »

Created at: 23.09.2011 19:56, source: Engine Yard Blog, tagged: community events Open Source javaone jruby san francisco jruby meetup

[caption id="attachment_10652" align="alignright" width="300" caption="Photo courtesy of Flickr user veganstraightedge*"][/caption]

On the tails of an excellent San Francisco JRuby meetup last night, we're delighted to be heading to JavaOne in a few weeks. JavaOne is October 2-5 and is held right in our backyard in downtown San Francisco. The multi-track conference is devoted to innovative developments and cutting edge implementations of Java. Naturally, we'll be dishing up all the sweet, sweet JRuby goodness you can handle.

Nick Sieger will be kicking it off right on Monday, October 3 with a talk about how you can add Ruby agility to your Java web stack.

On Tuesday night, October 4, be sure to stop by Engine Yard HQ around 7:30 for even more face time with the JRuby core team at our JavaOne after-party. Socialize with your fellow conference-goers and JRuby enthusiasts of all kinds at our rooftop penthouse. Don't miss special presentations by JRuby experts--drinks, snacks and networking will follow.

On Wednesday, October 5 our own Jacob Lehrbaum and Mike Piech will give you the inside scoop on how to leverage JRuby in your cloud-centric business. Tom Enebo, Charles Nutter and Dr Nic Williams will also be out and about repping JRuby, so don't be afraid to stop by the Engine Yard booth to discuss this awesome technology with the foremost experts over a beer. Great giveaways and prizes await!

JRuby will also be a contender again this year in the JavaOne Script Bowl! With competitors weighing in from the Clojure, Groovy and Scala sides, we need the community behind us to give JRuby the righteous victory it deserves. Come check it out on Wednesday, October 5 at 8:30am at the Hilton San Francisco, and judge for yourself which scripting language supplements your Java platform best. Support Dr Nic and the JRuby core team as they show off the latest and greatest in JRuby.

Lastly, for any interested JRubyists who missed the JRuby Jam virtual session last week, you can now watch the full presentation and view the slides.

JRuby Jam Session from Engine Yard on Vimeo.

We hope you’ll join us in celebrating JRuby virtually, or better yet, in-person!

*I <3 JRuby shirt photo courtesy of Flickr user veganstraightedge.

Fáilte Orchestra Contest Winner: Daniel McOrmond’s Irish Orchestra.io Tweets »

Created at: 21.09.2011 00:45, source: Engine Yard Blog, tagged: community Contests events contest funconf php

It's with pleasure that I announce Daniel McOrmond as the winner of our Fáilte Orchestra competition.

Daniel's "Irish Orchestra Tweets" entry is a dead simple app that takes the tweet-stream from the @orchestra_io Twitter account, displays them against a cringe-worthy 'oirish backdrop, and translates them to what appears to be an American's interpretation of what an Irish accent sounds like.

I mentioned last week that we wanted the "smartest, wittiest or most-charming" app, and Daniel's entry was most fitting. We really felt that it was in the spirit of the competition: combining some nice cheesy Irish humour with a nice little application of technology. Nice work, Daniel! We'll see you in Ireland for Funconf!

5 subtle ways you’re using MySQL as a queue, and why it’ll bite you »

Created at: 15.09.2011 03:14, source: Engine Yard Blog, tagged: Contests events Technology

I work for Percona, a MySQL consulting company. To augment my memory, I keep a quick-reference text file with notes on interesting issues that customers ask us to solve. One of the categories of frequent problems is attempts to build a job queue in MySQL. I have so many URLs under this bullet point that I stopped keeping track anymore. Customers have endless problems with job queues in their databases. By "job queue" I simply mean some list of things they've inserted, which usually need to be processed and marked as completed. I've seen scores -- maybe hundreds -- of cases like this.

Many people realize the difficulties in building a good job queue or batch processing system, and try not to create one inside MySQL. Although the job queue is a great design pattern from the developer's point of view, they know it's often hard to implement well in a relational database. However, experience shows me that job queues sneak up in unexpected ways, even if you're a seasoned developer.

Here are some of the most common ways I've seen the job-queue design pattern creep into an application's database. Are you using MySQL for any of the following?

1. Storing a list of messages to send: whether it's emails, SMS messages, or friend requests, if you're storing a list of messages in a table and then looking through the list for messages that need to be sent, you've created a job queue.
2. Moderation, token claims, or approval: do you have a list of pending articles, comments, posts, email validations, or users? If so, you have a job queue.
3. Order processing: If your order-processing system looks for newly submitted orders, processes them, and updates their status, it's a job queue.
4. Updating a remote service: does your ad-management software compute bid changes for ads, and then store them for some other process to communicate with the advertising service? That's a job queue.
5. Incremental refresh or synchronization: if you store a list of items that has changed and needs some background processing, such as files to sync for your new file-sharing service, well, by now you know what that is.

As you can see, queues are sly; they slip into your design without you realizing it. Frankly, many of them aren't really a problem in reality. But the potential is always there, and I've observed that it's hard to predict which things will become problems. This is usually because it depends on behavior that you don't know in advance, such as which parts of your application will get the most load, or what your users will promote to their friends.

Let's dive a little bit into why job queues can cause trouble, and then I'll show you some ways to help reduce the chance it'll happen to you. The problem is usually very simple: performance. As time passes, the job queue table starts to either perform poorly, or cause other things to perform badly through collateral damage. There are three primary reasons for this:

1. Polling. Many of the job queue systems I see have one or more worker processes checking for something to do. This starts to become a problem pretty quickly in a heavily loaded application, for reasons I'm about to explain.
2. Locking. The specific implementation of the polling often looks like this: run a SELECT FOR UPDATE to see if there are items to process; if so, UPDATE them in some way to mark them as in-process; then process them and mark them as complete. There are variations on this, not necessarily involving SELECT FOR UPDATE, but often something with similar effects. The problem with SELECT FOR UPDATE is that it usually creates a single synchronization point for all of the worker processes, and you see a lot of processes waiting for the locks to be released with COMMIT. Bad implementations of this (not committing until the workers have processed the items, for example) are really horrible, but even "good" implementations can cause serious pile-ups.
3. Data growth. I can't tell you how many times I've seen email list management applications that have a single huge emails table. New emails go into the table with a "new" status, and then they get updated to mark them as sent. As time passes, these email tables can grow into millions or even billions of rows. Even though there might only be hundreds to thousands of new messages to send, that big bloated table makes all the queries really, really slow. If you combine this with polling and/or locking and lots of load on the server, you have a recipe for epic disaster.

The solutions to these problems are actually rather simple: 1) avoid polling; 2) avoid locking; and 3) avoid storing your queue in the same table as other data. Implementing these solutions can take a bit of creativity, however.

First, let's look at how to avoid polling. I wish that MySQL had listen/notify functionality, the way that PostgreSQL and Microsoft SQL Server do (just to mention two). Alas, MySQL doesn't, but you can simulate it. Here are three ideas: use GET_LOCK() and RELEASE_LOCK(), or write a plugin to communicate through Spread, or make the consumers run a SLEEP(100000) query, and then kill these queries to "signal" to the worker that there's something to do. These can work quite well, although it'd be nice to have a more straightforward solution.

Locking is actually quite easy to avoid. Instead of SELECT FOR UPDATE followed by UPDATE, just UPDATE with a LIMIT, and then see if any rows were affected. The client protocol tells you that; there's no need for another query to the database to check. Make sure autocommit is enabled for this UPDATE, so that you don't hold the resultant locks open for longer than the duration of the statement. If you don't have autocommit enabled, the application must follow up with a COMMIT to release any locks, and that is really no different from SELECT FOR UPDATE. (The rest of the work can be done with autocommit disabled; you need to enable it for only this statement.) While I'm wishing for things, I wish that SELECT FOR UPDATE had never been invented. I haven't seen a case yet where it can't be done a better way, nor have I seen a case where it has failed to cause problems

Finally, it's also really easy to avoid the one-big-table syndrome. Just create a separate table for new emails, and when you're done processing them, INSERT them into long-term storage and then DELETE them from the queue table. The table of new emails will typically stay very small and operations on it will be fast. And if you do the INSERT before the DELETE, and use INSERT IGNORE or REPLACE, you don't even need to worry about using a transaction across the two tables, in case your app crashes between. That further reduces locking and other overhead. If you fail to execute the DELETE, you can just have a regular cleanup task retry and purge the orphaned row. (Hmm, sounds like another job queue, no?) You can do much the same thing for any type of queue. For example, articles or comments that are pending approval can go into a separate table. This is really required on a large scale, although you shouldn't worry that your Wordpress blog doesn't do things this way (unless you've been hired to rewrite CNN.com using Wordpress as a backend).

Finally, and I've saved perhaps the most obvious solution for last, don't use the database at all! Use a real queueing system, such as Resque, ActiveMQ, RabbitMQ, or Gearman. Be careful, however, that you don't enable persistence to a database and choose to use MySQL for that. Depending on the queue system, that can just reintroduce the problem in a generic way that's even less optimal. Some queue systems use all of the database worst practices I enumerated above.

I hope this article has given you some insight into the variety of ways that job queues inside of MySQL can sneak up on you and bite you in the tendermost parts. And I hope you can learn to recognize and avoid this design pattern yourself, or at least implement it in a way that won't hurt you. It really is such a common problem that it's become one of the classic questions I see. Now, I'm off to check my list of pending consulting requests and see what I should work on next.