bgror

There is no magic, there is only awesome (Part 4) »

Created at: 26.01.2010 06:51, source: the { buckblogs :here } - Home, tagged: Essays and Rants

This is the fourth (and final) article in a series titled “There is no magic, there is only awesome.” The first article introduced the four cardinal rules of awesomeness, the second was about knowing thy tools, and the third encouraged you to know thy languages .

First off, I apologize for dragging this out. It’s really become a weight on my shoulders. I’ve been fretting and fretting about writing the last two or three posts in this series, and I just couldn’t find the inspiration to make them come out like I wanted…and they’ve been holding up other posts I’ve been wanting to write.

So I’m going to cheat. You’re going to get a braindump, more or less, of the last two rules of awesomeness. Yes, I am entirely cognizant of the irony here. Nonetheless, here goes.

Rule #3: Know thy libraries!

If you want to be awesome, know your dependencies. At the very least, understand what each plugin, library, and technique you use, does. If called on to explain why you are using a particular design pattern, could you justify it? If someone asked you to compare plugin X and plugin Y, could you explain what their different strengths and weaknesses are? If you discover a particular library behaving erratically, would you be able to dig into the library’s guts to tell the author roughly where the problem is? Or would you be one of those who has to tug on the author’s sleeve and say, metaphorically, “it hurts when I do this”?

By libraries, I mean (essentially) any bit of code that you depend on, that has a life of its own outside your application. I’m including design patterns in this, although they aren’t really code, but they do represent a “library” of possibly ways to architect code. Basically, if you’re using someone else’s code, algorithms, or techniques, know why you’re using them. Make sure you can answer the four questions:

1. What does this do best?
2. What does this do worst?
3. Why should I use this in particular?
4. When was the last time I learned something new about this?

Rule #4: Know thy communities!

The last rule says that if you want to be awesome, you can’t do it in a vacuum. By writing code, you automatically join an ecosystem, whether it be a network of coworkers who will help you write, debug, deploy, and maintain that code, or a community of hobbyists who might wish to use your code in applications of their own (or who are writing code that you want to use). Even if all you do is pop up out of your cave long enough to toss some code into the public domain, you’re still contributing to a community, and if you’re using other people’s code, then you’re a consumer as well.

Don’t be blind to those communities. Stop for a minute and think about the communities you belong to, whether explicitly (where you’re a card-carrying, due-paying member) or implicitly (where you’re passively consuming someone else’s code), or somewhere in-between.

Why is this important? Mostly because communities are resources. Each community will be good for different kinds of help, collaboration, or criticism, and if you want to make the most of those resources (which, if you’re really awesome, you will), you need to know where each shines. Again, apply the four questions. What does this community do best? What is it worst at? Why should I participate in this particular community, over all the possible alternatives? And when was the last time I learned from this community?

That last is important, and surprisingly deep. If you are no longer learning from a community, what are you still doing there? Some of you are no doubt saying “but, but, but” and eagerly pointing out that maybe someone is part of a community so that they can lift others up, but note: if you’re doing that right, you’re still learning from those you teach. You learn new ways that your craft can be confusing to people. You learn new ways of looking at things that you’ve been taking from granted. You learn new ways to rephrase concepts that have grown stale in your own mind. It keeps you fresh. If you’re doing it wrong, though, then the only reason you remain in a community that teaches you nothing is complacency. And complacency is on the opposite end of the excellence spectrum from awesomeness.

Words of caution

So, those are the four rules. There is one overriding caveat to all of this, though, and that is: use moderation. You’re not going to master all of this in a day. Or even a week, or month, or year. I don’t know anyone that is perfect at all four things (though I know many who are much, much better than I am). To sound all zen and mysterious: awesomeness is a journey, not a destination. Enjoy the trip, appreciate the view, but don’t overdo it.

When you overdo it, you risk burning out. I’ve been riding that line for a couple years now, which was frightening. When writing code is your bread and butter, paying the bills and putting food on the table for you and your family, it is scary to contemplate “what do I do when I suddenly don’t want to do what I’ve been doing”. This is a big reason why I had to let go of Capistrano and other projects, and why I’ve discovered other hobbies (like woodcarving, string figuring, and cooking) that do not involve computers. I’m trying to find the middle ground, so that I can recapture the joy of writing code.

So: be awesome. But only in moderation.

There is no magic, there is only awesome (Part 3) »

Created at: 09.10.2009 19:48, source: the { buckblogs :here } - Home, tagged: Essays and Rants awesome keynote programming talk

This is the third article in a series titled “There is no magic, there is only awesome.” The first article introduced the “four cardinal rules of awesomeness”. The second article discussed knowing your tools.

Opening A.—Pass index finger of right hand distal to the little-finger loop, and passing round the ulnar side of that loop, bring it up from the proximal side into the thumb loop, and with the index finger pointing downard, take up with the back of the index finger the radial thumb string and return.

Even to string figure adepts, it can be challenging to parse those instructions. That paragraph is an extract from the instructions for an Eskimo Caribou string figure, written in 1903 by Dr. A. C. Haddon and published in American Anthropologist (you can read the whole thing on Google Books if you’re really feeling sleepless).

The original string figure notation described by Drs. Rivers and Haddon in 1902 used very technical anatomical terms to identify each finger, and the location of the string on each finger. As in that paragraph, you’ll see terms such as proximal (closer to the base of the finger), distal (closer to the finger tip), radial (closer to the thumb), and ulnar (closer to the little finger). These and other terms are used to describe locations relative to the fingers, as well as to name specific strings (“radial thumb string”) on the hand.

The strength of this notation is that it is very precise, and can be used with little need of external illustration. However, it is also fairly verbose, making it hard to parse without very focused attention and (potentially) multiple read-throughs. Also, the use of the technical anatomical terms makes the descriptions hard for novices to pick up.

To make her book more accessible to a larger audience, Caroline Furness Jayne (or “CFJ”) modified this notation so that “proximal” and “distal” were replaced with “lower” and “upper”, and “radial” and “ulnar” were replaced with “near” and “far”. (She left dorsal/palmar alone, for the most part, although in some figures she referred to the “front” or “back” of a hand or finger.) Her modifications introduced ambiguity, though, and required even more verbosity and copious illustrations to counter.

For example, she described the same movements (from the Eskimo Caribou figure) as follows:

First: Opening A. (The left palmar string must be taken up first). Second: Bend the right index away from you over the right far index string and over both strings of the right little finger loop and down on the far side of the right far little finger string; then draw toward you on the near side of the right index both right little finger strings and the right far index string, allowing the right near index string to slip over the knuckle of the index and to the far side of the finger. Now put the right index (still bent and holding the strings on its near side) from below into the thumb loop, by pressing the near side of the bent index toward you against the right far thumb string, and putting the tip down toward you over and on the near side of the right near thumb string. Pick up, on the far side of the bent right index, this right near thumb string, and lift it and the former near index string up by turning the index away from you and up to its usual position.

(I ommitted the illustrations for brevity’s sake, but if you want to see them, you can view the complete instructions here.)

The lack of technical terms is definitely an improvement, but the (significantly!) greater quantity of text means that the instructions require even more effort to parse. To combat this, the ISFA (International String Figure Association), in its publications, uses an abbreviated notation, numbering the fingers 1 (thumb) through 5 (little finger), and using other one-letter abbreviations for right (R), left (L), near (n), and far (f). They retain the use of proximal/distal in some cases, but also use “below”/”above” and “lower”/”upper” where it isn’t too ambiguous. The ISFA notation reduces the above text to this:

Opening A. R2 rotates away from you, over all strings, around 5f, then back under the strings, and proximally enters R1 loop. R2 hooks up R1n and retraces its path. (from http://www.isfa.org/arctic/30.htm)

This is much more concise, and is easier to scan. It settles on standard meanings of terms like “rotate away”, “hook up”, and “release”, which reduces verbosity, but also requires the reader to know how those words are defined in this context. Also, it is concise at the cost of specificity; there is often ambiguity that must either be resolved at “runtime”, or be countered with more verbose descriptions.

Two other notable attempts at describing string figures are SFN (String Figure Notation) by Eric Lee, and Mizz Code by Kyoichi Miyazaki (“Mizz”). The former is a variation on ISFA notation, employing various abbreviations to reduce verbosity, while the latter is an entirely new take on describing string figures that simply denotes how the string needs to move, without specifying finger movements at all. Both of these are much more concise than any of the others described above, but are also quite specialized and require some study before you can really make heads or tails of them.

So, that gives five different ways of describing string figure construction! How do you choose which one to use? Ultimately, the ISFA notation is the most widly used, as well as the verbose prose of CFJ (thanks to the popularity of her book). But a string figurist will benefit from knowing as many ways to describe a figure as possible, because each notation’s strategies reveal different insights into the techniques, and has different strengths in different situations. CFJ’s prose (if accompanied by a modicum of illustration) is good for beginners, because it does not require any specialized knowledge. The ISFA notation is a good general tool that allows people to quickly learn figures, with only a little training. And notations like SFN are great for quickly jotting down a figure as you learn it from someone else, or as you invent it yourself.

Really, as a string figurist, the more string figure languages you know, the more power you have.

Computer Languages

Thus, the second rule of awesomeness: know thy languages.

As a computer programmer, you’re faced with an even greater flood of languages than string figurists are. Even if you use primarily a single language, you might be surprised at how many incidental languages you use on a regular basis. Are you a web developer? Odds are that you dabble in both HTML and CSS, and probably Javascript, too. SQL is almost mandatory for most interactive systems. Furthermore, if you do anything at all in a text console, you probably know a little bit of shell script. Building your application and deploying it to production probably require a specialized DSL or two as well. And regular expressions are practically everywhere these days.

Despite there being more computer languages than string figure languages, the advice is the same: the more you know, the more power you have. And “knowing” is not merely “knowing about”. Look back at the four “knowledge gauge” questions from Part 2:

1. What does this do best?
2. What does this do worst?
3. Why should I use this in particular?
4. When was the last time I learned something new about this?

Take a minute and try to list all the languages you use on a regular basis. This includes even languages you use only tangentially, like SQL, regular expressions, Javascript, HTML, XML, CSS, and so forth. Try to answer each of the four gauge questions for every language in your list. If you struggle to answer any of them, consider that a jumping off point for further investigation.

If you aren’t really sure what a particular language is best or worst at, it may be because you aren’t familiar with enough programming languages to compare it with. You’ve probably heard the old saying, “to a man with a hammer, everything looks like a nail,” and that surely applies here. You may be at the pinnacle of guruship in your chosen programming language, but if that’s all you know, then you can’t give worthwhile advice on when that language should be used. You’re living in a house with no windows, and no matter how well you know that house, and no matter how big it is, you can never tell callers what’s just outside.

It’s been suggested by people much smarter than me that you should learn a new programming language every year. I can’t recommend this enough. I won’t deny that there have been years where I haven’t followed this advice, but every time I’ve spent a few months tinkering with a new programming language I’ve come away with my eyes open that much wider. It’s like having a lever, and finally discovering a fulcrum to use it with.

Learning a new language

Now, it’s all well and good to say “learn a new programming language”. It’s also remarkably easy to download the development kit for Ruby, Python, Java, Erlang, Haskell, or just about any other language you want to learn. And once you’ve started learning a language, the learning effect tends to snowball, so that’s not really a challenge either.

What’s hardest (for me, at least) is discovering a good “entry vector” into the language.

Here’s what works for me:

1. Find the documentation for the language. Look for a page or two that give you a glimpse into the syntax. Not every language provides an introduction like this, but when you can find one, it’s a great way to get a “feel” for how the language expresses things.
2. Next, look for a “hello world” example, or tutorial. Generally, these things are useless, but as an introduction to the language, it’s a good way to see what’s involved in a fully functional program. How much boilerplate does a program need? How is a program compiled or interpreted, and run? The “hello world” example is also a good place to start experimenting. As trivial as it sounds, try changing the string that gets printed. Try displaying an additional string. Hold on to this file; you can use it to play with as you progress through the next steps.
3. Now, look deeper. Try and find a more advanced tutorial. Look for examples that demonstrate writing procedures, objects, conditional branching, and so forth. They will almost certainly include concepts unique to the language, which will be new to you, but that’s the point! Work through those tutorials. Find additional documentation to clarify concepts that are unfamiliar to you. Go beyond the tutorials and experiment with these concepts, to solidify their meaning in your mind.
4. Once you’ve finished some tutorials, look for actual production code written in that language. GitHub is a fantastic resource for this kind of exploration, as are similar sites like SourceForge. Individual languages may have their own project “forges”, like RubyForge. Just find a project that looks interesting and read the code. This kind of “code spelunking” in a new language will give you valuable experience with the practical, day-to-day idioms used by experienced programmers, and will also give you solid examples of the concepts you read about and played with in the tutorials. (I intend to talk a lot more about “code spelunking” in the fourth article in this series.)
5. Lastly, think of a program or library you’d like to write, and write it in this new language. Nothing else you do will teach you the language as well as this step. You’re going to run into road blocks, but don’t get discouraged. If you need to do something that wasn’t covered in any of the tutorials, consider it an opportunity to read more documentation! Look at other projects and see how they accomplished similar tasks. Your first couple of programs in the new language will be very “unidiomatic”, but that’s how you start. Find someone experienced in the language and ask them to review your program, and point out better ways to do it.

By the end of the process, you won’t necessarily be an expert in the language, but you’ll know enough to compare it to other languages you know. You’ll also know whether you enjoyed the language enough to keep playing with it. Who knows? Maybe you’ll discover your new favorite language this way! I certainly wasn’t expecting to fall in love with Ruby when I started tinkering with it, but here I am, eight years later, and still tinkering.

Learning more

If you are already familiar with a language, the steps described above are not going to be as effective for increasing your knowledge. You’re probably in a rut, comfortable with what you know and able to accomplish almost everything you need to with it. That’s no excuse to stop learning, though; if you push yourself and learn even just a little more, you’ll almost certainly be surprised how useful that extra knowledge can be, even on a daily basis.

But how do you push yourself out of your comfort zone? This is probably harder than learning a new language, in many ways, because when you’re in the rut, it’s hard to see what the benefit of more knowledge would be.

One way is to find a good reference for the language, electronic or otherwise. Spend a little time each day (even just 15 minutes!) skimming that reference, looking for something you didn’t know before, or weren’t as familiar with as you would like to be. Once you find something, spend a little time playing with it. Find a way to add it to a project you’re working on. Blog about it. Tweet it. Just do something with it! Then, try to do something with it on a regular basis, so it stays in your memory.

Another technique is to find a library or program that uses concepts that are new to you. Unfamiliar with socket programming? Look for a networking library to read through. Want to be more familiar with threading techniques? Look for something that does a lot of parallel processing. Even if you have no specific learning goal, a good starting point is to select a programmer you admire, and find projects that they’ve worked on. You’re sure to find idioms, tips, tricks, and language features you’ve not seen before.

Ultimately, though, the game is not about “who can learn the most programming languages”. No one is keeping score. There is no tally scratched into the bedpost of your career, here. The goal is to educate your opinions. Awesome people have opinions born by experience. You want to be awesome? Then you need to be willing to keep pushing the envelope of your knowledge. Never settle for what you already have, even if you’re otherwise content. Keep looking. Lift the curtain on the unknown. If you want to be awesome, you need to explore.

Know thy languages.

There is no magic, there is only awesome (Part 2) »

Created at: 25.09.2009 17:15, source: the { buckblogs :here } - Home, tagged: Essays and Rants awesome keynote programming talk

This is the second article in a series titled “There is no magic, there is only awesome.” The first article introduced the “four cardinal rules of awesomeness”.

If you’ve ever watched someone make string figures, it’s pretty obvious that the tool set includes a loop of string, and your fingers. But if you haven’t played with string figures much, you might be surprised to learn that you’ve got a lot more than string and fingers at your disposal.

There are figures that require the use of your wrists to hold the string. Some require you to use your lips, teeth, tongue, or nose. I know a few that use the neck. There are some that use the elbow, knee, foot, or toes. A few require you to set the figure down on a flat surface to manipulate it.

There are many figures that require the use of another person, or several people. Some figures actually require multiple strings. Some require additional props, such as sticks.

Different figures might require different types of string. Some, like Eskimo figures, work best with a thicker, shorter, stiffer string, while those of the Pacific islands tend to prefer longer, more supple and slippery strings.

With so many variables, and so many ways of combining them, how then does one ever excel at string figures? Is it hopeless?

Of course it isn’t. As with any other activity, there are simply a set of tools to be employed, and the expert will be well acquainted with them. It’s rule #1 of being awesome: know thy tools.

Knowing vs. Knowing

But what does that mean? It certainly doesn’t mean just “knowing what your tools are”, otherwise you’d all be on your way to string figure mastery, just by reading the above paragraphs. Knowing your tools necessarily means knowing, deeply, how to use a tool. It means knowing the strengths and weaknesses of each tool, knowing why a particular tool is best in any given situation, and knowing how to learn more about them.

In fact, those statements can be generalized into four questions that can be used to gauge the depth of your knowledge about most anything:

1. What does this do best?
2. What does this do worst?
3. Why should I use this in particular?
4. When was the last time I learned something new about this?

We’ll hit those four questions in each of the articles that follow, but for now, I’m going to apply them specifically the tools of my own professional domain: writing software.

The tools of computing

If you write software, step back for a minute and think about the tools you use every day. This includes your operating system, your text editor, your command shell, your SCM. It’s your web browser, your terminal client, your email and IM clients. It includes command-line utilities you probably use without thinking, every day: grep, sed, awk, find, ls and cd, to name a few. It’s even your hardware: the keyboard you use, your mouse, your monitor, your computer. It probably includes many other things that I’ve completely overlooked! (Note, though, that programming languages will be covered separately; they are certainly tools of the trade, but they’re such spectacularly important tools that they deserve their own rule of awesomeness. More on that when we hit rule #2.)

So, now it’s your turn. Go ahead and grab a scrap of paper. Make a list, right now, of the tools you use every day. I’ll wait.

Done? Okay, awesome. Now, go through each item on your list and ask yourself each of the four “knowledge gauge” questions:

1. What does this do best?

Compared with similar tools, what does this tool do best? Don’t use a pre-recorded, generic answer. Think about it. Consider it carefully. Be specific. For particularly powerful or flexible tools, there may be multiple answers, and that’s okay. Your answer may quite possibly be debatable, and that’s okay, too. The point here is that you need to have an opinion. If someone asks you why in the world you choose to use Vim instead of Emacs, don’t let the silence stretch awkwardly. Know your mind!

2. What does this do worst?

Related to the previous question: At what is this tool particularly bad? Again, this is in comparison with similar tools. And there is no such thing as a perfect tool, so there’s no point trying to skip this question. That’s a cop out. If you don’t have a few pet peeves for each tool in your toolbox, then chances are you aren’t familiar enough with your tools. Have an opinion.

3. Why should I use this in particular?

Given all possible alternatives, why am I using this tool in particular? Was it recommended to you? Are you using it “by mandate”? Perhaps it was it a gift, or a hand-me-down? Have you personally tried any of the alternatives? The real test of this question is having someone ask you for a recommendation. If you were to recommend this tool, and they were to ask “why” in return, what would you tell them? Similarly, if someone were to ask you about one of the alternatives, would you be able to give them an educated opinion? (You do have one, right?)

4. When was the last time I learned something new about this?

In many ways, this is the most important of the four questions. It’s purpose is to make you aware of potential stagnation. Certainly there is a point at which you say can that you know a tool “well enough”. It gets your job done. It suffices. Perhaps you’re even something of an expert, with deep knowledge about the tool. But no matter how well you know a tool, if you stop learning and instead put your expertise on “cruise control”, you’ll stagnate. Expire. Rot. Guaranteed. To be exceptional, you must be active.

Don’t stagnate. Seriously, just don’t. If it’s been a while since you learned something new about one of your tools, take a good look at that tool. Look at its documentation. Find tutorials or blog articles. Books, videos, podcasts, whatever. Try to find something new about it, something that you didn’t know before. Perhaps it’s a feature you weren’t aware of. Maybe it’s just a new application of the tool. Perhaps-now, don’t freak out here-it’s another tool altogether! Never be afraid of replacing your tools when something better comes along. (It’s actually really, really valuable to continually experiment with similar tools, even if you’re satisifed with what you already have. It’s like learning a foreign language: it gives you a new perspective, and can introduce you to things, to which you might otherwise have never been exposed.)

Keep in mind, too, that “something new” does not have to mean “something mind-blowing”. In fact, such discoveries will be the exception. The point here is to simply be proactive, regularly flexing your knowledge muscles. And then, you must apply that new bit of knowledge in your workflow. Otherwise, it’s not really learned, is it? It’ll atrophy if you don’t exercise it.

Try to find at least one new thing per day. It may be cliche, but it’s also true: “practice makes awesome.” Or something like that.

Have an opinion.

These questions are intended to make you think about your tool set. It can be hard, at first, to step back and “meta-analyze” yourself and your habits, but you really can’t have a meaningful opinion if you don’t think about things. And trust me, you want to have opinions. People without opinions are, frankly, boring. They’re pretty much the opposite of awesome. Blindly accepting recommendations, mindlessly reciting others’ claims, and cargo-culting others’ solutions might be an okay place to start, if you must, but it’s always a lousy place to finish.

Get first-hand experience. Learn. Expose your hidden assumptions. Ask “why”. Have an opinion.

Know thy tools.

There is no magic, there is only awesome (Part 1) »

Created at: 17.09.2009 05:45, source: the { buckblogs :here } - Home, tagged: Essays and Rants awesome keynote programming talk

The following is the first of a series of articles that I will be posting in the coming weeks, based on the keynote address I gave at the 2009 Ruby Hoedown in Nashville, entitled “There is no magic, there is only awesome.” I originally intended to publish the entire series of articles as a single article, but it got too long. At any rate, I think it’ll be more easily digestible as multiple posts.

I’m always surprised to discover that there are people who have never heard of string figures. These are the games that are played (in western culture, at least) primarily by children, using a loop of string. They place the string on their hands and, either by themselves or with a friend, manipulate the string into various patterns. As a kid, I learned how to do a few such patterns, including Cat’s Cradle and the cup and saucer, but it was just a novelty, and I was interested in other things. I promptly forgot nearly everything about these games, except that they existed.

Fast-forward almost thirty years. My wife bought one of those Klutz books for our kids, this one about string games. It only described a handful of figures, but it was enough to pique my curiosity. I hopped online to see if there was any other information available about the subject.

Thus did the floodgates open! I discovered that string figures are a nearly universal pastime, being found in almost every culture around the world. In fact, many widely separated cultures share string figure repertoires—a discovery that fascinated and intrigued ethnologists over a century ago.

A Bit of History…

When Franz Boas first described an Eskimo string figure in 1888, there was a sudden flood of academic interest in string figures. Ethnologists began theorizing that the shared string figure repertoires could imply a common origin of mankind, or might indicate migration paths or intercultural interactions of “primitive” societies. Some of these figures were so elaborate that it seemed almost impossible to believe that they could come from independent invention. Efforts were made to “collect” these string figures by sketching them, photographing them, or even mounting the finished patterns on cardboard.

In 1902 there was another breakthrough: Drs. Rivers and Haddon, two ethnolgists, devised a way to describe the process by which string figures were constructed, allowing researchers to collect not only the finished products, but also the steps leading up to them. This helped reveal an interesting fact: many of these different cultures’ figures that looked the same superficially were actually produced by very different methods. The argument for independent invention was strengthened, but it also became possible to see where one culture may have learned a figure from another, by comparing the similarities between the two methods of construction.

A few years later, in 1906, a remarkable woman named Caroline Furness Jayne published one of the first books about string figures, called String Figures and How to Make Them. Amazingly, this book remains in print, and is still one of the definitive works on string figures. In this book, Jayne took the nomenclature devised by Rivers and Haddon, and with a few simple modifications, made it more accessible to the layman. Using this modified nomenclature, Jayne then proceeded to describe (and, with the help of a friend, illustrate) 97 string figures from around the world, many of which she had collected herself and which had never been published before.

It really is a spectacular book.

However, because the system for describing string figure construction was so new at the time, there was a large body of string figures that had been collected, but for which there was no known construction method. Among these figures were several from the Pacific island of Nauru. These Nauruan (which is a palindrome, making this a de facto awesome word) string figures were so elaborate and complex that Jayne was skeptical they could be created entirely “on the hand”, and she speculated that perhaps they were (at least partially) created “artificially” (e.g., by laying the string down and moving loops around).

Still, she included illustrations of 15 of these figures in her book (see the section entitled “Nauru Figures” in Chapter 9), and said:

“I have brought together such of these [drawings] as I could obtain, in the hope that other observers will find out the method by which they are made; with our present knowledge it is practically impossible to work back from the finished pattern to the opening movements.”—Caroline Furness Jayne, “String Figures and How to Make Them”, 1906 (emphasis added).

Although Jayne died just a few years after her book was published, her work remained influential. Years later it was read by a woman named Honor Maude, who was intrigued by the Nauruan patterns. When, in the 1930’s, she had the opportunity to visit Nauru for a few weeks, she immediately dove in and began interrogating the natives about their string figures. She was able to learn many (though not all) of the Nauruan figures from Jayne’s book, and many others besides.

It turns out that the figures were (for the most part) created entirely on the hand. For many of the figures, including fairly complex ones, the process of construction was even surprisingly straightforward. And armed with the results of Maude’s research, other researchers have since been able to reverse engineer the rest of the Nauruan patterns from Jayne’s book.

The mystery was dispelled. There was no magic. There were only awesome patterns that could now be documented, described, and duplicated.

This could never have happened if it were not for the willingness of a few to explore. Late 19th and early 20th century ethnologists explored the world, travelling to places most people had never even heard of. Caroline Jayne explored both the body of existing research, as well as travelled the US herself, collecting figures. And Honor Maude explored much of the south Pacific, including Nauru, collecting string figures.

More recently, string figurists have explored in other ways. By “playing with string”, attempting variations on existing patterns and testing combinations of maneuvers, and by working backward from known patterns, they were able to unravel figures that Caroline Jayne called “practically impossible” to reverse engineer. These people explored their domain, understood it at a new level, and did the impossible.

These people understood that string figures were not magic. String figures were only awesome.

Being Awesome

It is human nature to look at something exceptional and feel distanced from it. But calling that feeling of distance “magic” does the exceptional a disservice. It’s a barrier we build to give us an excuse for not being exceptional ourselves. It’s not magic that separates the exceptional from the mundane: it’s awesomeness.

“Awesome” is excellence. “Awesome” is the willingness to dig into one’s domain, to explore what has been discovered, and to look beyond. “Awesome” is understanding what you do, and how and why you do it. You cannot be awesome until you look inside that black box and dispel the magic around it.

As a string figurist and a computer programmer (and someone generally aspiring to awesomeness!), I suggest that there are four cardinal rules to being awesome.

1. Know thy tools.
2. Know thy languages.
3. Know thy libraries.
4. Know thy communities.

Now, when I say “know” here, I mean more than just “know about”. I mean the deeper, intuitive knowledge that comes from prolonged first-hand experience. It’s one thing to say that you know what is inside a cave. It’s another to actually go spelunking.

And it’s another entirely to meticulously explore a cave, acting as cartographer and biologist, geologist and poet, cowboy and scientist, all in one.

Part of meticulously exploring your domain, then, is asking questions. Make no assumptions. Have your own opinions. Look under every rock, and behind every corner. Ask why five times for every question.

When I worked at BYU, my boss at the time, Doug Walker, liked to hold what he called “obnoxious question sessions”. In them, he would ask “obnoxious” questions that challenged the assumptions we had made about our systems. These could be painful and awkward if we didn’t have good answers for them. It became obvious, quickly, that we needed to have opinions about everything we did, and that we shouldn’t take anything for granted. And neither should you.

The articles that follow in this series will each be a kind of obnoxious question session. They might be painful. They might be painfully obvious! Regardless, I’ll be taking each of those four cardinal “rules of awesomeness” one at a time.

And I hope you’ll bear with me as I drag string figuring into each one!

LEGOs, Play-Doh, and Programming »

Created at: 09.11.2008 13:46, source: the { buckblogs :here } - Home, tagged: Essays and Rants copland dependency-injection java needle net-ssh rubyconf

This article is based on a talk I gave at the 2008 RubyConf in Orlando, Florida, entitled “Recovering from Enterprise: how to embrace Ruby’s idioms and say goodbye to bad habits”.

The other day I went to Target with my son. Like most kids, I think, he’s convinced that Target is a toy store, which just happens to sell towels and shoes and cleaning supplies, too, so in his eyes it’d be criminal to not walk through the bare handful of toy aisles.

Besides, the toy section is across from the electronics section, which all geeks know is where the real toys are.

So, we went to the toy section and started browsing. I’ve always loved LEGO sets, and it’s a good thing they’re so expensive or I’d come home with a new box of bricks every time. At the Target near our home, they have half of an entire aisle devoted to boxes and boxes of LEGO sets. Need a battle-axe-wielding LEGO dwarf figure? A LEGO shark? How about a giant LEGO skull, a la Indiana Jones? And who could pass a LEGO Star Wars’ Star Destroyer model without a wistful thought or two?

It struck me at that time, though, how incredibly specific so many of these pieces are. With all of those sets in your possession, you could build a secret agent headquarters with a boulder trap that crushes angry battle-axe-wielding dwarves as they drive by in Martian exploration buggies. Which themelves are adorned with flower beds and creeper vines. And you could do all that in under 10 LEGO bricks! (Or, maybe a few more than that.)

Did you know that LEGO currently produces over 900 distinct LEGO pieces, or “elements” as they call them? Over the course of their history, there have been almost 13,000 distinct elements created. Now, that number includes variations in color and material, but even if you exclude those permutations, you’re still left with a staggering 2,800 different elements in the LEGO line.

It’s interesting that LEGO tends to encourage the use of specific pieces, rather than letting you build those pieces from more fundamental parts. It means that in order to master LEGO brick building, you have to know all of the pieces available to you, and have a good intuitive feel for how and when they should be used. That’s…a lot of information to keep tabs on. Myself, I just keep to the standard rectangular blocks and plug an exotic or two on as an afterthought when I see one that looks cool.

Also, if you’ve built up a model, and decide later that you want to change or extend some part of the model, you’ll often have to dismantle part (or all!) of it in order to do so. Kind of a pain.

Regardless, I still love building with LEGO bricks, and I suspect I always will.

Play-Doh

Now, my son being all of 6 years old, his attention span requires us to spend no more than a few minutes in any one toy aisle. So, long before I was ready to tear my eyes away from the LEGO sets, we found ourselves in the next aisle over. This was a much more colorful aisle, with bright pastels coloring various pre-school toys. My son, though, has nothing against pre-school toys, and was more than willing to drag me through them.

My eyes caught on the Play-Doh section.

The Play-Doh section at this Target is small, maybe 8 different hangers and a few square feet of shelf-space. You can get Play-Doh in as many as 50 different colors, but regardless of color, it’s all still the same thing: a bucket of malleable dough that you can pound, press, pinch, roll, and sculpt. (And rub into the carpet. And hair. And clothes. But we won’t get into that.)

Honestly, Play-Doh has a bad rap as a pre-schooler toy. It’s remarkably fun to play with. You can do all kinds of things with Play-Doh that you just can’t do with LEGO bricks. For example, the other day I built an arch out of cubes of Play-Doh that were held together only by friction. (You may not be impressed, but my 6-year-old was.)

The best part is that it doesn’t require so much memorization to become proficient in Play-Doh, though it might require more of an artistic streak than LEGO bricks do. Since I’m more engineer than artist, my Play-Doh creations tend to come out blocky and functional, rather than elegant and designed, but then, so do my LEGO creations.

Also, where LEGO models require significant work to alter or extend, Play-Doh models are dead-simple. If you want to add something to the base of your model, just graft more Play-Doh onto it. Want to change the shape of the keystone of your arch? Just pinch and mold in place. Simple!

Interestingly, I’ve found that while you can’t build with LEGO bricks using Play-Doh construction techniques, you can build with Play-Doh using LEGO construction techniques. Just build bricks out of Play-Doh. It’s unwiedly and impractical, but it can be done. The real question is: why would you want to? It’s pretty obvious that to build with Play-Doh, you should just embrace Play-Doh’s own strengths and run with it.

As obvious as that may seem, the lesson didn’t click for me for a long time. It’s not that I went about building Indiana Jones sets out of Play-Doh, one brick at a time. Rather, I didn’t realize that the same lesson applied to programming languages.

Java and LEGOs

Consider Java. Most would consider it the poster child of “enterprise” environments (though .NET is giving it a run for its money). And would you believe, Java and LEGO bricks have several things in common?

As of Java 1.6, there are well over 11,000 different classes and interfaces available to programmers in the standard library. (That’s not even counting the inner and anonymous classes that are usually not publicly documented.) Eleven. Thousand. Classes.

This is readily apparent when you consider the set of collection implementations that Java ships with.

• Collection
• Set
• List
• Queue
• Deque
• Map
• SortedSet
• SortedMap
• NavigableSet
• NavigableMap
• BlockingQueue
• BlockingDeque
• ConcurrentMap
• ConcurrentNavigableMap

• HashSet
• TreeSet
• LinkedHashSet
• ArrayList
• ArrayDeque
• LinkedList
• PriorityQueue
• HashMap
• TreeMap
• LinkedHashMap

• WeakHashMap
• IdentityHashMap
• CopyOnWriteArrayList
• CopyOnWriteArraySet
• EnumSet
• EnumMap

• ConcurrentLinkedQueue
• LinkedBlockingQueue
• ArrayBlockingQueue
• PriorityBlockingQueue
• DelayQueue
• SynchronousQueue
• LinkedBlockingDeque
• ConcurrentHashMap
• ConcurrentSkipListSet
• ConcurrentSkipListMap

• AbstractCollection
• AbstractSet
• AbstractList
• AbstractSequentialList
• AbstractQueue
• AbstractMap

Yes, that is FORTY-SIX different interfaces and implementations related to collections. Now, just like LEGO construction, having this volume of distinct elements on hand affects how you architect things. Writing software becomes more of a smorgasbord, where you pick and choose the specialized bricks you need, fitting them together just so. It also means that, in order to master Java, you need to have that intuitive grasp of how and when to use those thousands of classes. When do you use a HashSet versus a TreeSet? When would you use an ArrayDeque, and when would you want to subclass an AbstractQueue? It’s all part of the job.

Also, IDE’s are popular with Java in part because of the pain of refactoring. If you want to extend or modify a Java application, it can involve (like LEGO models) a lot of dismantling and reassembling.

Ruby as Play-Doh

But if Java is the LEGO of programming languages, then it could be argued that Ruby is the Play-Doh. Just as Play-Doh has been typically considered a pre-school toy, so Ruby has had a bad rap as a “toy” language, not fit for the “real world”. Also, compared to Java’s library of 11,000 classes, Ruby’s meager 1,400 classes (which number does include internal and anonymous ones, but not modules) seems paltry. And collections! Look what Ruby has to offer:

Modules:
  * Enumerable
  * Comparable (for elements within a collection)

Classes:
  * Hash
  * Array
  * Set
  * SortedSet

Just 6 options, to Java’s 46. What if you need a queue? Well, Ruby’s Array class has a queue-like interface; you could just use that. What about a sorted map? In that case, you might need to make do with a sorted set, or you could write your own, but it’s not hard. Most data structures are not rocket science, and for those that are, you can bet someone else has implemented it already.

But when you need to extend or modify your application, Ruby is a dream. Like Play-Doh, you can often just “pinch and mold” in place, grafting new code on or pulling old code out.

Ruby’s philosophy is like that of Play-Doh’s: provide a basic set of tools and make it relatively simple to build something complex with them. The very Ruby language itself is designed for this: closures, super-simple introspection of objects, runtime modification of existing objects, and the use of modules for extending classes and objects all tend to result in an environment that is simple, malleable, and extensible.

And just as you cannot use Play-Doh construction techniques with LEGO bricks, you also really cannot use Ruby programming techniques with Java. Using closures for delayed execution, or iteration, is tricky (at best) in Java, when it’s possible at all. Extending objects at runtime typically requires bytecode modification. And Ruby’s use of modules to extend classes and objects, while similar to both inheritance and interfaces, is slightly different (and arguably more powerful) than either.

You can write Ruby programs using Java programming techniques, but just as using LEGO techniques with Play-Doh is unwieldy and overcomplicated, so is mimicking Java in Ruby.

This is the lesson that I was slow to learn.

Copland

Consider exhibit A, from my Copland library.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

# copland/configuration/loader.rb (collapsed)
module Copland
  module Configuration
    class Loader
      attr_reader :search_paths
      attr_reader :loaders

      def initialize( search_paths=[] )
      def add_search_path( *paths )
      def add_loader( loader )
      def load( options={} )
      def load_path( path, options )
      def use_library( name )
    end
  end
end

Copland was my first stab at a dependency injection (DI) framework, and is more-or-less a feature-for-feature port of the HiveMind project in Java. (Ironically, it was the subject of my first presentation at a Ruby conference, in 2004!)

It was designed to automatically scan directories in the load path for YAML configuration files (I’ll mention those shortly), and load them up and parse them. The thing is, I imagined a case where someone might want to use XML instead of YAML. I couldn’t just leave these folks behind! So I made the whole configuration loading framework extendible. Want XML config files? Fine! Just implement an XML parser system and register it with the configuration loader framework, and you’re good to go!

That’s just wrong on so many levels. Always, always, always build just what you need, and only when you need it. You’re in Ruby, the Play-Doh of programming languages, and the cost of adding features later is really, really low. Remember YAGNI! Obviously, this principle holds in Java, too, but it really seems like the opposite philosophy has become the standard among many Java projects. It’s too bad, because it has contributed to a bad reputation that Java probably doesn’t entirely deserve.

Here’s a classic Java pattern that just really doesn’t translate to Ruby:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

# copland/class-factory.rb
module Copland
  class ClassFactory
    include Singleton

    def initialize
      @pools, @constructors = Hash.new, Hash.new
    end

    def create_pool( name, &block )
      block ||= proc { |k,*args| k.new( *args ) }
      @pools[ name ] = Hash.new
      @constructors[ name ] = block
    end

    def get_pool( name )
      pool = @pools[ name ] or raise NoSuchPoolException, name
      return pool
    end

    def register( pool_name, name, klass )
      pool = get_pool( pool_name )
      pool[ name ] = klass
    end

    def get( pool_name, name, *args )
      pool = get_pool( pool_name )
      klass = pool[ name ]
      raise NoSuchRegisteredClassException, "#{pool_name}:#{name}" unless klass
      constructor = @constructors[ pool_name ]
      return constructor.call( klass, *args )
    end
  end
end

This is an implementation of a class factory. In Ruby. The HiveMind project had a class factory, so the Copland project needed one, too!

But you know, class factories are absolutely pointless in Ruby. There are plenty of reasons for these in Java, but they just aren’t necessary in Ruby. Want a namespace? Declare the class in a module. Want the class to exist in multiple namespaces? Use constant assignment within whatever modules you desire. Need a dynamic lookup? Try #const_get. In the very worst case, just use a Hash if you need to map arbitrary strings to classes.

1
2
3
4
5
6
7
8
9
10
11
12
13

module A
  class B
  end 
end

# method #1, use const_get to dynamically look up classes
name = "B" 
klass = A.const_get(name)
object = klass.new

# method #2, use a hash to map arbitrary strings to classes
map = { "bimpl" => A::B }
map["bimpl"].new

Seriously. You don’t need explicit class factories in Ruby, because anything can be a class factory, implicitly.

I’ll mention one more painful Javaism that I ported to Copland. It’s so painful that I won’t even bother pasting it here—if you’re following along, look at examples/solitaire-cipher/lib/package.yml in the copland distribution.

If you do, what you’ll see are 106 lines of YAML describing how different Ruby objects in a simple, 250-line program should be initialized and connected. Yes. 106 lines of YAML. For 250 lines of Ruby.

Now, don’t get me wrong. YAML can be great for configuration. Rails, for instance, uses it for database connection information. The problem here, in Copland, was that I was using a static configuration for what would be better served with a block of code. Ruby reads elegantly; a YAML configuration file does not.

Fortunately, those wiser than myself showed me the way.

RubyConf 2004

I still remember Rich Kilmer, sitting in the front row in the October 2004 RubyConf. As I wrapped up my presentation on Copland and dependency injection, I asked if there were any questions.

Rich raised his hand. “Why didn’t you just use Ruby?”

I was confused by his question, and he had to explain. Why did I use YAML instead of just doing the configuration in Ruby code?

I think I mumbled something like “that would be a neat idea”. To me, it was a novel concept. I’d never heard of it before. You’d never see a Java program that was configured by writing Java code. That screams “hard coding”! But Ruby, you see, is different.

Ruby lets you write these beautiful little mini-languages. You’ll hear them called “Domain Specific Languages”, or DSL’s. They are subsets of the Ruby language, and you’ll find them in Rake, Capistrano, rspec, shoulda, and more. They’re really everywhere in Ruby, to varying degrees.

Although Rich tried to open my eyes, I think I would have continued to try and push Copland if it weren’t for Jim Weirich. Jim took the idea of a Ruby-ish DSL for dependency injection and made something concrete of it. A few days after the conference he forwarded me a draft of an article he was writing, in which he described dependency injection and gave a very simple (and very elegant) implementation of a DI framework in Ruby. Instead of static configuration, he’d written a basic DSL for declaring how the dependencies related to each other.

It was a moment of epiphany for me. Suddenly, I got it. I understood what DSL’s were about. I asked Jim for permission to take his simple implemention and build upon it.

The result was Needle.

Needle

Now, I’m much prouder of Needle than of Copland, because it is much closer to Ruby’s philosophy than Java. There are some pretty cool designs in there, too, though I use the term “cool” here to mean “neat without having any real practical application.”

Needle, though better, was still far from the mark.

As an example of why it misses the mark, consider Needle’s “pipeline” concept. Conceptually, it allowed you to specify a sequence of post-processors that operated on an object, allowing you to wrap code around it and mimicking (among other things) AOP-like operations. It also let me (as the author of the library) easily implement things like deferred instantiation, singleton services, and the like.

For example, suppose you wanted to declare a “deferred singleton” service, that logged all accesses to one of the methods. Underneath, Needle will create a pipeline of processors that operate on the service, returning a proxy object. The first time the proxy is accessed, it will check to see if the object has been instantiated yet. If it hasn’t, it’ll instantiate it (and cache it). The instantiation, though, actually just hands control to the next element in the pipeline, which in this case checks to see that the “singleton” constraint is enforced (e.g., all requests for this service return the same object, rather than instantiating a new object). The next pipeline element in the chain will wrap the interceptor code around the method in question, and yet another pipeline element would perform the actual object instantiation.

Pipelines really were pretty slick in Needle.

The problem, though, is that instead of leaving them as an implementation detail, I advertised them as one of Needle’s features. “Implement your own service models!” I cried. But, how often, really, is that likely to happen? Instead of exposing only the bare minimum of Needle’s API, I exposed as much of it as I could, because I could.

That’s a bad idea. Expose only what you need. The rest can be there, available, but not formally exposed. Only when (and if) you discover a need to expose more, should you expose more. This helps for several reasons.

• A smaller API is easier to describe, document, and support.
• A smaller API is easier for people to learn.
• A smaller API is easier for you to test.
• Extending a small API is much less onerous on your users than changing or restricting a larger API.

Net::SSH 1.x

Now, I’ve since come to my senses, but at one time I was completely head-over-heels in love with dependency injection. Like any schoolboy crush, it embarrasses me now to think about it, but there’s no denying it. The proof is everywhere in my project history.

Net::SSH, in particular.

At the time, I was looking for a good demonstration of the flexibility and power of dependency injection, and since Net::SSH was another of my pet projects at the time, it seemed like the perfect candidate.

I was still stuck in the “just in case” mindset, though, and Net::SSH 1.x reflected that. Badly. For instance, I isolated all the OpenSSL crypto interfaces into their own module, because “what if someone wanted to plug in a different crypto lib?” Nevermind that there was no other crypto lib for Ruby (and still isn’t, 4 years later). But WHAT IF?!?

Now, separation of concerns and modularity are good things, when used in moderation. But like any design pattern, it becomes evil when taken to extremes. Too much modularity and you wind up with component soup (and I hope you’re hungry, because you’re going to have a lot of it). With lots of tiny components, the interactions between those components can become difficult to test.

It also fuzzes the line between the public, documented API and the internal, private API. When you have two large components, it is very easy to say “A is public, and B is private”, but when you have two hundred components, where do you draw the line? It’s far too easy to let the “public” boundary meander a bit further into “private” territory than it should.

Even worse, when I added dependency injection to the mix, it became very, very difficult to follow the the flow of the program, and to understand the dependencies. Pull up Net::SSH 1.1.4, for instance, and find net/ssh.rb. Just try and figure out how a connection session is instantiated. It’s a mess. Unless you’re familiar with Needle, it’ll probably take you a long time to discover that the actual services are configured in the various services.rb files, but even after you figure that out, you still have to figure out how the different services interrelate. It’s a mess.

But, isn’t that the opposite of what DI is supposed to do? Isn’t DI supposed to improve the maintainability and testability of your code? Yeah. The problem, though, was three-fold.

First, Net::SSH, though complex in its way, was not really complex enough to need a dependency injection framework. DI itself adds complexity, and a framework for doing dependency injection adds even more, so before you go that route you need to be very sure that the trade-off in complexity is worth it. If your project is too small, you’ll actually increase the complexity of your project by adding a framework for doing DI.

Secondly, I was using a DI framework at a level that was really too granular. I was using the framework to wire together everything. No component was too small! No object too insignificant! I was on the dependency injection horse, and riding it for all it was worth. If I’d taken the time to really understand the pattern, though, I would have learned that though the pattern itself may be applied at the micro level, using a framework to do so is like nuking a mosquito—it works, but it leaves a mess behind.

Which leads to the last problem with Net::SSH’s use of Needle: it is really only appropriate for wiring together components of an application. Very, very few (Ruby) libraries will ever be complex enough, in themselves, to justify adding a dependency injection framework to them. Rather, let the application wire the libraries together as (and when) it needs to. Any more granular than that, and you’ll run into the same quagmire I did, I promise you.

Dependency Injection in Ruby

So, is there no room for DI in Ruby? There definitely is. I use DI nearly every day in Ruby, but I do not use a DI framework. Ruby itself has sufficient power to represent any day-to-day DI idioms you need. Consider this one:

1
2
3
4
5
6
7
8

class A
end

class B
  def new_client(with=A)
    with.new
  end
end

Here, B declares a factory method for generating new client objects. Because Ruby lets you declare default values for method arguments, you can let the default client implementation be A, which is the common case. But for testing, you can easily inject a mock into that method by passing an explicit parameter.

For cases where that doesn’t work, you can use a second factory method:

1
2
3
4
5
6
7
8
9
10
11
12

class A
end

class B
  def new_client
    client.new
  end

  def client
    A
  end
end

Then, in your tests, you can subclass B, overriding the client method to return your mock client implementation. It’s dependency injection, Jim, but probably not as you’ve known it.

Hashes, too, are your friend. You can allow optional arguments via hashes to specify implementation classes, defaulting to the standard implementation classes but allowing clients to inject their own implementations where needed:

1
2
3
4
5
6
7
8
9
10
11
12

class A
end

class B
  def initialize(options={})
    @client_impl = options[:client] || A
  end

  def new_client
    @client_impl.new
  end
end

“Loose coupling” and “high cohesion” are terms you’ll hear bandied about in defense of dependency injection, and those traits are certainly desirable. But strike a balance with pragmatism. There will be some who call me heretic for saying this, but don’t be afraid to introduce tighter coupling when it makes sense. Loose coupling everywhere is what I had with Net::SSH 1.x, and the result was nearly unmaintainable.

Be wise. You’re competent. Trust your instincts.

Lessons learned

If you read nothing else from this article, take to heart these bite-sized bullet-points:

• Direct translations are rarely accurate. Try using the Google translator to translate a paragraph from English, to Italian, to Japanese, and back to English, and you’ll see what I mean. The same is true of programming languages. Each language has it’s own idioms, and trying to take what works well in one language and force it directly into another language is doomed to fail, more often than not.
• Use your environment efficiently. Try as you might, you’ll never make a ball out of a LEGO brick by rolling it between your hands. You’ll just bloody your palms. Learn what your environment is capable of. Reading other people’s code is a great way to do this.
• DSL’s, not static configuration. Ruby excels at representing DSL’s. Whenever you can, consider using a DSL instead of static configuration for your applications. You’ll find it will simplify a lot more than it complicates.
• DI frameworks are unnecessary. In more rigid environments, they have value. In agile environments like Ruby, not so much. The patterns themselves may still be applicable, but beware of falling into the trap of thinking you need a special tool for everything. Ruby is Play-Doh, remember! Let’s keep it that way.
• Just in time. Not just in case. Don’t play “what if” games when you’re coding. Practice discipline, and implement only what you need, when you need it. You’ll wind up with tighter, more testable code that is easier to maintain in the long run.

Learning to program is a journey, and I’m still learning, myself. I’m not perfect at applying the rules above, but I’ve found that when I do, I’m much happier. I think you will be, too.