« December 2005 | Main | February 2006 »
After the war, Toyota revolutionized the way cars were built, first with their ideas of Just in Time inventory, and later by pushing control down to teams on the floor, capitalizing on the agility that enabled.
Detroit manufacturers noticed this, and sent representative to Toyota to see just how they did this. These invited spies brought back copious notes on what they saw, the the auto makers set about copying the Toyota process. Without exception, these early attempts at imitation failed: the car companies replicated what they saw Toyota doing, but only on the surface. They didn’t really understand what was behind these practices. It was like trying to become an artist by copying the angle and velocity of the brush held by a master.
Andy and I have tried to think differently about publishing. We came to it knowing nothing about the industry. We did, however, know how to make projects work. So we took what we knew and applied the principles, if not the practices, to what we did.
According to the feedback we get, we’ve caused something of stir in the publishing industry.
And it’s no surprise that other publishers do what the car companies did to Toyota. They try to copy. Just this month I’ve heard of two publishers who’ll be running beta-book programs (even, somewhat lamely, calling them Beta Books). And I just heard that a well-known technical publisher might be launching a series of cheap, PDF-only, books. (Perhaps they’ll call them Thursdays to show some originality.)
As the Toyota example showed, this kind of surface-level imitation is flattering, but ultimately it’s unlikely to succeed.
Behind the stuff that you see us doing, there’s an underlying philosophy and set of practices. They all reinforce each other. For example, the fact we have continuous builds and author-typesetting means we can create beta books that are living documents. The fact we have an errata system hyperlinked from these beta book pages means we can put feedback in the hands of our authors, and hence we can get updated revisions out faster. Each of these aspects of what we do is a small thing in isolation, but we have hundreds of them, and they all add up to a cohesive, and we feel revolutionary, whole. Copying just the visible aspects misses this depth.
For this reason, I honestly don’t mind other publishers blatantly ripping us off. But I’d rather they didn’t. Instead, I’d rather they found their own ways of innovating, and build their own ideas that others found useful. The publishing industry is in transition. It needs all the good ideas it can get. All publishers should contribute in their own way to the reshaping of the industry. Simply aping someone else’s success won’t help the community as a whole.
It started after Mike and I migrated a Rails app to a new, zippier server. Everything looked great (and the app ran a lot faster). However, after a while, I started getting problems with a database. To explain what went wrong, I need to chat a little about the application.
This piece of code is an attempt to migrate functionality away from a particularly nasty set of old CGIs, web applications, bailing wire, chewing gum, and other historical crud. There’s a lot of existing functionality, and I decided not to try to port it all across to Rails on day one. Instead, I gave the Rails application its own database to play with, and gave it access to the old legacy database for when it needed to look up stuff in the existing system. To do that, I created a set of ActiveRecord classes for the legacy tables, and explicitly connected them to the legacy database, rather than the default Rails application database. A typical legacy model class looked smething like this:
class LegacyOrder < ActiveRecord::Base
set_table_name "orders"
set_primary_key "o_id"
establish_connection "legacy_#{RAILS_ENV}"
...
end
(The "legacy_#{RAILS_ENV}" stuff lets me have development, test, and production versions of the legacy database in my database.yml file.)
So far, so good. I had four different legacy tables I mapped this way, and everything worked fine.
However, after a while, other applications accessing the legacy database started failing, claiming they couldn’t connect. Sure enough, it turned out that there were hundreds of active connections to that database: the database wasn’t going to accept any more.
The problem was that I was creating way too many connections to the legacy database. Each spawned Rails fcgi process established a separate connection to the legacy database for each legacy ActiveRecord class. That factor of four killed us.
But fixing it also raised an interesting question: in my original code, I established a separate connection for each legacy table in each running Rails process. This was clearly dumb. But what’s the best way of sharing a single connection between all of the legacy model objects in a Rails instance?
Some chatting on #rails-core came up wih three suggestions:
legacy_connection = ActiveRecord::Base.mysql_connection(...) LegacyOrder.connection = legacy_connection LegacyLineItem.connection = legacy_connection ...
class LegacyOrder < ActiveRecord::Base establish_connection ... end class LegacyLineItem < ActiveRecord::Base connection = LegacyOrder.connection end
Options (1) and (2) seemed messy: they added a bunch of coupling to the code where it wasn’t really justified. Option (3) seemed like the way to go. The only problem was that no one on #rails-core was sure it would work…
It turns out it does.
class LegacyBase < ActiveRecord::Base
establish_connection ...
end
class LegacyOrder < LegacyBase
...
end
class LegacyLineItem < LegacyBase
...
end
The parent class creates a single connection to the legacy database, and all the child classes share it. The reason it works is some smarts in Rails.
It turns out that Rails does just about everything lazily. That includes connecting to databases and reflecting on tables to extract the schema (needed to build the internals of the models). This improves performance, but it also makes this hack possible. In general, you’d expect the LegacyBase class to map to a database table called legacy_base. It would, if we ever tried to use it to access data. But because we don’t, and because Rails only reflects on the table the first time a data access occurs, we can safely create an ActiveRecord class with no underlying database table.
This scheme lets me specify the legacy connection once, and share that connection between all my legacy models. It’s tidy, expressive, and saves resources.
| htonl: | yeah we’ve canned agility here |
| htonl: | we’re doing the water fountain technique |
| htonl: | we mostly just spew code up in an arch and it falls down the drain |