Ruby, Rails, Micropayments, and Amazon FPS

Posted about 7 years back at zerosum dirt(nap) - Home

Last week Amazon unveiled a beta of their new Flexible Payments Service, a potential Paypal-killer and Google Checkout-killer, among other things. At first glance, FPS seems to be everything that Paypal isn’t: well designed, API-centric, and built with developers in mind.

While everyone is busy barking about how Amazon is going to go head to head with Google Checkout and Paypal for purchases, they seem to be missing the more interesting development here. I’m talking about something that neither Paypal or Google are even attempting to do, as far as I know: micropayments.

Ty, Steve, and I spent some serious time putting together a few different iterations of an idea/prototype earlier this year, that involves micropayments and the street performer protocol. And some other stuff, too :-). Unfortunately, when it came time to tie it into a payment gateway, it became painfully obvious that what we needed didn’t really exist, so we compromised. And that compromise led to a bunch of problems and eventually what we felt was an unworkable solution. So it’s been in mothballs since. This looks like it could be what we were maybe waiting for.

If you check out Amazon’s rate schedule you can see why I’m excited:

For Amazon Payments balance transfers < $0.05:
20% of the transaction amount, with a minimum fee of $0.0025

OK, OK, so 20% is a hefty fee but not when we’re talking about the alternative being somewhere in the neighborhood of $0.30 USD + 2-3%. So yeah, pfft. This is HUGE. And yes, it does look like these micropayments only work when you’re using Amazon payments (as opposed to using a credit card, etc) but so what? I’m not so sure that’s an obstacle.

Another reason it’s so interesting to me is that they’ve decided to support three languages/platforms in their initial beta rollout: Java, PHP, and yes, my friends, Ruby. Oh, and the sample app? Yep. Rails.

Why aren’t we more excited about this?

Flex vs HTML vs Time

Posted about 7 years back at work.rowanhick.com

The old simple triangle of time vs features vs quality has reared it's ugly head again. After one particularly unproductive weekend I've made the decision to place a hold on the Flex interface for XChain, in favour of a raw HTML interface, sprinkled with AJAX. I personally hate the decision, being enamored with Flex, I don't want to make it, but I look at the forward progress I've made in the past couple of weeks and it's been more or less stationary. I'm not trying to deliver the 'richest' interface possible, I'm going to deliver a working application. What impacts my client most, is that they can get their job done. Granted I am learning Flex, and grappling with the Cairngorm/No Cairngorm decision. I decided to go ahead with Cairngorm, particularly after reading the usual tales of woe 'i decided not to use it, now I've rebuild my app and am using it..'. However holy sh-t do you have to produce a lot of code to get things done, okay there are code generators out there, but still. For a pet project this blows out the window the time required to do the job. Sure if this was a 40hr work week project I'd be writing a completely different post right now. This makes me appreciate the elegant nature of Ruby and Rails™ so much more. It just gets out of your way and lets you get on with producing productive code. In just one hour I can do so much more. Firing up textmate I feel instantly at home. Is this a language familiarity thing, or a complexity problem, or am I using the right tool for the right job, within the given constraints, (probably not). I had a big discussion about it with a colleague at work, and it always boils down to trade-offs and compromises, there's no one magic silver bullet (unfortunately). Rails is fantastic at backend work, and helps some with front end work, Flex has some great time savers, rich interface components like datagrid which are cake to wire up and get working - but then you're spending a tonne of time on event and ui management. Where is the happy median ? Something like a super rich js framework ? What am I sorely missing from flex in going 'back' to HTML ? - Rich components - Consistent styling - Reduced network traffic A random thought floating through my mind is to publish it in AIR, that way I can still use an HTML interface hooked up to my back end, but a few extra features that might be very useful (drag n drop support, offline storage). More to investigate.. Grumpy but moving forward.

mms2r 1.1.4 Released

Posted about 7 years back at Mike Mondragon

mms2r version 1.1.4 has been released!

  1. DESCRIPTION:

MMS2R is a library that decodes the parts of an MMS message to disk while
stripping out advertising injected by the cellphone carriers. MMS messages are
multipart email and the carriers often inject branding into these messages. Use
MMS2R if you want to get at the real user generated content from a MMS without
having to deal with the cruft from the carriers.

If MMS2R is not aware of a particular carrier no extra processing is done
to the MMS other than decoding and consolidating its media.

Contact the author to add additional carriers to be processed by the
library. Suggestions and patches appreciated and welcomed!

Corpus of carriers currently processed by MMS2R:

  • AT&T => mms.att.net
  • AT&T/Cingular => mmode.com
  • Cingular => mms.mycingular.com
  • Cingular => cingularme.com
  • Dobson/Cellular One => mms.dobson.net
  • Helio => mms.myhelio.com
  • Nextel => messaging.nextel.com
  • Sprint => pm.sprint.com
  • Sprint => messaging.sprintpcs.com
  • T-Mobile => tmomail.net
  • Verizon => vzwpix.com
  • Verizon => vtext.com

Changes:

  1. 1.1.4 / 2007-08-07 (Dr. Rockso)
  • AT&T => mms.att.net support (thanks Mike Chen and Dave Myron)
  • get_body returns nil when there is not user text (sorry Will!)

mms2r 1.1.4 Released

Posted about 7 years back at Mike Mondragon

mms2r version 1.1.4 has been released!

  1. DESCRIPTION:

MMS2R is a library that decodes the parts of an MMS message to disk while stripping out advertising injected by the cellphone carriers. MMS messages are multipart email and the carriers often inject branding into these messages. Use MMS2R if you want to get at the real user generated content from a MMS without having to deal with the cruft from the carriers.

If MMS2R is not aware of a particular carrier no extra processing is done to the MMS other than decoding and consolidating its media.

Contact the author to add additional carriers to be processed by the library. Suggestions and patches appreciated and welcomed!

Corpus of carriers currently processed by MMS2R:

  • AT&T => mms.att.net
  • AT&T/Cingular => mmode.com
  • Cingular => mms.mycingular.com
  • Cingular => cingularme.com
  • Dobson/Cellular One => mms.dobson.net
  • Helio => mms.myhelio.com
  • Nextel => messaging.nextel.com
  • Sprint => pm.sprint.com
  • Sprint => messaging.sprintpcs.com
  • T-Mobile => tmomail.net
  • Verizon => vzwpix.com
  • Verizon => vtext.com

Changes:

  1. 1.1.4 / 2007-08-07 (Dr. Rockso)
  • AT&T => mms.att.net support (thanks Mike Chen and Dave Myron)
  • get_body returns nil when there is not user text (sorry Will!)

Introducing: Your Garage Online

Posted about 7 years back at Alloy Code - Home

Alloy Code is thrilled to announce the launch of a new web application. Your Garage Online can help you manage everything from your vehicle's routine maintenance to those major and minor repairs.

Features

Virtual garage to store your cars, including year, color, make, model, license and vehicle identification number data.
Searchable repair forms to look up any repair or maintenance record, regardless of when or where it was performed.
Comprehensive service history outlines work done by professional shops or in your own back yard.

YourGarageOnline.com was originally conceived as a Rails Day activity, but grew in scope beyond the confines of a single, 24 hour development project. The site was designed, coded and tested over a six week period, as an off-hours personal project and proof of concept application. The finished product serves as a demonstration of the quality and craftsmanship Alloy Code can deliver for our clients.

The site is coded in Ruby on Rails, using RESTful application design and leveraging cutting edge Web 2.0 technology.

Introducing: Your Garage Online

Posted about 7 years back at Alloy Code - Home

Alloy Code is thrilled to announce the launch of a new web application. Your Garage Online can help you manage everything from your vehicle's routine maintenance to those major and minor repairs.

Features

Virtual garage to store your cars, including year, color, make, model, license and vehicle identification number data.
Searchable repair forms to look up any repair or maintenance record, regardless of when or where it was performed.
Comprehensive service history outlines work done by professional shops or in your own back yard.

YourGarageOnline.com was originally conceived as a Rails Day activity, but grew in scope beyond the confines of a single, 24 hour development project. The site was designed, coded and tested over a six week period, as an off-hours personal project and proof of concept application. The finished product serves as a demonstration of the quality and craftsmanship Alloy Code can deliver for our clients.

The site is coded in Ruby on Rails, using RESTful application design and leveraging cutting edge Web 2.0 technology.

FragmentFu - Fun with Fragments

Posted about 7 years back at Revolution On Rails

First, please read the Advanced Rails Caching.. On the Edge for a primer on ESI, caching, and fragments.
To get started developing with ESI, I'm going to walk through a simple tutorial building an ESI enabled Rails application.
Get Mongrel-ESI
Find the latest mongrel ESI here: http://code.google.com/p/mongrel-esi/downloads/list


wget http://mongrel-esi.googlecode.com/files/mongrel-esi-0.0.5.gem
(Does anyone have a better solution of ruby projects hosted on Google Code? )

Install Mongrel ESI

sudo gem install mongrel-esi-0.0.5.gem

Create a new rails application (Currently using rails 1.2.3)

rails fragment_fu_demo

create
create app/controllers
create app/helpers
create app/models
.....

cd fragment_fu_demo


Create a home page for your application


Delete the public/index.html page

rm public/index.html

Edit the config/routes.rb and uncomment the following line:

map.connect '', :controller => "welcome"

Create a welcome controller

./script/generate controller welcome index


Install the FragmentFu plugin
   
./script/plugin install http://mongrel-esi.googlecode.com/svn/trunk/plugin/fragment_fu


Start your application

 
./script/server

visit http://localhost:3000 in your browser. You should see a "Welcome#index" page.

Start MongrelEsi
From the command line, run:

mongrel_esi start
** Starting Mongrel listening at 0.0.0.0:2000

visit http://localhost:2000 in your browser. You should see a "Welcome#index" page.

Create a fragment to cache
Edit the app/controllers/welcome_controller.rb
Add an action called now
 
def now
render :text => "#{Time.now} is #{Time.now.usec} nano-seconds"
end

Edit the app/views/welcome/index.html.erb and replace with the following:


<h1>Welcome</h1>
<p><%= render :esi => "/welcome/now" %></p>


Page without ESI parsing
visit http://localhost:3000 and you should see: "Time:" in the browser. If you view the source, you'll see


<h1>Welcome</h1>
<p><esi:include src="/welcome/now" max-age="0"/></p>


This is the simplest of esi tags, and the default max-age is 0, which means do not cache.

Page with ESI Parsing

visit http://localhost:2000 and you'll see:

Welcome
Date: Mon Aug 06 00:00:00 -0400 2007 which is 62972ns


Caching modules
Edit the index.html.erb and replace with the following:


<h1>Welcome</h1>
<p><%= render :esi => "/welcome/now" %></p>
<h2>What time is it again?</h2>
<p><%= render :esi => "/welcome/now"%></p>


If you refresh http://localhost:2000, you'll notice they nanoseconds are different

Lets add a ttl to first call


<h1>Welcome</h1>
<p><%= render :esi => "/welcome/now", :ttl => 45.seconds %></p>
<h2>What time is it again?</h2>
<p><%= render :esi => "/welcome/now"%></p>


If you refresh http://localhost:2000, you'll notice they are now the same. Refresh again in the next 45 seconds, and it will not change.


The next tutorial will cover a small TODO application, with inline invalidation and exception handling. Coming Soon!

FragmentFu - Fun with Fragments

Posted about 7 years back at Revolution On Rails

First, please read the Advanced Rails Caching.. On the Edge for a primer on ESI, caching, and fragments.
To get started developing with ESI, I'm going to walk through a simple tutorial building an ESI enabled Rails application.
Get Mongrel-ESI
Find the latest mongrel ESI here: http://code.google.com/p/mongrel-esi/downloads/list


wget http://mongrel-esi.googlecode.com/files/mongrel-esi-0.0.5.gem
(Does anyone have a better solution of ruby projects hosted on Google Code? )

Install Mongrel ESI

sudo gem install mongrel-esi-0.0.5.gem

Create a new rails application (Currently using rails 1.2.3)

rails fragment_fu_demo

create
create app/controllers
create app/helpers
create app/models
.....

cd fragment_fu_demo


Create a home page for your application


Delete the public/index.html page

rm public/index.html

Edit the config/routes.rb and uncomment the following line:

map.connect '', :controller => "welcome"

Create a welcome controller

./script/generate controller welcome index


Install the FragmentFu plugin
   
./script/plugin install http://mongrel-esi.googlecode.com/svn/trunk/plugin/fragment_fu


Start your application

 
./script/server

visit http://localhost:3000 in your browser. You should see a "Welcome#index" page.

Start MongrelEsi
From the command line, run:

mongrel_esi start
** Starting Mongrel listening at 0.0.0.0:2000

visit http://localhost:2000 in your browser. You should see a "Welcome#index" page.

Create a fragment to cache
Edit the app/controllers/welcome_controller.rb
Add an action called now
 
def now
render :text => "#{Time.now} is #{Time.now.usec} nano-seconds"
end

Edit the app/views/welcome/index.html.erb and replace with the following:


<h1>Welcome</h1>
<p><%= render :esi => "/welcome/now" %></p>


Page without ESI parsing
visit http://localhost:3000 and you should see: "Time:" in the browser. If you view the source, you'll see


<h1>Welcome</h1>
<p><esi:include src="/welcome/now" max-age="0"/></p>


This is the simplest of esi tags, and the default max-age is 0, which means do not cache.

Page with ESI Parsing

visit http://localhost:2000 and you'll see:

Welcome
Date: Mon Aug 06 00:00:00 -0400 2007 which is 62972ns


Caching modules
Edit the index.html.erb and replace with the following:


<h1>Welcome</h1>
<p><%= render :esi => "/welcome/now" %></p>
<h2>What time is it again?</h2>
<p><%= render :esi => "/welcome/now"%></p>


If you refresh http://localhost:2000, you'll notice they nanoseconds are different

Lets add a ttl to first call


<h1>Welcome</h1>
<p><%= render :esi => "/welcome/now", :ttl => 45.seconds %></p>
<h2>What time is it again?</h2>
<p><%= render :esi => "/welcome/now"%></p>


If you refresh http://localhost:2000, you'll notice they are now the same. Refresh again in the next 45 seconds, and it will not change.


The next tutorial will cover a small TODO application, with inline invalidation and exception handling. Coming Soon!

Advanced Rails Caching.. on the Edge

Posted about 7 years back at Revolution On Rails

When trying to scale a portal to millions of users with complex personalization, we had to rethink the application design. We needed partial page cacheability, and we wanted that close to the Edge... and recently the idea has been catching on. First with Components Are the New Black, and the idea of Nginx, SSI, and Memcache, what I'd like to briefly describe below is a partial page caching strategy that has worked well for the past year.

First, we use a technology called ESI. ESI stands for Edge Side Includes, which is a simple markup language for describing dynamic assembly of applications. At the core, its similar to SSI, but its a more versatile spec that has been accepted/implemented by a half dozen cache servers, both open source (Squid, Mongrel-ESI) and commercial (Oracle Web Cache, Akamai, among others). It also includes an invalidation protocol, exception handling on the edge, and a few other features.

The ESI W3C specification has been out for 6+ years, so these ideas are not new, but ESI is a diamond in the rough. No one seems to be using it.

A simple example of ESI in your rails application is including a common header. If the header is static, using a shared layout, or rendering a shared partial across applications, could be sufficient. But if you use the common idiom of a sign-in bar, like "Welcome Bob - Signout", on a page you want to fully cache, or a common header you want to share across multiple applications, then you may consider another approach.

If you leverage ESI, you can put the below in your application layout.

<div id="header">
<esi:include src="/header" max-age="300"/>
</div>


The <esi:include> tag will inject the response from /header into the page and cache that fragment for 300 seconds. You can also add a variety of options, request header parameters (which can be used to personalize the request) along with a dozen other optional parameters, some of which I will outline below. This is simple SSI (Server Side Includes) .

But if we take it a step further, lets look at how we can apply it to a site like Twitter.

Twitter


If you look at the image above, the majority of the page can be fully cached. I've outlined two green boxes, which appear to be dynamic content, which can have different TTLs. Using ESI, your markup could be:

<div id="latest">
<esi:include src="/latest" max-age="5"/>
</div>
...
<esi:include src="/featured" max-age="3600"/>


The ESI enabled cache server would parse this markup, make 2 separate HTTP requests (/latest, and /featured) and cache those with their corresponding TTLs. You can furthermore cache the wrapping template with a Surrogate-Control header, which tells the cache server to keep a cached copy of the template. A request to this page 4 seconds later would make 0 requests to your rails infrastructure. 8 seconds later would only hit the /latest, returning the rest of the page from cache. You can also envision a application pool of servers just to handle /latest, but I'll get into sharding applications via ESI in a later article.


Exception Handling

If you take this a step further, you can also define timeouts and exception behavior. As defined below, If the /latest request takes more than 1s, The cache server will give up, and retrieve the static snippet defined in the except block from your static web host.

<esi:try>
<esi:attempt>
<esi:include src="/latest" max-age="5" timeout="1"/>
</esi:attempt>
<esi:except>
<esi:include src="http://static.foo.com/latest" max-age="5" timeout="1"/>
</esi:except>
</esi:try>


We call these "Sorry" modules, because they often say "Sorry, this feature is having trouble", but the rest of the page may surface meaningful content. Even better, write out more meaningful content to disk once a day and serve that from Apache.


Invalidation


The other benefit of ESI is Invalidation support. There are various mechanisms to invalidate content, but my favorite is Inline Invalidation. Consider the common rails idiom of updating data. You post to a controller which redirects to a view of that data. Since the HTTP redirect (301) bubbles all the way back to the browser, any content in the body of the redirect can be parsed by the ESI server. Therefore you can put the invalidation xml is the redirect response body and not conditionally dirty your view logic.

<esi:invalidate>
<?xml version="1.0"?>
<!DOCTYPE INVALIDATION SYSTEM "internal:///WCSinvalidation.dtd">
<INVALIDATION VERSION="WCS-1.1">
<OBJECT>
<BASICSELECTOR URI="/foo/bar/baz"/>
<ACTION REMOVALTTL="0"/>
</OBJECT>
</INVALIDATION>
</esi:invalidate>


The above is a simple single URL example, but the specification supports regex, which would work well for restful resources (e.g. /people/#{person.id}/*), among other things.


FragmentFu


FragmentFu is a plugin that enables ESI support in rails applications. At this point it is an extraction from our internal plugin for modules, caching, and ESI. Its in active extraction/development, but I wanted to involve the community in gathering ideas, suggestions and comments.


Some sample snippets:
<% render :esi => widget_url(1) %>

<%= render :esi => latest_url, :ttl => 5.seconds, :timeout => 1.second %>

<%= render :esi => featured_url, :ttl => 10.hours, :except => '/some/static/path' %>


We've also toyed around with the idea of fragment respond_to's. Since ESI supports adding request header parameters, we can mimic Prototype's
X-Requested-With behavior to implement:

def latest
...
respond_to |wants| do
wants.html { do something }
wants.fragment { do something }
end
end



How to get started


Mongrel-ESI is a great starting place. It was built in-house and released a few months ago on Google Code. It supports a large subset of ESI, ESI Invalidation, and its a great tool for developing applications that utilize ESI. You can grab Mongrel-ESI, download the Oracle Web Cache Standalone or even Squid 3.0... and when you're site gets really big, its time to give Akamai a call. :)

Advanced Rails Caching.. on the Edge

Posted about 7 years back at Revolution On Rails

When trying to scale a portal to millions of users with complex personalization, we had to rethink the application design. We needed partial page cacheability, and we wanted that close to the Edge... and recently the idea has been catching on. First with Components Are the New Black, and the idea of Nginx, SSI, and Memcache, what I'd like to briefly describe below is a partial page caching strategy that has worked well for the past year.

First, we use a technology called ESI. ESI stands for Edge Side Includes, which is a simple markup language for describing dynamic assembly of applications. At the core, its similar to SSI, but its a more versatile spec that has been accepted/implemented by a half dozen cache servers, both open source (Squid, Mongrel-ESI) and commercial (Oracle Web Cache, Akamai, among others). It also includes an invalidation protocol, exception handling on the edge, and a few other features.

The ESI W3C specification has been out for 6+ years, so these ideas are not new, but ESI is a diamond in the rough. No one seems to be using it.

A simple example of ESI in your rails application is including a common header. If the header is static, using a shared layout, or rendering a shared partial across applications, could be sufficient. But if you use the common idiom of a sign-in bar, like "Welcome Bob - Signout", on a page you want to fully cache, or a common header you want to share across multiple applications, then you may consider another approach.

If you leverage ESI, you can put the below in your application layout.

<div id="header">
<esi:include src="/header" max-age="300"/>
</div>


The <esi:include> tag will inject the response from /header into the page and cache that fragment for 300 seconds. You can also add a variety of options, request header parameters (which can be used to personalize the request) along with a dozen other optional parameters, some of which I will outline below. This is simple SSI (Server Side Includes) .

But if we take it a step further, lets look at how we can apply it to a site like Twitter.

Twitter


If you look at the image above, the majority of the page can be fully cached. I've outlined two green boxes, which appear to be dynamic content, which can have different TTLs. Using ESI, your markup could be:

<div id="latest">
<esi:include src="/latest" max-age="5"/>
</div>
...
<esi:include src="/featured" max-age="3600"/>


The ESI enabled cache server would parse this markup, make 2 separate HTTP requests (/latest, and /featured) and cache those with their corresponding TTLs. You can furthermore cache the wrapping template with a Surrogate-Control header, which tells the cache server to keep a cached copy of the template. A request to this page 4 seconds later would make 0 requests to your rails infrastructure. 8 seconds later would only hit the /latest, returning the rest of the page from cache. You can also envision a application pool of servers just to handle /latest, but I'll get into sharding applications via ESI in a later article.


Exception Handling

If you take this a step further, you can also define timeouts and exception behavior. As defined below, If the /latest request takes more than 1s, The cache server will give up, and retrieve the static snippet defined in the except block from your static web host.

<esi:try>
<esi:attempt>
<esi:include src="/latest" max-age="5" timeout="1"/>
</esi:attempt>
<esi:except>
<esi:include src="http://static.foo.com/latest" max-age="5" timeout="1"/>
</esi:except>
</esi:try>


We call these "Sorry" modules, because they often say "Sorry, this feature is having trouble", but the rest of the page may surface meaningful content. Even better, write out more meaningful content to disk once a day and serve that from Apache.


Invalidation


The other benefit of ESI is Invalidation support. There are various mechanisms to invalidate content, but my favorite is Inline Invalidation. Consider the common rails idiom of updating data. You post to a controller which redirects to a view of that data. Since the HTTP redirect (301) bubbles all the way back to the browser, any content in the body of the redirect can be parsed by the ESI server. Therefore you can put the invalidation xml is the redirect response body and not conditionally dirty your view logic.

<esi:invalidate>
<?xml version="1.0"?>
<!DOCTYPE INVALIDATION SYSTEM "internal:///WCSinvalidation.dtd">
<INVALIDATION VERSION="WCS-1.1">
<OBJECT>
<BASICSELECTOR URI="/foo/bar/baz"/>
<ACTION REMOVALTTL="0"/>
</OBJECT>
</INVALIDATION>
</esi:invalidate>


The above is a simple single URL example, but the specification supports regex, which would work well for restful resources (e.g. /people/#{person.id}/*), among other things.


FragmentFu


FragmentFu is a plugin that enables ESI support in rails applications. At this point it is an extraction from our internal plugin for modules, caching, and ESI. Its in active extraction/development, but I wanted to involve the community in gathering ideas, suggestions and comments.


Some sample snippets:
<% render :esi => widget_url(1) %>

<%= render :esi => latest_url, :ttl => 5.seconds, :timeout => 1.second %>

<%= render :esi => featured_url, :ttl => 10.hours, :except => '/some/static/path' %>


We've also toyed around with the idea of fragment respond_to's. Since ESI supports adding request header parameters, we can mimic Prototype's
X-Requested-With behavior to implement:

def latest
...
respond_to |wants| do
wants.html { do something }
wants.fragment { do something }
end
end



How to get started


Mongrel-ESI is a great starting place. It was built in-house and released a few months ago on Google Code. It supports a large subset of ESI, ESI Invalidation, and its a great tool for developing applications that utilize ESI. You can grab Mongrel-ESI, download the Oracle Web Cache Standalone or even Squid 3.0... and when you're site gets really big, its time to give Akamai a call. :)

Episode 65: Stopping Spam with Akismet

Posted about 7 years back at Railscasts

The Railscasts site has been getting a lot of comment spam in the past, but no longer. In this episode I will show you how I solved this problem by using the Akismet web service.

Distributed SCMs: Be Smart, Use the Bleeding Edge

Posted about 7 years back at Wood for the Trees

I’ll preface this title by admitting I’ve not paid much attention to my version control software. When you’re smart (cue 16-ton weight), you try to think ahead and choose software or hardware which will vastly reduce the effort of programming. While staying on the edge with most things, for version control I neglected to really think about it. I had a very brief and painful time with CVS and promptly converted to Subversion because it was already installed, everyone used it, it was much easier, it solved the immediate problems with CVS I had, and it did what I needed to do at the time.

Of course, in programming, we all know that “what I needed to do” is never the same as “what I need to do now”. I have a lot more code to manage, I have a lot of projects which I am developing on my own or manage or take a serious part in, and I have more projects which I want to dip my feet into. The number of snippets, little things here and there, and feedback or changes from other people has meant a lot of branches, tags, svn cp and other management issues.

Now, this might sound like a “shiny thing” moment, but it is completely different from shiny thing syndrome. My advice to practically everyone reading this is to keep moving at the cutting edge of version control… and that doesn’t mean upgrading to Subversion 1.4.4. It means that there are genuine advances in version control which go well beyond Subversion and well beyond what it will ever be. I’ve known about distributed SCMs for a while, but haven’t really bothered looking into it. I’m writing this post because I think a lot of people are in the same boat. Subversion is supported, if not as the default SCM, practically everywhere: Sourceforge, Rubyforge, Google Code, Unfuddle, Lighthouse, etc. Every respectable project hosting service uses Subversion and almost every plugin, gem or developer out there at the moment references to Subversion repositories. Well, I’m going to make a serious splash in this department and say: it’s time to move on from Subversion. I have completely grown out of it, and I’m not even managing a codebase as large as Rails (or even ActiveRecord).

The single biggest development in version control over the last umpteen years, but specifically the last 2, is distributed version control. Basically, that means everyone’s working copy is a repository, every commit is local and every ‘commit’ to another repository is a ‘push’ (every ‘update’ from another repository is a ‘pull’)—that means merging is the default behaviour when you communicate with another repository. In other words, every repository is a branch. This way, distributed version control doesn’t force a single source code management architecture. These systems are completely open and the advantages to this model-less model are (obviously) endless:

  • most importantly, distribution is all about caring more about the coder-to-code relationship than the coder-to-project relationship
  • Distributed SCMs are orders of magnitude faster in almost every department
  • they usually use a bit more space
  • actual commits are almost instant, since they are local
  • merging is simplified enormously—it’s the daily business of a DSCM
  • no need for an update—you merge when ready
  • development can happen anywhere—no connection required
  • no default model of management or hierarchy
  • much more intuitive code structure—do what you like
  • no need for access control, but it’s there if you need it
  • etc.

You’ll see the benefits just in the development of your own code, without merging with anyone else’s repositories but your own, that distribution of management doesn’t require distribution of code. You can use any methodology, any hierarchy, any paradigm of managing your code that you wish.

Returning to my original point, staying at the bleeding edge doesn’t mean upgrade to the next minor or major version to get the latest feature. It means upgrade, or change software if necessary, to use the latest breakthroughs which will cut costs and save time for your own projects or your company’s projects.

Now, you’re probably asking which distributed SCM to use. I personally go for Mercurial for the following reasons:

  • SVK is a lie—it is not distributed; it just distributes SVN, which is like a multiplying a herd of turtles
  • much faster than Arch, Darcs and Bazaar
  • only Git is faster—but that was developed by Linus Torvalds
  • Mercurial and Git are relatively younger, meaning they’ve learnt from the mistakes of other DSCMs
  • Mercurial and Git come from the same background (the BitKeeper drama) and BitKeeper, according to Torvalds, was the only SCM worth using
  • Mercurial compares feature-for-feature with other DSCMs, and then some
  • Mercurial uses less space than others
  • very user friendly CLI—by far the easiest to learn of the DSCMs
  • decent documentation
  • a few project hosting sites already (like ShareSource)
  • ability to have Mercurial support using the HTTP front-end, even where you normally wouldn’t have it (i.e. Sourceforge and Rubyforge)
  • If you don’t have a feature in Mercurial, find or code a Mercurial extension—it is fully customisable

My Forceful Conclusion

Like the language you are using (Ruby 1.8), the framework (Rails 1.2), the testing suite (RSpec 1.0 and Mocha), the operating system (Mac OS X 10.4 and Linux 2.6), the web server (nginx, LiteSpeed, or lighttpd), the application server (LiteSpeed or Mongrel), the blogging software (Typo or Mephisto), even the hardware (Amazon S3/EC2 or many dual core 2+ GHz servers with many GBs of RAM), you need to use the latest version control software (Mercurial 0.9.4 or Git 1.5). If you don’t, it’s like using an old server with 333MHz & 256MB RAM, or Apache 1.3 & mod_ruby. But unlike an out-moded application server, SVN will make you a slower developer than the one sitting next to you using Mercurial, Git, Darcs or Bazaar. Oh, and I needn’t mention that he’ll get all the babes too.

Distributed SCMs: Be Smart, Use the Bleeding Edge

Posted about 7 years back at Wood for the Trees

I’ll preface this title by admitting I’ve not paid much attention to my version control software. When you’re smart (cue 16-ton weight), you try to think ahead and choose software or hardware which will vastly reduce the effort of programming. While staying on the edge with most things, for version control I neglected to really think about it. I had a very brief and painful time with CVS and promptly converted to Subversion because it was already installed, everyone used it, it was much easier, it solved the immediate problems with CVS I had, and it did what I needed to do at the time.

Of course, in programming, we all know that “what I needed to do” is never the same as “what I need to do now”. I have a lot more code to manage, I have a lot of projects which I am developing on my own or manage or take a serious part in, and I have more projects which I want to dip my feet into. The number of snippets, little things here and there, and feedback or changes from other people has meant a lot of branches, tags, svn cp and other management issues.

Now, this might sound like a “shiny thing” moment, but it is completely different from shiny thing syndrome. My advice to practically everyone reading this is to keep moving at the cutting edge of version control… and that doesn’t mean upgrading to Subversion 1.4.4. It means that there are genuine advances in version control which go well beyond Subversion and well beyond what it will ever be. I’ve known about distributed SCMs for a while, but haven’t really bothered looking into it. I’m writing this post because I think a lot of people are in the same boat. Subversion is supported, if not as the default SCM, practically everywhere: Sourceforge, Rubyforge, Google Code, Unfuddle, Lighthouse, etc. Every respectable project hosting service uses Subversion and almost every plugin, gem or developer out there at the moment references to Subversion repositories. Well, I’m going to make a serious splash in this department and say: it’s time to move on from Subversion. I have completely grown out of it, and I’m not even managing a codebase as large as Rails (or even ActiveRecord).

The single biggest development in version control over the last umpteen years, but specifically the last 2, is distributed version control. Basically, that means everyone’s working copy is a repository, every commit is local and every ‘commit’ to another repository is a ‘push’ (every ‘update’ from another repository is a ‘pull’)—that means merging is the default behaviour when you communicate with another repository. In other words, every repository is a branch. This way, distributed version control doesn’t force a single source code management architecture. These systems are completely open and the advantages to this model-less model are (obviously) endless:

  • most importantly, distribution is all about caring more about the coder-to-code relationship than the coder-to-project relationship
  • Distributed SCMs are orders of magnitude faster in almost every department
  • they usually use a bit more space
  • actual commits are almost instant, since they are local
  • merging is simplified enormously—it’s the daily business of a DSCM
  • no need for an update—you merge when ready
  • development can happen anywhere—no connection required
  • no default model of management or hierarchy
  • much more intuitive code structure—do what you like
  • no need for access control, but it’s there if you need it
  • etc.

You’ll see the benefits just in the development of your own code, without merging with anyone else’s repositories but your own, that distribution of management doesn’t require distribution of code. You can use any methodology, any hierarchy, any paradigm of managing your code that you wish.

Returning to my original point, staying at the bleeding edge doesn’t mean upgrade to the next minor or major version to get the latest feature. It means upgrade, or change software if necessary, to use the latest breakthroughs which will cut costs and save time for your own projects or your company’s projects.

Now, you’re probably asking which distributed SCM to use. I personally go for Mercurial for the following reasons:

  • SVK is a lie—it is not distributed; it just distributes SVN, which is like a multiplying a herd of turtles
  • much faster than Arch, Darcs and Bazaar
  • only Git is faster—but that was developed by Linus Torvalds
  • Mercurial and Git are relatively younger, meaning they’ve learnt from the mistakes of other DSCMs
  • Mercurial and Git come from the same background (the BitKeeper drama) and BitKeeper, according to Torvalds, was the only SCM worth using
  • Mercurial compares feature-for-feature with other DSCMs, and then some
  • Mercurial uses less space than others
  • very user friendly CLI—by far the easiest to learn of the DSCMs
  • decent documentation
  • a few project hosting sites already (like ShareSource)
  • ability to have Mercurial support using the HTTP front-end, even where you normally wouldn’t have it (i.e. Sourceforge and Rubyforge)
  • If you don’t have a feature in Mercurial, find or code a Mercurial extension—it is fully customisable

My Forceful Conclusion

Like the language you are using (Ruby 1.8), the framework (Rails 1.2), the testing suite (RSpec 1.0 and Mocha), the operating system (Mac OS X 10.4 and Linux 2.6), the web server (nginx, LiteSpeed, or lighttpd), the application server (LiteSpeed or Mongrel), the blogging software (Typo or Mephisto), even the hardware (Amazon S3/EC2 or many dual core 2+ GHz servers with many GBs of RAM), you need to use the latest version control software (Mercurial 0.9.4 or Git 1.5). If you don’t, it’s like using an old server with 333MHz & 256MB RAM, or Apache 1.3 & mod_ruby. But unlike an out-moded application server, SVN will make you a slower developer than the one sitting next to you using Mercurial, Git, Darcs or Bazaar. Oh, and I needn’t mention that he’ll get all the babes too.

Hobo 0.6 release date

Posted about 7 years back at The Hobo Blog

It seems you’re all itching to know when we’ll have a Hobo 0.6 without the ‘pre’ tag :-)

Just to let you know we’re shooting for Tuesday the week after next – 14th August.

Hobo 0.6 release date

Posted about 7 years back at The Hobo Blog

It seems you’re all itching to know when we’ll have a Hobo 0.6 without the ‘pre’ tag :-)

Just to let you know we’re shooting for Tuesday the week after next – 14th August.