De-ASCII Your Rails Logs

Posted almost 7 years back at Alloy Code - Home

Recently, I've been delving into some filter-related problems in one of my Rails applications. This has required me to trap and review specific segments of my Rails log files. As part of my initial stack setup when I begin a new project, I install the Query Analyzer and Query Trace plugins.

The upside is I get very detailed trace information for my application, pictured below:

Log output

The downside is, when these files are viewed with Textmate or Console, they wind up looking more like this:

    Rendered events/_event (0.00036)  
      [4;35;1mSlot Load (0.000361)   SELECT * FROM 'slots' WHERE (slots.event_id = 23)   
        [35;2mvendor/plugins/query_analyzer/lib/query_analyzer.rb:38:in 'select'  
        [35;2mlib/association_extensions/chronological.rb:9:in 'first'  
        [35;2mapp/models/event.rb:109:in 'begin_at'  
        [35;2mapp/views/events/_event.html.erb:6:in '_run_erb_47app47views47events47_event46html46erb'  
        [35;2mapp/views/events/index.html.erb:39:in '_run_erb_47app47views47events47index46html46erb'

Today's "stretch the brain" exercise centered around creating a Textmate command to clear out this unnecessary cruft. For starters, I cloned the Text bundle's "Remove Unprintable Characters in Document / Selection" command. It looks like the Textmate folks are using Perl to remove unprintable characters:

perl -pe 's/[^\t\n\x20-\xFF]|\x7F|\xC2[\x80-\x9F]//g'

I left all the settings unchanged:
Save: Nothing
Input: Selected Text or Document
Output: Replace Selected Text

And edited the Perl regular expression to match each of the three possible variants of ASCII instruction:

perl -pe 's/\[\d;?(\d+)?;?(\d)?m//g'

Success! Now, with one keystroke, I can remove all that extra information from my log file, and get down to business.

Rackspace trouble knocks 37signals offline [back!]

Posted almost 7 years back at Loud Thinking

Rackspace has had a major power incident at the data center keeping the 37signals suite of machines. Apparently, a traffic accident knocked power out to some vital cooling systems. When the power was restored through generators, the cooling systems failed to come back online.

They now have the cooling systems back up and are getting everything back online. We hope to be back very shortly.

No data has been harmed, the machines were preemptively shut down when the cooling systems failed.

A good number of other applications, such as Wesabe was hit by this as well. Hopefully they'll be back shortly as well.

UPDATE: Everything is back in working order. Read more on the product blog.

Rackspace trouble knocks 37signals offline [back!]

Posted almost 7 years back at Loud Thinking

Rackspace has had a major power incident at the data center keeping the 37signals suite of machines. Apparently, a traffic accident knocked power out to some vital cooling systems. When the power was restored through generators, the cooling systems failed to come back online.

They now have the cooling systems back up and are getting everything back online. We hope to be back very shortly.

No data has been harmed, the machines were preemptively shut down when the cooling systems failed.

A good number of other applications, such as Wesabe was hit by this as well. Hopefully they'll be back shortly as well.

UPDATE: Everything is back in working order. Read more on the product blog.

Digital Comics, Unlimited

Posted almost 7 years back at zerosum dirt(nap) - Home

I’m not ashamed to admit that I’m a bit of a comic book geek.

When I was a kid, I was a hardcore fanboy who spent pretty nearly all his paper route money on weekly trips to the local comic shop in Dover. I have no apology for that. As a technology geek also, I’ve built a social comic book pull list organizer that has yet to officially launch (we’re looking for someone who’s excited about comics and community building, no techie skills required, email me if you’re interested!) and I’ve done a little ancillary plugin work for the geeky folks over at Heavy Ink too.

These days I buy mostly graphic novels, but I’ve always loved the feel of a floppy, and back when I used to buy titles monthly, there was just something unspeakably awesome about ‘new comic book day’ and the excitement of pouring through the stacks in a local shop looking for that one key back issue…

Like most fans, I have mixed feelings about the digitization of comics. On one hand, I’d probably read a lot more of them if they were available (ahem, legally of course) through a cheap, easy to use digital distribution service like iTunes. On the other hand, I’m sort of glad this hasn’t happened, because it would kill a lot of the allure of it for me if the comics on paper were to become a thing of the past.

I imagine I’m not the only one who was both excited and, at the same time, just a little tiny bit bummed about the announcement of Marvel Comics Unlimited yesterday (NOTE: as of right now the site appears to be down for maintenance, coming soon). The basic premise is that Marvel will offer an online archive of over 2500 back issues online in high-resolution format, starting at about $5/mo. They’re also making a free sampler of 250 titles available to wet your appetites.

Marvel’s hedging their bets a little though, which is smart. New issues won’t appear on the Marvel Unlimited site until six months after their initial print publication. This is great for those of us who read graphic novels, and aren’t used to picking up books every week anyway, and it keeps the floppies in circulation. After a bit of consternation, I’ve decided that it’s pretty much a win/win. Of course, as it always is on the web, the user experience will determine the ultimate success or failure of the venture. The flash-based digital comics I’ve seen from Marvel up to this point haven’t exactly been the most pleasant things to read, so here’s to hoping this is an entirely new interface to the library.

Anyway, I’m looking forward to reading more about this, both as a comic book enthusiast and as a technologist. Welcome to the 21st century, comic fans. For better or for worse.

It’s about time, I guess!

Turn On Your Buzzword Filter

Posted almost 7 years back at zerosum dirt(nap) - Home

BigLove [12:37am]: my eyes crossed from all the buzzwords zapnap
zapnap [12:38am]: turn on your buzzword filter. OMFG WHERE DID T EH INTERNETS GO?

Episode 79: Generate Named Routes

Posted almost 7 years back at Railscasts

This more advanced episode will show you how to dynamically generate named routes by adding a method to the map object.

Werewolves in Charlotte

Posted almost 7 years back at Troubleseeker - Home

I recently attended an amazing conference, the RubyConf 2007 (held in Charlotte this year). The conference was held in the very quiet town Charlotte. The city’s downtown appeared so dead (where were all the pedestrians?), that it would have made a wonderful setting for a zombie movie a la 28 days later. So while we were running around aimlessly in the hopes of finding a place where you could buy a lawnmower (thank you Peter Jackson!), we had to come realizing that preparing against zombies wouldn’t help very much. We were facing a much greater evil (no, not dinosaur-riding Germans): Werewolves! Lots of them. Very very many. Most of the conference attendees actually (did THEY all eat the zombies?).

Werewolf, also known as Mafia is a game about cunning and accusing but mainly about surviving. The only thing you need is a deck of cards. The 6-16 players are divided randomly into two groups: villagers and werewolves. A narrator is supervising the game and informs the players when the night or day begins. At night, the werewolves get to kill one villager. During the day, everyone (including the werewolves disguised as villagers) is discussing and eventually voting on who to lynch merely based on suspicions. If the villagers happen to lynch a werewolf then they are one step closer to victory and prove the Spanish Inquisition right. If they manage to kill all the werewolves they win. The werewolves win if at one point there are as many villagers as there are werewolves.

The rules are really simple. The game is stressful but fun. Even if you end up playing a villager, you still need to convince all the others of your innocence. You can compare it to going through customs in certain countries: you do not smuggle anything, you do not intend to commit any crimes, you don’t have anything to hide – and yet, after the tenth question that points to a specific item on your 5th birthday while you’re spending the family’s vacation in Cuba you start to get wobbly knees.

OS X Leopard Upgrade Notes

Posted almost 7 years back at zerosum dirt(nap) - Home

Starting this blog entry to document any possible gotchas I experience on my upgrade to OS X 10.5. So far everything has been relatively clean. My only problem as of this writing has been with SSH and MacPorts. I have SSH installed via MacPorts and I noticed when doing any SVN+SSH operations, I was getting the following error:

percent_expand: NULL replacement
svn: Connection closed unexpectedly

Hmm, looks like Leopard is expanding some SSH environmental variable to NULL. An easy fix is to add the following line to your ~/.ssh/config:

IdentityFile ~/.ssh/id_rsa

This overrides the default search for your private key, and therefore you don’t get the percent_expand error. If anyone has more information about this particular issue, please let me know. Thanks!

Links For 11.10.07

Posted almost 7 years back at zerosum dirt(nap) - Home

Stuart Halloway - Ruby on Rails Podcast

Posted almost 7 years back at Ruby on Rails Podcast

Robert Stevenson interviews Stuart Halloway at eRubyCon in Ohio.

NH Ruby Next Monday Night

Posted almost 7 years back at zerosum dirt(nap) - Home

As Scott notes over on his blog, this month’s NH Ruby meeting will be off by a week and a day this time around, due to holiday stuff. Instead of the third Tuesday of the month, it’ll be the second Monday. Yessir, that’s this coming Monday.

My good friend Dave Berube will be the main speaker, and he’ll be discussing reporting techniques with Ruby and Rails. Dave is putting the finishing touches on a new book on the subject, so he knows of that which he speaks. It’s sure to be a good conversation and if you’re in the area, you should definitely check us out.

Flexible Payments With Rails & REST

Posted almost 7 years back at zerosum dirt(nap) - Home

Anyone who’s been playing around with the original Rails version of the Amazon Flexible Payments Service sample application has probably spent a fair amount of time swearing and complaining about how incredibly un-idiomatic and ugly it is. I know I have. It’s not that we don’t appreciate Amazon wanting to play ball with us, it’s just that it’s… well… it’s damn ugly.

But wait, a few weeks ago, a new REST-based Rails sample app was posted. This time around it’s a far more basic “hello world”-ish app, but it’s much more idiomatic and clean, a far better starting place for anyone who wants to learn how to make pay calls with FPS using Ruby and Rails. It’s also thankfully devoid of soap4r dependencies.

Just thought I’d note that, in case you’re working with FPS and haven’t stumbled upon it yet.

In related news, there’s a RubyForge project for a library called Remit, which purports to be a proper Ruby API wrapper for FPS. Tyler Hunt is the developer who’s registered the project and is working on it. With no updates since early September though, and no files released, I wonder if the project hasn’t maybe been aborted? Tyler, if you’re out there, give a shout and let us know if you need a hand.

Slow queries: Rails habtm gotcha

Posted almost 7 years back at poocs.net - Home

Back in the old days when some brave souls started with Rails, the framework would automatically supply helper methods to check the availability of associated records for every has_many or has_and_belongs_to_many association in the form of has_foos?.

For example, given a declaration like this:


class User
  has_and_belongs_to_many :interests
end

would get you User#has_interests? back in the days.

Fast forward to the present, these methods have long been removed from Rails core. There’s no 1:1 replacement, really. When you want to check a collection in a conditional for whether or not it’s empty, there are several things you can do.

Association Proxies, a quick review

Basically, every collection is a derivative of the Array class, as such most of Arrays methods have been (re-)implemented for the respective collection proxies.

First of all, there’s collection#length, which is the most brutal of them all because it simply loads the whole collection into memory and runs #size on it.

Next up is collection#count, which uses an SQL COUNT() statement to just fetch the number of associated records and is usually the quickest way at that.

Finally, there’s collection#size, which is basically a combination of both length and count since it behaves differently depending on whether or not the collection has already been loaded at the time it is being called. Given the collection has already been loaded (by a prior statement in your code, what have you) it’s identical to collection#length. If it hasn’t been loaded, its behavior is identical to collection#count.

Best practice

Judging from the rundown above, it’s usually best practice to simply use collection#size and trust it to do the right thing. In case you’re planning to use a collection as soon as you found out there are records associated with it, use collection#length to load the connection right away and then loop over it. Beware, however, that this is advisable only for those cases that don’t use pagination. (You’re using will_paginate for that, aren’t you?)

In a conditional, it makes most sense to use collection#empty?, which uses size#zero? internally.


unless user.interests.empty?
  # ...
end

The problem

99% of what I said above applies to both has_many and has_and_belongs_to_many associations. Admittedly, poor little habtm has had its momentum and seems to be on the way out of the cool kids’ block. Still, there are a few cases where it’s still the best fit (such as the User#interests example from first section of this article) and some people *do* have Rails code that’s been in existence for a few years. (Well, maybe only I do.)

So, the 1% that behaves differently is that size on a habtm collection is not as intelligent as its little brother from the has_many camp. In fact, it’s pretty dumb because it’s in every case behaving identical to length meaning it’s loading the whole collection. Ouch.

>> user.interests.size
  Join Table Columns (0.001284)   SHOW FIELDS FROM interests_users
  Interest Load (0.001263)   SELECT * FROM interests
    INNER JOIN interests_users
    ON interests.id = interests_users.interest_id
    WHERE (interests_users.user_id = 142841 )

The solution

You have to fill in the gaps of habtm#size’s lack of intelligence yourself. If you trap over habtm queries that take way too long for your taste, it might be about time to swap out that size call for an always speedy count call.

>> u.interests.count
  Interest Columns (0.001006)   SHOW FIELDS FROM interests
  SQL (0.009749)   SELECT count(*) AS count_all FROM interests
    INNER JOIN interests_users
    ON interests.id = interests_users.interest_id
    WHERE (interests_users.user_id = 142841 )

You would then also avoid using collection#empty? in your conditionals, since that would then resort back to using size internally. Instead, use the count#zero? form or define your own instance methods on the association.


if user.interests.count.zero?
  # ..
end

Postscript

To actually find out what Rails is transforming your method calls into (in terms of generated SQL) I prefer to use this technique by the always-wise Jamis Buck. Redirecting the ActiveRecord logger to the shell’s standard output will intermix the SQL statements with the output of your method calls, making everything relevant visible at first glance.

0.6.3 not quite there

Posted almost 7 years back at The Hobo Blog

It turns out that some of the changes in 0.6.3 have totally broken the “out of the box” Hobo app (which TBH is something we don’t look at very much over here). It’s getting into a better shape now – hopefully tomorrow (TM)

0.6.3 not quite there

Posted almost 7 years back at The Hobo Blog

It turns out that some of the changes in 0.6.3 have totally broken the “out of the box” Hobo app (which TBH is something we don’t look at very much over here). It’s getting into a better shape now – hopefully tomorrow (TM)