DEISA Sessions (Distributed European Infrastructure for Supercomputing Applications)

Posted about 7 years back at Spejman On Rails

This week I'm taking DEISA training sessions in Barcelona, they are teaching us how to submit tasks into most powerful supercomputers of Europe (DEISA Architecture).

Within this supercomputers we can meet Marenostrum, nowadays is the most powerful supercomputer in Europe and the World's fifth most powerful as you can see in www.top500.org. It's made of 10.240 CPUs Power PCs, 20 TeraBytes of RAM and 370 TeraBytes of disk. It's placed in a old chapel which gives it a special feeling.

Today I've been lucky and I could visit inside Marenostrum as you can see in this photos:

Marenostrum

Marenostrum

Marenostrum Core

More photos: http://www.flickr.com/photos/spejman/tags/marenostrum/

Episode 2: Dynamic find_by Methods

Posted about 7 years back at Railscasts

Shorten simple finds considerably and improve readability by using the dynamic find_all_by and find_by methods.

Tweaking The Rails Logger

Posted about 7 years back at zerosum dirt(nap) - Home

The Ruby Logger is a simple but pretty flexible tool — hopefully you’re already using it. If not, you should be. The default logger message format is pretty barebones though, so I thought I’d take a few minutes to talk about how to make it more useful by monkey patching format_message.

First, make sure to check out TopFunky’s ‘Hodel 3000 compliant logger’ article, posted a couple months ago. Geoffrey’s syslog-friendly modification works great when running in production mode. Customized log messages are just as important in the development environment, where we’ve made our own simple modifications to include the name and line number of the file:

class Logger
  def format_message(severity, timestamp, msg, progname)
    "#{Kernel.caller[2]}: #{severity.upcase}: #{progname.gsub(/\n/, '').lstrip}\n"
  end
end

This little snippet is especially handy if you’re using an IDE like IntelliJ that’s smart enough to hyperlink the file path. Got an error being logged in users_controller.rb on line 91? Click on the hyperlink and you’re there.

It works by accessing the execution stack Array returned by Kernel.caller. caller0 will refer to the line in Logger#add where format_message is called. caller1 is most likely going to point to one of the Logger#<severity> methods, one of [ debug, info, warn, error, fatal ]. This is the next level down in the stack, where the add method was called from. In most “application-level” code (code we actually write in our application, exterior to the benchmarking messages and such that Rails gives us for free), caller2 is going to be something in our application code itself, some place that we called logger.info, or logger.error, or whatever, from.

To get this installed in our environment we create a file development_logger.rb, containing the source above. Then, in development.rb (or the initializer block of environment.rb):

require 'development_logger.rb'

This will “monkey patch” the Rails Logger, effectively overriding it’s format_message method with our own mojo. Sure, it’s not perfect. But it works great for development purposes. A more elegant way to do the same thing would be to subclass Logger and then do something like:

config.logger = DevelopmentLogger.new(config.log_path)

However, this just doesn’t seem to have any effect when running script/server, which proceeds as if config.logger is set to the standard (unmodified) Rails Logger. I’m at a bit of a loss as to why. If anyone can explain why setting config.logger seems to have no effect, please (please!) let me know.

Episode 1: Caching with Instance Variables

Posted about 7 years back at Railscasts

Learn a quick way to improve performance. Just store the end result of an expensive command in an instance variable!

New Ruby/Rails IDE Comparison

Posted about 7 years back at zerosum dirt(nap) - Home

Oh, what an IDE snob I’ve become over the past couple years. I was an advocate of lightweight text editors for dev work for a long time, and was only truly bitten by the “heavyweight” environment bug once I was introduced to IntelliJ, which I can’t say enough nice things about. I was obviously pretty psyched when I found out that the Jetbrains team has decided to put some serious effort into a Ruby plugin. It’s become very usable in a very short amount of time, and I’m very happy with it, but it’s still relatively new and lacking some features.

This new comparison of Ruby / Rails IDEs does a great job summarizing the features in the three leading “heavyweight” IDEs for Ruby / Rails. Definitely check it out if you’re in the market for a good IDE. Interestingly, it seems like NetBeans is really giving both IntelliJ and RadRails a real run for their money, being the first of The Big Three to get a reasonable level of code completion working, amongst a plethora of other features.

Although I’ll probably download it and take a test drive, I’m pretty committed to IntelliJ at this point, so I’ll probably stick it out. But hopefully a little competition will keep things moving along at a good clip :-).

UPDATE: Switched over to NetBeans months ago and loving it! Nice work guys.

attr_readonly

Posted about 7 years back at zerosum dirt(nap) - Home

I was really shocked yesterday to discover that there’s literally no baked-in way to declare an ActiveRecord attribute as private or protected. Sometimes I don’t want publicly-accessible ActiveRecord attribute methods. Sorry, but not every field in my database should have a corresponding public mutator.

Consider an IP address field on a User model, that’s used to record the last IP the user logged in from. This is something that, although clients outside of the model should be able to access the value, they certainly shouldn’t be able to set it. Only the model itself should be able to update that field in the database. It’s private!

As another example, consider the case where a database field is just a cache of calculated relationship data. That cached value should never be set directly via a public instance method. One example is an average_rating as calculated from a bunch of user ratings on an article. Why would you want to expose a mechanism to set this directly? It should only be set through some sort of recalculate_average_rating public method that lives on your article model and fires as an after_save filter on a new rating object.

It turns out that counter_cache itself suffers from this same problem. Ticket #6896 in the Rails Trac points out this exact issue and proposes a solution in the form of attr_readonly. This is a solid solution, and would go a long ways towards enforcing proper encapsulation in AR.

I ranted about this yesterday in freenode and got yelled at a little bit. Yes, I was probably being a bit obnoxious, but I also don’t want the importance of this to be overlooked. The bottom line is that, like it or not, people are largely stupid. Programmers are not excluded from this. Even Ruby programmers, who admittedly are more self-conscious than most. If you give someone the opportunity to do something stupid, be it at the user level in your app, within your API, or right within the rest of your code base, they’ll do it. If we’re writing for an audience > 1, we should be writing code like we design user interfaces — the public methods available on our classes shouldn’t include things that give our audience permission to twiddle with the guts of that thing. This is a cornerstone of encapsulation and good OO design.

So how do you feel about all this? Is it not A Big Deal to you? To me, it’s a significant limiting factor when it comes to organizing large code bases amongst multiple programmers. If you’re the only guy working on a hobby app, then fine, I guess you can police yourself. Go somewhere and do your thing. But if you’re part of a larger team, or hope to build the foundation for something that can grow into a real enterprise-class app, then proper encapsulation is mandatory.

I really hope that patch gets committed, or that someone puts together a plugin that delivers the same functionality. In the meantime, sure, there are some things you can do that are better than nothing at all. Try declaring a public mutator in your class to override the AR attribute method of the same name and raise NoMethodError, or write a before_save filter that loads up the model into another variable and replaces the current field with it’s previous value. But just describing those “solutions” makes me feel somewhat ill. Rails is such a clean, elegant platform, and it needs a clean, elegant way to provide non-public methods for AR attributes. Let’s get that patch committed :-).

Breaking into Orkut with Mechanize

Posted about 7 years back at schadenfreude

Alternate title: Using Mechanize to Scrape Orkut for Love Letters from Lonely Brazilian Beauties and Translating their Wantings to English

Shout outs to Natai for the request.

In this tutorial we will use mechanize to login to Orkut, pull our scrap messages, we will assume the message is written in Portuguese and translate the message to English with Google Translate.

FUN!

Breaking into Orkut with Mechanize

Posted about 7 years back at schadenfreude

Alternate title: Using Mechanize to Scrape Orkut for Love Letters from Lonely Brazilian Beauties and Translating their Wantings to English

Shout outs to Natai for the request.

In this tutorial we will use mechanize to login to Orkut, pull our scrap messages, we will assume the message is written in Portuguese and translate the message to English with Google Translate.

FUN!

Breaking into Orkut with Mechanize

Posted about 7 years back at schadenfreude

Alternate title: Using Mechanize to Scrape Orkut for Love Letters from Lonely Brazilian Beauties and Translating their Wantings to English

Shout outs to Natai for the request.

In this tutorial we will use mechanize to login to Orkut, pull our scrap messages, we will assume the message is written in Portuguese and translate the message to English with Google Translate.

FUN!

Axiom Of The Empty Set

Posted about 7 years back at zerosum dirt(nap) - Home

A list without any items in it is still a list, I must insist. Just like a box without anything in it is still a box, or a Smurf village without any Smurfs (Smurves?) is still a frickin Smurf villiage..

But not according to the W3C, apparently. File this one under Rant Of The Day: none of the standard XHTML doctypes acknowledge the existance of the empty list.

<!ELEMENT ul (li)+>

That is, neither the <ul></ul> or the <ul/> representations are valid XHTML. This seems pretty broken to me at first glance, although I’m certianly willing to hear a rebuttal if anyone has one. Even if you don’t think it’s busted, it certainly adds complication to a very common web dev scenario…

So we have a list of resources in our view and there’s a good possibility that our list is empty. We want to be able to display that list, and have a button on the page that lets me add new items to the list with a little Ajax love so that we don’t have to reload the page. If we can represent an empty list in the page, it’s simple: we can just render :update in Rails…

render :update do |page|
  page.insert_html(:bottom, :my_list, "<li>#{item}</li>")
end

The code above assumes that there’s a DOM ID ‘my\_list’ that has 0 or more elements. But since XHTML won’t let us represent a 0-element list, the obvious thing to do is to bake some extra smarts into the update code.

Now it’s the job of the update to determine if a list exists, and if not to create it, but only for the special one-off case of the zero element list? Yuck, bleh. No thanks. This is probably what most people do, but it just strikes me as putting too much intelligence into something that’s supposed to be pretty dumb.

The alternate solution listed here also qualifies as a hack, no bones about it. But, at least in my opinion, it’s a somewhat more elegant hack than what was described above. File under simple Workaround Of The Day:

<ul id="list_things">
  <li class="invisible" style="display: none"/>
</ul>

Note that the W3C does support empty list item elements :-). Thanks to azta for the suggestion.

Super Mathematics

Posted about 7 years back at zerosum dirt(nap) - Home


Do the math. No matter what you think, nobody’s perfect.

Instance variables, class variables, and inheritance in Ruby

Posted about 7 years back at Sporkmonger

I’m seeing a lot of code that indicates to me that people don’t have a complete grasp of when to use class variables in Ruby.

Here’s a quick example script that should give people a better idea of what’s going on:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

class ExampleClass
  @variable = "foo"
  @@variable = "bar"
  
  def initialize
    @variable = "baz"
  end
  
  def self.test
    puts @variable
  end
  
  def test
    self.class.test
    puts @@variable
    puts @variable
  end
end

class ExampleSubclass < ExampleClass
  @variable = "1"
  @@variable = "2"
  
  def initialize
    @variable = "3"
  end
end

first_example = ExampleClass.new
first_example.test

puts "---"

second_example = ExampleSubclass.new
second_example.test

Output:

foo
2
baz
---
1
2
3

Essentially, when you declare a class variable in a base class, it’s shared with all subclasses. Changing its value in a subclass will affect the base class and all of its subclasses all the way down the inheritance tree. This behavior is often exactly what’s desired. But equally often, this behavior is not what was intended by the programmer, and it leads to bugs, especially if the programmer did not originally expect for the class to be subclassed by someone else.

I strongly encourage Ruby developers to sit down and think about how they want their classes to behave if subclassed by someone else. Be careful about using class variables, because often what you actually wanted was an instance variable on the class object.

NH Ruby UG Meeting.002 Tuesday

Posted about 7 years back at zerosum dirt(nap) - Home

If you’re a Ruby developer (or just an interested outsider) living in southern Maine/NH or the northern Mass area, don’t forget to attend the next meeting of the NH Ruby/Rails User Group. Discussion topic this week is RJS and Ajax. Come hang out with us in Portsmouth tomorrow and make sure to stick around afterwards for drinks, discussion, and merriment.

Oh yeah, and Scott has some free stuff to give away at the gathering too. You like free stuff, don’t you?

Initial reactions to Rails 1.2... annoyed

Posted about 7 years back at Wood for the Trees

I’ve held off my opinion about Rails 1.2 until I had a little more experience with it. First of all, I’m going to say I’m glad Rails has put out this release, because some of the features in Edge I’ve been needing for a while and, in cases where I couldn’t wait, like my simply_restful_backport, I hacked a ported version to 1.1.x. I also had unpublished ports for REST, routing, and a few deprecations to prepare myself. Most of the added features are bang on what is needed when following the Rails methodology of web development: singleton resources, nesting resources, the emphasis on REST, the respond_to(&block), etc.

But my optimism and preparations were in vain. I’m annoyed. My initial reaction is that Rails 1.2 is a fascist. The front-facing libraries, ActiveRecord and ActionPack, are not the cause of my annoyance. It’s the underbelly, ActiveSupport, which has acquired a rank smell.

How Deprecations are handled

I’ll be brief. Backward compatibility is beautiful and when you can’t maintain it, you prepare developers for the transition. Deprecrations are usually the answer: you say it is going to be removed for a long time, encourage people away from it, then finally remove it in either the next major release or the one after that, safe in the knowledge that everyone who will be affected by the change has got the message.

Rails has not held to the time-honoured traditions of deprecation. It does one of three things: fill up the logs with warning messages, raise annoying exceptions that apologise something is gone, or just plain crash because the old API had changed so much. That’s not deprecation. That’s just API upheaval.

And it’s unacceptable to have so many differences in the framework which, first, are undocumented, and second, change development so much. I would expect this behaviour from a 1.3 to 2.0 upgrade, but not between two minor releases. Maybe Rails Core should have re-numbered the release to warn users that Rails has changed quite drastically under the hood. The changes are fine, but it’s the way they came unannounced that is annoying.

Especially after so much time between the two minor versions, there was ample opportunity to tell developers… 1.1.6 was the last opportunity, and a better time was back at 1.1.2, when these additions already existed or were being considered. That is still too quick, but to give no warning, not even a proviso ‘this may change’, until it is too late is just insufferable.

ActiveSupport::Dependencies < ActiveSupport::Devil

For example, I don’t know why an innocuous ObjectSpace block in routes.rb should cause a blocking exception from ActiveSupport::Dependencies. It prevented my application from even loading and took half a day to track down. Or why I needed to alter ExceptionNotifier to require all its libs. Or track down 6 other ‘dependencies’ and make them explicit—I thought Rails was supposed to make this stuff easier, not more of a pain than C’s #include <hardtofind.h>?

It’s ridiculous for an ‘improvement’ release to be so fragile. Rails 1.1.x was pretty flexible on these terms. However, Rails, in its growing focus on encouraging programmers into standardised practices, is looking more like 37signals practices. And to say they are ‘best practices’ or good ‘conventions’ is a bit rich, because Rails is not the cleanest set of libraries. Just run it with $VERBOSE = true and you’ll see what I mean.[1]

In fact, after using ActiveSupport::Dependencies for a while, I’ve come to the conclusion that it is too clever for its own good and I foresee lots of bugs in Rails Core Trac about it. It’s too prying. It’s too observant of everything that is happening. It infects the production environment. It’s a glorified const_set trace_func.

ActiveSupport.include :bloat

What kind of overhead does ActiveSupport::Dependencies have? How many orders of magnitude slower is it than the previous Reloadable module, which just included a method, reloadable?, and was found via ObjectSpace? First of all, it didn’t need a revolution like Dependencies; it was clean, simple, elegant and extendable. It didn’t solve all the problems and had room for improvement, but removing it I think was a mistake. ActiveSupport::Dependencies, unlike it’s predecessor, is a beast. Here’s a quick LOC comparison between activesupport 1.3.1 and 1.4.1 (ignoring hooks in the other gems):

$ grep -v '^\s*#' activesupport-1.3.1/lib/active_support/reloadable.rb | wc -l
28
$ grep -v '^\s*#' activesupport-1.3.1/lib/active_support/dependencies.rb | wc -l
172

$ grep -v '^\s*#' activesupport-1.4.1/lib/active_support/reloadable.rb | wc -l
56
$ grep -v '^\s*#' activesupport-1.4.1/lib/active_support/dependencies.rb | wc -l
538

Not the most reliable comparison, but it gives a good idea of the bloat introduced by, first, dependency tracking and, second, simple deprecating. Most of those lines of code in Reloadable are about how it is not just deprecated, but downright crippled and how evil it was. This is not a good practice for a developing an open source framework which is not easy to upgrade (and we live with it, because Rails is useful). The difficulty of upgrading is precisely because of these badly planned deprecations and backward incompatibilities.

Enough ranting

In this sense, Rails 1.2 is by the far worst gem update I’ve experienced so far. How many blogs have posted about the many little things that needed to be changed to upgrade? How many forward-thinking developers looked in cringing, abject horror when their tests failed left and right after that (apparently) over-optimistic gem update rails? How many people flew to IRC and the mailing lists for help? Let me be so conceited as to introduce a ‘best practice’: discuss major changes with your users.

I realise that Rails is first and foremost a 37signals project, evolved from DHH’s work there and is primarily altered and improved for that company’s purposes, but as an open source project trying to get more users, it’s important to either branch those changes which are special to that company or to be more considerate to the user base. I think Rails development would benefit from:

  • a separate experimental branch (different from Edge trunk) for API changes—Radiant does this quite nicely
  • a highly visible discussion board on rubyonrails.org to discuss major changes with a link to it in IRC and regular updates to the Rails mailing lists

Granted, much of my qualms with Rails 1.2 is with Dependencies and with some good old fashioned hacking, they can fix it. However, I think it and the library from which it has spawned are worth my comment. Dependencies is the most invasive addition to the framework I’ve seen to-date. I appreciate the attempt at innovation, but Rails needs to stop doing this kind of widespread deprecation and fundamentalist intolerance to the unconventional.

I wish the development team would take more pleasure in the diversity of uses of Rails and less perverse pleasure in the militant drilling of young, transitional and experienced programmers. I like the framework, but I don’t work (or want to work) at 37signals.

1 I’ve been aware of my own transgressions for a while. I’m in complete agreement with Mauricio on the topic of warnings. They are there for a reason: to create semantically correct Ruby. We should all be able to run our fantastically short, readable, meta-programmed, overloaded and mixin-loving code with $VERBOSE = true. It isn’t difficult to achieve and it helps in the long-run to make that lovely little code more readable, usable and quiet.

Initial reactions to Rails 1.2... annoyed

Posted about 7 years back at Wood for the Trees

I’ve held off my opinion about Rails 1.2 until I had a little more experience with it. First of all, I’m going to say I’m glad Rails has put out this release, because some of the features in Edge I’ve been needing for a while and, in cases where I couldn’t wait, like my simply_restful_backport, I hacked a ported version to 1.1.x. I also had unpublished ports for REST, routing, and a few deprecations to prepare myself. Most of the added features are bang on what is needed when following the Rails methodology of web development: singleton resources, nesting resources, the emphasis on REST, the respond_to(&block), etc.

But my optimism and preparations were in vain. I’m annoyed. My initial reaction is that Rails 1.2 is a fascist. The front-facing libraries, ActiveRecord and ActionPack, are not the cause of my annoyance. It’s the underbelly, ActiveSupport, which has acquired a rank smell.

How Deprecations are handled

I’ll be brief. Backward compatibility is beautiful and when you can’t maintain it, you prepare developers for the transition. Deprecrations are usually the answer: you say it is going to be removed for a long time, encourage people away from it, then finally remove it in either the next major release or the one after that, safe in the knowledge that everyone who will be affected by the change has got the message.

Rails has not held to the time-honoured traditions of deprecation. It does one of three things: fill up the logs with warning messages, raise annoying exceptions that apologise something is gone, or just plain crash because the old API had changed so much. That’s not deprecation. That’s just API upheaval.

And it’s unacceptable to have so many differences in the framework which, first, are undocumented, and second, change development so much. I would expect this behaviour from a 1.3 to 2.0 upgrade, but not between two minor releases. Maybe Rails Core should have re-numbered the release to warn users that Rails has changed quite drastically under the hood. The changes are fine, but it’s the way they came unannounced that is annoying.

Especially after so much time between the two minor versions, there was ample opportunity to tell developers… 1.1.6 was the last opportunity, and a better time was back at 1.1.2, when these additions already existed or were being considered. That is still too quick, but to give no warning, not even a proviso ‘this may change’, until it is too late is just insufferable.

ActiveSupport::Dependencies < ActiveSupport::Devil

For example, I don’t know why an innocuous ObjectSpace block in routes.rb should cause a blocking exception from ActiveSupport::Dependencies. It prevented my application from even loading and took half a day to track down. Or why I needed to alter ExceptionNotifier to require all its libs. Or track down 6 other ‘dependencies’ and make them explicit—I thought Rails was supposed to make this stuff easier, not more of a pain than C’s #include <hardtofind.h>?

It’s ridiculous for an ‘improvement’ release to be so fragile. Rails 1.1.x was pretty flexible on these terms. However, Rails, in its growing focus on encouraging programmers into standardised practices, is looking more like 37signals practices. And to say they are ‘best practices’ or good ‘conventions’ is a bit rich, because Rails is not the cleanest set of libraries. Just run it with $VERBOSE = true and you’ll see what I mean.[1]

In fact, after using ActiveSupport::Dependencies for a while, I’ve come to the conclusion that it is too clever for its own good and I foresee lots of bugs in Rails Core Trac about it. It’s too prying. It’s too observant of everything that is happening. It infects the production environment. It’s a glorified const_set trace_func.

ActiveSupport.include :bloat

What kind of overhead does ActiveSupport::Dependencies have? How many orders of magnitude slower is it than the previous Reloadable module, which just included a method, reloadable?, and was found via ObjectSpace? First of all, it didn’t need a revolution like Dependencies; it was clean, simple, elegant and extendable. It didn’t solve all the problems and had room for improvement, but removing it I think was a mistake. ActiveSupport::Dependencies, unlike it’s predecessor, is a beast. Here’s a quick LOC comparison between activesupport 1.3.1 and 1.4.1 (ignoring hooks in the other gems):

$ grep -v '^\s*#' activesupport-1.3.1/lib/active_support/reloadable.rb | wc -l
28
$ grep -v '^\s*#' activesupport-1.3.1/lib/active_support/dependencies.rb | wc -l
172

$ grep -v '^\s*#' activesupport-1.4.1/lib/active_support/reloadable.rb | wc -l
56
$ grep -v '^\s*#' activesupport-1.4.1/lib/active_support/dependencies.rb | wc -l
538

Not the most reliable comparison, but it gives a good idea of the bloat introduced by, first, dependency tracking and, second, simple deprecating. Most of those lines of code in Reloadable are about how it is not just deprecated, but downright crippled and how evil it was. This is not a good practice for a developing an open source framework which is not easy to upgrade (and we live with it, because Rails is useful). The difficulty of upgrading is precisely because of these badly planned deprecations and backward incompatibilities.

Enough ranting

In this sense, Rails 1.2 is by the far worst gem update I’ve experienced so far. How many blogs have posted about the many little things that needed to be changed to upgrade? How many forward-thinking developers looked in cringing, abject horror when their tests failed left and right after that (apparently) over-optimistic gem update rails? How many people flew to IRC and the mailing lists for help? Let me be so conceited as to introduce a ‘best practice’: discuss major changes with your users.

I realise that Rails is first and foremost a 37signals project, evolved from DHH’s work there and is primarily altered and improved for that company’s purposes, but as an open source project trying to get more users, it’s important to either branch those changes which are special to that company or to be more considerate to the user base. I think Rails development would benefit from:

  • a separate experimental branch (different from Edge trunk) for API changes—Radiant does this quite nicely
  • a highly visible discussion board on rubyonrails.org to discuss major changes with a link to it in IRC and regular updates to the Rails mailing lists

Granted, much of my qualms with Rails 1.2 is with Dependencies and with some good old fashioned hacking, they can fix it. However, I think it and the library from which it has spawned are worth my comment. Dependencies is the most invasive addition to the framework I’ve seen to-date. I appreciate the attempt at innovation, but Rails needs to stop doing this kind of widespread deprecation and fundamentalist intolerance to the unconventional.

I wish the development team would take more pleasure in the diversity of uses of Rails and less perverse pleasure in the militant drilling of young, transitional and experienced programmers. I like the framework, but I don’t work (or want to work) at 37signals.

1 I’ve been aware of my own transgressions for a while. I’m in complete agreement with Mauricio on the topic of warnings. They are there for a reason: to create semantically correct Ruby. We should all be able to run our fantastically short, readable, meta-programmed, overloaded and mixin-loving code with $VERBOSE = true. It isn’t difficult to achieve and it helps in the long-run to make that lovely little code more readable, usable and quiet.