Lovebirds

Posted about 21 hours back at Mike Clark

Lovebirds

Two little lovebirds, just sittin' in a tree...

For You

Posted 1 day back at Mike Clark

For You

I love the complementary colors of male and female bluebirds in the spring. He flew in to feed her a tasty grub.

Episode #483 - July 25nd, 2014

Posted 3 days back at Ruby5

Rails Rumble, Debug Anything and Speeding up Rails

Listen to this episode on Ruby5

Sponsored by NewRelic

You should be using New Relic by now but do you regularly checkout their blog?
NewRelic

Rails Rumble

Rails Rumble 2014 will take place on the weekend of October 18th & 19th
Rails Rumble

How to Debug Anything

James Golick gave a talk at GoRuCo on how to debug anything
How to Debug Anything

3 Ways to Create Classes in Ruby

THUVA THARMA wrote a classy blog post on the many different ways to create a class
3 Ways to Create Classes in Ruby

Clone and Dup

Aaron Lasseigne just wrote a blog post Clone and Dup
Clone and Dup

Speedup Your Code by Aaron Patterson

Aaron Patterson from RedDotRuby 2014 on how he's been Speeding up rails
Speedup Your Code by Aaron Patterson

Closing

Thank you for listening to Ruby5. Be sure to tune in every Tuesday and Friday for the latest news in the Ruby and Rails community.
Closing

Why Postgres Won't Always Use an Index

Posted 4 days back at GIANT ROBOTS SMASHING INTO OTHER GIANT ROBOTS - Home

We were analyzing slow responses on a project recently and found ourselves questioning the performance of some PostgreSQL queries. As is typical when digging into queries, we had some questions around what the query was doing. So, we used our handy EXPLAIN command to shed some light on the database’s behavior. Upon inspecting the query it turned out an index we had created was not being used. We wanted to know why.

EXPLAIN explained

Let’s first look at how the EXPLAIN command works.

The Postgres docs have a helpful article for learning about EXPLAIN. The basics are that in your psql console prepend the word EXPLAIN in front of your query and you’ll get a query plan.

In SQL

Run psql <name of your database>, or rails dbconsole if you are inside of a Rails project. Prepend EXPLAIN to your query to see an explanation of how Postgres is going to break down your problem in the most efficient way possible.

EXPLAIN SELECT * FROM posts;
                        QUERY PLAN
----------------------------------------------------------
 Seq Scan on posts  (cost=0.00..53.84 rows=281 width=704)
(1 row)

In Rails

If you don’t feel like dropping down to the database you can get explain statements from ActiveRecord::Relations by calling the method explain. Calling the following command will give you the same result you’ll see from the Postgres console:

Post.all.explain

How to interpret the output

Now we have the query plan. What is it telling us? Query plans are read bottom to top and display the following information for each step of the query:

  • estimated statement execution cost: an arbitrary unit of measuring cost
  • estimated number of rows: the number of rows the query will produce
  • estimated width of rows: the size of the rows in bytes

For a more thorough explanation of how to analyze these query plans' costs and number of rows check out the Postgres docs on using explain.

What’s important to our question about indexes is the type of query being performed and whether an index is being used or not. If we see a step containing the words Index Scan we’re using the index.

A side note about query costs

As noted in the Postgres docs, the query plans are based on a sample of statistics about the tables in the query. These statistics are kept up to date by Postgres when your database is VACUUMed. As a result, even when the underlying data is identical, you may see slightly different values for the same query run at different times.

An Example

It doesn’t take much digging to find interesting results in the query plans. Let’s use a database with a posts table containing a couple hundred post records. id is the primary key and by default has an index. Let’s see what the query looks like to get the first 10 posts.

EXPLAIN SELECT * FROM POSTS WHERE id < 10;

Our plan is:

 Bitmap Heap Scan on posts  (cost=2.07..16.39 rows=9 width=704)
   Recheck Cond: (id < 10)
   ->  Bitmap Index Scan on posts_pkey  (cost=0.00..2.07 rows=9 width=0)
         Index Cond: (id < 10)
(4 rows)

This query is using the index because we see Index Scan in the plan. The meaning of the Recheck Cond: statement is beyond the scope of this article, but the postgres-performance mailing list has a helpful explanation.

Now, let’s see what it looks like to get the rest of the posts in the table.

EXPLAIN SELECT * FROM POSTS WHERE id > 10;

Our plan is:

 Seq Scan on posts  (cost=0.00..53.98 rows=272 width=704)
   Filter: (id > 10)
(2 rows)

This query is going to perform a sequential scan (Seq Scan) and not use the index. A sequential scan? This plan means that Postgres is going to read the whole table to find what we’re looking for. That approach seems inefficient given the index we already have. What’s going on here? To answer this question it’s helpful to think about what an index is. An index is a specialized representation in memory of the contents of a particular column (or multiple columns).

How indexes are used

As we saw above, running a couple of queries on our posts table reveals that even given an index to use, Postgres will not always choose to use it. The reason why this is the case is that indexes have a cost to create and maintain (on writes) and use (on reads).

When an index is used in a SELECT query, first the position of the requested rows is fetched from the index (instead of from the table directly). Then, the requested rows (a subset of the total) are actually fetched from the table. It’s a two step process. Given a large enough number of requested rows (such as all the posts with an id greater than 10), it is actually more efficient to go fetch all those rows from the table directly rather than to go through the extra step of asking the index for the position of those rows and next fetch those specific rows from the table. If the percentage of the rows is smaller than 5-10% of all rows in the table the benefit of using the information stored in the index outweighs the additional intermediate step.

What’s Next:

Hear more about Postgres indexes, caching, and performance from Harold Giménez, head of Heroku Postgres, on A Beautiful Thing, an episode of the Giant Robots Smashing Into Other Giants podcast. Or, watch the Improving Rails Performance screencast from Joe Ferris, our CTO.

Phusion Passenger 4.0.48 released

Posted 4 days back at Phusion Corporate Blog


Phusion Passenger is a fast and robust web server and application server for Ruby, Python, Node.js and Meteor. Passenger takes a lot of complexity out of deploying web apps, and adds powerful enterprise-grade features that are useful in production. High-profile companies such as Apple, New York Times, AirBnB, Juniper, American Express, etc are already using it, as well as over 350.000 websites.

Phusion Passenger is under constant maintenance and development. Version 4.0.48 is a bugfix release.

Phusion Passenger also has an Enterprise version which comes with a wide array of additional features. By buying Phusion Passenger Enterprise you will directly sponsor the development of the open source version.

Recent changes

4.0.47 was a hotfix release for an Enterprise customer. The changes in 4.0.47 and 4.0.48 combined are as follows.

  • Fixed a race condition while determining what user an application should be executed as. This bug could lead to applications being run as the wrong user. Closes GH-1241.
  • [Standalone] Improved autodetection of Rails asset pipeline files. This prevents Standalone from incorrectly setting caching headers on non-asset pipeline files. Closes GH-1225.
  • Fixed compilation problems on CentOS 5. Thanks to J. Smith. Closes GH-1247.
  • Fixed compilation problems on OpenBSD.
  • Fixed compatibility with Ruby 1.8.5.

Installing or upgrading to 4.0.48

OS X OS X Debian Debian Ubuntu Ubuntu
Heroku Heroku Ruby gem Ruby gem Tarball Tarball

Final

Fork us on Github!Phusion Passenger’s core is open source. Please fork or watch us on Github. :)

<iframe src="http://ghbtns.com/github-btn.html?user=phusion&amp;repo=passenger&amp;type=watch&amp;size=large&amp;count=true" allowtransparency="true" frameborder="0" scrolling="0" width="170" height="30"></iframe><iframe src="http://ghbtns.com/github-btn.html?user=phusion&amp;repo=passenger&amp;type=fork&amp;size=large&amp;count=true" allowtransparency="true" frameborder="0" scrolling="0" width="170" height="30"></iframe><iframe src="http://ghbtns.com/github-btn.html?user=phusion&amp;type=follow&amp;size=large&amp;count=true" allowtransparency="true" frameborder="0" scrolling="0" width="190" height="30"></iframe>

If you would like to stay up to date with Phusion news, please fill in your name and email address below and sign up for our newsletter. We won’t spam you, we promise.



An Explained psqlrc

Posted 5 days back at GIANT ROBOTS SMASHING INTO OTHER GIANT ROBOTS - Home

Let’s walk through my short psqlrc(5) to see what I’ve set, and to inspire you to find your own configuration that fits into your workflow. Here is my complete psqlrc:

\set ON_ERROR_ROLLBACK interactive
\set COMP_KEYWORD_CASE upper
\set HISTFILE ~/.psql/history- :DBNAME

\pset pager off
\pset null '(null)'

PostgreSQL’s shell, psql(1), can be configured using \set and \pset. \pset is for changing the output format — HTML, pager, field separator, and so on — while \set is for everything else.

ON_ERROR_ROLLBACK

The ON_ERROR_ROLLBACK settings affects how psql(1) handles errors. The default value is off.

When this setting is on, errors are effectively ignored at all times. So if you have this script, slint.sql:

BEGIN;
CREATE TABLE members (id SERIAL, name TEXT);
INSERT INTO member (name) VALUES ('David Pajo');
INSERT INTO members (name) VALUES ('Brian McMahan');
COMMIT;

And run it from the command line:

psql -f slint.sql

You would end up with a members table with Brian McMahan but without David Pajo.

When it is set to off, the default, then you get nothing: no members table and no Brian McMahan. It either all works or it doesn’t, just like a transaction should.

There is a third value: interactive. Under interactive, the above command in which statements are piped into psql(1) non-interactively is treated like off, but if you type them into the interactive prompt it is treated like on. This gives you a chance to fix things without starting over:

$ psql
bands=# BEGIN;
BEGIN
bands=# CREATE TABLE members (id SERIAL, name TEXT);
CREATE TABLE
bands=# INSERT INTO member (name) VALUES ('David Pajo');
ERROR:  relation "member" does not exist
LINE 1: INSERT INTO member (name) VALUES ('David Pajo');
bands=# INSERT INTO members (name) VALUES ('David Pajo');
INSERT 0 1
bands=# INSERT INTO members (name) VALUES ('Brian McMahan');
INSERT 0 1
bands=# COMMIT;
COMMIT

COMP_KEYWORD_CASE

Some people format their SQL with uppercase keywords; others go downcase. Some mix and match depending on their mood. psql(1) handles that!

Possibly the greatest feature of any shell is tab completion, and psql(1) doesn’t disappoint. However, there’s a question of which case it should use to complete keywords.

The straight-forward thing to do is to set it to lower or upper.

sel tab completes to SELECT

But even fancier are preserve-lower and preserve-upper, with preserve-upper as the default. These preserve whatever case you were using, falling back to lower (or upper). For example:

preserve the case but default to upper

There, up was completed to update and S was completed to SET, preserving the case as the user typed it; n was completed to name, preserving the case of the object in the database; and the space after order was completed to BY, favoring uppercase when the user has typed nothing.

HISTFILE

Like any good shell, psql(1) will save the commands you have entered so you can run them again (it’s full Readline; try a ^R some time). By default it stores the history in ~/.psql_history, but we can do better than that.

To start, let’s introduce another psql(1) command: \echo

bands=# \echo hello
hello
bands=# \echo :DBNAME 
bands

The variable :DBNAME is automatically set to the name of the database and available to all psql(1) commands. There are other pre-set variables like :PORT, :HOST, :USER, :ENCODING, and so on, but we’re going to use :DBNAME to start.

It just so happens that psql(1) will concatenate strings for you, so if you want different history for each database (the queries against the desserts table won’t make sense in the zoology database, for example), you can set that up:

\set HISTFILE ~/.psql_history- :DBNAME

You can combine these as much as you please, such as:

\set HISTFILE ~/.psql_history- :USER - :HOST - :PORT - :DBNAME

pager

The pager is the program that paginates text. The classic is more(1), and the improvement is less(1). Puns.

The default value for the pager setting is on which — unlike the name suggests — only uses the pager sometimes. A few lines are shown without a pager, but 25 or more lines invoke pagination. (Specifically, if the text would scroll off the screen, it invokes the pager.)

To always have a pager, use the value always. To never use the pager — useful inside a terminal multiplexer or terminal emulator with good scrolling — use the value off.

You can also change the pager itself by setting the PAGER environment variable. For example:

export PAGER="/usr/local/bin/gvim -f -R -"

This will use gvim(1) as your pager.

null

By default NULL values show as blank spaces. Also by default the empty string shows as a blank space.

bands=# INSERT INTO members (name) VALUES ('');
INSERT 0 1
bands=# INSERT INTO members (name) VALUES (NULL);
INSERT 0 1
bands=# SELECT * FROM members;
 id |     name      
----+---------------
  1 | David Pajo
  2 | Brian McMahan
  3 | 
  4 | 
(4 rows)

To better distinguish NULL values from empty strings, you can have psql(1) show anything you want instead:

bands=# \pset null '(null)'
Null display is "(null)".
bands=# SELECT * FROM members;
 id |     name      
----+---------------
  1 | David Pajo
  2 | Brian McMahan
  3 | 
  4 | (null)
(4 rows)

And more

You can find all of this and more in the psql(1) manpage or in the official PostgreSQL Web documentation. We have also written previously on this topic.

As you read the documentation we’d love to see your favorite settings as pull requests against the .psqlrc in our dotfiles.

Episode #482 - July 22nd, 2014

Posted 6 days back at Ruby5

Get your mind in the Gutter, agree that Programming is Not Math, be a RubyCritic, master Vim Plugins for Ruby, review 3 Ways to Create Classes in Ruby, and take a trip to RailsPacific.

Listen to this episode on Ruby5

Sponsored by CodeShip.io

Codeship is a hosted Continuous Delivery Service that just works.

Set up Continuous Integration in a few steps and automatically deploy when all your tests have passed. Integrate with GitHub and BitBucket and deploy to cloud services like Heroku and AWS, or your own servers.

Visit http://codeship.io/ruby5 and sign up for free. Use discount code RUBY5 for a 20% discount on any plan for 3 months.

Also check out the Codeship Blog!

CodeShip.io

Gutter

Rah-jeev Kannav Sharma wrote to us to let us know about a gem called Gutter, a low-overheard monitoring web dashboard for GNU and Linux machines. It features live, on-demand monitoring of RAM, load, uptime, disk allocation, users, and many more stats.
Gutter

Programming Is Not Math

Sarah Mei wrote a really interesting blog post last week called Programming Is Not Math. She talks about how most the time programming is in fact much more like a language skill. Yet, somehow most computer science degrees focus quite heavily on math while it’s very possible to not need it that much later on.
Programming Is Not Math

RubyCritic

Guilherme Simões sent us a note about RubyCritic, a gem he built for his Master's thesis. He describes it as an opinionated version of the MetricFu gem which does static code analysis.
RubyCritic

Vim Plugins for Ruby

Milos Dolobac sent us a note this week about a blog post he wrote called Vim Plugins for Ruby. According to him, these plugins are productivity boosters that every Ruby developer should know about.
Vim Plugins for Ruby

3 Ways to Create Classes in Ruby

If you’ve written almost anything in Ruby, you’ve probably come across a Ruby class, and as Thuva Tharma explains in his most recent blog post, there are actually three ways to create a class in Ruby. Each of these styles could come in handy if you know how to use them.
3 Ways to Create Classes in Ruby

RailsPacific

We recently found out about RailsPacific -- the first ever Ruby on Rails Conference in Asia. It runs September 26-27, one week after RubyKaigi, in Taipei, Taiwan. The conference is divided into one day of speaking and one day of workshops, including workshops on refactoring, performance tuning, object-oriented design, and TDD with RSpec.
RailsPacific

Sponsored by Top Ruby Jobs

Simon & Schuster is looking for a Ruby on Rails developer in New York, NY Adobe Systems is looking for a Senior Web Developer in San Jose & San Francisco, CA Underdog.io is looking for a Ruby on Rails Developer in New York, NY or remote and Cambridge Systematics is looking for a Ruby on Rails Engineer in Cambridge, MA
Top Ruby Jobs

Sponsored by Ruby5

Ruby5 is released Tuesday and Friday mornings. To stay informed about and active with this podcast, we encourage you to do one of the following:

Thank You for Listening to Ruby5

Spring Stroll

Posted 7 days back at Mike Clark

Spring Stroll

Just out for a leisurely stroll through the balsam root...

Ruby 2 Keyword Arguments

Posted 8 days back at GIANT ROBOTS SMASHING INTO OTHER GIANT ROBOTS - Home

Ruby 2.0 introduced first-class support for keyword arguments:

def foo(bar: 'default')
  puts bar
end

foo # => 'default'
foo(bar: 'baz') # => 'baz'

In Ruby 1.9, we could do something similar using a single Hash parameter:

def foo(options = {})
  bar = options.fetch(:bar, 'default')
  puts bar
end

foo # => 'default'
foo(bar: 'baz') # => 'baz'

Ruby 2.0 blocks can also be defined with keyword arguments:

define_method(:foo) do |bar: 'default'|
  puts bar
end

foo # => 'default'
foo(bar: 'baz') # => 'baz'

Again, to achieve similar behavior in Ruby 1.9, the block would take an options hash, from which we would extract argument values.

Required keyword arguments

Unfortunately, Ruby 2.0 doesn’t have built-in support for required keyword arguments. Luckily, Ruby 2.1 introduced required keyword arguments, which are defined with a trailing colon:

def foo(bar:)
  puts bar
end

foo # => ArgumentError: missing keyword: bar
foo(bar: 'baz') # => 'baz'

If a required keyword argument is missing, Ruby will raise a useful ArgumentError that tells us which required argument we must include.

Keyword arguments vs options hash

With first-class keyword arguments in the language, we don’t have to write the boilerplate code to extract hash options. Unnecessary boilerplate code increases the opportunity for typos and bugs.

With keyword arguments defined in the method signature itself, we can immediately discover the names of the arguments without having to read the body of the method.

Note that the calling code is syntactically equal to calling a method with hash arguments, which makes for an easy transition from options hashes to keyword arguments.

Keyword arguments vs positional arguments

Assume we have a method with positional arguments:

def mysterious_total(subtotal, tax, discount)
  subtotal + tax - discount
end

mysterious_total(100, 10, 5) # => 105

This method does its job, but as a reader of the code using the mysterious_total method, I have no idea what those arguments mean without looking up the implementation of the method.

By using keyword arguments, we know what the arguments mean without looking up the implementation of the called method:

def obvious_total(subtotal:, tax:, discount:)
  subtotal + tax - discount
end

obvious_total(subtotal: 100, tax: 10, discount: 5) # => 105

Keyword arguments allow us to switch the order of the arguments, without affecting the behavior of the method:

obvious_total(subtotal: 100, discount: 5, tax: 10) # => 105

If we switch the order of the positional arguments, we are not going to get the same results, giving our customers more of a discount than they deserve:

mysterious_total(100, 5, 10) # => 95

Connascence and trade-offs

Connascence between two software components A and B means either 1) that you can postulate some change to A that would require B to be changed (or at least carefully checked) in order to preserve overall correctness, or 2) that you can postulate some change that would require both A and B to be changed together in order to preserve overall correctness. - Meilir Page-Jones, What Every Programmer Should Know About Object-Oriented Design

When one Ruby method has to know the correct order of another method’s positional arguments, we end up with connascence of position.

If we decide to change the order of the parameters to mysterious_total, we must change all callers of that method accordingly. Not only that, but our mental model of how to use this method must change as well, which isn’t as simple as a find/replace.

Like most things, keyword arguments have their trade-offs. Positional arguments offer a more succinct way to call a method. Usually, the code clarity and maintainability gained from keyword arguments outweigh the terseness offered by positional arguments. I would use positional arguments if I could easily guess their meanings based on the method’s name, but I find this rarely to be the case.

What’s Next?

Episode #481 - July 18th, 2014

Posted 10 days back at Ruby5

Take a peek into your app, think about accessibility, write polyglot web apps, learn Rails, say goodbye to 1.8.7 and 1.9.2 support

Listen to this episode on Ruby5

Sponsored by New Relic

New Relic is _the_ all-in-one web performance analytics product. It lets you manage and monitor web application performance, from the browser down to the line of code. With Real User Monitoring, New Relic users can see browser response times by geographical location of the user, or by browser type.
This episode is sponsored by New Relic

Peek

Take a peek into your Rails application!
Peek

accessibility

Simple chrome extension to notify websites of your accessibility requirements.
accessibility

polyglot

Polyglot is a distributed web framework that allows programmers to create web applications in multiple programming languages.
polyglot

GoRails

GoRails is a series of screencasts and guides for all aspects of Ruby on Rails.
GoRails

EOL for 1.8.7 and 1.9.2

Extended maintenance of Ruby versions 1.8.7 and 1.9.2 will end on July 31, 2014.
EOL for 1.8.7 and 1.9.2

Notes on 'Notes on "Counting Tree Nodes"'

Posted 11 days back at interblah.net - Home

Having now finished watching Tom’s episode of Peer to Peer, I finally got around to reading his Notes on “Counting Tree Nodes” supplementary blog post. There are a couple of ideas he presents that are so interesting that I wanted to highlight them again here.

If you haven’t seen the video, then I’d still strongly encourage you to read the blog post. While I can now see the inspiration for wanting to discuss these ideas1, the post really does stand on it’s own.

Notes on ‘Enumerators’

Here’s the relevant section of the blog post. Go read it now!

I’m not going to re-explain it here, so yes, really, go read it now.

What I found really interesting here was the idea of building new enumerators by re-combining existing enumerators. I’ll use a different example, one that is perhaps a bit too simplistic (there are more concise ways of doing this in Ruby), but hopefully it will illustrate the point.

Let’s imagine you have an Enumerator which enumerates the numbers from 1 up to 10:

> numbers = 1.upto(10)
=> #<Enumerator: 1:upto(10)>
> numbers.next
=> 1
> numbers.next
=> 2
> numbers.next
=> 3
...
> numbers.next
=> 9
> numbers.next
=> 10
> numbers.next
StopIteration: iteration reached an end

You can now use that to do all sorts of enumerable things like mapping, selecting, injecting and so on. But you can also build new enumerables using it. Say, for example, we now only want to iterate over the odd numbers between 1 and 10.

We can build a new Enumerator that re-uses our existing one:

> odd_numbers = Enumerator.new do |yielder|
    numbers.each do |number|
      yielder.yield number if number.odd?
    end
  end
=> #<Enumerator: #<Enumerator::Generator:0x007fc0b38de6b0>:each>

Let’s see it in action:

> odd_numbers.next
=> 1
> odd_numbers.next
=> 3
> odd_numbers.next
=> 5
> odd_numbers.next
=> 7
> odd_numbers.next
=> 9
> odd_numbers.next
StopIteration: iteration reached an end

So, that’s quite neat (albeit somewhat convoluted compared to 1.upto(10).select(&:odd)). To extend this further, let’s imagine that I hate the lucky number 7, so I also don’t want that be included. In fact, somewhat perversely, I want to stick it right in the face of superstition by replacing 7 with the unluckiest number, 13.

Yes, I know this is weird, but bear with me. If you have read Tom’s post (go read it), you’ll already know that this can also be achieved with a new enumerator:

> odd_numbers_that_arent_lucky = Enumerator.new do |yielder|
    odd_numbers.each do |odd_number|
      if number == 7
        yielder.yield 13
      else
        yielder.yield number
      end
    end
  end
=> #<Enumerator: #<Enumerator::Generator:0x007fc0b38de6b0>:each>
> odd_numbers.next
=> 1
> odd_numbers.next
=> 3
> odd_numbers.next
=> 5
> odd_numbers.next
=> 13
> odd_numbers.next
=> 9
> odd_numbers.next
StopIteration: iteration reached an end

In Tom’s post he shows how this works, and how you can further compose enumerators to to produce new enumerations with specific elements inserted at specific points, or elements removed, or even transformed, and so on.

So.

A hidden history of enumerable transformations

What I find really interesting here is that somewhere in our odd_numbers enumerator, all the numbers still exist. We haven’t actually thrown anything away permanently; the numbers we don’t want just don’t appear while we are enumerating.

The enumerator odd_numbers_that_arent_lucky still contains (in a sense) all of the numbers between 1 and 10, and so in the tree composition example in Tom’s post, all the trees he creates with new nodes, or with nodes removed, still contain (in a sense) all those nodes.

It’s almost as if the history of the tree’s structure is encoded within the nesting of Enumerator instances, or as if those blocks passed to Enumerator.new act as a runnable description of the transformations to get from the original tree to the tree we have now, invoked each time any new tree’s children are enumerated over.

I think that’s pretty interesting.

Notes on ‘Catamorphisms’

In the section on Catamorphisms (go read it now!), Tom goes on to show that recognising similarities in some methods points at a further abstraction that can be made – the fold – which opens up new possibilities when working with different kinds of structures.

What’s interesting to me here isn’t anything about the code, but about the ability to recognise patterns and then exploit them. I am very jealous of Tom, because he’s not only very good at doing this, but also very good at explaining the ideas to others.

Academic vs pragmatic programming

This touches on the tension between the ‘academic’ and ‘pragmatic’ nature of working with software. This is something that comes up time and time again in our little sphere:

Now I’m not going to argue that anyone working in software development should have a degree in Computer Science. I’m pretty sympathetic with the idea that many “Computer Science” degrees don’t actually bear much of direct resemblance to the kinds of work that most software developers do2.

Ways to think

What I think university study provides, more than anything else, is exposure and training in ways to think that aren’t obvious or immediately accessible via our direct experience of the world. Many areas of study provide this, including those outside of what you might consider “science”. Learning a language can be learning a new way to think. Learning to interpret art, or poems, or history is learning a new way to think too.

Learning and internalising those ways to think give perspectives on problems that can yield insights and new approaches, and I propose that that, more than any other thing, is the hallmark of a good software developer.

Going back to the blog post which, as far I know, sparked the tweet storm about “programming and maths”, I’d like to highlight this section:

At most academic CS schools, the explicit intent is that students learn programming as a byproduct of learning CS. Programming itself is seen as rather pedestrian, a sort of exercise left to the reader.

For actual developer jobs, by contrast, the two main skills you need these days are programming and communication. So while CS still does have strong ties to math, the ties between CS and programming are more tenuous. You might be able to say that math skills are required for computer science success, but you can’t necessarily say that they’re required for developer success.

What a good computer science (or maths or any other logic-focussed) education should teach you are ways to think that are specific to computation, algorithms and data manipulation, which then

  • provide the perspective to recognise patterns in problems and programs that are not obvious, or even easily intuited, and might otherwise be missed.
  • provide experience applying techniques to formulate solutions to those problems, and refactorings of those programs.

Plus, it’s fun to achieve that kind of insight into a problem. It’s the “a-ha!” moment that flips confusion and doubt into satisfaction and certainty. And these insights are also interesting in and of themselves, in the very same way that, say, study of art history or Shakespeare can be.

So, to be crystal clear, I’m not saying that you need this perspective to be a great programmer. I’m really not. You can build great software that both delights users and works elegantly underneath without any formal training. That is definitely true.

Back to that quote:

the ties between CS and programming are more tenuous … you can’t necessarily say that they’re required for developer success.

All I’m saying is this: the insights and perspectives gained by studying computer science are both useful and interesting. They can help you recognise existing, well-understood problems, and apply robust, well-understood and powerful solutions.

That’s the relevance of computer science to the work we do every day, and it would be a shame to forget that.

  1. In the last 15 minutes or so of the video, the approach Tom uses to add a “child node” to a tree is interesting but there’s not a huge amount of time to explore some of the subtle benefits of that approach

  2. Which is, and let’s be honest, a lot of “Get a record out of a database with an ORM, turn it into some strings, save it back into the database”.

Baseimage-docker 0.9.12 released

Posted 11 days back at Phusion Corporate Blog

Baseimage-docker is a special Docker image that is configured for correct use within Docker containers. It is Ubuntu, plus modifications for Docker-friendliness. You can use it as a base for your own Docker images. Learn more at the Github repository and the website, which explain in detail what the problems are with the stock Ubuntu base image, and why you should use baseimage-docker.

Changes in this release

  • We now officially support nsenter as an alternative way to login to the container. With official support, we mean that we’ve provided extensive documentation on how to use nsenter, as well as related convenience tools. However, because nsenter has various issues, and for backward compatibility reasons, we still support SSH. Please refer to the README for details about nsenter, and what the pros and cons are compared to SSH.
    • The docker-bash tool has been modified to use nsenter instead of SSH.
    • What was previously the docker-bash tool, has now been renamed to docker-ssh. It now also works on a regular sh shell too, instead of bash specifically.
  • Added a workaround for Docker’s inability to modify /etc/hosts in the container (Docker bug 2267). Please refer to the README for details.
  • Fixed an issue with SSH X11 forwarding. Thanks to Anatoly Bubenkov. Closes GH-105.
  • The init system now prints its own log messages to stderr. Thanks to mephi42. Closes GH-106.

Using baseimage-docker

Please learn more at the README.

The post Baseimage-docker 0.9.12 released appeared first on Phusion Corporate Blog.

Tmux Only For Long-Running Processes

Posted 12 days back at GIANT ROBOTS SMASHING INTO OTHER GIANT ROBOTS - Home

This post describes a minimal Tmux workflow, used only for long-running processes. It is intended to reduce the cognitive load imposed by administrative debris of open tabs, panes, or windows.

Set up Tmux for a Rails project

From within a full-screen shell (to hide window chrome, status bars, notifications, the system clock, and other distractions), create a Tmux session for a Rails project:

cd project-name
tat

tat (short for “tmux attach”) is a command from thoughtbot/dotfiles that names the Tmux session after the project’s directory name. That naming convention will help us re-attach to the session later using the same tat command.

At this point, tat is the same thing as:

tmux new -s `basename $PWD`

Run the Rails app’s web and background processes with Foreman:

foreman start

The process manager is a long-running process. It is therefore a great candidate for Tmux. Run it inside Tmux, then forget it.

After only running one command inside Tmux, detach immediately:

<tmux-prefix> d

Ctrl+b is the default Tmux prefix. Many people change it to be Ctrl+a to match the API provided by GNU Screen, another popular terminal multiplexer.

Perform ad-hoc tasks

Back in a full-screen shell, we perform ad-hoc tasks such as:

vim .
git status
git add --patch
git commit --verbose

Those commands are targeted, “right now” actions. They are executed in a split second and focus us immediately on the task at hand.

Doing most of the work from inside Vim

A majority of our work is done from within a text editor, such as fast grepping in Vim:

\ string-i-am-searching-for

Or, running specs from Vim:

<Leader> s

In thoughtbot/dotfiles, <Leader> is <Space>.

Suspending the Vim process when necessary

To return control from Vim to the command line, suspend the process:

Ctrl+z

Run this command to see suspended processes for this shell session:

jobs

It will output something like:

[1]  + suspended  vim spec/models/user_spec.rb

This is when we might do some Git work:

git fetch origin
git rebase -i origin/master
git push --force origin <branch-name>
git log origin/master..<branch-name>
git diff --stat origin/master
git checkout master
git merge <branch-name> --ff-only
git push
git push origin --delete <branch-name>
git branch --delete <branch-name>

When we’re ready to edit in Vim again, we foreground the process:

fg

Re-attach to the Tmux session quickly

When we need to restart the process manager or start a new long-running process, we re-attach to the Tmux session:

tat

At this point, tat is the same thing as:

tmux attach -t `basename $PWD`

Compared to other Tmux workflows, this workflow does involve more attaching and detaching from Tmux sessions. That is why the tat shortcut is valuable.

Back inside Tmux, we can kill the Foreman process:

Ctrl+c

Or, we might want to open a long-running Rails console in order to maintain a history of queries:

<tmux-prefix> c
rails console

After poking around in the database, we might detach from Tmux again:

<tmux-prefix> d

Get things done

At that point, we might take a break, go home, or move on to another project.

The next time we sit (or stand!) at our desks, we start fresh by creating a branch, opening Vim, or doing whatever ad-hoc task is necessary in a clean slate, distraction-free environment.

Meanwhile, Tmux handles one responsibility for us: quietly manages long-running processes.

Solitary Unit Test

Posted 12 days back at Jay Fields Thoughts

Originally found in Working Effectively with Unit Tests

It’s common to unit test at the class level. The Foo class will have an associated FooTestsclass. Solitary Unit Tests follow two additional constraints:

  1. Never cross boundaries
  2. The Class Under Test should be the only concrete class found in a test.
Never cross boundaries is a fairly simple, yet controversial piece of advice. In 2004, Bill Caputo wrote about this advice, and defined a boundary as: ”...a database, a queue, another system...”. The advice is simple: accessing a database, network, or file system significantly increases the the time it takes to run a test. When the aggregate execution time impacts a developer’s decision to run the test suite, the effectiveness of the entire team is at risk. A test suite that isn’t run regularly is likely to have negative-ROI.

In the same entry, Bill also defines a boundary as: ”... or even an ordinary class if that class is ‘outside’ the area your [sic] trying to work with or are responsible for”. Bill’s recommendation is a good one, but I find it too vague. Bill’s statement fails to give concrete advice on where to draw the line. My second constraint is a concrete (and admittedly restrictive) version of Bill’s recommendation. The concept of constraining a unit test such that ‘the Class Under Test should be the only concrete class found in a test’ sounds extreme, but it’s actually not that drastic if you assume a few things.
  1. You’re using a framework that allows you to easily stub most concrete classes
  2. This constraint does not apply to any primitive or class that has a literal (e.g. int, Integer, String, etc)
  3. You’re using some type of automated refactoring tool.
There are pros and cons to this approach, both of which are examined in Working Effectively with Unit Tests.

Solitary Unit Test can be defined as:
Solitary Unit Testing is an activity by which methods of a class or functions of a namespace are tested to determine if they are fit for use. The tests used to determine if a class or namespace is functional should isolate the class or namespace under test by stubbing all collaboration with additional classes and namespaces.

Knowledge Base updates

Posted 12 days back at entp hoth blog - Home

Howdy everyone!

Today I would like to highlight two updates we deployed to the Knowledge Base in the past few weeks.

Versioning

KB articles are now versioned. You can see changes between versions, restore versions, and see who made the changes. This should make it safer to update your articles regularly and allow you to recover from mistakes more easily :)

You can see the versions on the KB list:

Showing the number of versions in the KB listing

On the KB page:

Showing the number of versions on the KB page

And look at the history:

Showing the versions history

Export

On the KB admin page, in the left sidebar, there is an option to export your whole KB as an HTML page:

Export your KB

We just improved this feature to add a Table of Contents, and fix all links between the different articles. This means that if you use the same anchor names in different articles, they will now work flawlessly in the exported file. If you moved or renamed articles but still have links to the old address, the export will take care of that as well.

This change will allow you to export your entire KB in one page, and print it as a PDF, and you have a complete manual for your application/service. Some of our customers do just that!

This last part is a bit experimental (the re-linking of everything), so if you experience any issue, just let us know.

Enjoy!