Ruby Access Control – Are Private And Protected Methods Only A Guideline?

ProtectA few days ago Chad Fowler ran the following quick quiz on Twitter:

Ruby quiz: in 140 chrs or less, why doesn't this work: class Y; def a;
self.x end; private; def x; puts "hi" end end; Y.new.a

Here is the formatted version:

```ruby class Y def a self.x end private def x puts “hi” end end

Y.new.a```

Running this produces the following error:

a.rb:3:in `a': private method `x' called for #<Y:0x7f819a82d548> (NoMethodError)
    from a.rb:11

I wasn’t immediately able to figure out what was wrong with this code and since access control is one of those fundamental things that you should know, if you want to be a decent Ruby developer, I decided to dig a little further to figure it out. Here is the scoop.

Private, Protected and Public – Ruby Method Visibility

The concept of private, protected and public methods in Ruby is somewhat different than it is in languages like Java (well, not so much public, that’s very similar :)). In Java if a method is declared private, it can only be accessed from other methods in the same class. When a method is declared protected it can be accessed by other classes in the same package as well as by subclasses of its class in a different package. When a method is public it is – of course – always visible to everyone. So, in essence with Java, these keywords protect the various members from access by classes, depending on where these classes are in the inheritance/package hierarchy.

In Ruby, the inheritance hierarchy or the package/module don’t really enter into the equation, it is rather all about which object is the receiver of a particular method call. When a method is declared private in Ruby, it means this method can never be called with an explicit receiver. Any time we’re able to call a private method with an implicit receiver it will always succeed. This means we can call a private method from within a class it is declared in as well as all subclasses of this class e.g.

```ruby class A def main_method method1 end

private def method1 puts “hello from #{self.class}” end end

class B < A def main_method method1 end end

A.new.main_method B.new.main_method```

alan@alan-ubuntu-vm:~/tmp$ ruby a.rb
hello from A
hello from B

However, as soon as we try to use an explicit receiver, even if the receiver is “self”, the method call will fail e.g.

```ruby class C < A def main_method self.method1 end end

C.new.main_method```

alan@alan-ubuntu-vm:~/tmp$ ruby a.rb
a.rb:36:in `main_method': private method `method1' called for #<C:0x7f67025a0648> (NoMethodError)
    from a.rb:40

Do you recognise the error? This was the answer to the quiz that Chad ran. We were trying to call a private method with an explicit receiver and even though we were doing it from within the class where our private method was defined, it would still fail for that reason. And of course, since you can’t call a private method with an explicit receiver, you can never call it from outside the class hierarchy where it was defined.

Protected methods are also a little different. You can always call a protected method with an implicit receiver, just like private, but in addition you can call a protected method with an explicit receiver as long as this receiver is self or an object of the same class as self. Let’s modify our example above and make the method protected:

```ruby class A def main_method method1 end

protected def method1 puts “hello from #{self.class}” end end

class B < A def main_method method1 end end

class C < A def main_method self.method1 end end```

alan@alan-ubuntu-vm:~/tmp$ ruby a.rb
hello from A
hello from B
hello from C

Now, the call with the implicit receiver in class B succeeds (as before) but the call with the explicit receiver in class C also succeeds (unlike when method1 was private). Furthermore, doing the following is also fine:

```ruby class D < A def main_method B.new.method1 end end

D.new.main_method```

alan@alan-ubuntu-vm:~/tmp$ ruby a.rb
hello from B

Everything works because B.new is the same type of object as self and so the call to method1 succeeds. If however we make class D NOT inherit from A, we get the following:

```ruby class D def main_method B.new.method1 end end

D.new.main_method```

alan@alan-ubuntu-vm:~/tmp$ ruby a.rb
a.rb:39:in `main_method': protected method `method1' called for #<B:0x7fe81d00efa8> (NoMethodError)
    from a.rb:46

In this case B.new is no longer the same type of object as self and so trying to call a protected method with B.new as the receiver – fails. So, I guess the inheritance hierarchy does actually play a role when it comes to access control in Ruby, it’s just not as big a role as it is in a language like Java.

Public methods are – of course – accessible with any kind of explicit or implicit receiver from anywhere (as the name implies).

It Is All Just A Guideline Anyway

Invisible

What if we don’t want to obey these access rules? For example, what if I want to test my private method? This is reasonably easy to do in Ruby, you simply need to “call” your private method using “send” (I will cover the use of “send” in more detail in a subsequent post I am planning to do, on passing methods as arguments in Ruby). For example, let’s modify our original example as follows:

```ruby class A private def method1 puts “hello from #{self.class}” end end

A.new.send(:method1)```

alan@alan-ubuntu-vm:~/tmp$ ruby a.rb
hello from A

That was just way too easy! It works fine in both Ruby 1.8 and 1.9, it was supposed to disappear in 1.9 but it hasn’t yet. However even if it does, we can always simply define a public method on the object in question which delegates to the private one or redefine our private methods as public and then change them back again when we’re done.

Testing private methods is often frowned upon since we’re breaking encapsulation and not testing through the interface. However, I do believe that sometimes, it simply makes sense to test private method separately, especially if they are used by multiple public methods. You can then simply mock out the private method when testing the public API rather than having to indirectly test its functionality over and over again. I think it makes for a better and more terse set of tests, which I think is smarter in a TDD context.

It is worth mentioning that in the Java world, this practice (testing private methods) is frowned upon much more strongly. I believe this attitude is more to do with refactoring support than with anything else. It is possible to invoke private methods in Java from outside the class. But, aside from the fact that this is much more labour intensive than in Ruby, the fact that you have to use reflection means you give up the refactoring support that Java IDEs provide. This is a big deal in Java, since it has excellent refactoring tools (being a strongly typed language). In Ruby it is not really much of an issue since refactoring support is minimal and arguably not as necessary due to the dynamic nature of the language.

Well, I hope this post has helped to clarify how access control works in Ruby as well as giving a bit more of an insight into the inner workings of the language. Until next time.

Images by hebedesign and vaXzine

On The Value Of Fundamentals In Software Development

Martial ArtsI believe in the value of the fundamentals, but it sometimes seems like I am the only one. I am firmly of the opinion that the key to true mastery and expertise in software development (and in everything else) lies in having a solid grip on the fundamentals, so solid that you don’t really need to think when you apply them. Whenever I trot this opinion out, however, I inevitably find myself in the minority. You see, we – software developers – view ourselves as problem solvers. We don’t really need to have deep knowledge and understanding of anything (and with the amount of tech we’re exposed to on a daily basis it is pretty much impossible anyway). As long as we know that something exists, we can always infer, look-up and work-out everything we need to be able to use it. This attitude is fine, but I believe it is wrong to apply it when it comes to fundamental knowledge, and this is where many people seem to disagree with me – to my constant amazement. Before looking at this further, let us define what fundamental knowledge actually is.

What Are Fundamentals

A couple of years ago I was asked the following question in an interview. Explain how optimistic locking works, in the context of Hibernate. At the time I had done a bit of work with Hibernate, but I was by no means an expert, so I drew a complete blank. What a total FAIL on my part that was, because optimistic locking is not really a Hibernate-centric idea, it is a fundamental concurrency control concept. I should have been able to fire-off an explanation of optimistic locking in general and I would have been 90% of the way there (regarding how it works within Hibernate).

Here is how I like to look at it. There are fundamentals at the macro level and at the micro level. At the macro level (i.e. looking at software development as a whole) there are concepts, algorithms and knowledge that will be applicable no matter the type of work you’re doing or the domain you’re working in. We’re talking things like arrays, hashes, trees, sorting, searching, some graph algorithms, caching, certain object oriented concepts and yes, even things like lazy loading and optimistic locking. There are a few others, but surprisingly not that many. You need to know these things backwards. The implications of using them on small and large data sets, common problems they apply to, limitations and how to overcome them etc. You will be playing with these every day, for the rest of your career; surely you owe it to yourself to understand them as deeply as you possibly can. Yet, most people have at best a working knowledge, which has remained at approximately the same level since university. 

At the micro level (i.e. looking at a specific technology, language or library), there will be API details, tools and quirks that you have to internalise fully in order to be truly effective. When I started learning Ruby, I read all sorts of books and blog posts. I found out about various techniques, tools, gems etc. Did any of it make a significant impact on the quality of my code or how fast I was able to write it? Not really, at least not that you would notice, beyond the natural improvements that you would expect from continued usage. Then, at one point, I spent a couple of hours closely looking at the API of the Array class, I don’t remember what my reasons were, but all of a sudden, I noticed myself coding faster and able to maintain a state of flow for longer. You see, I didn’t need to look-up much to do with the Array class any more, I was really surprised at how much of a difference that made (but if you think about how often we use/manipulate arrays, it is not actually that surprising). You know what my advice would be to someone just starting to learn Ruby? Arrays, Hashes, Strings and Files – learn their API’s off by heart as soon as you can, you will find yourself head and shoulders above most other Ruby programmers straight away. That is micro level fundamentals.

A Solid Understanding

Castle

What does it mean to internalise something completely? Let’s take martial arts as an example. You can pick up some books and read all of them. You can have someone show you all the moves and can even try some of them out a couple of times. Does that make you an expert martial artist? Not by a long shot, for that you need to practice, and keep practicing until muscle memory kicks in and the moves become second nature to you. You’ve internalised them. More than that, you don’t spend hours practicing complex combinations (not initially anyway), you spend your time working on basic movements, complex moves are made up of basic ones and if you have those down solid, it is a short step to be able to combine the simple moves to perform the more complex (my apologies to any real martial artists if I’ve oversimplified anything).

Yet, in software, most of us are just like that fan-boy amateur martial artist. We bite off a little bit of Ruby, then some Javascript, moving on to C# and coming back for a taste of Rails, then, back in Java land we take-in just enough Spring to get by and only as much Hibernate as we need to list it as a skill on the resume. We jump from technology to technology, wanting to know and be aware of everything (so we don’t fall behind), but all the while, we would probably have been better off practicing our “basic moves”. It doesn’t matter how many object oriented languages you know, if you’re not precisely sure what coupling, cohesion or abstraction means, your code is likely to be equally crappy in all of them. It doesn’t matter how many inversion of control containers you have used, if you don’t know what problem dependency injection is trying to solve, then you’re just another sheep ready to follow the leader off a cliff (it’s true, sheep will actually do that).

There is a difference between really understanding something and learning just enough to get by (i.e. having a working knowledge). It is usually ok to only have a working knowledge of most of the tech you use day-to-day; there is simply no need to have a deep understanding of the hundreds of libraries and technologies you deal with. The reason it is ok though, is because it’s assumed that you will have a solid understanding of the fundamental principles that all this tech is built upon. You don’t need to know the specifics of the Hibernate or ActiveRecord implementation if you understand how lazy loading, or optimistic locking works. If you have MVC down, then whether you’re using Struts or Rails makes no difference, it’s just an implementation detail. Basically, a good rule of thumb is, the more fundamental the concept the more solid your knowledge of it should be. It’s the whole, solid house – solid foundation thing.

I sometimes think that a generalist with really solid fundamentals could pass for an expert almost every time. Have you ever seen someone walk up to a keyboard, start coding and not stop for 10, 15, 20 minutes, no need to refer to anything? Impressive isn’t it? They seem to produce an amazing amount of useful code in a very short period time and when they leave people whisper the word genius. But there is nothing genius about it, just someone who took the time to learn the micro level fundamentals in that particular area – the rest was just practice. You can do the same, get your macro and micro level fundamentals down solid, after that it’s all about strategic practice, but that is a story for another post. To me, all this just makes a whole lot of sense. I would have thought it would make just as much sense to everyone else and yet I constantly find people to disagree with me on this point – it’s really quite puzzling :).

Images by thivierr and antonychammond

Solving The Towers Of Hanoi Mathematically And Programmatically – The Value Of Recursion

TowersMany programmers don’t really understand recursion – it’s a fact, I’ve seen it time and time again. Even when we understand how it works, we tend to only use it by rote, on problems that we expect to have recursive solutions. For example, we know that a recursive function is the best way to traverse a tree in order – it’s in every textbook and so that is how we do it. There are other problems, that we can come up with, which have intuitively recursive solutions and we don’t have too much trouble with those. It’s when a problem is not intuitively recursive, but has a recursive solution, that is where the difference between being able to use it and really understanding it comes out. Incidentally, this also makes a very good case for the value of math skills in a programming context. Let me demonstrate by looking at Towers of Hanoi as a programming exercise.

Solving Towers Of Hanoi Intuitively

The Towers of Hanoi problem is very well understood. You have 3 pegs (A, B, C) and a number of discs (usually 8) we want to move all the discs from the source peg (peg A) to a destination peg (peg B), while always making sure that a bigger disc never ends up on top of a smaller one. It is a pretty simple puzzle and you can quickly work out the correct sequence to move the discs and solve the problem. But, what if you need to translate your solution into code? For those of you who are aware that this problem has a simple recursive solution, pretend to forget it for a second :).

Being the good problem solvers that we are, we will approach this methodically. To solve the Towers of Hanoi programmatically, we need to find a pattern that will easily lend itself to being expressed in code. While it might seem like it should be obvious (after all we can easily move the disks around to solve it) it is actually very elusive. Before reading any further, do try and come up with the most intuitive solution that you can for this problem (and don’t cheat by looking it up :)). Once you have it, try and code it to work out all the bugs, go ahead, I’ll wait :). If it took you less than an hour, you’re a much smarter developer than I am. I came up with several possible intuitive solutions and got bogged down with every one of them before I got one that worked. Here is the pattern that I noticed which produced a working solution.

  • We start with X number of discs, and assign a parity to each one depending on whether we have an odd or an even number of discs. For example, if we start with 3 discs, the smallest disc (disc 1) will be odd, the next one (disc 2) will be even, and the last one (disc 3) will be odd. But if we start with 4 discs, disc 1 will be even, disc 2 will be odd and so forth.
  • Every disc with the same parity always moves through the same pattern when it comes to their turn to be shifted. Lets assume our three pegs are A, B, and C with A being the source peg (where all discs start) and B being the destination peg (where all discs end up), I call C the pivot peg.
    • If a disc has an odd parity, it always moves in the following pattern A -> B -> C -> A etc.
    • If a disc has an even parity, it always moves like so A -> C -> B -> A etc.
  • If we capture these rules, then the last thing we need to do is to work out which disk to move on every iteration. This is simple, we always, move the smallest disk we can, that wasn’t shifted in the previous iteration.

Feel free to walk through all of this yourself, but I will save you the trouble since I coded all of it up, so I know it works :). Here are the relevant code snippets:

```ruby class IterativeHanoi include HanoiHelpers EVEN_PARITY = “even” ODD_PARITY = “odd”

def initialize(options={:discs => 8}) @discs = options[:discs] @pegs = {:from => [], :to => [], :pivot => []} @peg_array = [@pegs[:from], @pegs[:to], @pegs[:pivot]] @discs.downto(1) do |num| @pegs[:from] << num end @disc_parities = assign_parities @move_rules_by_parity = movement_rules_by_parity end end```

We initialize the class with the number of discs we want to use for our problem. We create three arrays to represent out pegs (source, destination and pivot) and fill the source array with numbers to represent out discs. In the case of 3 discs, the initial state of the problem will look like this.

-------
 1    
 2    
 3    
-------
 A B C
-------

We also assign the parities to our discs and set up all the movement rules according to the parities. The actual code to solve the problem looks like so:

```ruby def solve puts “SOLVING ITERATIVELY!!!” print_special_state “Initial State” source_peg = nil destination = nil while @pegs[:to].length != number_of(@discs) source_peg = next_source_peg(destination) destination = @move_rules_by_parity[parity_of(source_peg.last)][source_peg.object_id] shift(1.disc, :from => source_peg, :to => destination) raise unless state_valid print_state end print_special_state “Final State” end

def next_source_peg(off_limits_peg) source_peg = nil @peg_array.each do |array| next if array == off_limits_peg next if array.length == 0 source_peg = array if source_peg == nil || array.last < source_peg.last end source_peg end```

This does exactly what I described above, picks the peg with the smallest disc that we’re allowed to move and shifts the disc off that peg according to the movement rules, that we set up based on the parity of the disc. It will also print out the state of the puzzle at every step, like so:

-------
      
 2    
 3 1  
-------
 A B C
-------


-------
      
      
 3 1 2
-------
 A B C
-------

etc.

And it will raise an exception if it find that the puzzle has entered an invalid state (a bigger disc is on top of a smaller one). For the full implementation with all the helper methods, have a look on github (I am trying to remember to put more of the code I play around with up there), more specifically here is the class http://github.com/skorks/hanoi/blob/master/lib/iterative_hanoi.rb.

This is actually a pretty decent iterative algorithm. It takes the minimum number of steps to solve the problem and is reasonably simple to implement and understand. And yet, it took quite a long time to figure out the pattern, implement, debug and this is the intuitive solution. Can we do better? There are many different iterative algorithms for solving the Towers of Hanoi, the point is that all the iterative solutions are, if anything, more complex than the one above.

The Mathematical Approach

Math

If we were inclined to think in a more mathematical fashion, we might approach this problem a little differently. Lucky for us the math behind Towers of Hanoi is very well understood, but even if it wasn’t the idea is the same. Let’s have a look.

Mathematically, we want to work out a formula for how many steps ((T_n)) it takes to transfer (n) discs from the source to the destination peg. Looking at the smaller cases and counting the steps manually we see the following.

  • when (n=1), (T_n=1)
  • when (n=2), (T_n=3)
  • when (n=3), (T_n=7)
  • when (n=4), (T_n=15)

From this we can start to infer a pattern. Every time we increase the number of discs by 1, we need to double the number of steps and add 1, which means we can express the number of steps (T_n) taken to transfer (n) discs based on the number of steps taken to transfer (n-1) discs:

\(T_n=2T_{n-1}+1\) where \(n>{0}\)

What we have is a recurrence (or it would be if we added the \(T_0=0\) to it :)). Now we can find the closed form of this recurrence relation and then prove that it always holds true using induction. I am not going to do this here. If you want a more in-depth look at the maths behind Tower of Hanoi (includig the proof), I refer you to Concrete Mathematics. For our purposes here, we can go with the gut-feel and assume that our recurrence always holds true. How does this help?

Well, because we can represent out problem as a recurrence, it means we should be able to code a recursive solution for it.

The Recursive Solution

This is where we need to be able to connect our theoretical math with a practical approach to solving the problem, which brings me back to the start of this post. I believe that being able to apply the math behind a recursive function, in a practical fashion, is where the really deep understanding of recursion lies. Let’s break it down.

What if we can rewrite our recurrence in the following fashion:

(Tn=T{n-1}+1+T_{n-1})

1
2
3
4
5
<p>
  This tells us that to solve the tower for \(n\) disks (<em>which will take \(T_n\) steps</em>) we need to sove the tower for \(n-1\) disks, then solve the tower for \(1\) disk and then solve the tower for \(n-1\) disks again. You can probably already see where I am going with this, there is only a slight intuitive leap we need to make here, regarding where to transfer the disks each time. We need to transfer \(n-1\) disks from the source peg to the pivot peg, then transfer \(1\) disk from the source peg to the destination peg and then transfer \(n-1\) disks from the pivot to the destination and we're done. Here is the code:
</p>

```ruby

class RecursiveHanoi include HanoiHelpers def initialize(options={:discs => 8}) @discs = options[:discs] @pegs = {:from => [], :to => [], :pivot => []} @peg_array = [@pegs[:from], @pegs[:to], @pegs[:pivot]] @discs.downto(1) do |num| @pegs[:from] << num end end

def solve puts “SOLVING RECURSIVELY!!!” print_special_state “Initial State” move(@discs, @pegs) print_special_state “Final State” end

def move(n, pegs = {}) move((n-1).discs, :from => pegs.start_peg, :to => pegs.pivot_peg, :pivot => pegs.end_peg) if n > 1 shift(1.disc, :from => pegs.start_peg, :to => pegs.end_peg) print_state raise unless state_valid move((n-1), :from => pegs.pivot_peg, :to => pegs.end_peg, :pivot => pegs.start_peg) if n > 1 end end```

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
<p>
  Once again we need to do similar setup to initialize the puzzle, but the <em>move </em>function is the interesting one here, it is recursive and it is all that is needed to solve the problem (<em>aside from printing out the state in the middle</em>). As you can see the code is simple and there is no need to have any special handling for larger or smaller numbers of disks, the recursive nature of the function will work it out. Notice how closely the function resembles our recurrence relation from above:
</p>

<p style="text-align: center;">
  \(T_n=T_{n-1}+1+T_{n-1}\) 

  <p>
    Had we gone through the math instead of trying to find the '<em>intuitive</em>' solution we would not only have been finished much faster (<em>it took way longer to code the iterative solution than it did to do the recursive one</em>), but the solution would have been a lot simpler and cleaner (the above code is also on github, in <a href="http://github.com/skorks/hanoi" target="_blank">the same place</a>).
  </p>

  <blockquote>
    <p>
      As an aside, did you notice how expressive the recursive solution looked when written in Ruby. Things like <em>pegs.start_peg</em> and <em>shift(1.disc)</em> etc. I went to a tiny bit of effort to code things in a DSL-like fashion, <a href="http://www.artima.com/rubycs/articles/ruby_as_dsl.html" target="_blank">Ruby being really good for that</a> and it turned out pretty nice (<em>considering the tiny amount of effort involved</em>). I opened up the <em>Integer </em>and <em>Hash </em>classes to add the DSL-like features:
    </p>

    ```ruby

class Integer def disc self end alias_method :discs, :disc end

class Hash def start_peg self[:from] end

def end_peg self[:to] end

def pivot_peg self[:pivot] end end```

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    <p>
      This made everything a lot more readable, the contrast with a $1.
    </p>
  </blockquote>

  <p>
    Recursion can be a very deep concept, functional languages are built on it, so I don't think you can ever understand it in one "<em>Aha!</em>" moment (<em>although you may have plenty of those along the way</em>). I think that what it does take, is a little mathematical grounding and then a lot of practice and it is tough to get the practice if we avoid it, or always go for the intuitive solution, like we tend to do in our day-to-day programming endeavors. This is where books such as <a href="http://www.amazon.com/Structure-Interpretation-Computer-Programs-Engineering/dp/0262011530" target="_blank">SICP</a> or even <a href="http://www.amazon.com/Little-Schemer-Daniel-P-Friedman/dp/0262560992/ref=sr_1_1?ie=UTF8&s=books&qid=1269780880&sr=1-1" target="_blank">The Little Schemer</a>, can be really valuable. I just wish I had time to read them myself :). If you have creative Towers of Hanoi solutions (<em>iterative or recursive</em>) or simply want to share your thoughts on recursion, math, etc., I'd love to hear from you.
  </p>

  <p>
    <span style="font-size: 10px; font-family: trebuchet ms;">Images by <a href="http://www.flickr.com/photos/vshioshvili/229207037/" target="_blank">shioshvili</a>, <a href="http://www.flickr.com/photos/35064820@N00/3950391591/" target="_blank">tkamenick</a> and <a href="http://www.flickr.com/photos/torley/2361164281/" target="_blank">Torley</a></span>
  </p>