Digital Compulsion

Function closures can encapsulate better than objects

2010-06-04T07:31:00.014+01:00

I was surprised to discover recently that a function closure can encapsulate code better than an object. Suppose I have the follow classes:


  public interface Car {
    void Drive();
  }


  public class CheapCar : Car {
    public CheapCar(int milesPerGallon, string name) {...}

    public void Drive() {
      Console.WriteLine("Crawling along...");
    }
  }


  public class FastCar : Car {
    public FastCar(double acceleration, string name) {...}

    public void Drive() {
      Console.WriteLine("Speeding along...");
    }
  }

Now suppose I want to build a DSL for constructing them, I want the usage to look like this:


  carFactory.FastCar(0.67).WithName("Speedy");
  carFactory.CheapCar(123).WithName("Efficient");

My DSL conveyor-belt will need a few stages, each receiving a parameter, each bundling it up with the parameters acquired thus far and handing over to the next stage or actually building a car:


  public class CarFactory {
    public NameCapturer FastCar(double acceleration) {
      return new FastCarNameCapturer(acceleration);
    }
    
    public NameCapturer CheapCar(int milesPerGallon) {
      return new CheapCarNameCapturer(milesPerGallon);
    }
  }


  public interface NameCapturer {
    Car WithName(string name);
  }

  
  class FastCarNameCapturer : NameCapturer {
    public FastCarNameCapturer(double acceleration) {...}
    
    public Car WithName(string name) {
      return new FastCar(acceleration, name);
    }
  }

  
  class CheapCarNameCapturer : NameCapturer {
    public CheapCarNameCapturer(int milesPerGallon) {...}
    
    public Car WithName(string name) {
      return new CheapCar(milesPerGallon, name);
    }
  }

Because there are two routes to specifying a name and thus two types of possible accumulation parameter there needs to be two NameCapturer implementations.

Let's see what happens when we accumulate parameters inside a function closure instead of directly inside a capturer object:


  public class NameCapturer {
    public NameCapturer(Func<string, Car> pipeline) {...}

    public Car WithName(string name) {
      return pipeline(name);
    }
  }


  public class CarFactory {
    public NameCapturer FastCar(double acceleration) {
      return new NameCapturer(name => new FastCar(acceleration, name) as Car);
    }

    public NameCapturer CheapCar(int mpg) {
      return new NameCapturer(name => new CheapCar(mpg, name) as Car);
    }
  }

The capturer object now provides just the language of the DSL whilst the function closure holds & fully encapsulates both the conversation that's been captured thus far and the final result.

It's very easy to generalise NameCapturer. Whilst we are at it let's add an age to each type of car and capture this with a generic AgeCapturer, a small helper class to smooth away noise from generics will be handy too.


  public class NameCapturer<T> {
    public NameCapturer(Func<string, T> pipeline) {...}

    public T WithName(string name) {
      return pipeline(name);
    }
  }


  public class AgeCapturer<T> {
    public AgeCapturer(Func<int, T> pipeline) {...}

    public T WithAge(int age) {
      return pipeline(age);
    }
  }


  public class DSLHelper {
    protected AgeCapturer<T> AgeCapturer(Func<int, T> pipeline) {
      return new AgeCapturer<T>(pipeline);
    }

    protected NameCapturer<T> NameCapturer(Func<string, T> pipeline) {
      return new NameCapturer<T>(pipeline);
    }
  }

All this stuff could conceivably exist in a 'DSL combinator' library. It's now possible to build a simple DSL for creating cars with just the following code:


  public class CarFactory : DSLHelper {
    public NameCapturer<AgeCapturer<Car>> FastCar(double acceleration) {
      return NameCapturer(name => AgeCapturer(age => new FastCar(acceleration, name, age) as Car));
    }
    
    public NameCapturer<AgeCapturer<Car>> CheapCar(int mpg) {
      return NameCapturer(name => AgeCapturer(age => new CheapCar(mpg, name, age) as Car));
    }
  }

Our Money Madness (an article by the Mises Institute)

2008-10-07T18:49:00.000+01:00

I've had an interest in economics for quite a while particularly the writings of Ludwig von Mises and more generally of the Austrian School. This school asserts (and I for one am quite convinced) that the present crisis was predicted by their theories of inflation, fiat currencies and the boom-bust business cycle. I'm no expert so I'd like to hand over to an article on the 'Mises Institude' website:

Our Money Madness
We flatter ourselves, in this technological age driven by financial innovation and mind-boggling efficiencies, that we know more than any previous generation. But there is lost knowledge, among which is the knowledge of what sound money feels and looks like, what it does, who makes it and why, and how it holds its value.
So let us revisit Robert Louis Stevenson's classic story, Treasure Island, and the climactic scene where the pirates and their companions have finally found their treasure and prepare to haul it away. The narrator reports as follows ...

Example of mutable extension to immutable domain model

2007-08-29T09:17:00.001+01:00

I must not have communicated the mutable extension idea clearly enough, as I've been accused of breaking the immutability and encapsulation of the domain model which would, I agree, make the domain model sloppy. I deplore mutability in domain models and anemic / 'behaviourally absent' nonsense such as setters / getters.

I haven't done either of those things (as far as production code is concerned) here's a worked example which demonstrates this.

Here's the immutable, encapsulated, distinctly unsloppy domain model:


class Person {
  private final String name;
  private final int age;

  public Person(String name, int age) {
    this.name = name;
    this.age = age;
 }
}

Here's code that's only available for testing:


class MutablePerson extends Person {
 private static final String DEFAULT_NAME = "Bob";
 private static final int DEFAULE_AGE = 32;

 public MutablePerson() {
  super(DEFAULT_NAME, DEFAULT_AGE);
}

public MutablePerson withName(String name) {
  return new MutablePerson(name, getAge());
}

public MutablePerson withAge(int age) {
  return new MutablePerson(getName(), age);
}

private String getName() {
  return accessor.get(this, "name", String.class);
}

private int getAge() {
  return accessor.get(this, "age", Integer.class);
}

private final ReflectiveAccessor accessor = new ReflectiveAccessor();
}

Now it's true that the testing code has broken encapsulation but only as far as the testing code is concerned, not for the production code. The domain model itself is unchanged. It should be noted that there is a one-to-one relationship between the immutable domain model class and the mutable extension and that the break in encapsulation for a given domain model class propagates no further than the corresponding mutable extension; and again this break only exists in the testing code.

So I would say the testing code has a minor encapsulation breach which allows the testing code to have a more succinct form than possible with the test builder (it essentially removes the need for all the build() methods).

Unlike the Test Builder it would not be suitable to move the Mutable Extensions to the production code. Immutability & non-anemicness helps keep domain models from becoming 'bags-o-data', having a mutable version of each domain model in the production code would be too much of a temptation.

Mutable extension to immutable domain model: an alternative to the Test Data Builder pattern

2007-08-28T01:24:00.000+01:00

Nat Pryce recently blogged about an alternative to the Object Mother pattern: creating a builder class for each class in the domain. He's able with this idea to get testing code to look like this:

Invoice invoiceWithNoPostcode = new InvoiceBuilder()
   .withRecipient(new RecipientBuilder()
       .withAddress(new AddressBuilder()
           .withNoPostcode()
           .build())
       .build())
   .build();

I think I can improve on this and achieve code that looks like this:

MutableInvoice invoiceWithNoPostcode = new MutableInvoice()
   .withRecipient(new MutableRecipient()
       .withAddress(new MutableAddress()
           .withNoPostcode()));

Instead of creating a builder for each domain class I've created a mutable extension to each domain class, it is the mutable extension I'm using in the test.

I don't quite like the code above, I think the following looks better:

Invoice invoiceWithNoPostcode = new Invoice()
   .withRecipient(new Recipient()
       .withAddress(new Address()
           .withNoPostcode()));

I've changed the code to use the same name for the mutable extension (with a different package of course) because I figure the tests can just refer to the mutable versions and it saves me having the 'Mutable' prefix on everything. Naturally I only make the mutable extensions available for testing.

Here's how the mutable Invoice extension might look:

package model.mutable;

public Invoice extends model.Invoice {
  public Invoice(Recipient recipient, InvoiceLines invoiceLines) {
      super(recipient, invoiceLines);
  }

  public Invoice() {
      // choose the same defaults here as you would in the InvoiceBuilder
      this(null, null);
  }

  public Invoice withRecipient(Recipient recipient) {
      return new Invoice(recipient, getInvoiceLines());
  }
}

I've had to add protected accessors to the domain model; this bothers me a little as they make it easier to slip towards an anemic data model. It might be worth using some reflective code in the extension class instead:

public class Invoice extends model.Invoice {
 ...

 private InvoiceLines getInvoiceLines() {
     return new ReflectiveAccessor().get(this, "invoiceLines", InvoiceLines.class);
 }
}


public class ReflectiveAccessor {
 public  T get(Object fieldOwner, String fieldName, Class fieldClass) {
    ...
 }
}

Branching by Abstraction

2007-04-30T04:15:00.000+01:00

Ok Paul, I'll bite.

Some time ago I was working for a client which was experiencing merge hell, come to think of it all the large clients I've worked for routinely experience merge hell, hopefully this isn't correlated with my presence. At the time I mused that when someone creates two implementations of an interface they are effectively branching and so I
voiced that instead of using version control to branch some code perhaps we should Branch by Abstraction. I was working with Paul Hammant at the time and he's apparently ran with this idea and achieved some notable improvements with it.

I don't think BBA it is really that novel, it's just another way of saying 'Hey!, Separate your Concerns', but since a lot of coding-land seems to be engaged in a energetic fight to do the opposite it's probably good to have as many ways of stating the bleedin' obvious as possible.

I'm quite a lazy creature by habit so I've only drafted an article on this on C2, and intend (honest) to polish it up over time, here's a copy of the first draft:

Summary: Don't separate differing concerns by using a VCS, use an abstraction instead.

Long story: Conceive the changes of a project as providing alternate implementations of abstractions already present in the software, or of extracting out an abstraction and making it explicit and them providing an alternate implementation. By doing so the abstraction and the varying implementations provide the separation and branches are need much less.

Consider a possible project that aims to add more data to a business report, say a customer invoice. Suppose that the data being added is the date on which a transaction took place. The current CustomerInvoice might obtain a bunch of LineItems and render them. The new requirement is to add date information to the LineItem and display this on the invoice. At the same time as this project is going on bug fixes are being applied the existing invoice, so we can't just make changes to the invoice.

Step 1) Extract out abstractions needed by the project which are not currently explicit in the code.

1a) Extract the code which provides line items into an interface and implementation: LineItemProvider, NameAndCostLineItemProvider, make the implementation the default the CustomerInvoice uses. Deprecate the implementation with a note to its impending replacement. Test and release to production.

1b) Extract the code which renders line items into an interface and implementation: LineItemRender, NameAndCostLineItemRenderer, make implementation the default the CustomerInvoice uses. Deprecate the implementation with a note to its impending replacement. Test and release to production.

Since the changes in Step 1 should be expressible as a series of refactorings it should be possible to make the test step very short, or arguably unnecessary.

Step 2) Create alternate implementations of the abstractions created in step 1, or already existing before the project began. NameCostAndDateLineItemProvider, NameCostAndDateLineItemRenderer.

Step 3) Release the new implementations in a QA environment configured to use the new implementations.

Step 4) Release the new implementations in production configured to use them.

Optional step) Back out the implementations from production, revert back to the previous implementations at run time.

Whilst most of the project is going on (steps 2 & 3) bug fixes can still be applied to all other aspect of the system, and the dated-invoice project just receives these changes for free, as they are only changing the line item providing / rendering aspect. If a bug fix is needed in the line item providing / rending aspect the bug fix team easily become aware of the new implementation via the deprecation notice, alternately the dated-invoice team can monitor changes to the undated-implementation.

If another project starts up whilst the dated-invoices is chugging along then it will have the same steps, and will involve work on some other abstraction. Should another project need to do work on the same abstractions they too will have the deprecation as a notice to be careful and consider interaction with the ongoing work on dated-invoice, otherwise they can work merrily in ignorant.

So in short the encapsulation that abstractions provide can work for projects just as well as they work for code. When changes occur in other projects then as long as it doesn't involve the abstractions I'm working on in my project then I don't need to care.

Compare this with BranchByVcs. In this one creates a fork of a much larger set of code, probably all of it. Other projects / bug fixes will occur in the same code and later the two changes will be merged together, the merge will take place a series of poorly understood textual changes with an unspecified focus, additionally the merging of changes from some project is usually done by people not on the project.

Poetic / pathetic version control

2007-01-25T00:31:00.000Z

Old codebase
Clearcase update
Flop

Anti-censorship

2007-01-20T13:47:00.000Z

A colleague raised a question about whether he should self-censor to remain visible to people behind the 'great firewall of China' or retain prohibited material and 'deny' all his content, prohibited or not.

I offer the links below as my answer.

Appeasing Dictatorship: From Munich to Hong Kong: http://www.aynrand.org/site/News2?page=NewsArticle&id=5137

Save Tibet website:
http://www.savetibet.org/index.php

Tiananmen Square, Wikipedia article:
http://en.wikipedia.org/wiki/Tiananmen_Square_protests_of_1989

Taiwan independence, Wikipedia article:
http://en.wikipedia.org/wiki/Taiwan_independence

China Human Rights Fact Sheet:
http://www.christusrex.org/www1/sdc/hr_facts.html

Java mocking without injection, same line count as Mocha

2006-09-14T09:54:00.000+01:00

The Ruby Mocha/Stubba mocking framework written by James Mead allows one to code collaboration expectations very clearly in very few lines.

Stephen Chu has expressed some disbelief that the same would be as readable in Java. However I've found that I can mock in Java with comparable clarity in the same number of lines as Ruby.


public class LaptopTest extends picounit.TestCase
  public void powerOnFailsWhenBatteryTooLow(Battery anyBatteryInstance) {
    should.call(anyBatteryInstance.meter()).andReturn(9);

    Laptop laptop = new Laptop();
    laptop.powerOn();

    verify.that(laptop.powerStatus()).isEqualTo(Power.Off);
  }
}

I've implemented many different ways of mocking in PicoUnit, all of them using some flavour of dependancy injection.

I haven't actually implemented the above one which requires much more infrastructure, it requires that PicoUnit modify every single class under test, specifically it requires all methods of all classes to have a pre-check injected into it (by altering the bytecode during class loading) for whether the method in question is being mocked and a subsequent redirect into PicoUnit if it is or a fall-through into the ordinary code if it isn't.

I've already done plenty of class bytecode manipulation / observation for PicoUnit, many of the features of PicoUnit would be impossible without this, there's so much bytecode stuff going on that PicoUnit has its own mini-framework for just this purpose, so I should be able to implement injectless mocking easily enough.

Convention, Configuration, and Communication Entropy

2006-04-28T06:20:00.000+01:00

The ever more popular web framework, Ruby on Rails, has at its core an interesting idiom: Convention over Configuration. The idea goes as follows: for the most part a data model will map directly to a database schema, and for the most part the display and editing of the data will map directly onto the model, therefore: don't specify the mapping in configuration, assume the mapping exists 1-1.

The long established ideas of Algorithmic Information Theory back up these design choices very well, in particular the idea of Communication Entropy applies directly to this situation. The entropy associated with a given communication relates to the amount of information that one learns upon receiving the communication. So for instance when one learns that a table PERSON has a data model equivalent Person and a view person.rhtml then one learns very little because this 1-1 type mapping is very common.

In communication theory one uses the entropy of a message to compress the message, so for instance in English the letter E crops up very often, one learns very little from a message from the presence of E (you can test this by removing all the E's from this text, or indeed all the vowels, you will mostly likely find the text completely readable without them), therefore when we want to send an 'E' down a communication line we send the smallest encoding: 0, leaving the whole of the rest of the alphabet to begin with 1 (t = 101, a = 110, etc.). When one encodes English like this on average a letter can be communicated with about 2 bits. Without encoding the average would be around 5 bits, over twice as large.

Perhaps if the same ideas were applied to software we could also see a reduction in message size. In fact this idea underpins Domain Specific Languages (like those Rails uses): don't communicate that which you can take as a given.

The parallels between TDD derived code and Artificial Neural Networks

2006-04-28T06:19:00.000+01:00

I'd like to draw an analogy between artificial neural networks and the code that often gets produced by poor TDD, i.e. TDD done with little respect paid to refactoring.

Artificial Neural Networks

An artificial neural network (ANN) is a graph like structure that maps some set of inputs to a set of outputs. The ANN can make very complex mappings due to having multiple layers and weightings between nodes in the network.

One example of an ANN is one which can recognise a human face. To obtain such a network one stimulates a randomized network with samples of a face and of non-faces. For each sample the network will produce an output. One takes the output and calculates the difference between it and the desired output and then that difference is back propogated through the network. Back propogation of the 'error' causes the network to learn, to produce smaller and smaller errors.

After training one has a very useful tool that can classify things sometimes more accurately than a human can. The faustain bargain one has agreed to is this: One doesn't know how the network classifies its input. The 'knowledge' is just a bunch of numbers, it's a black box.

Code produced by TDD

With TDD the tests take the same role as input to the code and expected output. The developer takes the difference in actual and expected output and back propogates changes to reduce this difference to zero.

At the point when all the tests pass it can be very tempting to move onto the next feature but if one succumbs to this temptation, and does so regularly then the resulting code will closely resemble the neural network, it will work, but you won't know how it works, it too will be a black box.

Danger of over-specificity

Once one has a trained network one might have the danger of having an over specific network: it can recognize all of the faces in the sample set but cannot recognize faces outside the set, even if we might feel those additional faces are too similar to warrant misclassification. To address this problem one usually divides the sample into two sets; one set being used for training and both being used to verify.

But with TDD there is no equivalent to having a sample of tests used to drive the code and additional tests used to verify the code. So not only will the code be a black box but it could very likely be an over specific black box.

Conclusion

I find the conclusion to all of this quite obvious: Refactor your code if you want to retain the ability to understand it! Even if you have great tests and great coverage you may still have a murky black box that nobody wants to work on.

(*) It may be that black box code cannot be tested with good tests and so the existence of black box code will be noticed by the developers as an increasing difficulty when trying to write good tests.