Testing Groovy code – an undesirable burden?

Many of the complaints I’ve heard directed at Groovy and Grails derive from the same issue: the compiler doesn’t pick up type errors. People worry that simple typos will make it into production and that they’ll be less productive due to MissingMethod and MissingProperty exceptions popping up when they run the application.

The standard answer to this is a simple one: write tests for your code. In fact, prefer test-driven (or test-first) development. Yet this doesn’t seem to satisfy some people. “What projects do you know that have 100% test coverage?” One or two that have close to 100%. But are the rest reliable? No, not even the Java-based ones. Any code that isn’t tested has a significantly high chance of being buggy. How can this be a serious argument against dynamic languages?

That’s not to say I think dynamic languages should take over from static languages. Both have their place and deciding which to use for any given task will often come down to personal preference. They both have their strengths and weaknesses, but a discussion of those will have to come another time.

Too much testing!

Another argument I’ve heard is that dynamic languages require more testing because you have to explicitly test for type errors that would otherwise be picked up by the compiler. I think this is frankly rubbish. What do we do when we’re testing? We’re checking that code behaves as expected given certain inputs. If there are any type errors in the code under test, you won’t see the expected behaviour.

Let me demonstrate. Say we have a Grails controller {{MyController}} with a corresponding unit test. The test starts simply:

package org.example

class MyControllerUnitTests extends grails.test.ControllerUnitTestCase {
    void testIndex() {
        controller.index()
    }
}

All we do is invoke the index action on the test controller. Now see what happens when we run the test against this controller code:

package org.example

class MyController {
    def index = {
        def id = parms.id
        render "You have reached ID ${ids}"
    }
}

If you run the test in SpringSource Tool Suite, you’ll see something like this:

As you can probably tell, the test is failing because the variable parms does not exist. Not surprising since this was a deliberate typo. Once you correct that typo, the test fails on the next one, ids. Change that to id and the test is now passing.

So, even though the test simply executes the action and doesn’t even bother checking the results, we get feedback on missing properties and methods. Granted, the feedback isn’t as instantaneous as you get in an IDE with a statically typed language, but in my experience it’s not a huge hit on productivity. And while we’re on IDEs, I know that both STS and IntelliJ IDEA will underline properties and methods it can’t resolve, so you would probably notice that parm and ids were mistyped before running the test.

This doesn’t really do much for my argument yet because I’ve just shown that you need an “extra test” to get the same checks you’d get with a static language. But remember that the aim of a test is to ensure that code behaves as it should, so we should check the action renders the string we expect! That’s easily done:

package org.example

class MyControllerUnitTests extends grails.test.ControllerUnitTestCase {
    void testIndex() {
        controller.params.id = 10
        controller.index()

        assertEquals "You have reached ID 10", mockResponse.contentAsString
    }
}

The test still passes, but it now checks that the action is rendering the appropriate string to the response. The key point I want to make here is that the test is no different than it would be if we were testing Java code, yet it will still pick up typos.

What about type-checking?

I may have justified my position when it comes to missing methods and properties, but what about real type errors, such as using a string where an integer is expected or vice versa? This is an interesting question because in dynamic languages, it’s typically the wrong one to ask. How can you have a type error in a dynamically typed language? You might have some code like this:

def n = 100
n = n.substring(1)

which you may see as a type error (a number is being treated like a string), but as far as the language is concerned all we have is a missing method on the value of n. This is important because you could very well add the method substring() to integers, in which case the code would work:

Integer.metaClass.substring = { start ->
    return delegate.toString().substring(start)
}

def n = 100
n = n.substring(1)

In other words, “type errors” in a dynamic language are simple missing method or property exceptions, which I’ve already demonstrated will be covered by your normal tests (without any extra shenanigans).

I would stop there, but that isn’t the whole story when it comes to Groovy. After all, it has static types! It also frequently interfaces with (statically typed) Java code. How does this affect our testing? Ah, I wish there was a simple answer to this. In some ways, passing the incorrect type to a method will be picked up. Try running this script:

def someMethod(String str) {
    println "Some method: $str"
}

someMethod(100)

Boom! You’ll get a missing method exception because Groovy can’t find an instance of someMethod() that takes an integer. Unit tests can handle cases like this, but in practice such problems raise their head once you start wiring your objects together and they start interacting with each other. At that point, you have to start thinking about integration tests – a topic for another time.

Have I convinced you?

The main thrust of this blog post has been to highlight that your bog standard unit tests, which you can run easily from your IDE, will pick up those typos and “type errors” you’re worried about without you having to do any extra work. All you need to do is focus on writing tests that check the behaviour of your code. It’s not trivial to write thorough tests, but that’s independent of the language you use.

Something else you should bear in mind is that It’s a lot easier to test code written in a dynamic language because you don’t have to always create objects of the correct type – they just have to have the appropriate methods and properties. So dynamic languages both strongly encourage you to adopt best practice and make that best practice easier.

Despite all this, I understand that it can be difficult in many environments to write tests for code, let alone write tests before writing any code. But I think this is a problem of culture and habit. Once you get used to writing tests, and then writing them first, the whole process becomes easier. You then find you have more confidence in your code. I can only strongly recommend that everyone takes any opportunity they can to try this approach out and get used to it. You never know, the managers may even come round to realising it’s a good idea!

9 thoughts on “Testing Groovy code – an undesirable burden?

  1. Pingback: Tweets that mention Peter Ledbrook -- Topsy.com

  2. Sakuraba

    I write code in static and dynamic languages and you have not completely convinced me despite having written a very good blog entry about the subject.

    The main point that compile-time-type-checking-people make is the fact that you dont even have to run your tests to find out about the error, whereas in dynamic languages such information is impossible to find out, since anything can be changed/added at runtime.

    In static systems you can tell wether your screws are in the right holes BEFORE the car touches the streets. Wether or not the engine works correctly is not the scope however.

  3. Dierk König

    Good blog post but not quite on the target. People don’t seek compiler checks, the seek _IDE_ checks!
    And a decent IDE can highlight the code above to show where methods are known at editing time and where they are expected to be known only at runtime.
    As a programmer I can match this against my expectations.

    In other words, in Groovy you can have as much checks as you want to have, albeit that the tool that enforces them is not the compiler.

  4. Pingback: Blog bookmarks 04/14/2010 « My Diigo bookmarks

  5. Sakuraba

    @Felipe
    No, as I said, I write code in both language styles. I dont favor any over the other. I would e.g. much rather work an a Django project dthan working on a Seam project, despite having much more experience with Java than with Python.

    The biggest productivity driver is motivation and not a tool. In my experience however libraries written in dynamic languages require extensive documentation to make it easy to get started, because no IDE can tell me what methods you can invoke on which objects. If such documentation is missing, I think then static languages have an edge, because it is easier to build code nagivation tools for them, because a tool just knows which method is going to be dispatched too.

    I favor maturity, developer usablity and proof that something has been put to the test much higher than what language it is written in.

  6. Peter Post author

    @Sakuraba That’s another argument and a perfectly valid one. Does it make you more productive? Certainly compared to developing with a static language without such an IDE. I remember when I first used Eclipse and how much quicker development became because I didn’t have to go through the compilation step.

    But are you less productive if you use a dynamic language and take a test-first approach? I think it depends on how comprehensively you embrace the test-first philosophy. Also, don’t ignore the benefits you get from the expressiveness of something like Groovy over Java and the newer IDE support that highlights potential missing methods and properties.

    As I said, though, it’s a different argument to the ones I focus on in this article. It’s also an argument that boils down more to personal preference and the task at hand than anything else.

    @Dierk That’s mostly true. When you are integrating code across teams, a compiler can be particular useful to detect breaking changes in interfaces and the like. But IDE feedback is probably the main benefit people see.

    Yes, you can get such feedback from the IDE with Groovy as I mentioned, but bear in mind that there’s a significant difference between a compile error, which is very difficult to ignore, and the simple underlining of method or property. The trouble is, a lot of valid things end up being underlined, particularly if you use a lot of untyped variables, method arguments, and return values. I suspect pair programming can make a significant difference here!

  7. Pingback: Choice of the week – 15/2010 « Ivko's Blog

  8. Wanderson Santos

    For some years IntelliJ IDEA Community Edition have an awesome static check compilation for Groovy, and do it in realtime (with highlighting), its blazing fast (Win/Linux) and its free.

    I honestly think the “IDE or not IDE” talk isn’t pragmatic. In the end, we just use what makes us more productive.

    Best regards!

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>