Solution #1: Exclude, exclude, exclude

If you don’t need commons-jxpath, you can add these three exclusions to the gt-xsd-core dependency declaration.

    <dependency>
        <groupId>org.geotools.xsd</groupId>
        <artifactId>gt-xsd-core</artifactId>
        <version>${gt.version}</version>
        <exclusions>
            <exclusion>
                <groupId>commons-jxpath</groupId>
                <artifactId>commons-jxpath</artifactId>
            </exclusion>
            <exclusion>
                <groupId>xml-apis</groupId>
                <artifactId>xml-apis</artifactId>
            </exclusion>
            <exclusion>
                <groupId>xml-apis</groupId>
                <artifactId>xml-apis-xerces</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

This at least works for me given the subset of geotools functionality I am using (primarily encoding and parsing OGC Filter objects to and from XML so I can persist them with Hibernate). It is still pulling in xercesImpl and that appears to be enough, at least to get my unit and integration tests working.

Solution #2: Using Dependency Management

If you need the commons-jxpath functionality in gt-xsd-core, add this to your dependencyManagement section of your POM:

    <dependencyManagement>
        <dependencies>
            <!-- The gt-xsd-core tries to bring commons-jxpath 1.2 in.  That messes up the maven junitreport plugin for
            some reason (like links to a version of xalan or xml-apis or something.  The 1.3 version doesn't have that
            issue. -->
            <dependency>
                <groupId>commons-jxpath</groupId>
                <artifactId>commons-jxpath</artifactId>
                <version>1.3</version>
            </dependency>
        </dependencies>
    </dependencyManagement>

and then keep the exclusions for the xml-api artifacts as above:

    <dependency>
        <groupId>org.geotools.xsd</groupId>
        <artifactId>gt-xsd-core</artifactId>
        <version>${gt.version}</version>
        <exclusions>
            <exclusion>
                <groupId>xml-apis</groupId>
                <artifactId>xml-apis</artifactId>
            </exclusion>
            <exclusion>
                <groupId>xml-apis</groupId>
                <artifactId>xml-apis-xerces</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

This is what I have setup in my POM right now because I think I might need jxpath soon. I also tried adding a node in dependency management to use the 1.3.03 version of xml-apis rather than 1.0b2 (which is what was pulled in by default), but that doesn’t appear to make any difference. I still had to have the 2 exclusions under gt-xsd-core for things to work.

So what was the problem?

Now, a little more about the problems I had. I had two types of errors:

Maven junitreport conflict – When I added the dependency for gt-xsd-core and then ran my unit tests using the maven grails plugin (mvn grails:exec -Dcommand=test-app -Dargs="-unit"), my tests would run fine (except for one which I’ll detail below), but then I would get this error when the junitreport plugin tried to run its XSL transform:

Embedded error: java.lang.reflect.InvocationTargetException
Provider org.apache.xalan.processor.TransformerFactoryImpl not found

Normally I would suspect that the geotools dependencies brought in some new version of the xalan library and it didn’t have an old class that the junitreport plugin required. To check this, I compared the output from both maven dependency:classpath and dependency:tree between my trunk branch and the branch where I had added the geotools dependencies. The comparision showed that only one one dependency had changed between the branches; a newer version of commons-pool was used. All other classpath mods were new libraries that were not previously included at all. In fact, xerces and xalan was not included by the trunk branch at all. I am sure that someone with more recent experiences with xml-apis, xerces, and xalan knows exactly what was happening. My guess is that by including the xml-apis-xerces dependency, some piece of the junitreport plugin thought that it could use xalan as opposed to some other XSL lib that it would use when xerces was not present. This is supported by the fact that if I added xalan as a dependency, the error went away. If I have time, I’ll look at the docs for junitreport one of these days and see if that sheds some light on the issue.

PDFImageWriter/NodeList conflict – One of the unit tests exercised our use of PDFBox. If I did not have the above exclusions and I include xalan, this error appeared during the PDFImageWriter constructor:

java.lang.LinkageError: loader constraint violation: loader (instance
of <bootloader>) previously initiated loading for a different type with
name "org/w3c/dom/NodeList"

I haven’t tracked down exactly why this linkage conflict occurred. The bizare thing is that I inserted some link-loading debugging that I used in the past to try to see where NodeList was coming from in both my trunk and geotools-enabled branches:

def clazz = org.w3c.dom.NodeList.class
def codeSource = clazz.getProtectionDomain().getCodeSource();
println ("Source Location: " + codeSource);


In trunk (i.e. no geotools and working fine), it came back as null; NodeList wasn’t loaded at all. In the geotools-enabled branch, it would load (from xerces, I think), but then had that conflict.

At this point, I have a solution (two, actually) and have spend pretty much a whole day researching various paths, so I need to wrap it up and move on. If anyone has a better solution or thoughts as to why exactly there were problems, feel free to comment.

One last note in case there is any confusion from me mentioning xalan so much – I did not end up adding a dependency for xalan to my POM. Just using the exclusion statements listed above solved the issue.

The luck part was that I remembered seeing a checkin from a teammate a couple of weeks ago where he got rid of direct use of a grails.converters.deep.JSON object (which is deprecated) and instead switched to a normal grails.converters.JSON object with a use('deep') statement:

def get = {
def o = OurClass.get(params.id);
JSON.use("deep")
render(contentType: "application/json", text: o as JSON)
}

This change was in a different controller for a different domain object though. Why was it affecting our other domain class? That’s where the experience part came in. A few years ago I had to track down why hibernate transactions would fail seemingly at random in another web app when running under JBoss. I could see why an initial transaction would fail (we’d get a timeout and not rollback properly), but there was no reason why later requests would cause failures. That’s when I discovered the downside of request thread pooling. Under most web contains, HTTP requests are handled by a pool of reusable threads. That means changes to thread local variables stick around between requests unless explicitly reset. In the case of the Hibernate bug, since the session was not getting closed properly, it wasn’t getting fully reset and cleared out, so subsequent requests that used that thread were getting the old session and failing because there was no way to get it back to a good state.

For the current JSON conversion bug, the JSON.use method was calling ConvertersConfigurationHolder.setTheadLocalConverterConfiguration(...), thus making the deep converter stick around for that thread. So when future requests came in to any controller on that thread, all JSON conversion calls were made with the deep configuration.

The solution is straightforward. Use the JSON.use(String, Closure) call instead when you want to make a deep call:

def get = {
def o = OurClass.get(params.id);
JSON.use("deep") {
render(contentType: "application/json", text: o as JSON)
}
}

This sets the configuration only for the closure, saving you from thread-based side effects later on.

I hear a great big, “Huh?”

First, a little background on converters. (You may want to get the Grails source code and have directories open to the org.codehaus.groovy.grails.web.converters and grails.converter packages before diving into this.) When a JSON converter is created, it obtains a ConverterConfiguration from the global ConvertersConfigurationHolder. ConverterConfiguration has several properties which influence the conversion process, but the most interesting for this discussion is its prioritized set of ObjectMarshallers. ObjectMarshallers are what do the actual work of turning an object into a JSON representation. Each ObjectMarshaller handles a certain set of classes, like arrays, maps, beans, grails domain objects, etc. When the JSON object is ready to convert a data object, it calls config.getMarshaller(data). The config iterates through its list of marshallers, calling marshaller.supports(data) on each. Since multiple marshallers may be able to handle a specific class (like GroovyBeanMarshaller and DomainClassMarshaller can both handle domain classes), whichever marshaller has highest priority (i.e. is called first) will be invoked.

When the standard ConverterConfiguration is initialized, the DomainClassMarshaller is given higher priority than the GroovyBeanMarshaller. Thus, it handles the conversion of domain objects during normal Grails application execution even though domain objects are also groovy beans. However, the DomainClassMarshaller.supports() method has a slight twist to it. It depends on there being a GrailsApplication object registered in the ConverterUtil class in order to tell it that a particular object is a domain object. Without the GrailsApplication object, ConverterUtil.isDomainClass() always returns false, causing DomainClassMarshaller.supports() to also return false. The ConverterConfiguration next checks the GroovyBeanMarshaller. Its supports() method returns true, so it handles the conversion in this case.

So why is this a big deal? For most domain object, it probably isn’t. Both marshallers render the primary properties of the domain object. However, the DomainClassMarshaller only renders properties return by domainClass.getPersistentProperties() while GroovyBeanMarshaller renders all properties, including transient ones. Because I needed some transient properties in the JSON representation, my unit tests worked great (since there is no grailsApplication setup, so the GroovyBeanMarshaller did the full rendering), but then my code failed when I ran the integration tests and the real app (since the DomainClassMarshaller handled the rendering).

How do you fix this? It’s pretty easy once you know where to look. First, I wanted to make my unit tests behave like the real app, so I had to get them to fail. Unfortunately, the conversion classes are written in java, not groovy, so you can’t just do a quick override of the ConverterUtil metaclass to get it to return what you want. You actually need to setup a GrailsApplication object. Rather than trying to create and initialize a whole one though, I determined that I could create a small stub class that overrode the two methods I needed:


import org.codehaus.groovy.grails.commons.*
class GrailsApplicationStub extends DefaultGrailsApplication {
def artefacts = [(DomainClassArtefactHandler.TYPE):[:]]

GrailsClass getArtefact(String artefactType, String name) {
def retVal = artefacts[artefactType] ? artefacts[artefactType][name] : null
return retVal
}
}

When initializing the unit test, I created the stub and set in the particular domain class that I wanted to be recognized. Then register the application stub with the ConverterUtil class:


def grailsApp = new GrailsApplicationStub();
grailsApp.artefacts[DomainClassArtefactHandler.TYPE][YourClass.name] = new DefaultGrailsDomainClass(YourClass);
ConverterUtil.setGrailsApplication(grailsApp);

Instances of YourClass will now be handled by the DomainClassMarshaller in the unit test. When the unit test checks for the existence of a transient property in a JSON representation, it should fail.

Next, I needed to change the marshaller prioritization so that the desired domain class was handled by a regular GroovyBeanMarshaller. Since I always wanted the GroovyBeanMarshaller to handle the class, I inserted this code into my unit test setup (and later into BootStrap so the behavior would apply to the full app):


JSON.registerObjectMarshaller(YourClass, {o, c ->
new GroovyBeanMarshaller().marshalObject(o, c)
})


This creates a ClosureObjectMarshaller that handles YourClass objects and gives the new marshaller the default priority of 1 (which puts it at the top of the priority list). When processing an object, this closure marshaller simply passes it through to a GroovyBeanMarshaller. Since the new marshaller only handles YourClass objects, the rendering behavior for all other domain objects is not affected.

This resolved my original issue. The JSON rendering of my object is now correct when I run my app. I did go down a side path while coming to this final solution. If you only want to override how a YourClass object is rendered some of the time, you could instead register a named configuration for it like this:


JSON.createNamedConfig("FullYourClassRender") {cfg ->
cfg.registerObjectMarshaller(new ClosureOjectMarshaller(YourClass, {o, c ->
new GroovyBeanMarshaller().marshalObject(o, c)
}))
}

Then when you want to use that particular configuration, you wrap the JSON object calls in a use statement:


JSON.use("FullYourClassRender") {
render(contentType: "application/json", text: myObject as JSON)
}


Other configuration options are also available. Take a look at the JSON class to see them.

I hope this helps document the conversion process a bit.

org.codehaus.groovy.grails.web.json.JSONException: “Misplaced object”

It turns out that the JSON converter has a writer property that is a JSONWriter object.  The act of showing the writer in the variables window of the Intellij debugger causes its toString() method to be called by the IDE (though any breakpoints set in the class do not get triggered by the call).  This, in turn, causes the writer to actually do the writing of the object which, when complete, changes the writer’s state property to DONE.  When the real code execution tries to process the data, the JSONWriter throws the exception because it thinks it has already finished rendering.

The trick to step through the JSON converter is to define a custom Type Renderer for the JSONWriter class that does not invoke the “toString” method.  A blog post on the Intellij site defines how to setup a Type Renderer for a class.  I set mine up so that the node renders simply as “JSONWriter” since I don’t need to see the details for it.  One caveat though – I found that the new Type Renderer didn’t affect entries already in my variables window until I right-clicked one and selected “View As -> JSONWriter”.  Then the view changed to use the new display settings.

Now back to debugging…maybe I’ll actually figure out the real problem sometime before the sun comes up.

Working with what I could find, I came across a blog post from March 2008 that covered integration testing with controllers, much like the grails User Guide.  It at least had some explicit examples of how to pass in parameters, so I hoped the approach would work with my ControllerUnitTestCase implementation.  Using the code snippets in the post as a guide, I setup my test something like this:

controller.request.params = [layoutType:'filmstrip',compKey:'testKey']
controller.create()

This looks good, right?  The test scaffolding connects the injected params property of the controller with the params property of the request.  Should work like a charm.

And so started my descent into WTF Land.

The tests compiled and ran fine, except that no params were getting passed through to the controller.  I assumed that the old documentation for integration testing no longer exactly applied for the new unit test setups.  Eventually I dug through enough of ControllerUnitTestCase, MvcUnitTestCase, and MockUtils that I figured out there was a way to get direct access to the params property of the controller.  “Makes sense,” I thought, “sort of a mini-mock.  Who needs the request for a unit test?”  Great, so let’s try this:

controller.params = [layoutType:'filmstrip',compKey:'testKey']
controller.create()

Run again and…

groovy.lang.ReadOnlyPropertyException: Cannot set readonly property: params for class: YourController

Erk.  Closer inspection of grails.test.MockUtils.addCommonWebProperties() reveals how the params property is added to the controller class with only a get method.  Okay, so we can still manipulate the map once we get it.  One more try:

controller.params.putAll([layoutType:"filmstrip",compKey:'testKey'])
controller.create()

Success!  Finally, information is going where it needs to be.  I later realized that section 9.2 of the user guide does actually show an example similar to this, except that it sets each item on params individually:

def controller = new AuthenticationController()
controller.params.login = "marcpalmer"
controller.params.password = "secret"
controller.params.passwordConfirm = "secret"
controller.signup()

“Oh well,” I say to myself, “at least I got it working.  I’d better post a comment on that old blog post so that others know the new way to set parameters.”  I returned to the blog page, moved down to the comments section… and saw that someone else had posted a comment just last month thanking the author for the good information.  Wait a second… you normally don’t thank someone if their examples don’t work.  So I scrolled up and took a closer look at the examples again:

controller.request.parameters = [sender:'me',msg:'test']
controller.create()

Anyone with a better eye for detail than me see the difference that sent me down the rat hole? It’s request.parameters, not request.params as I had entered.  For the 100th time today, Doh!

But wait a second (or an hour, as the case may be)… If I was calling request.params and there is no params property, how did my test run at all?  Dig, dig, dig… It turns out that the request is a GrailsMockHttpServletRequest object.  Scanning through that class, I see the ever so powerful and ever so dangerous setProperty(String name, value) override.  If you try to set a property that doesn’t exist, it adds the value as an entry in the request’s attributes map.  So the test scaffolding was perfectly happy with my request.params call.  It nicely tucked my params map away in the attributes and continued on its way.

Sigh…it almost makes me miss compile-time method checking… but not quite.

Fast forward to today when I finally had a strong need for a controller unit test setup.  I dove into the code and realized three things very quickly:

1. You don’t need to call mockController().  It does that for you based on the name of the test class itself.
2. You don’t want to call the constructor of your controller class in the test cases.  Use the newInstance() method instead so that all the proper scaffolding is put in place.
3. You don’t even have to call newInstance() most of the time because the setUp() method inits a controller variable for you already.

Doh!  I wish I had taken 15 minutes back in August to look at the source.  It would have saved quite a bit of time by encouraging us to get proper controller test cases in place faster.

To add insult to injury, after spending that time and more getting some test cases just right, I decided to try a search for more documentation online again.   This time there was an answer to my query for help.  A post to the grails-user mailing list less than a month ago generated several responses with useful info, including Scott Davis posting about the most recent addition to his Mastering Grails series which was published just days before the post.

So nine months after the release of Grails 1.1 there is now good documentation on this great piece.  Many thanks to those who pulled it together.

Who says theoreticians don’t know how to have fun?