Archive

Archive for the ‘Uncategorized’ Category

I am Alanis Morissette

December 15th, 2011 No comments

Isn’t it ironic that one of your posts attracts the attention of a grails guru…and it’s because you misspelled his name. (Sorry about that, Burt. I updated the post.) Even better, my blog software then blocked Sir Beckwith from posting because it thinks he is part of a spammer network. Shortly after it blocked him, it let through a comment from a spam bot. Sigh. Looks like I have some research to do since I can’t get WP-Spamfree to properly generate a log telling me why it blogged him.

Categories: Uncategorized Tags:

Running latest Groovy from Maven

April 5th, 2011 2 comments

Say you have a groovy-project that you build with maven.  You use the org.codehaus.gmaven:gmaven-plugin to compile your groovy code and run groovy tests without a problem.  Then you add some features or tests that need groovy 1.7.  You add the proper dependency and version to the <dependencies> section of your pom, run your test… and watch it blow up because the gmaven-plugin defaults to using groovy 1.6.  So you dig around on the web and find references for how to use the <providerSelection> tag of the gmaven-plugin to get your code compiled with 1.7 and to use 1.7 when running tests.  Things seem good.  Until…

You add a feature that requires some version of groovy greater than 1.7.4 (the version included with the latest gmaven-plugin, 1.3).  In my case, I used the @Delegate annotation with some inheritance in a test configuration and hit a bug that was fixed in groovy 1.7.6.  No matter what version I used in my dependencies section, my tests were executed under groovy 1.7.4.  I finally came up with the configuration below which let me run with a different groovy.  Note that it made no difference what I included in the dependencies section.  The gmaven-plugin configuration appears to be completely independent of that.

<plugin>
    <groupId>org.codehaus.gmaven</groupId>
    <artifactId>gmaven-plugin</artifactId>
    <version>1.3</version>
    <configuration>
        <providerSelection>1.7</providerSelection>
        <!-- This is only used if you want to run a groovy script from the command line using maven -->
        <source>${groovy.script}</source>
    </configuration>
    <executions>
        <execution>
            <goals>
                <goal>compile</goal>
                <goal>testCompile</goal>
            </goals>
        </execution>
    </executions>
    <!-- This block is required in order to make the gmaven plugins use a groovy other than 1.7.4.
     This is independent of the groovy entry in the dependencies section.  This does not affect the class path.

     What is interesting is that there must be both the gmaven.runtime entry with the explicit excludes
     and the additional dependency on whatever version we do want to use.  If you exclude the former,
     it will throw an exception. -->
    <dependencies>
        <dependency>
            <groupId>org.codehaus.gmaven.runtime</groupId>
            <artifactId>gmaven-runtime-1.7</artifactId>
            <version>1.3</version>
            <exclusions>
                 <exclusion>
                     <groupId>org.codehaus.groovy</groupId>
                     <artifactId>groovy-all</artifactId>
                 </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.codehaus.groovy</groupId>
            <artifactId>groovy-all</artifactId>
            <version>1.7.6</version>
        </dependency>
    </dependencies>
</plugin>

It can happen to you: SIOCSIFFLAGS: Unknown error 132

November 10th, 2010 No comments

Came home from work with my laptop. Brought it out of sleep mode. No wireless. Menu says “Wireless disabled.” Huh? Try “ifconfig wlan0 up” and get back this oh so helpful message:

SIOCSIFFLAGS: Unknown error 132

Tweak, tweak. Google, google. Find page with people joking about how they always forget to check if the “disable wireless” switch on their machine has been set. Naaahhh…#$%^$%&%^&^%!!! It must have gotten toggled while in my bag.  I now have a new use for duct tape holding that thing in place.

Categories: Uncategorized Tags:

Mr. Haki tears it up

June 15th, 2010 No comments

I always love the Groovy Goodness blog. So many gems in such a compact space. He just posted about a dozen tips in the past day alone, all on stuff that I could use in one project or another. The only thing I don’t like about it is it makes me want to go in and update all the places in my code where I could use the new calls right now. Major time suck… must resist… ohhhh… may just one little tweak…

Categories: Uncategorized Tags:

Grails, Maven, packaging, and skipTests fun

April 22nd, 2010 No comments

The grails maven plugin honors the maven.test.skip option (which turns off compiling and executing tests), but not the skipTests option (which only turns off executing tests). This means that you cannot easily create a package with test classes in it without running all of the tests. I finally come up with a close work around:

mvn clean grails:exec install -Dcommand=test-app -Dargs="-unit DoesNotExist" -Dmaven.test.skip=true

By explicitly listing the test-app command but specifying a test that does not exist, I was able to get the unit tests to be compiled. Since I only needed some common testing files in my package, this works for me. If someone wanted to package both the unit and integration tests, I don’t think this would work. I filed a new issue in the Grails Jira to try to get a real resolution.

Grails+Maven+GeoTools+PdfBox = PITA

March 20th, 2010 No comments

Here’s a bit of maven-voodoo for you. If you have a Grails (1.1.1) app that is built with Maven (2.1) and you try to link in GeoTools (specifically the gt-xsd-core module) and you try to bring in PdfBox as well, you will get really fascinating class loading and linking problems revolving around various XML pieces. I finally found two different ways to get rid of the errors. For those who like to cut to the chase, I’ll list the solutions first and then explain a bit more what errors I was seeing.

Solution #1: Exclude, exclude, exclude

If you don’t need commons-jxpath, you can add these three exclusions to the gt-xsd-core dependency declaration.

    <dependency>
        <groupId>org.geotools.xsd</groupId>
        <artifactId>gt-xsd-core</artifactId>
        <version>${gt.version}</version>
        <exclusions>
            <exclusion>
                <groupId>commons-jxpath</groupId>
                <artifactId>commons-jxpath</artifactId>
            </exclusion>
            <exclusion>
                <groupId>xml-apis</groupId>
                <artifactId>xml-apis</artifactId>
            </exclusion>
            <exclusion>
                <groupId>xml-apis</groupId>
                <artifactId>xml-apis-xerces</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

This at least works for me given the subset of geotools functionality I am using (primarily encoding and parsing OGC Filter objects to and from XML so I can persist them with Hibernate). It is still pulling in xercesImpl and that appears to be enough, at least to get my unit and integration tests working.

Solution #2: Using Dependency Management

If you need the commons-jxpath functionality in gt-xsd-core, add this to your dependencyManagement section of your POM:

    <dependencyManagement>
        <dependencies>
            <!-- The gt-xsd-core tries to bring commons-jxpath 1.2 in.  That messes up the maven junitreport plugin for
            some reason (like links to a version of xalan or xml-apis or something.  The 1.3 version doesn't have that
            issue. -->
            <dependency>
                <groupId>commons-jxpath</groupId>
                <artifactId>commons-jxpath</artifactId>
                <version>1.3</version>
            </dependency>
        </dependencies>
    </dependencyManagement>

and then keep the exclusions for the xml-api artifacts as above:

    <dependency>
        <groupId>org.geotools.xsd</groupId>
        <artifactId>gt-xsd-core</artifactId>
        <version>${gt.version}</version>
        <exclusions>
            <exclusion>
                <groupId>xml-apis</groupId>
                <artifactId>xml-apis</artifactId>
            </exclusion>
            <exclusion>
                <groupId>xml-apis</groupId>
                <artifactId>xml-apis-xerces</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

This is what I have setup in my POM right now because I think I might need jxpath soon. I also tried adding a node in dependency management to use the 1.3.03 version of xml-apis rather than 1.0b2 (which is what was pulled in by default), but that doesn’t appear to make any difference. I still had to have the 2 exclusions under gt-xsd-core for things to work.

So what was the problem?

Now, a little more about the problems I had. I had two types of errors:

Maven junitreport conflict – When I added the dependency for gt-xsd-core and then ran my unit tests using the maven grails plugin (mvn grails:exec -Dcommand=test-app -Dargs="-unit"), my tests would run fine (except for one which I’ll detail below), but then I would get this error when the junitreport plugin tried to run its XSL transform:

Embedded error: java.lang.reflect.InvocationTargetException
Provider org.apache.xalan.processor.TransformerFactoryImpl not found

Normally I would suspect that the geotools dependencies brought in some new version of the xalan library and it didn’t have an old class that the junitreport plugin required. To check this, I compared the output from both maven dependency:classpath and dependency:tree between my trunk branch and the branch where I had added the geotools dependencies. The comparision showed that only one one dependency had changed between the branches; a newer version of commons-pool was used. All other classpath mods were new libraries that were not previously included at all. In fact, xerces and xalan was not included by the trunk branch at all. I am sure that someone with more recent experiences with xml-apis, xerces, and xalan knows exactly what was happening. My guess is that by including the xml-apis-xerces dependency, some piece of the junitreport plugin thought that it could use xalan as opposed to some other XSL lib that it would use when xerces was not present. This is supported by the fact that if I added xalan as a dependency, the error went away. If I have time, I’ll look at the docs for junitreport one of these days and see if that sheds some light on the issue.

PDFImageWriter/NodeList conflict – One of the unit tests exercised our use of PDFBox. If I did not have the above exclusions and I include xalan, this error appeared during the PDFImageWriter constructor:

java.lang.LinkageError: loader constraint violation: loader (instance
of <bootloader>) previously initiated loading for a different type with
name "org/w3c/dom/NodeList"

I haven’t tracked down exactly why this linkage conflict occurred. The bizare thing is that I inserted some link-loading debugging that I used in the past to try to see where NodeList was coming from in both my trunk and geotools-enabled branches:

def clazz = org.w3c.dom.NodeList.class
def codeSource = clazz.getProtectionDomain().getCodeSource();
println ("Source Location: " + codeSource);

In trunk (i.e. no geotools and working fine), it came back as null; NodeList wasn’t loaded at all. In the geotools-enabled branch, it would load (from xerces, I think), but then had that conflict.

At this point, I have a solution (two, actually) and have spend pretty much a whole day researching various paths, so I need to wrap it up and move on. If anyone has a better solution or thoughts as to why exactly there were problems, feel free to comment.

One last note in case there is any confusion from me mentioning xalan so much – I did not end up adding a dependency for xalan to my POM. Just using the exclusion statements listed above solved the issue.

JSON conversion and request thread reuse

December 18th, 2009 No comments

Wrapping up (I hope) my experience with custom JSON conversions in Grails… The day before we were supposed to hit a major release milestone last week, I finally caught sight of a random bug that had been driving us nuts. For some AJAX requests, seemingly at random, the object that has the transient properties was not being rendered with them. Instead, it had only the regular, persistent properties and full representations of its associated objects rather than just the normal stubs of related objects with just the type and id. In other words, it was using deep JSON rendering, and this was overriding the custom JSON ObjectMarshaller that we had set for the class. But why was it doing it seemingly at random? This is where luck and experience combined to make the solution clear rather quickly and avoided some painful slogging through request handling on the server.

The luck part was that I remembered seeing a checkin from a teammate a couple of weeks ago where he got rid of direct use of a grails.converters.deep.JSON object (which is deprecated) and instead switched to a normal grails.converters.JSON object with a use('deep') statement:

def get = {
def o = OurClass.get(params.id);
JSON.use("deep")
render(contentType: "application/json", text: o as JSON)
}

This change was in a different controller for a different domain object though. Why was it affecting our other domain class? That’s where the experience part came in. A few years ago I had to track down why hibernate transactions would fail seemingly at random in another web app when running under JBoss. I could see why an initial transaction would fail (we’d get a timeout and not rollback properly), but there was no reason why later requests would cause failures. That’s when I discovered the downside of request thread pooling. Under most web contains, HTTP requests are handled by a pool of reusable threads. That means changes to thread local variables stick around between requests unless explicitly reset. In the case of the Hibernate bug, since the session was not getting closed properly, it wasn’t getting fully reset and cleared out, so subsequent requests that used that thread were getting the old session and failing because there was no way to get it back to a good state.

For the current JSON conversion bug, the JSON.use method was calling ConvertersConfigurationHolder.setTheadLocalConverterConfiguration(...), thus making the deep converter stick around for that thread. So when future requests came in to any controller on that thread, all JSON conversion calls were made with the deep configuration.

The solution is straightforward. Use the JSON.use(String, Closure) call instead when you want to make a deep call:

def get = {
def o = OurClass.get(params.id);
JSON.use("deep") {
render(contentType: "application/json", text: o as JSON)
}
}

This sets the configuration only for the closure, saving you from thread-based side effects later on.

Grails JSON converter and transient properties

November 23rd, 2009 2 comments

I figured out why my domain object was rendering as JSON differently in unit tests vs. when running the application and integration tests. Drum roll, please…No grailsApplication object is setup in unit tests.

I hear a great big, “Huh?”

First, a little background on converters. (You may want to get the Grails source code and have directories open to the org.codehaus.groovy.grails.web.converters and grails.converter packages before diving into this.) When a JSON converter is created, it obtains a ConverterConfiguration from the global ConvertersConfigurationHolder. ConverterConfiguration has several properties which influence the conversion process, but the most interesting for this discussion is its prioritized set of ObjectMarshallers. ObjectMarshallers are what do the actual work of turning an object into a JSON representation. Each ObjectMarshaller handles a certain set of classes, like arrays, maps, beans, grails domain objects, etc. When the JSON object is ready to convert a data object, it calls config.getMarshaller(data). The config iterates through its list of marshallers, calling marshaller.supports(data) on each. Since multiple marshallers may be able to handle a specific class (like GroovyBeanMarshaller and DomainClassMarshaller can both handle domain classes), whichever marshaller has highest priority (i.e. is called first) will be invoked.

When the standard ConverterConfiguration is initialized, the DomainClassMarshaller is given higher priority than the GroovyBeanMarshaller. Thus, it handles the conversion of domain objects during normal Grails application execution even though domain objects are also groovy beans. However, the DomainClassMarshaller.supports() method has a slight twist to it. It depends on there being a GrailsApplication object registered in the ConverterUtil class in order to tell it that a particular object is a domain object. Without the GrailsApplication object, ConverterUtil.isDomainClass() always returns false, causing DomainClassMarshaller.supports() to also return false. The ConverterConfiguration next checks the GroovyBeanMarshaller. Its supports() method returns true, so it handles the conversion in this case.

So why is this a big deal? For most domain object, it probably isn’t. Both marshallers render the primary properties of the domain object. However, the DomainClassMarshaller only renders properties return by domainClass.getPersistentProperties() while GroovyBeanMarshaller renders all properties, including transient ones. Because I needed some transient properties in the JSON representation, my unit tests worked great (since there is no grailsApplication setup, so the GroovyBeanMarshaller did the full rendering), but then my code failed when I ran the integration tests and the real app (since the DomainClassMarshaller handled the rendering).

How do you fix this? It’s pretty easy once you know where to look. First, I wanted to make my unit tests behave like the real app, so I had to get them to fail. Unfortunately, the conversion classes are written in java, not groovy, so you can’t just do a quick override of the ConverterUtil metaclass to get it to return what you want. You actually need to setup a GrailsApplication object. Rather than trying to create and initialize a whole one though, I determined that I could create a small stub class that overrode the two methods I needed:


import org.codehaus.groovy.grails.commons.*
class GrailsApplicationStub extends DefaultGrailsApplication {
def artefacts = [(DomainClassArtefactHandler.TYPE):[:]]

boolean isArtefactOfType(String artefactType, String className) {
return getArtefact(artefactType, className) != null
}

GrailsClass getArtefact(String artefactType, String name) {
def retVal = artefacts[artefactType] ? artefacts[artefactType][name] : null
return retVal
}
}

When initializing the unit test, I created the stub and set in the particular domain class that I wanted to be recognized. Then register the application stub with the ConverterUtil class:


def grailsApp = new GrailsApplicationStub();
grailsApp.artefacts[DomainClassArtefactHandler.TYPE][YourClass.name] = new DefaultGrailsDomainClass(YourClass);
ConverterUtil.setGrailsApplication(grailsApp);

Instances of YourClass will now be handled by the DomainClassMarshaller in the unit test. When the unit test checks for the existence of a transient property in a JSON representation, it should fail.

Next, I needed to change the marshaller prioritization so that the desired domain class was handled by a regular GroovyBeanMarshaller. Since I always wanted the GroovyBeanMarshaller to handle the class, I inserted this code into my unit test setup (and later into BootStrap so the behavior would apply to the full app):


JSON.registerObjectMarshaller(YourClass, {o, c ->
new GroovyBeanMarshaller().marshalObject(o, c)
})

This creates a ClosureObjectMarshaller that handles YourClass objects and gives the new marshaller the default priority of 1 (which puts it at the top of the priority list). When processing an object, this closure marshaller simply passes it through to a GroovyBeanMarshaller. Since the new marshaller only handles YourClass objects, the rendering behavior for all other domain objects is not affected.

This resolved my original issue. The JSON rendering of my object is now correct when I run my app. I did go down a side path while coming to this final solution. If you only want to override how a YourClass object is rendered some of the time, you could instead register a named configuration for it like this:


JSON.createNamedConfig("FullYourClassRender") {cfg ->
cfg.registerObjectMarshaller(new ClosureOjectMarshaller(YourClass, {o, c ->
new GroovyBeanMarshaller().marshalObject(o, c)
}))
}

Then when you want to use that particular configuration, you wrap the JSON object calls in a use statement:


JSON.use("FullYourClassRender") {
render(contentType: "application/json", text: myObject as JSON)
}

Other configuration options are also available. Take a look at the JSON class to see them.

I hope this helps document the conversion process a bit.

Categories: Uncategorized Tags: , ,

Fun with Grails JSON conversion

November 20th, 2009 No comments

I discovered a problem this afternoon with how one of the domain objects in my Grails app was being converted to JSON in its controller.  The conversion functioned as expected in a unit test, but failed in integration tests and when running the real app.  “Time to see how good the debugger in Intellij 9 is,” I say to myself.  So I dive into the grails.converters.JSON class to try to find where the difference comes from.  Unfortunately, every time I step into the render() method, the rendering quickly fails with this exception:

org.codehaus.groovy.grails.web.json.JSONException: “Misplaced object”

It turns out that the JSON converter has a writer property that is a JSONWriter object.  The act of showing the writer in the variables window of the Intellij debugger causes its toString() method to be called by the IDE (though any breakpoints set in the class do not get triggered by the call).  This, in turn, causes the writer to actually do the writing of the object which, when complete, changes the writer’s state property to DONE.  When the real code execution tries to process the data, the JSONWriter throws the exception because it thinks it has already finished rendering.

The trick to step through the JSON converter is to define a custom Type Renderer for the JSONWriter class that does not invoke the “toString” method.  A blog post on the Intellij site defines how to setup a Type Renderer for a class.  I set mine up so that the node renders simply as “JSONWriter” since I don’t need to see the details for it.  One caveat though – I found that the new Type Renderer didn’t affect entries already in my variables window until I right-clicked one and selected “View As -> JSONWriter”.  Then the view changed to use the new display settings.

Now back to debugging…maybe I’ll actually figure out the real problem sometime before the sun comes up.

Categories: Uncategorized Tags: , ,

When dynamic programming backfires

November 20th, 2009 2 comments

I love Groovy’s meta-programming capabilities.  Duck typing, DSLs, ExpandoMetaClass… it’s all great until you blow your arm off with them and spend two hours picking the pieces back up.  Such was my experience today as I was trying to set up my first unit test for a Grails controller.  I needed to pass in some parameters.  As I just wrote in my last post, the grails user guide has information on integration testing controllers, but nothing more than a brief mention of the ControllerUnitTestCase.  Unfortunately, searching for phrases like “grails unit test controller” on Google turn up posts from last Fall, before the ControllerUnitTestCase was released with Grails 1.1.  Don’t ask me why I didn’t just search for ControllerUnitTestCase from the start.  20/20 hindsight.

Working with what I could find, I came across a blog post from March 2008 that covered integration testing with controllers, much like the grails User Guide.  It at least had some explicit examples of how to pass in parameters, so I hoped the approach would work with my ControllerUnitTestCase implementation.  Using the code snippets in the post as a guide, I setup my test something like this:

controller.request.params = [layoutType:'filmstrip',compKey:'testKey']
controller.create()

This looks good, right?  The test scaffolding connects the injected params property of the controller with the params property of the request.  Should work like a charm.

And so started my descent into WTF Land.

The tests compiled and ran fine, except that no params were getting passed through to the controller.  I assumed that the old documentation for integration testing no longer exactly applied for the new unit test setups.  Eventually I dug through enough of ControllerUnitTestCase, MvcUnitTestCase, and MockUtils that I figured out there was a way to get direct access to the params property of the controller.  “Makes sense,” I thought, “sort of a mini-mock.  Who needs the request for a unit test?”  Great, so let’s try this:

controller.params = [layoutType:'filmstrip',compKey:'testKey']
controller.create()

Run again and…

groovy.lang.ReadOnlyPropertyException: Cannot set readonly property: params for class: YourController

Erk.  Closer inspection of grails.test.MockUtils.addCommonWebProperties() reveals how the params property is added to the controller class with only a get method.  Okay, so we can still manipulate the map once we get it.  One more try:

controller.params.putAll([layoutType:"filmstrip",compKey:'testKey'])
controller.create()

Success!  Finally, information is going where it needs to be.  I later realized that section 9.2 of the user guide does actually show an example similar to this, except that it sets each item on params individually:

def controller = new AuthenticationController()
controller.params.login = "marcpalmer"
controller.params.password = "secret"
controller.params.passwordConfirm = "secret"
controller.signup()

“Oh well,” I say to myself, “at least I got it working.  I’d better post a comment on that old blog post so that others know the new way to set parameters.”  I returned to the blog page, moved down to the comments section… and saw that someone else had posted a comment just last month thanking the author for the good information.  Wait a second… you normally don’t thank someone if their examples don’t work.  So I scrolled up and took a closer look at the examples again:

controller.request.parameters = [sender:'me',msg:'test']
controller.create()

Anyone with a better eye for detail than me see the difference that sent me down the rat hole? It’s request.parameters, not request.params as I had entered.  For the 100th time today, Doh!

But wait a second (or an hour, as the case may be)… If I was calling request.params and there is no params property, how did my test run at all?  Dig, dig, dig… It turns out that the request is a GrailsMockHttpServletRequest object.  Scanning through that class, I see the ever so powerful and ever so dangerous setProperty(String name, value) override.  If you try to set a property that doesn’t exist, it adds the value as an entry in the request’s attributes map.  So the test scaffolding was perfectly happy with my request.params call.  It nicely tucked my params map away in the attributes and continued on its way.

Sigh…it almost makes me miss compile-time method checking… but not quite.