Maven, ant, replacereexp, and native2ascii

Wow. What a lot of wasted time.

These are known bugs as you’ll see if you search it, but finding proper workarounds has cost a lot of time.

The first issue I got was in using replaceregexp in an ant script called from maven. I’d added all the proper jars to the dependency lists, and specified the ant.regexp.regexpimpl property to use the org.apache.tools.ant.util.regexp.JakartaOroRegexp implementation, but ant was unable to find it. If I defined a new task for replaceregexp with the proper classpath, then the definition would work fine, but it was unable to find the JakartaOroRegexp class when it executed.

To solve this, I used the Java task to call ant instead of the ant task.

<tasks>
<property name=”compile_classpath” refid=”maven.compile.classpath”/>

<!– This doesn’t set the paths correctly, so we need to use java to call ant  –>
<!– ant antfile=”build_translation.xml” inheritAll=”true” inheritRefs=”true” / –>

<java
classname=”org.apache.tools.ant.launch.Launcher”
fork=”true”
failonerror=”true”
>
<classpath path=”${compile_classpath}”/>

<!– arg value=”-debug”/ –>
<arg value=”-buildfile”/>
<arg file=”build.xml”/>
<arg value=”-Dcompile_classpath=${compile_classpath}” />
<arg value=”-Djdk.set=true”/>
</java>
</tasks>

This worked well, but my ant script used native2ascii and I kept getting the message Error starting Sun's native2ascii. No matter how hard I tried to fix this in the script by modifying class paths, JAVA_HOME definitions and otherwise, the only way I could fix this was by creating a symbolic link from JRE_HOME/lib/ext to JDK_HOME/lib/tools.jar. This means I have to do this on every build machine, and I really shouldn’t have to. Of course, the bug shouldn’t exist in the first place, so I guess that’s life.

At least now I can move on to solving the problem I was originally trying to solve instead of wasting time getting the builds to work.

Option 4

Well, it wasn’t all that clear to me after several reads of the documentation, but I worked around my Maven build issues using a fourth option. I have a pre-configured Geronimo server with all the pools created sitting idle and load it in using the tag instead of an assemblies tag. This means I have to pre-configure all my build servers and test machines, but at least it works.

I hope the Geronimo folks address this in an upcoming release. Of course, I would also hope that I could build pre-configured custom assemblies.

Maven, Geronimo, and continuous integration

I’ve been working on this problem for some time with varying degrees of success. My goal is to have continuous integration working for all the modules of the system, and Maven is well suited to this. For some of my modules, things work well. For example, for independent modules not using a database or other system facility. I’ve even got web services that use JNI building with success.

The problem child is the EJB module that uses the database. It depends on a database configuration that’s very easy to create using the console. This doesn’t work well for continuous integration, especially as modules are loaded and unloaded for testing. The standard Maven approach is to install an assembly, load the pieces you need to test, and unload when done. The standard assembly doesn’t include the database pool so that has to be set up separately. Maven doesn’t support this.

Option 1: Create a custom assembly.

For Geronimo 2.1.3, which I’ve been using, this fails. Period. It can’t be done. In order to include the modules I need, I consistently get in “IOException: HEAD full” error. 2.1.4 doesn’t give this error and builds an assembly, but it fails to start with a ClassNotFound error.

Option 2: Create and install plugins

This can be done, but Maven refuses to load them because the manifest is incomplete. I haven’t investigated this option fully, primarily because it annoys me that a plugin creator would create an unusable plugin.

Option 3: Create the pool in the deployment plan

When loading from Maven, it just says “Sorry, try again.” with no indication as to what the error is. I’m paraphrasing, but if you’ve done much work with maven you know what I’m talking about. Loading it using the deployer.jar fails because the MySQL .jar is missing. And there’s no way to load it. Well, I could load it into a custom assembly, but we’ve already covered that.

Summary:
I’m stumped. I’ve tried the recommended approaches and all the work arounds I can think of. Time for a nap.

Clusters, bug fixes, virtual machines, and systems administration

Well, it’s been a busy couple of weeks. I’m now in a new office and getting serious about getting our site in the hands of beta testers.

One aside I had to deal with was a bug in my Fedora package for ARM caused by a code reorganization in system libraries, but that was a simple fix and is now done.

The biggest accomplishment was moving my MySQL service to a clustered back end. This should give me significant scalability and reliability in the future. Since they’re all running on the same system in virtual machines means I get no such benefit now, but as machines are added this will improve. At this point it’s overkill, but I still want this configuration tested.

I’ve also set up a number of specialized servers, also in virtual machines. These host our subversion code management service, the maven artifact service, continuum build server, bugzilla, wiki, and others. How did people work before virtual machines? While these are overkill in terms of our performance needs, at least at this point, it sure makes the systems management simpler.

Of course, the downside is that a power bump is far more serious. Time to buy a UPS.

Broken dependencies

Ah the joys of being a Fedora contributor!

The first hint of trouble are the failed to build from source error messages. Apparently, the development versions just stopped building. Upon investigation, it appeared I had an incorrect dependency. Since I was using uuid libraries, I had a dependency on uuid-devel. Well, it turns out that that was for a different set of uuid functions. The ones I wanted were in e2fsprogs-devel… the package for file system maintenance ?!? Well, apparently I’m not the only one who found that odd, and they split that package into ones that made more sense. Hence the breakage.

Net result: Existing packages work, although I’ll update their dependencies to the correct ones. New packages (For Fedora 12, so you don’t have to worry about it yet) will be updated.

New issue of MeasureIT is out

This is the online publication of the Computer Measurement Group. Well worth the read.

http://www.cmg.org/measureit/

Odds and sods

So, as my infrequent posts show, I’m in a data connectivity lull. That doesn’t mean work stops, although it is challenged. Hopefully regular connectivity will resume shortly.

In the meantime, I’ve been working on infrastructure. Virtual machines are my friend. I’ve been setting up internal servers for subversion, wikis, bugzilla and so on. The most interesting one though has been for continuum.

Continuous integration is easy with one developer. For multiple developers, automation is key and continuum is the tool. It installs easily, works as advertised, and has a number of interesting features. It can automatically build whenever a change is committed, run the tests, and report the results. It all works well. One exception is that I’ve requested notifications on success, but it doesn’t seem to do this. Oh well. My one quibble with the tool is that I haven’t found a way to set it up for different development branches. That doesn’t mean one doesn’t exists, but I haven’t found it yet. It appears that the current tool requires a separate instance for each branch, which is doable but an obvious and seemingly avoidable pain. I hope it’s on their to do list. Of course, Maven has some issues with running offline, but that’s another story.

In the interim, development continues. Now that the testing infrastructure is in place, so does automated test development. Life is fun?

The frustration that is Java entity beans (Update)

So the issue was one of nested transactions. Apparently you can’t. I would expect an exception to be thrown when I tried to start the second transaction, not when the outer transaction commits though.

Now my next great frustration. I try to remove an entity, and the code following acts as if its removed, but the removal never makes it to the database. This appears to be referential integrity run amok, as another entity references this entity and disallows the removal. No amount of cascading seems to make it go away, so instead I have to find the entity that references it and remove it from there.

JDBC was so mush easier, and I never had any problems with referential integrity. Entity beans just aren’t making my life any easier.

The frustration that are entity beans

Entity beans promise a simplification of the database interface. On the surface that’s true. The methods are convenient. The logic is simple and straight forward. There are many published concerns about performance, but I’m not going to get into that here, primarily because I’m not at the performance testing stage, so I have no opinions. Yet. I’m sure I will.

The big issue I have is inconsistent and unmanageable behaviour. For example, I have a section of code that does this:

insert new item
if (Exception already exists)
rollback
throw error
do some other stuff
commit

Yes, there’s a unique field, specified both in the database definition and the entity definition. When initially written and tested, it worked. Add some code elsewhere and it doesn’t anymore.

When a duplicate entry is persisted, no error is thrown. When you commit, it throws a rollback error because of the duplicate entry. The net result is that there’s no way to differentiate between a duplicate entry and any error that may have occurred elsewhere in the code. Since that’s a special case for me, this is a pretty serious problem. I’ve found a workaround by adding an unnecessary explicit check before hand that increases my code path and duplicates work that is supposed to be done for me. In the meantime I’ve spent a lot of time and effort debugging “correct” code that worked at one point, and I have to ask myself where else is this happening?

Well, I’m getting my answer. In another section of code that “used to work”, I remove an entity, and then perform code that counts on it being removed. Now I’m getting an error saying that the entity is “detached”, and the remove operation throws an exception. If I catch and ignore the exception, the code following that counts on it being gone fails, because it’s not. Again, this isn’t a problem with program logic. This is a problem with entity bans not working as advertised.

I’m now faced with potentially tossing out about 15000 lines of code and starting again. The project is set back a few weeks at least. And this is due to “debugged” code that seems like it will never work in a consistent manner.

I’ve done a lot of DB interfacing in many languages and realize it has quirks. I’ve even done it in Java using JDBC. I’ve never had issues like this. JDBC is sure looking a lot better about now.

A busy yet unexciting couple of weeks

I haven’t posted in a while. It’s not that I haven’t been doing anything, it’s just that what I was doing was the basic stock and trade of any programmer – coding. J2EE with a bit of Google’s Web Toolkit (GWT) thrown in. And none of it pays the bills… just a hint to anyone out there with work that needs doing.

It’s not been a week without discovery though. I’ve been using Apache for quite a few years now and thought I knew it pretty well. Heck, I’ve even written Apache modules! But I’ve always been a bit of a small time user in that I’ve never worked on anything that really pushed it to the limits, except in a test environment. When I started testing my GWT application though, all that changed. Performance was sluggish, taking upwards of 30 seconds to load a page when tested remotely. This is obviously unacceptable, but I’d made no effort on the performance side, so I wasn’t really surprised. I’ve made a living developing and using tools that model the differences between a testing laboratory and the real world. And my real world consisted of multiple tiers that were actually multiple virtual machines running on my development machine and a DSL link (i.e. high speed on the requests, not on the responses) connected through DDNS. Of course it’s slow.

YSlow to the rescue. It’s a great plugin for Firefox from the folks at Yahoo that measures why pages are slow to load and more importantly offers suggestions for improvement. The first thing it suggested was I use mod_deflate to compress my files. I didn’t even know you could do this! Where have I been? What a difference this made! I’m now down to about 3 seconds a page.

Of course, it had other suggestions as well, including cache settings, ETags, and so on. I’m not so worried about them yet, as I’m still in pretty active development, but they’re good to know. Overall, GWT is pretty cache friendly, but there are some files, such as my CSS files, that are still changing pretty quickly. It did surprise me though how limited the configuration options are for Apache when it comes to caching files. I can set options according to a mime type, but not according to a file pattern. This would be useful for caching files such as *.cache.* and *.nocache.*. These are questions that will have to be addressed either by module or by process before deployment, but I’m happily ignoring them for now.

The other tool that’s been really useful is Firebug. This was required for YSlow, but the best use I made of it was in debugging my CSS. It allowed me to look at individual elements and discover what styles were being applied. This is very useful when using GWT as the styles may not always be what you expect. It definitely reduced the amount of cursing I did.

Then again, perhaps the most significant event of the last couple of weeks is that there were no significant events. My life as a programmer has taught me to expect otherwise.

There’s another week or so of heads down development, but then I tackle interesting things again, such as instrumenting Geronimo with ARM. AspectJ, here we come!