Debugging connection pool leak in Apache HTTP Client

I recently had an issue using the Apache HTTP Client pooling library where after a while threads would just block when trying to open connections. It didn’t take too much to figure out that the thread pool was being exhausted, a quick thread dump revealed all the threads were waiting for a connection from the pool. The issue was, which method was not releasing the connection? In the Apache HC library a connection is released to the pool when the response.getEntity().getContent() is closed (provided it isn’t already null). This is what EntityUtils.consume does - it checks if getContent() is null and if not then it calls .close(). ...

June 17, 2011 · Nigel Sim

Spring @Autowired - Use interfaces!

Here’s a little lesson that I had to relearn today: When using Spring use interfaces. The premise was I had a DAO bean that was configured with Spring, and it was @Autowired into my controller (or test case in this instance). Because I only intended to have a single implementation of this class, and because this was the first iteration of the project, I made the DAO bean concrete. However, when I tried to inject it into the test case got an exception: ...

May 31, 2011 · Nigel Sim

Preauth in Spring Security 3.x

Sometimes in a webapp you will be in a situation where a filter/app/container other than Spring will be responsible for authenticating a user and setting the user principal, leaving the authz to the Spring webapp. A portlet container is a typical example. There is a few examples floating around showing how to do this in Spring 2.x, but it appears some thing (packages, etc) have changed for Spring 3.x, so here is how to make it work. Use the following in your applicationContext.xml ...

January 20, 2011 · Nigel Sim

Projects for the new year

These are some project idea’s I’ve been sitting on all year and have not yet started, but hope to in the new year. 1. Python CMS framework in the vein of Drupal, based on Repoze BFG So Drupal is quite a popular community building framework, with good plugin system, and a data model which seems to work well. But I don’t think PHP is the way of the future, at least for me. I use Python for many other projects, I understand how to debug, unit test, etc. And BFG brings the really nice framework features from Zope and Plone, which makes it an ideal starting point. I’ve got a project plan for this which I’ll reveal when I get around to cutting some code. ...

December 24, 2008 · Nigel Sim

ZSI -> CXF: Parameters coming in as NULL

Recounting a strange little compatibility issue I had between ZSI 2.0 and CXF 2.0.x. I was using CXF as the server, running from Maven using Jetty, and ZSI as the client. The parameters from the ZSI were arriving at the service implementation as null. With the web services logging turned on I could see that the SOAP packet was arriving OK, and looked good. A little bit of digging round the web turned up this message. The issue was that ZSI wasn’t explicitly namespacing the elements, and so CXF was not seeing them. The solution was to add elementFormDefault=“qualified” to my WSDL definition, and rebuild the ZSI stubs. ...

November 17, 2008 · Nigel Sim

Trac taking a hammering

At work we have one VM which hosts all our project management software like Git, SVN, Trac and Bugzilla. However, recently it has been taking a hammering and essentially crashing. The issue was it was running out of RAM, and swapping like crazy. A little investigation into the situation uncovered that there were some Trac 0.10 CGI processes which were using >500MB of memory. Using the ps ewww -p <pid> command to look at the environment variables of the processes we determined that each of them was serving the same kind of request, SVN changesets. And essentially all of them were being hit by spiders. ...

October 15, 2008 · Nigel Sim

Zope3 Component Architecture (CA) style Adapters for Java

After programming for Zope3/Plone for the past year I’ve come to really admire the flexibility and elegance that their implementation of the adapter pattern gives us. And, after Martin Aspeli put the call out almost a year ago, and it has not yet been answered, I thought it was time to give it a go. How hard could it be. So, what I’ve developed is a very simple, no dependency, maven available library which should greatly improve the flexibility of your code. ...

October 15, 2008 · Nigel Sim

MyProxy server segfaulting

Setting up a new MyProxy (v3.9 12 Jun 2007 PAM OCSP) from VDT 1.8.1 I ran across an annoying segfault issue when running the server to run as a CA, in debug mode. # myproxy-server -dmax_proxy_lifetime: 43200 seconds PAM enabled, policy requiredCA enabledmax certificate lifetime: 43200 seconds using storage directory /var/myproxy Starting myproxy-server on localhost:7512... Connection from 127.0.0.1 using trusted certificates directory /opt/vdt/globus/TRUSTED_CA Authenticated client <anonymous> applying trusted_retrievers policy trusted retrievers policy matched applying authorized_retrievers policy applying authorized_renewers policy Program received signal SIGSEGV, Segmentation fault. Note, the client can connect and only fails once the client responds with their credentials. Using GDB I got the stack trace of the issue: ...

April 22, 2008 · Nigel Sim

Some thoughts on a dynamic (lazy) data access layer for web services

The problem I’ve been address in my most recent work block has been to develop an interface to a relational data store which can be used either as a local DB, or via webservices. The concept is quite straight forward, we want CRUD operations on a set of data objects. The schema is fairly straight forward too, with a core hierarchy of elements with a few enumerated lists. As well as a few cuts across the hierarchy. ...

April 3, 2008 · Nigel Sim

Spring and JNDI (Tomcat or Jetty)

Recently I had need to deploy some Spring webapps which required predeploy configuration. Being the first time I had to find a serious answer I looked to the mythical JNDI for an answer. This document is meant to complement other Spring JNDI documents out there. Essentially the problem is this. We need to deploy a webapp. The webapp needs configurations (database and webservice endpoint locations). Editing properties files or XML config within the webapp isn’t nice, because on a redeploy the config will be lost. Inside containers like Tomcat I am not aware of a way to easily add extra items to the classpath which won’t get nuked unexpectedly, so solutions like PropertyPlaceholderConfigurer don’t really fly as the properties file will end up within the webapp. And I don’t like the idea of setting environment variables for to locate such things. ...

February 14, 2008 · Nigel Sim