Fighting with a JVM bug

Had a really annoying issue on of the servers we host. We had glassfish v2 b58g running behind a load balancer and running on java 1.6.0_12. Everything seemed to work until i realized in our monitoring system zabbix that the connection was really unstable. In some cases the system was not accessible at all.
By using telnet i could see following behavior:

[user@jump ~]$ time wget --timeout=30 --tries=1 https://service.com/somesite
--08:30:07-- https://service.com/somesite
=> `service.3'
Resolving service.com... 199.44.48.112
Connecting to service.com|199.44.48.112|:443... connected.
Unable to establish SSL connection.

real 0m10.109s
user 0m0.009s
sys 0m0.025s
[user@jump ~]$

(some values are changed :-)

The interesting thing was that it always takes 10 seconds before the error happends.
Next thing was to use tcpdump and identify what is happening. It showed a lot of connections where closed with TCP RST. And glassfish seemed to throw and exception as it did not know either why the connections died. Anyway, some googling and i first found this ticket talking about possible error.
It lead me to a java bug 6403933 that still is not solved. in the latest JRE:s. And this problem was not in java 5 so it explains why i had not this problem before.

So, thanks to all my new knowledge i thought that upgrading glassfish to v2.1 b60g would help. And it did :-)

Monitoring JVM

I have been having some trouble with a Glassfish installation, version 2 build 58g. The HTTPS connections seems to be little slow sometimes.
Reading the logs did not get me any information so i needed to find some other tools.

First of all, using kill -3 on a process gives some information. It gives a dump of the threads and shows possible deadlocks. For glassfish look in log jmv.log.

Then we have jconsole that is pretty good. The only problem with that one is that it is almost impossible to access to a remote host because of firewalls because of some unknown ports that are used for the RMI connection. You can write your own JMX agent that only listens to one port and tunnel to it or you can run a VNC server on remote server, that might as well. What is good with Jconsole? It shows memory usage, gives possibility for JMX configuration, shows threads and a lot more.

Another tool i found today (it has been there all the time in Java 6) is jmap.
With it you can create a dump of all memory used. It shows all classes, instances, dependencies and a lot more. The amount of information is amazing and I'm still trying to learn how to find more.

To analyze the dump done by jmap you need to run jhat. It reads the dump and starts a small HTTP server where all the data can be accessed.

These three tools i mentioned can all be found in JRE/bin directory.

Another tool i found is Visualvm but i have just not figured out all the advantages with it, trying to learn.

iSendMMS

Yesterday we (Mobispine) released the first application in Sweden that makes it possible to send MMS from an iPhone. We called it for iSendMMS.
This application makes it possible to send one images and text to one recipient.
In the future we will add support for other content types as well.

I have mainly worked with the server side of this product and actually figured out the name it should have. Sometimes techies have ideas too :-)

The application works well but there is some known issues:
- Sometimes first registration fails - register one more time, it might help (if you are a Telia customer)
- Sometimes when using EDGE the MMS are not send - this is something to be solved in next release.
- Multiple recipients don't work even though you add them.
- Works only for Telia Sweden customer - will probably be expanded in the future.
- Can not receive MMS - that will come in the future. For this we need operator integration.
- The MMS is sent even though the information is lacking, wait pressing the send button! :-)

So, if you have a iPhone, download and try the app. If you have any comments, post them here or send mails. We can always make things better!

Drag and drop with GWT

Yesterday i started too look on drag-and-drop with GWT. It is pretty cool stuff and for example this example is pretty much what i want to do on the page I'm working on. I found this example by reading this blog about this functionality. Ajax really gives possibilities for the page and GWT makes it easy to implement.

Installing GWT for Netbeans

I don't know i did mention about how to install GWT for Netbeans. It is actually pretty easy and usually installing stuff in Netbeans in much easier than in Eclipse because the later usually gives these strange dependency errors that can drive you crazy.