Continuous Integration

Over the last few weeks I have found myself reflecting on the progress my team and I have made in migrating learndirect over the last few years. It is strange to think that I will have spent almost 10% of my working life nurturing the next decade of online learning for the learndirect community. We still have quite a few more releases to go, but I find myself surprisingly relaxed about every release we provide to our QA department.

In previous companies I have seen the day of release of a new version of software treated with trepidation by the testing team and with relief by development. Some developers might consider they had earned a brief respite having worked round the clock to hit the build target, others would be relieved to be able to finally share their hard work. Few would be prepared to second guess what the result might be in testing. In my experience the last month of development is often as much about sleep deprivation as anything else.

For those that feel this way I can strongly recommend a good dose of continuous integration. It may not quell all of your headaches, but it has served my team and I well for years.

CruiseControl

Adding automated unit tests to the continuous build is a great second step, but if you can work on a more thorough approach to script the compilation, packaging, deployment, installation, data seeding and integration of your system(s) you wont regret it.

Powered by ScribeFire.

Experimenting with NAS drives

http://comedyfriday.gotdns.com/photo/

Scribefire

Isn't technology wonderful. Download a few plugins and all of a sudden your browser is an application suite.

Powered by ScribeFire.

Post Conference Update

JBoss have now posted up all the presentations delivered during their sessions. A lot is changing around both Red Hat and JBoss so there's some good reading in there.

http://www.jboss.com/events/presentations

Freebies, photos and fly home

The event has proven incredibly useful. It has answered a lot of the questions I had around JBossCache and JBossRules, as well as questions I had not expected to be able to ask like JBossSX and SSO. The conference has probably raised more questions than it has answered, but then that is the nature of events like this.

It has been fascinating meeting so many of the people that work for JBoss and Red Hat, many of whom I had only previously read about. I dropped off the Ufi CD with the JBoss marketing team. Who knows if anything will come of that, but I thought it would be interesting for them at the very least.

Meeting the other delegates was also invaluable, hearing what they do, for whom and why. One or two may be in touch, let's hope so.

The conference itself was run incredibly well, and with an amazing focus on the fine detail. It was a pleasure to have been in Berlin, and the free JBoss top and bottle of German wine wasn't bad either ;)

Clustering and Storage / High Availability - Matthias Kranz and Jan Wildeboar

With my last session of the conference I decided to throw myself into the deep end completely and attended a hard core Red Hat session. I can see I really am going to have to work on my Red Hat skills quite a lot before doing this again, but anyway here are some points that I have picked up either from this session and from various conversations throughout the conference.

Red Hat Enterprise 5, the next release, is likely to include some substantial improvements.

iSCSI : low cost enterprise SAN connectivity
NFSv4
LVM2 : to include cluster wide snapshots

Application failover is possible when the correct start, stop and status scripts are available for the software. Some other features such as Membership, I/O Fencing, Lock Management and Heartbeats were discussed. I can imagine what some of these are when relating them to the JBoss clustering approach, but I was unable to follow the specifics. Two things which I did pick up on were STONITH (Shoot The Other Node In The Head?) and Lock Management.

STONITH is basically support for management cards that allow machines to be rebooted remotely. This is a well developed aspect of Red Hat which a member of the audience asked about.

Lock Management in clustered environments was previously made possible by a single lock manager. This has now been re-engineered so that each node has its own lock manager (DLM - Distributed Lock Management). This feature is new with Cluster Suite v4 and has brought enormous performance improvements.

The latest version of the Cluster Suite and Global File System have been shown to scale to 300 nodes in testing, but Red Hat are looking for a customer with a larger network to prove that it can scale beyond this.

The Global File System will now support 16TB on 32 bit systems, and 8EB on 64 bit systems.

Red Hat have had their Virtualization software on display at their stand. Apparently all the DVDs they wanted to bring are lost in transit somewhere so it looks like a download will be necessary to try this out for myself. They took great please in launching and destroying a Windows Vista virtual machine on the demonstration system. Virtualization support is, or will be, available in the Fedora distribution, meaning it's free for everyone and not just a part of the Enterprise suite.

The intent with future releases of Red Hat is to be able to replicate entire memory spaces from machine to machine, making it possible to provide fault tolerance for legacy systems far more easily. I'm not sure what kind of timescales may be on this, it sounds quite ambitious.

In conclusion, I needed a coffee badly.

JBossCache Advanced - Manik Surtani

At the start a diagram was displayed showing the stack. Broadly speaking it was shown that JGroups formed the base of the stack with various components, including JBossCache and Messaging (a new addition) plugged into it.

Clustering supports a variety of features, including the various flavours of HA services (High Availability) such as HA-Singleton, HA-Remoting, etc.

The application stack was defined as follows:

Application
Building Blocks
Channel
GMS
Unicast
NAKACK
FD
UDP
Physical Transport

Custom Channel implementations can be written, they are similar in nature to MulticastSocket.

The remainder of the session was spent describing in a little more detail 4 of the main areas; JGroups, JBossCache, PojoCache and HA services.

1. JGroups

Supports : UDP (IP Multicast), TCP and TCP NIO.
Note: TCP requires a dedicated thread per connection whereas NIO has been tweaked to use a connection pool.
Reliability : automatic retransmission
Ordering : FIFO, TOTAL
Failure Detection : using a heartbeat or ping approach
Group Membership
Fragmentation : allowing messages to be broken down into smaller units for transmission
Encryption, authentication : related to group membership
Compression, message batching : the example of compressing XML was used
State transfer : session replication
Merging after network partitioning : described as recovery from network faults such as failed routers
Flow control

2. JBossCache

.... some more to add ....