Friday, November 24, 2006

Freebies, photos and fly home

The event has proven incredibly useful. It has answered a lot of the questions I had around JBossCache and JBossRules, as well as questions I had not expected to be able to ask like JBossSX and SSO. The conference has probably raised more questions than it has answered, but then that is the nature of events like this.

It has been fascinating meeting so many of the people that work for JBoss and Red Hat, many of whom I had only previously read about. I dropped off the Ufi CD with the JBoss marketing team. Who knows if anything will come of that, but I thought it would be interesting for them at the very least.

Meeting the other delegates was also invaluable, hearing what they do, for whom and why. One or two may be in touch, let's hope so.

The conference itself was run incredibly well, and with an amazing focus on the fine detail. It was a pleasure to have been in Berlin, and the free JBoss top and bottle of German wine wasn't bad either ;)

Clustering and Storage / High Availability - Matthias Kranz and Jan Wildeboar

With my last session of the conference I decided to throw myself into the deep end completely and attended a hard core Red Hat session. I can see I really am going to have to work on my Red Hat skills quite a lot before doing this again, but anyway here are some points that I have picked up either from this session and from various conversations throughout the conference.

Red Hat Enterprise 5, the next release, is likely to include some substantial improvements.

iSCSI : low cost enterprise SAN connectivity
NFSv4
LVM2 : to include cluster wide snapshots

Application failover is possible when the correct start, stop and status scripts are available for the software. Some other features such as Membership, I/O Fencing, Lock Management and Heartbeats were discussed. I can imagine what some of these are when relating them to the JBoss clustering approach, but I was unable to follow the specifics. Two things which I did pick up on were STONITH (Shoot The Other Node In The Head?) and Lock Management.

STONITH is basically support for management cards that allow machines to be rebooted remotely. This is a well developed aspect of Red Hat which a member of the audience asked about.

Lock Management in clustered environments was previously made possible by a single lock manager. This has now been re-engineered so that each node has its own lock manager (DLM - Distributed Lock Management). This feature is new with Cluster Suite v4 and has brought enormous performance improvements.

The latest version of the Cluster Suite and Global File System have been shown to scale to 300 nodes in testing, but Red Hat are looking for a customer with a larger network to prove that it can scale beyond this.

The Global File System will now support 16TB on 32 bit systems, and 8EB on 64 bit systems.

Red Hat have had their Virtualization software on display at their stand. Apparently all the DVDs they wanted to bring are lost in transit somewhere so it looks like a download will be necessary to try this out for myself. They took great please in launching and destroying a Windows Vista virtual machine on the demonstration system. Virtualization support is, or will be, available in the Fedora distribution, meaning it's free for everyone and not just a part of the Enterprise suite.

The intent with future releases of Red Hat is to be able to replicate entire memory spaces from machine to machine, making it possible to provide fault tolerance for legacy systems far more easily. I'm not sure what kind of timescales may be on this, it sounds quite ambitious.

In conclusion, I needed a coffee badly.

JBossCache Advanced - Manik Surtani

At the start a diagram was displayed showing the stack. Broadly speaking it was shown that JGroups formed the base of the stack with various components, including JBossCache and Messaging (a new addition) plugged into it.

Clustering supports a variety of features, including the various flavours of HA services (High Availability) such as HA-Singleton, HA-Remoting, etc.

The application stack was defined as follows:

Application
Building Blocks
Channel
GMS
Unicast
NAKACK
FD
UDP
Physical Transport

Custom Channel implementations can be written, they are similar in nature to MulticastSocket.

The remainder of the session was spent describing in a little more detail 4 of the main areas; JGroups, JBossCache, PojoCache and HA services.

1. JGroups

Supports : UDP (IP Multicast), TCP and TCP NIO.
Note: TCP requires a dedicated thread per connection whereas NIO has been tweaked to use a connection pool.
Reliability : automatic retransmission
Ordering : FIFO, TOTAL
Failure Detection : using a heartbeat or ping approach
Group Membership
Fragmentation : allowing messages to be broken down into smaller units for transmission
Encryption, authentication : related to group membership
Compression, message batching : the example of compressing XML was used
State transfer : session replication
Merging after network partitioning : described as recovery from network faults such as failed routers
Flow control

2. JBossCache

.... some more to add ....

Wednesday, November 22, 2006

JBossWorld Party

It was a long day, lots of sessions, the Customer Panel, and then at the end of the day finding only light snacks rather than the full blown meal we had received the day before.

I got talking to a delegate from Camden Council whilst looking rather despondently at the nibbles, but was saved from starvation by the JBossWorld Party. If you want to know more I'll happily oblige, it was an amazing show; food, drinks, cocktails, dance routines, birthday cake ... all the usual things :)

Web Services Advanced - Thomas Diesler

It was perhaps a little unwise of me to throw myself into this session given how rusty my webservices are. Note to self: read more. This was compounded by the fact that JBoss have, as with so many other functional areas, apparently given this a serious overhaul. In addition it was mentioned that regular "plugfests" were being held with Microsoft to ensure interoperability.

Anyway, battling on, an impressive list of areas were walked through, some of which I will cover in more detail than others. I will revisit this and several other posts later on to provide links to online sources.

1. WS-Security

# Following the OASIS standards: SOAP, Token, REL and SAML (Important for SSO), W3CXML Encryption Signature

This is configured within within ... I need to locate the relevant config file ...

In a little more detail this area includes:

# Strong encryption
# Certificate authentication, signed content and public key infrastructure.
# JAAS integration
# Fine grained operation configuration
# Fine grained message element configuration
# Message expiration
# Interoperability with Microsoft WSE and Sun JWSDP

The key thing here was that this relates to securing the XML payload on the message and not on the transport. The scenario discussed described messages being recieved and stored by a relay for future transmission onto the destination. In this case the data remains secure whilst on the relay, and the transport could be open (HTTP) without breaching security of the data at any stage.

One warning that was given was to always consider the necessity for encryption, and whether it should be supplied within the message or on the transport given that the performance costs are quite different in each case. This overhead is one reason why WS-SEcurity supports fine grained control of message elements, meaning that messages need only be partially encrypted.

2. WS-Addressing

This refers to moving message addressing information out of the headers (as with cookies) and into the message body, making the message aware of its own routing needs. This allows applications to much more easily control message routing through standardised xml tags.

One advantage of this feature that was briefly described was the ability to dynamically assign "reply" and "fault" addresses for messages.

3. Message Transfer Optimization Messaging (MOTM) and XML Optimized Packaging (XOP)

Refering to the ability to encode binary data in base64 format automatically for transmission. Data formats currently supported include JPEG, text/XML, app/XML.

We were told that XOP is configured as a default within JBoss and that put simply, it handles base64 encoding so you don't have to.

4. JAXWS

Towards the end of the session some mention was made for Providers to be implemented as Message or Source. It appears that Source implementations are those that are able to work directly on the message payload, whereas those that are Message get access to the entire XML structure.

JBoss5 is said to be about 80% compliant with this standard at present, whereas JBoss4 is around 20% compliant

5. And Finally ...

Future consideration will be given to back porting the latest webservices technologies into JBoss4, but JBoss are not sure this will be technically possible. Other things being considered for the future include WS-ReliableMessaging and Fast Infoset.

Samples and documentation are high on the agenda. I look forward to these given how rusty my webservices knowledge is.

The key points for me were around the fine grained control of encryption within the message which is transport independent, the ability to work directly on the message payload, and the ability to auto-magically encode binary data.

JBoss Federated SSO Framework - Anil Saldhana and Sohil Shah

Much as with the previous session this proved immensely popular, but additional seating was provided so I managed to avoid sprawling on the floor at the back this time. Identity Federation is currently the domain of commercial vendors so it is hardly surprising that so much interest was shown around this subject, what was surprising was the low key manner with which it has been approached within the conference.

The benefits of SSO (Single Sign On) were described as follows:

Consolidation of identity stores
Improvements in user account provisioning
An improved user experience
Improved efficieny in the integration of new applications
Enabling secure inter-company application access

The architecture consists of 3 elements:

Token Marshalling: Kerberos, SAML, other pluggable formats. This is a configuration feature which is then managed automatically for the developer.
The Identity Management Framework: OpenLDAP by default, but this is pluggable and can support custom operations such as JDBC.
The Federation Server: which manages secure cross-domain identity propagation.

A demonstration was provided where two JBoss web applications running in their own independent JVM's where shown to support identity federation, logging in on one application and being auromatically logged in on the other, logging out on one and being automatically logged out on the other.

The developers described 4 key steps within the demonstration code:

1. Implementation of a LoginProvider to perform authentication.
2. Registration of the LoginProvider within the "sso.cfg.xml" file inside "jboss-sso.sar/conf".
3. Configuring the SSO Trust Server for your domain within this same config file.
4. Configuring the web app to activate federated SSO.

This last point could mean using JAAS, or some non-JAAS solution such as a Struts Action class. In the non-JAAS case it was said that 3 Tomcat valves would need to be configured. I imagine a trip through the demonstration code supplied on the web site would be best given that much of what was talked about related to the configuration of one aspect of the application server or another.

Areas which are being looked at for the future include integration with more specifications such as WS-Security, WS-Trust, WS-Federation, WS-Policy; support for Certificate based and SRP protocols; and cross system password synchronization.

JBoss Security - Anil Saldhana

Security is an area of JBoss that is being overhauled / re-written to an enormous extent. The sessions I attended were not announced as being part of the main product stream, yet we ran out of chairs in rooms which became stifling with the number of people that were packed in. Knowing the shortage of good open source initiatives in this area it is hardly surprising for me, but it would certainly appear that JBoss underestimated the interest around the subject.

This introductory session talked around the work that was ongoing, and gave an overview of the direction the team was taking with Security. I did then attend the advanced session around SSO which showed a working example, more on that in the next report.

The Security 2.0 Beta which is currently available includes SPI and an initial version of JBossSX, version 2.0 then is expected to include Identity Federation and a cleanup of SPI, at version 2.1 we should see SAML support which will make the Security module standards compliant for SSO.

The default distribution within JBoss 5.0 is apparently written to work with OpenLDAP, but this is a pluggable architecture meaning we could use JDBC ontop of our own identity store (Oracle) should we wish. Configuration changes where shown to have been made around the "login-config.xml" file.

Following a number of discussions on security over the course of the day it would appear that the work JBoss are undertaking on their security strategy is completely independent of Sun with their Josso project. In addition it does strike me that JBoss have made more advances into this area than Josso has been able too to date. All in all it certainly looks like JBossSX will prove a strong contender for software security measures we choose to investigate.

Introduction to JBoss Rules - Pete Bennett and Mark Proctor

Rules engines are a topic that we have considered a couple of times within the development team. The arguments around this have been quite fluid, making it a difficult for us to decide on a specific direction. I had hoped that I would be able to get a clearer impression from the conference on a suitable direction for Ufi and I believe I have achieved that, though not with the kind of result we would have liked.

JBoss Rules originates from the Drools technology. This is an advanced rules engine which takes rules and facts and makes assertions about them. From our perspective this means that if we write a set of business rules in DRL script, as used by the engine, we can then run these against sets of Fact objects (Hibernate objects?) to assert the results.

Rules engines, including JBossRules, look extremely appealing when considering aspects of the business such as the enrolment process and ILR. However, following discussion with the JBoss developers the fact remains that the technology is shown to work best on flat, static, data structures around which the rules are expected to change over time without any need to modify the underlying model. It is this which may make it difficult for us to realise the benefits of integrating a rules engine into something like the LASER codebase. It may still perhaps be possible for us to realise some benefits within FMI, but I suspect LASER is looking like a lost cause from this perspective unless there is some compelling reason supplied by a peripheral system.

Some interesting points about the session related to how well the tool support appears to have progressed for this technology. Plugins for Eclipse are now available for DRL, with all the usual syntax colouring, validation, etc, and DSL has evolved so that rules now have "natural language" support, making them more human readable. In addition debug and audit views were briefly shown in Eclipse, making it easy to trace activity within the engine.

Tuesday, November 21, 2006

Cocktail Reception - End of Day One

Well what can I say ... I got stuck in when they brought the food out. Trevor would be proud :)

I had a good chat with Pete Bennett and Manik Surtani (both JBoss) about JBossCache, and asked them for their opinions on the problems we had come across earlier this year. The view looks to be that we try the latest version of JBossCache within 4.0.3SP1, and that failing that we invoke the support service we have to investigate the replication problems we had further. They said that JBossCache had moved on quite a lot since January time, making it unlikely that we would experience the same problems again now. Infact Manik described a problem that they had fixed around the use of the JDBC Connector which sounds suspiciously like it to me. This looks like another piece of investigative coding for our Pete at the moment.

I bumped into Mladen Turk, writer of mod_jk as was pointed out to me. It's great to meet so many people responsible for so many of the technologies we use on a day by day basis.

Red Hat / JBoss Product Roadmap - Shaun Connolly

First up came remarks about the JBoss / Red Hat merger. This has infact proven to be a theme running through the event so far which is hardly surprising. The message that has been supplied is very much one of a merger of two non-conflicting vendors joining forces to unify their open source technology stacks. Although Red Hat had the Jonas application server already the point was made that JBoss has a broader product base making the adoption of JBoss into Red Hat highly appealing for them.

The JBoss Division are committed to staying platform independent and are looking for opportunities to integrate with other aspects of the Red Hat stack. They also made reference to the fact that 2/3 of all bug fixes for Release 4.0.5 came from the open source community, making it quite clear that the community is providing them with the help they need to compete against the likes of IBM, BEA and Oracle.

JBoss 5 is in beta and is currently expected to be production ready sometime around mid 2007. This is will include a raft of new features such as a replacement for the jmx-console. We can also expect a new version of the JBoss ON software which Demetri is establishing within Operations currently. A large number of the JBoss products will also see major enhancements as part of the release, and JBoss are currently back porting some of these into JBoss 4.0.5. Areas where we will benefit include JBossCache and Hibernate, but virtually every major JBoss product will see a considerable step change within the next 12 months from what I can see.

All in all then it came across that the two companies joined to better enable both to stand up against the other big players in the market. An area that I believe some significant advantage will be being sought is Virtualization. This has been mentioned several times both within this session and during others that I have attended. It looks as though Red Hat and JBoss are looking to the enterprise data centres to fuel a large part of their growth, which marks a serious shift into territories dominated by closed source vendors like IBM and BEA. Pure speculation I know, but it could be one of the big advantages that Red Hat provide. In the meantime JBoss are continuing to build up their application base with Single Sign On, Business Process Management and Rules Engine technologies (amongst others).

One final point which I suspect I will end up having to research after the event is ESB. According to the conference guide this is based on the Rosetta ESB which has been in production for 3 years already. Both ESB and SEAM have had a lot of air time at the conference.

Introduction to JBoss SEAM - Brian Leonard, Sun Microsystems

Brian is a Java Evangelist; see his Blog, see Joplin. He gave a demonstration about how JBoss SEAM could be integrated into a Java Server Faces (JSF) and EJB3 application.

Interestingly he gave a quick comparison between J2EE 4 and 5 before getting into the detail. Key statistical comparions for his demonstration application:

Java EE 4 Java EE 5
Java Files 17 7
Lines of Code 987 716
XML Files 9 1

He moved swiftly on before I could write the rest down unfortunately. Anyway it gives you a taste for the difference if you had ever been in doubt.

SEAM is being proposed as the reference for JSR 299: Web Beans, which is looking to unify JSF and EJB.

I could poke fun at the guy for writing all the code he did only to have problems launching the server because port 8080 was in use, followed by his random guess as to which java process to shut down (I wish Sun would sort that out), the fact is though he knew his stuff and had given me the most entertaining and comprehensible session of the day so far.

One point that was raised was that the use of SEAM implied tighter coupling of the code, and there appeared to be pollution of the code with responsibility for validation placed in the persistence layer. I would say that this was demonstration code so it should not be taken seriously. The JBoss team are extremely excited about SEAM and it may be worth investigating this further when more time is available. It's not clear to me from what I have seen whether it implies a move away from the current industry approach on internal application design.

The main point of the session was to show how easy it was to take a JSF + EJB application and bind it with SEAM rather than a ManagedBean. This the presenter achieved admirably, having to only really make configuration changes to achieve this. He pointed this out explicitly in his closing remarks, suggesting that tool support for SEAM would be simple enough to introduce given this fact. It should be interesting to see where JSR 299 goes from here, and indeed how JBoss develop SEAM to better integrate it with tools like Eclipse.

Introduction to JBoss jBPM - Tom Baeyens

This is one of a number of emerging tools that JBoss are currently pressing ahead with.

BPM (Business Process Managament) aims to represent business processes via a schematic which would then be used by developers to more easily provide a technical solution. The concept here is that it would be quicker and easier to move from the realm of the business analyst to the developer if a language can be used to more effectively described the business process to be rendered. At this level the problem then becomes one of providing an appropriate tool that the Business Analysts can use to trap their description, and which the developers can then use to generate the basis of the system.

JBoss are pushing jPDL which provides process descriptions based on XML Schema.

jBPM was described as offering a sophisticated state machine with workflow and service orchestration on a "multi-process language platform". Unfortunately we didn't see it in action so it looks like it needs some further investigation. I suspect this is akin to a rather expensive application Jim gave me a demo on a few weeks ago.

A couple of questions were asked which deserved a worthy mention around the following:

1. BAM (Business Activity Monitoring), or auditing, is currently being built into jBPM.
2. jBPM supports the deprecation of processes.

All in all it was quite a colourful session, but I suspect you need to have some practical experience with BPM tools before you can relate this to your own workplace. The reference to auditing caught my interest though, which is hard to do with subjects like that.

Introduction to SOA - Shaun Connolly

Service Oriented Architecture (SOA) is considered to be the current trend in software development. SOA aims to address the difficulties caused by businesses expanding their inventory of systems over time. The problem is considered to be the existance of independent systems, with independent storage, infrastructure, operational tools, and most importantly data. SOA presents the opportunity for systems to be brought together through the amalgamation of services ontop of the data repositories. It is believed that this is becoming an increasing concern with CIO's who are looking for flexible, innovative solutions with lower costs.

SOA is an underpinning feature of the JBoss Application Server, but is not a product in itself. The presentation extolled the virtues of the approach and appeared to be a leader into other presentations around, jBPM, SEAM and ESB, all part of the JBoss suite.

The following quotes provide a useful insite:

"If you don't like change, you're going to like irrelevance even less", General Eric Shinseki, Chief of Staff, US Army.

"The entire history of software engineering is one of rising levels of abstraction", Grady Booch.

It is believed that SOA offers greater flexibility and agility, that it creates more loosely coupled systems which are more process oriented, and that it helps to avoid the bing-bang approach to development as well as analysis paralysis. The presentation was not able to support these claims unfortunately, so I felt quite wary of the hype.

The presenter believed that perhaps as many as 40% to 50% of JBoss customers he has had dealings with are either pursuing or are planning to pursue an SOA strategy. One brief point to note is that the subject was tackled independently of web services; as he put it, the trick seems to be in choosing a suitable level of granularity at which to work.

I did feel that an excellent point was raised by another delegate at the end: how will businesses manage the rising tide of services they build? The implication here being that businesses are currently managing a large number of applications, so what is the killer case for SOA? This whole approach does feel intertwined with some of the later sessions around web services, business process management, etc, we shall see. However, in direct answer to the question he did say that JBoss were considering governance of services as an avenue of work for them to pursue in the next couple of years.

I wont comment further on this presentation except to say that the message here was really to put across a design style for systems. It all made a great deal of sense, which is no real shock given this is the approach we are taking on the LASER project. What we have yet to do though is to look at implementing services within other applications so that we can plug systems together. To date the best we have managed to do on this are the webservices that FMI has, but we have not benefited from re-use on this.

The ongoing review of systems for the purposes of defining the information map and capturing factual data about the architecture of those systems will ultimately need to consider where data items belong and whether they can be centralised. At this point we will be able to consider where services belong within the Ufi estate.

I've written quite a lot for what was quite an abstract discussion. This is probably because this, for me, is something that we will have to consider within the Ufi Technical Strategy.

That sinking feeling

Now it's worth pointing out straight away that sinking is not always a bad thing, after all fish do it quite happily every day.

I'm coming to the end of day 2 at JBossWorld Berlin and I can definitely say it's been worth it for me so far. It's not every week I can sit down and spend virtually 3 days sponging up information about our technology of choice so this has been absolutely fantastic for me.

To better help share some of the information that has come out of this I will provide an entry for each of the sessions I attended. Please excuse me if some of these are pretty much brain dumps at present, I will tidy them up and provide cross links over the next few days.

A lot of the material will largely be of interest to the developers, but there is some around the Red Hat / JBoss roadmap which could also be of interest to folks in Operations.

At the highest possible level he story so far is very much one of two companies combining forces, seeking stability and then working on making a major impact with JBoss 5 next year. The immediate advantages for us are that a number of products are going through a step change at present, and these will be back ported into JBoss 4 should we wish to adopt them within our current production version of JBoss.

Besides that Berlin is a fantastic location, the food and alcohol is apparently limitless, everyone is exceptionally helpful and it's great being able to put faces to names and vice versa!

Thursday, November 16, 2006

Welcome to my blog

I dedicate this blog to those people that wonder why I don't make it out to the pub on a Thursday night, or in fact on any other nights. The reasons for this are not too numerous to mention for those that know me, indeed they are 3. All very good reasons I might add, but still very young and with a tendency to throw themselves at me, or more specifically my ankles, when I walk through the door.

So, who am I?

Well as you can guess from the title I am Simon Mather, and to those for whom that means nothing or at least nothing new, I am father of 3 and Head of Software Engineering for Ufi / learndirect.

This blog is mostly work related, hardly a surprise I guess, but it wont all be. Well, it's a blog after all.

I am of the "spectrum generation", which apparently is a good thing (http://www.nvg.ntnu.no/sinclair/sinclair/clive_dt170998.htm). Anyway, thanks to Clive Sinclair I believe I have now been programming computers for one purpose or another for over 20 years. The weirdest system I have worked on would be programming a laboratory robot and ICP-AES when I was at polytechnic. However, the norm these days is for me to work on large online applications written in Java. It's not so unusual a claim today perhaps but then it pays the rent a little better.

Back to the plot: I joined Ufi in November 2004 to setup and lead the development team and became father of 3 in 2005 (told you). I have the responsibility for bringing to fruition the next generation frontline software solution (LSE) for learndirect and to provide ongoing development and support for the not insubstantial finance system which the team inherited. We are 30 strong now and are 12 months, or one and half releases, into the venture.

As this was only intended as a brief introduction I shall stop right here. Coming up shortly, notes on the JBossWorld 2006 conference in Berlin ...