Friday, 1 November 2013

Pumpkins are Composable Modules Too

Ludwigsburgers (if thats a reasonable way to describe the people of Ludwigsburg) know how to celebrate Hallowe'en. They have the most brilliantly mad display of pumpkin sculpture I've ever seen. I don't know how this exhibition ever got the green light, but I'm glad it did. "I know, lets model all the events of the Olympic games as pumpkin sculptures!". And why not? Swimming, ski-jumping, discus, bob-sleigh, weight-lifting...its all there in the beautiful baroque gardens of Ludwigsgburg palace.


I was in Ludwigsburg for Eclipsecon Europe on Hallowe'en this year to talk about how OSGi helps complex software engineering projects manage their own success. If you want to see more about the highs and lows of WebSphere's use of OSGi, the mistakes we made and lessons we learned - and where I see OSGi heading - then here are (most of) the slides I used.

One aspect of this story that's going to be hard to improve on is the one at the bottom of slide 2. But if that longest of long-haul journeys ends up in Berlin in 2015, I'll be there...



Tuesday, 29 October 2013

Travelling Light for the Long Haul - at EclipseCon Europe

I've been travelling a lot recently and this week I'll be speaking at EclipseCon Europe, in Ludwigsburg Germany, on.....Travelling Light for the Long Haul. I quite like using long-distance travel, with all its potential discomforts, as an analogy for long-lived software projects. The more you do the travel, the more you plan ahead with strategies for making it smoother and more comfortable for yourself. Today, for example, I'm travelling economy (as always...) but writing this from a business lounge.

I'll be giving a keynote on the OSGi track and I'll start off by looking at how OSGi has enabled one of IBM's most successful software products to survive and thrive in a challenging market.

One of the attractive qualities of OSGi is its role in enabling technologies that adopt it to manage the cost of their own success. Anything that gains adoption - in technology or elsewhere - picks up baggage as a result and needs to figure out how to deal with current installations while expanding in new directions. The WebSphere platform has been around for almost as long as Java and knows a thing or two about baggage but still manages to travel to many places with just a carry-on allowance. We adopted OSGi internally 8 years ago and have gradually increased our exploitation with each passing release, most recently and deeply with the lightweight WAS Liberty Profile. It hasn't all been plain sailing and we learned from a number of mistakes made along the way. When WebSphere Application Server first adopted OSGi it had over 10 million lines of code in a modest number of huge JARs. The engineering effort to modularize that into a “sensible” number of OSGi bundles was fairly significant. We had a global development team spread across a dozen labs and nearly as many timezones, all learning OSGi principles at the same time. What could possibly go wrong? I’ll spend a little time reviewing the consequences of our bundles-first-services-later approach but our success was initially limited to having the equivalent of a well-organized and large container ship which could travel at speed but needed a pretty wide berth. Our initial investment in OSGi delivered on most of the internal benefits we wanted but failed on some of the external ones that matter to our customers.

Application Servers are used in different ways by Developers and IT Operations. Ops teams care about the overall cost, including performance and availability, of the platform and the applications it supports; Dev teams care about how quickly and easily they can create and deliver their applications and treat the server as a tool. Only some of them know or care about OSGi; multi-channel enablement and cloud deployment are the current pressures they are under. Today, WebSphere is a consumer of OSGi in two distinct fashions. Internally we learned from our earlier experiences and embraced an OSGi services model to enable us to run the same runtime just as fast but in a far more dynamic fashion: it’s how we can start/stop individual technologies of the Java EE Web Profile independently on the WAS Liberty profile, in a 50MB install with a 2-second startup while still support all our customers’ existing deployments. Externally we support both Enterprise OSGi and traditional Java EE as application programming models, on the same runtime and using the same Eclipse-based tools. Our customers who understand and care about OSGi can develop and deploy web application bundles and multi-bundle enterprise applications. Those who don’t care about OSGi benefit from it indirectly through the OSGi subsystem features that provide the server capabilities their applications use.

One of the challenges OSGi continues to face is over when to be “front of office” and when to be “back”. As the industry accelerates towards cloud, OSGi is an internal part of IBM’s strategy for high-density virtualized Platform-as-a-Service through WebSphere Liberty. Today’s cloud provisioning strategies, for example the buildpacks used by Heroku and CloudFoundry, are designed to be technology-agnostic. As a programming model for the cloud, OSGi is in a position of strength with its heavily service-oriented architecture. But in the spirit of agnosticism, one of the next steps OSGi needs to take is simply greater availability of the core OSGi framework in some of the more popular cloud platforms. Once there are more OSGi services running in those environments then the value and simplicity of autowiring OSGi services as cloud services becomes more apparent.

We’ve taken some steps in this direction already with a WebSphere Liberty buildpack for the Cloud Foundry PaaS which provides an OSGi container (using WebSphere and Eclipse Equinox) for enterprise OSGi applications – you just bring your bundles and push them to the cloud. Come and hear more at EclipseCon. I’ll be speaking on Hallowe’en…

Monday, 31 December 2012

Why isn't Software Install just 'Unzip'?

I've been a big fan of Spotify since it first launched in the UK in 2008. I like it because it does what you expect it to and has a pretty good history of providing those "why hasn't anyone done this before" features....before anyone else does. (Or at least, before anyone else does them well enough for me to want to use them). Like collaborative playlists that don't require any money to change hands to create, share and enjoy. Brilliant. And the Spotify Download store was an obvious next step and a nice poke in the eye of the increasingly distasteful iTunes Store hegemony. But this isn't an advert for Spotify....like many others I was annoyed before Christmas at the abrupt closure of the Spotify store which I was relying on for the annual update to the Robinson christmas playlist. After enjoying the Gig of the Year in July (last year's christmas present to my wife) when Springsteen was on fire in London and then had the plug unceremoniously pulled at 10:30pm (the promoter's public-spirited conscience not wanting to disturb all the hard-working families who live next door to Hyde Park), an obvious omission that needed adding was Santa Claus is Comin' to Town. So after the terse "We're currently not supporting new download purchases on Spotify", and burdened by my old fashioned belief in the moral imperative of paying for legal music downloads, I went to Amazon (turning a blind eye to Amazon's UK tax avoidance. A Guardian reader's life is full of compromises). And having just started my Christmas vacation from work, I was immediately reminded of what I had recently left behind....the question of "why isn't software install just unzip?".

Bruce's addition to my playlist was an 8Meg MP3. Having purchased it, it should have taken me a couple of seconds to download it to wherever I liked on my hard drive and then 1 or 2 more to add it to my festive playlist. But wait....I'd forgotten about the Amazon MP3 Downloader - a 2M software installer for an 8M MP3 file that surely requires no "installation". Having spent the 89p I was obviously committed beyond the point of no return and so installed the damned installer. And after a few more seconds, Bruce was nicely sandwiched between Greg Lake and Louis Armstrong.

If I were ever to look back at my last week of 2012's mail/meeting/slidedecks/wiki summaries, I know I would find an inordinate amount of discussion over how we evolve the two approaches we have of installing WebSphere AppServer software. This may address a different scale of software and consumer from an MP3 file but some of the end-user install considerations are the same - certainly for those who don't want to have to manage the lifecycle of an Installer in addition to the software it installs. For the smallest distribution of the WebSphere AppServer - the WAS Liberty Profile - we provide two quite different ways to install the AppServer software and manage its lifecycle:
  1. through IBM Installation Manager
  2. through unzip
Actually, "unzip" isn't quite this because the archive file is actually an executable JAR whose Main-Class is a simple embedded unpack-installer and so requires java.exe to initiate the archive install. (For the sake of completeness, there is a third approach to install the WAS Liberty Profile which is via the WebSphere Developer Tools for Eclipse (WDT)). The reason there are two such different approaches is because we have two fairly different types of user - one type who prefers greater control and flexibility (and favours the unzip approach) and another who prefers a more managed approached that hides complexity, for example around managing dependencies between interim fixes (iFixes). I'm often asked about the differences between these two approaches so this post attempts to describe both our intentions and some of the end-user considerations around them.


Archive Install

Archive install (a more accurate name than "unzip install") was introduced in WAS V8.5, specifically for the WAS Liberty Profile - a component of WAS V8.5. It was initially motivated by developer-centric scenarios but expanded in ambition during the development of WAS V8.5 so that by the time we shipped WAS V8.5 it was a supported means for production scenarios too. The archive file to be installed is an executable JAR whose Main-Class displays an End User License Agreement and then unpacks the JAR content to the target installation directory. Distributions that contain a production license are available from IBM Passport Advantage. Distributions that contain a no-fee developer license are also available from WASdev.net > Downloads. This is the usual starting point for both a free try-it-and-see experience and a free Developer Desktop environment because it requires no accounts to be created or contracts to be procured - its a straightforward download of a 47M jar that is then installed through running:
java -jar jar-you-just-downloaded

There is no independent installer that has to be set up first, although this route does assume you already have Java 6 SE (or later) installed. If you also need to install Java then the IBM Java Development Kits can be downloaded from here. Lets assume you downloaded and used 'java -jar' to install the WAS Liberty runtime in c:\wlp. There is a README.TXT file in the root install directory (c:\wlp) that describes the defaults for a number of directories relative to where you installed the WAS Liberty runtime. For example, wlp.install.dir is the install directory and the default user directory, wlp.user.dir, can be found at c:\wlp\usr. These are important to understand, especially from an install perspective: the user directory is where all user information, such as server configurations, resources and applications will be located. The manner in which the install is laid out on disk and in which the user directory information is isolated from the rest of the install is designed to simplify the lifecycle management of WAS Liberty and to make archive install a practical option for production environments to which service can be applied. Consider a scenario where WAS Liberty V8.5.0.0 is installed with all the defaults and two servers, server1 and server2 have been created, with application snoop.war installed on both and application creditApp.war also on server1. The default directory structure looks like this:
WAS Liberty directory structure

Actually, the etc\server.env properties file is not created by default and is only needed if you want to override an environment default for all the servers under wlp.user.dir, in which case you can create it with any text editor. To be explicit about the location of the user directory, we can create a server.env file which contains:

WLP_USER_DIR=c:\wlp\usr

Lets now assume this is a production scenario (likely fronted by a Apache HTTP server with WAS plugin configured for these two servers): how would you now use archive install to move up to the next fix pack (8.5.0.1) with minimal disruption of service? This is actually very easy. You obtain the 8.5.0.1 fix pack jar wlp-base-8.5.0.1.jar from IBM Fix Central [http://www-933.ibm.com/support/fixcentral] and install it using java -jar to a new location e.g. c:\wlp8501\wlp. Next you override the default user directory of the new 8.5.0.1 install to point to your existing wlp.user.dir to pick up all the server configurations, applications and other user-defined resources. This is as simple as copying the etc\server.env file from c:\wlp\etc to c:\wlp8501\wlp\etc. Then you just stop each running 8.5.0.0 sever (e.g. 'c:\wlp\bin\server stop server1') and restart using the new fix pack level ('c:\wlp8501\wlp\bin\server start server1 --clean'). The '--clean' option is advisable on a first restart after updating binaries to flush any cached information from the previous run. Moving back to 8.5.0.0 is simply the same process in reverse (stop 8.5.0.1, start 8.5.0.0). This example didn't change any defaults in the WAS Liberty 8.5.0.0 install. In reality, if you're planning for this sort of service methodology you'd almost certainly create your initial wlp.user.dir somewhere that wasn't a subdirectory of the wlp.install.dir.

As you can see, moving up or down fix pack levels with archive install is very straightforward. What about applying interim fixes (iFixes)? The fix pack install I just described is a side-by-side replacement of (all) the platform binaries. An iFix is a jar-cumulative update of one of more jars in the platform install. By "jar-cumulative" I mean that if the same jar is fixed for two different APARs and you want the second APAR, then it is cumulative with the first APAR - you can't take the second without the first. Each iFix is another executable JAR obtained from Fix Central, named using the scheme -WS-WASProd_WLPArchive-IF.jar , along with a ReadMe.txt containing instructions on how to apply the iFix and the list of files that should be manually backed up before applying the iFix. The result of executing the iFix JAR replaces files in the wlp.install.dir\lib directory.There are further details on this process in the WAS InfoCenter. Having applied an iFix, you'll notice a new directory wlp.install.dir\lib\fixes containing metadata about the applied fix. This is used by the WAS Liberty productInfo utility to help you identify whether specific iFixes have been applied to an install and/or to compare two different WAS Liberty installs to see how the list of applied iFixes differs between the two. This is particularly useful if you deploy applications as part of packaged servers using the WAS Liberty command 'server package' to produce a 'packaged server' archive from an existing WAS Liberty install, including the applications, server configuration and other contents of the wlp.user.dir relevant to the server. This archive can be unzip-deployed to multiple target hosts; over a period of time multiple iFixes might be applied to each instance of the unzip-deployed servers as well as to the original install. The productInfo utility can help you to manage keeping the unzip-deployed servers at the same fix level (if desired). For example, suppose you produced a packaged server1 from the 8.5.0.1 scenario I described above:

C:\wlp8501\wlp>server package server1
Server server1 package complete in c:\wlp8501\wlp\usr\servers\server1\server1.zip.



This zip archive includes all of the WAS Liberty runtime and user content in wlp.user.dir\servers\server1 and shared.app.dir. This can be copied and unzipped to multiple hosts. Any future iFixes can be applied directly to each target install or could be applied to the original master install if you prefer to create a new packaged server image (now including the iFix) and redistribute/unzip on the target hosts. For example, against the WAS Liberty 8.5.0.1 install:
C:\wlp8501>java -jar 8.5.0.1-ws-wasprod_wlparchive-ifpm75505.jar
Applying fix to Liberty install directory at C:\wlp8501\wlp now.
Replacing file at 'C:\wlp8501\wlp\lib\com.ibm.ws.kernel.boot_1.0.1.jar'.
Fix has been applied successfully.

You can use the WAS Liberty productInfo command to compare the contents of any two installations, including an installation in its packaged form resulting from a server package operation. For example, the following command starts with the WAS Liberty installation at c:\wlp8501\wlp (which now includes an iFix) and compares it against the earlier packaged server1.zip (without the iFix):

C:\wlp8501\wlp>productinfo compare --target=c:\wlp8501\wlp\usr\servers\server1\server1.zip
Some of the iFixes in the image at c:\wlp8501\wlp are missing in the image at c:\wlp8501\wlp\usr\servers\server1\server1.zip.

The following iFixes are in the image at c:\wlp8501\wlp but are missing in the image at c:\wlp8501\wlp\usr\servers\server1\server1.zip.
PM75505 in the iFix(es): [8.5.0.1-WS-WASProd_WLPArchive-IFPM75505]

All in all, the archive install procedure is a quick and easy way to get started, requires no external installer and provides a lot of flexibility around managing and maintaining WAS Liberty instances so installed. But it does not manage the install and maintenance of the Java platform used by WAS, it does not implicitly calculate any dependencies between iFixes, it doesn't provide automatically back up for files replaced when an iFix is applied and nor does it search Fix Central for available updates. For this sort of managed install capability you do need a discrete installer which brings us on to......

Installation Manager


Installation Manager (IM) is an IBM technology used to install many IBM products, including WAS since WAS V8.0. The WAS Liberty Profile was introduced into all editions of WAS V8.5 and can also be installed using IM. The initial result of installing WAS Liberty through IM is exactly the same as an archive install except that IM also lays down additional metadata that it uses to manage future feature and service updates. Installation Manager itself needs to be downloaded first, for example from the IBM Support Portal and unzipped before it can be used. The thing you download here is imaginatively referred to as the "Installation Manager installer". There was probably an IBM prize for coming up with that name. Once you've unzipped this you have the choice of using this directly as an "install kit" for scripted/silent installs of the target product (e.g WAS) or using it to fully install Installation Manager, including the IM user interface, and then using the result of that to install and service WAS. Details about the differences in these two approaches are in the IM InfoCenter but for the sake of simplicity in this article, I'll assume the latter.

IM installs and services product offerings like WAS from one or more installation repositories, the URLs for which needs to be configured in IM. Each edition of WAS (e.g. WAS Network Deployment edition) actually consists of a set of installable offerings, each with its own installation repository. The repositories themselves are available in a number of different forms depending on the product offering - for example the repository for the WAS ND offering is provided at three locations: on the WAS ND product media, downloadable from IBM Passport Advantage and in an IBM-hosted web-based repository. The complete list of all installable WAS V8.5 offerings (across all editions) can be seen in the WAS InfoCenter. You can also create custom enterprise repositories containing the service levels and iFixes desirable for an enterprise using the Packaging Utility as described here. IM installs WAS Liberty as part of one of the higher-level product offerings that include Liberty. You can see in the WAS InfoCenter that for V8.5, this includes:
  • com.ibm.websphere.BASE.v85
  • com.ibm.websphere.BASETRIAL.v85
  • com.ibm.websphere.EXPRESS.v85
  • com.ibm.websphere.EXPRESSTRIAL.v85
  • com.ibm.websphere.DEVELOPERS.v85
  • com.ibm.websphere.DEVELOPERSILAN.v85
  • com.ibm.websphere.zOS.v85
  • com.ibm.websphere.ND.v85
  • com.ibm.websphere.NDTRIAL.v85
To use IM to install WAS Liberty from a hosted web repository, you start IM and configure it, using the IM File->Preferences->Repositories menu, to point to the repository for the desired offering. The URLs use the following convention:
http://www.ibm.com/software/repositorymanager/
for example
http://www.ibm.com/software/repositorymanager/com.ibm.websphere.BASE.v85

Choosing the IM "Install" option will offer the choice to install, in this example, the base edition of WAS V8.5. As part of the install you choose whether to include the WAS full profile (with various options) and/or the WAS Liberty profile.
Installation Manager install of WAS V8.5 showing WAS full and Liberty profiles


By default, IBM Java 6 is also installed with WAS when using IM (unlike the archive install route). IBM Java 7 can be optionally installed as part of WAS if the following repository ID is added to IM:
http://www.ibm.com/software/repositorymanager/com.ibm.websphere.IBMJAVA.v70

While there is more overhead to installing WAS Liberty by this route rather than the archive install, the benefits of IM become more apparent once you need to apply service to the installed product. When you start IM and choose the "Update" option, IM searches the configured repositories (IBM hosted repositories or custom enterprise repositories) for all fixes applicable to the products installed by IM and presents a list of those that have not yet been applied - unlike the archive install route you don't have to know about these a priori.

Installation Manager simplifies the application of service to installed packages

If there are any pre-requisites to this iFix, IM will resolve these and by default it back up any updates so they can be undone - all capability beyond archive install. IM can only manage installations of WAS Liberty that have been installed by IM. If you use IM to install WAS Liberty, develop and test an application using that install, use the Liberty runtime server package command to create a packaged server with your application resource and server configuration, and then unzip deploy that on a remote host, you will not be able to use IM to apply service directly to the unzip deployed packaged server. For this scenario you can either:
  1. use IM to apply service to the original install and then repeating the server package and unzip deploy of the result to the remote host
  2. use the methods described in the archive install section above to apply an iFix directly to the unzip deployed server on the remote host.

In summary, "unzip" can be good enough for software install. But after a few service updates have been applied, the complexity of managing these manually may start to make a managed installer look more cost effective. Both approaches are valid for WAS Liberty in production - I hope this post helps explain the pros and cons of each.

- Ian.


Saturday, 15 October 2011

WAS V8.5 Alpha and the WebSphere Liberty Profile

Earlier this year I took on the role of Chief Architect of the WebSphere foundation and have been busy with the team since then working on the development of our next release. We launched the first fruits of our labour last week through our new WASdev community site: the WAS V8.5 Alpha. This is an early glimpse of what's coming next in WAS and it is heavily focussed on making the WAS runtime and tools the best environment possible for developing and testing web applications. At the heart of the Alpha is a new dynamic profile of the WebSphere runtime that we've called the WAS Liberty Profile. Rather than being a static profile of runtime features, Liberty adapts to the requirements of the application and ensures - at a really fine-grained level - that only the necessary application container functions are started. It gives you the freedom to deploy web applications with wide-ranging requirements and provides all the pieces they need (like security, transaction management, connection pooling, persistence through JPA or JDBC, and so on) without pulling in any more than they need. And it provides fidelity with the full-profile WAS runtime to which applications are deployed in production because the containers and services started by the Liberty profile are the same.

One of the key innovations in Liberty is a new kernel that processes the specific "features" required by an application to load the required container(s) and WebSphere platform services: WAS has been an OSGi-based runtime for many years (since WAS V6.1) but, with the Liberty profile in the WAS V8.5 Alpha, we've moved beyond simply modularity and gone to town on exploitation of dynamic OSGi services. This is what delivers the tiny runtime footprint and a WAS server that starts up in a couple of seconds. And this is only one aspect of what's new for developers - we've also dramatically simplified server runtime configuration so that a test/debug WAS server instance can be configured easily, either inside or outside an Eclipse environment, with a single XML file covering all aspects of the server, the applications and the resources required by the applications. If you've ever looked at a WAS configuration and wished you could treat it more like a development artefact - storing the configuration in an SCM system, versioning or diffing it, sharing it between developers - well now you can. While the server config can be as simple as a single XML file, there are also flexible ways to compose a configuration from fragments that are aggregated through include statements.

The WASdev home page lists "8.5 Reasons why the WAS Liberty Profile is awesome". For more details, including an overview of the new simplified server config, take a look at the Introduction to the Liberty Profile article I posted to WASdev.



And did I mention that its free? And so, now, are our most popular web development tools which we're making available as Eclipse features (in addition to being bundled inside RAD). You can get all this from WASdev downloads page.

But enough talk....lets see how quickly we can download the tools, the WAS server, configure a server instance, install a simple app and bring the whole thing up. I'll assume you already have the Eclipse IDE for Java EE but if not you can get that from the Eclipse site. The only other thing you need is an IBM ID for the downloads - if you don't have one then simply register for one here.

OK. Open up your Eclipse IDE (3.7 or 3.6.2), start your stopwatch and follow these steps:
  1. Go the WASdev download page to install the WAS Liberty Profile tools into Eclipse. This gives you a rich set of web development tools along with the ability to launch and control instances of the WAS Liberty Profile server. For Eclipse 3.7, its as easy as dragging the install button from that page onto your Eclipse IDE title bar.
  2. Accept the license terms (I'll take the liberty of assuming that, under these "beat the clock" conditions, you have already squared this with your friends in the legal department) for the install to proceed. The total size of the tools being added to your Eclipse environment here is less than 3M.
  3. Restart Eclipse. The tools are now installed - we can use these to now install the WAS server runtime itself for use testing and debugging applications. This is going to download and install a whole WAS Liberty profile server. Sounds scary? It isn't - the whole runtime is just another 25M and there's no additional installation program required.
  4. In the Eclipse workbench, open the Servers view. Right-click the Servers view and select New > Server. Under the server type list, expand IBM then select the WebSphere Application Server v8.5 Alpha Liberty Profile server type. Click Next.
  5. The WebSphere Runtime Environment page is displayed. In the Installation folder section, click download and install. Choose the WebSphere Application Server V8.5 Alpha site and enter your IBM ID and password and click Next. Accept the license terms for the runtime. (You'll have already mentioned this to the same lawyer you talked to about the tools license). Click Next.
  6. In the Download and Install page, specify the installation target folder for the WAS runtime to be installed to. (Just 25M, remember).
  7. Click Next. WAS is now installed. (Note that, if you just want to install the WAS Liberty profile runtime independently of the tools for test and debug puposes, then you can skip all the steps above and simply download the WAS Liberty Profile runtime directly from the WASdev downloads page and unzip it).
  8. There's a pre-configured defaultServer instance but lets go all the way and create our own server instance using the New button. Call it whatever you like e.g. firstserver. Click Finish. You now have a WAS server instance to which you can install and test web applications. On your filesystem, there will be a firstserver directory structure in the usr/servers directory of your new WAS install.
  9. Now we need to deploy a web app to this server and run it. You can download this simple StopWatch WAR file and import it into Eclipse.
  10. In Eclipse: File > Import WAR file. Click Next. Enter the location of the WAR file on your filesystem and make sure the Target runtime is set to WebSphere Application Server V8.5 Alpha Liberty Profile. Click Finish.
  11. In the Project Explorer, right click on the new StopWatch project and choose Run As > Run on Server.
  12. The WebSphere Application Server V8.5 Alpha Liberty Profile should already be highlighted. Click Finsh.
  13. Stop the Clock!
If you're looking for the fastest, cheapest and easiest way to develop web apps for WAS then this is it.

I've been busy spreading the message about Liberty since we started the WAS V8.5 Alpha - you can take a look at my slides from the WebSphere Technical Conference in Berlin, where we launched the Alpha:


And if you happen to be in Mumbai on Oct 19 or 20, come and hear me talk about Liberty at the IBM Software Universe.

Saturday, 18 June 2011

God Dad, You're Soooo Embarrassing

...the words every proud parent longs to hear, to know they've succeeded in keeping their offspring alive long enough for them to become a teenager. I heard this as I walked through the door yesterday and was doubly pleased when I found out why. The video I made a little while ago to kick off the new Enterprise OSGi YouTube channel has successfully attracted enough attention for my daughter to be teased at school, once again proving the value and reach of social media. Result.

Friday, 28 May 2010

OSGi and SCA come together in the WebSphere OSGi Application Feature Pack

28 May 2010: The WebSphere V7 OSGi Application and JPA 2.0 feature pack is now generally available (and free!) and supported for production use. This is something we're pretty excited about in the WebSphere team as it offers some truly powerful techniques to simplify the development, management and maintenence of complex enterprise applications. I described some of the primary use cases for enterprise OSGi applications in a previous post about the early program we've been running since the end of 2009. Some significant additional capabilities have been added since then.

The first one I'd like to talk about is SCA. If you install both the OSGi Application feature pack and SCA feature pack then you'll find not only do you have the ability to assemble SCA composites and OSGi applications but you can also include OSGi applications as components in SCA composites - the combination of these two feature packs is greater than the sum of the parts. So what does it mean to use SCA with OSGi? The two technologies are used for quite different purposes but are extremely complimentary.

OSGi provides a means to break down your enterprise application into (i) modules (bundles) that are specific to the applicaton and (ii) bundles that are common across applications. The latter can be separated out from the application archive and placed in a common OSGi bundle repository that WebSphere is configured to use during application deployment. Immediate benefits include smaller applications, a standard mechanism for using shared libraries in enterprise applications and a smaller memory footprint when two or more enterprise applications share common libraries. And you haven't needed to write any new or different Java code to take advantage of this. Another feature OSGi brings is a standard extensibility model - through OSGi services. You can design your application to take advantage of OSGi's rich and dynamic service-based model to consume services from the OSGi service registry. Service interfaces then become extension points that you can design into your application and extend it in the future via service provider implementations introduced through separate bundles with no change to the original application code. OSGi defines Java APIs for registering and discovery OSGi services but a declarative approach to use OSGi services is much simpler. This is where the OSGi Blueprint container comes in - the Blueprint container (which is added to the WebSphere runtime when the feature pack is installed) manages the lifecycle and dependency injection of POJO bean components and is configured through an  application-level XML bean definition file which is a standards-based evolution of the Spring bean definition XML. Blueprint provides a fine-grained bean assembly model for OSGi applications and simple declarative means to publish a service provided by a POJO bean component. For example, the following bean definition snippet defines a bloggingServiceComponent bean, implemented by the BloggingServiceImpl class for which a BloggingService service is registered to the OSGi service registry.

<blueprint>
<bean id="bloggingServiceComponent" class="com.ibm.ws.eba.example.blog.BloggingServiceImpl">
<property name="blogEntryManager" ref="blogEntryManager"/>
<property name="blogAuthorManager" ref="blogAuthorManager"/>
<property name="blogCommentManager" ref="blogCommentManager"/>
</bean>
<service ref="bloggingServiceComponent" interface="com.ibm.ws.eba.example.blog.api.BloggingService"/>
</blueprint>


So now we can build a modular, extensible application assembled from POJO components described by a standardized XML bean definition file. No SCA so far - when does that become interesting?

Say you wanted to remotely publish the BloggingService and make is available over an ATOM binding. Or you wanted to compose your OSGi application into a coarse-grained composite with other non-OSGi components. These two scenarios are where SCA comes in. The WebSphere OSGi and SCA feature packs support both these patterns through the introduction of a new SCA implementation type, impl.osgiapp. By defining an SCA component for your OSGi application you can assemble it into an SCA composite with any other SCA implementation type - for example an EJB component (impl.ejb). And by promoting the OSGi <service> definition into an SCA component definition, the service becomes remotable with a choice of any of the standard SCA bindings to determine how the service invocation and parameters should be serialized. Thats a pretty powerful extension to raw OSGi applications and enables OSGi applications to be used as part of a SOA application deployed to WebSphere.

And then there's the tooling. The second major addition to our OSGi Application support that I wanted to mention is our Eclipse-based tooling to simplify the development and deployment of OSGi Applications. Integrated into the new RAD Beta are new project types for OSGi Applications, graphical tools for editing OSGi Application and OSGi bundle manifests and a Blueprint XML editor for blueprint module definition files. These offer familiar design and source views so you can choose to edit the metadata directly or through a form-based editor. And of course there's all the content assist, validation and re-factoring productivity tools familiar in RAD. The RAD Beta also integrates a test server environment augmented with the OSGi feature pack to enable simple run-on-server testing.

RAD Beta OSGi Blueprint XML Editor


The feature pack contains a number of comprehensive samples that illustrate the primary use cases. Try it out - and if you have any questions or comments, visit the OSGi Application feature pack technical discussion forum.

Monday, 8 March 2010

Apache Aries - An open source project for Enterprise OSGi Applications

I spoke at the recent OSGi DevCon London conference (part of JAX-London 2010) as it extended late into the evening - Oracle's Mike Keith and I were competing with the bar in the 9-10pm slot as we talked about the Apache Aries and Eclipse Gemini open source projects. For those of you who chose the bar, or who couldn't make it to the gathering in West London, here are my slides on Apache Aries. They describe the primary goals and capabilities of Apache Aries, including how to get involved.
The 1-line summary is: Take a look at Aries if you want ready-to-use components that provide enterprise OSGi capability or if you want to join the development community to drive the technology forward.