Why isn't Software Install just 'Unzip'?

I've been a big fan of Spotify since it first launched in the UK in 2008. I like it because it does what you expect it to and has a pretty good history of providing those "why hasn't anyone done this before" features....before anyone else does. (Or at least, before anyone else does them well enough for me to want to use them). Like collaborative playlists that don't require any money to change hands to create, share and enjoy. Brilliant. And the Spotify Download store was an obvious next step and a nice poke in the eye of the increasingly distasteful iTunes Store hegemony. But this isn't an advert for Spotify....like many others I was annoyed before Christmas at the abrupt closure of the Spotify store which I was relying on for the annual update to the Robinson christmas playlist. After enjoying the Gig of the Year in July (last year's christmas present to my wife) when Springsteen was on fire in London and then had the plug unceremoniously pulled at 10:30pm (the promoter's public-spirited conscience not wanting to disturb all the hard-working families who live next door to Hyde Park), an obvious omission that needed adding was Santa Claus is Comin' to Town. So after the terse "We're currently not supporting new download purchases on Spotify", and burdened by my old fashioned belief in the moral imperative of paying for legal music downloads, I went to Amazon (turning a blind eye to Amazon's UK tax avoidance. A Guardian reader's life is full of compromises). And having just started my Christmas vacation from work, I was immediately reminded of what I had recently left behind....the question of "why isn't software install just unzip?".

Bruce's addition to my playlist was an 8Meg MP3. Having purchased it, it should have taken me a couple of seconds to download it to wherever I liked on my hard drive and then 1 or 2 more to add it to my festive playlist. But wait....I'd forgotten about the Amazon MP3 Downloader - a 2M software installer for an 8M MP3 file that surely requires no "installation". Having spent the 89p I was obviously committed beyond the point of no return and so installed the damned installer. And after a few more seconds, Bruce was nicely sandwiched between Greg Lake and Louis Armstrong.

If I were ever to look back at my last week of 2012's mail/meeting/slidedecks/wiki summaries, I know I would find an inordinate amount of discussion over how we evolve the two approaches we have of installing WebSphere AppServer software. This may address a different scale of software and consumer from an MP3 file but some of the end-user install considerations are the same - certainly for those who don't want to have to manage the lifecycle of an Installer in addition to the software it installs. For the smallest distribution of the WebSphere AppServer - the WAS Liberty Profile - we provide two quite different ways to install the AppServer software and manage its lifecycle:

1. through IBM Installation Manager
2. through unzip

Actually, "unzip" isn't quite this because the archive file is actually an executable JAR whose Main-Class is a simple embedded unpack-installer and so requires java.exe to initiate the archive install. (For the sake of completeness, there is a third approach to install the WAS Liberty Profile which is via the WebSphere Developer Tools for Eclipse (WDT)). The reason there are two such different approaches is because we have two fairly different types of user - one type who prefers greater control and flexibility (and favours the unzip approach) and another who prefers a more managed approached that hides complexity, for example around managing dependencies between interim fixes (iFixes). I'm often asked about the differences between these two approaches so this post attempts to describe both our intentions and some of the end-user considerations around them.

Archive Install

Archive install (a more accurate name than "unzip install") was introduced in WAS V8.5, specifically for the WAS Liberty Profile - a component of WAS V8.5. It was initially motivated by developer-centric scenarios but expanded in ambition during the development of WAS V8.5 so that by the time we shipped WAS V8.5 it was a supported means for production scenarios too. The archive file to be installed is an executable JAR whose Main-Class displays an End User License Agreement and then unpacks the JAR content to the target installation directory. Distributions that contain a production license are available from IBM Passport Advantage. Distributions that contain a no-fee developer license are also available from WASdev.net > Downloads. This is the usual starting point for both a free try-it-and-see experience and a free Developer Desktop environment because it requires no accounts to be created or contracts to be procured - its a straightforward download of a 47M jar that is then installed through running:
java -jar jar-you-just-downloaded

There is no independent installer that has to be set up first, although this route does assume you already have Java 6 SE (or later) installed. If you also need to install Java then the IBM Java Development Kits can be downloaded from here. Lets assume you downloaded and used 'java -jar' to install the WAS Liberty runtime in c:\wlp. There is a README.TXT file in the root install directory (c:\wlp) that describes the defaults for a number of directories relative to where you installed the WAS Liberty runtime. For example, wlp.install.dir is the install directory and the default user directory, wlp.user.dir, can be found at c:\wlp\usr. These are important to understand, especially from an install perspective: the user directory is where all user information, such as server configurations, resources and applications will be located. The manner in which the install is laid out on disk and in which the user directory information is isolated from the rest of the install is designed to simplify the lifecycle management of WAS Liberty and to make archive install a practical option for production environments to which service can be applied. Consider a scenario where WAS Liberty V8.5.0.0 is installed with all the defaults and two servers, server1 and server2 have been created, with application snoop.war installed on both and application creditApp.war also on server1. The default directory structure looks like this:

WAS Liberty directory structure

Actually, the etc\server.env properties file is not created by default and is only needed if you want to override an environment default for all the servers under wlp.user.dir, in which case you can create it with any text editor. To be explicit about the location of the user directory, we can create a server.env file which contains:

WLP_USER_DIR=c:\wlp\usr

Lets now assume this is a production scenario (likely fronted by a Apache HTTP server with WAS plugin configured for these two servers): how would you now use archive install to move up to the next fix pack (8.5.0.1) with minimal disruption of service? This is actually very easy. You obtain the 8.5.0.1 fix pack jar wlp-base-8.5.0.1.jar from IBM Fix Central [http://www-933.ibm.com/support/fixcentral] and install it using java -jar to a new location e.g. c:\wlp8501\wlp. Next you override the default user directory of the new 8.5.0.1 install to point to your existing wlp.user.dir to pick up all the server configurations, applications and other user-defined resources. This is as simple as copying the etc\server.env file from c:\wlp\etc to c:\wlp8501\wlp\etc. Then you just stop each running 8.5.0.0 sever (e.g. 'c:\wlp\bin\server stop server1') and restart using the new fix pack level ('c:\wlp8501\wlp\bin\server start server1 --clean'). The '--clean' option is advisable on a first restart after updating binaries to flush any cached information from the previous run. Moving back to 8.5.0.0 is simply the same process in reverse (stop 8.5.0.1, start 8.5.0.0). This example didn't change any defaults in the WAS Liberty 8.5.0.0 install. In reality, if you're planning for this sort of service methodology you'd almost certainly create your initial wlp.user.dir somewhere that wasn't a subdirectory of the wlp.install.dir.

As you can see, moving up or down fix pack levels with archive install is very straightforward. What about applying interim fixes (iFixes)? The fix pack install I just described is a side-by-side replacement of (all) the platform binaries. An iFix is a jar-cumulative update of one of more jars in the platform install. By "jar-cumulative" I mean that if the same jar is fixed for two different APARs and you want the second APAR, then it is cumulative with the first APAR - you can't take the second without the first. Each iFix is another executable JAR obtained from Fix Central, named using the scheme -WS-WASProd_WLPArchive-IF.jar , along with a ReadMe.txt containing instructions on how to apply the iFix and the list of files that should be manually backed up before applying the iFix. The result of executing the iFix JAR replaces files in the wlp.install.dir\lib directory.There are further details on this process in the WAS InfoCenter. Having applied an iFix, you'll notice a new directory wlp.install.dir\lib\fixes containing metadata about the applied fix. This is used by the WAS Liberty productInfo utility to help you identify whether specific iFixes have been applied to an install and/or to compare two different WAS Liberty installs to see how the list of applied iFixes differs between the two. This is particularly useful if you deploy applications as part of packaged servers using the WAS Liberty command 'server package' to produce a 'packaged server' archive from an existing WAS Liberty install, including the applications, server configuration and other contents of the wlp.user.dir relevant to the server. This archive can be unzip-deployed to multiple target hosts; over a period of time multiple iFixes might be applied to each instance of the unzip-deployed servers as well as to the original install. The productInfo utility can help you to manage keeping the unzip-deployed servers at the same fix level (if desired). For example, suppose you produced a packaged server1 from the 8.5.0.1 scenario I described above:

C:\wlp8501\wlp>server package server1
Server server1 package complete in c:\wlp8501\wlp\usr\servers\server1\server1.zip.



This zip archive includes all of the WAS Liberty runtime and user content in wlp.user.dir\servers\server1 and shared.app.dir. This can be copied and unzipped to multiple hosts. Any future iFixes can be applied directly to each target install or could be applied to the original master install if you prefer to create a new packaged server image (now including the iFix) and redistribute/unzip on the target hosts. For example, against the WAS Liberty 8.5.0.1 install:
C:\wlp8501>java -jar 8.5.0.1-ws-wasprod_wlparchive-ifpm75505.jar
Applying fix to Liberty install directory at C:\wlp8501\wlp now.
Replacing file at 'C:\wlp8501\wlp\lib\com.ibm.ws.kernel.boot_1.0.1.jar'.
Fix has been applied successfully.

You can use the WAS Liberty productInfo command to compare the contents of any two installations, including an installation in its packaged form resulting from a server package operation. For example, the following command starts with the WAS Liberty installation at c:\wlp8501\wlp (which now includes an iFix) and compares it against the earlier packaged server1.zip (without the iFix):

C:\wlp8501\wlp>productinfo compare --target=c:\wlp8501\wlp\usr\servers\server1\server1.zip
Some of the iFixes in the image at c:\wlp8501\wlp are missing in the image at c:\wlp8501\wlp\usr\servers\server1\server1.zip.

The following iFixes are in the image at c:\wlp8501\wlp but are missing in the image at c:\wlp8501\wlp\usr\servers\server1\server1.zip.
PM75505 in the iFix(es): [8.5.0.1-WS-WASProd_WLPArchive-IFPM75505]

All in all, the archive install procedure is a quick and easy way to get started, requires no external installer and provides a lot of flexibility around managing and maintaining WAS Liberty instances so installed. But it does not manage the install and maintenance of the Java platform used by WAS, it does not implicitly calculate any dependencies between iFixes, it doesn't provide automatically back up for files replaced when an iFix is applied and nor does it search Fix Central for available updates. For this sort of managed install capability you do need a discrete installer which brings us on to......

Installation Manager

Installation Manager (IM) is an IBM technology used to install many IBM products, including WAS since WAS V8.0. The WAS Liberty Profile was introduced into all editions of WAS V8.5 and can also be installed using IM. The initial result of installing WAS Liberty through IM is exactly the same as an archive install except that IM also lays down additional metadata that it uses to manage future feature and service updates. Installation Manager itself needs to be downloaded first, for example from the IBM Support Portal and unzipped before it can be used. The thing you download here is imaginatively referred to as the "Installation Manager installer". There was probably an IBM prize for coming up with that name. Once you've unzipped this you have the choice of using this directly as an "install kit" for scripted/silent installs of the target product (e.g WAS) or using it to fully install Installation Manager, including the IM user interface, and then using the result of that to install and service WAS. Details about the differences in these two approaches are in the IM InfoCenter but for the sake of simplicity in this article, I'll assume the latter.

IM installs and services product offerings like WAS from one or more installation repositories, the URLs for which needs to be configured in IM. Each edition of WAS (e.g. WAS Network Deployment edition) actually consists of a set of installable offerings, each with its own installation repository. The repositories themselves are available in a number of different forms depending on the product offering - for example the repository for the WAS ND offering is provided at three locations: on the WAS ND product media, downloadable from IBM Passport Advantage and in an IBM-hosted web-based repository. The complete list of all installable WAS V8.5 offerings (across all editions) can be seen in the WAS InfoCenter. You can also create custom enterprise repositories containing the service levels and iFixes desirable for an enterprise using the Packaging Utility as described here. IM installs WAS Liberty as part of one of the higher-level product offerings that include Liberty. You can see in the WAS InfoCenter that for V8.5, this includes:

• com.ibm.websphere.BASE.v85
• com.ibm.websphere.BASETRIAL.v85
• com.ibm.websphere.EXPRESS.v85
• com.ibm.websphere.EXPRESSTRIAL.v85
• com.ibm.websphere.DEVELOPERS.v85
• com.ibm.websphere.DEVELOPERSILAN.v85
• com.ibm.websphere.zOS.v85
• com.ibm.websphere.ND.v85
• com.ibm.websphere.NDTRIAL.v85

To use IM to install WAS Liberty from a hosted web repository, you start IM and configure it, using the IM File->Preferences->Repositories menu, to point to the repository for the desired offering. The URLs use the following convention:
http://www.ibm.com/software/repositorymanager/
for example
http://www.ibm.com/software/repositorymanager/com.ibm.websphere.BASE.v85

Choosing the IM "Install" option will offer the choice to install, in this example, the base edition of WAS V8.5. As part of the install you choose whether to include the WAS full profile (with various options) and/or the WAS Liberty profile.

Installation Manager install of WAS V8.5 showing WAS full and Liberty profiles

By default, IBM Java 6 is also installed with WAS when using IM (unlike the archive install route). IBM Java 7 can be optionally installed as part of WAS if the following repository ID is added to IM:
http://www.ibm.com/software/repositorymanager/com.ibm.websphere.IBMJAVA.v70

While there is more overhead to installing WAS Liberty by this route rather than the archive install, the benefits of IM become more apparent once you need to apply service to the installed product. When you start IM and choose the "Update" option, IM searches the configured repositories (IBM hosted repositories or custom enterprise repositories) for all fixes applicable to the products installed by IM and presents a list of those that have not yet been applied - unlike the archive install route you don't have to know about these a priori.

Installation Manager simplifies the application of service to installed packages

If there are any pre-requisites to this iFix, IM will resolve these and by default it back up any updates so they can be undone - all capability beyond archive install. IM can only manage installations of WAS Liberty that have been installed by IM. If you use IM to install WAS Liberty, develop and test an application using that install, use the Liberty runtime server package command to create a packaged server with your application resource and server configuration, and then unzip deploy that on a remote host, you will not be able to use IM to apply service directly to the unzip deployed packaged server. For this scenario you can either:

1. use IM to apply service to the original install and then repeating the server package and unzip deploy of the result to the remote host
2. use the methods described in the archive install section above to apply an iFix directly to the unzip deployed server on the remote host.

In summary, "unzip" can be good enough for software install. But after a few service updates have been applied, the complexity of managing these manually may start to make a managed installer look more cost effective. Both approaches are valid for WAS Liberty in production - I hope this post helps explain the pros and cons of each.

- Ian.