Getting started with Siebel ADM

Hello again,

after quite a while has passed since my last post, I'd like to go a step further. Judging from the number of site visits and a couple of PMs, there seems to be high interest in what we've achieved so far.

Unfortunately, since I'm not only in charge of Siebel deployments, but also of our Siebel project, and projects sometimes do not go too well, I had to re-prioritize my activities. My deep apologies to everyone who asked for information and got neglected promises only.

I'll at least try to deliver information in smaller chunks, thus being able to post more frequently.

Setting up Siebel ADM Framework

In our installation, Siebel Application Deployment Manager proved a valuable tool for automating deployments. If you've heard rumours about ADM framework being buggy or insufficient, chances are that you've heard them a long time ago.

In our experience, Siebel ADM framework does its job reasonably well, once set up properly, an can readily be extended if you've got standard Siebel expertise around.

A drawback of Siebel ADM framework is that you find only few experts worldwide who come with ready-to-use Siebel ADM framework know-how, so on many occasions you will have to proceed hands-on with Oracle's bookshelf.

Long story short: do not hesitate to use Siebel ADM framework, at least as of v8.1 - most of it requires only standard Siebel expertise, even though some of the pre-defined BCs might not be commonly known to all Siebel experts around.

So, in order to step forward towards Siebel Continuous Integration, the first thing to do is to set up a Siebel ADM framework environment. The best source of doing that is the Siebel Application Deployment Manager Guide, which can be found in Oracle's bookshelf here.

You should make yourself familiar with it - in my point of view, there is no way around this; you must know about it, otherwise it will not help you. So, if you are working in an SCM team which have not been in contact with Siebel before, you should make sure you get at least one guy with Siebel expertise for setting this up and getting it running.

Siebel ADM framework in general

The high-level architecure of Siebel ADM is shown in a nice chart in Oracle's bookshelf, reproduced here:

As you can see, you have four main components:

• a source environment, e.g. the staging environment where you do your development work
• one or many target environments, i.e. the environment(s) you want to deploy to
• an orchestration environment, i.e. Siebel ADM's core running the management server, and
• a client environment, merely a couple of batch scripts for controlling what the manager server should do

As far as I know, there is no GUI to the orchestration environment and the management server, but there is one for a couple of ADM artifacts in Siebel's standard 360-degree view. Although this might feel like a disadvantage, it is actually an advantage when you think of embedding all of this into an existing automation infrastructure, which is obviously what we want to achieve with our CI system.

The basic idea of deploying some customization with the ADM framework is as follows:

• develop something in your source environment
• extract artifacts from the source environment, optionally committing them to a versioning system
• create and validate a deployable package with the ADM package utility (one of the tools in the client environment)
• use ADM command-line interface to control ADM deployments through the management server

There are two important things to know about these steps:

• The input to the package utility basically consists of text-based files in a pre-defined directory structure. This lends itself extremely well to versioning, so what we did is to put all ADM-deployable artifacts into this directory structure and commit this to our versioning system. We then use the versioning system as input channel for the package utility, thus creating an ADM package from a particular version of these file. More about versioning in one of the subsequent posts, though.
• A given ADM package can be deployed to more than one environment, with the management server keeping track of what package was deployed where. With the help of management server, you should even be able to roll back to a previously deployed ADM package, a feature we have not tested thoroughly yet.

When you want to take advantage of the latter item, it is very important to make sure that each of your Siebel staging environments has an enterprise name different from the others, because this is the ID the management server uses for controlling where to deploy a package. If you cannot ensure this, you need an own orchestration environment per staging environment, which is obviously not what you really want.

However, before we started to use ADM framework, our Siebel systems where simply created by cloning virtual systems. This meant that the enterprise names of the environments where equal to each other, preventing us from using ADM framework the way it is meant to be used. We figured out, however, how to rename the enterprise of a Siebel environment without having to re-install it. Drop me a note or a comment if I should blog about this, too.

So far for the basics of ADM framework. If you are thinking about Siebel CI, to me it is a must to use it because it is the officially supported way to automate Siebel deployments.

Although there are many details to explain here, I think next time I'll blog about putting Siebel and Siebel ADM artifacts under version control, and why we stopped using the standard recommended script of integrating Siebel Tools and our versioning system Subversion.

Comments are welcome, as usual, as are PMs. Have fun.

CI Blueprint

In today's blog post, let me first of all summarize the main approach we took for all of the software we produce in our development department. I'm not talking about Siebel only, but we develop on a variety of platforms with a variety of programming languages and paradigms - in this blog I'll call this a "platform".

In our SCM team, there was growing demand to standardize development and deployment procedures as much as possible and to be able to monitor and report progress independently of the respective platform. Since we've got a lot of database-like platforms (PL/SQL-based, or also Siebel) where the code cannot simply be compiled and packaged to some kind of executable, but instead needs a server environment to run and interact with its sister applications, we set up a couple of staging environments

• a development environment (for none-integrated development with stubs and mocks)
• an integration environment (for integrative tests of the development team)
• a QA environment (for QA)
• a production environment (obviously enough)

In order to change a piece of software, a developer needs a ticket in our ticket system. This is mainly for being able to track software changes in the sense of an Application Lifecycle Management, but also required of our audit department.  (We even link these implementation tickets to our Scrum artifacts, but that's a totally different story.) As a ticket system, we use Atlassian's Jira - others were suitable, as well.

As a versioning system, we use Subversion - again, other systems were suitable, as well. Our main goal is to be able to say that Subversion is the only valid source of deployments to any environment except for the development environment. Our auditors would love that.

As a build system, we usually use Maven, even for none-Java systems. Systems which do not lend themselves well to being built with Maven, are still built with it, but of course without the standard Maven voodoo, dependency management, and so on. In its simplest case, Maven is used to call batch scripts which actually do the work. Nevertheless, we require all development teams to provide a pom.xml file containing instructions resp. configuration on how to build a piece of software. You could use any other build system, as well - if you are interested in building and deploying Siebel only, forget about Maven and stick to batch.

Build automation is done by a Hudson system. The SCM team uses Hudson for almost any automated task, not only for building software, but also for deploying it, as well as for smoke test, housekeeping, preparing, or clean-up tasks.

The standard way of implementing a software change is depicted in the overview workflow chart below.

Fig. 1: Development Workflow

This chart does not include the paths necessary for dealing with failed tests, but let's stick to this optimistic view for the time being.

As you can see, we have three teams in charge of a software change

• development team
• SCM team
• QA team

The development team is responsible for delivering a working, integrated solution. The SCM team is in charge of deploying the deliverables to the respective environment, an the QA team is to make sure the quality criteria are met.

The so-called "builds" (DEV, QA, and RELEASE builds) are artifacts we produce in Hudson. They contain ready-to-deploy, comprehensive packages of the respective piece of software and differ from each other in currentness, frequency of generation, and quality.

The two yellow action state bubbles in the chart depict automatic activities (almost) without manual interaction. The DEV build is usually automatically produced on commit of an artifact to Subversion. If the DEV build ends successfully, it is immediately deployed to the integration environment. This standard approach does not lend itself well to systems which require a restart of the entire system; in these cases (including Siebel), DEV builds are produced and deployed during nights.

With a DEV build deployed, the developers can test if their changes work in an integrated environment. If so, they signal the software change as "done" in the ticket system.

QA builds are produced on demand of the QA team based on the tickets in "done" state. This means that

• only sources being quality-controlled by developers on an integrated environment are passed to QA, and
• QA builds are produced less frequently than DEV builds.

Only when all tickets associated with a QA build are in "tested" state, they are ready for being included in the next RELEASE build. We produce RELEASE builds on the occasion that

• a release is about to be rolled-out (six times a year), or
• production fixes are to be rolled-out (once or twice a month).

We differentiate between these two development streams starting with the versioning systems (separate branches) and continuing with the environments (one set of dev, int, and QA environments per branch).

This post is not much about Siebel CI, but it's about our development blueprint for all of our platforms. Not all build and deployment jobs obey to this blueprint, and especially Siebel imposes some restrictions to it. Nevertheless, we got quite far down the road, even with Siebel. 

An yes, it is possible and it does make sense to commit SIFs to Subversion ;-).

Stay tuned for more.

Blog Introduction

In our current project, the SCM team encountered several issues concerning packaging and deployment of Siebel customizations.

During the past year, we managed to automate Siebel packaging and deployment with the help of Oracle's Application Deployment Manager (ADM) tool, PLUS we integrated it with our Continuous Integrations toolchain we had already set up when we started, consisting of

• Jira
• Subversion
• Maven
• Hudson

This blog is all about our experiences with setting up a CI toolchain with and for Siebel, which helps a lot with improving deployment quality, manageability, and monitorability in the sense of an Application Lifecycle Management (ALM) approach.

Stay tuned for more on this.