User Tools

Site Tools


middleware:deploy:ed:j2ee

Enterprise Directory J2EE Environment

Introduction

This document describes the process for installing the J2EE environments used by the Enterprise Directory project.
The J2EE environments consists of 5 separate machines: 2 clustered web application servers, 2 clustered EJB servers, and 1 messaging server.
The instructions for each machine are almost identical, however I will point out the differences as needed.

Subversion

A subversion client is provided by the systems administration group and should be available on your path.

Shell

Middleware uses BASH for its shell environment.

  1. Checkout the bashrc script:
    svn co https://svn.middleware.vt.edu/svn/ed/bash-conf $HOME/bash-conf
  2. Symlink a .bash_profile script from your home directory:
    ln -s $HOME/bash-conf/bash_profile $HOME/.bash_profile
  3. Symlink a .bashrc script from your home directory:
    ln -s $HOME/bash-conf/bashrc $HOME/.bashrc
  4. Symlink the .subversion directory from your home directory:
    ln -s $HOME/bash-conf/subversion $HOME/.subversion
  5. Symlink the .vimrc directory from your home directory:
    ln -s $HOME/bash-conf/vimrc $HOME/.vimrc

Logout and log back in to confirm your environment is configured correctly.

Scripts

Various scripts are kept in subversion and must be checked out or updated before deployments.

  1. Checkout the sys scripts:
    svn co https://svn.middleware.vt.edu/svn/ed/sys-scripts /apps/local/bin

Java

The J2EE container, Ant, and Maven all depend on a working JVM.

  1. Checkout the JDK:
    svn checkout https://svn.middleware.vt.edu/svn/ed/jdk-conf/jdk1.6.0_24-64 /apps/local/jdk1.6.0_24-64
  2. Create a symlink for this directory:
    ln -s /apps/local/jdk1.6.0_24-64 /apps/local/jdk

Java Changes

included in the subversion checkout

  • Added BouncyCastle provider jar to jre/lib/ext
  • Reordered security providers in jre/lib/security/java.security
    • security.provider.1=sun.security.provider.Sun
    • security.provider.2=com.sun.crypto.provider.SunJCE
    • security.provider.3=sun.security.jgss.SunProvider
    • security.provider.4=com.sun.security.sasl.Provider
    • security.provider.5=org.bouncycastle.jce.provider.BouncyCastleProvider
    • security.provider.6=sun.security.rsa.SunRsaSign
    • security.provider.6=com.sun.net.ssl.internal.ssl.Provider
    • security.provider.8=org.jcp.xml.dsig.internal.dom.XMLDSigRI
    • security.provider.9=sun.security.smartcardio.SunPCSC
  • Added Unlimited Strength Policy Files in jre/lib/security
  • Set networkaddress.cache.ttl=14400 in jre/lib/security/java.security
  • Added vt-root, vt-server, and vt-middleware certificates to jre/lib/security/cacerts

Ant

The jboss-conf project depends on Ant to build.

  1. Checkout Ant:
    svn co https://svn.middleware.vt.edu/svn/ed/ant-conf/apache-ant-1.7.1 /apps/local/apache-ant-1.7.1
  2. Create a symlink for this directory:
    ln -s /apps/local/apache-ant-1.7.1 /apps/local/ant

Maven

The j2ee project depends on Maven to build.

  1. Checkout Maven:
    svn co https://svn.middleware.vt.edu/svn/ed/mvn-conf/apache-maven-3.0.3 /apps/local/apache-maven-3.0.3
  2. Create a symlink for this directory:
    ln -s /apps/local/apache-maven-3.0.0 /apps/local/maven
  3. Edit $HOME/.m2/settings.xml and provide the following profile configuration:
    <settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
                          http://maven.apache.org/xsd/settings-1.0.0.xsd">
      <activeProfiles>
        <!-- where ENV is one of dev, pprd, or prod -->
        <activeProfile>edu.vt.middleware.ed.env-ENV</activeProfile>
        <!-- where TYPE is either apps or msg -->
        <activeProfile>edu.vt.middleware.ed.TYPE</activeProfile>
      </activeProfiles>
      <profiles>
        <profile>
          <id>edu.vt.middleware.ed.env-ENV</id>
          <properties>
            <env>ENV</env>
            <skipTests>true</skipTests>
          </properties>
        </profile>
        <profile>
          <id>edu.vt.middleware.ed.TYPE</id>
        </profile>
      </profiles>
    </settings>

Truecrypt

Truecrypt (TC) is now a dependency of JBoss since sensitive information needed to deploy JBoss is stored in a TC volume. In particular the TC volume is mounted by the JBoss startup script, jbctl, prior to starting JBoss and then unmounted once the container has successfully started. This limits the access to sensitive data on the filesystem to the time frame of container startup. Note that jbctl is now an interactive startup script since the TC mount password is required to successfully mount the volume.

Download and install TC from http://www.truecrypt.org/downloads. The text-based CLI version is all that is needed, but the GUI version may be more convenient for development workstations.

In order to create a TC volume, it is necessary to choose locations for the encryption container volume file and mount point. The following are used:

touch ~/.tcvolume
mkdir ~/private

It is important that both the TrueCrypt volume and mount location are readable only by the application user. The following permission changes ensure proper security:

chmod 600 ~/.tcvolume
chmod 700 ~/private

In addition to paths, use of a key file is strongly recommended to add entropy to the passphrase used to generate the encryption key. The key file should be a small file containing 1M or less of random data. The following command is recommended for creating a key file with good randomness:

dd if=/dev/urandom of=~/private/tckeyfile bs=1k count=512
chmod 400 ~/private/tckeyfile

Create the TC volume using the following command.

truecrypt -t -c ~/.tcvolume ~/private

Note that this command starts an interactive setup process. The following values are recommended for volume creation options:

  • Volume type: Normal
  • Volume size: 100M
  • Encryption algorithm: AES
  • Hash algorithm: RIPEMD-160
  • Filesystem: Linux Ext3
  • Password: <easily-typed, strong password of 20 characters or more>
  • Keyfile path: /apps/mw/private/tckeyfile

Record the path to the mount point for use in JBoss setup below. It's important to ensure that unprivileged users can mount and unmount the volume to accommodate the assumptions of the JBoss control script. On platforms that support sudoers, this is accomplished by adding an entry like the following:

middleware ALL = NOPASSWD: /usr/bin/truecrypt --core-service

Replace “middleware” with the user account to which you wish to grant this privilege. Note that this must be done by a system administrator.

It's also important to note that the fuse kernel module is required to mount volumes on Linux. Ensure this module is available and loaded prior to mounting. (It should be loaded or load on demand for most modern Linux flavors.)

Now mount the volume as a normal user:

truecrypt -t -k ~/private/tckeyfile --protect-hidden=no ~/.tcvolume ~/private

Now unmount the volume:

truecrypt -t -d ~/.tcvolume

JBoss

JBoss is the J2EE container that middleware uses.

Installing

  1. Checkout JBoss:
    svn checkout https://svn.middleware.vt.edu/svn/ed/jboss-conf/5.1.0 /apps/src/jboss-conf/5.1.0
  2. Change your current directory to the newly created JBoss directory:
    cd /apps/src/jboss-conf/5.1.0
  3. Mount the TC private directory and copy the following files from a known good location, set 600 permission, then unmount:
    • Servlet container keystore (e.g. apps-dev.keystore for Apps node; only needed for nodes that provide Web services)
    • j2ee.properties
    • login-config.xml
  4. Build a version of JBoss:
    • For apps machines:
      • DEV:
        ant -Dtype=apps-1 -Denv=dev install
      • PPRD:
        ant -Dtype=apps-1 -Denv=pprd install
      • PROD:
        ant -Dtype=apps-1 -Denv=prod install
    • For msg machines:
      • DEV:
        ant -Dtype=msg -Denv=dev install
      • PPRD:
        ant -Dtype=msg -Denv=pprd install
      • PROD:
        ant -Dtype=msg -Denv=prod install
  5. Create a symlink for this directory:
    ln -s /apps/local/jboss-5.1.0 /apps/local/jboss
Local Workstation Considerations

For a local workstation with hostname myserver:

  • TrueCrypt volume contents
    • The j2ee.properties and login-config.xml files should be versions that reference the crash and burn database. This information can be obtained from the TrueCrypt volume on apps-dev-1.middleware.vt.edu but still must be placed in files named j2ee.properties and login-config.xml on the local workstation.
    • The keystore file is one generated for the local workstation.
  • ant build
    • A customized version of the build properties file is used with the name myserver.dev.build.properties.
    • The designation of the keystore file is one of the properties to update.
    • The build command becomes
      ant -Dtype=myserver -Denv=dev install

Configuration

The following files need to be placed in /apps/mw/private in the TrueCrypt volume before JBoss can be started:

  • login-config.xml
  • j2ee.properties
  • any keystores needed for HTTPS
Local Workstation Considerations

Update the customized properties file myserver.dev.build.properties to set the values for TrueCrypt (including volume, mount point and keyfiles) as needed to reflect local paths.

Starting

JBoss can be started by executing the jbctl script (jboss.sh is symlinked to it to facilitate transition), which should be in your $PATH:

$ jbctl
USAGE: jbctl status|start|stop|restart|kill

This script interactively mounts the private TC volume, so it no longer supports automated container startup or batch processing. The tcmp and tcup helper scripts are called by jbctl to mount and unmount, respectively, the TC volume. They should also be in your $PATH for convenience.

Apps

All the entity beans, session beans, and web applications are deployed in a single ear.
All svn commands require a path to the appropriate version, this is denoted as VERSION in these instructions.

  1. Checkout the j2ee project:
    svn co https://svn.middleware.vt.edu/svn/ed/j2ee/tags/j2ee-VERSION /apps/src/j2ee/tags/j2ee-VERSION
  2. Change your current directory to the newly created j2ee directory:
    cd /apps/src/j2ee/tags/j2ee-VERSION
  3. Build the j2ee ear:
    mvn install
  4. Deploy the j2ee ear, where ENV is one of dev|pprd|prod:
    j2eectl deploy apps ENV
  • Note that the deploy only needs to occur on one of the apps machines, the ear will automatically be deployed on the other machines by the j2eectl script. The j2eectl script will handle placing each machine into the load balancer pool (if necessary) after deploying the ear to that machine.
Local Workstation Considerations

Before executing mvn install some changes to the pom.xml files in the j2ee project are needed.

  • Update to use the crash and burn database. This setting is in the root pom.xml
    <persistence.url>jdbc:oracle:thin:@or-dvlp-1.db.vt.edu:1251:MIDW</persistence.url>
  • Adjustments to the minimum and maximum database pool sizes can be made (if needed) via properties in several pom.xml files:
    • ears/apps/pom.xml
    • ears/msg/pom.xml
    • tests/ears/container-test/pom.xml

Deploying is simply a copy of the j2ee ear to the JBoss deploy directory followed by a (re)start of JBoss.

Msg

All replication services are deployed in a single ear.
All svn commands require a path to the appropriate version, this is denoted as VERSION in these instructions.

  1. Checkout the j2ee project:
    svn co https://svn.middleware.vt.edu/svn/ed/j2ee/tags/j2ee-VERSION /apps/src/j2ee/tags/j2ee-VERSION
  2. Change your current directory to the newly created j2ee directory:
    cd /apps/src/j2ee/tags/j2ee-VERSION
  3. Some replication services require a keystore to perform updates. The following is the current list of required keystores that should be placed in the properties directory:
    • edid.keystore
  4. All projects retrieve their configuration data from properties.
    A different decryption key is required to access the data for dev, pprd, and prod .
    Copy the appropriate password.key to the META-INF directory:
    cp password.key /apps/src/j2ee/tags/j2ee-VERSION/src/main/resources/META-INF
  5. Build the j2ee ear:
    mvn install
  6. Deploy the j2ee ear:
    deploy-msg.sh
middleware/deploy/ed/j2ee.txt · Last modified: 2015/06/01 12:02 (external edit)