User Tools

Site Tools


Enterprise Directory Post Mortems

Author Daniel Fisher

July 2012 (ED 3.3)

Replication Issues

  • Replication using changed attributes may be marked as stale when transactions bundle multiple message

NameArbiter Issues

  • Mail server LDAP entry CN used for regex comparison

Google Migration Issues

  • Mixed case aliases failed NA check and could not be copied
  • Preferred address not copied for active GE accounts

April 2011 (ED 3.1)

Web Service Issues

  • Nullpointer bug fix was not merged from trunk

Audit Issues

  • Suppression of passwords broken due to method signature changes

Replication Issues

  • New entity validation caused some Banner replication failure

Scheduler Issues

  • Several schedulers not wired and therefore not running

June 2010 (ED 3.0 - 3.0.1)

DAT Issues

  • JBoss ignored JAAS cache configuration and re-authenticated users at seemingly random times.

Guest Issues

  • Guest scheduler process failed to run due to NullPointerException

PID Issues

  • PID release scheduler process will block if associated e-mail account still exists on the mail server

Carilion Issues

  • E-mail aliases not in the PID namespace
  • Shibboleth configuration required testing in prod

Service Issues

  • Service expiration was configured to use the group expiration email

DPID Issues

  • DPID applications are now secured with CAS which caused problems for users with existing CAS sessions.

Replication Issues

  • Address suppression attributes were not being removed.
  • Hornetq netty bridge would hang at random intervals

Logging Issues

  • New logging application did not have enough testing and required some refactoring for production use

November 2009 (ED 2.3)

Google Issues

  • Service URL was incorrectly configured using the '' domain. The correct domain is ''.
  • Google API user was required to unlock via capcha, despite having already agreed to the terms of use.
  • Google mail host was incorrectly configured using ''. The correct host is ';
  • DAT interface for managing google e-mail accounts contained a bug which caused all e-mail management tiles to appear
  • DAT interface for resetting google password was not using the correct action name to allow authorization for all password resets
  • address no longer in service for bounce addresses

Guest Issues

  • Entity validation on the user account was incorrectly configured to 30 characters. The correct userid length is 500 characters.
  • Guest password resets from MyVT were being set with the expire flag.
  • Guest cleanup was not working for guests without a sponsor.

MyVT Issues

  • Password changes made from the portal did not correctly copy the password to the e-mail account.
  • Schema implementation was incorrectly requiring bannerAcademicLevel

LDAP Issues

  • The EDID proxy was incorrectly configured.
  • Peoplesearch contained a bug that prevented the last query template from being used.

Email Issues

  • Welcome messages for new VT e-mail accounts are bouncing if their replication doesn't occur fast enough.

DPID Issues

  • Dpid DSML was not updated to use contactName which caused replication to be rejected by downstream clients.

March 2009 (ED 2.2)

LDAP Issues

  • When entries under ou=accounts with no email address were returned, a double free of the entry would occur, resulting in a SIGABRT

DAT Issues

  • Sponsored people should have their password expired on creation, this is not happening.

(This may have been broken for more than one deployment.)

July 2008 (ED 2.1)

LDAP Issues

  • Service entries now require the serviceDn attribute, which broke existing services until they replicated from the registry

In the future

When adding required attributes, make them optional until all the data has replicated.

DAT Issues

  • People with limited view permissions could not see any data
  • Address editing throws null pointer exception

In the future

SETI Testing should add the limited view permission as one they test.
Address editing should be part of the test cycle.

March 2008 (ED 2.0)

Portal Web Service Issues

  • suppressibleAttributes missing from person DSML
  • mailAccount and mailPreferredAddressCandidates missing from mail DSML
  • removal of labeledURIs and instantMessagingIDs broken for users with more than 1 of this data type
  • Some portlets displaying raw exception message back to users

In the future

All of these problems can be fixed by thorough testing of the portal web service integration.

Pidgen Web Service Issues

  • AxisFaultHelper stopped working because JBoss implementation of SOAP fault code stopped including the exception class name
  • Naming conflict on PidGen.checkPassword() with PasswordChange.checkPassword()

In the future

All of these problems can be fixed by thorough testing of the pidgen web service integration.

Replication Issues

  • Phone numbers for addresses were not formatted properly causing them to be removed from addresses in the ou=addresses branch
  • employee-confidential value missing from suppressedAttributes in person DSML
  • DPID email account DSML not updated for the new mail replication XSL

In the future

ED Directory testing should include looking at changes in the ou=addresses branch.
Replication should be verified for employees who are marked confidential.
Since we do not have a test mail server LDAP, email replication problems are very hard to avoid.

DAT Issues

  • Responsible UID field displayed for non-sponsored people
  • Wildcard search times are slow

In the future

Minor field changes are very hard to catch, not much can be done to prevent this.
SETI Testing should focus some time on looking at wildcard searches are performance.

General Improvements

  • IRM should do a RPPR refresh before pre-production deployments to aid in testing.
  • SETI Testing should include Rhonda Randel in DAT testing.
  • Before testing begins, a pre-test meeting should be held to walk the testing group through interface changes.
    • One or more meetings should follow during the test cycle to discuss questions
middleware/deploy/ed/post-mortem.txt · Last modified: 2015/06/01 12:02 (external edit)