Welcome • Services • Getting Started • Support and Tools • Documentation  
 
 

CalNetAD Directory Integration: Technical Design

Updated: 05/08/2003

Summary

We decided to implement a phased integration of the data in our campus CalNet Directory (which is based on a Sun/iPlanet LDAPv3 directory server) with our Active Directory (AD). We describe the design and coding for stage one in this document. Basic user account information is synchronized between the two directories as detailed in the table of data elements below. Our implementation is based on the IBM Directory Integrator (IBMDI) engine and custom ADSI scripting using primarily JScript code. To minimize the data to be modified in the AD, we create a local repository for the directory data using the IBMDI "Delta mechanism" feature to help manage the synchronization process.

The synchronization process is scheduled to run every night with Active Directory. This means that any CalNet account changes, including CalNet ID changes, will be automatically synchronized overnight and available the next working day. If this overnight processing delay is acceptable, then nothing else needs to be done. However, if there is a need to immediately use the self-selected ID before the automatic nightly process, an OU administrator can make a manual synchronization of the user attributes in Active Directory. This manual synchronization procedure is available in the document Dealing with CalNet ID Changes.

Data elements

The following data elements are being synchronized between our campus Directory and the AD:

User attribute: GUI label: Unique within: Initial values (CNID=CalNetID):
altSecurityIdentities Kerberos Principal Name Forest Kerberos:CNID@BERKELEY.EDU
cn Full name OU CNID (when first created)
displayName Display name [not unique] "displayname" from CalNet directory (CNID for students)
sAMAccountName User logon name (pre-Win2K) Domain CNID
userPrincipalName User logon name Forest CNID@BERKELEY.EDU
uid [none] Forest CalNet directory uid (CPDID)

 

CNDirPub2: data from the iPlanet directory

To begin the integration process, a complete LDIF dump of the iPlanet directory is performed daily. This LDIF file is processed using an IBMDI AssemblyLine, CNDirPub2AL, configured with a Delta B-tree database to record all entries into a local repository. The AssemblyLine (AL) consists of four connectors: input, save_add_mod, save_del, and send. The latter connector is a passive-mode connector invoked by custom code in the epilog of CNDirPub2AL to send securely a file of transactions created in the epilog from those saved by the save_* connectors. This file, in the "LDIF" data format is sent to a remote instance of IBMDI where the final processing and AD updates occur. See the Appendix for a detailed description of CNDirPub2AL and its connectors. This system of LDIF data, the B-tree repository, and IBMDI ALs accessible via HTTP is termed the CNDirPub2, and is designed to support multiple data consumers of which the CalNetAD service is the first working example.

 

CNDirPub2: SSL configuration

The Certificate Authority (CA) certificate for a locally run Microsoft Certificate Authority was transferred to the Sun platform. The IBM Certificate Manager was used to import the CA certificate into the jssecacert keystore.

 

ADCon2: using an ADSI-based script connector to AD

To process the daily updates from the CNDirPub2 system, an IBMDI EventHandler, ADCon2EH, writes the data received to a file and invokes an AL, ADCon2AL, which uses a file connector, input, to read the saved data, and a custom script connector, ADSIScriptConnector, to make add, delete, and modify transactions into the AD via the add_user, del_user, and mod_user IBMDI connectors. The script connector is implemented as a set of JScript functions which use ADSI interfaces to query and update the AD data to comply with our requirements for data synchronization and security and privacy policies. See the Appendix for a detailed description of ADCon2AL and its custom script connector, ADSIScriptConnector. The system of IBMDI components which receive and process the directory data to keep the AD synchronized with the campus CalNet Directory is called the ADCon2.

 

ADCon2: SSL configuration

The following procedure was used to set-up the SSL configuration for the ADconsumer AL:

1. Generate a key pair and a self-signed certificate. The keytool utility that ships with the J2SE SDK is used to generate a keystore containing a public/private keypair.

     keytool -genkey -keyalg RSA -alias metamerge -keystore <keystore_filename>

2. Generate a Certificate Signing Request (CSR).

     keytool -certreq -alias metamerge -keyalg RSA 
	    -file <csr_filename> -keystore <keystore_filename>
The contents of the CSR was copied into the web-based certificate request form for a locally run Microsoft Certificate Authority. After the CSR was approved, the signed certificate, the CA certificate, and the certificate path (PKCS #7 Certificate) were retrieved from the Microsoft Certificate Authority web interface and saved in local files.

3. Import the certificates. The keytool utility was used to import the certificate path into the keystore created in step 1.

     keytool -import -alias root -trustcacerts 
	    -file <certificate_path_filename> -keystore <keystore_filename>

The Metamerge Certificate Manager was used to import the signed certificate into the jssecacert keystore.

4. Configure the Metamerge ADCon2 AL. Using the Metamerge Administrative Console, the following System Properties were set:

com.architech.certificatesFile: <keystore_filename>
com.architech.certificatesFile.password: <keystore_password>
javax.net.ssl.trustStore: <path>jssecacerts

 

ADval2: checking data integrity

Because data within the AD can be changed by other processes outside of the publisher/consumer cycle, a method to validate against the data source of record was required. To begin the validation process, a complete LDIF dump of selected attributes for all user objects in the AD is created each night using the LDIFDE AD utility. This file is securely transferred to the CNDirPub2 site, then, using a process much like the ADCon2AL, the ADval2AL parses the LDIF file using a Delta B-tree database to record all entries. The resulting B-tree database is substituted each night for the normal B-tree repository before the publisher cycle to generate any corrective transactions needed to bring the AD data back into synchrony with the iPlanet directory.

 

Design considerations

Our initial design for the publisher/consumer system for directory synchronization had implemented an IBMDI process which transferred individual transactions between the publisher IBMDI instance and the consumer one. By contrast, in the current design the entire set of transactions is saved to a data file and this file is transferred in a single operation. During our testing of the initial design we encountered an IBMDI memory-leak error generated during each invocation of the IBMDI AssemblyLine used for transferring individual records. This problem, combined with our lack of a robust messaging infrastructure for carrying these individual transactions to remote consumers, prompted us to move to the current file-oriented, single-transfer design.

 

Policies

The following policies relating to data conversions and security and privacy of data elements have been implemented:

  • All employee (faculty and staff) and affiliate user accounts are by default created within the OU=Users,OU=FSA container
  • All student accounts are created by default within the OU=Users,OU=Students container using the CalNetID as the displayName attribute
  • The cn attribute is not modified automatically following the initial creation unless the account to be changed still resides within the FSA or any Students OU
  • Student accounts are created to have access control entries (ACEs) which prohibit access to the uid attribute for general users
  • If a modification involves moving an account into a Students OU, the ACEs mentioned above are added; if an account is moved out of a Students OU, the ACEs are removed

 

Appendix

Details for the IBMDI ALs, EventHandlers, connectors and Java Service Wrapper as currently implemented are documented in this Appendix. XML-based IBMDI export files for the IBMDI code is available under the following links:

All custom code for the CNDirPub2 and ADval2 systems use the native IBMDI Rhino-based Javascript interpreter. The ADCon2 system uses the IBMDI support for WSH languages and uses the ECMAScript-based JScript language to facilitate access to ADSI COM-based objects.

ADCon2AL

This AL uses a FileSystem connector, input, with an LDIF parser in Iterator mode to read the input data file, DUMPER.LDIF, making the following input mappings between LDAP attributes (on the left side) and work entry attributes:

1. [non-student] displayname --> displayname
2. [student] ucbstuid --> displayname
3. [custom code] --> is_student
4. [custom code] --> ou_default
5. ucbkerberosprincipalstring --> ucbkerberosprincipalstring
6. uid --> uid

Custom code in the after_getnext IBMDI AL hook is used to test incoming entries for validity and to set various flags used in the custom code during the attribute mappings. The work objects are linked into a B-tree database, ADusers.deltaDB, via the Delta process keyed to the uid attribute for entries.

Two FileSystem connectors, save_add_mod and save_del, with Simple parsers in AddOnly mode write the work objects into two data files, SIMPLE_ADD_MOD.TXT and SIMPLE_DEL.TXT, with the following attribute mappings between the work object (on the left side) and the output records:

1. [custom code] --> action
2. ucbkerberosprincipalstring --> calnetid
3. displayname --> displayname
4. is_student --> is_student
5. ou_default --> ou_default
6. uid --> uid

The string value for action is either "add", "modify", or "delete", based on the operation code value returned by the work.getOperation() method. This value is automatically set for each work object during the Delta processing of the input data entries by the input connector.

To control which output file is selected for a given work object, the operation code value also is determined by custom code within the before_add IBMDI AL hook for the save_* connectors. The two output files are later merged within the AL epilog into a single data file, CNDirPubSave.LDIF, which contains all of the "delete" transactions listed first. This technique was chosen to avoid errors encountered when the corrected records for certain types of data errors in the source iPlanet directory were propagated to the AD in the default order generated by the Delta process in which deletes are placed last in the sequence of transactions.

Finally, a passive-mode HTTPClient2 connector, send, is invoked within the AL epilog to send via HTTP POST the CNDirPubSave.LDIF file to a remote HTTPS-based EventHandler. The http.body attribute is mapped using custom code to a java.io.FileInputStream object pointing to the data file to be transferred.

processLDAPchanges

This AL is launched by an HTTPEventHandler, ADCon2EH, upon receipt of a valid data file of transactions to be processed. Custom code in the EventHandler saves the data in the http.body attribute into a file, CNDirPubSave.LDIF and runs the AL. A FileSystem connector, input, with a LDIF parser in Iterator mode then reads the data file and maps the incoming entry attributes (on the left side) into the work entry attributes as follows:

1. [custom code] --> UPN
2. action --> action
3. [custom code] --> altsecID
4. [custom code] --> cn
5. displayname --> displayname
6. is_student --> is_student
7. [custom code] --> key
8. ou_default --> ou_default
9. [custom code] --> samaccount
10. uid --> uid

The work entry then is processed by one of three connectors, add_user, del_user, or mod_user, to perform the transaction within the AD. These three connectors are all based on the custom Script Connector, ADSIScriptConnector, and operate in AddOnly, Delete, and Update modes, respectively. add_user and mod_user both make the following mappings of the work object attributes (on the left side) to the Connection object attributes sent to the custom script connector for processing:

1. UPN --> UPN
2. altsecID --> altsecID
3. cn --> cn
4. displayname --> displayname
5. is_student --> is_student
6. key --> key
7. ou_default --> ou
8. samaccount --> samaccount
9. uid --> uid

To control which action is performed, custom code within the before_add (for add_user) and before_execute (for both del_user and mod_user) IBMDI AL hooks retrieves the value of the work object's action attribute and ignores the entry if it doesn't match the appropriate operation for the connector. For example:

// Ignores entries not for MODIFY operation
if (work.getString("action") == "modify")
{
  task.logmsg("UPDATING:");
  task.dumpEntry(work);
}
else
  system.ignoreEntry();

ADSIScriptConnector

This JScript custom Script Connector implements the required functions for any IBMDI Script Connector:

  • selectEntries() [stub only]
  • getNextEntry() [stub only]
  • putEntry()
  • findEntry()
  • modEntry()
  • deleteEntry()

In addition, several helper functions are implemented to assist in utility, security, privacy, or error control operations:

  • pack(Ary) - converts JScript array to Scripting.Dictionary object
  • checkError(sOperation, sErr, iCount, sAction) - general ADSI and COM error handling
  • randPass(sPWChars, iSize) - creates random password string
  • searchError(sErr, iCnt, sAct) - search error handling
  • removeACE(oUsr, sTrstee, iCnt, sAct) - remove ACE from AD user object ACL
  • restoreACE(oUsr, sTrstee, iCnt, sAct) - restore ACE from AD user object ACL

These latter two functions were implemented to allow privacy policies to be reflected in the access control lists (ACLs) for the uid property of AD user objects.

To access the AD for searching, the ADODB.Conneciton object is used with the ADsDSOObject ADSI provider interface. Standard ADSI interfaces are used for adding, modifying, and deleting user objects. A special End-of-File (EOF) entry is processed by the putEntry() function to signal the end of data input to the parent IBMDI AL process and to close out the custom log file generated by the script connector.

Most of the policies relating to security and privacy are implemented as part of the putEntry() function (setting initial random passwords and initial ACEs), or within the modEntry() function (adding or removing ACEs and setting the value for the cn attribute).

Java Service Wrapper

To support running the IDI process as a Windows service we use the Java Service Wrapper (JSW) from TanukiSoftware.org on our Windows 2000 application server. JSW consists of a small Windows xecutable program (WRAPPER.EXE), a native library (WRAPPER.DLL) and a JAR file (WRAPPER.JAR) with all of the Wrapper Java classes. These components use a text-based configuration file to control the launching and monitoring of the IBM JVM and to run the IDI application classes with the proper environment paths and settings.

This same configuration file is used by JSW for installing and removing the custom IDI service from the Windows 2000 system. Most of the parameters in the JSW configuration file are derived from the batch files that IBM includes for the manual startup of the IDI application. We customized the JSW configuration file and the JSW template batch files to start the JSW system from the root IDI directory and to include support for logging to the Windows event logs as well as to the standard JSW log file. An edited version of the JSW configuration file and the service installation batch file we use are given below.

wrapper-adcon2.conf 

#********************************************************************
# Wrapper parameters
#********************************************************************
# Java Application
wrapper.java.command=[path to IDI]/_jvm/bin/java.exe


# Java Main class
wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp


# Java Classpath (include wrapper.jar)  Add class path elements as
#  needed starting from 1
wrapper.java.classpath.1=libs/wrapper.jar
wrapper.java.classpath.2=jars/activation.jar
wrapper.java.classpath.3=jars/comm.jar
wrapper.java.classpath.4=jars/ibmjndi.jar
wrapper.java.classpath.5=jars/imap.jar
wrapper.java.classpath.6=jars/jaas.jar
wrapper.java.classpath.7=jars/jcert.jar
wrapper.java.classpath.8=jars/jndi.jar
wrapper.java.classpath.9=jars/jnet.jar
wrapper.java.classpath.10=jars/jsse.jar
wrapper.java.classpath.11=jars/ldap.jar
wrapper.java.classpath.12=jars/ldap12.jar
wrapper.java.classpath.13=jars/mail.jar
wrapper.java.classpath.14=jars/mailapi.jar
wrapper.java.classpath.15=jars/ncsow.jar
wrapper.java.classpath.16=jars/pop3.jar
wrapper.java.classpath.17=jars/providerutil.jar
wrapper.java.classpath.18=jars/smtp.jar
wrapper.java.classpath.19=jars/xalan.jar
wrapper.java.classpath.20=jars/xerces.jar
wrapper.java.classpath.21=miloader.jar
wrapper.java.classpath.22=jars/ldapbp.jar
wrapper.java.classpath.23=jars/ibmjsse.jar
wrapper.java.classpath.24=jars/ibmjceprovider.jar
wrapper.java.classpath.25=jars/ibmjcefw.jar
wrapper.java.classpath.26=jars/local_policy.jar
wrapper.java.classpath.27=jars/US_export_policy.jar
wrapper.java.classpath.28=jars/ibmjlog.jar


# Java Library Path (location of Wrapper.DLL or libwrapper.so)
wrapper.java.library.path.1=libs


# Java Additional Parameters
wrapper.java.additional.1="-Duser.dir=[path to IDI]"


# Initial Java Heap Size (in MB)
wrapper.java.initmemory=3


# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=64


# Application parameters.  Add parameters as needed starting from 1
wrapper.app.parameter.1=com.metamerge.loader.miloader
wrapper.app.parameter.2=com.architech.RS
wrapper.app.parameter.3=-c"[path to IDI]/conf/ADCon2/ADCon2.cfg"
wrapper.app.parameter.4=-l"[path to IDI]/conf/ADCon2/logs/ADCon2.log"


# Port which the native wrapper code will attempt to connect to
wrapper.port=[wrapper port]


#********************************************************************
# Wrapper Logging parameters
#********************************************************************
# Format of output for the console.  (See docs for formats)
wrapper.console.format=PM


# Log Level for console output.  (See docs for log levels)
wrapper.console.loglevel=INFO


# Log file to use for wrapper output logging.
wrapper.logfile=logs/wrapper.log


# Format of output for the log file.  (See docs for formats)
wrapper.logfile.format=LPTM


# Log Level for log file output.  (See docs for log levels)
wrapper.logfile.loglevel=INFO


# Maximum size that the log file will be allowed to grow to before
#  the log is rolled. Size is specified in bytes.  The default value
#  of 0, disables log rolling.  May abbreviate with the 'k' (kb) or
#  'm' (mb) suffix.  For example: 10m = 10 megabytes.
wrapper.logfile.maxsize=0


# Maximum number of rolled log files which will be allowed before old
#  files are deleted.  The default value of 0 implies no limit.
wrapper.logfile.maxfiles=0


# Log Level for sys/event log output.  (See docs for log levels)
wrapper.syslog.loglevel=INFO


#********************************************************************
# Wrapper NT Service parameters
#********************************************************************
# WARNING - Do not modify any of these parameters when an application
#  using this configuration file has been installed as a service.
#  Please uninstall the service before modifying this section.  The
#  service can then be reinstalled.


# Name of the service
wrapper.ntservice.name=IDI_ADCon2


# Display name of the service
wrapper.ntservice.displayname=IBM DI ADCon2 Service


# Description of the service
wrapper.ntservice.description=IBM Directory Integrator ADCon2 Service


# Service dependencies.  Add dependencies as needed starting from 1
wrapper.ntservice.dependency.1=


# Mode in which the service is installed.  AUTO_START or DEMAND_START
wrapper.ntservice.starttype=AUTO_START


# Priority at which the service is run.  NORMAL, LOW, HIGH, or
#  REALTIME
wrapper.ntservice.process_priority=NORMAL


# Allow the service to interact with the desktop.
wrapper.ntservice.interactive=false
=======


======= install-diserver.cmd =======
@echo off
rem
rem Find the application home.
rem
if "%OS%"=="Windows_NT" goto nt
echo This is not NT, so please edit this script and set _APP_HOME manually
set _APP_HOME=..
goto conf


:nt
rem %~dp0 is name of current script under NT
set _APP_HOME=%~dp0
rem : operator works similar to make : operator
set _APP_HOME=%_APP_HOME:\cmd\=%



rem
rem Find the wrapper.conf
rem
:conf
set _WRAPPER_CONF="%~f1"
if not %_WRAPPER_CONF%=="" goto startup
set _WRAPPER_CONF="%_APP_HOME%\conf\wrapper-adcon2.conf"



rem
rem Run the application.
rem At runtime, the current directory will be that of Wrapper.exe
rem
:startup
"%_APP_HOME%\Wrapper.exe" -i %_WRAPPER_CONF%
if not errorlevel 1 goto end
pause


:end
set _APP_HOME=
set _WRAPPER_CONF=
 
Contact Us