389 libs of glue – aka how to get rid of resurrecting entries

A directory server topology could be quite complex and it could happen that deleted entries are resurrected due to replication conflicts.

Those entries are marked with:
– the “glue” and “extensibleObject” objectclass
– the “nsds5ReplConflict” attribute

You may need to filter those entries, and the suggested way is to use an ACI, as suggested in the official documentation.

r(h)e(v)set password

RHEV uses free-ipa services to authenticate users to its portal. If password expired, the administrator can only set an one-time-password via the IPA portal eg. https://freeipa.rhevlab.local/ipa/ui/ and the user must change it *before* logging in.

The user – eg. u01@rhevlab.local – needs only to
#kinit u01;

The kinit executable reads kerberos configuration from /etc/krb5.conf and sets the default realm (eg. rhevlab.local) thus associating the user with the ldap entry

default_realm = RHEVLAB.LOCAL
dns_lookup_realm = false
dns_lookup_kdc = false
rdns = false
ticket_lifetime = 24h
forwardable = yes

You can otherwise specify the UPPERCASE domain in your kinit request
#kinit u01@RHEVLAB.LOCAL

While all your *lowercase* requests are doomed to fail ;
#kinit u01@rhevlab.local
kinit: Cannot find KDC for requested realm while getting initial credentials

certutil cheatsheet for 389 Directory Server

389 certificates – like google crome – are managed via nss-util, a crypto suite which includes the following tools:
– certutil for managing certificates
– pk12util for managing PKCS12 certificate bundles

NSS store certificates in a directory containing the following files:
* cert8.db, secmod.db, key3.db
and an optional file with the keystore password to be used by 389
* pin.txt

To shorten paths we replace
# certutil -d /path/to/pki/
# certutil -d.

We can list certs with
# certutil -d. -L

Exporting and re-importing a CA Certificate

Export our certificate in txt
# certutil -d. -L -n babel.it -a > babel.it.asc

Import our certificate (eg. on another server).
# certutil -d. -L -n babel.it -t "CT,," -a -i babel.it.asc

Export SSL certificate+key

PKCS12 is a container format for bundling certs and keys. It uses an encrypted format.

Exporting a key requires two passwords:
1- the keystore password
2- the key-certificate password

certutil DOES NOT allow to store both cert and key. We must use pk12util both for
* exporting the cert+key from a keystore
* importing the cert+key in a new keystore
# pk12util -d. -L -n foo.babel.it -i foo.babel.it.p12

NSS can’t retrieve keys in PEM format, so we can use openssl to do it and strip off the encryption part used by p12 to protect the bundle. NOTE: certutil doesn’t allow to IMPORT a non-p12 key but we may want to export and reimport to change eg. the certificate name.

# openssl pkcs12 -in foo.babel.it.p12 -nodes -clcerts -out bar.babel.it.pem
# openssl pkcs12 -export -in bar.babel.it.pem -inkey bar.babel.it.pem -nodes -name bar.babel.it -out bar.babel.it.p12

Import SSL keys under a new 389 instance

Importing a new key+cert under a keystore which is not password-protected will lead to the following error – that arises when we try to list private keys

# certutil -d. -K
certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services "
certutil: could not authenticate to token NSS Certificate DB.: An I/O error occurred during security authorization.

This means that our keystore is not password-protected and cannot store certificate keys. We can fix with
# certutil -d. -W

Now that our keystore is fine, we can just add the keypair.
NOTE: IMPORT DOES NOT SUPPORT NAMING. You must change the certificate name with openssl *before* importing
# pk12util -d. -i bar.babel.it.p12

Verify your keystore
Check that we have both keys and certs

# certutil -d. -L
# certutil -d. -K

python dsadmin module: 389 made easy

Rich Megginson, the 389 Keymaster, created this wonderful python library https://github.com/richm/scripts

To create a new root suffix just

credential = {
'host': 'localhost,
'port': 10389,
'cn=directory manager',
'bindpw': 'secret'

Connect to an already-existing instance passing our dict variable as function arguments
conn = DA(**credential)

Create a backend database
backend_name = conn.setupBackend('o=addressbook1')

Bind it to an entry
conn.setupSuffix(suffix='o=addressbook1', bename=backend_name)

And create the entry

e = Entry('o=addressbook1')
e.setValue('objectclass', ['top','organization'])

Easy, isn’t it?

Nobody backups 389 Directory Server!

389 Directory Server  (like Sun/Oracle DS) has a nice tool for taking logical backups: db2ldif. Logical backups are – simply – backups taken in an exportable format – like LDIF or SQL. They differ from raw backups – which are taken copying or snapshotting data files.

db2ldif works triggering a backup in 389: it simply tells to slapd – the ldap daemon – to export a backup on a given file

As 389DS usually runs as user “nobody”, there’s a common error you’ll se when running it:

# db2ldif -s "dc=babel,dc=it" -U -a /backup-ldap/myds.$(date -I).ldif
db2ldif: can't open /backup-ldap/myds.2012-06-14.ldif: 13 (Permission denied)

Actually it’s not the command you’re running that lacks permissions, but the user “nobody”.

# chown -R nobody /backup-ldap/
And everything will work fine!

Apple Calendar Server and 389org Directory Server


Welcome to a short guide about binding Apple Calendar Server (ACS) to a 389 Directory Server. Aim of this guide is to point a couple of common pittfalls.


  1. install ACS with default authentication. Remember:
    1. enable user_xattr on your directory
  2. install 389org

Download ldapdirectory.py – the backend written for OpenLdap from http://trac.calendarserver.org/attachment/ticket/260/ldapdirectory.patch

Modify ldapdirectory.py to reflect your attributes (eg I use “mail” instead of “uid”, “nsUniqueId” instead of “entryUUID” ).

I suggest to add some more log and error lines to ldapdirectory to speed up debugging.

Modify your configuration file like http://trac.calendarserver.org/attachment/ticket/260/caldavd-sample.plist

Remember to fix it with your parameter and to DISABLE all authentication types but BASIC-AUTH.

This is because ACS uses MD5-DIGEST by default, making impossible to forward the password to the Ldap Server (see http://en.wikipedia.org/wiki/Digest_access_authentication)

Creating root suffixes with 389org

Creating a simple root suffix with 389org consist in 3 passages:

  1. create a backend database where to save our data;
  2. create a root suffix pointing to the new db;
  3. create the ldap entry associated to the root suffix.

These steps are executed by the following ldif.

# create an LDBM database named AddressbookDb
# eventually setting cache to 2G
dn: cn=AddressbookDb,cn=ldbm database,cn=plugins,cn=config
objectclass: extensibleObject
objectclass: nsBackendInstance
nsslapd-cachememsize: 2048576000
nsslapd-suffix: o=db8

# associate the database with an ldap entry
dn: cn="o=db8",cn=mapping tree,cn=config
objectclass: top
objectclass: extensibleObject
objectclass: nsMappingTree
nsslapd-state: backend
nsslapd-backend: Addressbookdb8
cn: "o=db8"

# create the ldap entry
dn: o=db8
objectclass: top
objectclass: organization
o: db8

now you can issue a successful
# ldapsearch -x -b “o=db8” -s base -LLL

You can even create a local database which refers to other servers
for update

# replace state: backend -> state: referral on update
# and point to your referrals
dn: cn="o=db7",cn=mapping tree,cn=config
objectclass: top
objectclass: extensibleObject
objectclass: nsMappingTree
nsslapd-state: referral on update
nsslapd-backend: Addressbookdb7
nsslapd-referral: ldap://ab1.example.com:389/o%3Ddb7
nsslapd-referral: ldap://ab2.example.com:389/o%3Ddb7

389org on Sabayon

Babel sponsored a Fedora DS release for Sabayon/Gentoo. The work has been done by the Sabayon maintainer lxnay with my support: we involved richm aka Mr. Fedora DS.

This work lead us to discover some issues on the server:

1. the AdminServer – using mod_cgi – plays the dup2/close game to close stdfd: this caused an error on Sabayon but not on Fedora.

  • We discovered that, even if Admin Server requires Apache2 mpm  (multithread), the Fedora  mod_cgi was build using prefork (single-threaded)
  • On Sabayon, like Gentoo,  you have to compile from scratch all packages, so you’re not supposed to mix prefork/mpm, and the mod_cgi was compiled in a multithread environment.
  • obviousily the dup2/close game won’t fit on multithread environment, because will close the fds of the main process (and not the child’s one)
  • further info on http://www.spinics.net/lists/fedora-directory/msg11697.html
2. the 389org schema files dropped the ldap aliases: we filed a bug and now they’re recovering them.

Shadows on FedoraDS

No bad news, don’t worry! Just a post on migrating password from /etc/shadow to Fedora Directory Server – aka port389.org

In /etc/shadow you’ll find


Just ldapmodify and set on

dn: uid=jon,dc=babel,dc=it
userPassword: {CRYPT}$6$tJwPWaez$02dXHTj/olD9xG3kDZ9gG1fkdlGC3k0l

and bind

# ldapsearch -x -b "dc=babel,dc=it" "uid=jon" \ 
-D"uid=jon,dc=babel,dc=it" -wguessme

Ldap Server Side Sorting

Walking thru the net I found this nice hint…

There is a way to sort the ldap query results. 389 (and some other
LDAP servers) have a functionality called server-side sorting.

You can set a special LDAP control while making your ldap query.
This control has a certain number of parameters, for example, which attributes
should be used for sorting and in which order you want your results to
be sorted.
Be careful with the performance implications, be sure that
your search result is small enough.

Some more details are available here :