Double colon in LDIF

[rob fielding]

Hi,

I'm still finding my feet in LDAP and LDIFs but I've been led to think
entries such as these were 'corruptions' :

cisSetting:: Ym0ubmV3QGRl
userPassword:: eXZxZDJkbHk=
dn:: AGNuPWRzdnIwMDAwLG91PUF

Entries like this were originally found in a Netscape Directory LDIF,
which I might add a) is very old, b) deprecated in the organisation c)
completely unfamiliar to me and I do not have direct access to. I'm
working on an OpenLDAP replacement, and only have the LDIF and little
inhouse knowledge to lean on. As the NSD LDIF contains many schema
violations I've written a perl script which 'fixes' many aspects of the
LDIF, including various techniques for resolving the corrupt double
colon entries.

After a successful ldapadd into my provider, verifying my fixed LDIF is
syntactically sound, and successful syncrepl to my consumer (including
resolution of ITS#3546 using OPENLDAP_REL_ENG_2_2 cvs) I decided to have
a look at a slapcat. To my astonishment, I've found many entries such as
quoted above.

Things to note: *all* userPassword entries are double colon entries in
the LDIF - they are infact plaintext at this point, visible in gq. Not
all cisSettings are double colon entries, and those that are render in
plain text in gq. They render as above with ldapsearch. The dn's I
haven't checked out as they are clearly not human readable.

Can someone clarify whether the above is 'normal' or not? I'm begining
to think they are encoded in some way, however why would gq render them
in plaintext, yet slapcat and ldapsearch render them in this way? What
does the double colon signify? Perhaps the NSD LDIF isn't as corrupt as
we thought?

Seen on:
OpenLDAP 2.3.1a w db-4.3.27
OpenLDAP 2.2.23 w db-4.3.27
OpenLDAP OPENLDAP_REL_ENG_2_2 w db-4.3.27

Best regards,

[Dusty Doris]

Post by rob fielding
Hi,
I'm still finding my feet in LDAP and LDIFs but I've been led to think
cisSetting:: Ym0ubmV3QGRl
userPassword:: eXZxZDJkbHk=
dn:: AGNuPWRzdnIwMDAwLG91PUF
Can someone clarify whether the above is 'normal' or not? I'm begining
to think they are encoded in some way, however why would gq render them
in plaintext, yet slapcat and ldapsearch render them in this way? What
does the double colon signify? Perhaps the NSD LDIF isn't as corrupt as
we thought?

This is the base64 encoding of the value. Go to
http://makcoder.sourceforge.net/demo/base64.php to see.
For example,
dn:: AGNuPWRzdnIwMDAwLG91PUF = dn: cn=dsvr0000,ou=A
Is that correct?
Just copy the encoded part, such as AGNuPWRzdnIwMDAwLG91PUF, paste in that
website and hit decode. It will show you.

BTW, this will explain ldif format.

http://www.ietf.org/rfc/rfc2849.txt

[Bjorn Ove Grotan]

Post by rob fielding
Hi,
I'm still finding my feet in LDAP and LDIFs but I've been led to think
cisSetting:: Ym0ubmV3QGRl
userPassword:: eXZxZDJkbHk=
dn:: AGNuPWRzdnIwMDAwLG91PUF

<snip>

Post by rob fielding
Can someone clarify whether the above is 'normal' or not? I'm begining
to think they are encoded in some way, however why would gq render them
in plaintext, yet slapcat and ldapsearch render them in this way? What
does the double colon signify? Perhaps the NSD LDIF isn't as corrupt as
we thought?

Double colon in LDIF like this means that the value is encoded with
base64. Run it through mimedecode or python's base64.decodestring(value)
and you'll see what the real value is. Inside the ldap-database, values
are utf8, and if you use LDAP API, you can use utf8 directly. But if you
use plain old LDIF, you must convert utf8 etc to base64 and state that
the following value is base64 encoded using double colon.

example ->
description:: SGVsbG8gd29ybGQ= which translates to
description: Hello world

This example is wrong usage though, since values with only us-ascii
doesn't have to be base64-encoded.

[Dusty Doris]

Post by rob fielding
Hi,
I'm still finding my feet in LDAP and LDIFs but I've been led to think
cisSetting:: Ym0ubmV3QGRl
userPassword:: eXZxZDJkbHk=
dn:: AGNuPWRzdnIwMDAwLG91PUF
Can someone clarify whether the above is 'normal' or not? I'm begining
to think they are encoded in some way, however why would gq render them
in plaintext, yet slapcat and ldapsearch render them in this way? What
does the double colon signify? Perhaps the NSD LDIF isn't as corrupt as
we thought?

This is the base64 encoding of the value. Go to
http://makcoder.sourceforge.net/demo/base64.php to see.

For example,

dn:: AGNuPWRzdnIwMDAwLG91PUF = dn: cn=dsvr0000,ou=A
cisSetting:: Ym0ubmV3QGRl = cisSetting: ***@de

Is that correct?

Just copy the encoded part, such as AGNuPWRzdnIwMDAwLG91PUF, paste in that
website and hit decode. It will show you.

[Hallvard B Furuseth]

Post by rob fielding
cisSetting:: Ym0ubmV3QGRl
userPassword:: eXZxZDJkbHk=
dn:: AGNuPWRzdnIwMDAwLG91PUF

In LDIF format, this is a valid representation of attribute values.
The '::' means that the following value is base64-encoded in the file.
(It is not so encoded in the directory; the client which reads the
value must base64-decode it before using it.)

If you are parsing LDIF files "by hand", you also need to know that a
line which starts with a space is a continuation of the previous line.
Remove 1 space at the beginning of continuation lines and append them
to the previous line.

LDIF format is described in RFC 2849, which is included in doc/rfc/ in
OpenLDAP.

Post by rob fielding
Things to note: *all* userPassword entries are double colon entries in
the LDIF - they are infact plaintext at this point, visible in gq.

Good. Applications should try to avoid displaying plaintext passwords.
While base64 is easy to "decrypt", at least the password can't be seen
at a glance.

[Adam Tauno Williams]

Post by rob fielding
Can someone clarify whether the above is 'normal' or not? I'm begining
to think they are encoded in some way, however why would gq render them
in plaintext, yet slapcat and ldapsearch render them in this way? What
does the double colon signify? Perhaps the NSD LDIF isn't as corrupt as
we thought?

Yes, this is normal. It indicates the value is base64 (?) encoded.

[Quanah Gibson-Mount]

Post by rob fielding
OpenLDAP 2.3.1a w db-4.3.27
OpenLDAP 2.2.23 w db-4.3.27
OpenLDAP OPENLDAP_REL_ENG_2_2 w db-4.3.27

Since everyone else has answered your initial question, I'll note that the
recommended BDB version to use with OpenLDAP is 4.2.52 + patches. BDB 4.3
has various issues (the requirement in the README of 2.2.23 is wrong).

--Quanah

--
Quanah Gibson-Mount
Principal Software Developer
ITSS/Shared Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html

"These censorship operations against schools and libraries are stronger
than ever in the present religio-political climate. They often focus on
fantasy and sf books, which foster that deadly enemy to bigotry and blind
faith, the imagination." -- Ursula K. Le Guin

[Todd Lyons]

Post by rob fielding
I'm still finding my feet in LDAP and LDIFs but I've been led to think

Others have already mentioned that it is simply base64 encoded. The
easiest way to see what's really in there is a good old commandline

Post by rob fielding
cisSetting:: Ym0ubmV3QGRl
dn:: AGNuPWRzdnIwMDAwLG91PUF

[***@tlyons ~]$ echo "AGNuPWRzdnIwMDAwLG91PUF" | mimencode -u ; echo
Warning: base64 decoder saw premature EOF!
cn=dsvr0000,ou

That actually looks like some kind of error, but not sure where.

Post by rob fielding
Things to note: *all* userPassword entries are double colon entries in
the LDIF - they are infact plaintext at this point, visible in gq. Not

Normal.

Post by rob fielding
Can someone clarify whether the above is 'normal' or not? I'm begining

As others have said, yes.

[Owen DeLong]

Post by rob fielding
Hi,
I'm still finding my feet in LDAP and LDIFs but I've been led to think
cisSetting:: Ym0ubmV3QGRl
userPassword:: eXZxZDJkbHk=
dn:: AGNuPWRzdnIwMDAwLG91PUF

I don't know about NS Dir., but, in OpenLDAP, those represent values
that are already server-encrypted. It's how the server differentiates
between:

userPassword: mypasswordintheclear
userPassword: {md5}anmd5hash
userPassword:: base64alreadyserverencryptedpassword

Post by rob fielding
LDIF, including various techniques for resolving the corrupt double
colon entries.

That might not be such a good thing.

Post by rob fielding
After a successful ldapadd into my provider, verifying my fixed LDIF is
syntactically sound, and successful syncrepl to my consumer (including
resolution of ITS#3546 using OPENLDAP_REL_ENG_2_2 cvs) I decided to have
a look at a slapcat. To my astonishment, I've found many entries such as
quoted above.

Yep.

Post by rob fielding
Things to note: *all* userPassword entries are double colon entries in
the LDIF - they are infact plaintext at this point, visible in gq. Not
all cisSettings are double colon entries, and those that are render in
plain text in gq. They render as above with ldapsearch. The dn's I
haven't checked out as they are clearly not human readable.

That's the opposite of what I would expect and the opposite of the
behavior on my system, although I'm not using the cisSettings attribute,
but the userPassword attribute behaves as I described above.

Post by rob fielding
Can someone clarify whether the above is 'normal' or not? I'm beginning
to think they are encoded in some way, however why would gq render them
in plaintext, yet slapcat and ldapsearch render them in this way? What
does the double colon signify? Perhaps the NSD LDIF isn't as corrupt as
we thought?

I'm not familiar with gq, so, I can't answer that question.

I'm still pretty new to this myself, but, that's been my experience
so far.

Owen

[Mike Jackson]

Post by Owen DeLong
I don't know about NS Dir., but, in OpenLDAP, those represent values
that are already server-encrypted. It's how the server differentiates
userPassword: mypasswordintheclear
userPassword: {md5}anmd5hash
userPassword:: base64alreadyserverencryptedpassword

Huh? That last userPassword value does not signify that it is encrypted. It only signifies that it
contains a character which RFC2849 says must be base64-encoded. When you decode it, it may or may
not be a password hash, depending on your server configuration when you loaded that value.

--
mike