Using back-ldap as a client-side proxy/cache

[Ryan Steele]

Hey folks,

In order to provide stability to my OpenLDAP clients in the event of a network outage, I would like to implement some
client-side caching. I've done some research, and have concluded that nscd is evil and should be avoided at all costs,
and thus eventually settled on using back-ldap as a proxy and caching mechanism on the clients. Ideally, clients would
query a local cache first, and if the information was not available, back-ldap would then forward the connection on to
my root OpenLDAP server(s). However, I didn't see much information in the admin guide with respect to such
configurations other than a reference to the back-ldap man page, and given that I've got no real experience with setting
up back-ldap, I was wondering if somebody who did/does would have some recommendations, advice, or knew of a good
documentation source describing this sort of setup?

The other question I have is that it seems most people use back-ldap with a slapd.conf-style configuration, versus a
cn=config type of setup. In this sort of circumstance, where one is not configuring a full-on OpenLDAP server/replica,
that seems like it might be a good thing in the interest of keeping the client configurations simple. Nonetheless, I
wanted to verify that it was the recommended way, since slapd.conf (in the context of a fully fleshed-out OpenLDAP
server) is deprecated.

Thanks as always for insights, advice, and criticisms.

Respectfully,
Ryan

[Howard Chu]

Post by Ryan Steele
Hey folks,
In order to provide stability to my OpenLDAP clients in the event of a

network outage, I would like to implement some

Post by Ryan Steele
client-side caching. I've done some research, and have concluded that nscd

is evil and should be avoided at all costs,

It's not necesarily evil, it just doesn't work...

Post by Ryan Steele
and thus eventually settled on using back-ldap as a proxy and caching

mechanism on the clients.

And just to be clear, back-ldap is only a proxy. For caching support you must
add the pcache overlay.

Post by Ryan Steele
Ideally, clients would
query a local cache first, and if the information was not available,

back-ldap would then forward the connection on to

Post by Ryan Steele
my root OpenLDAP server(s).

That's exactly what back-ldap+pcache will do.

Post by Ryan Steele
However, I didn't see much information in the

admin guide with respect to such

Post by Ryan Steele
configurations other than a reference to the back-ldap man page, and given

that I've got no real experience with setting

Post by Ryan Steele
up back-ldap, I was wondering if somebody who did/does would have some

recommendations, advice, or knew of a good

Post by Ryan Steele
documentation source describing this sort of setup?

Generally there's not much to set up. back-ldap needs whatever information any
client would need to communicate successfully with the remote LDAP server.

Post by Ryan Steele
The other question I have is that it seems most people use back-ldap with
a

slapd.conf-style configuration, versus a

Post by Ryan Steele
cn=config type of setup. In this sort of circumstance, where one is not

configuring a full-on OpenLDAP server/replica,

Post by Ryan Steele
that seems like it might be a good thing in the interest of keeping the

client configurations simple. Nonetheless, I

Post by Ryan Steele
wanted to verify that it was the recommended way, since slapd.conf (in the

context of a fully fleshed-out OpenLDAP

Post by Ryan Steele
server) is deprecated.

It only seems that most are using slapd.conf because cn=config is new and most
sites with existing slapd.conf deployments haven't migrated to cn=config yet.
For new installs, just use cn=config.

[Ryan Steele]

Post by Howard Chu

Post by Ryan Steele
Hey folks,
In order to provide stability to my OpenLDAP clients in the event of a
network outage, I would like to implement some client-side caching.
I've done some research, and have concluded that nscd is evil and
should be avoided at all costs,

It's not necesarily evil, it just doesn't work...

Well, to me it's one and the same if it affects my service availability. :)

Post by Howard Chu

Post by Ryan Steele
and thus eventually settled on using back-ldap as a proxy and caching
mechanism on the clients.

And just to be clear, back-ldap is only a proxy. For caching support you
must add the pcache overlay.

Post by Ryan Steele
Ideally, clients would query a local cache first, and if the
information was not available, back-ldap would then forward the
connection on to my root OpenLDAP server(s).

That's exactly what back-ldap+pcache will do.

Yeah, I think I've settled on using libnss-ldapd/nslcd + back-ldap + pcache. I would like to get nssov involved too,
but it's not clear to me from the available documentation whether nssov is to be used in conjunction with pcache or
instead of pcache. It almost seemed like nssov would be most suitable on the server side of things, whereas pcache was
geared more towards client-side back-ldap setups. Am I misinterpreting anything there? If not, would anybody care to
expound on that subject a little to provide some framing context?

Post by Howard Chu

Post by Ryan Steele
However, I didn't see much information in the admin guide with respect
to such configurations other than a reference to the back-ldap man page,
and given that I've got no real experience with setting up back-ldap, I
was wondering if somebody who did/does would have some recommendations,
advice, or knew of a good documentation source describing this sort of
setup?

Generally there's not much to set up. back-ldap needs whatever
information any client would need to communicate successfully with the
remote LDAP server.

That's what I was starting to gather, but I'm still trying to figure out exactly what's supported since it's only a
proxy. In particular, there doesn't seem to be many examples of folks using ACL's in their back-ldap configs, so I
still want to make sure that the contents of the client-side cache are subject to the same visibility restrictions
imposed on the server. To that end, if someone could confirm that the ACL syntax available in fully-fleshed out
back-hdb slapd installations is available (in its entirety) in back-ldap configurations, I would be most appreciative.

Post by Howard Chu

Post by Ryan Steele
The other question I have is that it seems most people use back-ldap with
a slapd.conf-style configuration, versus a cn=config type of setup. In
this sort of circumstance, where one is not configuring a full-on
OpenLDAP server/replica, that seems like it might be a good thing in the
interest of keeping the client configurations simple. Nonetheless, I
wanted to verify that it was the recommended way, since slapd.conf (in
the context of a fully fleshed-out OpenLDAP server) is deprecated.

It only seems that most are using slapd.conf because cn=config is new
and most sites with existing slapd.conf deployments haven't migrated to
cn=config yet. For new installs, just use cn=config.

Gotcha, thanks for clarifying, Howard.


Respectfully,
Ryan

[Neil Dunbar]

Post by Howard Chu

Post by Ryan Steele
Hey folks,
In order to provide stability to my OpenLDAP clients in the event of a
network outage, I would like to implement some client-side caching.
I've done some research, and have concluded that nscd is evil and
should be avoided at all costs,

It's not necesarily evil, it just doesn't work...

Damn straight it doesn't work. I end up shooting it in the face wherever I can. For some reason, SLES seems to be particularly good at breaking things, IME. Debian/Ubuntu and RHEL/CentOS seem to be more forgiving - slightly.

Anyway - I've also been hacking out a caching proxy config for our enterprise directory, so for what it's worth, this is it. It's by no means optimised - so feel free to hack it to pieces.

I've omitted the access controls and ssf constraints, since that's not going to be useful for all setups.

------
moduleload back_hdb.la
moduleload back_ldap.la
moduleload pcache.la

database ldap
suffix "dc=llnw,dc=com"
uri "ldap://ldap.llnw.com"
tls start tls_cacert=/etc/ssl/certs/cacert.org.pem tls_reqcert=demand
rootdn "cn=admin,dc=llnw,dc=com"
rootpw <secret squirrel stuff>
network-timeout 10
idle-timeout 30

overlay pcache
pcache hdb 10000 1 1000 100
pcacheAttrset 0 cn uid llnwUnixUsername uidNumber gidNumber member memberUid homeDirectory loginShell
pcacheTemplate (&(objectClass=)(llnwUnixUsername=)) 0 3600
pcacheTemplate (&(objectClass=)(cn=)) 0 3600
pcacheTemplate (&(objectClass=)(gidNumber=)) 0 3600
pcacheTemplate (objectClass=) 0 3600
pcacheTemplate (&(objectClass=)(|(member=)(memberUid=))) 0 3600
pcacheTemplate (&(objectClass=)(member=)) 0 3600
pcacheTemplate (&(objectClass=)(memberUid=)) 0 3600
pcacheBind (&(objectClass=posixAccount)(llnwUnixUsername=)) 0 1800 sub dc=llnw,dc=com
pcachePersist TRUE
pcacheOffline FALSE
cachesize 1000
pcachePosition head
directory /var/lib/ldap
-----

[This gets set up to listen only to the ldapi:/// URI, and nslcd.conf has a "uri ldapi:///" line in its config]

Our internal schema has uid set to <name>@llnw,com, so that won't do as a login name - we have an internal attribute llnwUnixUsername which does the same thing. You might well want to substitute uid where that exists.

And while nssov is really cute, since it exists in the same process space as slapd, it doesn't end up triggering the pcache, which does gets fired upon incoming LDAP requests from an external process (nslcd). It's probably that I just suck, and didn't configure slapd quite right, but that's why I ended up still using nslcd and slapd on the same box.

On the plus side, this works quite well for laptop configurations which have partial connectivity. Mind you, when I mentioned having enterprise credentialing on personal laptops to my colleagues, the response was ... less than enthusiastic.

Anyway - hope this helps.

Cheers,

Neil





NEIL DUNBAR
Systems Architect

[phone] work
[phone] mobile
[phone] US mobile
www.llnw.com

[Howard Chu]

Post by Neil Dunbar

Post by Howard Chu

Post by Ryan Steele
Hey folks,
In order to provide stability to my OpenLDAP clients in the event of a
network outage, I would like to implement some client-side caching.
I've done some research, and have concluded that nscd is evil and
should be avoided at all costs,

It's not necesarily evil, it just doesn't work...

Damn straight it doesn't work. I end up shooting it in the face wherever I
can. For some reason, SLES seems to be particularly good at breaking things,
IME. Debian/Ubuntu and RHEL/CentOS seem to be more forgiving - slightly.
Anyway - I've also been hacking out a caching proxy config for our enterprise
directory, so for what it's worth, this is it. It's by no means optimised - so
feel free to hack it to pieces.
And while nssov is really cute, since it exists in the same process space as
slapd, it doesn't end up triggering the pcache, which does gets fired upon
incoming LDAP requests from an external process (nslcd). It's probably that I
just suck, and didn't configure slapd quite right, but that's why I ended up
still using nslcd and slapd on the same box.

Hm, you probably have them configured in the wrong order. I specifically
designed nssov and pcache to work together, and they do.

Post by Neil Dunbar
On the plus side, this works quite well for laptop configurations which have
partial connectivity. Mind you, when I mentioned having enterprise
credentialing on personal laptops to my colleagues, the response was ... less
than enthusiastic.

[Neil Dunbar]

Post by Neil Dunbar
And while nssov is really cute, since it exists in the same process space as
slapd, it doesn't end up triggering the pcache, which does gets fired upon
incoming LDAP requests from an external process (nslcd). It's probably that I
just suck, and didn't configure slapd quite right, but that's why I ended up
still using nslcd and slapd on the same box.

Hm, you probably have them configured in the wrong order. I specifically designed nssov and pcache to work together, and they do.

Well, I did try and flip them in either order (pcache then nssov, and vice versa); it did seem strange that it didn't work for me, but felt that it should. It could well be that my pcache templates weren't being triggered by nssov (but were being triggered by nslcd). I'll have another pop at a later point - it would certainly be to everyone's advantage to have one fewer daemon living in the system.

Cheers,

Neil





NEIL DUNBAR
Systems Architect

[phone] work
[phone] mobile
[phone] US mobile
www.llnw.com

[Neil Dunbar]

Post by Neil Dunbar
And while nssov is really cute, since it exists in the same process space as
slapd, it doesn't end up triggering the pcache, which does gets fired upon
incoming LDAP requests from an external process (nslcd). It's probably that I
just suck, and didn't configure slapd quite right, but that's why I ended up
still using nslcd and slapd on the same box.

Hm, you probably have them configured in the wrong order. I specifically designed nssov and pcache to work together, and they do.

OK - I'm stuck: yes - I realise I suck, but at the moment, I'm just spinning on this one. Howard/Quanah/anyone - can you post a sample config which lashes together nssov and pcache? At least it would help me see what's supposed to be the order of events.

I think my config is about as simple as it gets, back-ldap proxying to our real directory, then loading nssov, then pcache on a BDB backend (Although I've tried reversing pcache and nssov to no avail). pcache configured to cache the filters issued from nslcd/nssov.

The nssov overlay works perfectly - fetching passwd and group databases from the directory like a champ.

Similarly, if I do an ldapsearch on the proxy, using the nslcd filters - all is well, the local cache DB gets populated, with the right indices. If I fire up nslcd pointing to ldapi:///, I get the both the system databases working and caching working fine.

But nssov just doesn't seem to find its way through to the pcache. I can post the config - but it's pretty much identical to the ones in Symas's AAA paper. It's so annoying - I absolutely believe that I'm about a heartbeat away from getting it cracked, but the final solution is beyond my tired brain.

If it helps - OpenLDAP 2.4.21 running on Ubuntu Karmic amd64, BDB 4.7.

Cheers,

Neil





NEIL DUNBAR
Systems Architect

[phone] work
[phone] mobile
[phone] US mobile
www.llnw.com

[Howard Chu]

Post by Neil Dunbar

Post by Howard Chu

Post by Neil Dunbar
And while nssov is really cute, since it exists in the same process space as
slapd, it doesn't end up triggering the pcache, which does gets fired upon
incoming LDAP requests from an external process (nslcd). It's probably that I
just suck, and didn't configure slapd quite right, but that's why I ended up
still using nslcd and slapd on the same box.

Hm, you probably have them configured in the wrong order. I specifically
designed nssov and pcache to work together, and they do.

OK - I'm stuck: yes - I realise I suck, but at the moment, I'm just spinning
on this one. Howard/Quanah/anyone - can you post a sample config which lashes
together nssov and pcache? At least it would help me see what's supposed to be
the order of events.
I think my config is about as simple as it gets, back-ldap proxying to our
real directory, then loading nssov, then pcache on a BDB backend (Although
I've tried reversing pcache and nssov to no avail). pcache configured to cache
the filters issued from nslcd/nssov.
The nssov overlay works perfectly - fetching passwd and group databases from
the directory like a champ.
Similarly, if I do an ldapsearch on the proxy, using the nslcd filters - all
is well, the local cache DB gets populated, with the right indices. If I fire
up nslcd pointing to ldapi:///, I get the both the system databases working
and caching working fine.
But nssov just doesn't seem to find its way through to the pcache. I can post
the config - but it's pretty much identical to the ones in Symas's AAA paper.
It's so annoying - I absolutely believe that I'm about a heartbeat away from
getting it cracked, but the final solution is beyond my tired brain.

This is the essence of it:

database ldap
suffix "dc=example,dc=com"
rootdn "dc=example,dc=com"
rootpw "secret"
uri "ldap://localhost:9011/"

overlay pcache
proxycache bdb 100 2 6 1m
proxyattrset 0 uid userPassword uidNumber gidNumber gecos
cn homeDirectory loginShell objectClass
proxyattrset 1 cn userPassword gidNumber memberUid
uniqueMember objectClass
proxytemplate (&(objectclass=)(uid=)) 0 3600
proxytemplate (&(objectclass=)(uidnumber=)) 0 3600
proxytemplate (&(objectclass=)(gidnumber=)) 1 3600
proxytemplate (&(objectclass=)(|(memberuid=)(uniquemember=))) 1 3600
response-callback head

cachesize 20
directory /home/hyc/OD/hobj/tests/testrun/db.2.a
index objectClass eq
index cn,sn,uid,mail pres,eq,sub
index uidnumber,gidnumber,uniquemember,memberuid eq

overlay nssov

[Neil Dunbar]

Post by Howard Chu
overlay pcache
proxycache bdb 100 2 6 1m
proxyattrset 0 uid userPassword uidNumber gidNumber gecos
cn homeDirectory loginShell objectClass
proxyattrset 1 cn userPassword gidNumber memberUid
uniqueMember objectClass
proxytemplate (&(objectclass=)(uid=)) 0 3600
proxytemplate (&(objectclass=)(uidnumber=)) 0 3600
proxytemplate (&(objectclass=)(gidnumber=)) 1 3600
proxytemplate (&(objectclass=)(|(memberuid=)(uniquemember=))) 1 3600
response-callback head

Got it! My pcacheTemplates were written using original case (objectClass, memberUid, etc.), which gets normalized to lower case inside slapd (via nssov). Hence external queries maintaining case would trigger pcache, but the internal ones didn't.

D'oh!

Changed the templates to lower case. All working now. Thanks, Howard.

Cheers,

Neil





NEIL DUNBAR
Systems Architect

[phone] work
[phone] mobile
[phone] US mobile
www.llnw.com

[m***@aero.polimi.it]

Post by Neil Dunbar

Post by Howard Chu
overlay pcache
proxycache bdb 100 2 6 1m
proxyattrset 0 uid userPassword uidNumber gidNumber gecos
cn homeDirectory loginShell objectClass
proxyattrset 1 cn userPassword gidNumber memberUid
uniqueMember objectClass
proxytemplate (&(objectclass=)(uid=)) 0 3600
proxytemplate (&(objectclass=)(uidnumber=)) 0 3600
proxytemplate (&(objectclass=)(gidnumber=)) 1 3600
proxytemplate (&(objectclass=)(|(memberuid=)(uniquemember=))) 1 3600
response-callback head

Got it! My pcacheTemplates were written using original case (objectClass,
memberUid, etc.), which gets normalized to lower case inside slapd (via
nssov). Hence external queries maintaining case would trigger pcache, but
the internal ones didn't.
D'oh!
Changed the templates to lower case. All working now. Thanks, Howard.

Your temporary fix is probably fine. Though, I'd consider this a bug: the
case should not matter. I suggest you file an ITS.

p.

[m***@aero.polimi.it]

Post by m***@aero.polimi.it

Post by Neil Dunbar

Post by Howard Chu
overlay pcache
proxycache bdb 100 2 6 1m
proxyattrset 0 uid userPassword uidNumber gidNumber gecos
cn homeDirectory loginShell objectClass
proxyattrset 1 cn userPassword gidNumber memberUid
uniqueMember objectClass
proxytemplate (&(objectclass=)(uid=)) 0 3600
proxytemplate (&(objectclass=)(uidnumber=)) 0 3600
proxytemplate (&(objectclass=)(gidnumber=)) 1 3600
proxytemplate (&(objectclass=)(|(memberuid=)(uniquemember=))) 1 3600
response-callback head

Got it! My pcacheTemplates were written using original case
(objectClass,
memberUid, etc.), which gets normalized to lower case inside slapd (via
nssov). Hence external queries maintaining case would trigger pcache, but
the internal ones didn't.
D'oh!
Changed the templates to lower case. All working now. Thanks, Howard.

Your temporary fix is probably fine. Though, I'd consider this a bug: the
case should not matter. I suggest you file an ITS.

Although I'm not quite seeing where in the code the case of that
configuration may matter.

p.

[Neil Dunbar]

Post by m***@aero.polimi.it

Post by m***@aero.polimi.it

Post by Neil Dunbar
Changed the templates to lower case. All working now. Thanks, Howard.

Your temporary fix is probably fine. Though, I'd consider this a bug: the
case should not matter. I suggest you file an ITS.

Although I'm not quite seeing where in the code the case of that
configuration may matter.

Not sure either. The pCacheQueryURL attribute description has an MR of caseExactMatch, could that be it?

Cheers,

Neil





NEIL DUNBAR
Systems Architect

[phone] work
[phone] mobile
[phone] US mobile
www.llnw.com

[m***@aero.polimi.it]

Post by Neil Dunbar

Post by m***@aero.polimi.it

Post by m***@aero.polimi.it

Post by Neil Dunbar
Changed the templates to lower case. All working now. Thanks, Howard.

Your temporary fix is probably fine. Though, I'd consider this a bug: the
case should not matter. I suggest you file an ITS.

Although I'm not quite seeing where in the code the case of that
configuration may matter.

Not sure either. The pCacheQueryURL attribute description has an MR of
caseExactMatch, could that be it?

I don't think so. That MR (and that attribute) is only used when
storing/retrieving the cached rules stored in the database between server
restarts (AFAIK). In the code, all times a filter in a request is parsed,
and a query template is analyzed, attribute names are taken from the
schema. Whatever the configuration (or the client request) contained, it
is mapped to the corresponding value in the schema.

p.

[Neil Dunbar]

Post by Neil Dunbar

Post by m***@aero.polimi.it

Post by m***@aero.polimi.it

Post by Neil Dunbar
Changed the templates to lower case. All working now. Thanks, Howard.

Your temporary fix is probably fine. Though, I'd consider this a bug: the
case should not matter. I suggest you file an ITS.

Although I'm not quite seeing where in the code the case of that
configuration may matter.

Not sure either. The pCacheQueryURL attribute description has an MR of caseExactMatch, could that be it?

Looks like it's not a case issue after all. It seems like the results of some of my queries are being cached (ie, groups) but some aren't (ie, users).

Which results in some oddities: getent group foo (while not in cache) will correctly fetch the group details and its membership, but next time I run the command, I get the group name gid fetched from cache, but with no users. Running debug trace shows that the DNs of the members are indeed stored in cache, but it fails to fetch those DNs and their corresponding entries from cache.

Whereas getent passwd bar seems always to fetch from the backend directory (back-ldap), and not attempt to store the results in cache. Very odd indeed...

I'll keep on hacking on this one. Howard's config advanced the position considerably - I just need to work out why some of the templates don't trigger cache population.

Cheers,

Neil





NEIL DUNBAR
Systems Architect

[phone] work
[phone] mobile
[phone] US mobile
www.llnw.com

[Neil Dunbar]

Post by m***@aero.polimi.it

Post by Howard Chu
overlay pcache
proxycache bdb 100 2 6 1m
proxyattrset 0 uid userPassword uidNumber gidNumber gecos
cn homeDirectory loginShell objectClass
proxyattrset 1 cn userPassword gidNumber memberUid
uniqueMember objectClass
proxytemplate (&(objectclass=)(uid=)) 0 3600
proxytemplate (&(objectclass=)(uidnumber=)) 0 3600
proxytemplate (&(objectclass=)(gidnumber=)) 1 3600
proxytemplate (&(objectclass=)(|(memberuid=)(uniquemember=))) 1 3600
response-callback head

Your temporary fix is probably fine. Though, I'd consider this a bug: the
case should not matter. I suggest you file an ITS.

I think I'm starting to see the light now (yeah - like I thought a few posts ago...). The thing that Howard's config fixed (for me) wasn't the templates, as I first thought - it was the attribute sets for the queries. Our company schema needs to map a few attributes to alternative names (we don't use uniqueMember, we use plain old member instead; and our Unix login names go under the attribute llnwUnixUsername rather than uid). Fine - no problem there - nssov maps the names trivially.

However, the pcache will cache a query (and results) if the attribute set specified is a subset of the pcacheAttrSet - so clearly I needed to add those attributes to their relevant attribute sets. With that done, getent group <foo> and getent passwd <bar> would indeed cause results to be cached in the local DB.

What I was missing, it seems, was a couple of pCacheTemplates capable of triggering on the results for the searches prompted by nssov_passwd_all() and nssov_group_all() (e.g., when someone calls getent group, or getent passwd). So I added

pcacheTemplate (objectClass=) 0 3600 # attr set 0 is the user attributes
pcacheTemplate (objectClass=) 1 3600 # set 1 is the group attrs

and both the complete passwd db and group db ended up being cached as well. Great stuff (although maybe not a good idea for a very large company).

I don't think this is buggy behaviour, but it lead me to some unexpected results (ie, groups that weren't empty appearing to be so - meaning that the groups were cached, but their members were not in cache). I'm still hacking away on it, but it does seem that the relevant subset of our directory is being cached on my laptop outside of the office - so that's a major plus. Apologies to all for the false starts and so on, but perhaps my findings will help others make this step more easily. Especially when combined with sudo-ldap, it does make for a really good enterprise AAA system - even for partially connected use.

Cheers,

Neil





NEIL DUNBAR
Systems Architect

[phone] work
[phone] mobile
[phone] US mobile
www.llnw.com