Using back-meta or back-relay plus slapo-rwm as a proxy to a local database

[Ryan Steele]

Ok, fair warning - this is a little long-winded, but I'd rather give too much detail than not enough. Also, all
examples are in slapd.conf format, since there is no documentation for cn=config, and I'm using slapd with -f and -F to
make the conversion.


Anyways, I'm working on implementing some rewrite rules with slapo-rwm, using back-relay (or possibly back-meta if that
doesn't work out) as the backend on top of which slapo-rwm will sit, to make available some features that would be
otherwise not be present with back-ldap, or in the absence of a proxy backend altogether.


In its most basic form, the slapo-rwm overlay can be used without a proxy or relay backend, but the feature I'm after
appears to require either the slapd-meta backend or the slapd-relay backend. For now, I've chosen to try with
slapd-relay, as my research has indicated it supports the feature I'm after. For reference, this feature I desire is
the 'searchResult' rewriteContext, as I'm interested in rewriting dynamic group membership DN's (i.e., the 'member'
attribute, as defined in rfc2307bis) as plain uid's, so clients that only know about rfc2307 (and thus, look only for
entries of the same format as memberUid and fail to perform any mappings to translate membership DN's to UID's) can make
use of the dynamic groups I've created in my directory.


Since my slapd-hdb database's rootDSE is dc=example,dc=com, I really couldn't find any way to get around using a virtual
naming context; creating a back-meta database on the same host with the same suffix as the back-hdb database seemed
ambiguous at best (in terms of how to ensure that clients would always speak to the back-meta db first), and using the
suffix "dc=com" seemed inappropriate, as did creating several relay databases, one for each of the entries below the
slapd-hdb database's rootDSE (e.g., ou=Users,dc=example,dc=com, ou=Groups,dc=example,dc=com, etc.). Creating several
relay databases in that fashion presents other problems as well, and wouldn't do me any good for clients which query for
entries using dc=example,dc=com as their search base.


Given that, and the fact that I'm only looking to remap/rewrite data going to and from a single local database with a
single naming context, I decided it was a better idea to create a single back-relay database with a bogus "proxy" naming
context and use a suffixmassage to rewrite it, e.g.:


database relay
suffix "dc=example-proxy,dc=com"
relay "dc=example,dc=com"

overlay rwm
rwm-rewriteEngine on
rwm-suffixmassage "dc=example-proxy,dc=com" "dc=example,dc=com"
rwm-rewriteContext searchResult
rwm-rewriteRule "^uid=([^,]+?),ou=Users,dc=example,dc=com$" "$1"


For reference, here is the back-meta database configuration I was going to use before I discovered that back-relay
supported the searchResult rewriteContext:

database meta
suffix "dc=example-proxy,dc=com"

overlay rwm
rwm-rewriteEngine on
uri "ldap://localhost/dc=example-proxy,dc=com"
rwm-suffixmassage "dc=example-proxy,dc=com" "dc=example,dc=com"
rwm-rewriteContext searchResult
rwm-rewriteRule "^uid=([^,]+?),ou=Users,dc=example,dc=com$" "$1"


In either case, I have a few questions going forward from this point:

1. Is it legal to rewrite the search result DN's as described in the example above? I'm hoping that since the data
being manipulated is going from server -> client and is not being stored in the directory, the attributes I'm rewriting
would not be subjected to the syntax requirements of the entry that holds them.

2. Are there any benefits or drawbacks to using a rewriteMap to accomplish this task? In my mind, it just seems like
another means to the same end, at least based on the following configuration I came up with:

rwm-rewriteMap ldap dn2uid "ldap://localhost/ou=Users,dc=example,dc=com?uid?sub"
rwm-rewriteContext searchResult
rwm-rewriteRule "^uid=([^,]+?),ou=Users,dc=example,dc=com$" "${dn2uid($1)}"

3. Is there a better way to go about doing this that I've failed to see or consider?

4. Considering the slapd-meta man page, which mentions that some of its features could potentially result in excessive
overhead for some installations, are the features used in the aforementioned configuration(s) efficient enough to scale
well in a very busy environment? I'm aware that's a pretty vague and loaded question with lots of variables, so I'm
just looking for an educated opinion from a 10,000 foot view, assuming excellent hardware, high network capacity, and
heavy - say, 5k, 10k, 15k queries per second - traffic. Performance will obviously be tested before implementing this,
but it'd still be nice to have a rough idea of what to expect going in.


Lastly, on a somewhat tangential note, given that there's no documentation with respect to cn=config for setting up
slapo-rwm (or doing it in conjunction with a proxy backend), is there any way to discover the cn=config attributes and
objectclasses for those backend and overlays that's better or more efficient than creating a standard slapd.conf
configuration and converting it with slapd using -f and -F? Perhaps an official or unofficial bit of documentation
floating around out there other than a few miscellaneous postings on the mailing lists and some ITS contents? On a
similar note in the same vein, as far as I can tell, none of the man pages, admin guide sections, or FAQ-O-Matic entries
on slapd-relay mention which rewrite contexts are available with it, and I didn't realize that slapd-relay accepted the
searchResult rewrite context until I read the servers/slapd/back-relay/README file (while searching for code that might
reveal slapo-rwm's cn=config attribute equivalents). Would it be objectionable to update these reference points with
said information? I'd be happy to file an ITS and submit patches and/or content for updating the documentation, if
Gavin is too busy, unavailable, or would like some help.


Thanks for any and all advice, musings, criticisms, and opinions!

-Ryan

[m***@aero.polimi.it]

Post by Ryan Steele
Ok, fair warning - this is a little long-winded, but I'd rather give too
much detail than not enough. Also, all
examples are in slapd.conf format, since there is no documentation for
cn=config, and I'm using slapd with -f and -F to
make the conversion.
Anyways, I'm working on implementing some rewrite rules with slapo-rwm,
using back-relay (or possibly back-meta if that
doesn't work out) as the backend on top of which slapo-rwm will sit, to
make available some features that would be
otherwise not be present with back-ldap, or in the absence of a proxy backend altogether.
In its most basic form, the slapo-rwm overlay can be used without a proxy
or relay backend, but the feature I'm after
appears to require either the slapd-meta backend or the slapd-relay
backend. For now, I've chosen to try with
slapd-relay, as my research has indicated it supports the feature I'm
after. For reference, this feature I desire is
the 'searchResult' rewriteContext, as I'm interested in rewriting dynamic
group membership DN's (i.e., the 'member'
attribute, as defined in rfc2307bis) as plain uid's, so clients that only
know about rfc2307 (and thus, look only for
entries of the same format as memberUid and fail to perform any mappings
to translate membership DN's to UID's) can make
use of the dynamic groups I've created in my directory.
Since my slapd-hdb database's rootDSE is dc=example,dc=com, I really
couldn't find any way to get around using a virtual
naming context; creating a back-meta database on the same host with the
same suffix as the back-hdb database seemed
ambiguous at best (in terms of how to ensure that clients would always
speak to the back-meta db first), and using the
suffix "dc=com" seemed inappropriate, as did creating several relay
databases, one for each of the entries below the
slapd-hdb database's rootDSE (e.g., ou=Users,dc=example,dc=com,
ou=Groups,dc=example,dc=com, etc.). Creating several
relay databases in that fashion presents other problems as well, and
wouldn't do me any good for clients which query for
entries using dc=example,dc=com as their search base.
Given that, and the fact that I'm only looking to remap/rewrite data going
to and from a single local database with a
single naming context, I decided it was a better idea to create a single
back-relay database with a bogus "proxy" naming
database relay
suffix "dc=example-proxy,dc=com"
relay "dc=example,dc=com"
overlay rwm
rwm-rewriteEngine on
rwm-suffixmassage "dc=example-proxy,dc=com" "dc=example,dc=com"
rwm-rewriteContext searchResult
rwm-rewriteRule "^uid=([^,]+?),ou=Users,dc=example,dc=com$" "$1"

The result of this mapping does not yield a valid DN; as such, this
rewriting is incorrect. Also, I don't see how it could be useful, as LDAP
clients usually expect DNs to be DN-valued.

Post by Ryan Steele
For reference, here is the back-meta database configuration I was going to
use before I discovered that back-relay
database meta
suffix "dc=example-proxy,dc=com"
overlay rwm
rwm-rewriteEngine on
uri "ldap://localhost/dc=example-proxy,dc=com"
rwm-suffixmassage "dc=example-proxy,dc=com" "dc=example,dc=com"
rwm-rewriteContext searchResult
rwm-rewriteRule "^uid=([^,]+?),ou=Users,dc=example,dc=com$" "$1"
1. Is it legal to rewrite the search result DN's as described in the
example above? I'm hoping that since the data
being manipulated is going from server -> client and is not being stored
in the directory, the attributes I'm rewriting
would not be subjected to the syntax requirements of the entry that holds them.
2. Are there any benefits or drawbacks to using a rewriteMap to accomplish
this task? In my mind, it just seems like
another means to the same end, at least based on the following
rwm-rewriteMap ldap dn2uid
"ldap://localhost/ou=Users,dc=example,dc=com?uid?sub"
rwm-rewriteContext searchResult
rwm-rewriteRule "^uid=([^,]+?),ou=Users,dc=example,dc=com$"
"${dn2uid($1)}"

Same as above. You're performing an internal search to obtain information
you already have, and cast it in invalid form.

Post by Ryan Steele
3. Is there a better way to go about doing this that I've failed to see or consider?

Consider why you need it: broken client? Fix the client. Anything else?
Perhaps LDAP is not what you need; in any case, move such mangling into
the client.

Post by Ryan Steele
4. Considering the slapd-meta man page, which mentions that some of its
features could potentially result in excessive
overhead for some installations, are the features used in the
aforementioned configuration(s) efficient enough to scale
well in a very busy environment? I'm aware that's a pretty vague and
loaded question with lots of variables, so I'm
just looking for an educated opinion from a 10,000 foot view, assuming
excellent hardware, high network capacity, and
heavy - say, 5k, 10k, 15k queries per second - traffic. Performance will
obviously be tested before implementing this,
but it'd still be nice to have a rough idea of what to expect going in.

Pure rewriting may not impact too much; rewriteMaps that rewuire further
internal lookups may yield significant overhead. Of course, if you setup
things like ACL checking that involves significant internal searches on a
database that makes extensive use of rewriting, things can get even worse.

Post by Ryan Steele
Lastly, on a somewhat tangential note, given that there's no documentation
with respect to cn=config for setting up
slapo-rwm (or doing it in conjunction with a proxy backend), is there any
way to discover the cn=config attributes and
objectclasses for those backend and overlays that's better or more
efficient than creating a standard slapd.conf
configuration and converting it with slapd using -f and -F? Perhaps an
official or unofficial bit of documentation
floating around out there other than a few miscellaneous postings on the
mailing lists and some ITS contents? On a
similar note in the same vein, as far as I can tell, none of the man
pages, admin guide sections, or FAQ-O-Matic entries
on slapd-relay mention which rewrite contexts are available with it, and I
didn't realize that slapd-relay accepted the
searchResult rewrite context until I read the
servers/slapd/back-relay/README file (while searching for code that might
reveal slapo-rwm's cn=config attribute equivalents). Would it be
objectionable to update these reference points with
said information? I'd be happy to file an ITS and submit patches and/or
content for updating the documentation, if
Gavin is too busy, unavailable, or would like some help.

None that I'm aware of. The FAQ is interactive; if by extensive
experimenting, or by looking at the code, you want to contribute, it'd be
more than welcome. Please note that 2.4.22 contains some improvements to
slapo-rwm cn=config handling; you may want to stick with that release.

p.

[m***@aero.polimi.it]

Post by m***@aero.polimi.it

Post by Ryan Steele
Ok, fair warning - this is a little long-winded, but I'd rather give too
much detail than not enough. Also, all
examples are in slapd.conf format, since there is no documentation for
cn=config, and I'm using slapd with -f and -F to
make the conversion.
Anyways, I'm working on implementing some rewrite rules with slapo-rwm,
using back-relay (or possibly back-meta if that
doesn't work out) as the backend on top of which slapo-rwm will sit, to
make available some features that would be
otherwise not be present with back-ldap, or in the absence of a proxy
backend altogether.
In its most basic form, the slapo-rwm overlay can be used without a proxy
or relay backend, but the feature I'm after
appears to require either the slapd-meta backend or the slapd-relay
backend. For now, I've chosen to try with
slapd-relay, as my research has indicated it supports the feature I'm
after. For reference, this feature I desire is
the 'searchResult' rewriteContext, as I'm interested in rewriting dynamic
group membership DN's (i.e., the 'member'
attribute, as defined in rfc2307bis) as plain uid's, so clients that only
know about rfc2307 (and thus, look only for
entries of the same format as memberUid and fail to perform any mappings
to translate membership DN's to UID's) can make
use of the dynamic groups I've created in my directory.
Since my slapd-hdb database's rootDSE is dc=example,dc=com, I really
couldn't find any way to get around using a virtual
naming context; creating a back-meta database on the same host with the
same suffix as the back-hdb database seemed
ambiguous at best (in terms of how to ensure that clients would always
speak to the back-meta db first), and using the
suffix "dc=com" seemed inappropriate, as did creating several relay
databases, one for each of the entries below the
slapd-hdb database's rootDSE (e.g., ou=Users,dc=example,dc=com,
ou=Groups,dc=example,dc=com, etc.). Creating several
relay databases in that fashion presents other problems as well, and
wouldn't do me any good for clients which query for
entries using dc=example,dc=com as their search base.
Given that, and the fact that I'm only looking to remap/rewrite data going
to and from a single local database with a
single naming context, I decided it was a better idea to create a single
back-relay database with a bogus "proxy" naming
database relay
suffix "dc=example-proxy,dc=com"
relay "dc=example,dc=com"
overlay rwm
rwm-rewriteEngine on
rwm-suffixmassage "dc=example-proxy,dc=com" "dc=example,dc=com"
rwm-rewriteContext searchResult
rwm-rewriteRule "^uid=([^,]+?),ou=Users,dc=example,dc=com$" "$1"

The result of this mapping does not yield a valid DN; as such, this
rewriting is incorrect. Also, I don't see how it could be useful, as LDAP
clients usually expect DNs to be DN-valued.

In this case, dscl (Mac OS X's directory services client) expects a UID,
not a DN, as is the POSIX standard for group
members, and doesn't know how to parse usernames in groups that use DN's
to identify their members. Instead, they
expect UID's, and since there appears to be no way to do this client side
with dscl (believe me, I did try to find one
before resorting to this), I have constructed the above rewriteRule to do
this for them. This should not impact my
Linux clients either, as nslcd (a daemon created in a fork of nss_ldap by
Arthur de Jong) only performs DN-to-UID
translations if the group members are represented as DN-valued attribute
values. My question wasn't whether or not the
result is a valid DN - it's whether or not it would actually work,
functionally speaking. Is that the case, regardless
of how ugly or undesirable it might be?

You seem to be confusing LDAP groups (groupOfNames objectClass, member
attribute) with POSIX groups (posixGroup objectClass, memberUid
attribute). This has been sorted out long ago. Use the right object for
each application. The rest of the discussion, at this point, seems moot.

p.

Post by m***@aero.polimi.it

Post by Ryan Steele
For reference, here is the back-meta database configuration I was going to
use before I discovered that back-relay
database meta
suffix "dc=example-proxy,dc=com"
overlay rwm
rwm-rewriteEngine on
uri "ldap://localhost/dc=example-proxy,dc=com"
rwm-suffixmassage "dc=example-proxy,dc=com" "dc=example,dc=com"
rwm-rewriteContext searchResult
rwm-rewriteRule "^uid=([^,]+?),ou=Users,dc=example,dc=com$" "$1"
1. Is it legal to rewrite the search result DN's as described in the
example above? I'm hoping that since the data
being manipulated is going from server -> client and is not being stored
in the directory, the attributes I'm rewriting
would not be subjected to the syntax requirements of the entry that
holds
them.
2. Are there any benefits or drawbacks to using a rewriteMap to accomplish
this task? In my mind, it just seems like
another means to the same end, at least based on the following
rwm-rewriteMap ldap dn2uid
"ldap://localhost/ou=Users,dc=example,dc=com?uid?sub"
rwm-rewriteContext searchResult
rwm-rewriteRule "^uid=([^,]+?),ou=Users,dc=example,dc=com$" "${dn2uid($1)}"

Same as above. You're performing an internal search to obtain information
you already have, and cast it in invalid form.

If there was any other way at all to do this, I would, believe me. But,
OS X's LDAPv3 plugin for dscl simply does not
know how to utilize DN-valued group members, and provides no way to
customize the way search results are parsed. So
again, I understand that the information is not a valid DN, but again, if
functionally speaking this works, then that's
what I'll have to do to leverage my rfc2307bis-style dynamic groups on OS
X.

Post by m***@aero.polimi.it

Post by Ryan Steele
3. Is there a better way to go about doing this that I've failed to see
or
consider?

Consider why you need it: broken client? Fix the client. Anything else?
Perhaps LDAP is not what you need; in any case, move such mangling into
the client.

LDAP is what I need, and unfortunately, "fixing the client" (as in hacking
and recompiling dscl's LDAPv3 plugin) would
potentially invalidate a warranty of service and support.

Post by m***@aero.polimi.it

Post by Ryan Steele
4. Considering the slapd-meta man page, which mentions that some of its
features could potentially result in excessive
overhead for some installations, are the features used in the
aforementioned configuration(s) efficient enough to scale
well in a very busy environment? I'm aware that's a pretty vague and
loaded question with lots of variables, so I'm
just looking for an educated opinion from a 10,000 foot view, assuming
excellent hardware, high network capacity, and
heavy - say, 5k, 10k, 15k queries per second - traffic. Performance will
obviously be tested before implementing this,
but it'd still be nice to have a rough idea of what to expect going in.

Pure rewriting may not impact too much; rewriteMaps that rewuire further
internal lookups may yield significant overhead. Of course, if you setup
things like ACL checking that involves significant internal searches on a
database that makes extensive use of rewriting, things can get even worse.

Great, thanks so much for that assessment, Pierangelo.

Post by m***@aero.polimi.it

Post by Ryan Steele
Lastly, on a somewhat tangential note, given that there's no
documentation
with respect to cn=config for setting up
slapo-rwm (or doing it in conjunction with a proxy backend), is there any
way to discover the cn=config attributes and
objectclasses for those backend and overlays that's better or more
efficient than creating a standard slapd.conf
configuration and converting it with slapd using -f and -F? Perhaps an
official or unofficial bit of documentation
floating around out there other than a few miscellaneous postings on the
mailing lists and some ITS contents? On a
similar note in the same vein, as far as I can tell, none of the man
pages, admin guide sections, or FAQ-O-Matic entries
on slapd-relay mention which rewrite contexts are available with it, and I
didn't realize that slapd-relay accepted the
searchResult rewrite context until I read the
servers/slapd/back-relay/README file (while searching for code that might
reveal slapo-rwm's cn=config attribute equivalents). Would it be
objectionable to update these reference points with
said information? I'd be happy to file an ITS and submit patches and/or
content for updating the documentation, if
Gavin is too busy, unavailable, or would like some help.

None that I'm aware of. The FAQ is interactive; if by extensive
experimenting, or by looking at the code, you want to contribute, it'd be
more than welcome. Please note that 2.4.22 contains some improvements to
slapo-rwm cn=config handling; you may want to stick with that release.
p.

Duly noted, and thanks again for your advice here, and any future advice
on this topic.
Respectfully,
Ryan

[m***@aero.polimi.it]

Post by m***@aero.polimi.it

In this case, dscl (Mac OS X's directory services client) expects a UID,
not a DN, as is the POSIX standard for group
members, and doesn't know how to parse usernames in groups that use DN's
to identify their members. Instead, they
expect UID's, and since there appears to be no way to do this client side
with dscl (believe me, I did try to find one
before resorting to this), I have constructed the above rewriteRule to do
this for them. This should not impact my
Linux clients either, as nslcd (a daemon created in a fork of nss_ldap
by
Arthur de Jong) only performs DN-to-UID
translations if the group members are represented as DN-valued attribute
values. My question wasn't whether or not the
result is a valid DN - it's whether or not it would actually work,
functionally speaking. Is that the case, regardless
of how ugly or undesirable it might be?

You seem to be confusing LDAP groups (groupOfNames objectClass, member
attribute) with POSIX groups (posixGroup objectClass, memberUid
attribute). This has been sorted out long ago. Use the right object for
each application. The rest of the discussion, at this point, seems moot.
p.

No, I know the difference. What I'm saying is that the OS X clients
aren't translating DN-valued LDAP group membership
attributes to UID-valued POSIX group memberships. On Linux, this is done
a) parsing the group membership attributes, which are listed as DN's, and
returning just the portion between the 'uid='
and the next comma (e.g., if the DN was
'uid=jdoe,ou=Users,dc=example,dc=org', nslcd would translate that to
'jdoe', for
use as a POSIX group member), or
b) issuing a second lookup to map the UID corresponding to that particular
DN (kind of the way slapo-rwm can if
configured to).
In other words, I'm what I'm proposing to do is exactly what
nss_ldapd/nslcd does, only in a proxy database on the
server side in the server's response to the client, instead of on the
client side of that response.
The *only* reason I'm doing this is because it's not being done on OS X
machines by Directory Service's LDAPv3 plugin
(the equivalent to Linux's nss_ldap plugin for NSS), and I cannot change
the way the LDAPv3 plugin works by hacking its
code, as doing so could void the warranty and support and all that other
proprietary nonsense. So my only choice is to
do it in the response on the server side, instead of in the response on
the client side.
And, since this is going to be done by means of a virtual naming context
with slapd-meta and slapo-rwm, only the OS X
clients will be using that virtual naming context. Everybody else, who
does it properly on the client side, will use
the real naming context. The main reason for the OP was to verify whether
or not the implementation I proposed would
actually *work*, regardless of whether or not it was advisable from a
policy standpoint. If by "invalid", you mean
"will not work" (and not "it is not advisable to do so"), then that is
fine and I'll find some other way around this
problem (somehow...) - so if you wouldn't mind clarifying your response, I
would be appreciative.

Well, perhaps you should then have a look at slapo-nssov
(contrib/slapd-modules/nssov/); someone else on this list may give you
better support than I could in setting it up appropriately.

p.

[m***@aero.polimi.it]

Post by m***@aero.polimi.it

No, I know the difference. What I'm saying is that the OS X clients
aren't translating DN-valued LDAP group membership
attributes to UID-valued POSIX group memberships. On Linux, this is done
a) parsing the group membership attributes, which are listed as DN's, and
returning just the portion between the 'uid='
and the next comma (e.g., if the DN was
'uid=jdoe,ou=Users,dc=example,dc=org', nslcd would translate that to
'jdoe', for
use as a POSIX group member), or
b) issuing a second lookup to map the UID corresponding to that particular
DN (kind of the way slapo-rwm can if
configured to).
In other words, I'm what I'm proposing to do is exactly what
nss_ldapd/nslcd does, only in a proxy database on the
server side in the server's response to the client, instead of on the
client side of that response.
The *only* reason I'm doing this is because it's not being done on OS X
machines by Directory Service's LDAPv3 plugin
(the equivalent to Linux's nss_ldap plugin for NSS), and I cannot change
the way the LDAPv3 plugin works by hacking its
code, as doing so could void the warranty and support and all that other
proprietary nonsense. So my only choice is to
do it in the response on the server side, instead of in the response on
the client side.
And, since this is going to be done by means of a virtual naming context
with slapd-meta and slapo-rwm, only the OS X
clients will be using that virtual naming context. Everybody else, who
does it properly on the client side, will use
the real naming context. The main reason for the OP was to verify whether
or not the implementation I proposed would
actually *work*, regardless of whether or not it was advisable from a
policy standpoint. If by "invalid", you mean
"will not work" (and not "it is not advisable to do so"), then that is
fine and I'll find some other way around this
problem (somehow...) - so if you wouldn't mind clarifying your response, I
would be appreciative.

Well, perhaps you should then have a look at slapo-nssov
(contrib/slapd-modules/nssov/); someone else on this list may give you
better support than I could in setting it up appropriately.
p.

No; nssov is irrelevant here - that would only work for Linux clients, and
they're the ones who perform the DN-to-UID
translations appropriately. I know that this isn't just an issue for me;
O'Reilly have even made passing references in
their books about the fact that Apple's LDAPv3 Directory Services plugin
does not handle DN-valued group memberships
http://macdevcenter.com/pub/a/mac/2003/08/26/active_directory.html?page=2
That was 7 years ago, and they still haven't fixed it. Wonder what the
odds are that they'll have a change of heart? :P
Thanks anyways, though, for taking a gander at this. I'm going to try and
implement the workaround I mentioned above
for those OS X hosts, as a stopgap (hopefully a stopgap and not permanent,
anyways...) until and if Apple updates the
plugin to allow DN-valued group memberships, as they do for Active
Directory. Hopefully it works. I wish I could say
that I find it ironic that they would get it right with AD and not with
*nix, but I digress...

Slapo-rwm (and back-meta, which mimics it) by design only rewrite
DN-valued attrs, and only rewrite them to DN values. This is for two
reasons:

1) DN-rewriting makes sense in distributed directories, virtual views and
so on

2) otherwise, people would abuse it, opening too many cans of worms, or
would use it, like in your case, to workaround broken clients, thus
allowing those clients to live forever shifting to the DSA all the burden
of working around their issues.

In any case, hacking slapo-rwm for your purpose shouldn't be very
difficult: you should simply hack where, after rewriting, the syntax of
the rewritten attrs is checked. In the case of searchResult this occurs
in rwm_dnattr_result_rewrite() by calling
rwm_dn_massage_pretty_normalize(). You can replace that call simply by
rwm_dn_massage() (with some memory bookkeeping, possibly).

If you succeed, please do not submit patches, as they'll likely be
rejected. The "right" way to solve the issue would be to design a
"slapo-apple" overlay that hides all the hacks required by that client.

p.

[Ryan Steele]

Post by m***@aero.polimi.it

No; nssov is irrelevant here - that would only work for Linux clients, and
they're the ones who perform the DN-to-UID
translations appropriately. I know that this isn't just an issue for me;
O'Reilly have even made passing references in
their books about the fact that Apple's LDAPv3 Directory Services plugin
does not handle DN-valued group memberships
http://macdevcenter.com/pub/a/mac/2003/08/26/active_directory.html?page=2
That was 7 years ago, and they still haven't fixed it. Wonder what the
odds are that they'll have a change of heart? :P
Thanks anyways, though, for taking a gander at this. I'm going to try and
implement the workaround I mentioned above
for those OS X hosts, as a stopgap (hopefully a stopgap and not permanent,
anyways...) until and if Apple updates the
plugin to allow DN-valued group memberships, as they do for Active
Directory. Hopefully it works. I wish I could say
that I find it ironic that they would get it right with AD and not with
*nix, but I digress...

Slapo-rwm (and back-meta, which mimics it) by design only rewrite
DN-valued attrs, and only rewrite them to DN values.

Bummer for me.

Post by m***@aero.polimi.it
1) DN-rewriting makes sense in distributed directories, virtual views and
so on

Agreed; I saw a lot of other practical uses for it during my research.

Post by m***@aero.polimi.it
2) otherwise, people would abuse it, opening too many cans of worms, or
would use it, like in your case, to workaround broken clients, thus
allowing those clients to live forever shifting to the DSA all the burden
of working around their issues.

Agreed here as well. I can completely understand the justifications.

Post by m***@aero.polimi.it
In any case, hacking slapo-rwm for your purpose shouldn't be very
difficult: you should simply hack where, after rewriting, the syntax of
the rewritten attrs is checked. In the case of searchResult this occurs
in rwm_dnattr_result_rewrite() by calling
rwm_dn_massage_pretty_normalize(). You can replace that call simply by
rwm_dn_massage() (with some memory bookkeeping, possibly).

I may give this a shot, thanks for the tip.

Post by m***@aero.polimi.it
If you succeed, please do not submit patches, as they'll likely be
rejected. The "right" way to solve the issue would be to design a
"slapo-apple" overlay that hides all the hacks required by that client.
p.

I think this is a good idea, given the speed with which Apple usually addresses things like this. But it will probably
have to wait until I've had a chance to resolve some other outstanding issues, such as ITS#6540, before I can take the
time to write an overlay from scratch.


Again, many thanks for the advice.


Respectfully,
Ryan

[Ryan Steele]

Post by m***@aero.polimi.it

No, I know the difference. What I'm saying is that the OS X clients
aren't translating DN-valued LDAP group membership
attributes to UID-valued POSIX group memberships. On Linux, this is done
a) parsing the group membership attributes, which are listed as DN's, and
returning just the portion between the 'uid='
and the next comma (e.g., if the DN was
'uid=jdoe,ou=Users,dc=example,dc=org', nslcd would translate that to
'jdoe', for
use as a POSIX group member), or
b) issuing a second lookup to map the UID corresponding to that particular
DN (kind of the way slapo-rwm can if
configured to).
In other words, I'm what I'm proposing to do is exactly what
nss_ldapd/nslcd does, only in a proxy database on the
server side in the server's response to the client, instead of on the
client side of that response.
The *only* reason I'm doing this is because it's not being done on OS X
machines by Directory Service's LDAPv3 plugin
(the equivalent to Linux's nss_ldap plugin for NSS), and I cannot change
the way the LDAPv3 plugin works by hacking its
code, as doing so could void the warranty and support and all that other
proprietary nonsense. So my only choice is to
do it in the response on the server side, instead of in the response on the client side.
And, since this is going to be done by means of a virtual naming context
with slapd-meta and slapo-rwm, only the OS X
clients will be using that virtual naming context. Everybody else, who
does it properly on the client side, will use
the real naming context. The main reason for the OP was to verify whether
or not the implementation I proposed would
actually *work*, regardless of whether or not it was advisable from a
policy standpoint. If by "invalid", you mean
"will not work" (and not "it is not advisable to do so"), then that is
fine and I'll find some other way around this
problem (somehow...) - so if you wouldn't mind clarifying your response, I
would be appreciative.

Well, perhaps you should then have a look at slapo-nssov
(contrib/slapd-modules/nssov/); someone else on this list may give you
better support than I could in setting it up appropriately.
p.

No; nssov is irrelevant here - that would only work for Linux clients, and they're the ones who perform the DN-to-UID
translations appropriately. I know that this isn't just an issue for me; O'Reilly have even made passing references in
their books about the fact that Apple's LDAPv3 Directory Services plugin does not handle DN-valued group memberships
appropriately (third paragraph from the bottom): http://macdevcenter.com/pub/a/mac/2003/08/26/active_directory.html?page=2

That was 7 years ago, and they still haven't fixed it. Wonder what the odds are that they'll have a change of heart? :P

Thanks anyways, though, for taking a gander at this. I'm going to try and implement the workaround I mentioned above
for those OS X hosts, as a stopgap (hopefully a stopgap and not permanent, anyways...) until and if Apple updates the
plugin to allow DN-valued group memberships, as they do for Active Directory. Hopefully it works. I wish I could say
that I find it ironic that they would get it right with AD and not with *nix, but I digress...


Respectfully,
Ryan

[Ryan Steele]

Post by m***@aero.polimi.it

In this case, dscl (Mac OS X's directory services client) expects a UID,
not a DN, as is the POSIX standard for group
members, and doesn't know how to parse usernames in groups that use DN's
to identify their members. Instead, they
expect UID's, and since there appears to be no way to do this client side
with dscl (believe me, I did try to find one
before resorting to this), I have constructed the above rewriteRule to do
this for them. This should not impact my
Linux clients either, as nslcd (a daemon created in a fork of nss_ldap by
Arthur de Jong) only performs DN-to-UID
translations if the group members are represented as DN-valued attribute
values. My question wasn't whether or not the
result is a valid DN - it's whether or not it would actually work,
functionally speaking. Is that the case, regardless
of how ugly or undesirable it might be?

You seem to be confusing LDAP groups (groupOfNames objectClass, member
attribute) with POSIX groups (posixGroup objectClass, memberUid
attribute). This has been sorted out long ago. Use the right object for
each application. The rest of the discussion, at this point, seems moot.
p.

No, I know the difference. What I'm saying is that the OS X clients aren't translating DN-valued LDAP group membership
attributes to UID-valued POSIX group memberships. On Linux, this is done with nss_ldap/nss_ldapd/nslcd by either:

a) parsing the group membership attributes, which are listed as DN's, and returning just the portion between the 'uid='
and the next comma (e.g., if the DN was 'uid=jdoe,ou=Users,dc=example,dc=org', nslcd would translate that to 'jdoe', for
use as a POSIX group member), or

b) issuing a second lookup to map the UID corresponding to that particular DN (kind of the way slapo-rwm can if
configured to).

In other words, I'm what I'm proposing to do is exactly what nss_ldapd/nslcd does, only in a proxy database on the
server side in the server's response to the client, instead of on the client side of that response.

The *only* reason I'm doing this is because it's not being done on OS X machines by Directory Service's LDAPv3 plugin
(the equivalent to Linux's nss_ldap plugin for NSS), and I cannot change the way the LDAPv3 plugin works by hacking its
code, as doing so could void the warranty and support and all that other proprietary nonsense. So my only choice is to
do it in the response on the server side, instead of in the response on the client side.

And, since this is going to be done by means of a virtual naming context with slapd-meta and slapo-rwm, only the OS X
clients will be using that virtual naming context. Everybody else, who does it properly on the client side, will use
the real naming context. The main reason for the OP was to verify whether or not the implementation I proposed would
actually *work*, regardless of whether or not it was advisable from a policy standpoint. If by "invalid", you mean
"will not work" (and not "it is not advisable to do so"), then that is fine and I'll find some other way around this
problem (somehow...) - so if you wouldn't mind clarifying your response, I would be appreciative.


Respectfully,
Ryan

[Ryan Steele]

Post by m***@aero.polimi.it

Post by Ryan Steele
Ok, fair warning - this is a little long-winded, but I'd rather give too
much detail than not enough. Also, all
examples are in slapd.conf format, since there is no documentation for
cn=config, and I'm using slapd with -f and -F to
make the conversion.
Anyways, I'm working on implementing some rewrite rules with slapo-rwm,
using back-relay (or possibly back-meta if that
doesn't work out) as the backend on top of which slapo-rwm will sit, to
make available some features that would be
otherwise not be present with back-ldap, or in the absence of a proxy backend altogether.
In its most basic form, the slapo-rwm overlay can be used without a proxy
or relay backend, but the feature I'm after
appears to require either the slapd-meta backend or the slapd-relay
backend. For now, I've chosen to try with
slapd-relay, as my research has indicated it supports the feature I'm
after. For reference, this feature I desire is
the 'searchResult' rewriteContext, as I'm interested in rewriting dynamic
group membership DN's (i.e., the 'member'
attribute, as defined in rfc2307bis) as plain uid's, so clients that only
know about rfc2307 (and thus, look only for
entries of the same format as memberUid and fail to perform any mappings
to translate membership DN's to UID's) can make
use of the dynamic groups I've created in my directory.
Since my slapd-hdb database's rootDSE is dc=example,dc=com, I really
couldn't find any way to get around using a virtual
naming context; creating a back-meta database on the same host with the
same suffix as the back-hdb database seemed
ambiguous at best (in terms of how to ensure that clients would always
speak to the back-meta db first), and using the
suffix "dc=com" seemed inappropriate, as did creating several relay
databases, one for each of the entries below the
slapd-hdb database's rootDSE (e.g., ou=Users,dc=example,dc=com,
ou=Groups,dc=example,dc=com, etc.). Creating several
relay databases in that fashion presents other problems as well, and
wouldn't do me any good for clients which query for
entries using dc=example,dc=com as their search base.
Given that, and the fact that I'm only looking to remap/rewrite data going
to and from a single local database with a
single naming context, I decided it was a better idea to create a single
back-relay database with a bogus "proxy" naming
database relay
suffix "dc=example-proxy,dc=com"
relay "dc=example,dc=com"
overlay rwm
rwm-rewriteEngine on
rwm-suffixmassage "dc=example-proxy,dc=com" "dc=example,dc=com"
rwm-rewriteContext searchResult
rwm-rewriteRule "^uid=([^,]+?),ou=Users,dc=example,dc=com$" "$1"

The result of this mapping does not yield a valid DN; as such, this
rewriting is incorrect. Also, I don't see how it could be useful, as LDAP
clients usually expect DNs to be DN-valued.

In this case, dscl (Mac OS X's directory services client) expects a UID, not a DN, as is the POSIX standard for group
members, and doesn't know how to parse usernames in groups that use DN's to identify their members. Instead, they
expect UID's, and since there appears to be no way to do this client side with dscl (believe me, I did try to find one
before resorting to this), I have constructed the above rewriteRule to do this for them. This should not impact my
Linux clients either, as nslcd (a daemon created in a fork of nss_ldap by Arthur de Jong) only performs DN-to-UID
translations if the group members are represented as DN-valued attribute values. My question wasn't whether or not the
result is a valid DN - it's whether or not it would actually work, functionally speaking. Is that the case, regardless
of how ugly or undesirable it might be?

Post by m***@aero.polimi.it

Post by Ryan Steele
For reference, here is the back-meta database configuration I was going to
use before I discovered that back-relay
database meta
suffix "dc=example-proxy,dc=com"
overlay rwm
rwm-rewriteEngine on
uri "ldap://localhost/dc=example-proxy,dc=com"
rwm-suffixmassage "dc=example-proxy,dc=com" "dc=example,dc=com"
rwm-rewriteContext searchResult
rwm-rewriteRule "^uid=([^,]+?),ou=Users,dc=example,dc=com$" "$1"
1. Is it legal to rewrite the search result DN's as described in the
example above? I'm hoping that since the data
being manipulated is going from server -> client and is not being stored
in the directory, the attributes I'm rewriting
would not be subjected to the syntax requirements of the entry that holds them.
2. Are there any benefits or drawbacks to using a rewriteMap to accomplish
this task? In my mind, it just seems like
another means to the same end, at least based on the following
rwm-rewriteMap ldap dn2uid
"ldap://localhost/ou=Users,dc=example,dc=com?uid?sub"
rwm-rewriteContext searchResult
rwm-rewriteRule "^uid=([^,]+?),ou=Users,dc=example,dc=com$" "${dn2uid($1)}"

Same as above. You're performing an internal search to obtain information
you already have, and cast it in invalid form.

If there was any other way at all to do this, I would, believe me. But, OS X's LDAPv3 plugin for dscl simply does not
know how to utilize DN-valued group members, and provides no way to customize the way search results are parsed. So
again, I understand that the information is not a valid DN, but again, if functionally speaking this works, then that's
what I'll have to do to leverage my rfc2307bis-style dynamic groups on OS X.

Post by m***@aero.polimi.it

Post by Ryan Steele
3. Is there a better way to go about doing this that I've failed to see or consider?

Consider why you need it: broken client? Fix the client. Anything else?
Perhaps LDAP is not what you need; in any case, move such mangling into
the client.

LDAP is what I need, and unfortunately, "fixing the client" (as in hacking and recompiling dscl's LDAPv3 plugin) would
potentially invalidate a warranty of service and support.

Post by m***@aero.polimi.it

Post by Ryan Steele
4. Considering the slapd-meta man page, which mentions that some of its
features could potentially result in excessive
overhead for some installations, are the features used in the
aforementioned configuration(s) efficient enough to scale
well in a very busy environment? I'm aware that's a pretty vague and
loaded question with lots of variables, so I'm
just looking for an educated opinion from a 10,000 foot view, assuming
excellent hardware, high network capacity, and
heavy - say, 5k, 10k, 15k queries per second - traffic. Performance will
obviously be tested before implementing this,
but it'd still be nice to have a rough idea of what to expect going in.

Pure rewriting may not impact too much; rewriteMaps that rewuire further
internal lookups may yield significant overhead. Of course, if you setup
things like ACL checking that involves significant internal searches on a
database that makes extensive use of rewriting, things can get even worse.

Great, thanks so much for that assessment, Pierangelo.

Post by m***@aero.polimi.it

Post by Ryan Steele
Lastly, on a somewhat tangential note, given that there's no documentation
with respect to cn=config for setting up
slapo-rwm (or doing it in conjunction with a proxy backend), is there any
way to discover the cn=config attributes and
objectclasses for those backend and overlays that's better or more
efficient than creating a standard slapd.conf
configuration and converting it with slapd using -f and -F? Perhaps an
official or unofficial bit of documentation
floating around out there other than a few miscellaneous postings on the
mailing lists and some ITS contents? On a
similar note in the same vein, as far as I can tell, none of the man
pages, admin guide sections, or FAQ-O-Matic entries
on slapd-relay mention which rewrite contexts are available with it, and I
didn't realize that slapd-relay accepted the
searchResult rewrite context until I read the
servers/slapd/back-relay/README file (while searching for code that might
reveal slapo-rwm's cn=config attribute equivalents). Would it be
objectionable to update these reference points with
said information? I'd be happy to file an ITS and submit patches and/or
content for updating the documentation, if
Gavin is too busy, unavailable, or would like some help.

None that I'm aware of. The FAQ is interactive; if by extensive
experimenting, or by looking at the code, you want to contribute, it'd be
more than welcome. Please note that 2.4.22 contains some improvements to
slapo-rwm cn=config handling; you may want to stick with that release.
p.

Duly noted, and thanks again for your advice here, and any future advice on this topic.


Respectfully,
Ryan