ldapsearch against AD for large groups (>1000 members)

[Antti Tikkanen]

Hi,

I am not sure if this is an OpenLDAP thing or an AD thing. When I use
ldapsearch to get the members of an AD group with more than 1000 members,
I only receive the first 1000:

# ldapsearch -LL '(cn=testgroup)'

--clip--

dn: CN=testgroup,<...>
member;range=0-999:CN=test999,<...>
member;range=0-999:CN=test998,<...>
member;range=0-999:CN=test997,<...>

and so on. This is not a thing with AD's MaxPageSize, which I have changed
accordingly and tested (I used nss_ldap without paging with variable
values for MaxPageSize and used 'getent passwd' with ~1500 users). Is
there any way I can list all the users in a group with more than 1000
members?

Best regards,
Antti Tikkanen

[Michael Ströder]

Post by Antti Tikkanen
I am not sure if this is an OpenLDAP thing or an AD thing.

Don't know the details about AD but I guess it's not an OpenLDAP
issue since I used OpenLDAP client in a test to query a group
entry with 200000 member values from Netscape DS 4.1x.

Post by Antti Tikkanen
When I use
ldapsearch to get the members of an AD group with more than 1000 members,
# ldapsearch -LL '(cn=testgroup)'
--clip--
dn: CN=testgroup,<...>
member;range=0-999:CN=test999,<...>
member;range=0-999:CN=test998,<...>
member;range=0-999:CN=test997,<...>

Seems playing with the ;range sub-type when explicitly requesting
attributes might be interesting...

But why do you want to retrieve all members? In most applications
this does not scale very well anyway.

Ciao, Michael.

[Antti Tikkanen]

Post by Michael Ströder

Post by Antti Tikkanen
When I use
ldapsearch to get the members of an AD group with more than 1000 members,
# ldapsearch -LL '(cn=testgroup)'
--clip--
dn: CN=testgroup,<...>
member;range=0-999:CN=test999,<...>
member;range=0-999:CN=test998,<...>
member;range=0-999:CN=test997,<...>

Seems playing with the ;range sub-type when explicitly requesting
attributes might be interesting...

I tried this with the following:

1) ldapsearch -LL '(cn=testgroup)' 'member;range=0-999'
2) ldapsearch -LL '(cn=testgroup)' 'member;range=0-*'
3) ldapsearch -LL '(cn=testgroup)' 'member;range=1000-1999'
4) ldapsearch -LL '(cn=testgroup)' 'member;range=1000-*'
5) ldapsearch -LL '(cn=testgroup)' 'member;range=0-999' 'member;range=1000-1999'
6) ldapsearch -LL '(cn=testgroup)' 'member;range=0-999' 'member;range=1000-*'

First two will return the 1000 first users. Numbers 3&4 will return the
500 remaining users (I have a total of 1500), but not the first 1000.
Numbers 5&6 will return only the first 1000 users. I see no way of getting
all the users with one query.. anyone?

Seems this is indeed an AD thing, though.

Best regards,
Antti

[Dave Snoopy]

Michael,

I posted to the newsgroup about this same issue a
couple of weeks ago. The 1,000 entires is a maximum
page size that AD will send in responses. It can be
changed if you have admin access on your PDC.

Better yet though, you can look at the code for the
program "net" which comes with Samba 3.0 Alpha 17. It
includes a working example of how to do a "paged"
query against AD, and thus request the remainder of
the results. The name of the function which does it is
"ads_do_paged_search".

--Dave

Post by Antti Tikkanen

Post by Antti Tikkanen
I am not sure if this is an OpenLDAP thing or an

AD thing.
Don't know the details about AD but I guess it's not
an OpenLDAP
issue since I used OpenLDAP client in a test to
query a group
entry with 200000 member values from Netscape DS
4.1x.

Post by Antti Tikkanen
When I use
ldapsearch to get the members of an AD group with

more than 1000 members,

Post by Antti Tikkanen
# ldapsearch -LL '(cn=testgroup)'
--clip--
dn: CN=testgroup,<...>
member;range=0-999:CN=test999,<...>
member;range=0-999:CN=test998,<...>
member;range=0-999:CN=test997,<...>

Seems playing with the ;range sub-type when
explicitly requesting
attributes might be interesting...
But why do you want to retrieve all members? In most
applications
this does not scale very well anyway.
Ciao, Michael.

__________________________________________________
Do You Yahoo!?
Sign up for SBC Yahoo! Dial - First Month Free
http://sbc.yahoo.com

[Antti Tikkanen]

Post by Antti Tikkanen
Hi,
I am not sure if this is an OpenLDAP thing or an AD thing. When I use
ldapsearch to get the members of an AD group with more than 1000 members,
# ldapsearch -LL '(cn=testgroup)'
--clip--
dn: CN=testgroup,<...>
member;range=0-999:CN=test999,<...>
member;range=0-999:CN=test998,<...>
member;range=0-999:CN=test997,<...>

Getting back to this, I think I know what the problem is. A different
thing is what to do about it.

I gather that MS is using the Internet Draft:

"Incremental Retrieval of Multi-valued Properties"
(http://http://www.watersprings.org/pub/id/draft-kashi-incremental-00.txt
is the only link I found valid)

Probably no one else supports this? Nor ever will? At least not OpenLDAP?
The draft has expired in May 2002, and is written by Microsoft people.. :)

Regards,
Antti

[Luke Howard]

Aren't you confusing paged results with the range attribute description?

-- Luke

Subject: Re: ldapsearch against AD for large groups (>1000 members)
Date: Mon, 8 Jul 2002 09:52:06 -0700 (PDT)
Michael,
I posted to the newsgroup about this same issue a
couple of weeks ago. The 1,000 entires is a maximum
page size that AD will send in responses. It can be
changed if you have admin access on your PDC.
Better yet though, you can look at the code for the
program "net" which comes with Samba 3.0 Alpha 17. It
includes a working example of how to do a "paged"
query against AD, and thus request the remainder of
the results. The name of the function which does it is
"ads_do_paged_search".
--Dave

Post by Antti Tikkanen

Post by Antti Tikkanen
I am not sure if this is an OpenLDAP thing or an

AD thing.
Don't know the details about AD but I guess it's not
an OpenLDAP
issue since I used OpenLDAP client in a test to
query a group
entry with 200000 member values from Netscape DS
4.1x.

Post by Antti Tikkanen
When I use
ldapsearch to get the members of an AD group with

more than 1000 members,

Post by Antti Tikkanen
# ldapsearch -LL '(cn=testgroup)'
--clip--
dn: CN=testgroup,<...>
member;range=0-999:CN=test999,<...>
member;range=0-999:CN=test998,<...>
member;range=0-999:CN=test997,<...>

Seems playing with the ;range sub-type when
explicitly requesting
attributes might be interesting...
But why do you want to retrieve all members? In most
applications
this does not scale very well anyway.
Ciao, Michael.

__________________________________________________
Do You Yahoo!?
Sign up for SBC Yahoo! Dial - First Month Free
http://sbc.yahoo.com

--
Luke Howard | lukehoward.com
PADL Software | www.padl.com