Need help syncing with syncrepl 2.3

[L.B.]

Hi;

I've finally decided to make the move to syncrepl after much delay and
procrastination. I've read the guide and also reviewed several howto's
on the topic... It still isn't running correctly for me because it
doesn't replicate a few new users I've added to the provider. Also I'm
seeing the following issue over and over (every time it tries a sync
on my 10m interval):

#########
Mar 5 20:25:19 admin-agis01 slapd2.3[6147]: do_syncrep2: rid 001
LDAP_RES_INTERMEDIATE - SYNC_ID_SET
Mar 5 20:25:19 admin-agis01 slapd2.3[6147]: syncrepl_del_nonpresent:
rid 001 be_delete
uid=airftp,ou=SystemUsers,ou=SystemAccounts,dc=swa,dc=com (0)
Mar 5 20:25:19 admin-agis01 slapd2.3[6147]: syncrepl_entry: rid 001
LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)
Mar 5 20:25:19 admin-agis01 slapd2.3[6147]: syncrepl_entry: rid 001
be_search (0)
Mar 5 20:25:19 admin-agis01 slapd2.3[6147]: syncrepl_entry: rid 001
uid=airftp,ou=SystemUsers,ou=SystemAccounts,dc=swa,dc=com
Mar 5 20:25:19 admin-agis01 slapd2.3[6147]: syncrepl_entry: rid 001 be_add (0)
Mar 5 20:25:19 admin-agis01 slapd2.3[6147]: do_syncrep2: rid 001
LDAP_RES_SEARCH_RESULT
#########

My setup is RHEL4 with Buchan's RPMs
(openldap2.3-servers-2.3.39-3.rhel4, etc.). I have a fairly simple
setup, one provider and one consumer.

Here is my provider config:
######################

include /usr/share/openldap2.3/schema/core.schema
include /usr/share/openldap2.3/schema/cosine.schema
include /usr/share/openldap2.3/schema/inetorgperson.schema
include /usr/share/openldap2.3/schema/nis.schema
include /usr/share/openldap2.3/schema/misc.schema
include /usr/share/openldap2.3/schema/corba.schema
include /usr/share/openldap2.3/schema/openldap.schema
include /usr/share/openldap2.3/schema/ppolicy.schema
include /usr/share/openldap2.3/schema/ldapns.schema

access to *
by dn.exact="cn=Replicator,dc=swa,dc=com" read
by self read
by * none break

limits group="cn=Replicator,dc=swa,dc=com"
size=unlimited
time=unlimited

access to *
by dn.exact="uid=agis-ldap,ou=SystemUsers,ou=SystemAccounts,dc=swa,dc=com"
read
by self read
by * none break

access to attrs=userPassword
by self write
by * auth

pidfile /cluster/agis-ldap/ldap-master/var/run/slapd.pid
argsfile /cluster/agis-ldap/ldap-master/var/run/slapd.args

modulepath /usr/lib/openldap2.3
moduleload ppolicy.la
moduleload syncprov.la

TLSCertificateFile /cluster/agis-ldap/ldap-master/etc/cacerts/ldap.pem
TLSCertificateKeyFile /cluster/agis-ldap/ldap-master/etc/cacerts/ldap.pem
TLSCACertificateFile /cluster/agis-ldap/ldap-master/etc/cacerts/ldap.pem

loglevel 256

database bdb
suffix "dc=swa,dc=com"
rootdn "cn=Manager,dc=swa,dc=com"
rootpw {SSHA}YADYADAYADA

directory /cluster/agis-ldap/ldap-master/var/lib/ldap

overlay ppolicy
ppolicy_default "cn=swaPasswordPolicy,ou=Policies,dc=swa,dc=com"
ppolicy_use_lockout

overlay syncprov
syncprov-checkpoint 1 10
syncprov-sessionlog 100
serverid 001

cachesize 100000
idlcachesize 100000

checkpoint 256 5

index objectClass eq
index ou,cn,mail,givenname eq,subinitial
index uidNumber,gidNumber,memberUid,loginShell eq
index uid eq,subinitial
index uniqueMember pres
index entryCSN,entryUUID eq
######################

Here is my consumer config:
######################
include /usr/share/openldap2.3/schema/core.schema
include /usr/share/openldap2.3/schema/cosine.schema
include /usr/share/openldap2.3/schema/inetorgperson.schema
include /usr/share/openldap2.3/schema/nis.schema
include /usr/share/openldap2.3/schema/misc.schema
include /usr/share/openldap2.3/schema/corba.schema
include /usr/share/openldap2.3/schema/openldap.schema
include /usr/share/openldap2.3/schema/ppolicy.schema
include /usr/share/openldap2.3/schema/ldapns.schema

access to *
by dn.exact="uid=agis-ldap,ou=SystemUsers,ou=SystemAccounts,dc=swa,dc=com"
read
by self read
by * none break

access to attrs=userPassword
by self write
by * auth


pidfile /cluster/agis-ldap/ldap-slave/var/run/slapd.pid
argsfile /cluster/agis-ldap/ldap-slave/var/run/slapd.args

modulepath /usr/lib/openldap2.3
moduleload ppolicy.la
moduleload syncprov.la

TLSCertificateFile /cluster/agis-ldap/ldap-slave/etc/cacerts/ldap.pem
TLSCertificateKeyFile /cluster/agis-ldap/ldap-slave/etc/cacerts/ldap.pem
TLSCACertificateFile /cluster/agis-ldap/ldap-slave/etc/cacerts/ldap.pem

loglevel sync

database bdb
suffix "dc=swa,dc=com"
rootdn "cn=Manager,dc=swa,dc=com"
rootpw {SSHA}YADYADAYADA

directory /cluster/agis-ldap/ldap-slave/var/lib/ldap

overlay ppolicy
ppolicy_default "cn=swaPasswordPolicy,ou=Policies,dc=swa,dc=com"
ppolicy_use_lockout

cachesize 100000
idlcachesize 100000

checkpoint 256 5

index objectClass eq
index ou,cn,mail,givenname eq,subinitial
index uidNumber,gidNumber,memberUid,loginShell eq
index uid eq,subinitial
index uniqueMember pres
index entryCSN,entryUUID eq

syncrepl rid=001
provider=ldap://ldap-agis01.mascorp.com
type=refreshOnly
interval=00:00:10:00
retry="60 10 300 +"
searchbase="dc=swa,dc=com"
filter="(objectClass=*)"
binddn="cn=Replicator,dc=swa,dc=com"
bindmethod=simple
credentials=yadayadayada
schemachecking=off
updateref ldap://ldap-agis01.mascorp.com/
######################

Any help would be much appreciated!

Thanks!!

Rafael

[Buchan Milne]

This post might be inappropriate. Click to display it.

[L. B.]

Hi Buchan - I updated the limits statement to the following:

limits dn.exact="cn=Replicator,dc=swa,dc=com"
size=unlimited
time=unlimited

and now it appears to be working as expected!

On a side note, I never received a "Size limit exceeded" using the same parameters from the syncrepl configuration (I'm under 500 entries).

Thanks!

Rafael

Below is the new output after a synchronization:

May 20 22:16:06 admin-agis01 last message repeated 3 times
May 20 22:16:48 admin-agis01 slapd2.3[32501]: do_syncrep2: rid 001 LDAP_RES_INTERMEDIATE - SYNC_ID_SET
May 20 22:16:48 admin-agis01 slapd2.3[32501]: syncrepl_del_nonpresent: rid 001 be_delete uid=dyrnaesd,ou=Software Applications,dc=swa,dc=com (0)
May 20 22:16:48 admin-agis01 slapd2.3[32501]: syncrepl_entry: rid 001 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)
May 20 22:16:48 admin-agis01 slapd2.3[32501]: syncrepl_entry: rid 001 be_search (0)
May 20 22:16:48 admin-agis01 slapd2.3[32501]: syncrepl_entry: rid 001 cn=users,ou=groups,dc=swa,dc=com
May 20 22:16:48 admin-agis01 slapd2.3[32501]: syncrepl_entry: rid 001 be_modify (0)
May 20 22:16:48 admin-agis01 slapd2.3[32501]: syncrepl_entry: rid 001 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)
May 20 22:16:48 admin-agis01 slapd2.3[32501]: syncrepl_entry: rid 001 be_search (0)
May 20 22:16:48 admin-agis01 slapd2.3[32501]: syncrepl_entry: rid 001 cn=swa,ou=groups,dc=swa,dc=com
May 20 22:16:48 admin-agis01 slapd2.3[32501]: syncrepl_entry: rid 001 be_modify (0)
May 20 22:16:48 admin-agis01 slapd2.3[32501]: syncrepl_entry: rid 001 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)
May 20 22:16:48 admin-agis01 slapd2.3[32501]: syncrepl_entry: rid 001 be_search (0)
May 20 22:16:48 admin-agis01 slapd2.3[32501]: syncrepl_entry: rid 001 uid=barreror,ou=Software Applications,dc=swa,dc=com
May 20 22:16:48 admin-agis01 slapd2.3[32501]: syncrepl_entry: rid 001 be_modify (0)
May 20 22:16:48 admin-agis01 slapd2.3[32501]: syncrepl_entry: rid 001 LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)
May 20 22:16:48 admin-agis01 slapd2.3[32501]: syncrepl_entry: rid 001 be_search (0)
May 20 22:16:48 admin-agis01 slapd2.3[32501]: syncrepl_entry: rid 001 uid=airftp,ou=SystemUsers,ou=SystemAccounts,dc=swa,dc=com
May 20 22:16:48 admin-agis01 slapd2.3[32501]: syncrepl_entry: rid 001 be_modify (0)
May 20 22:16:48 admin-agis01 slapd2.3[32501]: do_syncrep2: rid 001 LDAP_RES_SEARCH_RESULT
May 20 22:17:23 admin-agis01 slapd2.3[32501]: <= bdb_equality_candidates: (uniqueMember) not indexed

Post by Buchan Milne

Post by L.B.
Hi;
I've finally decided to make the move to syncrepl after much delay and
procrastination. I've read the guide and also reviewed several howto's
on the topic... It still isn't running correctly for me because it
doesn't replicate a few new users I've added to the provider. Also I'm
seeing the following issue over and over (every time it tries a sync

This normally indicates that the consumer didn't get the final control, usually
because it didn't have sufficient (size/time) access to get the full search
results.

Post by L.B.
#########
Mar 5 20:25:19 admin-agis01 slapd2.3[6147]: do_syncrep2: rid 001
LDAP_RES_INTERMEDIATE - SYNC_ID_SET
rid 001 be_delete
uid=airftp,ou=SystemUsers,ou=SystemAccounts,dc=swa,dc=com (0)
Mar 5 20:25:19 admin-agis01 slapd2.3[6147]: syncrepl_entry: rid 001
LDAP_RES_SEARCH_ENTRY(LDAP_SYNC_ADD)
Mar 5 20:25:19 admin-agis01 slapd2.3[6147]: syncrepl_entry: rid 001
be_search (0)
Mar 5 20:25:19 admin-agis01 slapd2.3[6147]: syncrepl_entry: rid 001
uid=airftp,ou=SystemUsers,ou=SystemAccounts,dc=swa,dc=com
Mar 5 20:25:19 admin-agis01 slapd2.3[6147]: syncrepl_entry: rid 001 be_add
(0) Mar 5 20:25:19 admin-agis01 slapd2.3[6147]: do_syncrep2: rid 001
LDAP_RES_SEARCH_RESULT
#########
My setup is RHEL4 with Buchan's RPMs
(openldap2.3-servers-2.3.39-3.rhel4, etc.).

2.3.43 has been available for a long time ...

Post by L.B.
I have a fairly simple
setup, one provider and one consumer.
######################
include /usr/share/openldap2.3/schema/core.schema
include /usr/share/openldap2.3/schema/cosine.schema
include /usr/share/openldap2.3/schema/inetorgperson.schema
include /usr/share/openldap2.3/schema/nis.schema
include /usr/share/openldap2.3/schema/misc.schema
include /usr/share/openldap2.3/schema/corba.schema
include /usr/share/openldap2.3/schema/openldap.schema
include /usr/share/openldap2.3/schema/ppolicy.schema
include /usr/share/openldap2.3/schema/ldapns.schema
access to *
by dn.exact="cn=Replicator,dc=swa,dc=com" read
by self read
by * none break
limits group="cn=Replicator,dc=swa,dc=com"
size=unlimited
time=unlimited

The intention in my limits example is that you would create a groupOfNames for
cn=Replicator, and add additional host-specific DNs to this groupOfNames
object. But, it seems you have only one cn=Replicator non-group entry, changed
the ACL appropriately, but not the limits statement.
[...]

Post by L.B.
syncrepl rid=001
provider=ldap://ldap-agis01.mascorp.com
type=refreshOnly
interval=00:00:10:00
retry="60 10 300 +"
searchbase="dc=swa,dc=com"
filter="(objectClass=*)"
binddn="cn=Replicator,dc=swa,dc=com"
bindmethod=simple
credentials=yadayadayada
schemachecking=off
updateref ldap://ldap-agis01.mascorp.com/

Assuming you have more than 500 entries, if you do a search as this syncrepl
binddn, with the rest of the search parameters based on the syncrepl
configuration, do you get all entries, or a "Size limit exceeded" ?
Regards,
Buchan