syncrepl LDIF kickstart file

[Scott Classen]

Hello openldap community,

I have openldap 2.4.6 running on 2 machines.

one master server with a BDB database acting as the syncrepl provider (the syncrep[l overlay has been added to the database configuration directive).

I now have set up a second machine also running openldap 2.4.6 and I've been playing around with it trying to get it to act as a syncrepl consumer.

I remember reading somewhere in the openldap documentation that it should be fairly straight forward with a simple 10-12 line LDIF file to get the synrepl consumer synched up with the provider.

Can someone please post an example of such a simple kickstart file?

Thanks ^6,

Scott

[Buchan Milne]

Post by Scott Classen
Hello openldap community,
I have openldap 2.4.6 running on 2 machines.
one master server with a BDB database acting as the syncrepl provider (the
syncrep[l overlay has been added to the database configuration directive).
I now have set up a second machine also running openldap 2.4.6 and I've
been playing around with it trying to get it to act as a syncrepl consumer.
I remember reading somewhere in the openldap documentation that it should
be fairly straight forward with a simple 10-12 line LDIF file to get the
synrepl consumer synched up with the provider.

Uh, if you are using a configuration file (and not back-config, and implying
that you want to bootstrap the configuration via sync-repl too), then
assuming that your configuration is correct, starting the consumer should
result in it doing an initial sync.

Maybe you could provide more details on your configuration if you don't come
right.

Regards,
Buchan

[Howard Chu]

Post by Scott Classen
Hello openldap community,
I have openldap 2.4.6 running on 2 machines.
one master server with a BDB database acting as the syncrepl provider (the

syncrep[l overlay has been added to the database configuration directive).

Post by Scott Classen
I now have set up a second machine also running openldap 2.4.6 and I've
been

playing around with it trying to get it to act as a syncrepl consumer.

Post by Scott Classen
I remember reading somewhere in the openldap documentation that it should
be

fairly straight forward with a simple 10-12 line LDIF file to get the synrepl
consumer synched up with the provider.

Post by Scott Classen
Can someone please post an example of such a simple kickstart file?
Thanks ^6,

Sounds vaguely like you want this
(from http://www.openldap.org/pub/hyc/LDAPcon2007s.pdf page 20)

###
dn: cn=config
objectclass: olcglobal
cn: config

dn: olcdatabase={0}config,cn=config
objectclass: olcdatabaseconfig
olcdatabase: {0}config
olcsyncrepl: rid=001 provider=$URI binddn="cn=config" bindmethod=simple
credentials=$CONFIGPW searchbase="cn=config" type=refreshOnly
interval=00:00:00:10
###

Slapadding this will fully initialize a new server if you point it at an
existing provider, but of course you need to have the syncprov overlay
configured on the config database of the provider.

See test049 in the test suite for a more detailed example.

[Michael Ströder]

Post by Howard Chu
olcsyncrepl: rid=001 provider=$URI binddn="cn=config" bindmethod=simple
credentials=$CONFIGPW searchbase="cn=config" type=refreshOnly

Just to make it very clear avoiding confusion. Is this LDIF snippet
directly usable? I guess $URI and $CONFIGPW have to be replaced by valid
values.

Ciao, Michael.

[Howard Chu]

Post by Michael Ströder

Post by Howard Chu
olcsyncrepl: rid=001 provider=$URI binddn="cn=config" bindmethod=simple
credentials=$CONFIGPW searchbase="cn=config" type=refreshOnly

Just to make it very clear avoiding confusion. Is this LDIF snippet
directly usable? I guess $URI and $CONFIGPW have to be replaced by valid
values.

Correct. It's just an example, you have to fill in the proper values yourself.
That's why I said to look at test049, which has an actual working config.

[Scott Classen]

Post by Scott Classen

Post by Scott Classen
Hello openldap community,
I have openldap 2.4.6 running on 2 machines.
one master server with a BDB database acting as the syncrepl

provider (the
syncrep[l overlay has been added to the database configuration
directive).>

Post by Scott Classen
I now have set up a second machine also running openldap 2.4.6

and I've

Post by Scott Classen
been

playing around with it trying to get it to act as a syncrepl consumer.

Post by Scott Classen
I remember reading somewhere in the openldap documentation that

it should

Post by Scott Classen
be

fairly straight forward with a simple 10-12 line LDIF file to get
the synrepl
consumer synched up with the provider.

Post by Scott Classen
Can someone please post an example of such a simple kickstart file?
Thanks ^6,

Sounds vaguely like you want this
(from <A HREF="http://www.openldap.org/pub/hyc/LDAPcon2007s.pdf" target="l">http://www.openldap.org/pub/hyc/LDAPcon2007s.pdf</A> page 20)
###
dn: cn=config
objectclass: olcglobal
cn: config
dn: olcdatabase={0}config,cn=config
objectclass: olcdatabaseconfig
olcdatabase: {0}config
olcsyncrepl: rid=001 provider=$URI binddn="cn=config"
bindmethod=simple&nbsp;&nbsp; credentials=$CONFIGPW searchbase="cn=config"
type=refreshOnly&nbsp;&nbsp; interval=00:00:00:10
###
Slapadding this will fully initialize a new server if you point it
at an
existing provider, but of course you need to have the syncprov
overlay
configured on the config database of the provider.
See test049 in the test suite for a more detailed example.

Yes, that is what I am interested in.

I added the syncprov overlay to the cn=config directive on the PROVIDER as such:

cn=config
olcDatabase={0}config
olcOverlay={0}syncprov

Now on a brand new CONSUMER machine I created the following LDIF file (sync-seed.ldif):

dn: cn=config
objectClass: olcGlobal
cn: config

dn: olcDatabase={0}config,cn=config
objectClass: olcDatabaseConfig
olcDatabase: {0}config
olcRootDN: cn=ldapadmin,cn=config
olcRootPW: {SSHA}mysoopersecretpasswd
olcsyncrepl: rid=001 provider=ldap://my.provider.machine binddn="cn=ldapadmin,cn=config" bindmethod=simple
credentials=secret searchbase="cn=config" type=refreshOnly
interval=00:00:00:10


I then put the file in my openldap dir and cd there:
cd /usr/local/etc/openldap
mkdir slapd.d
slapadd -b "cn=config" -F slapd.d -l sync-seed.ldif


all is OK

I then start slapd:
/usr/sbin/slapd -d 256
...
slapd starting
syncrepl_message_to_entry: rid=001 mods check (olcDbConfig: value #6 provided more than once)
do_syncrepl: rid=001 quitting


Hmmm what have I done wrong?

Thanks,
Scott

[Gavin Henry]

Post by Scott Classen

Post by Scott Classen

Post by Scott Classen
Hello openldap community,
I have openldap 2.4.6 running on 2 machines.
one master server with a BDB database acting as the syncrepl

provider (the
syncrep[l overlay has been added to the database configuration
directive).>

Post by Scott Classen
I now have set up a second machine also running openldap 2.4.6

and I've

Post by Scott Classen
been

playing around with it trying to get it to act as a syncrepl consumer.

Post by Scott Classen
I remember reading somewhere in the openldap documentation that

it should

Post by Scott Classen
be

fairly straight forward with a simple 10-12 line LDIF file to get
the synrepl
consumer synched up with the provider.

Post by Scott Classen
Can someone please post an example of such a simple kickstart file?
Thanks ^6,

Sounds vaguely like you want this
(from <A HREF="http://www.openldap.org/pub/hyc/LDAPcon2007s.pdf" target="l">http://www.openldap.org/pub/hyc/LDAPcon2007s.pdf</A> page 20)
###
dn: cn=config
objectclass: olcglobal
cn: config
dn: olcdatabase={0}config,cn=config
objectclass: olcdatabaseconfig
olcdatabase: {0}config
olcsyncrepl: rid=001 provider=$URI binddn="cn=config"
bindmethod=simple&nbsp;&nbsp; credentials=$CONFIGPW searchbase="cn=config"
type=refreshOnly&nbsp;&nbsp; interval=00:00:00:10
###
Slapadding this will fully initialize a new server if you point it
at an
existing provider, but of course you need to have the syncprov
overlay
configured on the config database of the provider.
See test049 in the test suite for a more detailed example.

Yes, that is what I am interested in.
cn=config
olcDatabase={0}config
olcOverlay={0}syncprov
dn: cn=config
objectClass: olcGlobal
cn: config
dn: olcDatabase={0}config,cn=config
objectClass: olcDatabaseConfig
olcDatabase: {0}config
olcRootDN: cn=ldapadmin,cn=config
olcRootPW: {SSHA}mysoopersecretpasswd
olcsyncrepl: rid=001 provider=ldap://my.provider.machine binddn="cn=ldapadmin,cn=config" bindmethod=simple
credentials=secret searchbase="cn=config" type=refreshOnly
interval=00:00:00:10
cd /usr/local/etc/openldap
mkdir slapd.d
slapadd -b "cn=config" -F slapd.d -l sync-seed.ldif
all is OK
/usr/sbin/slapd -d 256
...
slapd starting
syncrepl_message_to_entry: rid=001 mods check (olcDbConfig: value #6 provided more than once)
do_syncrepl: rid=001 quitting
Hmmm what have I done wrong?
Thanks,
Scott

Can you slapcat your config back out and show us everything?

slapdcat -n 0 > config.ldif

/usr/sbin/slapd not /usr/local/libexec/slapd ?
--
Kind Regards,

Gavin Henry.
OpenLDAP Engineering Team.

E ***@OpenLDAP.org

Community developed LDAP software.

http://www.openldap.org/project/

[Scott Classen]

Post by Gavin Henry

Post by Howard Chu
cn=config
olcDatabase={0}config
olcOverlay={0}syncprov
dn: cn=config
objectClass: olcGlobal
cn: config
dn: olcDatabase={0}config,cn=config
objectClass: olcDatabaseConfig
olcDatabase: {0}config
olcRootDN: cn=ldapadmin,cn=config
olcRootPW: {SSHA}mysoopersecretpasswd
olcsyncrepl: rid=001 provider=ldap://my.provider.machine
binddn="cn=ldapadmin,cn=config" bindmethod=simple
credentials=secret searchbase="cn=config" type=refreshOnly
interval=00:00:00:10
cd /usr/local/etc/openldap
mkdir slapd.d
slapadd -b "cn=config" -F slapd.d -l sync-seed.ldif
all is OK
/usr/sbin/slapd -d 256
...
slapd starting
syncrepl_message_to_entry: rid=001 mods check (olcDbConfig: value
#6 provided more than once)
do_syncrepl: rid=001 quitting
Hmmm what have I done wrong?
Thanks,
Scott

Can you slapcat your config back out and show us everything?
slapdcat -n 0 > config.ldif
/usr/sbin/slapd not /usr/local/libexec/slapd ?

I have a symlink so ... yes, I'm using the right slapd (2.4.6)

here is the output from

slapdcat -n 0 > config.ldif
on the CONSUMER machine

dn: cn=config
objectClass: olcGlobal
cn: config
structuralObjectClass: olcGlobal
entryUUID: 2a37e848-2cba-102c-9007-85f7c9c1cd1d
creatorsName: cn=config
modifiersName: cn=config
createTimestamp: 20071121201453Z
modifyTimestamp: 20071121201453Z
entryCSN: 20071121201453.411262Z#000000#000#000000

dn: cn=schema,cn=config
objectClass: olcSchemaConfig
cn: schema

snip snip snip
lots of schema definitions
snip snip snip



dn: olcDatabase={-1}frontend,cn=config
objectClass: olcDatabaseConfig
objectClass: olcFrontendConfig
olcDatabase: {-1}frontend
olcLastMod: TRUE
olcMaxDerefDepth: 0
olcReadOnly: FALSE
olcSchemaDN: cn=Subschema
olcMonitoring: FALSE
structuralObjectClass: olcDatabaseConfig
entryUUID: 2a37fc84-2cba-102c-9009-85f7c9c1cd1d
creatorsName: cn=config
createTimestamp: 20071121201453Z
entryCSN: 20071121201453.412021Z#000000#000#000000
modifiersName: cn=config
modifyTimestamp: 20071121201453Z

dn: olcDatabase={0}config,cn=config
objectClass: olcDatabaseConfig
olcDatabase: {0}config
olcSyncrepl: {0}rid=001 provider=ldap://myprovidermachine:389
binddn="cn=ldap
admin,cn=config" bindmethod=simple credentials=secret
searchbase="cn=config"
type=refreshOnly interval=00:00:00:10
structuralObjectClass: olcDatabaseConfig
entryUUID: 2a37f888-2cba-102c-9008-85f7c9c1cd1d
creatorsName: cn=config
modifiersName: cn=config
createTimestamp: 20071121201453Z
modifyTimestamp: 20071121201453Z
entryCSN: 20071121201453.411896Z#000000#000#000000

dn: olcOverlay={0}syncprov,olcDatabase={0}config,cn=config
objectClass: olcOverlayConfig
objectClass: olcSyncProvConfig
olcOverlay: {0}syncprov
olcSpCheckpoint: 20 600
olcSpSessionlog: 100
structuralObjectClass: olcSyncProvConfig
entryUUID: fd14c728-2bdd-102c-8b89-17e8c59360fa
creatorsName: cn=ldapadmin,cn=config
createTimestamp: 20071120175848Z
entryCSN: 20071120175848.403267Z#000000#000#000000
modifiersName: cn=ldapadmin,cn=config
modifyTimestamp: 20071120175848Z


I've been thinking about this some and I am still confused about what
is probably a fairly simple syncrepl concept.

What I have been trying to do is synchronize the cn=config base from
the PROVIDER to the CONSUMER. My hope was that by replicating
cn=config then all the other databases (well, really only the primary
BDB for now) would then be automagically synched too. Does this make
sense? My concern is that I am using TLS and currently the names of
the crt and key files are different for the PROVIDER and CONSUMER so
simply replicating the cn=config may not actually work unless I remain
consistent in my naming of the SSL files. I guess I can do this, but I
thought to clarify the idea with the openldap experts first.

I'm still hopeful. I really like the idea of building a new machine,
compiling openldap, slapadding a seed LDIF file and instantly having a
backup slave LDAP server.

Thanks,
Scott

[Gavin Henry]

Post by Scott Classen
I've been thinking about this some and I am still confused about what
is probably a fairly simple syncrepl concept.
What I have been trying to do is synchronize the cn=config base from
the PROVIDER to the CONSUMER. My hope was that by replicating
cn=config then all the other databases (well, really only the primary
BDB for now) would then be automagically synched too. Does this make
sense?

Rather than try to debug all of above, I'm stepping back and trying to
understand what you are doing.

So you are trying to sync the providers config to the consumer?

This is like MirrorMode or Multi-Master.

If you have a master and a slave (the slave with a syncprov overlay on),
then plan to bootstrap from the slave for another slave, then that makes
sense, i.e. what you plan above is ok.

Can you clarify?

Post by Scott Classen
My concern is that I am using TLS and currently the names of
the crt and key files are different for the PROVIDER and CONSUMER so
simply replicating the cn=config may not actually work unless I remain
consistent in my naming of the SSL files. I guess I can do this, but I
thought to clarify the idea with the openldap experts first.
I'm still hopeful. I really like the idea of building a new machine,
compiling openldap, slapadding a seed LDIF file and instantly having a
backup slave LDAP server.
Thanks,
Scott

[Scott Classen]

Post by Gavin Henry

Post by Scott Classen
I've been thinking about this some and I am still confused about what
is probably a fairly simple syncrepl concept.
What I have been trying to do is synchronize the cn=config base from
the PROVIDER to the CONSUMER. My hope was that by replicating
cn=config then all the other databases (well, really only the primary
BDB for now) would then be automagically synched too. Does this make
sense?

Rather than try to debug all of above, I'm stepping back and trying to
understand what you are doing.
So you are trying to sync the providers config to the consumer?
This is like MirrorMode or Multi-Master.
If you have a master and a slave (the slave with a syncprov overlay on),
then plan to bootstrap from the slave for another slave, then that makes
sense, i.e. what you plan above is ok.
Can you clarify?

OK. The BIG picture. I am running a small operation here. I will
probably never have more than 100 users in my LDAP directory.
I have one machine that is currently running openldap 2.4.6 as my one
and only (i.e. Master) LDAP server.
Now that LDAP is up and running I am starting to slowly migrate
existing users over to the LDAP directory.
It is absolutely imperative that if the master server becomes
unavailable that users can still function.
Therefore, I was trying to set up a backup LDAP server. My research
has led me to the conclusion that I should be using syncrepl (of the
regular sort ... not mirrormode or multi-master) to create my second
backup LDAP server.

I thought by installing openldap 2.4.6 on a second machine and using a
short 10-15 line seed.ldif file it would talk to the master LDAP
server, get the cn=config from the master and see that in addition to
the cn=config base there is also another BDB base
(dc=example,dc=als,dc=lbl,dc=gov), and then it would sync up that guy
too.

Post by Gavin Henry

Post by Scott Classen
My concern is that I am using TLS and currently the names of
the crt and key files are different for the PROVIDER and CONSUMER so
simply replicating the cn=config may not actually work unless I remain
consistent in my naming of the SSL files. I guess I can do this, but I
thought to clarify the idea with the openldap experts first.
I'm still hopeful. I really like the idea of building a new machine,
compiling openldap, slapadding a seed LDIF file and instantly
having a
backup slave LDAP server.
Thanks,
Scott

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Scott Classen, Ph.D.
SIBYLS Beamline 12.3.1
http://bl1231.als.lbl.gov
Advanced Light Source
Lawrence Berkeley National Laboratory
1 Cyclotron Rd
MS6R2100
Berkeley, CA 94720
O) 510.495.2697
Beamline) 510.495.2134
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[Gavin Henry]

<quote who="Greg Martin">

Post by Gavin Henry
Can you slapcat your config back out and show us everything?
slapdcat -n 0 > config.ldif
/usr/sbin/slapd not /usr/local/libexec/slapd ?

Gavin, this is slightly of-topic. I'm curious as to your statement above.
my Slackware package installs slapd to /usr/libexec/slapd. There is no
other version. Your statement seems to imply there could be two
versions. Can you help me understand why?

Default configure; make depend; make and make install installs slapd into
/usr/local/libexec

Distros put the binaries where they see fit.

And while I'm at it, can someone explain how slapd knows to act
differently when called from one link, say, slapcat, over another, say
slapdn. I've been using linux for 5-6 years and have not run across
this particular behavior. Does slapd read argv(0) and act differently
based on that?

symlinks

Test:

[***@suretec ~]$ cat test.pl
#!/usr/bin/perl

use strict;
use warnings;

print $0, "\n";

[***@suretec ~]$ ./test.pl
./test.pl
[***@suretec ~]$ ln -s test.pl slapcat
[***@suretec ~]$ ./slapcat
./slapcat

[***@suretec ~]$ ls -l /usr/local/sbin/slapcat
lrwxrwxrwx 1 root root 16 2007-10-31 19:47 /usr/local/sbin/slapcat ->
../libexec/slapd

[Gavin Henry]

<quote who="Scott Classen">

Post by Scott Classen

Post by Gavin Henry

Post by Scott Classen
I've been thinking about this some and I am still confused about what
is probably a fairly simple syncrepl concept.
What I have been trying to do is synchronize the cn=config base from
the PROVIDER to the CONSUMER. My hope was that by replicating
cn=config then all the other databases (well, really only the primary
BDB for now) would then be automagically synched too. Does this make
sense?

Rather than try to debug all of above, I'm stepping back and trying to
understand what you are doing.
So you are trying to sync the providers config to the consumer?
This is like MirrorMode or Multi-Master.
If you have a master and a slave (the slave with a syncprov overlay
on),
then plan to bootstrap from the slave for another slave, then that
makes
sense, i.e. what you plan above is ok.
Can you clarify?

OK. The BIG picture. I am running a small operation here. I will
probably never have more than 100 users in my LDAP directory.
I have one machine that is currently running openldap 2.4.6 as my one
and only (i.e. Master) LDAP server.
Now that LDAP is up and running I am starting to slowly migrate
existing users over to the LDAP directory.
It is absolutely imperative that if the master server becomes
unavailable that users can still function.
Therefore, I was trying to set up a backup LDAP server. My research
has led me to the conclusion that I should be using syncrepl (of the
regular sort ... not mirrormode or multi-master) to create my second
backup LDAP server.

Ok.

Post by Scott Classen
I thought by installing openldap 2.4.6 on a second machine and using a
short 10-15 line seed.ldif file it would talk to the master LDAP
server, get the cn=config from the master and see that in addition to
the cn=config base there is also another BDB base
(dc=example,dc=als,dc=lbl,dc=gov), and then it would sync up that guy
too.

But if you replicate the config, you are creating another master. You have
to create a seperate config for the slave. There is no harm from then on
to create a seed.ldif and replicate your cn=config from another slave.

Gavin.

[Scott Classen]

Post by Gavin Henry

Post by Scott Classen
I thought by installing openldap 2.4.6 on a second machine and using a
short 10-15 line seed.ldif file it would talk to the master LDAP
server, get the cn=config from the master and see that in addition to
the cn=config base there is also another BDB base
(dc=example,dc=als,dc=lbl,dc=gov), and then it would sync up that guy
too.

But if you replicate the config, you are creating another master. You have
to create a seperate config for the slave. There is no harm from then on
to create a seed.ldif and replicate your cn=config from another slave.
Gavin.

Yes, I see your point. This was a concern of mine. I was suspicious
that simply replicating the cn=config from the master would not work
as a seed for a slave I was envisioning.

So I need a slave-specific cn=config?

Is it possible to store a second cn=config on the master server that
is specific for slave machines?
How would you do this? What would it look like?

Could I make a cn=slave,cn=config and use that as the cn=config for
all slave machines.

Thanks,
Scott

[Quanah Gibson-Mount]

Is it possible to store a second cn=config on the master server that is
specific for slave machines?
How would you do this? What would it look like?

I worked on doing this a long time back, but haven't revisited it yet. It
wasn't possible in 2.3. You can see some detail in ITS#4211.

--Quanah

--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc
--------------------
Zimbra :: the leader in open source messaging and collaboration

[Gavin Henry]

<quote who="Scott Classen">

Post by Scott Classen

Post by Gavin Henry

Post by Scott Classen
I thought by installing openldap 2.4.6 on a second machine and
using a
short 10-15 line seed.ldif file it would talk to the master LDAP
server, get the cn=config from the master and see that in addition to
the cn=config base there is also another BDB base
(dc=example,dc=als,dc=lbl,dc=gov), and then it would sync up that guy
too.

But if you replicate the config, you are creating another master.
You have
to create a seperate config for the slave. There is no harm from
then on
to create a seed.ldif and replicate your cn=config from another slave.
Gavin.

Yes, I see your point. This was a concern of mine. I was suspicious
that simply replicating the cn=config from the master would not work
as a seed for a slave I was envisioning.
So I need a slave-specific cn=config?

Or just one master to replicate from.

Post by Scott Classen
Is it possible to store a second cn=config on the master server that
is specific for slave machines?
How would you do this? What would it look like?

Hmmm..., you could do (not tried this):

slapcat -n 0

Do this on a configured slave to get the right config.ldif etc.

change all DNs to whatever you plan to store the data under, and then use
the rwm overlay to deliver it all out of the directory in the original
cn=config format.

You could then replicate from that.

Just a thought.

Post by Scott Classen
Could I make a cn=slave,cn=config and use that as the cn=config for
all slave machines.
Thanks,
Scott