Introduction

Sun Java System Directory Server (DSEE) uses an underlying Berkeley DB for storing data. This is done with an internal B Tree implementation. There are different versions used depending on what version of DSEE is used. There are a few tools that are made available by the vendor to peer into the database. One of particular interest is db_stat. This page will attempt to describe how to use this tool to provide information about databases used by a DSEE directory.

Note: Sun is making this available as it has helped us in the past on various POC and benchmarks, however, Sun provides no support for use of the db_stat utility. In addition, this utility is not made available by the packaged DSEE bits made available by Sun.

The documentation page is here .

nsslapd-db-page-size

The default page size in Directory Server is 8K (8192 bytes). The name of the attribute is nsslapd-db-page-size, in dn: cn=config,cn=ldbm database,cn=plugins,cn=config.

Background

When a directory is imported, it creates a set of ./db3 files in it's database directory. An example directory
listing of what this directory might look like for an example directory is:

# of Entries: 42.8 million
Avg entry size: 1K (without replication meta-data)
Avg entry size: 2K (with replication meta-data)

These average entry sizes are based on some calculations from LDIF exports (with and without the -r option).
So, based on this info, if we calculate 42.8 million * 2K, we our userRoot_id2entry.db3 should only be around
81.8 GB. Instead we see that it is around 210 GB.

Why the big difference?

There are several reasons (replication meta data, database fragmentation) however, the biggest reason in this
database is due to the large number of overflow pages.

Database Overflow Pages

The berkeley db utilizes a B-Tree implementation. When each DB is built, it uses some info to decide on how
to build that B-Tree. A B-Tree page can include several entries (each entry consiting of a key and data). In our
directory, for the id2entry.db3 database, the key is the DN (i.e. uid=jdoe,ou=People,dc=example,dc=com) and
the data is the entry (i.e. all the attributes, values, replication meta-data csn's).

If the size of the key or the data exceeds 25% of the page size, then instead of placing that data into that
B-Tree node, it instead creates an overflow page that will only contain that one entry. Some of the settings in a
default directory implementation are:

Default DB Record/Page Size: 8K (or 8,192 bytes)
Max Size of entry before overlow page is used: 25% of 8K = 2K or (2048 bytes)

Running db_stat

To run it, you'll need to:

$ export LD_LIBRARY_PATH=...{ds lib directory}... (location of the libdb.so file)

$ ### Stop the directory instance
$ db_stat_sparc_64 -N -d {db_location}/...id2entry.db3

Example:

# ./db_stat_sparc_64 -N -d /export01/ds6data/db/userRoot_id2entry.db3
Tue Apr 3 00:18:41 2007 Local time
53162 Btree magic number
8 Btree version number
Big-endian Byte order
record-numbers Flags
2 Minimum keys per-page
8192 Underlying database page size
4 Number of levels in the tree
43M Number of unique keys in the tree (43051538)
43M Number of data items in the tree (43051538)
15977 Number of tree internal pages
640800 Number of bytes free in tree internal pages (99% ff)
7196657 Number of tree leaf pages
2758M Number of bytes free in tree leaf pages (95% ff)
0 Number of tree duplicate pages
0 Number of bytes free in tree duplicate pages (0% ff)
18M Number of tree overflow pages (18004268)
703M Number of bytes free in tree overflow pages (99% ff)
0 Number of empty pages
409672 Number of pages on the free list

If the number of overflow pages is large (in the example over 18,000,000) increase the dbpagesize. The dbpagesize defaults to 8k (8192) and can be set to 8, 16, 32, or 64K. The database must be re-created (imported) when the pagesize is changed. Note that this is a good time to make sure one is using an appropriate value for allidsthreshold.

Notes

• Ideally, the file system block size should match the dbpagesize.
• Do not use CTRL-C to interrupt the running of db_stat
• As noted above, the version of db_stat used must match the version of the database

Contributors

From an article by Terry.Sigle@Sun.COM