This post is archived and probably outdated.

Berkeley DB 5 and PHP

2010-06-03 15:54:00

Everybody working on Unix or in the database world stumbles over Oracle Berkeley DB every now and then. DB is an Open Source embedded database used by applications like OpenLDAP or Postfix. Traditionally it followed mostly a key-value access pattern. Now what caught my attention was the fact that the recently released DB 5.0 provides an SQLite-like C API with the promise of providing better concurrency and performance than regular SQLite. Time to give it a shot.

So I grabbed the source distribution, checked the documentation and saw that I shall use the --enable-sql_compat configure option so DB creates a libsqlite3.so which can be used by PHP's build system. On my system compiling this worked like a charm and after linking PHP against it the SQLite3 extension as well as th SQLite PDO driver worked as before. I didn't do any benchmarks, yet, but I would be interested to read some results. Oh and yes, according to the documentation there are a few minor changes in behavior, this should only affect very few PHP users though.

So the whole process after extracting DB and PHP sources looked like this:

$ cd db-5.0.21/build_unix
$ ../dist/configure --prefix=/opt/db-5.0.21 --enable-sql_compat --enable-cxx
$ make
# make install
$ cd ../..
$ cd php-src
$ ./configure --with-sqlite3=/opt/db-5.0.21 --with-pdo-sqlite=/opt/db-5.0.21 --with-other-php-options
$ make
# make install

Now you don't have to use the SQLite interface for using DB but can also use PHP's dba extension. In current releases of PHP PHP expects at most DB version number 4 for the DBA driver, but a patch was committed to PHP's svn repository so upcoming PHP releases can be build using the latest DB version by adding --with-db4=/opt/db-5.0.21 to PHP's configure line.

Now I can hear you complain: "three ways to access a database isn't enough!" - which is true as the SQLite3 extension as well as the PDO SQLite driver are limited to SQLite capabilities and DBA, as an abstraction layer for Berkeley DB-like databases, only offers some basic operations, too. But there's hope:

While browsing through the DB source distribution I stumbled over a directory called php_db4 which obviously caught my further attention, so the DB developers are bundling a DB-specific PHP extension wrapping DB's native C++ API in an PHP OO API. To try it simply follow the steps from the PHP manual for compiling custom extensions:

$ cd db-5.0.21/php_db4
$ phpize
$ ./configure --with-db4=/opt/db-5.0.21
$ make
# make install

And then a quick check using reflection to see what functionality is provided by this extension:

$ php -dextension=db4.so --re db4
Extension [  extension #45 db4 version 0.9 ] {

  - Constants [210] {
    Constant [ integer DB_VERSION_MAJOR ] { 5 }
    Constant [ integer DB_VERSION_MINOR ] { 0 }
    Constant [ integer DB_VERSION_PATCH ] { 21 }
    Constant [ string DB_VERSION_STRING ] { Berkeley DB 5.0.21: (March 30, 2010) }
    Constant [ integer DB_MAX_PAGES ] { 4294967295 }
    Constant [ integer DB_MAX_RECORDS ] { 4294967295 }
    Constant [ integer DB_DBT_APPMALLOC ] { 1 }
    Constant [ integer DB_DBT_ISSET ] { 8 }
    Constant [ integer DB_DBT_MALLOC ] { 16 }
    Constant [ integer DB_DBT_PARTIAL ] { 64 }
    Constant [ integer DB_DBT_REALLOC ] { 128 }
    Constant [ integer DB_DBT_USERMEM ] { 1024 }
    Constant [ integer DB_DBT_DUPOK ] { 4 }
    Constant [ integer DB_CREATE ] { 1 }
    Constant [ integer DB_CXX_NO_EXCEPTIONS ] { 2 }
    Constant [ integer DB_FORCE ] { 1 }
    Constant [ integer DB_NOMMAP ] { 8 }
    Constant [ integer DB_RDONLY ] { 1024 }
    Constant [ integer DB_RECOVER ] { 2 }
    Constant [ integer DB_MULTIVERSION ] { 4 }
    Constant [ integer DB_TXN_SNAPSHOT ] { 2 }
    Constant [ integer DB_THREAD ] { 16 }
    Constant [ integer DB_TRUNCATE ] { 32768 }
    Constant [ integer DB_TXN_NOSYNC ] { 1 }
    Constant [ integer DB_TXN_NOT_DURABLE ] { 2 }
    Constant [ integer DB_USE_ENVIRON ] { 4 }
    Constant [ integer DB_USE_ENVIRON_ROOT ] { 8 }
    Constant [ integer DB_AUTO_COMMIT ] { 256 }
    Constant [ integer DB_DIRTY_READ ] { 512 }
    Constant [ integer DB_DEGREE_2 ] { 1024 }
    Constant [ integer DB_READ_COMMITTED ] { 1024 }
    Constant [ integer DB_READ_UNCOMMITTED ] { 512 }
    Constant [ integer DB_NO_AUTO_COMMIT ] { 8192 }
    Constant [ integer DB_RPCCLIENT ] { 1 }
    Constant [ integer DB_INIT_CDB ] { 64 }
    Constant [ integer DB_INIT_LOCK ] { 128 }
    Constant [ integer DB_INIT_LOG ] { 256 }
    Constant [ integer DB_INIT_MPOOL ] { 512 }
    Constant [ integer DB_INIT_REP ] { 1024 }
    Constant [ integer DB_INIT_TXN ] { 2048 }
    Constant [ integer DB_JOINENV ] { 0 }
    Constant [ integer DB_LOCKDOWN ] { 4096 }
    Constant [ integer DB_PRIVATE ] { 8192 }
    Constant [ integer DB_RECOVER_FATAL ] { 16384 }
    Constant [ integer DB_SYSTEM_MEM ] { 65536 }
    Constant [ integer DB_EXCL ] { 64 }
    Constant [ integer DB_FCNTL_LOCKING ] { 2048 }
    Constant [ integer DB_RDWRMASTER ] { 16384 }
    Constant [ integer DB_WRITEOPEN ] { 65536 }
    Constant [ integer DB_TXN_NOWAIT ] { 16 }
    Constant [ integer DB_TXN_SYNC ] { 4 }
    Constant [ integer DB_ENCRYPT_AES ] { 1 }
    Constant [ integer DB_CDB_ALLDB ] { 64 }
    Constant [ integer DB_DIRECT_DB ] { 128 }
    Constant [ integer DB_NOLOCKING ] { 1024 }
    Constant [ integer DB_NOPANIC ] { 2048 }
    Constant [ integer DB_OVERWRITE ] { 4096 }
    Constant [ integer DB_PANIC_ENVIRONMENT ] { 8192 }
    Constant [ integer DB_REGION_INIT ] { 16384 }
    Constant [ integer DB_TIME_NOTGRANTED ] { 32768 }
    Constant [ integer DB_TXN_WRITE_NOSYNC ] { 32 }
    Constant [ integer DB_YIELDCPU ] { 65536 }
    Constant [ integer DB_UPGRADE ] { 1 }
    Constant [ integer DB_VERIFY ] { 2 }
    Constant [ integer DB_DIRECT ] { 16 }
    Constant [ integer DB_EXTENT ] { 64 }
    Constant [ integer DB_ODDFILESIZE ] { 128 }
    Constant [ integer DB_CHKSUM ] { 8 }
    Constant [ integer DB_DUP ] { 16 }
    Constant [ integer DB_DUPSORT ] { 4 }
    Constant [ integer DB_ENCRYPT ] { 1 }
    Constant [ integer DB_RECNUM ] { 64 }
    Constant [ integer DB_RENUMBER ] { 128 }
    Constant [ integer DB_REVSPLITOFF ] { 256 }
    Constant [ integer DB_SNAPSHOT ] { 512 }
    Constant [ integer DB_STAT_CLEAR ] { 1 }
    Constant [ integer DB_JOIN_NOSORT ] { 1 }
    Constant [ integer DB_AGGRESSIVE ] { 1 }
    Constant [ integer DB_NOORDERCHK ] { 2 }
    Constant [ integer DB_ORDERCHKONLY ] { 4 }
    Constant [ integer DB_PR_PAGE ] { 16 }
    Constant [ integer DB_PR_RECOVERYTEST ] { 32 }
    Constant [ integer DB_PRINTABLE ] { 8 }
    Constant [ integer DB_SALVAGE ] { 64 }
    Constant [ integer DB_REP_NOBUFFER ] { 2 }
    Constant [ integer DB_REP_PERMANENT ] { 4 }
    Constant [ integer DB_LOCKVERSION ] { 1 }
    Constant [ integer DB_FILE_ID_LEN ] { 20 }
    Constant [ integer DB_LOCK_NORUN ] { 0 }
    Constant [ integer DB_LOCK_DEFAULT ] { 1 }
    Constant [ integer DB_LOCK_EXPIRE ] { 2 }
    Constant [ integer DB_LOCK_MAXLOCKS ] { 3 }
    Constant [ integer DB_LOCK_MINLOCKS ] { 5 }
    Constant [ integer DB_LOCK_MINWRITE ] { 6 }
    Constant [ integer DB_LOCK_OLDEST ] { 7 }
    Constant [ integer DB_LOCK_RANDOM ] { 8 }
    Constant [ integer DB_LOCK_YOUNGEST ] { 9 }
    Constant [ integer DB_LOCK_NOWAIT ] { 1 }
    Constant [ integer DB_LOCK_RECORD ] { 2 }
    Constant [ integer DB_LOCK_SET_TIMEOUT ] { 4 }
    Constant [ integer DB_LOCK_SWITCH ] { 8 }
    Constant [ integer DB_LOCK_UPGRADE ] { 16 }
    Constant [ integer DB_HANDLE_LOCK ] { 1 }
    Constant [ integer DB_RECORD_LOCK ] { 2 }
    Constant [ integer DB_PAGE_LOCK ] { 3 }
    Constant [ integer DB_LOGVERSION ] { 17 }
    Constant [ integer DB_LOGOLDVER ] { 8 }
    Constant [ integer DB_LOGMAGIC ] { 264584 }
    Constant [ integer DB_ARCH_ABS ] { 1 }
    Constant [ integer DB_ARCH_DATA ] { 2 }
    Constant [ integer DB_ARCH_LOG ] { 4 }
    Constant [ integer DB_ARCH_REMOVE ] { 8 }
    Constant [ integer DB_FLUSH ] { 1 }
    Constant [ integer DB_LOG_CHKPNT ] { 2 }
    Constant [ integer DB_LOG_COMMIT ] { 4 }
    Constant [ integer DB_LOG_NOCOPY ] { 8 }
    Constant [ integer DB_LOG_NOT_DURABLE ] { 16 }
    Constant [ integer DB_LOG_WRNOSYNC ] { 32 }
    Constant [ integer DB_user_BEGIN ] { 10000 }
    Constant [ integer DB_debug_FLAG ] { 2147483648 }
    Constant [ integer DB_LOG_DISK ] { 1 }
    Constant [ integer DB_LOG_LOCKED ] { 2 }
    Constant [ integer DB_LOG_SILENT_ERR ] { 4 }
    Constant [ integer DB_MPOOL_CREATE ] { 1 }
    Constant [ integer DB_MPOOL_LAST ] { 16 }
    Constant [ integer DB_MPOOL_NEW ] { 32 }
    Constant [ integer DB_MPOOL_DIRTY ] { 2 }
    Constant [ integer DB_MPOOL_DISCARD ] { 1 }
    Constant [ integer DB_MPOOL_NOFILE ] { 1 }
    Constant [ integer DB_MPOOL_UNLINK ] { 2 }
    Constant [ integer DB_TXNVERSION ] { 1 }
    Constant [ integer DB_GID_SIZE ] { 128 }
    Constant [ integer DB_EID_BROADCAST ] { -1 }
    Constant [ integer DB_EID_INVALID ] { -2 }
    Constant [ integer DB_REP_CLIENT ] { 1 }
    Constant [ integer DB_REP_MASTER ] { 2 }
    Constant [ integer DB_RENAMEMAGIC ] { 198656 }
    Constant [ integer DB_BTREEVERSION ] { 9 }
    Constant [ integer DB_BTREEOLDVER ] { 8 }
    Constant [ integer DB_BTREEMAGIC ] { 340322 }
    Constant [ integer DB_HASHVERSION ] { 9 }
    Constant [ integer DB_HASHOLDVER ] { 7 }
    Constant [ integer DB_HASHMAGIC ] { 398689 }
    Constant [ integer DB_QAMVERSION ] { 4 }
    Constant [ integer DB_QAMOLDVER ] { 3 }
    Constant [ integer DB_QAMMAGIC ] { 270931 }
    Constant [ integer DB_AFTER ] { 1 }
    Constant [ integer DB_APPEND ] { 2 }
    Constant [ integer DB_BEFORE ] { 3 }
    Constant [ integer DB_CONSUME ] { 4 }
    Constant [ integer DB_CONSUME_WAIT ] { 5 }
    Constant [ integer DB_CURRENT ] { 6 }
    Constant [ integer DB_FAST_STAT ] { 1 }
    Constant [ integer DB_FIRST ] { 7 }
    Constant [ integer DB_GET_BOTH ] { 8 }
    Constant [ integer DB_GET_BOTHC ] { 9 }
    Constant [ integer DB_GET_BOTH_RANGE ] { 10 }
    Constant [ integer DB_GET_RECNO ] { 11 }
    Constant [ integer DB_JOIN_ITEM ] { 12 }
    Constant [ integer DB_KEYFIRST ] { 13 }
    Constant [ integer DB_KEYLAST ] { 14 }
    Constant [ integer DB_LAST ] { 15 }
    Constant [ integer DB_NEXT ] { 16 }
    Constant [ integer DB_NEXT_DUP ] { 17 }
    Constant [ integer DB_NEXT_NODUP ] { 18 }
    Constant [ integer DB_NODUPDATA ] { 19 }
    Constant [ integer DB_NOOVERWRITE ] { 20 }
    Constant [ integer DB_NOSYNC ] { 21 }
    Constant [ integer DB_POSITION ] { 23 }
    Constant [ integer DB_PREV ] { 24 }
    Constant [ integer DB_PREV_NODUP ] { 26 }
    Constant [ integer DB_SET ] { 27 }
    Constant [ integer DB_SET_LOCK_TIMEOUT ] { 1 }
    Constant [ integer DB_SET_RANGE ] { 28 }
    Constant [ integer DB_SET_RECNO ] { 29 }
    Constant [ integer DB_SET_TXN_NOW ] { 8 }
    Constant [ integer DB_SET_TXN_TIMEOUT ] { 2 }
    Constant [ integer DB_UPDATE_SECONDARY ] { 30 }
    Constant [ integer DB_WRITECURSOR ] { 8 }
    Constant [ integer DB_WRITELOCK ] { 16 }
    Constant [ integer DB_OPFLAGS_MASK ] { 255 }
    Constant [ integer DB_MULTIPLE ] { 2048 }
    Constant [ integer DB_MULTIPLE_KEY ] { 16384 }
    Constant [ integer DB_RMW ] { 8192 }
    Constant [ integer DB_DONOTINDEX ] { -30998 }
    Constant [ integer DB_KEYEMPTY ] { -30996 }
    Constant [ integer DB_KEYEXIST ] { -30995 }
    Constant [ integer DB_LOCK_DEADLOCK ] { -30994 }
    Constant [ integer DB_LOCK_NOTGRANTED ] { -30993 }
    Constant [ integer DB_NOSERVER ] { -30990 }
    Constant [ integer DB_NOSERVER_HOME ] { -30989 }
    Constant [ integer DB_NOSERVER_ID ] { -30988 }
    Constant [ integer DB_NOTFOUND ] { -30987 }
    Constant [ integer DB_OLD_VERSION ] { -30986 }
    Constant [ integer DB_PAGE_NOTFOUND ] { -30985 }
    Constant [ integer DB_REP_DUPMASTER ] { -30984 }
    Constant [ integer DB_REP_HANDLE_DEAD ] { -30983 }
    Constant [ integer DB_REP_HOLDELECTION ] { -30982 }
    Constant [ integer DB_REP_ISPERM ] { -30980 }
    Constant [ integer DB_REP_NEWMASTER ] { -30893 }
    Constant [ integer DB_REP_NEWSITE ] { -30976 }
    Constant [ integer DB_REP_NOTPERM ] { -30975 }
    Constant [ integer DB_REP_UNAVAIL ] { -30974 }
    Constant [ integer DB_RUNRECOVERY ] { -30973 }
    Constant [ integer DB_SECONDARY_BAD ] { -30972 }
    Constant [ integer DB_VERIFY_BAD ] { -30970 }
    Constant [ integer DB_VERB_DEADLOCK ] { 1 }
    Constant [ integer DB_VERB_RECOVERY ] { 8 }
    Constant [ integer DB_VERB_REPLICATION ] { 32 }
    Constant [ integer DB_VERB_WAITSFOR ] { 32768 }
  }

  - Classes [4] {
    Class [  class db4txn ] {

      - Constants [0] {
      }

      - Static properties [0] {
      }

      - Static methods [0] {
      }

      - Properties [0] {
      }

      - Methods [8] {
        Method [  public method abort ] {
        }

        Method [  public method commit ] {
        }

        Method [  public method discard ] {
        }

        Method [  public method id ] {
        }

        Method [  public method set_timeout ] {
        }

        Method [  public method set_name ] {
        }

        Method [  public method get_name ] {
        }

        Method [  public method db4txn ] {
        }
      }
    }

    Class [  class db4cursor ] {

      - Constants [0] {
      }

      - Static properties [0] {
      }

      - Static methods [0] {
      }

      - Properties [0] {
      }

      - Methods [7] {
        Method [  public method close ] {
        }

        Method [  public method count ] {
        }

        Method [  public method del ] {
        }

        Method [  public method dup ] {
        }

        Method [  public method get ] {

          - Parameters [2] {
            Parameter #0 [  &$param0 ]
            Parameter #1 [  &$param1 ]
          }
        }

        Method [  public method put ] {
        }

        Method [  public method pget ] {

          - Parameters [2] {
            Parameter #0 [  &$param0 ]
            Parameter #1 [  &$param1 ]
          }
        }
      }
    }

    Class [  class db4 ] {

      - Constants [0] {
      }

      - Static properties [0] {
      }

      - Static methods [0] {
      }

      - Properties [0] {
      }

      - Methods [15] {
        Method [  public method db4 ] {
        }

        Method [  public method open ] {
        }

        Method [  public method close ] {
        }

        Method [  public method cursor ] {
        }

        Method [  public method del ] {
        }

        Method [  public method get ] {
        }

        Method [  public method get_encrypt_flags ] {
        }

        Method [  public method pget ] {
        }

        Method [  public method get_type ] {
        }

        Method [  public method join ] {
        }

        Method [  public method put ] {
        }

        Method [  public method set_encrypt ] {
        }

        Method [  public method stat ] {
        }

        Method [  public method sync ] {
        }

        Method [  public method truncate ] {
        }
      }
    }

    Class [  class db4env ] {

      - Constants [0] {
      }

      - Static properties [0] {
      }

      - Static methods [0] {
      }

      - Properties [0] {
      }

      - Methods [11] {
        Method [  public method db4env ] {
        }

        Method [  public method close ] {
        }

        Method [  public method dbremove ] {
        }

        Method [  public method dbrename ] {
        }

        Method [  public method get_encrypt ] {
        }

        Method [  public method open ] {
        }

        Method [  public method remove ] {
        }

        Method [  public method set_data_dir ] {
        }

        Method [  public method set_encrypt ] {
        }

        Method [  public method txn_begin ] {
        }

        Method [  public method txn_checkpoint ] {
        }
      }
    }
  }
}

The extension bundles some examples to get you started with Berkeley DB.

Even though I'd prefer a relational client-server database for most use cases I can image situations where a simple embedded solutions has its benefits, especially if combined with other Oracle Berkeley DB based applications.