Capture from PostgreSQL using Direct DBMS Log Reading - DSS 6 | Data Source Solutions Documentation

Documentation: Capture from PostgreSQL using Direct DBMS Log Reading - DSS 6 | Data Source Solutions Documentation

Capture From PostgreSQL Using Direct DBMS Log Reading

This capture method has been deprecated since 6.1.5/5 and is disabled as of 6.2.5/1. Use the Logical Replication capture method instead.

This section describes the configuration requirements for capturing changes from PostgreSQL using DIRECT capture method. In this capture method (Capture_Method=DIRECT), Data Source Solutions DSS reads transaction log records directly from the DBMS log file using the file I/O.

  • The DSS Agent must be installed on the PostgreSQL source database server.

  • PostgreSQL configuration file postgresql.conf should have the following settings:

    • wal_level = logical

      SHOW wal_level;
      
      ALTER SYSTEM SET wal_level = logical; -- server restart needed
      
    • archive_mode = on

      SHOW archive_mode;
      
      ALTER SYSTEM SET archive_mode = on; -- server restart needed
      
    • archive_command
      The value of archive_command depends on the location of the archive directory, the operating system and the way archiving is done in a PostgreSQL installation. For example:
      In Unix & Linux,

      SHOW archive_command;
      
      ALTER SYSTEM SET archive_command = 'test ! -f /var/lib/pgsql/9.5/data/archive/%f && cp %p /var/lib/pgsql/9.5/data/archive/%f';   -- server restart needed
      

      In Windows,

      SHOW archive_command;
      
      ALTER SYSTEM SET archive_command = 'copy "%p" "c:\\Program Files\\PostgreSQL\\9.5\\data\\archive\\%f"';   -- server restart needed
      
  • The location property XLOG DIRECTORY (PostgreSQL_XLog) must be defined with the directory path to the PostgreSQL transaction log file directory.

The operating system user as which DSS is running when connecting to PostgreSQL should have read permission to the files in this directory either by directly running DSS as the DBMS owner (postgres) or via a trusted executable named dss_postgres.

directory_path is the directory where the PostgreSQL transaction log files are archived (e.g. /distr/postgres/935/archive). The operating system user as which DSS is running when connecting to PostgreSQL should have read permission to the files in the directory_path either by directly running DSS as the DBMS owner (postgres) or via a trusted executable named dss_postgres.

Creating Trusted Executable

Following are the steps to create the trusted executable dss_postgres in the DSS_HOME/sbin directory,

  1. Execute the following commands while logged in as the DBMS owner (postgres):

    $ cd /usr/dss/dss_home
    $ cp bin/dss sbin/dss_postgres
    $ chmod 4755 sbin/dss_postgres
    

    If the user postgres does not have permission to write to the DSS installation directories, execute the following commands as user root:

    $ cd /usr/dss/dss_home
    $ cp /usr/dss/dss_home/bin/dss /usr/dss/dss_home/sbin/dss_postgres
    $ chown postgres:postgres /usr/dss/dss_home/sbin/dss_postgres
    $ chmod 4755 /usr/dss/dss_home/sbin/dss_postgres
    
  2. Additionally, on Linux the trusted executable needs to be patched using:

    $ /usr/dss/dss_home/lib/patchelf --set-rpath /usr/dss/dss_home/lib --force-rpath /usr/dss/dss_home/sbin/dss_postgres