Log Mode and Log Archive Retention

For DSS to capture changes from SAP NetWeaver on HANA, the automatic backup of transaction logs must be enabled in HANA. Normally, DSS reads changes from the 'online' transaction log file. However, in the event of an interruption (e.g. 2 hours), DSS must be able to read from transaction log backup files to capture older changes. Full backups are not necessary as DSS only reads transaction log backup files.

To enable automatic log backup in HANA, the log mode must be set to normal. Once the log mode is changed from overwrite to normal, a full data backup must be created. For more information, search for Log Modes in SAP HANA Documentation.

The log mode can be changed using HANA Studio. For detailed steps, search for Change Log Modes in SAP HANA Documentation. Alternatively, you can execute the following SQL statement:

alter system alter configuration ('global.ini', 'SYSTEM') set ('persistence', 'log_mode') = 'normal' with reconfigure;

Transaction Log (archive) Retention

If a backup process has already moved these files to tape and deleted them, then DSS's capture will give an error and a refresh will have to be performed before replication can be restarted. The amount of 'retention' needed (in hours or days) depends on organization factors (how real-time must it be?) and practical issues (does a refresh take 1 hour or 24 hours?).

When performing log-shipping (Archive Only capture), file names must not be changed in the process because begin-sequence and timestamp are encoded in the file name and capture uses them.

OS Level Permissions or Requirements

To capture from HANA database, DSS should be installed on the HANA database server itself, and the DSS Agent listener should be configured to accept remote connections. The operating system (OS) user under which the DSS is running should have READ permission on the HANA database files. This can typically be achieved by adding this user to the sapsys user group.

Channel Setup Requirements

It is not possible to enable 'supplemental logging' on HANA. This means that the real key values are not generally available to DSS during Capture. A workaround for this limitation is capturing the Row ID values and use them as a surrogate replication key.

The following two additional actions should be defined prior to Adding Tables to a Channel to instruct DSS to capture Row ID values and to use them as surrogate replication keys.

HANA Encrypted Log Files and Log Backups

**Since** v6.1.0/36

DSS supports capturing changes from HANA encrypted log files and log backups.

To capture from encrypted log files and log backups, the following must be configured:

• Execute the dsshanaviews.sql script under the schema with SAP tables. This script is available in the DSS_HOME/dbms/hana directory. It will create a set of views that DSS uses to read from SAP dictionaries on Netweaver HANA.

• The DSS database user must be granted the following privilege to read the encrypted log files and log backups:

  grant execute on <em>SAP_Schema</em>."/DSS/ROOT_KEYS_EXTRACT" to <em>username</em>;
  

• If using a multi-tenant configuration, the encryption configuration management should be delegated to tenants. Execute the following command on the system database to change control over to the tenant databases:

  alter system encryption configuration controlled by local databases;
  

• Set password for the encryption root keys backup in the database from which the capture is to be done:

  alter system set encryption root keys backup password <em>password</em>;
  

• Define the location property ROOT KEYS BACKUP PASSWORD (HANA_Root_Keys_Backup_Password) while creating a SAP HANA location or by editing the existing location's source and target properties). The password that was set for the encryption root keys backup in the previous step should be set for this property.

  To set this location property from the CLI, use the command dsslocationconfig:

  dsslocationconfig <em>hub</em> <em>hana_location_name</em> HANA_Root_Keys_Backup_Password=<em>password</em>;
  

Recognizing SAP Archived Records

<b>Since</b> v6.1.5/7

DSS can distinguish between records manually deleted by a user and records automatically archived by SAP NetWeaver on HANA. By recognizing deletions performed by the archiving process, DSS can mark these records as archived instead of deleted, ensuring the data remains relevant for reporting purposes.

Perform the following to enable this feature:

• Configure SAP NetWeaver on HANA database:

  1. Import the latest DSS transport files available in the DSS_HOME/dbms/netweaver directory.

  2. In the SAP NetWeaver UI, click Set archive username to add an SAP application user to the user list in /N/DSS/TRIGGERS transaction. This user will perform archive deletion in SAP NetWeaver on HANA. All deletions made by this user will be marked as archived.

The internal table /DSS/IC_ARCHBLK tracks archive deletions by creating a single record for each process. It serves only as a transactional marker and does not serve any other purposes. To prevent data accumulation, this table requires periodic cleanup. You can use the built-in functionalities of your database system to automate the cleanup of this table. Alternatively, you can manually delete old entries from the table.

• Create the SAP NetWeaver on HANA location and define the required actions in DSS:

  1. Select the Recognize SAP Archiving Process option (or define the equivalent location property SAP_Archiving) while creating a location or by editing the existing location's source and target properties.

  2. Define the following three actions on the target location:

     Location	Action	Parameter(s)
     Target	ColumnProperties	Name={col_name_for_arch_deletes} ArchiveDelete Extra
     Target	Restrict	CompareCondition=“{col_name_for_arch_deletes}=0” Context=!compare_deleted
     Target	Restrict	Context=refresh_keep_deletes RefreshCondition=“{col_name_for_arch_deletes}=0"

     These action definitions are required to create an extra column col_name_for_arch_deletes in the target location. The value populated in this column indicates the type of record deletion.

     • If a row is automatically archived by SAP, the value in this column will be set to 2.
     • If parameter SoftDelete is also defined, value 1 indicates the record was manually deleted by a user (who is not registered as an SAP application user that will perform archive deletion).
     • Value 0 indicates the record is not deleted.

If you use ArchiveDelete and SoftDelete for the target location simultaneously, the columns created for ArchiveDelete and SoftDelete must use the same column names. Otherwise, errors will occur.

 Defining [**ColumnProperties**](/docs/dss6-action-reference-columnproperties) action with **ArchiveDelete** parameter is similar to defining [**SoftDelete**](/docs/dss6-action-reference-columnproperties#softdelete) for identifying the archived deletes.

Capturing from Backint

**Since** v6.1.0/36

DSS supports capturing changes from log files stored in the Backint for SAP HANA interface.

Both capture methods (Direct DBMS Log Reading and Archive Only) are supported when the DSS and HANA database are located on the same node. However, if they are on separate nodes, only the Archive Only capture method is supported.

In order to capture data from log files stored in Backint for SAP HANA, the backup application should be installed and configured on the node where the DSS is installed.

The DSS uses the HANA system views (backup catalogs) to get the list of existing log files.

Configuration for Capturing from Backint

To enable DSS to retrieve data from log files stored in Backint for SAP HANA, the following configuration steps are required:

• If you are upgrading from 6.1.0/35 or older version of DSS, run the script dsshanaviews.sql available in the DSS_HOME/dbms/hana directory under schema with SAP tables. The script creates a set of views that DSS uses to read from SAP dictionaries on Netweaver SAP HANA.

• If DSS and HANA database are located on different nodes, define the location properties BACKINT EXECUTABLE PATH (Hana_Backint_Executable_Path) and BACKINT CONFIGURATION FILE PATH (Hana_Backint_Configuration_Path).

• Add the Backint application as a trusted external application. By default, DSS does not trust external applications.

  1. Copy the file dssosaccess_example.conf from DSS_HOME/etc to DSS_CONFIG/etc and rename it to dssosaccess.conf.

  2. Edit the dssosaccess.conf file to add the full path of the Backint application under Allowed_Plugin_Paths. Typically, the default path is /usr/sap/HDB/SYS/global/hdb/opt. Example:

     {
       ## Following section 'safelists' two directories for command
       ## execution. Otherwise LDP will only run binaries and scripts
       ## inside '$DSS_CONFIG/plugin/agent' and '$DSS_CONFIG/plugin/transform'   
       #
       Allowed_Plugin_Paths: [
       /usr/sap/HDB/SYS/global/hdb/opt
       ]
     }
     

• If DSS and Backint agent are located on the same node, the values for the location properties BACKINT EXECUTABLE PATH (Hana_Backint_Executable_Path) and BACKINT CONFIGURATION FILE PATH (Hana_Backint_Configuration_Path) are received automatically from the SAP HANA database.
  
  Note that specific configuration steps are required for the AWS Backint agent:

  1. Create a separate AWS Backint configuration file on the HANA node. For example,

     cp /hana/shared/aws-backint-agent/aws-backint-agent-config.yaml  \ 
     /hana/shared/aws-backint-agent/aws-backint-agent-config-dss.yaml
     

  2. Modify the content of the LogFile parameter in the new file. For example,

     LogFile: "/hana/shared/aws-backint-agent/aws-backint-agent-catalog-dss.log"
     

  3. Define the file path for the new file (e.g. aws-backint-agent-config-dss.yaml) in the location property BACKINT CONFIGURATION FILE PATH (Hana_Backint_Configuration_Path).

  These configuration steps are necessary because the Backint log file is created with permissions such as 622 (read-write for the owner, read for the group, and read for others). By default, the owner for the HANA database is the hdbadm user, while the DSS instance user belongs only to the same group as hdbadm. To address this issue, it is possible to use a umask policy, but it is generally considered unsafe.

  As a result of this configuration, two Backint log files are generated. The first log file records all standard backup operations. The second log file specifically contains information related to the DSS operations.

Capture Limitations

This section describes the limitations for capturing changes from SAP NetWeaver on HANA using DSS.

• DSS does not support Capture from multi-node HANA clusters.

• Until version 6.1.0/35, DSS does not support Capture from HANA encrypted logs.

HANA allows encryption of transaction logs and transaction log backups separately. So if only the transaction logs are encrypted and not the transaction log backups, then DSS can capture changes using the Archive Only method.

• Since HANA does not support supplemental logging, DSS cannot process specific actions/parameters that require the value of a column, and that column is not present in the HANA logs.

  • The following action/parameter definitions will not function due to this limitation:

    • Action CollisionDetect
    • Parameter TimeKey in action ColumnProperties
    • Parameter DbProc in action Integrate
    • Parameter Resilient in action Integrate (will not function only if a row is missing on the target)
    • Parameter BeforeUpdateColumns in action FileFormat
    • Parameter BeforeUpdateColumnsWhenChanged in action FileFormat

  • The following action/parameter definitions will not function if it requires the value of a column, and that column is not present in the HANA logs:

    • Parameter CaptureExpression in action ColumnProperties (however, CaptureExpressionType=SQL_WHERE_ROW will function normally)
    • Parameter IntegrateExpression in action ColumnProperties
    • Parameter RenameExpression in action Integrate
    • Parameter CaptureCondition in action Restrict
    • Parameter IntegrateCondition in action Restrict
    • Parameter HorizColumn in action Restrict
    • Parameter AddressTo in action Restrict

The above-mentioned Restrict parameters for compare and refresh will function normally.