Statistics

The Statistics page offers a centralized and interactive data dashboard with monitoring and reporting features that allow you to track data flow in real-time and get automated error notifications in case something goes wrong. The data dashboard provides an easy view of the most important replication metrics at a minute or even second frequency.

Stats Job

The stats job (dssstats) is responsible for gathering data that is displayed on the Statistics and Topology pages. The stats job is created in a RUNNING state along with the repository tables during the installation of Data Source Solutions DSS. If the stats job is not manually suspended, it will always run whenever the hub server is running.

Information for the stats job is gathered from the following files:

• Data Source Solutions DSS log files, in which all data from capture/integrate/refresh jobs is written to. The script called dssstats reads information from the log files and pulls them into the statistics system.

• Router transaction files. These are the files, to which the capture job transports changes from the source location, while the integrate job reads from the router files and applies changes to the target location. The stats job does not read the transaction files, it only counts them and looks at their names.

• *.cap_state file.

The DSS_STATS repository table is responsible for maintaining statistical data and consists of a number of columns that store statistical information about data replication. The stats job modifies the DSS_STATS table based on the data read from the above files. In particular, the DSS_STATS table includes the metric_name and metric_value columns storing data on a variety of metrics captured by Data Source Solutions DSS, such as capture/integrate latency, captured row counts, integrated change counts. For a complete list of columns in the DSS_STATS repository table and the type of data stored in them, see section Repository Tables.

You can also store the statistics data in a secondary external location separate from the repository database. For more information, see Secondary Storage for Statistics.

Stats Aggregation

The stats job performs two types of aggregations when writing to the DSS_STATS table:

• Scope aggregation

  Metrics are taken from DSS log files and are written to DSS_STATS table at a certain scope defined by a channel name (column chn_name), location name (column loc_name) and table name (column tbl_name), which can be either named explicitly or regarded as '*' (which means applies to all channels, locations, tables).

  For example:

  There are 5 'captured inserts' with chn_name='chn1', loc_name='src' and tbl_name='tbl1' and 5 for 'captured inserts' with chn_name='chn1', loc_name='src' and tbl_name='tbl2'. The DSS_STATS table will store these values, but it will also store value 10 for tbl_name='*', the sum of both values.

For more information on various scopes that can be defined, see option -s of command dssstats showing how the aggregation behaves with different scopes.