Capabilities for Databricks

This section lists the Capabilities of Data Source Solutions DSS when using 'Databricks'. For more information about the pre-requisites, access privileges, and other configuration requirements, see Databricks Requirements.

Capture

DSS does not support Capture changes from location on Databricks.

Repository Database

DSS does not support Repository database on Databricks.

Integrate

DSS supports the following capabilities on Databricks:

• Integrate changes into location (Databricks from version 9.0 to version 15.0).
• Burst integration (Integrate with parameter Method=BURST) (Databricks from version 9.0 to version 15.0).
• Append integration into TimeKey (Integrate with parameter Method=APPEND) (Databricks from version 9.0 to version 15.0).
• Integrate with parameter BurstCommitFrequency (Databricks from version 9.0 to version 15.0).
• Continuous integration (Integrate with parameter Method=CONTINUOUS) (Databricks from version 9.0 to version 15.0).
• ColumnProperties with parameter SoftDelete (Databricks from version 9.0 to version 15.0).
• Bulk load requires a staging area (location property Staging_Directory) (Databricks from version 9.0 to version 15.0).
• Creation and update of DSS state tables (Databricks from version 9.0 to version 15.0).

DSS does not support the following capabilities on Databricks:

• Tables without a key and without TableProperties with NoDuplicateRows for continuous integration.
• Continuous integrate with parameter OnErrorSaveFailed.
• Disable/enable database triggers during integrate (with parameter NoTriggerFiring).
• Integrate with parameter DbProc.

Bi-directional Replication

DSS does not support Bi-directional Replication on Databricks.

Refresh and Compare

DSS supports the following capabilities on Databricks:

• dssrefresh or dsscompare from source location (Databricks from version 9.0 to version 15.0).
• dssrefresh into target location (Databricks from version 9.0 to version 15.0).
• Isolated table dssrefresh (option -i) into target location (Databricks from version 9.0 to version 15.0).
• Row-wise dssrefresh into target location (option -g) (Databricks from version 9.0 to version 15.0).

Other Capabilities

DSS supports the following capabilities on Databricks:

• International table and column names where DBMS is not configured with UTF-8 encoding (Databricks from version 9.0 to version 15.0).
• Always treat DBMS table names and column names as case sensitive, even if not configured (equivalent to location property Case_Sensitive_Names always set to TRUE) (Databricks from version 9.0 to version 15.0).
• Always treat DBMS schema names case sensitive (Databricks from version 9.0 to version 15.0).

DSS does not support the following capabilities on Databricks:

• Use distribution key for parallelizing changes within a table (ColumnProperties with parameter DistributionKey).
• Lossless binary float datatypes. No data loss when transporting float values (because base 2 fractions are never converted to base 10). The values for these datatypes, when selected back from the database, might be slightly different from the ones that were inserted. This leads to Compare differences..