Location Connection for Databricks - DSS 6 | Data Source Solutions Documentation

Documentation: Location Connection for Databricks - DSS 6 | Data Source Solutions Documentation

Location Connection for Databricks

This section describes the connection details required for creating Databricks location in Data Source Solutions DSS. For information about the pre-requisites, access privileges, and other configuration requirements for creating Databricks location, see Databricks Requirements.

If the managed secrets feature is enabled, option USE TOKEN INSTEAD is displayed in the fields designated for entering secrets.

Field	Description	Equivalent Location Property
CATALOG `Since v6.1.0/33`	Catalog name in a Unity Catalog metastore. If the target database is implemented in the Unity Catalog, and if a value is not supplied in this field, Databricks will use the default catalog hive_metastore.	Databricks_Catalog
DATABASE	Name of the database in Databricks.	Database_Name
SERVER	Hostname or IP-address of the server on which the database in Databricks is running.	Database_Host
PORT	Port number on which the Databricks server is expecting connections.	Database_Port
HTTP PATH	URL for the Databricks compute resource. For more information, refer to Databricks documentation.	Databricks_HTTP_Path
AUTHENTICATION METHOD	Authentication method for connecting DSS to Databricks server. Available options are: User Name and Password: Authenticate using the username and password. Personal Access Token: Authenticate using the Databricks personal access tokens (PAT). Client Credentials `Since v6.1.5/9`: Authenticate with a service principal using the OAuth access tokens. For more information about these authentication methods, refer to the Databricks documentation.	Databricks_Authentication_Method
USER	Username for connecting DSS to the database in Databricks. This field is enabled only if AUTHENTICATION METHOD is set to User Name and Password.	Database_User
PASSWORD	Password for the USER. This field is enabled only if AUTHENTICATION METHOD is set to User Name and Password.	Database_Password
PERSONAL ACCESS TOKEN	Databricks personal access token for your workspace. This field is enabled only if AUTHENTICATION METHOD is set to Personal Access Token.	Databricks_Personal_Access_Token
CLIENT ID `Since v6.1.5/9`	Client ID of your service principal, used to obtain the OAuth access token. This field is enabled only if AUTHENTICATION METHOD is set to Client Credentials.	Databricks_Client_Id
CLIENT SECRET KEY `Since v6.1.5/9`	Secret key associated with the CLIENT ID of your service principal. The key is used in combination with the CLIENT ID to authenticate and obtain the OAuth access token. This field is enabled only if AUTHENTICATION METHOD is set to Client Credentials.	Databricks_Client_Secret

Advanced Settings

Field Description Equivalent Location Property

Field	Description	Equivalent Location Property
LINUX / UNIX ODBC DRIVER MANAGER LIBRARY PATH	Directory path where the ODBC Driver Manager Library is installed. This field is applicable only for Linux/Unix operating system. For a default installation, the ODBC Driver Manager Library is available at /usr/lib64 and does not need to be specified. However, when UnixODBC is installed in for example /opt/unixodbc the value for this field would be /opt/unixodbc/lib.	ODBC_DM_Lib_Path
LINUX / UNIX ODBCINST	Directory path where the odbcinst.ini file is located. This field is applicable only for Linux/Unix operating system. The odbcinst.ini file should contain information about the Simba Spark ODBC Driver under the heading [Simba Spark ODBC Driver 64-bit]. For example: `[Simba Spark ODBC Driver 64-bit] Description=Simba Spark ODBC Driver (64-bit) Driver=/opt/simba/spark/lib/64/libsparkodbc_sb64.so`	ODBC_Inst
ODBC DRIVER	Name of the user defined (installed) ODBC driver used for connecting DSS to the Databricks database.	ODBC_Driver

LINUX / UNIX ODBC DRIVER MANAGER LIBRARY PATH

Directory path where the ODBC Driver Manager Library is installed. This field is applicable only for Linux/Unix operating system.

For a default installation, the ODBC Driver Manager Library is available at /usr/lib64 and does not need to be specified. However, when UnixODBC is installed in for example /opt/unixodbc the value for this field would be /opt/unixodbc/lib.

ODBC_DM_Lib_Path

LINUX / UNIX ODBCINST

Directory path where the odbcinst.ini file is located. This field is applicable only for Linux/Unix operating system.

The odbcinst.ini file should contain information about the Simba Spark ODBC Driver under the heading [Simba Spark ODBC Driver 64-bit]. For example:


[Simba Spark ODBC Driver 64-bit]
Description=Simba Spark ODBC Driver (64-bit)
Driver=/opt/simba/spark/lib/64/libsparkodbc_sb64.so

ODBC_Inst

ODBC DRIVER Name of the user defined (installed) ODBC driver used for connecting DSS to the Databricks database. ODBC_Driver

Staging

DSS supports staging for Databricks location. For information about the location properties for staging, see Staging for Databricks.