Databricks

This guide sets up the Databricks connector using an account-level service principal with OAuth machine-to-machine (M2M) authentication. It works for Databricks on AWS, Azure, and Google Cloud. Datalogz reads metadata from your workspaces and Unity Catalog; it never accesses the data inside your tables.

Step 1: Create a service principal in Databricks

1. In the Databricks account console, create a new service principal.

2. Grant it the account admin role. This is required for Datalogz to enumerate workspaces and read Unity Catalog metadata.

3. Generate an OAuth secret for the service principal and note the Client ID and Secret.

Step 2: Create the connector in Control Tower

1. Sign in to Datalogz Control Tower and go to Settings > BI Connectors.

2. Click New Connector and select Databricks.

3. Enter your Databricks account ID and the service principal's Client ID and Secret, then click Connect.

4. Select the workspaces you want to monitor.

5. Select a SQL warehouse for metadata queries. Serverless warehouses are preferred and selected automatically when available.

6. Enter a Connector Name and select the snapshot frequency (for example Daily or Weekly).

7. Click Next, then Confirm and Finish.

Control Tower ingests workspaces, Unity Catalog tables and columns, volumes, functions, Genie (AI/BI) Spaces, and SQL warehouses. Large Unity Catalog estates can take a while on the first sync.

For questions or help with this setup, contact [e-mail].