Skip to main content
Connect Databricks to OPTIMAZE with a service principal that can read the system billing tables. OPTIMAZE imports usage and cost by workspace and SKU, priced at your account prices where Databricks has them and at list prices otherwise.
Databricks is in beta. The tile on the Integrations page is marked Beta.

​
Prerequisites

  • Access to the Databricks account console, to create a service principal.
  • A workspace with system tables enabled and a SQL warehouse the service principal can use.

​
Set up the Databricks integration

1

Create a service principal in Databricks

  1. In the Databricks account console, open User management, then Service principals, and add a service principal.
  2. On the service principal, generate an OAuth secret and copy the client ID and secret. The secret is shown only once, so note its expiry date too.
  3. Assign the service principal to the workspace.
  4. Grant it SELECT on system.billing.usage and system.billing.list_prices.
  5. Grant it CAN USE on a SQL warehouse.
  6. Copy your account ID, the workspace URL and the SQL warehouse ID. The warehouse ID is on the warehouse’s Connection details tab.
2

Enter the details in OPTIMAZE

Under Data, select Integrations. On the Databricks tile, select Connect, then fill in the form.
3

Connect

Select Connect.

​
What happens next

When you select Connect, OPTIMAZE runs a connection check before it starts importing. Each part of the check shows as passed, failed or not checked, and a later part is not checked when an earlier one fails. Once the check passes, the integration appears under Connected on the Integrations page. OPTIMAZE imports the previous six months of history on the first sync, and each later sync re-reads the last three days to pick up late changes. If you entered an expiry date, the integration shows Secret expiring soon 30 days before it, and Secret expired after it. Generate a new secret in Databricks and update the integration before then.

​
Troubleshooting

  • Authentication fails: the client ID or secret is wrong, or the secret has expired.
  • Access fails: the service principal is missing SELECT on the billing tables, or CAN USE on the warehouse, or is not assigned to the workspace.
  • Secret expiring soon: generate a new OAuth secret on the service principal and enter it on the integration.
  • Data available fails: Databricks has no cost data for the period yet. Wait for Databricks to report usage, then connect again.