In general, Databricks recommends using OAuth instead of Personal Access Tokens (PATs) for authentication with Databricks to enhance security. We are now extending this recommendation to Databricks Git credentials and encourage the use of OAuth over Git providers’ PATs when authenticating with your Git providers.
Today, we are excited to announce the General Availability of OAuth Git credential support for Service Principals with GitHub and Azure DevOps, improving Git connection security for automated workloads.
Databricks Git integration initially supported only PATs for authentication. Users created personal access tokens with their Git provider and saved the tokens in Databricks. This approach is no longer recommended for a few reasons, including:
- [Long lifetimes] PATs offer longer access durations (weeks/months) than short-lived tokens (hours/days). Although administrators can enforce shorter PAT lifespans, this creates operational challenges as users must frequently update their Databricks Git credentials to avoid workflow failures upon expiration.
- [Insecure storage and transfer] Users often manually copy PATs, which could leave traces in clipboards and documents.
- [Wide scopes] Some PATs, such as GitHub Classic PATs, apply to every repo the user can access. This behaviour can easily lead to unintended privilege escalation and allow for lateral movement.
- [Missing service principal support] Some Git providers, such as Azure DevOps, do not support generating PATs for service principals.
Our most popular Git providers discourage the use of PATs: GitHub and Azure DevOps do not recommend using PAT for long-lasting integrations. Bitbucket recommends Bitbucket Cloud integration or app developers use OAuth for user authentication instead of access tokens.
Databricks has supported OAuth 2.0-based user authentication with GitHub and Azure DevOps for several years, but this support was previously limited to interactive user sessions.
Now that Service Principal support is generally available, our recommendation is to use OAuth instead of PATs when integrating with these Git providers for both interactive and automated workflows. What are the benefits? Take our GitHub App integration as an example:
- OAuth tokens are automatically refreshed by default. Users no longer encounter errors when their PAT token expires.
- OAuth offers enhanced administrative control, especially regarding the viewing and access of integrated repos.
- OAuth allows you to configure access to specific GitHub repos.
- Access tokens have a short lifespan (in this case, 8 hours), which reduces the risk of credential exposure.
Some customers have requested SSH authentication and GPG commit signing. However, we chose to invest in OAuth support instead, as SSH and GPG would require users to upload private keys to Databricks, similar to storing a PAT, leading to the same drawbacks: long-lived credentials and manual rotation. Moreover, if an improperly scoped SSH key were compromised, it could grant an attacker direct access to the Git server host, significantly increasing the risk of exploitation.
Getting Started
For GitHub, you can configure the Service Principal GitHub App connection on the Service Principal’s settings page, following a similar process as a user’s configuration. For Azure DevOps, we now support OAuth connections for service principals using federated credentials based on OpenID Connect (OIDC). OIDC is an authentication protocol built on top of OAuth 2.0 that provides login and profile information about the logged-in user. OIDC enables secure and user-friendly login experiences by allowing users to authenticate once with a trusted identity provider (IdP, in this case, Microsoft EntraID) and be remembered without needing to re-enter credentials. This new feature replaces the earlier scripting-based approach described in this blog, significantly simplifying and shortening this critical user journey from hours to just a few minutes.
