Mounting ADLS with Databricks

 Mounting of ADLS contianer Using SAS token

# Container name
# Storage account name
# Storage account SAS token
# Mount point name (it could be anything /mnt/.....)

#code Template
dbutils.fs.mount( source = 'wasbs://<conatiner-name>@<storage-account-name>.blob.core.windows
.net',
                 mount_point= '<mount-point-name>', extra_configs ={'fs.azure.sas.<conatiner-
name>.<storage-account-name>.blob.core.windows.net':'<SAS-Token>'})

# Example will be like this
dbutils.fs.mount( source = 'wasbs://landing@savimaladls.blob.core.windows.net',
                 mount_point= '/mnt/abc2', extra_configs ={'fs.azure.sas.landing.savimaladls.
blob.core.windows.net':'?sv=2021-06-08&ss=bfqt&srt=sco&sp=rwdlacupyx&se=2022-09-26T12:11:
56Z&st=2022-09-26T04:11:56Z&spr=https&sig=XO45qIekFi%2FJV9z%2F5mec5DicdsGvgGCdbzxoykEDd5i0M%3
D'})

#in next cell
%fs ls /mnt/abc2


Mounting of ADLS contianer Using SAS token with secret scope and key vault

# Container name
# Stoage account name
# Mount point name (it could be anything /mnt/.....)
# Databricks secret scope name
# Azure key vault key name containing the secret info

#code Template
dbutils.fs.mount( source = 'wasbs://<conatiner-name>@<storage-account-name>.blob.core.windows
.net',
                 mount_point= '<mount-point-name>', extra_configs ={'fs.azure.sas.<conatiner-
name>.<storage-account-name>.blob.core.windows.net': dbutils.secrets.get(scope="<databricks-
secret-scope-name>",key="<key-vault-secret-name>")})

# Example will be like this
dbutils.fs.mount( source = 'wasbs://landing@savimaladls.blob.core.windows.net',
                 mount_point= '/mnt/abc2', extra_configs ={'fs.azure.sas.landing.savimaladls
.blob.core.windows.net':dbutils.secrets.get(scope="dbscopemissionbatch2",key="dbserviceprinci
palkey")})

Mounting of ADLS contianer Using Application Registration
# Container name
# Stoage account name
# Tennant Id
# Application Id
# Secret value
# Mount point name (it could be anything /mnt/.....)

#Code Template
configs = {"fs.azure.account.auth.type": "OAuth",
       "fs.azure.account.oauth.provider.type": "org.apache.hadoop.fs.azurebfs.oauth2.
ClientCredsTokenProvider",
       "fs.azure.account.oauth2.client.id": "<application-id>"",
       "fs.azure.account.oauth2.client.secret": "<service-credential-key>",
       "fs.azure.account.oauth2.client.endpoint": "https://login.microsoftonline.com/
<tennant-id>/oauth2/token",
       "fs.azure.createRemoteFileSystemDuringInitialization": "true"}

dbutils.fs.mount(source = "abfss://<container-name>@<storage-account-name>.dfs.core.windows.
net",mount_point = "<mount-point-name>",extra_configs = configs)

#Example code
configs = {"fs.azure.account.auth.type": "OAuth",
       "fs.azure.account.oauth.provider.type": "org.apache.hadoop.fs.azurebfs.oauth2.
ClientCredsTokenProvider",
       "fs.azure.account.oauth2.client.id": "d9c97ba4-8943-4b06-b9d5-eb239baf9113",
       "fs.azure.account.oauth2.client.secret": "uwsajdbwj3u392smsknas",
       "fs.azure.account.oauth2.client.endpoint": "https://login.microsoftonline.com/
3cc553a5-eec4-4b67-9863-7e488c9fa455/oauth2/token",
       "fs.azure.createRemoteFileSystemDuringInitialization": "true"}

dbutils.fs.mount(source = "abfss://landing@savimaladls.dfs.core.windows.net",mount_point =
"/mnt/mission100landing",extra_configs = configs)

Mounting of ADLS contianer Using Application Registration with secret scope and key vault

# Container name
# Stoage account name
# Tennant Id
# Application Id
# Mount point name (it could be anything /mnt/.....)
# Databricks secret scope name
# Azure key vault key name containing the secret info

#Code Template

configs = {"fs.azure.account.auth.type": "OAuth",
       "fs.azure.account.oauth.provider.type": "org.apache.hadoop.fs.azurebfs.oauth2.
ClientCredsTokenProvider",
       "fs.azure.account.oauth2.client.id": "<application-id>"",
       "fs.azure.account.oauth2.client.secret": dbutils.secrets.get(scope="<databricks-secret-
scope-name>",key="<key-vault-secret-name>"),
       "fs.azure.account.oauth2.client.endpoint": "https://login.microsoftonline.com/
<tennant-id>/oauth2/token",
       "fs.azure.createRemoteFileSystemDuringInitialization": "true"}

dbutils.fs.mount(source = "abfss://<container-name>@<storage-account-name>.dfs.core.windows
.net",mount_point = "<mount-point-name>",extra_configs = configs)


#Example code
configs = {"fs.azure.account.auth.type": "OAuth",
       "fs.azure.account.oauth.provider.type": "org.apache.hadoop.fs.azurebfs.oauth2.
ClientCredsTokenProvider",
       "fs.azure.account.oauth2.client.id": "d9c97ba4-8943-4b06-b9d5-eb239baf9113",
       "fs.azure.account.oauth2.client.secret": dbutils.secrets.get(scope="
Batch5_Databricks_Secret_Scope",key="APPSECRET"),
       "fs.azure.account.oauth2.client.endpoint": "https://login.microsoftonline.com/
3cc553a5-eec4-4b67-9863-7e488c9fa455/oauth2/token",
       "fs.azure.createRemoteFileSystemDuringInitialization": "true"}

dbutils.fs.mount(source = "abfss://landing@savimaladls.dfs.core.windows.net",mount_point = "
/mnt/mission100landing",extra_configs = configs)


Check all the mount points
print(dbutils.fs.mounts())

How to unmount the mount point
dbutils.fs.unmount("<mount-point-name-to-be-unmounted>")

How to Refresh mount points
dbutils.fs.refreshMounts()

Check all mount releated function use this help function
dbutils.fs.help()

No comments:

Post a Comment