Before diving into the process, it’s important to understand what a shortcut in OneLake entails. Essentially, a shortcut in this context is a reference or link to a dataset or specific data object within OneLake. This allows users to access data quickly without navigating through multiple layers of the data lake’s structure, facilitating faster decision-making and smoother workflows.
Which Fabric Items can I create Shortcut for?
Shortcuts can be created within Lakehouses and Kusto Query Language (KQL) databases.
Once shortcut created, any Fabric or non-Fabric service that can access data in OneLake can use shortcuts. Shortcuts are transparent to any service accessing data through the OneLake API. Shortcuts just appear as another folder in the lake. Spark, SQL, Real-Time Analytics, and Analysis Services can all use shortcuts when querying data.
*** You can’t create or access shortcut directly from the warehouse.
What’s more, these shortcuts can be configured to link to other OneLake locations, Azure Data Lake Storage (ADLS) Gen2, Amazon S3 storage accounts, Dataverse or Google Cloud Storage. (Microsoft is adding new external sources everyday, the list will likely increase)
How to I Create shortcut?
Despite varying sources and authentication methods, the overall process for creating shortcuts remains largely the same. Let’s demonstrate how to create a shortcut for Azure Data Lake Storage Gen2.
- Open a lakehouse.
- Right-click on a directory within the Lake view of the lakehouse.
- Select New shortcut.
- Under External sources, select Azure Data Lake Storage Gen2.
- Enter the Connection settings according to the following table
- Select Next and browse to the target location for the shortcut.
If you just used the storage account in the connection URL, all of your available containers appear in the left navigation view. If you specified a container in connection URL, only the specified container and its contents appear in the navigation view. You can select one or more shortcut target locations by clicking the checkbox next a folder and click on next.
Verify the shortcut you are trying to create and click on Create button.
Once you have shortcut in Lakehouse Tables, you can query that shortcut-ed tables from Warehouse Mode and/or create views with joins, unions, etc.
What do shortcuts look like and behave?
Shortcuts function similarly to symbolic links. They exist as separate entities from their targets, meaning that deleting a shortcut does not impact the target. However, if you move, rename, or delete the target path, the shortcut may become invalid.
Important Limitations of Shortcut
- The maximum number of shortcuts per Fabric item is 100,000.
- The maximum number of shortcuts in a single OneLake path is 10.
- The maximum number of direct shortcuts to shortcut links is 5.
- OneLake shortcut names, parent paths, and target paths can’t contain “%” or “+” characters.
- Shortcuts don’t support non-Latin characters.
- Additional shortcuts can’t be created inside ADLS or S3 shortcuts. (AKA you can’t create shortcut on shortcut)
- Lineage for shortcuts to Data Warehouses and Semantic Models is not currently available.