Using Databricks FILE type against data on NAS: what works today with FSx for ONTAP S3 Access Points

A follow-up to Part 2: Databricks and FSx for ONTAP S3 Access Points in the "FSx for ONTAP S3 Access Points x Lakehouse Deep Dive" series. Some results changed since Part 2 — if you read that one, see "Why reads fail" below. TL;DR I tested whether images and PDFs already sitting on FSx for ONTAP can be referenced from Databricks without copying them into the lakehouse. The result is: registration works, reads do not. If you only need the zero-copy verdict, read "Connecting to an FSx for ONTAP S3 Access Point" and "Design tips" below. If you want the behaviour of the FILE type itself, start from the top. Databricks shipped FILE as a beta column type: a reference to an unstructured file held as a single column in a Delta table. The value is a struct carrying uri / offset / size / content_type / checksum. Because it holds the reference and the metadata as a struct, you can pass it straight from a table into an AI function such as ai_parse_document. There are two variants — FILE EXTERNAL, which holds only the reference, and FILE MANAGED, which copies the bytes into Databricks. I tested this against files on Amazon FSx for NetApp ONTAP, reached through an FSx for ONTAP S3 Access Point — the feature that exposes an ONTAP volume over an S3-compatible API. Here is what I found: Registration against an FSx for ONTAP S3 AP succeeds; reads do not. The cause sits in the session policy attached to the temporary credentials Unity Catalog vends, and there is no way to work around it from the user side The FILE type itself works. But you have to choose between FILE EXTERNAL and FILE MANAGED before you ingest If your design puts metadata in object tags, tag values are effectively ASCII. Most CJK strings are rejected while a minority are accepted. AWS Support has escalated that split to the service team as a potential defect The operator surface is inconsistent. GROUP BY and DISTINCT are accepted; = and ORDER BY are rejected Below: what is established for each of those four, and how to turn it into design decisions. The templates and scripts are in the repository, so you can re-run the same tests in your own account. Repository: Yoshiki0705/fsxn-lakehouse-integrations Test environment Item Value Measured 2026-08-12 Storage One FSx for ONTAP file system Access Point One, INTERNET origin, UNIX root identity Workspace One non-trial workspace, purpose-built, same account and region Compute Serverless SQL warehouse, and a classic DBR 18.2 cluster Control A S3 general bucket in the same account, running the same operations alongside Method SQL Statement Execution API, and boto3 from my workstation Running the S3 general bucket control alongside matters. Looking only at the Access Point side, you cannot separate a defect in your own environment from behaviour specific to the Access Point. Except where stated otherwise, every conclusion below rests on the same operation succeeding against S3 general bucket. Note that the object tag results depend on the ONTAP tag validation implementation, so a different ONTAP version may behave differently. The FSx console and describe-file-systems do not expose the version; use the ONTAP REST API (GET /api/cluster?fields=version) to check yours. What the FILE type can and cannot do First, the behaviour of the type itself. There is prior work on this feature from Databricks Japan: パス文字列でもバイナリでもない。DatabricksのFILE型を試す by taka_yayoi. Every operational caveat it lists reproduced in my environment, so I have tabulated those first. (What follows is my paraphrase and my own measurements, not a translation.) Some terms first. FILE MANAGED copies bytes into a Databricks-side location called a FileSpace. list_files enumerates files on a Volume and returns FILE values; create_file constructs a FILE value explicitly. _object_metadata reads an object's tags and user metadata. Behaviour Measured Design implication FileSpace on the same volume as the source CREATE TABLE succeeds; INSERT fails with Cannot get file metadata under managed storage Give the FileSpace its own volume checksum on FILE values from list_files Null on every row Not usable for integrity checks checksum under FILE MANAGED Populated with ETAG:"…", matching the etag from _object_metadata on S3 general bucket exactly as a string Comes from the object store, so it is usable for reconciliation Filenames under FILE MANAGED Replaced with an opaque UUID; the extension does not survive either Copy the name to a column before ingesting Format of uri Prefixed with dbfs: Normalise before writing string comparisons Serverless notebooks Unsupported Use the SQL Statement Execution API against a serverless SQL warehouse Automatic garbage collection in beta Does not run The FileSpace grows on every re-run. Plan for manual deletion The Previews page toggle Used FILE EXTERNAL / FILE MANAGED without touching it (confirmed on 2 workspaces) May not be required. Do not assume either way content_type from create_file binary/octet-stream for both Japanese and ASCII Pass it explicitly if anything downstream branches on it What this tells us The difference between FILE EXTERNAL and FILE MANAGED is not only whether bytes are copied. Whether checksum is populated and whether the filename survives both change. Reversing the decision later means re-ingesting, so decide before you ingest. On content_type, I got binary/octet-stream for both Japanese and ASCII input. For the same files, list_files reported text/plain, so the two paths infer differently. If anything downstream branches on content_type, pass it explicitly. The state of operator support Both the documentation and the prior article state that FILE columns cannot be used in grouping expressions. Testing it, support splits more finely than that. Operation Result GROUP BY file accepted SELECT DISTINCT file accepted GROUP BY file.uri accepted (the documented approach) d.file = r.attachment rejected — The `=` does not support ordering on type "FILE" ORDER BY file rejected — The `sortorder` does not support ordering on type "FILE EXTERNAL" What this tells us GROUP BY and DISTINCT both need equality semantics to decide group membership. Both are accepted while the equality operator itself is explicitly rejected. Since the error names the type's operator surface, this does not look like row-count-dependent behaviour. That said, this was confirmed on two rows. I cannot rule out that grouping degrades to identity comparison rather than value equality. In practice, use GROUP BY file.uri and do not depend on GROUP BY file being accepted. This is beta, so the behaviour may change. Connecting to an FSx for ONTAP S3 Access Point This is the main question. The answer is "registration works, reads do not", and the cause is not a lack of Access Point support. The four things registration needs Item Required form Symptom when wrong External ID in the trust policy The Databricks account UUID 403 Forbidden from the storage provider. The metastore ID and the workspace ID do not work Self-assume in the trust policy Account root as the principal, with an aws:PrincipalArn condition naming the role ARN Naming the role as its own principal fails at creation with Invalid principal, because IAM validates that the principal exists. Unity Catalog requires the role to be able to assume itself Resource in the IAM permission policy The access point ARN (arn:aws:s3:::accesspoint/ and .../object/*) AccessDeniedException. The alias-as-bucket-name form alone does not work External location URL The alias form (s3:///) An ARN-style URL is rejected immediately with url does not specify a valid bucket name What this tells us There is an asymmetry worth committing to memory: the same Access Point must be written as an ARN in the IAM policy and as an alias in the external location URL. The AWS CLI works with the alias form of the ARN, so verifying connectivity with the CLI first and then writing the IAM policy will trip you up. The aws:PrincipalArn approach works because that key resolves to the role ARN rather than the assumed-role session ARN. Rather than trust my memory of the condition key, I deployed the template, assumed the role, and assumed it again with those credentials to confirm. With all four satisfied, CREATE EXTERNAL LOCATION succeeds with skip_validation=False — meaning Unity Catalog's own validation passed. The external volume on top can be created too. Why reads fail After registration, every read path returns 403 or an authorisation error. The same on serverless SQL and on a classic DBR 18.2 cluster. This is the test that isolated the cause. Unity Catalog will vend the temporary credentials it uses if you ask for them. I requested them for both paths and used them from my workstation, with no Databricks compute or network involved. Vended for ListObjectsV2 HeadObject S3 general bucket control path success success FSx for ONTAP S3 AP path AccessDenied 403 Same role, same session, same network; the only variable is which path the credentials were scoped to. The error states the cause explicitly: is not authorized to perform: s3:ListBucket on resource: "arn:aws:s3:::accesspoint/" because no session policy allows the s3:ListBucket action What this tells us because no session policy allows is the decisive part. What is named is the session policy, not the role's permissions. By session policy I mean the down-scoped policy passed to AssumeRole to constrain the role's permissions on a per-request basis. The policy body is generated by Unity Catalog and is not readable by the user. But since adding an access-point-ARN allowance to the role changes nothing, the session policy appears not to mention the access point ARN resource form at all. Only the overlap between the session policy and the role policy is effective, so additions on the role side have no effect. There is no workaround on the user side. Unity Catalog generates the policy, so the fix has to happen there. Emitting the access point ARN form in the session policy should resolve it, though I do not know what implementation constraints apply. Note also that validation at registration time (skip_validation=False) passes while reads return 403. What that validation checks is not published, so I cannot account for the difference. What changed since Part 2 Part 2 of this series (measured 2026-05) reported that listing top-level files and reading an explicit file did work through a UC external location. In this run (2026-08-12), read_files, list_files, to_file and dbutils.fs.ls all returned 403; no read path partially succeeded. That environment is gone, so I cannot account for the difference conclusively. The likely explanation is that the earlier partial successes were observed on an instance-profile path rather than through UC-vended credentials — Part 2 lists instance-profile direct access as a separate approach that works but sits outside UC governance, so the paths may not have been cleanly separated. This run had a S3 general bucket control alongside and includes the vended-credential test outside Databricks entirely, so take these results as the current ones. I will annotate the Part 2 table accordingly. This test has prerequisites. Enable External Data Access on the metastore and grant EXTERNAL USE SCHEMA and EXTERNAL USE LOCATION. Both are disabled by default. They are the controls that allow credentials to be used externally, so they are worth understanding independently of this issue. A table to work through when you see a 403 Arranged so that anyone hitting the same 403 can check top to bottom. None of these is the cause. What to suspect Finding Unity Catalog volume privileges The failing volume had READ VOLUME; the working control volume had no privileges at all IAM role permissions Assumed the same role from my workstation and listed successfully. HEAD on a real object returned 200 HEAD on a missing key returning 403 (killing the _delta_log probe) The Access Point returned 404 for every shape I tried — same as S3 general bucket x-amz-expected-bucket-owner being rejected Accepted by both the Access Point and the control bucket A different endpoint Both Databricks and the AWS CLI use .s3..amazonaws.com, which CNAMEs to s3-r-w..amazonaws.com. Same host Insufficient compute role permissions sts:GetCallerIdentity raised NoCredentialsError on the driver. The cluster has no default AWS credentials; the compute role is not in the path The S3 gateway endpoint No change after disassociating the route. The failure reproduces outside the VPC with vended credentials, so this path is not the problem One note on the last row. A Databricks-managed VPC creates an S3 gateway endpoint at workspace creation, and the private route tables point the S3 managed prefix list at it. That was not the cause here, but for designs that reach an S3 AP from inside a VPC, check first whether the alias resolves into the prefix list's range. I did not reconcile the resolved CIDRs against the prefix list contents in this environment. What object tags can actually hold If you want metadata to live with the file, object tags are one option: tag at write time, read from the table side later. That path has constraints. The following was measured without Databricks in the picture — boto3 from my workstation straight against the FSx for ONTAP S3 Access Point, with the same operations running against the S3 general bucket control bucket. Start with what works. PutObjectTagging, GetObjectTagging and DeleteObjectTagging all function, as do x-amz-meta-* headers and x-amz-tagging on the same PutObject as the data. The limits (10 tags per object, 128 characters for a key, 256 for a value) are the same values S3 general bucket documents; I did not measure the boundaries. Reading through a second access point on the same volume returned tags written through the first. Tag retention is not scoped per access point. For existing NAS assets shared across several paths, that property works in your favour. But I did not observe where the tags are physically stored, so visibility from NFS or SMB, and retention across SnapMirror, FlexClone and Snapshot restore, remain unverified. Two constraints change how you design. An object overwrite clears tags and user metadata. Nothing errors. A pipeline that rewrites a file has to re-apply them in the same PutObject. My understanding is that S3 general bucket behaves the same way, since PutObject replaces the object, so treat this as a general S3 design point rather than something specific to FSx for ONTAP. I did not run a side-by-side control for this particular case. Tag values are effectively ASCII. Every printable Latin-1 character I tried (U+00A1–U+00FF) was accepted. Above U+0100, most are rejected with InvalidTag — but not all. A minority of CJK strings are accepted, deterministically, 6 runs out of 6. To find the unit of validation, I tested single characters: Input Result U+5206 InvalidTag U+985E InvalidTag U+5206 U+985E accepted U+6771 InvalidTag U+4EAC InvalidTag U+6771 U+4EAC InvalidTag What this tells us Two two-character strings, every constituent character rejected on its own, and the two strings disagree with each other. Validity is determined by the complete byte sequence, not as a function of the characters in it — so this is not a per-character allowlist. For the two-character pairs I tested, results were identical forwards and backwards, and identical as a tag key and as a tag value. I did not test longer strings or other scripts. I raised this with AWS Support. After checking it against the S3 and FSx for ONTAP documentation, they gave the view that the pattern does not match any intentional validation they could identify, and escalated it to the service team as a potential defect in the tag validation layer. Their reply also quoted the documented character set: letters, whitespace, and + - = . _ : / @. That does not predict the split under either reading. If "letters" means Unicode letters, every string I tested qualifies yet half are rejected. If it means ASCII letters, all of them should be rejected yet half are accepted. The design guidance is simple: keep tags ASCII and keep localised text in a column. Today the situation is "some strings pass and some do not", which is more awkward to handle than a uniform restriction. I will update the repository and this article when a determination arrives. Design tips The same material, ordered the way you actually make the decisions. FILE type Decide between FILE EXTERNAL and FILE MANAGED before ingesting. Checksum and filename handling differ, and changing course later means re-ingesting Give the FileSpace its own volume. On the same volume, INSERT fails — and because the error surfaces at ingest rather than at configuration time, diagnosis is slower Copy the filename to its own column before ingesting into FILE MANAGED. It is replaced with a UUID and the extension does not survive Do not rely on content_type inference; pass it explicitly Group by file.uri. Do not depend on GROUP BY file being accepted Automatic garbage collection does not run in beta. Build manual FileSpace deletion into operations Unity Catalog and FSx for ONTAP S3 Access Points The external ID in the trust policy is the Databricks account UUID. The metastore ID does not work The trust policy needs self-assume: account root as the principal with an aws:PrincipalArn condition naming the role ARN The IAM permission policy needs the access point ARN, including .../object/*. The alias-form ARN alone does not work The external location URL takes the alias form. ARN-style URLs are rejected Reads do not work today. If zero-copy is a requirement, evaluate AWS-native engines that authorise in the caller's own IAM context alongside it (this repository has walkthroughs under integrations/athena, integrations/glue, integrations/emr-spark, integrations/redshift-spectrum and integrations/lake-formation) Object tags Keep tags ASCII and keep localised text in a column Re-apply tags in the same PutObject when you rewrite a file. An overwrite clears them Tag retention is not scoped per access point, so the same tags are readable through several access points How to test Run a S3 general bucket control alongside. Without it you cannot separate a defect in your environment from Access Point behaviour, and the first 403 looks like the answer If the platform exposes credential vending, use those credentials from your own machine and compare. Isolating the problem is much faster without the vendor's compute in the path Verify teardown by reconciling against a baseline. NAT Gateways do not disappear when you delete the workspace Run it yourself All of the above is in the repository as templates and scripts: one template, one script, one runbook. Artefact Role integrations/databricks/uc-storage-credential-role.yaml The IAM role the Unity Catalog storage credential assumes, plus a S3 general bucket control bucket shared/scripts/probe_uc_external_location.py Seeds identical object tags on both sides, registers both external locations with validation on, reads _object_metadata through each, prints a verdict docs/en/databricks-verification-runbook.md Prerequisites, where each parameter comes from, the verdict branches, dependency-ordered teardown, measured cost # 1. Baseline first. This is what makes "teardown looks clean" verifiable python3 shared/scripts/audit_databricks_workspace_footprint.py \ --region --save /tmp/baseline.json # 2. The IAM role and control bucket aws cloudformation deploy --region \ --stack-name fsxn-databricks-uc-credential \ --template-file integrations/databricks/uc-storage-credential-role.yaml \ --capabilities CAPABILITY_NAMED_IAM \ --parameter-overrides file://cfn-params/databricks-uc-storage-credential.json # 3. Compare. --control-bucket is deliberately mandatory .venv/bin/python shared/scripts/probe_uc_external_location.py \ --profile --role-arn \ --ap-alias -ext-s3alias --ap-name \ --control-bucket --region --vend-check # 4. Tear down and reconcile against the baseline python3 shared/scripts/audit_databricks_workspace_footprint.py \ --region --compare /tmp/baseline.json The script prints one of four verdicts. The first is the important one. Inconclusive — the S3 general bucket control could not be read either, so this run says nothing about the Access Point. Fix the environment first Registration succeeded, reads denied — the result in this article Registration and reads both succeeded — the platform changed. Please tell me which release Registration itself failed — almost certainly a missing Access Point ARN in the IAM policy One note on the parameter file: a _comment key in a CloudFormation parameter file makes the CLI fail with Unknown parameter in Parameters[n]. The description of each parameter lives in cfn-params/README.md instead, so fill in values with that open alongside. If you are evaluating a different platform The specific bug is Databricks'. The shape is not, and the shape is the part worth taking away if you are looking at a different product. Managed lakehouse platforms generally reach into your cloud account by assuming a role you create, then constrain that role at runtime with a down-scoped session policy so that credentials handed to a query cannot roam. That is good design. But the policy has to name your storage. If the storage is addressed through an access point while the policy names it in bucket-form ARNs, the two do not match and authorisation fails — however correct your own role policy is. So the question to put to a platform you are evaluating, before you commit to an architecture, can be narrow and answerable: When you vend temporary credentials for an external location, which resource ARN forms does the session policy contain? Specifically, does it include the S3 access point ARN form in addition to the bucket form? And if the platform exposes credential vending, you can verify it without asking anyone: request the credentials, use them from your own machine, and compare a native bucket path against an access point path. That is what --vend-check above does, and it was the only test in this exercise that produced a conclusion with no room for interpretation. Cost and cleanup I created a test workspace and destroyed it the same day. The workspace mode that uses your own AWS account creates a VPC with a NAT Gateway. A NAT Gateway runs roughly 45 USD a month per gateway (hourly charge only, excluding data processing, at the August 2026 Tokyo region rate). These resources are created directly rather than through a CloudFormation stack, so deleting the workspace does not remove them. This is the easiest place to make a foolish mistake during cleanup. I recorded a baseline before starting and reconciled afterwards. Available NAT Gateways back to zero; VPC, both IAM roles and the bucket gone, with no unassociated Elastic IPs. I turned that check into a script (shared/scripts/audit_databricks_workspace_footprint.py). In a shared account a plain listing is not a verdict, because most of what it shows belongs to someone else. Run it once before creating anything and once after teardown; it exits non-zero if anything exists that was not in the baseline. Running it confirmed this teardown was clean and also surfaced leftover roles from an earlier workspace. The cost breakdown is in the runbook with measured figures. Current state and open items Established CREATE STORAGE CREDENTIAL / CREATE EXTERNAL LOCATION / CREATE EXTERNAL VOLUME succeed against an FSx for ONTAP S3 AP alias, with Unity Catalog validation left on Reads through it return 403, on both serverless SQL and a classic DBR 18.2 cluster The same failure reproduces outside Databricks, using vended credentials from a workstation _object_metadata on S3 general bucket returns object tags and user metadata correctly Object tag acceptance is determined by the complete byte sequence, and within the range tested is neither order- nor position-dependent Not established VPC-origin Access Points, and Access Points with WINDOWS identity Whether _object_metadata would read tags through an Access Point if the session policy were fixed ai_parse_document over files served from an Access Point Throughput and latency of these paths Whether GROUP BY on a FILE column is correct at scale, or merely accepted Where object tags are physically stored; visibility from NFS or SMB; retention across SnapMirror, FlexClone, Snapshot restore and FabricPool tiering The boundary values for tag limits (10 tags, 128-character keys, 256-character values) Reporting status The object tag behaviour has been escalated through AWS Support to the FSx for ONTAP service team and is awaiting a determination. What I asked for is not a timeline but the distinction: a defect that will be fixed, or an undocumented restriction that gets written down. Either answer lets me state the constraint accurately. The session policy issue and the operator inconsistency go to Databricks when this article publishes. This repository carried an entry from May 2026 saying Unity Catalog external locations do not support S3 Access Points. Registration does work; the entry has been corrected. If you read the earlier version, please note the change. Closing The FILE type is a well-built mechanism for bringing unstructured data under table governance. For referencing files on an FSx for ONTAP S3 Access Point with zero copies, though, it does not get there today. Registration works, so you can try it, but reads fail, and because the cause sits in the session policy on the vending side, users cannot fix it. If zero-copy is the requirement, AWS-native engines that authorise in the caller's own IAM context are the more straightforward path for now — in exchange for assembling table-level governance yourself with Lake Formation. If Unity Catalog governance is the requirement, then for the time being data placement has to be part of the design. The object tag character set is awaiting a determination. When it moves, I will update both the article and the repository. I hope this is useful to someone. See you next time. Related posts Previous: Part 2: Databricks and FSx for ONTAP S3 Access Points Next: Part 8: OpenSharing and FSx for ONTAP Series index: What works, what does not, and why About this article This series is based on verification in the fsxn-lakehouse-integrations repository Author: Yoshiki Fujiwara Affiliation: NetApp G.K. Japan Role: Senior Cloud Solutions Architect - AWS Topic: Amazon FSx for NetApp ONTAP / Hybrid Cloud / Generative AI Data Architecture Level: 200-300 All test environments have been deleted. The behaviours described are measurements from 2026-08-12 in one specific configuration and may change with platform updates. Beta features are involved, so please check against current documentation before making production decisions.