NetApp Instaclustr for ClickHouse can query data stored in Amazon FSx for NetApp ONTAP through ONTAP’s S3-compatible interface. That means you can analyze data in place, without first loading it into ClickHouse-managed storage. For many teams, this reduces duplicated storage and lowers infrastructure costs for large, infrequently queried datasets such as logs, clickstream events, and telemetry.
Why query Amazon FSx for NetApp ONTAP data with ClickHouse?
Analytics teams usually face a familiar trade-off:
- Load data into native database storage for fast performance, but pay more for duplicated data and higher storage costs.
- Keep data in object storage to reduce cost, but accept slower access and extra ingestion complexity.
NetApp Instaclustr for ClickHouse with FSx for ONTAP offers a practical middle path: keep data in external storage and query it directly using familiar SQL, while avoiding a full copy into native ClickHouse tables.
How the Amazon FSx for NetApp ONTAP and ClickHouse integration works
Amazon FSx for NetApp ONTAP supports file protocols and an S3-compatible interface. Instaclustr for ClickHouse connects through that S3-compatible endpoint.
Key behavior to understand:
- ClickHouse reads data through the ONTAP S3 interface, not POSIX/NFS access.
- Source files should already be in ClickHouse-readable formats such as Parquet or CSV.
- SQL remains familiar, but external S3-backed tables do not behave like MergeTree tables in performance, indexing, or partitioning.
Cross-VPC and cross-account access
Instaclustr for ClickHouse supports architectures where compute and storage are in different VPCs or AWS accounts. This is useful for organizations that separate environments for security, ownership, or billing. Once configured, a central analytics cluster can query distributed storage across accounts.
Common use cases
This pattern works best when storage efficiency matters more than ultra-low-latency query performance:
- Historical logs: retain years of logs cost-effectively and query for audit/compliance.
- Clickstream analytics: run analysis directly on large event datasets in your data lake.
- Security archives: investigate incidents using stored detection and activity data.
- IoT telemetry: store high-volume sensor data once and query on demand.
Query patterns in ClickHouse
Instaclustr for ClickHouse uses Named Collections to store connection settings securely (endpoint, region, credentials), so queries do not expose plaintext secrets.
1) Ad-hoc exploration with the s3 table function
Use this for one-off inspection and validation. ClickHouse reads files directly at query time without creating a persistent table definition.
|
1 2 |
SELECT * FROM s3(<named_collection>, filename='<file>'); |
2) Repeatable access with the S3 table engine
Use this when querying the same external dataset repeatedly. You define schema and storage path once, then query it as a persistent external table.
|
1 2 3 4 5 6 |
CREATE TABLE s3_fsx_for_ontap_table ( id Int32, name String ) ENGINE = S3(<named_collection>, filename='<file>'); |
Important note: This is not MergeTree storage. If you INSERT into an S3-engine table, data is written to object-backed storage on FSx for ONTAP.
Reference architecture for Amazon FSx for NetApp ONTAP and ClickHouse
A typical deployment includes:
- Instaclustr for ClickHouse cluster (provisioning, upgrades, operations handled by Instaclustr)
- Amazon FSx for NetApp ONTAP file system with S3-compatible access
- Applications or pipelines writing data to FSx for ONTAP
At query time, ClickHouse reads required object ranges, processes data in memory, and returns results. Because storage and compute are decoupled, you can scale each independently.
Final takeaway
If you have large datasets that are expensive to keep in native database storage but only queried occasionally, this integration is a strong fit. You can reduce duplication, control storage spend, and still run SQL-based analytics.
A practical next step is to run a pilot on one high-volume, low-frequency dataset (for example, historical logs), then compare:
- Monthly storage cost
- Query latency for representative workloads
- Operational overhead versus current ingestion flows
Most alternatives solve only one layer of the stack. NetApp Instaclustr gives you a fully managed open source data platform across the lifecycle – from Kafka messaging and streaming to ClickHouse analytics – so you can reduce complexity, avoid duplicated storage, and scale with one partner.
Watch this video for another overview and then start a trial or talk to our team to validate the architecture on your own workloads.
Frequently Asked Questions
-
Does this use standard file access protocols? +
No. Instaclustr for ClickHouse accesses FSx for ONTAP data through the ONTAP S3-compatible interface.
-
Will performance match native ClickHouse tables? +
Not exactly. S3-backed tables do not provide the same indexing and partitioning behavior as MergeTree tables.
-
Does this remove all data engineering work? +
No. You still need data in compatible formats and proper access/network setup.
-
Who manages the ClickHouse infrastructure? +
Instaclustr manages the ClickHouse cluster lifecycle, including provisioning, upgrades, and routine operations.
