Skip to content

Commit b483ae4

Browse files
committed
Move to a sub-directory
1 parent 034d8ba commit b483ae4

2 files changed

Lines changed: 7 additions & 4 deletions

File tree

src/data_engineering/clojure_support_for_popular_data_tools.clj renamed to src/data_engineering/support_for_popular_data_tools/snowflake.clj

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,9 @@
55
:type :post
66
:date "2025-09-04"
77
:category :clojure
8-
:tags [:metadata :civitas]}}}
9-
(ns data-engineering.clojure-support-for-popular-data-tools)
8+
:tags [:metadata :civitas]
9+
:canonical-url "https://alza-bitz.github.io/clojure-support-for-popular-data-tools"}}}
10+
(ns data-engineering.support-for-popular-data-tools.snowflake)
1011

1112
;; In this article I look at the extent of Clojure support for some popular on-cluster data processing tools that Clojure users might need for their data engineering or data science tasks. Then for [Snowflake](https://snowflake.com) in particular **I go further and present a new Clojure API.**
1213

@@ -32,6 +33,8 @@
3233

3334
;; Please note, I don't wish to make any critical judgments based on either the summary analysis above or the more detailed analysis below. The goal is to understand the situation with respect to Clojure support and highlight any gaps, although I suppose I am also inadvertently highlighting the difficulties of maintaining open source software!
3435

36+
;; With that said, let's dive into the details for Spark, Kafka and Snowflake.
37+
3538
;; ### Spark Interop with Geni
3639

3740
;; [Geni](https://github.com/zero-one-group/geni) is the go-to library for Spark interop. Some months back, I was motivated to evaluate the coverage of Spark features. In particular, I wanted to understand what would be involved to support [Spark Connect](https://spark.apache.org/spark-connect/) as it would reduce the complexity of computing on-cluster directly from the Clojure REPL.
@@ -82,13 +85,13 @@
8285
{:id 3 :name "Charlie" :age 35 :department "Engineering" :salary 80000}])
8386

8487
;; Create session and save data
85-
(with-open [session (sp/create-session "src/data_engineering/snowflake.edn")]
88+
(with-open [session (sp/create-session "src/data_engineering/support_for_popular_data_tools/snowflake.edn")]
8689
(-> (sp/create-dataframe session employee-data)
8790
(sp/save-as-table "employees" :overwrite)))
8891

8992
;; #### Feature 2. Compute over Snowflake table(s) on-cluster to produce a smaller result for local processing
9093

91-
(with-open [session (sp/create-session "src/data_engineering/snowflake.edn")]
94+
(with-open [session (sp/create-session "src/data_engineering/support_for_popular_data_tools/snowflake.edn")]
9295
(let [table-df (sp/table session "employees")]
9396
(-> table-df
9497
(sp/filter (sp/gt (sp/col table-df :salary) (sp/lit 70000)))

src/data_engineering/snowflake.edn renamed to src/data_engineering/support_for_popular_data_tools/snowflake.edn

File renamed without changes.

0 commit comments

Comments
 (0)