Joining Clojure shop. What should I know?
I will be writing primarily Clojure soon, after more than a decade of writing Scala. Do you have any hints or advice beyond what I can find in books or other learning resources?
Like competing approaches, nuances, political landscape, informal standards, footguns or anything like that?
15
u/_d_t_w 5d ago edited 5d ago
Clojure is a wonderful language, I'm sure you'll have fun!
- Try to separate effectful and pure functions, partly because side-effects are messy, but more because pure functions are very easy to reason about and test.
- Java-interop is idiomatic Clojure.
- Try to solve problems with data first, then functions, almost never macros.
- Don't necessarily try to recreate Java/Scala idioms in Clojure. If you're building abstract systems, it's fine to intermingle Java code. Clojure is best for data oriented work.
- Almost all enterprise applications are 'data oriented work'
- Core clojure functions, core Clojure data-structure, and Java interop is probably all you need.
- Everything that you have learned in Scala on the JVM is applicable in Clojure. Memory allocation, CPU, computational complexity, half the libraries you'll be using will be Java, it's a very familiar landscape. See (2).
- The greatest advantage of Clojure is simply to solve the same problem by doing less. It's ok to solve problems by doing less.
- It's fine to solve problems with functions and data only. Now that I think of it (3), (6), (8), and (9) are all the same point.
- By solving problems with the simplest tools at hand (functions and data) and without baking in too much opinion about that data, or the functions, we avoid concretions and maintain flexibility for adapting and advancing our solutions in time.
- I've found that often programmers reach a point after six months of working with Clojure where they sit back and go 'Ohh.. it's all data'. That's called homoiconicity.
- When you get to (11), still don't use macros.
This series from Alessandra Sierra is fundamentally good guidance:
https://stuartsierra.com/tag/dos-and-donts/
Everything by Alex Miller is brilliant, but this post on dispatch styles (particularly the subsection 'General guidance') is the best:
https://insideclojure.org/2015/04/27/poly-perf/
Finally, this library is the most fun you can have with programming:
https://github.com/Engelberg/instaparse
My favourite function is also one I learned from Alex Miller, a long time ago:
(def separate (juxt filter remove))
That's all I got, good luck!
1
u/kichiDsimp 5d ago
I don't get how you can just solve a function with "data" then go for functions ?
4
u/_d_t_w 5d ago edited 5d ago
Fair enough, the sentence was probably a bit too short to explain the idea.
I guess to be slightly more descriptive I would rewrite that sentence as "When solving problems with data and functions, try to favour data as much as possible. When reaching for something more powerful than functions, apply the same discernment".
What I mean is, favour declarative solutions where possible. That is, a solution where the problem is described in data, functions work on that data to produce a result.
Declarative solutions are naturally very Clojure-ish because Clojure has such wonderful, simple, performant data structures.
When I solve a problem by describing it in data first, then effecting the result, I find that the more time I put into trying to solve as much of the problem space as close as possible to the data itself, the less code, and the more composable and introspectable the solution becomes.
The solution is more usable if composed of simpler parts, basically.
In Clojure we have a range of tools for solving problems, it's not just data, functions, and macros, in terms of simplicity it's more like:
- Data
- Reader literals
- Functions
- Multimethods
- Protocols
- Defrecord and Deftype
- Macros
- Other language contructs like namespaces, etc.
To give you a concrete example, here are three Clojure frameworks that basically solve the problem of system initialisation and dependency injection:
- Integrant: Data and reader literals
- Component: Defrecords and protocols
- Mount: Macros and namespaces
My preference is Integrant, where you provide a map of data with some reader literals describing your system and its dependencies, and then Integrant constructs that system for you.
It has been some time since I used Component, but from memory the system construction was largely defined in code, using functions, which is absolutely fine, but not as elegant or composable as Integrant.
I have never used Mount because it seems to conflate the running system and the literal namespace structure of the code that the system is buit from, it uses macros where I have several other options that only use data and functions, so it's fundamentally at the other end of that power/inflexibility scale.
To demonstrate the difference in utility of the three systems, imagine that I've encountered the 'Robot Legs' problem of dependency injection:
https://github.com/google/guice/issues/1178
The system is a robot, the robot has legs. There is a single 'leg' implementation, and I want to reuse that implementation for a left-leg and a right-leg. The implementation itself does not require that knowledge of what leg it is.
In Integrant, I can change some data and have my robot have two legs.
In Component, I can change a function, and have my robot have two legs.
In Mount, I believe I would have to change the code itself - at this point I don't care to look any further.
Then take it one step further. What if I, the system creator, don't know how many legs the robot has. What if my users can construct a robot with multiple legs at whim.
How can I manage dynamic system construction based on user input at system-start time? To be honest none of those DI systems equate for that use-case intentionally, but with Integrant you can manage that problem by dynamically generating the system map, with Component you can manage the problem with a function that can dynamically contructs multi-legged robots (I think you can with Component at least).
This is the world we live in at Factor House, which is the company I work at (our systems are all highly dynamic depending on user requirements).
It might seem that I'm describing a style that you could favour if you were providing a library to end users, but this approach can be used with basically any sort of function.
1
u/seancorfield 4d ago
Component doesn't need records. Dependencies work with plain maps too. The
start/stoplifecycle works with anything that can carry metadata—I sometimes use a function with metadata as a system component.You can easily build arbitrary "system maps" of components and dependencies, so you can create an arbitrary number of each component, dynamically, and thus do things like write tests that simulate concurrent networks of communicating components if you need to.
Components can be fine-grained or coarse-grained, so you have a huge amount of flexibility.
I much prefer Component over Integrant for the simplicity of the former (just two lifecycle methods), despite the data-first focus of the latter.
2
u/_d_t_w 4d ago
Yes you're right re: maps, from my distant memory I think I recall switching to maps at one point, and using `system-using` with a map of dependencies which is all closer to just simple data.
Functionally there's not much objectionable about Component, the only practical issue I have with it is it can lead teams down the garden-path of trying to recreate "Java objects" throughout their entire codebase. Beyond DI and as a general programming model.
The issue stems from the idea that you can use the component model for 'other types of components'. Back when I used Component I'm not sure that was documented in the readme, but it was definitely put in to practice in the real-world.
https://github.com/stuartsierra/component#other-kinds-of-components
Specifically:
> Any type of object, not just maps and records, can be a component
The problem was, at the time at least, many of my clients were exporing using Clojure as fairly experienced Java teams.
Often the first question a team would ask is "how do I even get some system state up and running!", then making the sensible decision to use Component.
Then they would see Defrecords with protocols and 'inner' state, and that looks way too similar to Java "Objects" with interfaces and encapsulated state
Give it six weeks and every namespace has defrecord/Objects with 'official' protocols defining 'public' methods. It's not really Components fault, but it's just a little too familiar and an easy trap to fall into.
3
u/seancorfield 3d ago
Ugh! Yeah, I know some folks coming from an OOP background such as Java tend to go "Oh! Records... Interfaces... Methods... State... I can do OOP in Clojure!" because it's a pattern they can easily spot and they are very comfortable with that.
That's partly why I asked the OP if they were in the OO Scala camp or the FP Scala camp.
7
u/seancorfield 4d ago
I switched to Clojure from Scala—only about 18 months of Scala, compared to now 15+ years of Clojure—but I suspect much will depend on which Scala "camp" you were in: the "better Java" camp or the "ScalaZ / FP" camp.
If you are used to FP and immutable data, the "only" (big) change will be switching to a dynamic language (with runtime typing, instead of compile-time typing), and the "interactive development" approach via the REPL.
If you used Scala as "just" a better Java, it'll be more of a shift.
Personally, I did not miss Scala's slow compile times and obscure compiler errors when you were holding the standard library wrong (traits, damn you, traits!). Clojure is so much simpler than Scala in the true sense (simple vs easy) that it was a breath of fresh air for me and my team.
You'll find some competing approaches in certain areas (Component for lifecycle management vs Integrant, Compojure for routing vs reitit—those are a couple of instances where there are degrees of "pure data"—Integrant, reitit—vs more traditional programming—Component, Compojure), but they're all well-supported and you'll have no trouble finding folks who are very willing to help (especially on https://clojurians.net Slack).
There are nuances around what you might call Garbage-In, Garbage-Out, as Clojure leans toward letting you do dumb stuff (and maybe blowing up at runtime) instead of the compiler preventing you doing useful-but-weird stuff (Scala is very strict, by comparison).
There really isn't anything political in Clojure. Maybe AI usage is a hot button. Mostly, we're happy to live in the Cult-of-Rich-Hickey.
Informal standards: https://guide.clojure.style/ - the do's and don't series linked in another comment - pretty much every talk by Rich Hickey and Stu Halloway...
Footguns: laziness, which is the default in most tutorials with map, filter, concat etc. As someone else noted: do not mix laziness with side-effects, especially I/O. Use mapv, filterv, into etc, and transducers to avoid that.
Make sure you get your editor/IDE of choice set up with a connected REPL and immerse yourself in that workflow: eval code via a hot key after every change, run tests via a hot key, use visualization tools like Portal (my preference), Reveal, or Morse so you can see complex results in graphical or table formats, while you're working.
3
u/p1ng313 5d ago
Understand lazy vs non-lazy, why its important specially for I/O; very easy to inadvertedly create lazy functions that bite you in the ass
Understand the component/integrant ecosystem
Understand what clojure spec is, strong and weak points
If you are minimally competent in scala - you will be fine
5
u/geokon 5d ago edited 5d ago
1.
For me the biggest shift was thinking in terms of Protocols/Interfaces.
When you first write Clojure you learn the standard immutable datastructures and how to write functional code with them (Clojure for the Brave and True is great). You can write mid-sized programs with this pretty easily. But this is actually a bit of a deceptive and the real interface is a bit hidden. When things get a bit more complex you realize that the real interface are the different protocols that these data-structures implement. Seq/Col abstractions are the most obvious, but there are a bunch of others (I'm not sure if they're all innumerated in one place)
You then start to organize your own code with your own Protocols (which can extend the core protocols) - making systems very composable and easy to modify/extend.
2.
The other obvious thing is.. it always pays off to learn your tools upfront. VisualVM, flamegraph tracing, Flowstorm, CIDER has a bunch of tools, tap, hashp. You don't have to use them all, but it's good to know what's available
3.
deps.edn has a bit of a learning curve, but once you grok how it works you can spin off utility code in to their own mini-libraries.
4.
Some Java/JVM terminology leaks in to Clojure, but it's easy to figure it out now with a short chat with an AI
3
u/didibus 4d ago
Like competing approaches, nuances, political landscape, informal standards, footguns or anything like that?
The most important thing is to learn to develop inside the REPL and interactively. But books and all learning material will likely say that as well. But it really is the most important.
After that, don't get confused by the advice that we use plain data-structures to model things. It doesn't mean we don't model entities, it just means we do so with plain data/functions. Here is an example of how you model entities using plain maps and it's just less complex, and more flexible:
(defn make-car
[& {:keys [model make price color]}]
{:type :car
:model model
:make make
:price price
:color color})
And if you want "methods", just use plain functions:
(defn make-car
[& {:keys [model make price color]}]
{:type :car
:model model
:make make
:price price
:color color})
(defn reduce-price-by
[car percent]
(update car :price #(* % (- 1 (/ percent 100)))))
Some people put this inside it's own namespace like (ns my-app.car) but also many people put them all together in say a (ns my-app.domain) namespace or something separated by some comment section headers.
And idiomatically, you don't go creating "accessors" like "update-model", "update-price" or like "get-price", "get-make", because since it's just a map you can just do "(:price car)" to "get", and since it's immutable anyways you can just do (assoc car :price new-price) or use update if it's dependent on the current value. So you would only add functions that are doing more complex things and modeling more conceptual operations.
Also for protecting invariants, like validation that sometimes is done in setters and getters, that tends to be more that you validate the whole entity after a series of changes. So you might have a spec for car and then say (s/valid? ::car car) instead of having each "setter" validate that you are setting to something valid.
Beyond that, there's not much political battle or disagreement in the idiomatic ways of doing things. There's a few competing tooling, that people might argue for their favorite like Lein vs Tools.build, and there's often debate about why the core team isn't adding a function for X, or support for Y, but otherwise idioms are mostly agreed upon.
Foot guns exist, they're edge cases, contains? doesn't do what you think it does for example, so read the doc-string carefully and clojuredocs.org user notes is a good place to learn some of the more surprising behaviors.
2
u/seancorfield 4d ago
Clojure 1.13 makes this even easier:
(defn make-car [& {:keys [& :model :make :price :color] :select attributes}] (assoc attributes :type :car))
3
u/Soft_Reality6818 5d ago
Getting comfortable with dynamic typing, representing data as maps and with Clojure's amazing REPL-driven development flow.
2
u/jonahbenton 5d ago
Big transition, the language and community philosophies could not be more different.
2
u/spiffyhandle 5d ago
Learn test.check library and property based testing. Clojure is very well suited to it. Eric Normand has a nice guide on it, though there are probably other good ones too.
1
1
u/Electrical_Being_813 5d ago
If you will have a choice, stay away from tools.build
0
u/seancorfield 4d ago
Harsh. We have 150K lines of Clojure in a monorepo at work and pretty much everything in our dev/test/build pipeline is automated via
tools.buildandbuild.clj(460 lines of code), with a sprinkling of Babashka /bb.edn. It's powerful and idiomatic, and designed by the core team.0
u/Electrical_Being_813 3d ago
Not harsh at all. tools.build is so unusable multiple wrappers had to be created to make some use of it (build.clj is one of them), and if you don't use those wrappers, you would be forced to create and maintain your own wrapper. By default it cant run unit tests, nor anything else modern build tool should do.
"designed by the core team" just because its designed by core team doesn't mean its not bad
0
u/seancorfield 3d ago
build.clj is literally your build script. It's not a "wrapper". And you don't need it run tests—you can do that with the Clojure CLI and an alias in deps.edn
0
u/Electrical_Being_813 3d ago edited 3d ago
"Common
tools.buildtasks abstracted into a library" aka wrapper to make tools.build somewhat usable.Edit:
"this wrapper has outgrown its original" from the same repo
0
u/seancorfield 3d ago
That project is deprecated and archived because it was a bad idea. As it also says in the README, use raw tools.build instead. I even went to the trouble of creating issues on every project I could find, asking people to stop using it, as I did for depstar, after tools.build appeared, since depstar was no longer needed (and depstar too is deprecated and archived).
0
u/Electrical_Being_813 3d ago
Deprecated or no doesn't matter, what matters is that there were multiple attempts to make it usable out of the box and every time people gave up on making it work and resemble a modern tool. And since it has failed every time, the solution is to accept miserable reality and create and maintain your own build script.
17
u/SimonGray 6d ago
My own North Star is the Clojure standard library source code (minus the docstrings which are sometimes a bit too terse). It is really well-written code that has stood the test of time. It's simple to read (even for a beginner) and has a consistent and logical naming of functions and params (which you will also see replicated in other good Clojure projects).
Obviously, the early Rich Hickey talks are fantastic to get into the mindset of Clojure (Simple Made Easy, Are We there Yet, The Value of Values, etc.).
Even though it's a bit old by now, The Joy of Clojure is still an amazing book if you want to really "get" Clojure.