On wrapping DSLs with more powerful languages
Recently I discovered Apple's Pkl language for generating static configuration files such as JSON and YAML. I found the idea ludicrous: writing code in a powerful language to generate code in a weaker language, which is then consumed by a program written in another powerful language. I mean, at this point why not get rid of the weaker language altogether and just write code in the initial powerful language? Why sandwich a weak language in between two powerful ones?
Here are some more similar examples that come to mind:
- CSS being extended by Sass
- CMake generating Make files
- Vim's configuration language Vimscript being surpassed by "Vim9 script"
What these pairs of languages have in common is that a more powerful language is being created to address the limitations of a simpler domain-specific language (DSL).
Consider CSS, which started off as a simple and elegant DSL, more than enough for styling static HTML documents. However, as web applications have increased in complexity and the demands on CSS have grown, its limitations have transformed from a bliss to a curse for web developers.
For example, you couldn't define variables, which meant that if you had a brand color (#4A90E2) that you needed to change, you would have to do a project-wide search and replace. Also, you couldn't write functions, or reusable snippets of code, which led to duplication.
Fundamentally, CSS lacked means of abstractions.
This led to the creation of Sass, a full-fledged language that provides many means of abstraction, but compiles to CSS to maintain compatibility.
I think this trend can be summarized in the following way. First, developers create a small and simple DSL to elegantly express themselves in a very specific domain (e.g. rules in Makefiles). They don't include means of abstraction (such as functions and variables) because the use-case doesn't warrant them. Over time, the DSL becomes popular and developers start demanding more from it (e.g. functions with arguments). When they can't express a new idea in an elegant manner, they resort to duplication and hacks, makig the DSL code unreadable and unmaintainable. Eventually, somebody fed up with the amount of duplication creates a new language that provides the abstractions necessary to deal with the new ideas, but is forced to make it compile to the DSL for compatibility reasons.
I believe that this trend of wrapping a DSL with a more powerful language is a workaround rather than a proper solution, which leads to technical debt. The reason for this is that It makes the stack of abstractions grow unnecessarily: there are now two languages instead of one which need to be maintained and kept compatible. This is an instance of accidental complexity rather than essential complexity.
I think that when writing code in a DSL becomes unwieldy, a proper solution is to either update the DSL with the needed abstractions or create a new, more powerful language and update the target program to be able to read it.
I'm not saying that Pkl, CMake, Sass, etc. are evil and should not be used. I recognize that hacks and crutches have a place to exist, that there are circumstances such as deadlines, backwards compatibility and inertia. I just want us to recognize and point out when a perceived solution is in fact a hack so that developers don't accidentally normalize them and are able to factor in the potential technical debt when considering such solutions.