techWebsite/content/posts/cli-renaissance/index.md

+++
title = "A modern CLI renaissance?"
date = 2024-03-04T12:20:02-06:00
draft = true
+++
<!--
tool,link,year,language
ls,,1961,c
cat,,1971,c
cd,,1971,c
cp,,1971,c
man,,1971,c
rm,,1971,c
grep,,1973,c
diff,,1974,c
sed,,1974,c
bc,,1975,c
make,,1976,c
vi,,1976,c
TeX,,1978,pascal
bourne shell,,1979,c
awk,,1985,c
screen,,1987,c
bash,https://www.gnu.org/software/bash/,1989,c
zsh,https://www.zsh.org/,1990,c
vim,https://www.vim.org/,1991,c
midnight commander,,1994,c
ssh,,1995,c
curl,https://github.com/curl/curl,1996,c
fish,https://fishshell.com/,2005,c++ (currently being rewritten in rust)
fossil,https://fossil-scm.org/home/doc/trunk/www/index.wiki,2006,c
tmux,https://github.com/tmux/tmux,2007,c
git,https://git-scm.com/,2008,c
the silver searcher,https://github.com/ggreer/the_silver_searcher,2011,c
jq,https://github.com/jqlang/jq,2012,c
go 1.0,https://go.dev/,2012,go
fzf,https://github.com/junegunn/fzf,2013,go
eza/exa,https://github.com/eza-community/eza,2014,rust
neovim,https://neovim.io/,2015,c
pueue,https://github.com/Nukesor/pueue,2015,rust
rust 1.0,https://www.rust-lang.org/,2015,rust
just,https://github.com/casey/just,2016,rust
micro,https://micro-editor.github.io/,2016,go
nnn,https://github.com/jarun/nnn,2016,c
ripgrep,https://github.com/BurntSushi/ripgrep,2016,rust
fd,https://github.com/sharkdp/fd,2017,rust
bat,https://github.com/sharkdp/bat,2018,rust
broot,https://dystroy.org/broot/,2018,rust
difftastic,https://difftastic.wilfred.me.uk/,2018,rust
hyperfine,https://github.com/sharkdp/hyperfine,2018,rust
lazygit,https://github.com/jesseduffield/lazygit,2018,go
lsd,https://github.com/lsd-rs/lsd,2018,rust
nushell,https://www.nushell.sh/,2018,rust
scc,https://github.com/boyter/scc,2018,go
sd,https://github.com/chmln/sd,2018,rust
git-delta,https://github.com/dandavison/delta,2019,rust
bottom,https://github.com/ClementTsang/bottom,2019,rust
grex,https://github.com/pemistahl/grex,2019,rust
starship,https://starship.rs/,2019,rust
tre,https://github.com/dduan/tre,2019,rust
typst,https://typst.app/,2019,rust
diskonaut,https://github.com/imsnif/diskonaut,2020,rust
helix,https://helix-editor.com/,2020,rust
duf,https://github.com/muesli/duf,2020,go
pijul,https://pijul.org/,2020,rust
zellij,https://zellij.dev/,2020,rust
zoxide,https://github.com/ajeetdsouza/zoxide,2020,rust
btop,https://github.com/aristocratos/btop,2021,c++
ast-grep,https://github.com/ast-grep/ast-grep,2022,rust
yazi,https://github.com/sxyazi/yazi,2024,rust
-->

<!--
open tmp.csv |
sort-by year tool |
update tool {|i| if ($i.link != "") {$"[($i.tool)]\(($i.link)\)"} else {$i.tool}} |
select tool year language |
to md |
xsel -bi
-->

<!--
ls specifically can trace its history to 1961 
-->

Take a look at this [table](#appendix-the-tools) at the bottom of the page.
I'll wait.
Notice the relative scarcity between ~1995 and ~2015?
Id like to talk about a trend I've seen these past few years,
where people are rewriting and rethinking staples of the command line interface,
why I think this trend might be happening,
and why I think this trend is a good thing.

## History

The terminal and the command line interface have been staples of computer user interfaces before computer monitors were even available,
with some of the first computers offering an interactive mode in the late 1950's.
The recognizable Linux terminal traces its linage to the very first version of Unix in 1971.
Many utilities that a Linux user interacts with every day,
commands like `rm`, `cat`, `cd`, `cp`, `man`, and a host of other core commands trace their initial versions to this first version of Unix.
Other tools are a bit newer, such as `sed` (1974), `diff` (1974) `bc` (1975), `make` (1976), or `vi` (1976).
There were a few more tools introduced in the 90's, such as `vim` (1991) and `ssh` (1995), but you get the picture.
The majority of the foundational CLI tools on a Linux pc, even one installed yesterday, are older than Linux itself is.

## Ok, so?

Now, there's nothing wrong with this.
The tools still work fine, but in the half-century since they were first written,
terminals and the broader Linux ecosystem have all changed.
Terminals now have capacity to display more colours, Unicode symbols, and even inline images.
Terminal programs now coexist with graphical user interfaces,
and only a small subset of computer users even know they exist,
whereas in the past, terminals were the only way one interacted with the computer.

Perhaps more importantly, our knowledge has expanded:
our knowledge of user interfaces,
of what works and what doesn't,
of what usecases are common and what usecases are niche,
the way that error messages can teach,
the value of a good out of the box experience,
and the value of documentation that is easy to find and digest.

These changes to the environment surrounding CLI apps in recent years have
led to a resurgence in development of command line utilities.
Instead of just developing completely new tools or cloning old tools,
I've noticed that people are rethinking and reinventing tools that have existed since the early days of Unix.

This isn't just some compulsive need to rewrite every tool out there in your favorite language.
People are looking at the problem these tools set out to solve,
and coming up with their own solutions to them,
exploring the space of possible solutions and taking new approaches.

Its this exploration of the solution space that id like to take a look at:
the ways that tools are changing,
why people are changing them,
and what kicked off this phenomenon.

## The lessons learned from the past

A large amount of the innovation in the area, I think,
can be attributed to lessons that have been learned in 50 years of using software:
sharp edges we have repeatedly cut ourselves on,
unintuitive interfaces that repeatedly trip us up,
and growing frustration at the limitations that maintaining decades of backwards compatibility imposes on our tools.

These lessons have been gathering in the collective conciousness;
through cheatsheets, guides, and FAQs;
resources to guide us through esoteric error messages, complex configurations, and dozens upon dozens of flags.

Id like to go over a couple of the more prominent lessons that I feel terminal tools have learned in the past several decades.

### A good out of the box experience

While configurability is great,
one should not need to learn a new configuration language and dozens or hundreds of options to get a usable piece of software.
Configuration should be for customization, not setup.

One of the earliest examples of this principle may be the fish shell.
Both zsh and fish have powerful prompt and autocompletion engines,
but zsh requires you to setup a custom prompt and enable completions in order to use the features that set it apart from the competition.
With no config file, zsh is no better than bash.
When starting fish for the first time, however,
its powerful autocompletion and information rich prompt are front and center with no configuration required.
Of course, fish still has the same level of configurability as zsh, it just also has sensible defaults.

To demonstrate my point, this is the default prompt for zsh with no configuration.
It *only* shows the hostname, none of the advanced features you can get out of a zsh prompt even without plugins.
![zsh prompt, only shows hostname](zsh-prompt.png)
Here is bash's prompt.
It actually gives more info than zsh's, even though zsh can do more when properly configured.
![bash prompt, shows hostname and current directory](bash-prompt.png)
And here is fish's default prompt.
It has a few colours, shows everything the bash prompt does, and additionally shows the git branch we are on.
![fish prompt, has colours, shows hostname, current directory, and git info](fish-prompt.png)

Text editors are another great example of the evolution of out of the box defaults.
Vim and Neovim both improved on their predecessors,
but much of that improvement is locked behind extremely complex configuration experiences and plugins.
Here's four different terminal text editors with no configuration applied:

![vi, vim, neovim, and helix editors in their default
configuration](editors.png)

Vi, (top left) is our baseline, and, as far as I can tell, doesn't actually
support much for configuration. What you see out of the box is more or less
whats there.

Vim (top right) greatly improved on Vi, adding things such as syntax highlighting, line numbers, spellchecking, split windows, folding, and even basic autocompletion.
However, everything but syntax highlighting is either extremely clunky or outright disabled without configuration.
(for example, the earliest things I did when I first made a `.vimrc` was to enable indent folding,
make some better keybinds for navigating windows, and adding a line number ruler to the side)

Neovim (bottom left) further improved on Vim, adding support for Treesitter and the Language Server Protocol, 
but the out of the box experience is the *exact* same as vim!
In order to take advantage of the LSP and Treesitter support, you have to install plugins,
which means learning a Nvim package manager, learning how to configure LSPs,
and configuring a new LSP for every language you want to use it with
(or finding out about Mason and being OK with having multiple levels of package management in your Nvim install alone).
Don't get me wrong: Neovim is a great editor once you get over the hump.
I still use it as my daily driver, but so much of its functionality is simply hidden.

Then we have the Helix (bottom right) editor.
Slightly glaring default colour scheme aside, everything is just *there*.
Helix doesn't have plugin support [yet](https://github.com/helix-editor/helix/discussions/3806),
but it has so much stuff in core that,
looking through my neovim plugins,
pretty much all of them are in the core editor!
(Ironically, the one feature that I feel helix is missing, [folding](https://github.com/helix-editor/helix/issues/1840),
is a core part of neovim, albeit one that requires some configuration to get good use out of).
Helix does have a config file where you can change a huge amount of settings,
but its an extremely usable IDE out of the box thanks to having all of its features enabled by default.

### Helpful error messages

<!-- look at nushells error messages -->

TODO [before](../nushell)

### Concise and discoverable documentation

In my Nvim config, I use [which-key](https://github.com/folke/which-key.nvim),
a plugin that displays available keybindings as you type.
I've been using vim for almost a decade, including a long time without which-key,
so its not like I never learned the keybindings, but I still find which-key useful.
Why is that, you may ask?
Well, because even though I use (n)vim every day, I don't use all the keybindings every day.
I might go months between using, for example, `dap` (delete current paragraph), or `C-w x` (swap current window for next).
Naturally, when you go months without using certain parts of a program, you tend to forget they exist.
Which-key solves that handily, by offering quick, non-intrusive reminders of what is available.
Here's what my which-key config looks like:

![Which-key.nvim, showing contextual key hints](nvim-which-key.png)

Now, which-key and its like have been around for a while,
but other TUI programs have integrated contextual hints without the need for a plugin.
The two that I am aware of are zellij and helix.

![Helix's contextual hint](helix-contextual-hint.png)

Helix both has autocompletion for its built in command line and a contextual hint that appears when you press the first key in a multi key combo.
This drastically helps both new and experienced users learn and remember keybinds without making the editor any less powerful.

![Zellij's hint bar](zellij-hints.png)

Zellij has a bottom bar displaying keybindings available in the current mode.
This has proven invaluable for me, as I don't use a terminal multiplexer much
(On GUI systems, I use the window manager for managing multiple terminals), and as such tend to forget the keybinds.
Though it does take up screen space, and a person who used Zellij every day would most likely disable it,
the hints bar is more than worth it for new and occasional users.

### Common usecases should be easy

Where possible, documentation should not even be required for the most common use cases.
Whenever I want to use `find`, I almost always have to first look at the man page,
as I don't use it quite often enough to memorize it.
But that's totally unneeded! 90% of my uses of `find` take the form of `find ./ -name "*foo*"`.
With [fd](https://github.com/sharkdp/fd), the exact same invocation is a simple `fd foo`, dead simple, no man page needed.
Of course, 10% of the time I'm doing something else and have to look at the manual even with fd,
but the point is that manuals are for when you want to do something with the tool that is not the most common usecase.

There are many other examples as well. How many of your grep invocations are in the form of `grep -R 'foo' ./`?
Most of mine are. [Ripgrep](https://github.com/BurntSushi/ripgrep) shortens that to `rg foo`
while still having all the power of grep when I need it, and it is faster to boot!

This isn't to say that tools should 'dumb themselves down' or hobble themselves to make them easier to use.
However they should keep in mind the most common usecase that their tool is likely to be used in,
and streamline that usecase.

## Shedding historical baggage

Many tools were made for one thing, and over time have evolved into another thing entirely.
This can happen by conscious design or, more commonly,
from an industry or community picking a tool up and using it for something it was not originally designed for.
While hacking tools for uses they were not designed for is always fun and in many cases the only way to do something,
its perhaps better to make a dedicated tool when the design choices made for the old usecases start hindering the new usecases.

A great example of this is [just](https://github.com/casey/just), a command runner heavily inspired by GNU make.
Make was (and in large part still is) a C build system.
As such, it includes features such as implicit rules
(if a file called `foo.o` is needed and there is no explicit rule is there, the C compiler will be invoked on the file `foo.c`
(there are similar rules for C++ and linking))
and file modification time laziness
(fantastic for a build system, needs liberal sprinkling of `.PHONY` rules when used as a task runner).

These features are *good* features when make is being used as a build system,
but another major use of make that has emerged has been as a way to run common tasks.
So alongside `make build` to build your program, you would have `make bootstrap`, `make test`, `make config`, etc.
This is where the design decisions behind make the build system start to hinder make the task runner,
making one learn about make the build system in order to work around those features to use make the task runner.
However, make cant drop these features, both because projects still actively use make as a build system,
and because even makefiles that are just used as taskrunners still work around the footguns and
would be broken by make making large changes to its syntax and semantics.

However, Just was designed from the outset to be a command runner, and as a result,
it is much easier to pick up the just language and make a quick set of commands that can be run.
By leaving behind the old tool, a new tool can be made that better fits the tasks that people use the tool for.

## The languages

Now, looking back at that table, one might notice a pattern in the languages used.
While almost all tools before 2010 or so were written in C,
the language selection after that are dominated by Rust and Go.

Of course, there are exceptions to the pattern.
TeX was written in pascal, Neovim kept C as its primary language,
and there is the occasional new tool written in C or C++.
But there is still a clear pattern in the language choice for newly written tools.

Now, why do you think that pattern has changed?
And have these new languages led to an increase in the number of tools being written?
I think so, and I don't actually think its the languages itself, so much as the libraries surrounding them.

Both Rust and Go have healthy package ecosystems surrounding interaction with the terminal.
Rust has Clap for argument parsing,
crossterm for dealing with ANSI escape codes and other terminal interaction,
and Ratatui for making TUIs.
Go has a similar set of tools, with Cobra for CLI argument parsing,
Viper for config file management integrated with Cobra,
Gocui, tview, and Termui for TUIs,
or Bubbletea for pretty UI components.

These libraries combined with the extra ergonomics offered by the languages themselves,
make the barrier to entry lower,
allowing for more people to experiment with the design and ergonomics of CLI tools.

## Conclusion

TODO
<!-- emphasize that the new tools are not 'better' just because they are new,
but because they take the old tools and learn what did and did not work for them. -->

## Appendix: the tools

This is an extremely unscientific table of command line tools that I have tried, have used, or currently use.
It is assuredly incomplete, but *should* be broadly representative.
The date data has been gathered from the first git commit where available,
wikipedia otherwise, and sorting is by year first, then alphabetical.

|tool|year|language|
|-|-|-|
|ls|1961|c|
|cat|1971|c|
|cd|1971|c|
|cp|1971|c|
|man|1971|c|
|rm|1971|c|
|grep|1973|c|
|diff|1974|c|
|sed|1974|c|
|bc|1975|c|
|make|1976|c|
|vi|1976|c|
|TeX|1978|pascal|
|bourne shell|1979|c|
|awk|1985|c|
|screen|1987|c|
|[bash](https://www.gnu.org/software/bash/)|1989|c|
|[zsh](https://www.zsh.org/)|1990|c|
|[vim](https://www.vim.org/)|1991|c|
|midnight commander|1994|c|
|ssh|1995|c|
|[curl](https://github.com/curl/curl)|1996|c|
|[fish](https://fishshell.com/)|2005|c++ (currently being rewritten in rust)|
|[fossil](https://fossil-scm.org/home/doc/trunk/www/index.wiki)|2006|c|
|[tmux](https://github.com/tmux/tmux)|2007|c|
|[git](https://git-scm.com/)|2008|c|
|[the silver searcher](https://github.com/ggreer/the_silver_searcher)|2011|c|
|[go 1.0](https://go.dev/)|2012|go|
|[jq](https://github.com/jqlang/jq)|2012|c|
|[fzf](https://github.com/junegunn/fzf)|2013|go|
|[eza/exa](https://github.com/eza-community/eza)|2014|rust|
|[neovim](https://neovim.io/)|2015|c|
|[pueue](https://github.com/Nukesor/pueue)|2015|rust|
|[rust 1.0](https://www.rust-lang.org/)|2015|rust|
|[just](https://github.com/casey/just)|2016|rust|
|[micro](https://micro-editor.github.io/)|2016|go|
|[nnn](https://github.com/jarun/nnn)|2016|c|
|[ripgrep](https://github.com/BurntSushi/ripgrep)|2016|rust|
|[fd](https://github.com/sharkdp/fd)|2017|rust|
|[bat](https://github.com/sharkdp/bat)|2018|rust|
|[broot](https://dystroy.org/broot/)|2018|rust|
|[difftastic](https://difftastic.wilfred.me.uk/)|2018|rust|
|[hyperfine](https://github.com/sharkdp/hyperfine)|2018|rust|
|[lazygit](https://github.com/jesseduffield/lazygit)|2018|go|
|[lsd](https://github.com/lsd-rs/lsd)|2018|rust|
|[nushell](https://www.nushell.sh/)|2018|rust|
|[scc](https://github.com/boyter/scc)|2018|go|
|[sd](https://github.com/chmln/sd)|2018|rust|
|[bottom](https://github.com/ClementTsang/bottom)|2019|rust|
|[git-delta](https://github.com/dandavison/delta)|2019|rust|
|[grex](https://github.com/pemistahl/grex)|2019|rust|
|[starship](https://starship.rs/)|2019|rust|
|[tre](https://github.com/dduan/tre)|2019|rust|
|[typst](https://typst.app/)|2019|rust|
|[diskonaut](https://github.com/imsnif/diskonaut)|2020|rust|
|[duf](https://github.com/muesli/duf)|2020|go|
|[helix](https://helix-editor.com/)|2020|rust|
|[pijul](https://pijul.org/)|2020|rust|
|[zellij](https://zellij.dev/)|2020|rust|
|[zoxide](https://github.com/ajeetdsouza/zoxide)|2020|rust|
|[btop](https://github.com/aristocratos/btop)|2021|c++|
|[ast-grep](https://github.com/ast-grep/ast-grep)|2022|rust|
|[yazi](https://github.com/sxyazi/yazi)|2024|rust|