Introduction to Dotfiles

Author(s): Tiu Wee Han
Reviewers: James Pang, Liu Yiwen

What are Dotfiles?

Dotfiles are plain text configuration files on Unix like systems (e.g. MacOS, Linux, BSD). Dotfiles store settings of almost every application, service and tool running on your system. These files control the behavior of applications from boot to termination and everything in between.

Some common uses of dotfiles include:

• Scripts that run when a Shell is started: .profile, .bashrc, .zshrc
• Customising text editors: .nanorc, .vimrc, .emacs.d
• Storing settings for commonly used tools (e.g. git, ssh): .gitconfig, .gitignore, .ssh/config

Why Dotfiles?

Benefit 1: Easy and Centralised Way to Configure Applications

With dotfiles, users have an easy and centralised way to configure their environment and applications. The usage of a file to store configurations also makes it easily shareable and reuseable by other people, as opposed to having to change settings within the application. Dotfiles can be used to configure almost all popular command line tools. In this section, we will explore dotfiles related to git and ssh - tools frequently used by developers.

Feature 1.1: Git Dotfiles

git is an indispensible tool for many developers and projects, and it is undoubtedly the most popular version control software. Hence, it is highly beneficial to learn how to configure git dotfiles.

Global settings for git, such as the user's name, email and GitHub username can be specified in the ~/.gitconfig file. The code below is a bare-bones ~/.gitconfig file.

[user]
  name = John Doe
  email = johndoe@gmail.com.

[github]
  user = johndoe123

Apart from just storing basic details, it can also contain other information such as merge tools, git color scheme and aliases. This provides a centralised location for a developer to view and edit their git configurations.

Another commonly used git related dotfile is the .gitignore file, which contains a list of files and directories that a developer wants to exclude from git's version control, such as packages, binaries or secret information. Some files commonly included in .gitgnore include:

node_modules/ # Project specific packages for node projects
.DS_Store     # MacOS file to store folder layout information in the GUI
.vs_code      # Project specific VS code text editor configuration
bin/          # Binary executables

This example also illustrates how using dotfiles can add project specific configurations that all developers working on a project will adhere to universally, as opposed to system level settings which may vary from developer to developer.

Feature 1.2: SSH Dotfiles

For developers and engineers working with remote servers, ssh is arguably the most essential tool for connecting and running commands on remote servers. However, for many beginners, working with ssh can be a daunting prospect. Configuring dotfiles is a good way to both simplify the usage of ssh and learn how to master it effectively.

Say a user John wants to connect to a remote server called remoteA. Typically, he will run the following command to connect to the server

ssh -i ~/.ssh/id_rsa john@remoteAServer.com

However, this is a relatively long command, and can be tedious to type especially if John accesses remoteA frequently. Thankfully for John, he can configure the different hosts by adding the following lines in the ~/.ssh/config file:

Host remoteA
    Hostname remoteAserver.com
    User john
    IdentityFile ~/.ssh/id_rsa

With the configurations above, John is now able to simply just run the following to connect to remoteA:

ssh remoteA

This evidently saves time and effort needed to connect to a server. Aside from just connecting to servers, ssh can also be used for other purposes, all of which are easily configurable using dotfiles. These include SSH proxies, launching GUI applications remotely and SSH forwarding.

Benefit 2: Personalised Shortcuts and Commands

Dotfiles allow users to create custom shortcuts and commands, including aliases, functions and key-bindings, that can tremendously improve productivity. These can be both system level and project specific. In this section, we will look into aliases and functions.

Feature 2.1: Aliases

An alias is a shortcut command to a longer and more complex command. A new alias is defined by assigning a string with the command to a name in the format alias <name>=<command>. Aliases are often set in the ~/.bashrc or ~/.zshrc file. The examples below illustrate the use of aliasing.

alias c='clear'
alias lf='/bin/ls --color -CF'
alias ll='ls -l --color=auto'
alias ls='ls --color=auto'
alias r='fc -s'
alias vi='vim'

In this case, a user is able to type c instead of typing out clear in order to clear the terminal screen, a very commonly used command. Aliases are especially useful for abstracting long commands that are used often, and when compounded can save the user a lot of time and effort.

Feature 2.2: Functions

Much like aliases, functions allows a user to abstract longer commands into short commands. However, there is additional functionality such as being able to pass command line arguments and flags. This makes the commands more extensible and reuseable for a wide array of functionality.

The example function mcd below takes in a single argument represented by $1, creates a directory with the same name and enters it in a single command.

function mcd() {
  mkdir -p $1;
  cd $1;
}

So if the user were to type the following into the terminal:

mcd my_folder

it would create a folder called my_folder and enter into it in a single step.

A highly useful and commonly used function is the extract function. This combines a lot of utilities to allow you to decompress just about any compressed file format, such as tar, rar and zip:

function extract() {
 if [ -z "$1" ]; then
    # display usage if no parameters given
    echo "Usage: extract <path/file_name>.<zip|rar|bz2|gz|tar|tbz2|tgz|Z|7z|xz|ex|tar.bz2|tar.gz|tar.xz>"
    echo "       extract <path/file_name_1.ext> [path/file_name_2.ext] [path/file_name_3.ext]"
    return 1
 else
    for n in $@
    do
      if [ -f "$n" ] ; then
          case "${n%,}" in
            *.tar.bz2|*.tar.gz|*.tar.xz|*.tbz2|*.tgz|*.txz|*.tar) 
                         tar xvf "$n"       ;;
            *.lzma)      unlzma ./"$n"      ;;
            *.bz2)       bunzip2 ./"$n"     ;;
            *.rar)       unrar x -ad ./"$n" ;;
            *.gz)        gunzip ./"$n"      ;;
            *.zip)       unzip ./"$n"       ;;
            *.z)         uncompress ./"$n"  ;;
            *.7z|*.arj|*.cab|*.chm|*.deb|*.dmg|*.iso|*.lzh|*.msi|*.rpm|*.udf|*.wim|*.xar)
                         7z x ./"$n"        ;;
            *.xz)        unxz ./"$n"        ;;
            *.exe)       cabextract ./"$n"  ;;
            *)
                         echo "extract: '$n' - unknown archive method"
                         return 1
                         ;;
          esac
      else
          echo "'$n' - file does not exist"
          return 1
      fi
    done
fi
}

The use of functions essentially allows a user to create custom command-line tools to suit their needs. Defining the function above in ~/.bashrc or ./.zshrc will load it whenever a new shell is created.

Benefit 3: Shell Styling and Utility

Dotfiles are also used to style shells (e.g. bash, zsh, fish) and enhance or add features to them. The styling of shells not only improves the overall aesthetic, it also makes them more readable (shell output can be tedious to read and digest). Features such as adding system information to the prompt, tab completion and syntax highlighting also improves the overall utility and feedback of the shell, contributing to a better developer experience.

Feature 3.1: Prompt

The PS1 environment variable provides information for configuring and styling the Shell prompt. By default, PS1 is set to display only the username, host name and current working directory in the prompt.

_{Figure 2. Default prompt}

PS1 can be configured to add (or remove) features, such as status code of the previous command, git branch, git status, dates, etc. In Figure 3 below, the prompt is able to detect and show the git branch and status when it is inside a git repository, a useful and convenient feature for developers.

_{Figure 3. Enhanced prompt}

Feature 3.2: Tab Completion

Tab completion is a feature that allows the user to select options from a drop down menu. This is a default feature when using zsh, but can be configured using dotfiles when using other shells like bash.

_{Figure 4. cd tab completion (Tutorial)}

Tab completion can also be used for other purposes, such as selecting flags from a drop down menu, complete with an accompanying description.

_{Figure 5. Flag autocomplete}

Feature 3.3: Syntax Highlighting

Dotfiles are also often used to style and colorise the terminal. This is not just for aesthetic reasons - the use of color enhances readability of programs and makes it easier to debug. One such example would be the use of syntax highlighting as seen in Figure 6 below. It is especially useful for long shell commands, escape sequences or string interpolation.

_{Figure 6. Syntax Highlighting (Tutorial)}

Management Strategies

There are many stratgies for managing dotfiles, but virtually all of them revolve around storing them in git repositories. The nature of dotfiles make git and ideal management tool - they continously evolve over time as the user adds more configurations, and it may be useful to track old dotfiles for future reference.

Another major benefit of managing dotfiles with git is that they can then be pushed to online repositories like GitHub or GitLab. These dotfiles can then be easily reused on other systems by simply pulling from these online repositories, allowing users to port their painstakingly created dotfiles anywhere. Since certain dotfiles may also vary across different systems (e.g. bash configurations may be different), users can use different branches to differentiate between these systems. It almost seems as though git was created for managing dotfiles!

The main difference between different strategies are how these dotfiles should be linked from the git repository into the system, since dotfiles can exist in different directories. The 3 most commonly used strategies are detailed below:

1. Use a git worktree:
   The strategy uses a lesser-known Git functionality of storing the Git worktree separately from the Git directory. This consists of a Git bare repository in a "side" folder (like $HOME/.cfg or $HOME/.myconfig) using a specially crafted alias so that commands are run against that repository and not the usual .git local folder, which would interfere with any other Git repositories around.

2. Use symlinking:
   This strategy involves using symlinks (The ln command) to create a shortcut linking to the dotfiles repository. The ln command is used to create links in your (unix-based) system. The syntax is ln -s <actual location of the file> <name and location you want to see that file under>. For example, the command below will result in the gitconfig in the dotfiles directory to be accessible from the ~/.gitconfig location, which is where Git is expecting to see all the Git preferences set.

   ln -s ~/dotfiles/gitconfig ~/.gitconfig
   

3. Use an existing dotfiles management tool such as yadm: yadm is a command-line tool for managing dotfiles. It provides many features out of the box, which saves the user the time needed to manually configure his/her own dotfiles. It also supplies additional features, such as the ability to manage a subset of secure files, which are encrypted before they are included in the repository. However, the caveat of this strategy is that it requires new systems to have yadm installed, which significantly reduces the portability.

Getting Started

It is easy to get started with dotfiles - a simple way to start is just by adding aliases for commonly used commands! For developers looking to optimise their productivity even further with dotfiles beyond the examples in this article, here are some useful readings/resources to get started with dotfiles:

• Unofficial Guide to Dotfiles on GitHub
• Introduction to Dotfiles
• Awesome dotfiles: A curated list of dotfiles resources.

Conclusion

In a nutshell, dotfiles are highly useful tools that can provide virtually unlimited customisability, and can tremendously improve the productivity of a developer. For junior developers that are keen on improve their linux or shell scripting knowledge, dotfiles are a good way to get started.