I don't Git it

Hey there,

It’s Robin from CFD Engine and I need your help with this one. I’ve been experimenting with using Git to manage my CFD projects and I can’t quite make it fit.

I just don’t Git it. It feels like it should be a fit, but 20 years of custom & practice are holding me back.

Git what now?

Before I git carried away, I should spend a few lines on what Git is. In short, it’s an open-source tool, used by developers, to track changes to source code.

Tracking changes allows you to trace the history of your code, from its first words, to its latest version, including who made what change, when & why.

Saved changes are called commits in case you’re wondering when that term crops up later 👍

It’s decentralised, meaning you can work locally, by yourself, or remotely, with lots of other people. It has great features for transferring data to & from remote machines, plus neat ways of figuring out how changes, made by different people at different times, should all fit together.

You may have come across it when you installed OpenFOAM (especially if you’re building from source). It’s the tool that both OpenCFD & The Foundation use to manage their respective OpenFOAM codebases.

In this issue, I’m not talking about using Git to manage source code, I’m talking about using it to manage your CFD projects.

It’s an immensely powerful and flexible tool. But it can be immensely complex & it needs a different way of working to get the best out of it.

And that’s where my problems start…

A typical project

A typical project of mine looks something like this:

My typical OpenFOAM project layout - one directory per run

One folder for each CFD run, which contains (or links to) all of the files needed to run it. There are shared folders, templates & included files sprinkled throughout, but that’s the general gist.

There’s usually a little text file in there (runList.txt) giving a handy index of which run is which & the basic relations between them.

I’ve been doing something like this for 20+ years & it’s a hard habit to break.

A Git-enabled version looks more like this:

An OpenFOAM project with all runs committed to a Git repository

A single folder (a Git repository) with all of the run configurations represented by commits (nodes in the line graph) which form a kind of family tree, showing how the runs are linked.

Note: I don’t typically keep results in these folders, just run config, we may well differ on that.

Work In Progress

I make use of Git when I’m building a baseline case, committing changes as I iterate towards the final run config (the one historically labelled R01_v3_Final_Final_Fixed 🤫).

Say I (randomly) wanted to go back to the mesh recipe I was using last Tuesday? No problem if the changes are tracked in Git. A little more difficult if I didn’t stash Tuesday’s snappyHexMeshDict somewhere safe.

However, committing geometry files, meshes and results to your Git repository isn’t such a great idea. It can be done, but tracking changes to large binary files is not something that Git excels at.

So, despite all Git’s promise, this is where I run out of steam.

Here’s an idea

I’m convinced that there’s something in this though, particularly for those of us using remote compute. I could see it working along these lines…

Each project lives in its own local Git repository. All changes are tracked and, when appropriate, they’re pushed to a remote repository (for backup, if nothing else, but also for collaboration).
New runs are tagged which, when pushed to the remote repository, automatically triggers their running in the cloud.
Geometry files are stored in cloud storage & retrieved at run time.
Results are archived in cloud storage & not version controlled.

This is roughly where I see my future workflow. But, is it too different to be a fit for anyone else? 🤔

I’m thinking too much (again)

I feel like Git has something to offer us as OpenFOAM users. I just can’t shrug off 20 years of multiple directories, managing projects with a little text file and checking for changes with diff -qr CASE1 CASE2.

Perhaps, in this case, Git is a hammer looking for a nail & the switching costs don’t outweigh the benefits?

I don’t have an answer, so I’m going to hand it over to you. How do you manage your projects? What do your project directories look like? How do you keep track of changes? Do you have any nifty tools that help? Or, is it all managed using process, custom & practice? Keen to hear back from you – especially if you have this all figured out.

Until next week, stay safe,