data engineering
Contact

Publishing a GitHub action to clean LookML: Looker Janitor

Aug 1, 2024

I wanted a way to keep my Looker view files consistent and comprehensive, so I created a GitHub Marketplace Action to do just that.

Repos:

GitHub Marketplace:

The Problem

Every time I open a Looker view file, it looks slightly different from the one I just opened before. I don't mean its content, I mean its structure: ordering of dimensions, ordering of field types, ordering of their parameters, etc...are all slightly different. Enough to add just that bit of extra mental load that could be easily avoided.

Also, not all fields have the parameters that I expect them to have and finding which ones are missing the parameters we need, is a hassle: e.g. which average measures are missing a value format parameter?

Is the primary key at the top of the page?

Are filters, parameters, dimensions, measures all grouped together or are they mixed together?

The Solution

I created a GitHub Action to sort fields, either by name or label (including if the label is localized), groups by field type, and orders parameters in a uniform fashion. Also, it will check for a given list of required parameters (per field type) so that those can be annotated as part of a PR review process using a tool like Review Dog.

I called the action Looker Janitor. Check it out on the GitHub Marketplace

LookML Parsing

At first, I created my own parser for LookML. To do so, at first, I used a set of regex rules based on the type of parameter I was dealing with. This worked fine, but then I decided to not let it become something I'd need to maintain every time there was an update to LookML. I decided to use the lkml Python package instead for parsing purposes.

Testing

I wanted to test every input in both python and as part of the action. So I created a test setup script that generates tests for all inputs in python as well as an action that uses the published action to test the same inputs. This way I get close to end-to-end testing, closely simulating the actual real-world usage of the tool.

---
Last update: Aug 1, 2024
Privacy