CI-CD Pipeline for Obsidian Notes

2025-05-24

1. Write a python script to move files from Obsidian vault to Hugo content path

 1import frontmatter
 2import os
 3import shutil
 4
 5def main(vault_path, hugo_path):
 6    notes_to_publish = find_publish_notes(vault_path)
 7    for note in notes_to_publish:
 8        note_name = os.path.basename(note)
 9        publish_path = os.path.join(hugo_path, note_name)
10
11        # Check if the file already exists in the publish directory
12        if os.path.exists(publish_path):
13            print(f"File {note_name} already exists in {hugo}. Skipping.")
14            continue
15        
16        # Copy the file to the publish directory
17        shutil.copy(note, hugo_path)
18        print(f"Copied {note_name} to {hugo}")
19
20# Find notes in the vault with the publishNote frontmatter set to true
21def find_publish_notes(path):
22    publish_notes = []
23    for root, dirs, files in os.walk(path):
24        for file in files:
25            if file.endswith(".md"):
26                file_path = os.path.join(root, file)
27                with open(file_path, 'r') as f:
28                    note = frontmatter.load(f)
29                    if("publishNote" in note.metadata and note.metadata['publishNote'] == "true"):
30                        publish_notes.append(file_path)
31    return publish_notes
32
33if __name__ == "__main__":
34    vault = "/var/home/josh/Documents/Obsidian/josh-obsidian-vault"
35    hugo = "/var/home/josh/gitea/notes-joshrnoll/content/notes"
36    main(vault, hugo)

2. Write Gitea Actions to publish website changes on git commit:

First attempt - runner doesn’t have hugo installed by default. Added install command to first step:

1- name: Install Hugo
2
3run: /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" && brew install hugo

The run still failed after that with this in the logs:

Warning: /home/linuxbrew/.linuxbrew/bin is not in your PATH.
  Instructions on how to configure your shell for Homebrew
  can be found in the 'Next steps' section below.
==> Installation successful!
==> Homebrew has enabled anonymous aggregate formulae and cask analytics.
Read the analytics documentation (and how to opt-out) here:
  https://docs.brew.sh/Analytics
No analytics data has been sent yet (nor will any be during this install run).
==> Homebrew is run entirely by unpaid volunteers. Please consider donating:
  https://github.com/Homebrew/brew#donations
==> Next steps:
- Run these commands in your terminal to add Homebrew to your PATH:
    echo >> /root/.bashrc
    echo 'eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"' >> /root/.bashrc
    eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"
- Install Homebrew's dependencies if you have sudo access:
    sudo apt-get install build-essential
  For more information, see:
    https://docs.brew.sh/Homebrew-on-Linux
- We recommend that you install GCC:
    brew install gcc
- Run brew help to get started
- Further documentation:
    https://docs.brew.sh
/var/run/act/workflow/1: line 2: brew: command not found

Looks like after installing Homebrew, you need to add it to $PATH (Homebrew is installed by default on Bluefin linux, which is what i’m using as my main workstation at the time of this writing, so I totally missed this).

I added the following to first step in the action:

1      - name: Install Homebrew and Hugo
2        run: |
3          /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
4          echo >> /root/.bashrc
5          echo 'eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"' >> /root/.bashrc
6          eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"
7          brew install hugo

After adding this, the above step in the action finally ran successfully, however, the next step, which was simply meant to run the hugo command to build the site failed with the error /var/run/act/workflow/2: line 2: hugo: command not found… I suspect that modifying the environment doesn’t persist between steps.

As I suspected. The docs say:

Because steps run in their own process, changes to environment variables are not preserved between steps.

My guess is that when running brew install hugo, homebrew is actually adding the path to hugo’s executable to the system’s $PATH, which isn’t persisted between steps because this is a modification of an environment variable.

I’ll have to look into this, but it seems like the simplest fix is to add everthing into one step:

 1      - name: Install Homebrew and Hugo
 2        env:
 3          AZURE_STORAGE_ACCOUNT_AUTH_MODE: key
 4          AZURE_STORAGE_ACCOUNT: ${{ secrets.AZURE_STORAGE_ACCOUNT }}
 5          AZURE_STORAGE_SAS_TOKEN: ${{ secrets.AZURE_STORAGE_SAS_TOKEN }}
 6        run: |
 7          /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
 8          echo >> /root/.bashrc
 9          echo 'eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"' >> /root/.bashrc
10          eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"
11          brew install hugo
12          hugo && hugo deploy --target=production

This worked! 🥳🥳

3. Write a bash script that runs git add, commit and push

 1#!/bin/bash
 2
 3set -euo pipefail # Set strict mode to exit script on errors
 4
 5REPO_DIR="/var/home/josh/gitea/publish-notes"
 6HUGO_DIR="/var/home/josh/gitea/notes-joshrnoll"
 7CURRENT_DATE=$(date '+%Y-%m-%dT%H:%M:%S%:z')
 8
 9cd "$REPO_DIR"
10
11source ./venv/bin/activate
12
13python ./copy-notes.py
14
15cd "$HUGO_DIR"
16
17git checkout main
18git add .
19
20set +euo pipefail # Unset strict mode to allow for custom error handling
21
22git commit -m "Update notes $CURRENT_DATE"
23if [ $? -ne 0 ]; then
24    echo "No changes published, skipping push"
25    exit 0
26fi
27
28git push origin main
29if [ $? -ne 0 ]; then
30    echo "Issue pushing to remote repository"
31    exit 1
32fi

#ci-cd #automation #gitea #hugo