Making it some what dynamic

2023-4-27

Table of Contents

Reason for this

Although this website is meant to be written the form of a book, I realized it is nessary to add a bit more blog-like features for the returning readers. This way, both the "exploration" of the book and "what's been added since last time I visited" aspect of the expirence can be covered.

The solution for now

I embeded a javascript file into the welcome page markedown file.

The code look like this 


fetch("https://api.github.com/<my-repo-owner>/<my-repo-name>/commits?per_page=3&sha=master", {headers: header})
  .then(response => response.json())
  .then(data => {
  let promises = [];
  for (let commit of data) {
      // sort through the fetch
      promises.push(
          fetch("https://api.github.com/repos/<my-repo-owner>/<my-repo-name>/commits/"+commit.sha, {headers: header})
          .then(response => response.json())
          .then(commitData => {
              // Check if the commit includes file data
              if (commitData.files) {
                  const newFilesInThisCommit = commitData.files.filter(file => file.status === 'added' && file.filename.startsWith('book/')  && file.filename.endsWith('.html')  && !file.filename.endsWith('index.html'));
                  return newFilesInThisCommit.map(file => file.filename);
              } else {
                  return [];
              }
          })
      );
  }
  return Promise.all(promises);
  })
  .then(filesInCommits => {
      let html = "<ul>";
      for (let filesInCommit of filesInCommits) {
        for(let file of filesInCommit) {
            //String manimulation
            file = file.substring(5);
            file = file.substring(0, file.length - 5);
            let temp = file.substring(file.lastIndexOf('/') + 1);
            temp = temp.replace(/-/g, ' ');
            temp = temp.replace(/\w\S*/g, function(txt){return txt.charAt(0).toUpperCase() + txt.substr(1).toLowerCase();});
            file = file.substring(0, file.lastIndexOf('/')+1);
            file = temp + ' in ' + "<code class='hljs'>" +file + '</code>';
            html += `<li>${file}</li>`;
        }
      }
      html += "</ul>";
      //put the html in the document
      const element = document.getElementById('latestBlog');
      element.innerHTML = html;
  })
  .catch(error => console.error(error));

And then I put it in the book.toml(the default "setting" file for mdbook) in my book folder like this:

[output.html]
additional-js = ["my-javascript-file.js"]

What was the challange

Mdbook is static website generator, so its missing a backend. There are "custom backend" that you can do, but they are mainly for rendering( generating the book in a different format other than html) and is not an actual service.

The loops I have to go through

script during compile

I explored seveal script like this

import subprocess

def get_git_diff(num):
    git_diff_cmd = f'git diff HEAD~{num} ./src/SUMMARY.md'
    grep_cmd = "grep -E '^\+'"
    cmd = f'{git_diff_cmd} | {grep_cmd}'
    result = subprocess.run(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
    return result.stdout.splitlines()

i = 1
latest_content = None

# Keep incrementing the number until we find a diff with more than 5 lines
while True:
    content = get_git_diff(i)
    if len(content) > 5:
        latest_content = content
        break
    i += 1

# Print the latest content
for line in latest_content:
    if (line.startswith('@@') and line.endswith('@@')):
        continue
    if ("README.md" in line):
        continue
    
    stripped_line = line.strip('+').strip()
    print(line)

which aim to monitor the changes in SUMMARY.md(the table of content file) and do string manipulation afterwards. This seemed like a good idea at the time until i realized i could just monitor the file addition instead.

The true custom backend

Then i thought, hmm, why not just make a backend for my satic "frontend", and create a backend i did.

I used this template generator and creates a simple backend. I specified an api call that gave me the necessary strings, and I than uploaded it to cyclic.sh.

I then went on my welcome page and embedded this code:

    fetch("<my-custom-backend-app>/api/v1/latest")
    .then(response => response.json())
    .then(data => {
        // Get a reference to the HTML element where we want to display the data
        const element = document.getElementById('latestBlog');
        // Update the HTML content with the fetched data
        let html = "</ul>";
        for (let i = 0; i < data.length; i++) {
            html += "<li>" + data[i] + "</li>";
        }
        html += "</ul>";

        element.innerHTML = html;
    })
    .catch(error => console.error(error));

This sounds good on paper, so lets test it out.

sound effect, this was a major fail.

The reason turns out to be quite simple, not only did i have to do the do the api call from the backend app, i then have to fetch again from my custom backend, so the process is like this:

antzed.com <-- my custom backend app <-- github api.

I even tried to simplied fetch through fetching the minimum commit from the github api, meaning changing https.../commits?per_page=<big number> to https.../commits?per_page=2, and do direct index calles in my embeds like this,

let html = `<ul>
    <li>${data[0]}</li>
    <li>${data[1]}</li>
    <li>${data[2]}</li>
    <li>${data[3]}</li>
    <li>${data[4]}</li>
</ul>`;

but that doesn't really help.

So in the end, I cut off the middleman that is my custom backend.

What's next

Even though the current solution works, I still believe I can achieve a faster loading time. So I'm going to keep working on this.

Am I going to keep the custom backend?

Probably. Using it to do this simple fetches is definitly not going to happen. But it definitly can be used for some other complex operation.