Inner workings

2023年3月20日月曜日 14:36 by [ - Junko - ]
Tags [ - Tech - Meta - ]

<Junko> edit for 2023年7月22日: We are no longer using org mode or github to publish our content. This post is outdated.

Introduction

<Junko> honestly, we didn't have enough time yesterday to fully talk about the blog, the last update was way past midnight.

as you can probably see, we spent the morning working on the css for the page, the gruvbox theme is one of the most pleasant things to look at.

<Anuma> I decided the theme myself, something that felt good to look at, plus it went along with our system theme. If had been up to her, she would have made the site look like crus.cc

most of the trouble setting this up came at the time of exporting source blocks into html with the color highlighting, some languages like gdscript still are not supported for reasons still unknown to me.

coming up, we’ll add rss support, not that anyone will use it, but it’ll be a fun experiment since i’ve never tried anything like that.

now, let me explain how the inner workings of this site works more in depth. the following block represents the hierarchy of the project:

├── build-site.el
├── build.sh
├── content
│   ├── Alicia
│   │   └── letter.org
│   ├── Anuma
│   ├── Junko
│   │   └── inner-workings.org
│   ├── about.org
│   ├── css
│   │   └── styles.css
│   ├── html
│   │   ├── html_postamble.html
│   │   └── html_preamble.html
│   ├── images
│   │   ├── blog.png
│   │   └── initial.png
│   ├── index.org
│   ├── pfp
│   │   ├── Alicia.png
│   │   ├── Anuma.png
│   │   └── Junko.png
│   └── sitemap.org
└── docs

the workflow of this system is:

write the articles in org, inside the folder of whoever is the main author.
run build.sh which opens emacs and runs the build-site.el script, generating all the html files that are the actual webpage in the /docs folder, which is where the site is built from.
commit all the changes to the github repo and will be submitted to github pages automatically.

Walkthrough

now we’ll go step by step through every folder and relevant file of this project, explaining a the code and how it works.

Content

Member folders

these are the Alicia, Anuma and Junko folders, which is where the posts of the blog are stored, as classified by main author.

Images, pfp and CSS

also very straightforward folders, store the site images, member profile pictures, and the sylesheet for the site.

the reason i have the pfp folder outside of the images folder is mostly just convenience.

<Anuma> She gets very lazy with these things, she'd do anything to save herself from typing one (x1) more word in a path. Having the profile pictures inside the image folder would be way cleaner.

the base css file is grabbed from here as per the systemcrafters’ post. the only thing of note i added besides changing the color theme into gruvbox is the following snippet:

pre span.org-builtin                     {color:#006FE0;font-weight:bold;}
pre span.org-string                      {color:#b8bb26;}
pre span.org-keyword                     {color:#fb4934;}
pre span.org-variable-name               {color:#83a598;}
pre span.org-function-name               {color:#b8bb26;}
pre span.org-type                        {color:var(--accent);}
pre span.org-preprocessor                {color:#808080;font-weight:bold;}
pre span.org-constant                    {color:#D0372D;}
pre span.org-comment-delimiter           {color:#8D8D84;}
pre span.org-comment                     {color:#8D8D84;font-style:italic}
pre span.org-outshine-level-1            {color:#8D8D84;font-style:italic}
pre span.org-outshine-level-2            {color:#8D8D84;font-style:italic}
pre span.org-outshine-level-3            {color:#8D8D84;font-style:italic}
pre span.org-outshine-level-4            {color:#8D8D84;font-style:italic}
pre span.org-outshine-level-5            {color:#8D8D84;font-style:italic}
pre span.org-outshine-level-6            {color:#8D8D84;font-style:italic}
pre span.org-outshine-level-7            {color:#8D8D84;font-style:italic}
pre span.org-outshine-level-8            {color:#8D8D84;font-style:italic}
pre span.org-outshine-level-9            {color:#8D8D84;font-style:italic}
pre span.org-rainbow-delimiters-depth-1  {color:#707183;}
pre span.org-rainbow-delimiters-depth-2  {color:#7388d6;}
pre span.org-rainbow-delimiters-depth-3  {color:#909183;}
pre span.org-rainbow-delimiters-depth-4  {color:#709870;}
pre span.org-rainbow-delimiters-depth-5  {color:#907373;}
pre span.org-rainbow-delimiters-depth-6  {color:#6276ba;}
pre span.org-rainbow-delimiters-depth-7  {color:#858580;}
pre span.org-rainbow-delimiters-depth-8  {color:#80a880;}
pre span.org-rainbow-delimiters-depth-9  {color:#887070;}
pre span.org-sh-quoted-exec              {color:#FF1493;}

this will allow us to specify the colors for the code blocks later.

Html

here we store the actual hard coded html for the project, at the current moment it’s only the preamble and the postamble.

the preamble is added to every single page on the site, while the postamble is just added on blog entries. we’ll see how to add that to the bulder script later.

Preamble

the following snippet of code adds the title of the site, along with table that contains the buttons above. this is bound to have some more stuff added in the future as we implement rss and other functions.

these links are dead when testing the website locally, only work on github pages.

<div class='header'>
  <h1>Lavender field</h1>
  <table style="border-collapse: collapse">
    <tr>
      <th><a href='/index.html'>Home</a></th>
      <th><a href='/about.html'>About us</a></th>
      <th><a href='https://mastodon.online/@Arisu' target='_blank'>Mastodon</a></th>
    </tr>
  </table>
</div>

Postamble

a very simple piece of html, it’s just the “go back” link at the end of the posts.

<a href='/index.html'>go back</a>
<br>
<br>
<br>

Index.org

the index file of this site is probably a joke but it works. the only thing it does is add a title for the Posts and import the sitemap.org, which is a file that indexes the posts of the site. is generated automatically by the build script.

#+title: Index
#+options: title:nil

*Posts
    #+INCLUDE: sitemap.org

About.org

this is the about page, where we talk a bit about ourselves. it’s just an org file like any other post.

Builder script

here is the code that builds the webpage into the docs folder.

Build.sh

the bash script just launches emacs with the -Q argument to make it run without any interface, just evals the build-site script. this is what is launched to build the site.

#!/usr/bin/env sh

emacs -Q --script build-site.el

Build-site.el

lastly this is the emacs lisp script that actually does all the heavy lifting. i’ll go through it bit by bit.

the following snippet of code makes the script import ox-publish, which is a built in emacs functionality that will let us translate org documents into html.

it will also download htmlize from melpa, and place it inside the project root, so it can work in any system.

(require 'ox-publish)

;; Set the package installation directory so that packages aren't stored in the
;; ~/.emacs.d/elpa path.
(require 'package)
(setq package-user-dir (expand-file-name "./.packages"))
(setq package-archives '(("melpa" . "https://melpa.org/packages/")
                         ("elpa" . "https://elpa.gnu.org/packages/")))

;; Initialize the package system
(package-initialize)
(unless package-archive-contents
  (package-refresh-contents))

;; Install dependencies
(package-install 'htmlize)

following that, i declare a function that is used to format the entry name in the sitemap, where it adds the date of the entry to the display name in the index. the function is grabbed from this blog post that explains how to make an org-mode website much better than i ever could.

(defun junko/sitemap-format-entry (entry style project)
  (cond ((not (directory-name-p entry))
         (format "[[file:%s][[%s] %s]]"
                 entry
                 (format-time-string "%Y-%m-%d"
                                     (org-publish-find-date entry project))
                 (org-publish-find-title entry project)))
        ((eq style 'tree)
         (file-name-nondirectory (directory-file-name entry)))
        (t entry)))

next up, it’s probably the most important part of this file, this is where we declare what parts of the project are built, where to and with what properties. according to what type of content and how i want to handle it, it’s divided in multiple categories.

first one of them, is all the actual posts on the site. this part doesn’t add title, author or date automatically, because of the way ox-publish handles the autogenerated conent, so it’s concatenated manually in the preamble.

in this part too, we create the sitemap, so it only indexes the files that are affected by this section.

(setq org-publish-project-alist
      (list
       (list "org-site:main"
             :recursive t
             :base-extension "org"
             :base-directory "./content"
             :publishing-function 'org-html-publish-to-html
             :publishing-directory "./docs"
             :auto-sitemap t
             :sitemap-filename "sitemap.org"
             :sitemap-title ""
             :sitemap-format-entry 'junko/sitemap-format-entry
             :sitemap-sort-files 'anti-chronologically
             :with-author nil       ;; Don't include author name
             :with-creator nil            ;; Include Emacs and Org versions in footer
             ;;:with-toc nil                ;; Include a table of contents
             :with-date nil
             :with-title nil
             :section-numbers nil       ;; Don't include section numbers
             :time-stamp-file nil
             :exclude "index.org\\|about.org"
             :html-postamble (with-temp-buffer (insert-file-contents "./content/html/html_postamble.html")(buffer-string))
             :html-preamble (concat (with-temp-buffer (insert-file-contents "./content/html/html_preamble.html")(buffer-string))
                                    "<h1 class=\"title\">%t</h1>
                                    <p class=\"subtitle\">%d BY %a</p>"))

the second and third parts, export the index and about pages respectively. we do them separately for them to not be indexed in the sitemap, not have the postamble, and to manage properties like the table of content (toc).

       (list "org-site:index"
             :base-directory "./content"
             :base-extension nil
             :exclude ".*"
             :include ["index.org"]
             :publishing-function 'org-html-publish-to-html
             :publishing-directory "./docs"
             :auto-sitemap nil
             :section-numbers nil
             :with-toc nil
             :html-preamble (with-temp-buffer (insert-file-contents "./content/html/html_preamble.html")(buffer-string)))
       (list "org-site:about"
             :base-directory "./content"
             :base-extension nil
             :exclude ".*"
             :include ["about.org"]
             :publishing-function 'org-html-publish-to-html
             :publishing-directory "./docs"
             :auto-sitemap nil
             :with-toc t
             :section-numbers nil
             :html-preamble (with-temp-buffer (insert-file-contents "./content/html/html_preamble.html")(buffer-string)))

finally, we export the images and css separately, notice how the publishing function is not trying to translate them to html but just publishing them as attachments.

       (list "css"
             :recursive t
             :base-extension "css"
             :base-directory "./content"
             :publishing-directory "./docs"
             :publishing-function `org-publish-attachment)
       (list "images"
             :recursive t
             :base-extension "jpg\\|png\\|gif"
             :base-directory "./content"
             :publishing-directory "./docs"
             :publishing-function 'org-publish-attachment)))

and in the last part of the file we just set some other variables to make the site, mostly related to styling and we set the site to publish.

(setq org-html-validation-link nil)
(setq org-html-htmlize-output-type 'css)
(setq org-html-htmlize-font-prefix "org-")

;; Customize the HTML output
(setq org-html-validation-link nil            ;; Don't show validation link
      org-html-head-include-scripts nil       ;; Use our own scripts
      org-html-head-include-default-style nil ;; Use our own styles
      ;;org-html-head "<link rel=\"stylesheet\" href=\"/css/styles.css\" />") ;; CSS for Github
      org-html-head "<link rel=\"stylesheet\" href=\"/home/alicia/blog/content/css/styles.css\" />") ;; CSS for local testing
;; Generate the site output
(org-publish-all t)

(message "Build complete")

and that’s it! that’s how our site works currently!

<Anuma> You sure wasted a lot of time and didn't explain much. The people you sighted have much better explained content.

<Junko> the point is to show how our site works, not a tutorial on how to make it, i just gave some very basic guidelines and pointed towards sources that can help you take this basic idea further.

anyways, hope the content was of any use, the next post will happen whenever any of us feels like writting and it’ll be about pretty much whatever.

Sources

Go back