commit 5cc902bd9ce452cacd0fbb3fd25778624885a2db
from: Alexander Arkhipov <aa@alearx.org>
date: Fri Nov 10 02:25:59 2023 UTC

add article 011.building_this_site.txt

commit - 13478d29a8dfeb5531cb75493d89bb56a68fd21f
commit + 5cc902bd9ce452cacd0fbb3fd25778624885a2db
blob - /dev/null
blob + 6d03ce854405604d3558f1a5e3039f5d665007ec (mode 644)
--- /dev/null
+++ art/011.building_this_site.txt
@@ -0,0 +1,77 @@
+You might have noticed that this web/gopher/gemini site is a bit
+"weird". Part of the reason is that it's not just a web site, it's a
+web/gopher/gemini site. Another reason is that while many people seem
+to think about things like "how to best convert markdown to html" and
+"can I add tags?", I usually think about things like "how do I just
+upload this nonsense already?" and "how do I maintain a history of
+changes without it taking a lot of effort?". Finally, it's also
+because I just want to serve a few files: the server daemon running
+CGI scripts is an anti-goal.
+
+The idea of how to satisfy all these needs correctly did not come to
+me immediately: over a couple of years I maintained a variety of
+synchronisation scripts of various complexity. A common feature among
+them was that each time I ended up thinking "can't I just get rid of
+$THIS, or $THAT?". The current (2023-11-09) system is also done via a
+few scripts, but it's much more simple and harmonious than it used
+to be.
+
+
+THE SYSTEM
+
+Firstly, I have several remote directories that I am interested in
+updating from time to time, such as /var/ftp/pub, /var/gopher &c.
+These are all owned by $USER:$(id -gn).
+
+I maintain a bunch of mostly-binary files on a local disc in a
+directory $d. This gets synced with remote:/var/ftp/pub/ manually via
+a script that just says:
+
+rsync -a --del "$@" $d/ remote:/var/ftp/pub/
+
+UTF-8 files, however, are mostly stored inside a git repository [0].
+All the *.html *.gph and *.gmi files there are maintained by hand.
+From this point of view the site update looks so:
+
+	$ got ci -m'message' # = git add . && git commit -m'message'
+	$ got se             # = git push
+
+Once I do so, however, I may have to wait for up to half an hour for
+the updates to appear (same goes for automatically updating stagit).
+That's because the update is actually performed by this cronjob:
+
+	*/30 * * * * $HOME/libexec/update
+
+The update script does several things, but they are simple
+individually:
+
+1. /var/ftp/pub/ is rsync(1)ed to appropriate gopher, gemini and www
+   locations with --link-dest=/var/ftp/pub/.
+2. The working tree of the gopher git repository is updated.
+3. The gopher, gemini and www directories are rsync(1)ed from the
+   working directory with --exclude-from=common.exclude and
+   --exclude-from={gopher,www,gemini}.exclude.
+4. ~/.{plan,project} are just symlinks to
+   $working_tree/{plan,project}.txt.
+5. stagit is also updated.
+
+5., though, is its own algorithm:
+
+1. I determine which repositories are public with some grep(1)ping
+   and sed(1)ing of /etc/gotd.conf.
+2. For each repository:
+2.1. I find the newest timestamp like so:
+
+	find $repository -exec stat -f%m {} + | sort -r | head -1
+
+2.2. If the timestamp is newer than the one stored in
+     $repository.mtime, or such file doesn't exist yet, I run stagit(1)
+     and stagit-gopher(1), then store the new mtime.
+2.3  Regardless of 2.2., I also check to see if the index files
+     contain a link to that repository; if not I flip the
+     appropriate "boolean" variable.
+5. I check for the "boolean" variables from 2.3, and run
+   stagit-index(1) and stagit-gopher-index(1) as appropriate.
+
+
+[0] ssh://anon@alearx.org/gopher