Comments with a static website generator are a difficult thing. There are not too many options to implement this functionality.
Basically comments are contradicting the paradigm of a static website. They are not exactly static, aren’t they? So either you integrate a commenting-service into your website or implement an own feedback-cycle.
Cloud-services for comments
… and why they simply are a “fail”, these days
At first it sounds like a brillant idea … you give them money and in return, they manage the comments for your website. All you need to do is include their forms and javascript, job done. Some even offer this for free, provided the traffic is reasonably low.
The problem with these services is that they track everything. And they do not just employ their own tracker: disqus, e.g. had so many 3rd-party-trackers involved that I got lost in count. Needless to say, that in the days of the GDPR (German: DSGVO), this would violate the regulation. Even if you could make it look legal by fully describing the usage by all involved trackers, they do not comply to e.g. deletion requests. Thus they might look legal, but they never really will be.
There are cloud-services that do not excessively involve 3rd-party-trackers. The do ask money, instead. If you earn money with your site, these are surely an option for you.
Myself, I do not directly monetarise my published whatevers. To be honest, I hardly believe that anybody will actually be reading it … ever. So why paying a cloud-service’s monthly plan?
Thus, I had to either:
- forget about static website generation and fall back to a CMS, with all its disadvantages
- live without eventual user’s comments
- find a way to implement comments that keep me out of harm’s (a.k.a. GDPR) way and that fits into the picture of static website generation
I chose to try the third approach.
Presumptions
My solution makes a few presumptions:
- you got mail-accounts with IMAP with your web-hosting (usually true, trivially), at least one
- you work with Linux or alike (you need “imapfilter”)
- you work with a CI/CD, e.g. Jenkins or some cloud-based service
Requirements
- reduce the exposure to GDPR-issues to zero
- reduce SPAM to zero
- keep the effort in processing/reviewing a comment to the minimum possible level
High Level Design
One of the oldest mechanism in HTML is the mailto-link. It opens a mail-client on the user’s machine and prepares it to send an email. While this is pretty much old-school, these days it experiences a rennaisance:
- all happens on the user’s machine
- the user is responsible to manage the security (encryption and signing) of his client and MTA
- email does not have a widely accepted standard for encryption, so … unencrypted is state of the art and conforms the GDPR
So: help the user to send an email with enough template in it to allow automatic processing.
In order to prepare this, a content-page will have a commentid assigned in the frontmatter, if I expect it to receive comments. A partial template called in my basepage will show an explanation and a button to initiate the mail, if a page has a commentid. It also shows the comments, if there are any.
A comment is just a page in the comment-section. It is also marked with a commentid and is of type “comment”.
Up to here, this is how much the static website generator is involved. So how do we get the mails turned into comment-pages?
I found a tool called imapfilter. It can be scripted in LUA (yuck!) and I learned how to make it pump new mails from a mail-account into some shell-script or alike Linux-command.
The logic is pretty obvious from here: I need a LUA-script that downloads new emails, identifies them and if they are valid, pumps them into the shell script. What makes an email valid? The commentid and an API-key can be parsed from the body! This in one blow eliminates almost all SPAM to be expected: spammers will usually not customise their SPAM with an API-key.
The script that is called by the imapfilter is a TCL-script. It will need to invest some effort to properly parse the email and then use the fields in the mail-body to build the frontmatter of the new comment:
- store the sender’s email
- adapt the mail’s date
- store the commentid
- extract the subject as title
- recode the body
- decide whether the body is MD or plain-text
- mark it as draft!
Once this is done, the comment will be stored as page in the comment-section.
A shell-script may find all drafts in the comment-section and pull them into an editor. Thus their format can be fixed, the content masked, where necessary and the draft-status removed. Alternatively the comment may just be turn down and deleted.
There are some goodies with this:
- as the comments are in draft mode, you may generate a list-page especially for the draft-comments, so when starting the hugo-server with draft-mode on, you can (pre-)view the incoming comments.
- you may decide to check-in every comment into GIT, regardless. Being in draft-mode, they do not show up on the website.
- once automated, your email-account is maintenance-free: unidentified mails get deleted, the others get processed
- you may generate a list of comments that are queued for review
- you may delay the deletion of processed emails so you can directly answer some
Workflow
So what does that mean for our workflow?
- Users send mails
- imapfilter in a cronjob pulls the mails and creates draft pages
- a shell script helps to review the draft pages
- all pages that survived the review process will exit the draft-stage and
- these pages will be checked in to git
- which gets them deployed on the website
So all I need to do is, implementing a little monitor-script that informs me when draft-pages appear in my comments-section. Then login and run a shell script to deal with them. Done!
For small amounts of comments, this is a totally acceptable way to handle this. Plus: I am forced to keep track of what happens on my site. I am free of conflict with the GDPR. I am free of trackers. I am free of charges from cloud-services. Still I have a functional comment mechanism.
Implementation
At first we need to be able to collect comments. So we setup a mail-account with IMAP-access. Configure your mail-client so you can fully operate it.
Site configuration
Then configure your mail-address in the config.toml (or whatever format you prefer for your hugo-site-config):
baseURL = "https://road-coder.de/"
languageCode = "en-uk"
title = "An IT-freelancer's toolbox"
theme = "first"
enableGitInfo = true
[params]
AuthorName = "A. Wallaschek"
CustomCSS = [ "css/site.css" ]
CommentsEmail = "comments-xyz@road-coder.de"
CommentsKey = "CMT1337"
Copyright = "Copyright © 2019 by Adrian Wallaschek - all rights reserved"
[author]
name = "Adrian Wallaschek"
email = "my-email-address@example.com"
[privacy]
[privacy.disqus]
disable = true
[privacy.googleAnalytics]
anonymizeIP = true
disable = true
[privacy.instagram]
disable = true
[privacy.twitter]
disable = true
[privacy.vimeo]
disable = true
[privacy.youtube]
disable = true
This is my site config. There is nothing secret to it: the most blocks is standard, your’s will look alike. I just did obfuscate the mail-addresses, though.
The last block, I have taken from some hint on the internet. Depending on the theme, that you use, you might be better off configuring this explicitely.
What is related to the comments is two lines in the params. (And do not ask me, why I have the AuthorName there, I will check and fix this!) These lines are CommentsEmail and CommentsKey. The email is obviously the email-address that receives comments. Be nice to yourself and do not use any existing email-addresses that you currently own and use. You will be flooded with SPAM! The CommentsKey is any arbitrary Random String that you use to be able to identify comments from SPAM. You may change this from time to time (in the config and in the receiving LUA script) to keep the security level a bit higher.
The comments-partial
My actual theme is of no further interest, here. Just note that I have mine based on bootstrap, your mileage may vary.
{{/*
This partial will display a comments section, if
- the commentid is set in the frontmatter
- the page itself is not a comment
*/}}
{{ if and (isset .Params "commentid") (not (eq .Type "comment")) }}
The following block contains a description of the wait to send a comment. The page /legal/comments actually contains a full-prosa description of the process and its aspects regarding privacy.
The mailto-link is the key-element: it uses some of the mailto-syntax options to prepare the email. Note that the mail-address and the CommentsKey from the global site configuration are being used here.
<div class="alert alert-warning">
<p>You are invited to send me a comment about this page. Please read
the <a href="/legal/comments/" class="alert-link">introduction about sending comments</a> as this informs you about
how to send a comment <strong>and what this comment-email means to your privacy</strong>.
</p>
<a href="mailto:{{ .Site.Params.CommentsEmail }}?subject=Comment about {{ .Permalink }}&body=**({{ .Params.commentid }}/{{ .Site.Params.CommentsKey }}) Do not modify neither the subject-line nor this line! **%0D%0A%0D%0AYour nickname (will be published with your comment): ...%0D%0AComment:%0D%0A...write your comment here ...%0D%0A%0D%0A " class="btn btn-primary">Send a comment by mail ...</a>
</div>
This loop will simply check if there are existing comments to be displayed. Honestly, I still try to find a better way to express this, but the syntax of the Go-template system is a piece of crap. I hate it.
{{ $.Scratch.Set "CommentsFound" false }}
{{ range where .Site.Pages "Type" "comment" }}
{{ if eq .Params.commentid ($.Param "commentid") }}
{{ $.Scratch.Set "CommentsFound" true }}
{{ end }}
{{ end }}
If comments have been found, a message will introduce the listing:
{{ if $.Scratch.Get "CommentsFound" }}
<p>Here some comments that I received about this page:</p>
{{ end }}
Then here they are: all the comments. Let us not discuss, wether or not to put this into the if-statement, above and so on. It works, but if you find a better way … feel free to send me a comment!
{{ range where .Site.Pages.ByDate.Reverse "Type" "comment" }}
{{ if eq .Params.commentid ($.Param "commentid") }}
<blockquote class="blockquote rounded">
<a href="{{ .Permalink }}"><h5>{{ .Title }}</h5></a>
<p>{{ .Summary }}</p>
<footer class="blockquote-footer">
<div class="d-inline tipped" data-toogle="tooltop" data-placement="top" title="{{.Date.Format "2006-01-02 15:04:05 MST"}}">
{{ index .Params "author" | default "anon" }} on {{.Date.Format "2006-01-02"}}
</div>
</footer>
</blockquote>
{{ end }}
{{ end }}
Here we just repeat the link from the beginning, i.e. if comments got displayed.
{{ if $.Scratch.Get "CommentsFound" }}
<a href="mailto:{{ .Site.Params.CommentsEmail }}?subject=Comment about {{ .Permalink }}&body=**({{ .Params.commentid }}/{{ .Site.Params.CommentsKey }}) Do not modify neither the subject-line nor this line! **%0D%0A%0D%0AYour nickname (will be published with your comment): ...%0D%0AComment:%0D%0A...write your comment here ...%0D%0A%0D%0A " class="btn btn-primary">Send a comment by mail ...</a>
{{ end }}
{{ end }}
I know all of this could be done … leaner … more efficient and more maintainable. This is a proof-of-concept and it works. I will care for amendments, later. You are invited to support me with comment!
Now this partial, let us called comments.html for the lack of a better name, will be called in my base
The master template: baseof.html
This will likely not need a big explanation. I just cut a few irrelevant lines.
<!DOCTYPE html>
<html lang="en">
{{- partial "head.html" . -}}
<body>
<div class="container-fluid">
{{- partial "navbar-top.html" . -}}
{{- partial "header.html" . -}}
<div class="row justify-content-center">
<div class="col-md-10 col-lg-8 col-xl-6">
{{- block "main" . }}{{- end }}
</div>
</div>
<div class="row justify-content-center">
<div class="col-md-10 col-lg-8 col-xl-6">
<!-- This is, where the music plays
... between content and footer -->
{{- partial "comments.html" . -}}
</div>
</div>
<!-- some lines cut away, here -->
{{- partial "footer.html" . -}}
</div>
<script src="/js/jquery-3.3.1.min.js"></script>
<script src="/js/popper.min.js"></script>
<script src="/js/bootstrap.min.js"></script>
<!-- some lines cut away, here -->
</body>
</html>
Remember: despite of the comments-partial included here, it will only generate output if the commentid is set for this page.
The LUA-script
The tool imapfilter can be installed from a package on most Linux distribution.
You will then have a directory ~/.imapfilter and create a file config.lua in it:
mbox = IMAP {
server = 'imap.your-email-hoster.site'
username = 'your-accounts-username',
password = 'your-accounts-password',
ssl = 'ssl3'
}
-- messages = mbox["INBOX"]:contain_body("bla")
-- messages:move_messages(mbox["Archiv"]);
-- Iterate all new messages
-- Pipe those into a command
--
--
allnew = mbox["INBOX"]:is_unseen()
results = Set {}
for _, mesg in ipairs(allnew) do
mbox, uid = table.unpack(mesg)
text = mbox[uid]:fetch_message()
if (pipe_to('./blubb', text) == 1) then
table.insert(results, mesg)
end
end
-- results:delete_messages()
This kind of demonstrates the structure: the first block (IMAP) creates an mbox-object to access the imap-account with it. The allnew-variable will receive a list of “unseen” messages from the INBOX. “unseen” in this context means what most mail-clients call “unread”. At least I believe that ;-).
This list will be iterated through and each message will be fetched and piped into the script that is called for now ./blubb (it is clearly in a proof-of-concept-stage ;-)).
Depending on the scripts results, actions may be taken: move it to spam, delete it, mark it as unseen, whatever. This part is obviously not implemented, yet.
The TCL-script
This is work in progress. Be warned. This script will have to fully parse the email: headers and mime-blocks.
I chose TCL, because it is powerful, I know it, I had to fresh up my TCL skills and it has a mime-library to parse the mime blocks for me.