2021 October 3
Dug into the Let's Encrypt failure and by examining the logs with:
cd ~/discourse/image
./launcher logs app | grep -i letsencrypt
and was directed to:
/var/discourse/shared/standalone/letsencrypt/acme.sh.log
which reported:
scanalyst.fourmilab.ch:Verify error:CAA record for
scanalyst.fourmilab.ch prevents issuance
This was because we have a record in the fourmilab.ch DNS:
fourmilab.ch CAA 0 issue "thawte.com"
which restricted certificate issue to Thawte. Since CAA records are
processed in a hierarchical manner, I added:
scanalyst.fourmilab.ch CAA 0 issue "letsencrypt.org"
to authorise issuance by Let's Encrypt for the subdomain.
Re-ran:
./discourse-setup
to re-try Let's Encrypt and now it works. We're still getting some
mixed content warnings about logos, etc., but it basically works.
The host secret keys and certificates are in:
/var/discourse/shared/standalone/ssl
For details and debugging tips, see:
https://meta.discourse.org/t/setting-up-https-support-with-lets-encrypt/40709
Qualys SSL Labs scores the SSL implementation as A+.
https://www.ssllabs.com/ssltest/analyze.html?d=scanalyst.fourmilab.ch&hideResults=on
Chromium does not issue the mixed content watnings that Firefox does.
Firefox seems to be complaining about access to the favicon files.
Installed:
yum install xauth
in the hope this will allow X11 forwarding on SSH logins. After
logging out and back in, it re-created ~/.Xauthority and now X11
tunnelling works.
Installed:
yum install openmotif-devel
and now the binary version of nedit built on AWS works.
Installed:
yum install gtk2-devel
yum install intltool
yum install gcc
yum install gcc-c++
to allow building Geany.
This permitted building and installing Geany, which I downloaded
from:
https://www.geany.org/download/releases/
into ~/linuxtools/geany-1.36 and built with:
./configure
make
super
make install
It now works.
Under Settings/User Preferences, set:
Open external links in a new tab by default.
This sets the defaults for new users.
Created regular user account for testing:
E-mail: REDACTED
User name: Kelvin
Full name: Kelvin Throop
Password: REDACTED
You can now update the reverse DNS for an Elastic IP address without
applying to a human. Go to the Elastic IP address page, select the
address, and choose "Update reverse DNS". Enter the domain name and
you're done.
Updated the SPF record for the server to:
scanalyst.fourmilab.ch. TXT "v=spf1 include:amazonses.com ip4:193.8.230.0/24 ip4:18.195.73.61 ~all"
to reflect that Amazon SES that is our mail transfer agent.
To test E-mail configuration, go to:
https://www.mail-tester.com/
and send E-mail to the custom address it gives you. If the mail
arrives, it will be diagnosed for SPF, etc. configuration.
You can test your reverse DNS with:
https://mxtoolbox.com/ReverseLookup.aspx
According to this test, it reports:
scanalyst.fourmilab.ch
which is correct.
After all of this, I could send E-mail to my own address, but E-mail
to other addresses disappeared without a trace. After checking the
logs, etc., I finally stumbled upon Admin/Emails/Skipped and found
the E-mails listed, all with the reason:
554 Message rejected: Email address is not verified. The following
identities failed the check in region US-EAST-1:
REDACTED
After researching this, I found:
https://aws.amazon.com/premiumsupport/knowledge-center/ses-554-400-message-rejected-error/
where we learn:
"Check whether your account is in the Amazon SES sandbox for the
AWS Region that you're using to send emails. If your account is in
the Amazon SES sandbox, then you must verify the recipient email
address, in addition to verifying your sender identity. Or, you can
request to move your account out of the Amazon SES sandbox."
So, while you're in the sandbox, you can only sent to addresses you've
verified, so I was able to send to my own (verified), but not any other.
I verified REDACTED, and then I was able to send a test message
to it. I submitted a request to escape the sandbox.
Production access request
Service: SES Sending Limits
Region: us-east-1
Please enable production access
------------
Use case description: This is a Discourse-based discussion site
intended to provide interactive discussion of the content on my
main https://www.fourmilab.ch/ site, which has been on the Web
since 1994, hosted at AWS since January 2016, and has an Alexa
global site rank of 201,268 (119.254 in the U.S.). E-mail will
be used exclusively for communications with users who sign up
to participate in the site and restricted to verification of
E-mail addresses for user registration, password reset, and
opt-in notifications of activity on a user's account (new
comments on users' posts, etc.). The volume of mail should be
low, only a few per day if a user opts in to everything
Discourse permits. No mail other than that generated by the
Discourse system will be sent, and no mail to those who have
not created accounts on the server.
Mail Type: TRANSACTIONAL
Website URL: https://scanalyst.fourmilab.ch/
Within one minute of submitting this request, sent late on a Sunday
night Western European time, and Sunday afternoon at Amazon's
headquarters, I received a reply which began:
This is precisely what my original message specified. So, I suppose I
have to appease the fickle deity of AWS by jumping like a trained seal
when they clap their hands. So I composed and sent the following.
I believe I provided most of the information requested in your
reply in my original communication, which is quoted in the
conversation thread below. Here are answers to the specific
information requested in the third paragraph of your reply.
>> For example, tell us how often you send email,
All E-mail is sent by the Discourse discussion site software
(https://www.discourse.org/) and only to users who have registered
to create accounts on the site. Under no circumstances is E-mail
sent to any person who has not first visited the site and requested
an account, providing their E-mail address to confirm their address
and complete registration. This is a standard procedure with
Discourse. We run an unmodified version of the standard Discourse
software (version 2.8.0), using its standard E-mail policies.
After a user has replied to the account registration account, no
further E-mail is sent unless the user opts in to request E-mail
notifications of activity on the site, such as comments on posts
they have made or private messages from other users, or the user
requests an operation such as a password reset. In my experience
with Discourse sites, I find that few users request E-mail
notifications, so they receive no regular E-mail from the site, and
those who enable all notifications will get, with typical usage,
around one E-mail per day. I would expect the total volume of
E-mail from the site, which I expect to attract around 50 to 100
users in the first year, to be on the order of 25 to 50 E-mails per
week total across all users.
>> how you maintain your recipient lists
Recipient lists are exclusively drawn from users who have
registered for the site and confirmed their registration, and only
for notifications for which the user has opted in. If a user does
not explicitly request notifications, they will receive no E-mail
from the site. Each user can view their E-mail settings and opt
out of any notifications they have enabled at any time. If a user
deletes their account, they will under no circumstances receive any
further E-mail from the site. The user list with E-mail addresses
is private to the site, not visible to anybody other than site
administrators, and will not be shared with or sold to any third
party under any circumstances.
>> and how you manage bounces, complaints, and unsubscribe requests
Bounces are logged when they occur and administrators are notified
of them. Bounces which appear to be consistent (as opposed to a
transient delivery problem) will result in the administrator
suspending E-mail notifications for the user and posting a private
message to the user on the site (not via E-mail) notifying the user
of the problem and requesting them to update their E-mail address
to one which does not bounce. Complaints are very rare in a system
which only sends E-mail for notifications a user has explicitly
requested. I ran another similar site for three years with in
excess of 250 users and do not recall receiving a single complaint
about its E-mails. I would respond to complaints by disabling
E-mail notifications for that user so they received no further
E-mail. Since all E-mail is opt-in notifications, a user can
unsubscribe from any or all E-mail from their own account
preferences page on the site. If for some reason the user cannot
do this, they can send an E-mail or message on the site to the site
administrator who can do this for them.
>> It is also helpful to provide examples of the email you plan to send
By far, the most common E-mails the site will send are registration
confirmation messages when a new user creates an account. Below I
quote verbatim one of these messages, sent when I created my own
administrator account. When the site goes into production, the
From and Reply-to addresses, which in this test mode are my
personal address, will be replaced with a group mail box for
administrators, REDACTED. (I have elided the HTML copy
of the same message embedded in the E-mail sent, as the user will
see only one of the plain text or HTML, depending on their E-mail
client.)
* * *
Date: Sun, 3 Oct 2021 01:27:14 +0000
From: Discourse
Reply-To: Discourse
To:REDACTED
Message-ID: <0100017c43c30cef-cdbb47f1-fd56-4cc4-8266-c4506b29384d-000000@email.amazonses.com>
Subject: [Discourse] Confirm your new account
Mime-Version: 1.0
X-Auto-Response-Suppress: All
Auto-Submitted: auto-generated
Feedback-ID: 1.us-east-1.5morRTaxg4m9BLcFh6GhNFImf4r42yeSFXEyu0V2py4=:AmazonSES
X-SES-Outgoing: 2021.10.03-54.240.8.51
Welcome to Discourse!
Click the following link to confirm and activate your new account:
http://scanalyst.fourmilab.ch/u/activate-account/ac497e68eef4248c61b1690443747d2c
If the above link is not clickable, try copying and pasting it into the address bar of your web browser.
* * *
I hope that the above information is useful in deciding to grant
the request for production status.