2021 October 3
Dug into the Let's Encrypt failure and by examining the logs with: cd ~/discourse/image ./launcher logs app | grep -i letsencrypt and was directed to: /var/discourse/shared/standalone/letsencrypt/acme.sh.log which reported: scanalyst.fourmilab.ch:Verify error:CAA record for scanalyst.fourmilab.ch prevents issuance This was because we have a record in the fourmilab.ch DNS: fourmilab.ch CAA 0 issue "thawte.com" which restricted certificate issue to Thawte. Since CAA records are processed in a hierarchical manner, I added: scanalyst.fourmilab.ch CAA 0 issue "letsencrypt.org" to authorise issuance by Let's Encrypt for the subdomain. Re-ran: ./discourse-setup to re-try Let's Encrypt and now it works. We're still getting some mixed content warnings about logos, etc., but it basically works. The host secret keys and certificates are in: /var/discourse/shared/standalone/ssl For details and debugging tips, see: https://meta.discourse.org/t/setting-up-https-support-with-lets-encrypt/40709 Qualys SSL Labs scores the SSL implementation as A+. https://www.ssllabs.com/ssltest/analyze.html?d=scanalyst.fourmilab.ch&hideResults=on Chromium does not issue the mixed content watnings that Firefox does. Firefox seems to be complaining about access to the favicon files. Installed: yum install xauth in the hope this will allow X11 forwarding on SSH logins. After logging out and back in, it re-created ~/.Xauthority and now X11 tunnelling works. Installed: yum install openmotif-devel and now the binary version of nedit built on AWS works. Installed: yum install gtk2-devel yum install intltool yum install gcc yum install gcc-c++ to allow building Geany. This permitted building and installing Geany, which I downloaded from: https://www.geany.org/download/releases/ into ~/linuxtools/geany-1.36 and built with: ./configure make super make install It now works. Under Settings/User Preferences, set: Open external links in a new tab by default. This sets the defaults for new users. Created regular user account for testing: E-mail: REDACTED User name: Kelvin Full name: Kelvin Throop Password: REDACTED You can now update the reverse DNS for an Elastic IP address without applying to a human. Go to the Elastic IP address page, select the address, and choose "Update reverse DNS". Enter the domain name and you're done. Updated the SPF record for the server to: scanalyst.fourmilab.ch. TXT "v=spf1 include:amazonses.com ip4:193.8.230.0/24 ip4:18.195.73.61 ~all" to reflect that Amazon SES that is our mail transfer agent. To test E-mail configuration, go to: https://www.mail-tester.com/ and send E-mail to the custom address it gives you. If the mail arrives, it will be diagnosed for SPF, etc. configuration. You can test your reverse DNS with: https://mxtoolbox.com/ReverseLookup.aspx According to this test, it reports: scanalyst.fourmilab.ch which is correct. After all of this, I could send E-mail to my own address, but E-mail to other addresses disappeared without a trace. After checking the logs, etc., I finally stumbled upon Admin/Emails/Skipped and found the E-mails listed, all with the reason: 554 Message rejected: Email address is not verified. The following identities failed the check in region US-EAST-1: REDACTED After researching this, I found: https://aws.amazon.com/premiumsupport/knowledge-center/ses-554-400-message-rejected-error/ where we learn: "Check whether your account is in the Amazon SES sandbox for the AWS Region that you're using to send emails. If your account is in the Amazon SES sandbox, then you must verify the recipient email address, in addition to verifying your sender identity. Or, you can request to move your account out of the Amazon SES sandbox." So, while you're in the sandbox, you can only sent to addresses you've verified, so I was able to send to my own (verified), but not any other. I verified REDACTED, and then I was able to send a test message to it. I submitted a request to escape the sandbox. Production access request Service: SES Sending Limits Region: us-east-1 Please enable production access ------------ Use case description: This is a Discourse-based discussion site intended to provide interactive discussion of the content on my main https://www.fourmilab.ch/ site, which has been on the Web since 1994, hosted at AWS since January 2016, and has an Alexa global site rank of 201,268 (119.254 in the U.S.). E-mail will be used exclusively for communications with users who sign up to participate in the site and restricted to verification of E-mail addresses for user registration, password reset, and opt-in notifications of activity on a user's account (new comments on users' posts, etc.). The volume of mail should be low, only a few per day if a user opts in to everything Discourse permits. No mail other than that generated by the Discourse system will be sent, and no mail to those who have not created accounts on the server. Mail Type: TRANSACTIONAL Website URL: https://scanalyst.fourmilab.ch/ Within one minute of submitting this request, sent late on a Sunday night Western European time, and Sunday afternoon at Amazon's headquarters, I received a reply which began: This is precisely what my original message specified. So, I suppose I have to appease the fickle deity of AWS by jumping like a trained seal when they clap their hands. So I composed and sent the following. I believe I provided most of the information requested in your reply in my original communication, which is quoted in the conversation thread below. Here are answers to the specific information requested in the third paragraph of your reply. >> For example, tell us how often you send email, All E-mail is sent by the Discourse discussion site software (https://www.discourse.org/) and only to users who have registered to create accounts on the site. Under no circumstances is E-mail sent to any person who has not first visited the site and requested an account, providing their E-mail address to confirm their address and complete registration. This is a standard procedure with Discourse. We run an unmodified version of the standard Discourse software (version 2.8.0), using its standard E-mail policies. After a user has replied to the account registration account, no further E-mail is sent unless the user opts in to request E-mail notifications of activity on the site, such as comments on posts they have made or private messages from other users, or the user requests an operation such as a password reset. In my experience with Discourse sites, I find that few users request E-mail notifications, so they receive no regular E-mail from the site, and those who enable all notifications will get, with typical usage, around one E-mail per day. I would expect the total volume of E-mail from the site, which I expect to attract around 50 to 100 users in the first year, to be on the order of 25 to 50 E-mails per week total across all users. >> how you maintain your recipient lists Recipient lists are exclusively drawn from users who have registered for the site and confirmed their registration, and only for notifications for which the user has opted in. If a user does not explicitly request notifications, they will receive no E-mail from the site. Each user can view their E-mail settings and opt out of any notifications they have enabled at any time. If a user deletes their account, they will under no circumstances receive any further E-mail from the site. The user list with E-mail addresses is private to the site, not visible to anybody other than site administrators, and will not be shared with or sold to any third party under any circumstances. >> and how you manage bounces, complaints, and unsubscribe requests Bounces are logged when they occur and administrators are notified of them. Bounces which appear to be consistent (as opposed to a transient delivery problem) will result in the administrator suspending E-mail notifications for the user and posting a private message to the user on the site (not via E-mail) notifying the user of the problem and requesting them to update their E-mail address to one which does not bounce. Complaints are very rare in a system which only sends E-mail for notifications a user has explicitly requested. I ran another similar site for three years with in excess of 250 users and do not recall receiving a single complaint about its E-mails. I would respond to complaints by disabling E-mail notifications for that user so they received no further E-mail. Since all E-mail is opt-in notifications, a user can unsubscribe from any or all E-mail from their own account preferences page on the site. If for some reason the user cannot do this, they can send an E-mail or message on the site to the site administrator who can do this for them. >> It is also helpful to provide examples of the email you plan to send By far, the most common E-mails the site will send are registration confirmation messages when a new user creates an account. Below I quote verbatim one of these messages, sent when I created my own administrator account. When the site goes into production, the From and Reply-to addresses, which in this test mode are my personal address, will be replaced with a group mail box for administrators, REDACTED. (I have elided the HTML copy of the same message embedded in the E-mail sent, as the user will see only one of the plain text or HTML, depending on their E-mail client.) * * * Date: Sun, 3 Oct 2021 01:27:14 +0000 From: Discourse Reply-To: Discourse To:REDACTED Message-ID: <0100017c43c30cef-cdbb47f1-fd56-4cc4-8266-c4506b29384d-000000@email.amazonses.com> Subject: [Discourse] Confirm your new account Mime-Version: 1.0 X-Auto-Response-Suppress: All Auto-Submitted: auto-generated Feedback-ID: 1.us-east-1.5morRTaxg4m9BLcFh6GhNFImf4r42yeSFXEyu0V2py4=:AmazonSES X-SES-Outgoing: 2021.10.03-54.240.8.51 Welcome to Discourse! Click the following link to confirm and activate your new account: http://scanalyst.fourmilab.ch/u/activate-account/ac497e68eef4248c61b1690443747d2c If the above link is not clickable, try copying and pasting it into the address bar of your web browser. * * * I hope that the above information is useful in deciding to grant the request for production status.