Development Log: 2021-10-05

2021 October 5

Created an Amazon AWS S3 bucket, arn:aws:s3:::scanalyst-backups, in
the EU (London) eu-west-2 region.
    AWS Region:     EU (London) eu-west-2
    Resource name:  arn:aws:s3:::scanalyst-backups
    Permissions:    Not public
Obtained access key and secret password from AWS console:
    Selected (account name)/My Security Credentials from top bar.
    Clicked Users in left bar.
    Clicked user kelvin.
    Selected "Security credentials" tab.  Access keys shows the one
        for the Fourmilab backup bucket.
    Clicked "Create access key:
        Access key ID:      REDACTED 
        Secret access key:  REDACTED
        CSV:    Access key ID,Secret access key
                REDACTED,REDACTED

In Discourse Admin/Settings/Files:
    s3 access key id:       REDACTED
    s3 secret access key:   REDACTED
    s3 region:              EU (London)
Do *not* set "s3 upload bucket.  This is for directing uploaded files
to S3 as opposed to local storage.  You do not want this, and if you
set it, you cannot store backups there.

In Discourse Admin/Settings/Backups:
    backup location:        S3
    backup frequency:       1
    s3 backup bucket:       scanalyst-backups
    backup time of day:     4:17

Ran a manual backup.  It completed OK and the file showed up in the S3
bucket as default/scanalyst-2021-10-05-110356-v20210922064213.tar.gz.

We'll see if regular nightly backups run as scheduled.

Set Admin/Security force_https on.  This is supposed to promote http:
URLs from users to https:.  It appears to be working.

Installed a copy of Bacula from AWS into /server/src/bacula, and
created the /server/bin/bacula directory into which it will be
installed.  Built and installed the Bacula file daemon.  Since the
daemon is entirely configured from its configuration file, no
parameters need be set on the build.

Created a configuration file, /server/bin/bacula/bacula-5.2.10/etc/bacula-fd.conf,
adapting the one from AWS for our client name, Scanalyst.

Installed the /server/init files, bacula and functions, needed to
control the daemon.

Started the Bacula file daemon:
    /server/init/bacula start
and verified it is running.

Updated the Bacula director configuration on Pallas to replace the
retired Ratburger client definition with one for Scanalyst, retaining
its scheduled full backup day of the 20th of the month.  Added an
entry for "scanalyst" to /etc/hosts.  Tested with client status
query and it looks fine.

Added Scanalyst_AWS_Server to the firewall Hosts list and to the
"Bacula for Remote Servers" rule so that the Bacula File Daemon
on Scanalyst can send data to the Storage Daemon on Pallas.

Started a full backup from Pallas.
Full backup completed successfully in 41 minutes, 9.53 Gb written.

Created a ~/Scanalyst_Backup on Juno based upon its analogue for
Ratburger.  Ran initial full RSYNC backups of Server and / partitions.
The ./Backup_all script in this directory can be used to update
incremental backups.

In order to enable DKIM (DomainKeys Identified Mail) a message signing
scheme which guards against main in the middle attacks, according to
the directions in:
    https://docs.aws.amazon.com/ses/latest/DeveloperGuide/send-email-authentication-dkim-easy-setup-domain.html
I set up Easy DKIM for the entire fourmilab.ch domain as follows.  On
the page:
    https://console.aws.amazon.com/ses/home?region=us-east-1#verified-senders-domain:
I clicked "Verify a New Domain", which generated the following records
to be inserted in the fourmilab.ch DNS:
    "Domain verification record set:"
    "Record name","Record type","Record Value"
    "_amazonses.fourmilab.ch","TXT","REDACTED"

    "Alternate domain verification record:"
    "Record name","Record type","Record Value"
    "fourmilab.ch","TXT","amazonses:REDACTED"

    "DKIM record set:"
    "Record name","Record type","Record Value"
    "REDACTED._domainkey.fourmilab.ch","CNAME","REDACTED.dkim.amazonses.com"
    "REDACTED._domainkey.fourmilab.ch","CNAME","REDACTED.dkim.amazonses.com"
    "REDACTED._domainkey.fourmilab.ch","CNAME","REDACTED.dkim.amazonses.com"
(Sent as CSV: enter as regular DNS declarations in Route 53.)  After you
add the DNS records, you wait for your domain to change from "Pending
verification" to "Verified" for both the main domain verification and
DKIM.  Now it should be sending DKIM.  Let's see.

Yes, mail-tester.com reports:
    Your DKIM signature is valid
        Message has at least one valid DKIM or DK signature
        Message has a valid DKIM or DK signature from author's domain
        Message has a valid DKIM or DK signature from envelope-from domain

Terminated the original instance, i-082d04418d82bcfa1, which was
accidentally created in Availability Zone eu-central-1a and without IPv6
and retired when we moved over to the current instance in eu-central-1b.
Deleted its volumes:
    /       vol-02ccb8d0ba2bd3ef9
    /server vol-0057760635b5e0129
At this time all volumes are assigned to (not necessarily running)
instances.

Added a CloudWatch dashboard for Scanalyst CPU Balance and usage.
Added an alarm for Scanalyst CPU balance falling below 250.