Last Updated: March 2021

#Overview

This diagram was created with https://app.diagrams.net/. To edit it, download the diagram file and edit it with the https://app.diagrams.net/ web application, or you may run the application from source if you wish.

#Physical Hosts
  • baikal

    • 69.61.110.118
    • cyberia's first rack server, installed in CyberWurx datacenter in Atlanta Georgia
    • j3s is the only authorized support contact right now and the only one who can log into the CyberWurx portal
  • dredd

    • dynamic(ish) ip address
    • NOTE: dredd uses port 3217 for ssh. connect with ssh -p 3217 dredd.cyberia.club
    • olde desktop located in j3s's house
  • magnataur

    • dynamic(ish) ip address
    • NOTE: magnataur uses port 3216 for ssh. connect with ssh -p 3216 magnataur.cyberia.club
    • olde desktop located in j3s's house
#Cloud Service Accounts
  • namecheap

    • fack's namecheap account is currently being used to manage DNS entires for:
      • cyberia.club
      • nullhex.com
      • capsul.org
    • all DNS updates are being done manually by j3s.
    • conventions:
      • A records are named after hostnames & point to VMs / physical hosts
      • CNAMEs are named after the service & point to the A record of the host the service runs on
  • CyberWurx portal

    • Allows us to add reverse DNS entries for Capsuls
    • View metrics, get datacenter information, support tickets, etc
    • Right now j3s is the only one who can log in / be authorized for support. Can add others though!
#Capsul

Most of cyberia's services run on Capsul, our Virtual Machine Management tool & service.

Ansible Managed Capsuls:
capsul-ay3yh10q2q  f1-xs  69.61.2.230  alpine311  Jun 20 2020 domechild.cyberia.club  (email server)
capsul-c04bbf593b  f1-s   69.61.2.246  alpine311  Jun 01 2020 raaz.cyberia.club       (NSHC / North Star Health Collective) 
capsul-pfgy2tthx9  f1-xs  69.61.2.167  alpine311  May 10 2020 legion.cyberia.club     (postgres for forge & others in the future)
capsul-id502edkg0  f1-xs  69.61.2.170  alpine311  Apr 01 2020 rosewater.cyberia.club  (cyberia forge)
capsul-t6tfb2dh5p  f1-m   69.61.2.183  alpine311  May 10 2020 mothership.cyberia.club (prometheus & grafana & future logg agg)
capsul-w6hsx09r7v  f1-xs  69.61.2.213  alpine311  Aug 20 2020 leckie.cyberia.club     (ansible bastion + build submitter)
capsul-f6crtfzx5c  f1-xs  69.61.2.218  alpine313  Mar 01 2021 comet.cyberia.club      (owncast server)
capsul-e1tfrw0637  f1-xs  69.61.2.201  alpine313  Mar 13 2021 kindred.cyberia.club    (mastodon server)

Legacy Capsuls:
capsul-yi9ffqbjly  f1-x   69.61.2.188  debian10   Apr 15 2020 btcpay.cyberia.club     (btcpay) 
cvm-lqj2x9nxic	   f1-l   69.61.2.190  debian10   Mar 07 2020 matrix.cyberia.club     (cyberia matrix) 
cvm-m1tjv0lljd	   f1-xs  69.61.2.178  debian10   Mar 10 2020 elliot.cyberia.club     (websites & git.cyberia.club, nullhex.com)

The Ansible Managed servers should have a user account for each user. The Legacy servers & baikal only have one user named cyberian, with everyone's keys authorized for that server.

Contact j3s, forest, or vvesley for more information on cyberia's capsul account.

#Host Key Fingerprints

NOTE: you can control what kind of host key your ssh client will use like this:

ssh -o HostKeyAlgorithms=ssh-ed25519 example.cyberia.club

baikal.cyberia.club
  ECDSA    SHA256:85GTFfUpDDefcNcIROtFpuTiHC1j3iNU74aaKFO03+0
  ED25519  SHA256:v9MEa97wnmA75CyzQC5lW8nOI56LJ4jTmD2f68udK80

magnataur.cyberia.club
  NOTE:    magnataur uses port 3216 for ssh. connect with ssh -p 3216 magnataur.cyberia.club
  ECDSA    SHA256:kPOBn03CH176zrTlFDVmjFJpWi1OGHhkNCiK6stNn/0
  ED25519  SHA256:7M8ppVJ534Axz1ZXt6NheBxYkqY9UJ3AAmb9BmY9bYk

dredd.cyberia.club
  NOTE:    dredd uses port 3217 for ssh. connect with ssh -p 3217 dredd.cyberia.club
  ECDSA    SHA256:5157aYG7PT8Y0I4sTzlpQ5i/E3bq4aPF9T1P+xj+l9Q
  ED25519  SHA256:w6F0NXBoLCXG60yXoI3QhYGiLlPCr6YrK/OUSSDcmAw

mothership.cyberia.club
  ECDSA    SHA256:3XJG2fyaPDJWjnEOW3q2KiWg5qLV6hmEPczvp8GqhE0
  ED25519  SHA256:njIT2k1t6hHuOO0VjBNmHW1QSGN4GEqQQMj/BGpnBa0

domechild.cyberia.club
  ECDSA    SHA256:IQqTPv14u3dG62hS0q2Mr6pef6KwpjPKM2uVP+SK+qA
  ED25519  SHA256:3z5BI2ZEZjzDEh0B7a2GxgMa4faqA3Y6bQdGcQp4G88

rosewater.cyberia.club
  ECDSA    SHA256:dAbABreDUpV9AG7kChcx9S6+6f+fmnhqwwInqYoxcwU
  ED25519  SHA256:nT+ISIGV95MBKkIpcHTKo30lx4qRQ0Cpu1iM3w6+Sh0

legion.cyberia.club
  ECDSA    SHA256:EW9ydcgLg/pwoA0GPsI0VVeIBpnSi7aIHhvXOQBa+Xg
  ED25519  SHA256:cWLBFESOHrmVFrLRLjxrY4tcPmVRerJe1SB/+6tXSxAv

leckie.cyberia.club
  ECDSA    SHA256:KbzxzEKP21B0S3A/SKqqGmjiymnkk7byvoc6W4SxEwM
  ED25519  SHA256:M1QPflfIrsbhVlMaomvGQsr5AZS5YRkBHv+pnyI7bg4

raaz.cyberia.club
  ECDSA    SHA256:AJb0bZN2PTTm83zf5zI1IOEIVfeXUxQl/vTode/88jA
  ED25519  SHA256:zJv6E6lG4dAsqNmDHTO/qFVlTESKYq/KD29e8Nt/6j4

matrix.cyberia.club
  ECDSA    SHA256:VlRPAqLGxY4JUVhYirOVlfuDFtgTbaiw3x29xYizEeU
  ED25519  SHA256:BExhsVPNTp49jyJ6ezRf+Nn4TxPj8D9VZMhnjMABq6g

elliot.cyberia.club
  ECDSA    SHA256:/tsASDZ+MX519DC/Y7mHbV2CYCPnyMAbX1e0GHBOin0
  ED25519  SHA256:B9QNCnz57agsI40tMVU8UwyvZqMbz/p1ZNH5E1gL3io

btcpay.cyberia.club
  ECDSA    SHA256:CdqdUvG0Obfdq9kkeQSETVhSJO2oCAdEAjDCydQWcDI
  ED25519  SHA256:WcjrJtvev3+rAu98TFGJoxx/CytLCg+GfEXBMVOl5Hw

comet.cyberia.club
  ECDSA    SHA256:UcDUCFd/U3F8ECG/RKxLbJRAAiMBSRKVKqDM0hmjwJ8
  ED25519  SHA256:SoOuSzKmpUd4x8Y8G32EAfQTY15agz1z7zJJCWdI8Tw

kindred.cyberia.club
  ECDSA    SHA256:M2oWKPgOqynag2nXrxnideac+r4Vb2tAsEz5ddEh/EM
  ED25519  SHA256:wCyMJYgoPAwlFKTXw41v/q8kypuand4fmhY4zsWdGlc
#Automation (Ansible)

The Ops Handbook is still on the old git server, it is the main repo with the ansible inventory & playbooks.

Ansible bastion host/automation is on leckie.cyberia.club

#Service Inventory
User-oriented Name URL Developer-oriented Name Host Deployment Code Application Code
cyberia's matrix server https://matrix.cyberia.club/ synapse matrix.cyberia.club ansible/roles/synapse matrix-org/synapse
cyberia's matrix server https://riot.cyberia.club/ element (used to be called riot) matrix.cyberia.club ansible/roles/riot vector-im/element-web
cyberia's matrix server N/A postgres matrix.cyberia.club ansible/roles/postgresql git.postgresql.org
cyberia's matrix server N/A irc bridge to freenode matrix.cyberia.club TBD matrix-org/matrix-appservice-irc
cyberia's matrix server https://matrix.cyberia.club/_synapse/metrics matrix prometheus exporter matrix.cyberia.club TBD matrix-org/synapse/metrics
nullhex email https://nullhex.com/ alps elliot.cyberia.club TBD ~emersion/alps/
nullhex email nullhex.com ports 25 & 587 (STARTTLS) opensmtpd domechild.cyberia.club ansible/roles/opensmtpd OpenSMTPD/OpenSMTPD
nullhex email nullhex.com:993 (imap) dovecot domechild.cyberia.club ansible/roles/dovecot dovecot/core
nullhex email N/A rspamd domechild.cyberia.club TBD rspamd/rspamd
capsul https://capsul.org capsul baikal.cyberia.club TBD ~forest/capsul-flask/
forge (cyberia's git server) https://forge.cyberia.club/ sourcehut rosewater.cyberia.club see the ops-handbook ~sircmpwn/sourcehut
forge (cyberia's git server) N/A postgres legion.cyberia.club TBD git.postgresql.org
concourse (the new build server) https://concourse.cyberia.club/ concourse rosewater.cyberia.club TBD concourse/concourse
vault (build secrets manager) N/A vault rosewater.cyberia.club ansible/roles/concourse-vault hashicorp/vault
cyberia's website https://cyberia.club/ nginx static site elliot.cyberia.club TBD services/website
the old git server https://git.cyberia.club/ cgit elliot.cyberia.club TBD git.zx2c4.com/cgit
prometheus https://prometheus.cyberia.club/ prometheus mothership.cyberia.club rules & ansible/roles/prometheus prometheus/prometheus
alertmanager N/A alertmanager mothership.cyberia.club same as prometheus prometheus/alertmanager
grafana https://grafana.cyberia.club/ grafana mothership.cyberia.club ansible/roles/grafana grafana/grafana
Jackal https://bot.j3s.sh go-neb (matrix bot) mothership.cyberia.club TBD matrix-org/go-neb (forest's fork)
Stream https://stream.cyberia.club owncast comet.cyberia.club TBD owncast/owncast
Mastodon https://social.cyberia.club hometown kindred.cyberia.club TBD hometown-fork/hometown
#LetsEncrypt Certificate Inventory

For information on certificates which are managed by uacme automatically, see https://git.cyberia.club/services/ops-handbook/tree/ansible/hosts and the tls_certs variable in https://git.cyberia.club/services/ops-handbook/tree/ansible/group_vars

Certificates which are exceptions to the rule:

btcpay.cyberia.club
 - btcpay.cyberia.club certificate is automatically managed by btcpayserver-docker

elliot.cyberia.club
The following are managed by a script called acme.sh located at `/root/.acme.sh/`
 - capsul.org
 - www.capsul.org
 - nullhex.com
 - cyberia.club
 - git.cyberia.club

matrix.cyberia.club
The following are managed by a script called acme.sh located at `/root/.acme.sh/`
  - matrix.cyberia.club
  - riot.cyberia.club

magnataur.cyberia.club
The following are managed by Caddy on the router which sits in front of magnataur. Ask j3s or fack about this.
  - cafe.cyberia.club
  - mumble.cyberia.club
#How to use acme.sh:
systemctl stop nginx ; acme.sh --renew --domain git.cyberia.club; systemctl start nginx ;

If you get an error like

Please specify at least one validation method: '--webroot', '--standalone', '--apache', '--nginx' or '--dns' etc.

Then you must edit the config file, for example

nano root@elliot:~/.acme.sh# nano cyberia.club/cyberia.club.conf

and change Le_Webroot='' to Le_Webroot='no' inside the <domain-name>/<domain-name.conf> file. see: github issue

#certificate expiry alerts

The certificate expiry alerts are defined here: https://git.cyberia.club/services/ops-handbook/tree/rules/alerts.yml#n112

The probe_ssl_earliest_cert_expiry metric is written by the blackbox exporter, configured here: https://git.cyberia.club/services/ops-handbook/tree/ansible/roles/prometheus/templates/prometheus.yml.j2#n82

#Notes

https://poolp.org/posts/2019-09-14/setting-up-a-mail-server-with-opensmtpd-dovecot-and-rspamd/

About this wiki

commit 88007aa68da52b148a764374b9e45ae284669445
Author: j3s <j3s@c3f.net>
Date:   2021-07-25T13:19:56-05:00

Add minutes to list
Clone this wiki
https://giit.cyberia.club/~cyberia/docs (read-only)
git@giit.cyberia.club:~cyberia/docs (read/write)