c/reddthat · by ticoombs@reddthat.com · 238d

[Migration Complete] Moving to Dedicated Server (& Lemmy v0.19.15)

I found some time. and in 15 minutes from this post we will go down for ~1 hour to ensure we have complete data consistency.

09:00 UTC to 10:00 UTC

~~See you soon!~~

HI!!!!!!!!

32 points · 20 comments · view on lemmy.world

20 Comments

ticoombs@reddthat.com · 7 pts · 238d (14 replies)

Migrated Successfully!

Pictrs Test:

ticoombs@reddthat.com · 5 pts · 238d (12 replies)

Working on fixing old.reddthat.com.

ticoombs@reddthat.com · 6 pts · 238d

Fixed up. All seems to be working. 😪

For those of you who were watching, check out our possible new status page: https://statusv2.reddthat.com/ (it has our CPU/Ram/Disk usages on display for everyone to peruse)

e0qdk@reddthat.com · 2 pts · 238d (10 replies)

old.reddthat.com is currently not working (at time of posting this comment). Most pages show unable to retrieve site., although I did get dial tcp (an IP address):443: connect: connection refused. (where (an IP address) is an IP that I've redacted) on some pages.

ticoombs@reddthat.com · 2 pts · 237d (8 replies)

I believe I've fixed it.

A recent docker update no longer supports overriding hosts with hostnames. (I was telling mlmym that 'reddthat.com' is 'lemmy-ui'. Which would make it request directly to the lemmy-ui container rather than going back out to CF and back again.

(For others/if you want to know the nitty gritty). I was doing this:

    networks:
      lemmy_internal:
    extra_hosts:
      - "reddthat.com:lemmy-ui"

Which did actually work, but now it doesn't since I updated/installed docker compose v5.0.0. (Seems the correctly enforce the host:IP format now). So I had to do a work around.

e0qdk@reddthat.com · 2 pts · 237d (7 replies)

Thank you. It seems to be working now.

I remember having a devil of a time trying to get mlmym and lemmy to talk to each other on my local system when I was poking around at trying to fix some issues in mlmym a few months back. (I should really get back to that at some point...) They had "https://" hard-coded in front of the domain name parameter -- and I didn't want to have to change the Go source code just to get a dev environment up. I ended up working around it by modifying my hosts file (to add a .INTERNAL domain), ran things under podman with host networking, configured Apache on the host to reverse proxy into both containers, and set up a self-signed cert that I had to rebuild into the mlmym docker (by copying in the appropriate crt files + running update-ca-certificates via the Dockerfile). In retrospect, I probably should've just modified the code...

What was the workaround you had to implement to fix this one?

ticoombs@reddthat.com · 2 pts · 237d (6 replies)

I had to tell our main_nginx that it is "reddthat.com".

services:
  proxy:
    image: docker.io/library/nginx
    ...etc...
    networks:
      default:
      lemmy_internal:
        aliases:
          - reddthat.com

Now its: {user} -> CF -> {server_nginx} -> {frontend_nginx} -> mlmym => ((docker internal network)) -> {main_nginx} -> {{lemmy-ui/api}}
Instead of: {user} -> CF -> {server_nginx} -> {frontend_nginx} -> mlmym => CF -> {main_nginx} -> {{lemmy-ui/api}}

e0qdk@reddthat.com · 1 pts · 237d (5 replies)

...and it's down again. 🫠️

ticoombs@reddthat.com · 2 pts · 236d

SORTED! Ended up patching it myself. Enjoy :D

ticoombs@reddthat.com · 2 pts · 237d

😥. I will fix it! I think because we are still going via nginx, it's still has to pass the go-away challenges, which it must be failing and thats why...

Annnd, as we just moved I also only have key access to the server and I made all new keys and don't have one on my other devices yet. (A classic)

Edit: nope, it's now attempting to connect over https to the container... I think I'll need to write a proper patch for it to allow for a override.
It also looks dead in the water/feature complete. So I've moved to FSFs container. (https://github.com/Fedihosting-Foundation-Forks/mlmym).

If I end up making a patch I'll upstream. (I'm surprised LW hasn't made one already)

ticoombs@reddthat.com · 2 pts · 238d

:( yeah the outage notification woke me up.

As it was late I did a work around to temporarily fix it. I'll be looking at it once I've had coffee.

AppleStrudel@reddthat.com · 2 pts · 225d

Wooo! 🎉

ticoombs@reddthat.com · 5 pts · 237d

Fun Pics: