OdiousStooge

Huge thank you! I had a feeling something like this was going on but had no idea how to troubleshoot/fix.

My pictrs and lemmy containers were the biggest between 3-8 GB (significant for a smaller instance) after a couple weeks.

For anyone who finds this, in addition to what OP provided here, another command I found helpful (since I am a docker noob 🙂) to find the name of the currently running containers:

docker ps --format '{{.Name}}'

After some tinkering, **yes ** this indeed was my issue. The logs for pictrs and lemmy in particular were between 3 and 8 gb only after a couple weeks of info level logging.

Steps to fix (the post above has more detail but adding my full workflow in case that helps folks, some of this wasn't super apparent to me) - these steps assume a docker/ansible install:

1. SSH to your instance.

2. Change to your instance install dir

most likely: cd /srv/lemmy/{domain.name}

1. List currently running containers

docker ps --format '{{.Name}}'

Now for each docker container name:

1. Find the path/name of the associated log file:

docker inspect --format='{{.LogPath}}' {one of the container names from above}

1. Optionally check the file size of the log

ls -lh {path to log file from the inspect command}

1. Clear the log

truncate -s 0 {path to log file from the inspect command}

After you have cleared any logs you want to clear:

1. Modify docker-compose.yml adding the following to each container:

logging:
      driver: "json-file"
      options:
        max-size: "100m"

1. Restart the containers

docker-compose restart

UPDATE:

If anyone else is running into consistently rising disk I am pretty sure this is my issue (RE logs running with no cap):

https://lemmy.eus/post/172518

Trying out ^ and will update with my findings if it helps.

Gotcha thanks! Thats good to know. Based on the originating ticket: https://github.com/LemmyNet/lemmy/issues/1133

Sounds like it might be safe for me to purge that table a bit more often as well.

Dumb question, how are you profiling (RE your mention of getting a better idea of which tables might be bloated) your DB? Just SSHing into your box and direct connecting to DB? Or are there other recommended workflows?

💖 man ima dumb dumb. Most excellent, thank you!

Thanks!. Yeah I gotta figure that out. Something funky going on.

There are several communities in there atm but I'm getting some "odd" federation behavior.

If you are interested in federating there are two ways I have seen it work (again, warning, note that it is NSFW 😀 ):

• use your search feature on your main instance and enter !butt_frenzy@booty.world then you can subscribe

• or, from your main instance, alter your url to: {your main instance}/c/butt_frenzy@booty.world so for example if your main instance is lemmy.ml that would be: https://lemmy.ml/c/butt_frenzy@booty.world

^ butt_frenzy is one of the communities on booty.world for example.

Gotta be a better way, and I am probs doing something wrong. Will work on it. Open to feedback if anyone has any advice though haha.

Thanks.

Noice many thanks! I'll give it a shot.

Oh snap, yes I am using Digital Ocean.

Looks like you might have forgotten to add the link, mind chucking that in when you get a chance?

Thanks, this seems like a good lead as well.

Cool thanks! Last thing for now, and then I'll go do some debugging.

Dumb Q: The lemmy install includes an SMTP server correct? Like I don't need to use an external service for that?

Nice! Thanks your mention of the config.hjson makes me wonder lol. I probably goofed that too.

I did the ansible install which I believe just adds orchestration on top of the Docker install. I'll ssh in and try the docker-compose command.

Do you know, if I did goof the email config, can I just tweak the config locally and then re-run ansible? Or do I need to do some manual tweaks to the deployed solution? Or I suppose at this point it might be easier to just blow the instance away and start fresh.