Hello everyone,
I recently set up a drop-in systemd unit file that adds an "on failure" command to all user-level services on a home server. Specifically the on failure command sends a notification to ntfy.sh detailing the service that has failed. E.g. if my miniflux container fails, it will send a notification such as Le service miniflux.service a échoué. Vérifier : journalctl --user -u miniflux.service
However, particularly on server startup, I have been getting some weird failure notifications for services I cannot find.
Any ideas on what I can do to troubleshoot this? An example is provided bellow.
Title:
⚠️ ❌ 303a3d11aa5781aaca14faa1e36fbe4465466f74e49821fee252d01227f8fdfb-73342cf5eacdadbd.service a échoué
Body:
Le service 303a3d11aa5781aaca14faa1e36fbe4465466f74e49821fee252d01227f8fdfb-73342cf5eacdadbd.service a échoué. Vérifier : journalctl --user -u 303a3d11aa5781aaca14faa1e36fbe4465466f74e49821fee252d01227f8fdfb-73342cf5eacdadbd.service
6 Comments
dustbin@thelemmy.club · 7 pts · 158d
The naming convention looks suspiciously like an autogenerated (dynamic) unit; one typically sees these from /etc/fstab being parsed at boot by systemd-fstab-generator into /run/systemd/.
Try perusing https://www.freedesktop.org/software/systemd/man/latest/systemd.generator.html and /run/systemd/ on your system, see if anything catches your eye...
kaki@sh.itjust.works · 3 pts · 158d
OP mentionned containers, and Podman/Quadlet generates such services for health-checking a container service, so it might be that.
dustbin@thelemmy.club · 2 pts · 158d
It's probably most effective at this point to add
systemd.log_level=debug systemd.log_target=kmsgto the kernel boot options to ferret out what's creating it then go from thereKerplunk@mastodon.scot · 1 pts · 143d
@dustbin @kaki
It’s probably most effective at this point to add systemd.log_level=debug systemd.log_target=kmsg to the kernel boot options to ferret out what’s creating it then go from there
It is 100% more effective to move to a nosystemd distribution.
Whooping_Seal@sh.itjust.works · 1 pts · 147d
I havent had the chance to reboot with the parameters mentioned below, but I think it most definitely is the health check (which I have manually configured for everything but syncthing).
Kerplunk@mastodon.scot · 0 pts · 143d
@Whooping_Seal
Can't figure out what systemd service is failing.
over 2 million lines of slop coding, 2000 plus bugs open.
systemd is a fail