How often do you update software on your servers?

Or asked the other way around: How long do you keep your servers running without installing any software updates?

update means something like

sudo dnf update

or something ....

apt-get upgrade
apt-get update
103 points · 76 comments · view on lemmy.world

76 Comments

mjr@infosec.pub · 57 pts · 291d (2 replies)

Those apt commands are in a less-good order. It's usually better to update apt, then upgrade the system.

I upgrade as soon as reasonably possible after the notification appears, if the system isn't on auto-upgrade.

njordomir@lemmy.world · 3 pts · 290d (1 reply)

I do sudo apt update && sudo apt upgrade

Is there any reason to not combine the commands since the output always prompts prior to changes anyway?

cyberwolfie@lemmy.ml · 5 pts · 290d

I think their point was to make sure they are done in order, i.e. update before upgrade, not the other way around as in OPs example.

dust_accelerator@discuss.tchncs.de · 40 pts · 291d (6 replies)

Every night at ~ 12-1am

unattended updates / transactional-update are awesome.

Stuff has been running for years, and it's still up to date.

southernbeaver@lemmy.world · 23 pts · 291d

This guy scares me

DasFaultier@sh.itjust.works · 5 pts · 291d

This is the way! At least install security upgrades nightly using unattended-upgrades and reboot from time to time to get the latest Kernel version.

gopher@programming.dev · 2 pts · 291d

Once per week for me. Works really great on openSUSE MicroOS. Had to roll back maybe a couple of times the last few years.

That said, I run basically everything in containers so the OS installed things are lean.

JustEnoughDucks@feddit.nl · 1 pts · 290d (1 reply)

I wish I could use unattended-upgrade.

It literally restarts my server even when I disable the option, leaving it hung if the USB boot key isn't in there.

I had to stop using it, so now I just manually upgrade because that doesn't auto-restart without my permission...

vegetaaaaaaa@lemmy.world · 2 pts · 290d

unattended-upgrades doesn't do that unless you explicitly specify Unattended-Upgrade::Automatic-Reboot "true"; in the config. Check /usr/share/doc/unattended-upgrades/README.md.gz

The main configuration file is /etc/apt/apt.conf.d/50unattended-upgrades, maybe you put your config in the wrong place?

here is mine

Ugurcan@lemmy.world · 1 pts · 290d

Tell me you’re using nightly builds as well.

Dran_Arcana@lemmy.world · 23 pts · 291d (1 reply)

Unattended-upgrade does security-only patching once every 4 hours (in rough sync with my local mirror)

Full upgrades are done weekly, accompanied by a reboot

I find that the split between security patching and feature/bug patching maintains a healthy balance knowing when something is likely to break but never being behind on the latest cve.

cenzorrll@piefed.ca · 3 pts · 291d

For me, unattended-upgrade does it's thing. Updating other packages happens whenever I think about it. Very few things are not containerized and there's very little added beyond the base Debian install, so when I do update its maybe a dozen packages.

I would previously reboot during thunderstorms if we lost power, but now that I've got a UPS I probably ought to come up with a different plan.

sneezycat@sopuli.xyz · 17 pts · 291d (11 replies)

Well, one of the reasons I'm using debian on my server is so I can kinda forget about it...

I'll update maybe once a month, or every couple months. I don't always restart though, so my kernel is probably a bit behind :'D

eksb@programming.dev · 8 pts · 291d (1 reply)

I use Debian stable and subscribe to the debian-security-announce mailing list, so I update each time I get an email from it.

amju_wolf@pawb.social · 1 pts · 291d

This is the way. (At least for a server)

PlanterTree@discuss.tchncs.de · 4 pts · 291d

lol. Same issue for me. I run it for months, and surprisingly (for me) nothing breaks at all.

But fucking ssh shows warnings regarding some "post quantum crypto" stuff; recommending software update, that was not there before lol.

atzanteol@sh.itjust.works · 4 pts · 291d (7 replies)

That's... Not how it works.. Debian is "stable" not "secure". You use Debian so that is easier to run updates frequently since they'll be unlikely to break things.

sneezycat@sopuli.xyz · 0 pts · 291d (6 replies)

If I wanted to run updates frequently I would run arch lmao. Even if I did apt update every day, debian stable doesn't get that many updates.

I could just run auto-update but meh.

atzanteol@sh.itjust.works · 3 pts · 291d (3 replies)

If I wanted to run updates frequently I would run arch lmao. Even if I did apt update every day, debian stable doesn’t get that many updates.

You're not updating for features you're updating for bug and security fixes. That's why Debian stable doesn't have many updates. But the ones they do are typically important.

sneezycat@sopuli.xyz · -2 pts · 291d (2 replies)

Yeah, I know. Until I get ransomware'd and my nudes leaked, I won't care 💅🏻✨

atzanteol@sh.itjust.works · 2 pts · 291d (1 reply)

Clearly you don't know.

sneezycat@sopuli.xyz · -1 pts · 291d

I guess people smoke because they don't know smoking causes cancer ;3

lastweakness@lemmy.world · 1 pts · 291d (1 reply)

Are you talking about desktop use?

sneezycat@sopuli.xyz · 1 pts · 291d

No, my home server. My desktop and laptop both have arch, because I do interact with them more often.

CompactFlax@discuss.tchncs.de · 13 pts · 291d (2 replies)

When I remember. About once a month.

spacelord@sh.itjust.works · 5 pts · 291d (1 reply)

Same here. No auto updates, no touching of a stable system without my manual intervention. 😅

CompactFlax@discuss.tchncs.de · 4 pts · 291d

Last thing I need in my life is a broken system at home when I don’t have time for it!

pHr34kY@lemmy.world · 11 pts · 290d

Whenever I ssh into it.

slazer2au@lemmy.world · 8 pts · 291d

Once a week. I have a bash script that does an apt update upgrade and pulls new docker images.

rikudou@lemmings.world · 6 pts · 290d

Monthly unless I learn about a vulnerability that would require it sooner.

hexagonwin@lemmy.sdf.org · 5 pts · 291d

maybe like once in 3 months. i usually update when i need to setup something new on the server that needs to install new packages.

Sir_Kevin@lemmy.dbzer0.com · 5 pts · 291d

On Windows, almost never since it was a disruptive shitshow. Now that I've got everything running Linux it's weekly. Often sooner if I happen to be remoting in and manually update.

lambalicious@lemmy.sdf.org · 4 pts · 290d

Only mostly when I want to. Which tends to be on Mondays and Saturdays.

I'm running Sid on servers, so automatic updates are actually a risk. Used to be Debian Stable, but maaan the docker and podman improvements.... make me drool.

Evotech@lemmy.world · 4 pts · 291d

Weekly. Cronjob.

deleted@lemmy.world · 4 pts · 291d

I do it every 3 to 5 days. I usually do it when I have time to fix things if it goes south.

First_Thunder@lemmy.zip · 3 pts · 291d

Using nix :P

I update the flake every now and then via nix flake updated and then do a rebuild

reseller_pledge609@lemmy.dbzer0.com · 3 pts · 291d

I SSH in and run an update manually, once a week.

I'm not knowledgable and comfortable enough to let updates happen automatically and feel like I could trust it to keep running. Not yet, anyway.

Edit: But at some point I might do what another commenter said and make sure security updates run automatically and check other updates weekly.

hydrian@twit.social · 3 pts · 291d (4 replies)

@PlanterTree Systems facing public internet, security updates are applied daily automatically.

PlanterTree@discuss.tchncs.de · 4 pts · 291d

up to now I install all my updates manually, maybe I should look into this: how to auto-update.

Passerby6497@lemmy.world · 2 pts · 291d (2 replies)

Can I ask how you do that? I have some debian and fedora boxes I should configure for that

communism@lemmy.ml · 3 pts · 290d

Automatic daily updates for system packages. Automatic daily container updates with watchtower. I normally have things pinned to a reasonable major or minor release, so I do manual upgrades for new OS release branches and usually pin to a major version for Docker containers but depends on the container.

Auli@lemmy.ca · 3 pts · 290d (1 reply)

Every day or at least once a week. Should automate it.

PlanterTree@discuss.tchncs.de · 1 pts · 286d

Should; Could; How high of a priority is this update automation for you? This is also how I run my server. Configuration possibilities are infinite.

lemming741@lemmy.world · 3 pts · 291d

Probably every 2 months. When I have a day off work with nothing to do. I have a few VMs that are more fragile than I want to admit and if something breaks I want to have time to tinker instead of just restoring a backup.

Shimitar@downonthestreet.eu · 3 pts · 291d

Gentooer here. Emerge sync &; world daily at night.

Weekly a manual check for stuff that doesn't autoupdate for reasons.

Monthly / biweekly podman compose pull for containers. Manual, because i don't trust that kind of autoupdate.

Edit: opnSense updates are manual only when I remember because if it breaks, I must be at home to fix it or i lose remote access and that's bad.

melfie@lemy.lol · 3 pts · 291d
[ removed ]
confusedpuppy@lemmy.dbzer0.com · 3 pts · 291d (4 replies)

On Alpine Linux I update my two Pi servers at 2 in the morning daily. It's simpler compared to Debian which needs unattended-updates. Just add apk update && apk upgrade to a cron job and you're good to go.

I only have three docker services which is simple enough to update manually.

I like to keep things as simple as possible for my already chaotic brain.

non_burglar@lemmy.world · 4 pts · 291d (1 reply)

Be careful with unattended upgrades, even on alpine. A recent breaking change in python3 broke my alpine 23 ansible instance. Thankfully I have backups, but if you're going to automate the upgrade, you should automate tests as well.

confusedpuppy@lemmy.dbzer0.com · 2 pts · 291d

My web facing server has just enough packages installed to (kinda securely) host a Caddy and Kiwix docker container to work with my domain name and make a comfortable work environment through SSH. My Pi for my HomeAssistant docker container has less because it's locked down to just my local network.

I also wrote my own install scripts so reinstalling everything and getting it back to a running state would take about 15 minutes for each device.

And I also wrote my own backup/restore scripts that evolved over 3/4 of a year. I use them often so I have confidence in those scripts.

I personally don't really care too much. I have multiple ways of dealing with issues for something that's a hobby to me. Which is why I stick to simplicity.

I'm sure this is a thing for people to worry about when dealing with more complex setups. I just wanna vibe out in my tiny corner of the internet.

stratself@lemdro.id · 2 pts · 290d (1 reply)

To make it even simpler, apk -U upgrade

confusedpuppy@lemmy.dbzer0.com · 2 pts · 290d

apk seems to have some tricks in there that aren't as well known.

I managed to catch in the IRC channel that apk add doc will automatically download any related man pages for packages with any future downloads through apk. That made life a bit more convenient instead of downloading all those packages separately.

snekerpimp@lemmy.world · 2 pts · 291d

Almost everything I have runs Debian or NixOS, so…….. once a month? Except for VMs I’m playing around with, which usually get updated every time I log into them, or instal stuff.

Mikelius@lemmy.ml · 2 pts · 290d

Daily on my Gentoo server, through a Cronjob every morning. It's a custom script though, so there's more than just doing an emerge update. It'll send me ntfy notifications for the update results, if there are new news items, and if there are any time config merge updates to make. A few other things as well but that's the main stuff.

Other servers, typically weekly or only manually when I ssh into them (for the ones I don't really feel the need to update frequently).

bjoern_tantau@swg-empire.de · 2 pts · 291d

Automatic upgrades handle the security patches. Everything else maybe once a month. My big services like Nextcloud auto update as well.

sobchak@programming.dev · 2 pts · 290d (2 replies)

When something doesn't work. I.e. when an app update causes incompatibility with a service. I think I have one server that's a few years without an update (distro version may actually be EOL for all I know).

Auli@lemmy.ca · 1 pts · 290d (1 reply)

Why probably so may unpatched issues.

sobchak@programming.dev · 1 pts · 290d

Ain't broke and I can't be bothered to update. Not accessible publicly either. It also runs some software with very specific and brittle dependencies and I don't care to risk breaking it. If distro is EOL (probably is) then it'd be a pretty time consuming getting everything set up again.

Treczoks@lemmy.world · 2 pts · 290d

Apt update and upgrade happen automatically.

illusionist@lemmy.zip · 2 pts · 291d

On my ubuntu I use unattended updates but that doesn't work reliably. I have to update it manually most of the time. Once every other month.

On my fedora server it auto updates every day at 4 reliably.

The next server is going to be atomic such that the server restart is even shorter (not that I would care about it at 4).

eager_eagle@lemmy.world · 2 pts · 291d

everyday to once a month, depending how often I use the server

IME usually waiting longer to apply larger updates causes more issues than smaller and more frequent ones

suicidaleggroll@lemmy.world · 2 pts · 291d

Every couple of days. I don't auto-update, but I've streamlined the process to the point that I can just open a single web page and see the number of pending updates for every system on my network, docker containers included, each one with a button. Clicking the button applies the update and reboots if necessary. So it takes about 15 seconds of effort to update everything, which is why I don't mind doing it so often.

jbk@discuss.tchncs.de · 2 pts · 291d

podman quadlets with auto updates running on opensuse microos

im not yet self hosting a ton of services tho

Kolanaki@pawb.social · 2 pts · 291d

Mine is set to update all the stuff I use, and the OS, automatically whenever an update is available. 🤷‍♂️

jimmy90@lemmy.world · 2 pts · 290d

my nixos containers and the podman containers inside them update nightly around 03:00

troed@fedia.io · 2 pts · 291d

All services are dockerized, updated nightly.

Server OS runs a kernel-patch service for real time exploit patching.

All other updates as soon as they appear.

Yeah, sometimes I'll need to go in a repair - but that's way better than having to clean up after having been exploited due to not keeping up on security patches.

vga@sopuli.xyz · 2 pts · 290d

If I have something serious, I will set up automatic upgrades. If short downtimes are ok, also with automatic reboots when the kernel updates, but if they are not, with notifications that I should go reboot them.

If it's not anything serious, whenever I remember to.

30p87@feddit.org · 1 pts · 291d

Every day to once a week, depending on free time

ilco@feddit.nl · 1 pts · 291d

Usely every 3/4 months roughly. I try to remeber to update. The base. Server. And docker based things! /webserices. I update. Sparingly. Every few new versions. As I am the only user of my server. I don't have a high need to update. So I update only if a new future. Is added or a mayor bug /security patch.

atzanteol@sh.itjust.works · 1 pts · 291d

All systems, daily via a single ansible script. That's apt update, upgrade and reboot if needed (some systems set to only reboot with a separate script so I can handle them separately).

Rarely have any sort of problems.

ShortN0te@lemmy.ml · 1 pts · 291d (1 reply)

Got apticron set up on my servers or similar solutions to get notified when updates are available. Then usually, from time of notification +1 or 2 days.

ShortN0te@lemmy.ml · 1 pts · 291d

And for containers auto updates once every day.

BCsven@lemmy.ca · 1 pts · 291d

Depends, on how critical something is...since we deal with servers / customers at work that often are purposely not adjusted for years...because introducing a different behaviour (even if better) would grind production to a halt, I take a not careful approach.

I was using OpenSUSE Leap, and with zypper you can review which patches are available, whether they are critical or run recommended or not needed. You can then apply which specific patch you want be CVE if necessary.

But with Leap's path seaming messy at the moment, I moved to Tumbleweed, since you have snapshotying built in. If an update did mess something up you just rollback to the previous snapshot and in less than a minute it is fixed

MTK@lemmy.world · 1 pts · 291d

Anything exposed to the internet gets a daily / weekly update, depending on how exposed it is, how stable the updates are and how critical a breach would be. For example nginx would be a daily update.

Anything behind a vpn gets a more random update schedule mostly based on when I feel like it (probably around once a month or every other month)

Luckyfriend222@lemmy.world · 1 pts · 291d

First Friday of the month. Easy to remember.

corsicanguppy@lemmy.ca · 0 pts · 290d

Yum-cron. Daily. Rolling bounce on a schedule.

It has been rock-solid for 20 years, but lennart's cancer and the growing amount of shite they're shoveling into EL has caused a few issues here and there with 7, 9 and 10. (Skipped 8 because f that)

But, today, it works. So that's year 23 and 8 months.