the problem here does not originate in zfs, this is a hardware failure due to a poorly supported SoC (vendor says PCIe works 'normally' since kernel 7.1, this system ran 6.19) and bad electricity supply
do you ever get a new shiny thing and it looks like it works very nice and coherently and then you dig a bit deeper and realize that it is just a bunch of semi-broken shit held together by glue and duct tape? that's the state of android, that is nicely obscured on pixels by google customizations
haha, no, depending on your use case zfs is mostly set and forget, you just have to get a set of sane defaults, which, at least for me, is way easier and saner than stuff like btrfs. Arch wiki is a good source for that.
scrub is a command that reads data from disks and verifies checksums, it can automatically heal the corrupt data if some redundancy is available. 0 bytes repaired means nothing was repaired, 5 errors mean that there are 5 instances where the recalculated checksums of data did not match the expected values that were calculated when initially written. Normally the command would output 'scrub repaired 0B with 0 errors', that's how you know that the pool is healthy.
shprutzling the cache vdevs sideways is generally required only for obscure optimizations, I've never seen a case where a wrong setting would lead to data loss.
The gotchas come from a single thing: the stupid CDDL licence, which is not compatible with GPL. That means that zfs is not included in linux kernel and most of the normal linux userspace filesystem stuff do not work with zfs.
Also, keep in mind that zfs is generally almost impossible to fuck up, what I've done here is a result of extreme dedication and like 8 years of experience.
I've seen a calculation somewhere for drive speed, size and scrub frequency where optimal number of extra drives was calculated. On fast storage and with weekly scrubs one extra drive should be mostly fine.
My philosophy is any raid is not really a data contingency solution - it just saves some time when the drive fails.
This is raidz1. Three 4TB NVME drives. I've built this machine as the primary server for services like immich as a fun project that enables me to say that my photo service runs faster than all the commercial stuff and is on ARM64. The downside is that the SoC is not supported by upstream very well so vendor patches had to be applied and there was some instability between that and my shitty electricity supply.
I feel very sorry for you. I have the same problem (panic on mount) with this one, but at least it works when mounting read-only. I've been moving data with zfs send the entire day
I was using a mix of debian servers, arch thinkpads and macos macbooks at the point where I moved everything to NixOS (macbooks did not survive this transition and were donated due to never working properly because of t2 issues).
Nix is extremely hard. I mean, don't get me wrong, initial setup is easy, adding and updating packages is the easiest experience between all package managers out there, nixpkgs is overpowered as it has almost all the software that exists since there are no conflicting dependencies in nix. Getting something that is not in nixpkgs to run is impossible for a beginner (nixpkgs has steam-run to partially solve this, but results may vary). Getting to the point where a tornado may pass through your home and destroy all the hardware inside, and the whole recovery process from that point is about 20 minutes of your time, including the trip to the store for new hardware, requires significant effort, both time and mental, that most of us don't have. But the entire point is that it is possible with Nix!
If Debian is like owning a 90s Corolla, NixOS is like owning a factory that can make any car you like. You can choose to get a 90s Corolla, you can choose to get a newer Corolla, you can skip corollas entirely and make something new and fast. When the new and fast thing breaks, you can make a new car or go back to the Corolla. The Corolla comes completely ready to use by you from the production line. If you need to adjust the mirrors, you just make a new car with the correct mirrors setup. All the cars you make after that will come with the mirrors setup how you like them, because that is now a part of the manufacturing instructions the factory has.
The only caveat is that factory gets updates every 6 months and then your car building pipeline may break when you get the update. In three years I've been using nixos fixing those took about 5 minutes and usually boiled down to stuff like 'previously we had a list of car colors, passed as a string, now the factory expects to get a hex color code'.
Judging by the commit history, I moved most of my machines to NixOS after about 6 months building my config, then, 8 months later I rebuilt the entire thing from scratch when I understood nix enough to make everything truly modular.
I really doubt that once you've embraced NixOS you'd be tempted to use anything else or try LFS. NixOS is beautiful. NixOS just works (after 6 months of setup).
Profiles are nice and all, but remember, that there are still ways to track you across profiles (serious determination required), like the network stack being mostly shared. Here's the thread.
If you truly need isolated apps, consider getting multiple phones. I personally have three devices for this. For example, my main device (where I actually use the web browser and which is connected to my private network) runs only a limited set of foss apps. This will solve the alarms issue as well as this device has only one profile.
I have a fucking thinkpad that does not follow the USB PD spec!
The thing has only type-c ports, the thing is relatively recent (2023), the thing accepts 140W via type-c, I never bothered to check the actual specs and assumed PD 3.1
Turned out that no, this is not PD, this works only via proprietary power brick that outputs 20v@6.75A, that I didn't get as I already have a bajilion of PD 3.0/3.1 power supplies at home.
I work for a company that is one of the top worldwide plastic polluters. We use plastic in packaging. I was in an astonishing series of meetings recently, organized by the global corporate office for legal, R&D and supply chain that went like this:
First, we had a series of trainings about climate change that had none of the typical bullshit. They straight explained that the whole idea of recycling is stupid and shifts focus to the part where it makes the least impact, that carbon offsets don't really work, that we can't rely oil running out because the environment and the economy will collapse way sooner than that will happen.
The second part was I guess appealing to people's emotions by showing devastating effects that humans have caused already, for example there was a picture of starved baby bird that died because the parents were feeding it plastic garbage from the ocean.
The third part was practical plans to make various industries sustainable, the point was that drastic measures are required for most companies and that the worst offenders are not consumer-facing so relying on public pressure will not work.
The fourth part was about practical challenges that are not solved yet in our company and everyone got a list of goals they should achieve by 2030. Removing the plastic entirely was one of the goals.
They actually talked about the number go up problem and how we should lobby for extremely strict sustainability legislation. I'm now personally responsible for implementation of glass or aluminum packaging that should be returned and reused. I even became a bit optimistic about the future, because typically I imagine my kids (if I had any) in 30 years living in a fascist state and their job is manning the machine guns against waves of migrants from Africa where the environment collapsed and crops failed.
implementation of my proposal would still require a minor revolution as the current system will not allow to elect a legislator that will pass these points:
Designate any newspaper, TV/YouTube/telegram channel, blog, Instagram account, etc with more than a 100k monthly views/impressions/followers/whatever as a public faced media.
Make the only allowed ownership form of these media outlets something like worker cooperatives.
Somewhat limit the ownership stakes so that no single member of this cooperative can make all the decisions.
The cooperative should have public financial records so the anyone can see where the revenue comes from.
This still does not protect against a billionaire buying out everyone in the cooperative through donations via multiple shell companies, but it is an improvement.
What I like is that a large media holding can still exist under this scheme, but any corruption would be somewhat apparent.
you guys are hilarious, I used to live somewhere where this system exists and was designed with all these right reasons and now a social media post that conflicts with the state's viewpoint will land you a 10 year sentence.
this seems like an especially likely outcome if designed in the current US political climate.
my point is regulation is not a solution and I don't see any way to overcome this under capitalism. the only thing that might work is some kind of worker-operated cooperative like what the guys at 404 media are doing.
who do you expect is the person deciding what reporting is free and fair? is there a governmental regulatory body, created with the purpose of determining if the reporting was factual? shouldn't the efficiency of this process be improved via pre-approving any media by this regulator to avoid fines?
the problem here does not originate in zfs, this is a hardware failure due to a poorly supported SoC (vendor says PCIe works 'normally' since kernel 7.1, this system ran 6.19) and bad electricity supply
do you ever get a new shiny thing and it looks like it works very nice and coherently and then you dig a bit deeper and realize that it is just a bunch of semi-broken shit held together by glue and duct tape? that's the state of android, that is nicely obscured on pixels by google customizations
haha, no, depending on your use case zfs is mostly set and forget, you just have to get a set of sane defaults, which, at least for me, is way easier and saner than stuff like btrfs. Arch wiki is a good source for that.
scrub is a command that reads data from disks and verifies checksums, it can automatically heal the corrupt data if some redundancy is available. 0 bytes repaired means nothing was repaired, 5 errors mean that there are 5 instances where the recalculated checksums of data did not match the expected values that were calculated when initially written. Normally the command would output 'scrub repaired 0B with 0 errors', that's how you know that the pool is healthy.
shprutzling the cache vdevs sideways is generally required only for obscure optimizations, I've never seen a case where a wrong setting would lead to data loss.
The gotchas come from a single thing: the stupid CDDL licence, which is not compatible with GPL. That means that zfs is not included in linux kernel and most of the normal linux userspace filesystem stuff do not work with zfs.
Also, keep in mind that zfs is generally almost impossible to fuck up, what I've done here is a result of extreme dedication and like 8 years of experience.
I've seen a calculation somewhere for drive speed, size and scrub frequency where optimal number of extra drives was calculated. On fast storage and with weekly scrubs one extra drive should be mostly fine.
My philosophy is any raid is not really a data contingency solution - it just saves some time when the drive fails.
that's no fun. also, it turned out that this is not a disk error, the zfs metadata got corrupted and now I get a panic when trying to mount this.
This is raidz1. Three 4TB NVME drives. I've built this machine as the primary server for services like immich as a fun project that enables me to say that my photo service runs faster than all the commercial stuff and is on ARM64. The downside is that the SoC is not supported by upstream very well so vendor patches had to be applied and there was some instability between that and my shitty electricity supply.
I feel very sorry for you. I have the same problem (panic on mount) with this one, but at least it works when mounting read-only. I've been moving data with zfs send the entire day
my achievement here is that this is a proper raidz setup with multiple disks and ecc ram
I was using a mix of debian servers, arch thinkpads and macos macbooks at the point where I moved everything to NixOS (macbooks did not survive this transition and were donated due to never working properly because of t2 issues).
Nix is extremely hard. I mean, don't get me wrong, initial setup is easy, adding and updating packages is the easiest experience between all package managers out there, nixpkgs is overpowered as it has almost all the software that exists since there are no conflicting dependencies in nix. Getting something that is not in nixpkgs to run is impossible for a beginner (nixpkgs has steam-run to partially solve this, but results may vary). Getting to the point where a tornado may pass through your home and destroy all the hardware inside, and the whole recovery process from that point is about 20 minutes of your time, including the trip to the store for new hardware, requires significant effort, both time and mental, that most of us don't have. But the entire point is that it is possible with Nix!
If Debian is like owning a 90s Corolla, NixOS is like owning a factory that can make any car you like. You can choose to get a 90s Corolla, you can choose to get a newer Corolla, you can skip corollas entirely and make something new and fast. When the new and fast thing breaks, you can make a new car or go back to the Corolla. The Corolla comes completely ready to use by you from the production line. If you need to adjust the mirrors, you just make a new car with the correct mirrors setup. All the cars you make after that will come with the mirrors setup how you like them, because that is now a part of the manufacturing instructions the factory has.
The only caveat is that factory gets updates every 6 months and then your car building pipeline may break when you get the update. In three years I've been using nixos fixing those took about 5 minutes and usually boiled down to stuff like 'previously we had a list of car colors, passed as a string, now the factory expects to get a hex color code'.
Judging by the commit history, I moved most of my machines to NixOS after about 6 months building my config, then, 8 months later I rebuilt the entire thing from scratch when I understood nix enough to make everything truly modular.
I really doubt that once you've embraced NixOS you'd be tempted to use anything else or try LFS. NixOS is beautiful. NixOS just works (after 6 months of setup).
also, machines wipe themselves at each boot! service state data is distributed via zfs send! zfs that actually works!
Profiles are nice and all, but remember, that there are still ways to track you across profiles (serious determination required), like the network stack being mostly shared. Here's the thread.
If you truly need isolated apps, consider getting multiple phones. I personally have three devices for this. For example, my main device (where I actually use the web browser and which is connected to my private network) runs only a limited set of foss apps. This will solve the alarms issue as well as this device has only one profile.
I have a fucking thinkpad that does not follow the USB PD spec!
The thing has only type-c ports, the thing is relatively recent (2023), the thing accepts 140W via type-c, I never bothered to check the actual specs and assumed PD 3.1
Turned out that no, this is not PD, this works only via proprietary power brick that outputs 20v@6.75A, that I didn't get as I already have a bajilion of PD 3.0/3.1 power supplies at home.
::: spoiler back of the device
I work for a company that is one of the top worldwide plastic polluters. We use plastic in packaging. I was in an astonishing series of meetings recently, organized by the global corporate office for legal, R&D and supply chain that went like this:
First, we had a series of trainings about climate change that had none of the typical bullshit. They straight explained that the whole idea of recycling is stupid and shifts focus to the part where it makes the least impact, that carbon offsets don't really work, that we can't rely oil running out because the environment and the economy will collapse way sooner than that will happen.
The second part was I guess appealing to people's emotions by showing devastating effects that humans have caused already, for example there was a picture of starved baby bird that died because the parents were feeding it plastic garbage from the ocean.
The third part was practical plans to make various industries sustainable, the point was that drastic measures are required for most companies and that the worst offenders are not consumer-facing so relying on public pressure will not work.
The fourth part was about practical challenges that are not solved yet in our company and everyone got a list of goals they should achieve by 2030. Removing the plastic entirely was one of the goals.
They actually talked about the number go up problem and how we should lobby for extremely strict sustainability legislation. I'm now personally responsible for implementation of glass or aluminum packaging that should be returned and reused. I even became a bit optimistic about the future, because typically I imagine my kids (if I had any) in 30 years living in a fascist state and their job is manning the machine guns against waves of migrants from Africa where the environment collapsed and crops failed.
Do I understand correctly that there is no way to implement SSO on the self-hosted version?
also there is still a problem with foreign based media outlets, I don't have a way to deal with them at this point
implementation of my proposal would still require a minor revolution as the current system will not allow to elect a legislator that will pass these points:
This still does not protect against a billionaire buying out everyone in the cooperative through donations via multiple shell companies, but it is an improvement. What I like is that a large media holding can still exist under this scheme, but any corruption would be somewhat apparent.
what you are proposing is basically censorship, and state censorship is much easier to consolidate than the current billionaire-based one
you guys are hilarious, I used to live somewhere where this system exists and was designed with all these right reasons and now a social media post that conflicts with the state's viewpoint will land you a 10 year sentence.
this seems like an especially likely outcome if designed in the current US political climate.
my point is regulation is not a solution and I don't see any way to overcome this under capitalism. the only thing that might work is some kind of worker-operated cooperative like what the guys at 404 media are doing.
who do you expect is the person deciding what reporting is free and fair? is there a governmental regulatory body, created with the purpose of determining if the reporting was factual? shouldn't the efficiency of this process be improved via pre-approving any media by this regulator to avoid fines?
I've been hosting grist for a while and it is quite nice. Wasn't able to move all the stuff from classic spreadsheets though