Announcing the Data.gov Archive

https://lil.law.harvard.edu/blog/2025/02/06/announcing-data-gov-archive/

366 points · 17 comments · view on lemmy.world

17 Comments

just_another_person@lemmy.world · 62 pts · 1y (3 replies)

At least private institutions are stepping up. Where's the rest of the universities?

NaibofTabr@infosec.pub · 29 pts · 1y (1 reply)

Dependent on federal money.

corsicanguppy@lemmy.ca · 7 pts · 1y

... Thus, not saying a thing if they do actually cache it.

rbos@lemmy.ca · 12 pts · 1y

I work for a Canadian university library and I assure you that data archival efforts are proceeding apace. I can't really say more without possibly endangering American colleagues.

Bakkoda@sh.itjust.works · 58 pts · 1y (2 replies)

I'll officially be seeding more of this type of stuff (annas, cdc, etc) than all of my "Linux isos". Feels good.

Tikiporch@lemmy.world · 5 pts · 1y (1 reply)

Did they create a torrent for this, or just straight downloads?

finicky_foyer@sh.itjust.works · 5 pts · 1y

It appears to be hosted via S3 compatible object store. Their docs provide examples to browse/download via awscli or boto3 (as well as rclone).

https://source.coop/repositories/harvard-lil/gov-data/description https://source.coop/repositories/harvard-lil/gov-data/access

errer@lemmy.world · 33 pts · 1y (1 reply)

16TB is within the capacity of individuals. Given I don’t trust even Harvard to be safe with the impending onslaught, anyone with the means of downloading a copy for themselves should do so.

Xanza@lemm.ee · 6 pts · 1y

Cold storage is $2 per TB/mo. Could easily be crowdfunded. $410/yr to keep it around.

tastysaganaki@reddthat.com · 19 pts · 1y

🙏

tonytins@pawb.social · 5 pts · 1y

Thank goodness. I was hoping for something like this.

NudeNewt@lemm.ee · 3 pts · 1y (5 replies)

Remember folks, to avoid Linux ISO honeypots you must enable Super-Seeding.

Xanza@lemm.ee · 2 pts · 1y (4 replies)

Is there a list of super-seeding complaint clients? I can't seem to find one. Hell, I can't even check to see if my client supports it.

NudeNewt@lemm.ee · 1 pts · 1y (3 replies)

Most p2p clients support it, you'll need to refer to their documentation to enable.

For the most part all you need to do to enable it is right click a completed download then select "super seeding" mode. It's at least that straightforward for QBT.

Xanza@lemm.ee · 2 pts · 1y (2 replies)

I never knew. My version of Deluge is 1.3.5, and super seeding requires 2.0+. So that looks like the issue. lol

NudeNewt@lemm.ee · 2 pts · 1y (1 reply)

Hmm, in that case I recommend you read into "Bandwidth Tweaking":

https://deluge-torrent.org/userguide/bandwidthtweaking/

Xanza@lemm.ee · 2 pts · 1y

I'm gonna look into upgrading, but I've implemented this as well. Thanks.