At least 3 major outlets — The New York Times, The Guardian, and Reddit — have blocked the Internet Archive’s Wayback Machine from accessing their content
https://www.mediapost.com/publications/article/412911/not-in-our-back-yard-publishers-block-wayback-mac.html
526 points · 64 comments · view on lemmy.world
64 Comments
tonytins@pawb.social · 141 pts · 174d
Really? They think Internet Archive is the problem?
AmbitiousProcess@piefed.social · 67 pts · 174d
They think AI companies are using it as a "backdoor" to scrape their content. Which is patently ridiculous, but that won't stop them.
Tollana1234567@lemmy.today · 2 pts · 173d
reddit already allows AI to scrape the site, specifically google.
AmbitiousProcess@piefed.social · 2 pts · 172d
*for a fee
If the wayback machine was an actually workable backdoor to Reddit's content, none of these companies would ever have any reason to pay them, and that's what they're scared about.
ohulancutash@feddit.uk · 22 pts · 174d
They think they want their revenue streams
ameancow@lemmy.world · 5 pts · 173d
Tollana1234567@lemmy.today · 2 pts · 173d
its thier excuse to silence dissent, when the time comes to censor overreaching "political trends"
9tr6gyp3@lemmy.world · 87 pts · 174d
Wait until they find out that AI is scraping their web sites.
Blackfeathr@lemmy.world · 18 pts · 174d
Well, Reddit's got a contract for AI companies to scrape their content, so pig boy Spez is getting paid, he don't give a fuck
Fuckfuckmyfuckingass@lemmy.world · 13 pts · 174d
I'm sure they don't care, or are all about it.
ameancow@lemmy.world · 9 pts · 173d
Tollana1234567@lemmy.today · 2 pts · 173d
reddit trying to achieve what FB is doing, meta already has complete control of FB what they push as propaganda.
teslekova@sh.itjust.works · 2 pts · 173d
Ideally, we would have our own personally run AI instances that can give us a probability that what we are reading is LLM generated. It's still pretty good at recognising itself. That will be an arms race, though.
Saryn@lemmy.world · 74 pts · 173d
What an absolute ridiculous thing to say.
ameancow@lemmy.world · 18 pts · 173d
Cantaloupe@lemmy.fedioasis.cc · 13 pts · 173d
Gee, I wonder what else could be scraping content from all these websites?
REDACTED@infosec.pub · 12 pts · 173d
To be fair, the archive indeed got heavily abused into simply reading without paywalls. I know this is a controversial opinion, but seeing comments on other threads like "Remember to support news media", then "use archive to bypass paywalls" then anger towards said companies for caring about getting paid or growing, makes one question where exactly does Lemmy draw the line between pirating and paid content. Or are we simply altogether against sites like 404Media just because of paywalls?
Saryn@lemmy.world · 13 pts · 173d
That's not the point. The point is content scraping (and crawling) is the cornerstone of the contemporary information environment. It's how we got to this technological paradigm in the first place.
This whole "people are bypassing paywalls" is a badly evidenced non-issue, and all too convenient. What these companies are really saying is "Content scraping is bad when others do it. Only I and other big fish get to do it and profit billions out of it. Fuck ordinary citizens. Fuck everyone and everything but me and my dreams of endless wealth and power."
To be fair.
gagcar@lemmus.org · 2 pts · 173d
You say bypassing paywalls is a non-issue, but it is basically the only thing I have heard people say to use it for on social media. You can have your problems about data harvesting, but don’t pretend like getting around paywalls was not what the average individual user was using it for.
CombatWombat@feddit.online · 44 pts · 174d
I’m certain they’ve wanted to do this for a long time, and AI is a convenient way to justify it, rather than admitting they don’t want humans using it to circumvent the paywall. It does solidify for me personally that the LA Times is the paper of record for the United States going forward, rather than the New York Times.
gAlienLifeform@lemmy.world · 15 pts · 173d
The LA Times also blocks the Internet Archive unfortunately. I'd recommend PBS NPR ProPublica or some other nonprofit organization for your US paper of record.
CombatWombat@feddit.online · 7 pts · 173d
Ugh. Thanks for the heads’ up — I’ve definitely posted archive links without noticing they’re blocked before. PBS and NPR have really gone downhill with the budget cuts. ProPublica is great, but their coverage is pretty narrow, so there’s a lot of stories they don’t cover at all. It’s getting harder and harder to find a quality source.
cecinestpasunbot@lemmy.ml · 2 pts · 173d
Unfortunately, I think most quality sources with broad coverage aren’t free. Even the paid sources almost always have a corporate bias. Of those the financial times probably does the least to editorialize. Beyond that I think you just have to find independent journalists or outlets with a narrower investigative focus that you can trust.
hector@lemmy.today · 7 pts · 173d
I just got a gift subscription to the NYTimes, for the first time since I quit in 2018, and it's really gone downhill. I am learning about more big scoops from the guardian from lemmy posts than I see in their paper. I think Israel's final solution for gaza here broke their brain, they had an identity crisis and sided with Israel and fascism over all the fourth estate democracy mumbo jumbo.
They haven't broken a single big story that I recall in the past year. Not a single one, even the wall street journal published epstein's birthday letter from the president. The NYTimes gave up, they are no longer the paper of record, whatever problems before they covered events more thoroughly and had courage to break big stories, and now they don't.
teslekova@sh.itjust.works · 3 pts · 173d
That's actually pretty sad. Also a serious problem for the USA. NYT, for all its faults, really was the best one.
WesternInfidels@feddit.online · 5 pts · 173d
The South African billionaire paper that wouldn't endorse Harris? Well, our options all suck, I guess.
traxex@lemmy.dbzer0.com · 41 pts · 173d
Reminder to donate to the Internet Archive so they can keep fighting the good fight.
green_goglin@thelemmy.club · 35 pts · 173d
Nobody tell NYT about being able to add another “.” Subsequent to”.com” to bypass their paywall.
stegosaur@lemmy.world · 11 pts · 173d
Awesome, this is the best paywall hack I have ever seen!
gAlienLifeform@lemmy.world · 6 pts · 173d
I'm probably screwing it up here, but neither of these are working for me
https://www.nytimes.com.2026/02/04/us/politics/supreme-court-california-congressional-map.html
https://www.nytimes..com/2026/02/04/us/politics/supreme-court-california-congressional-map.html
SocialMediaRefugee@lemmy.world · 8 pts · 173d
Put the extra "." after the ".com" so ".com."
gAlienLifeform@lemmy.world · 8 pts · 173d
Ah, https://www.nytimes.com/./2026/02/04/us/politics/supreme-court-california-congressional-map.html won't work on my usual browser (which just ends up loading NYTs homepage) but it does work in a Chrome incognito window
Thank you!
green_goglin@thelemmy.club · 2 pts · 173d
you’re welcome:
gAlienLifeform@lemmy.world · 4 pts · 173d
I think auto complete or something might have messed with what you intended to post, that link still hits the paywall for me, but using your guidance I was eventually able to figure out that
works in a Chrome incognito window. The "." after "com" and the "/" after that "." are apparently the critical bits
M0oP0o@mander.xyz · 2 pts · 173d
You are missing the .com. part.........
green_goglin@thelemmy.club · 2 pts · 173d
oof I brainfarted - markdown and code auto formats and in doing so autocorrects.
M0oP0o@mander.xyz · 2 pts · 173d
It happens, just wanted to point it out in case people trying it thought it did not actually work
Viking_Hippie@lemmy.dbzer0.com · 1 pts · 173d
Still getting this bullshit:
green_goglin@thelemmy.club · 2 pts · 173d
beep boop
brucethemoose@lemmy.world · 3 pts · 173d
Why does this work? Is it a deliberate bypass on the NYT's part?
green_goglin@thelemmy.club · 3 pts · 173d
No idea, but I love it.
anon_8675309@lemmy.world · 1 pts · 173d
Hmmm. Interesting.
tackleberry@thelemmy.club · 28 pts · 173d
Fuck Reddit. That website has been selling our data and using it to train AI... I say fuck 'em
ameancow@lemmy.world · 10 pts · 173d
tackleberry@thelemmy.club · 4 pts · 173d
great catch! you can actually see the AI slop when it pops up. REddit is dead, and you should delete your data from that cesspool
ameancow@lemmy.world · 5 pts · 173d
Buddahriffic@lemmy.world · 4 pts · 173d
FYI, any data on Lemmy can be used for the same for free. The federation infra can even be used to give AI models more direct access than even reddit is likely giving them. Just in case anyone is assuming that because this is community-run that it means the data isn't being sold. It's not, but it is being accessed by the same entities, if they want it.
brucethemoose@lemmy.world · 6 pts · 173d
The users are still the users though, not the product like Reddit.
Buddahriffic@lemmy.world · 2 pts · 173d
Oh yeah, not saying they are generally equivalent, just in that one particular aspect: access to comment data for any purpose.
brucethemoose@lemmy.world · 3 pts · 173d
Yep.
TBH I think it’s kind of silly for the Fediverse to try and block scraping, as long as that scraping isn’t effectively a DDoS. It’s public.
DarrinBrunner@lemmy.world · 27 pts · 174d
Gotta control the press before you can rewrite history.
M0oP0o@mander.xyz · 24 pts · 173d
I noticed a few days ago when looking into americans leaving loaded firearms in ovens that we are losing archived news. I would find an article or story that is just missing now, all it is is a headline link to no where. And I have seen this trend on all things, we are losing the knowledge and for no other reason then the possibility of an extra dollar at some point. Take this and mix in the overwhelming amount of LLM generated bullshit pretending to be information tailored to peoples perceived interests (if you live in a religious area for example you see more religious bullshit) and we have almost inescapable silos.
I don't think I need to explain how dangerous this is.
AlexLost@lemmy.world · 19 pts · 173d
Remember to support your local library. Only physically written words are going to be safe in the coming age.
M0oP0o@mander.xyz · 6 pts · 173d
As someone that has been on my local library board.... I got bad news for you on that one. Libraries are culling books like never before, facing licensing issues like never before, and funding issues like never before.
Formfiller@lemmy.world · 18 pts · 174d
That’s very 1984 of them
gAlienLifeform@lemmy.world · 14 pts · 173d
Is the Guardian actually blocking the Internet Archive? Seems to work for me
https://web.archive.org/web/20260224104430/https://www.theguardian.com/us-news/2026/feb/23/trump-iran-airstrikes-nuclear-deal
Meanwhile,
https://web.archive.org/web/20260224121247/https://www.mediapost.com/publications/article/413017/ai-basic-training-newsrooms-offer-little-practica.html?initial_article=412911&es_index_start=3&es_index=0
e; huh, Mediapost article did in fact start loading on the Archive a few minutes after I posted this
Takeshidude@lemmy.world · 14 pts · 173d
Start self-hosting archive box They cant block everyone
fierysparrow89@lemmy.world · 4 pts · 173d
If you have a concrete suggestion as to the stack don't hold back 😃
Liketearsinrain@lemmy.ml · 2 pts · 172d
user314_lemmus_v3s@lemmy.world · 13 pts · 174d
I wander what happened to Archive in 2024 when it was "hacked" and some pages "disappeared"...
lechekaflan@lemmy.world · 9 pts · 173d
Fuck you spez
SpicyLizards@reddthat.com · 7 pts · 174d
Buuuut they all say that we need to donate to save free speech! It can't be a lie right?
Mainly pointing at the guardian here as they are sliding down the same slope that the other two slops did.
FishFace@piefed.social · 4 pts · 174d
By "donate" you mean "buy a subscription"?
turdburglar@piefed.social · 6 pts · 174d
that’s fuken lame.
LibertyLizard@slrpnk.net · 5 pts · 174d
Should be fairly easy to defeat, no?
AnchoriteMagus@lemmy.world · 3 pts · 174d
Is this related to the owner of those multiple archive sites being outed as a Russian DDoS perpetrator?
KickMeElmo@sopuli.xyz · 39 pts · 174d
Nope. Different org, unrelated rationale.
Zerot@fedia.io · 22 pts · 174d
No. This is a different archive site
DaMummy@hilariouschaos.com · 8 pts · 174d
I'm sure it's more to do with archive bypassing their advertising and subscription.
Aritoteles@lemmy.world · -4 pts · 173d
is this a meme? I don't get the joke.