We all migrate to smaller websites try not to post outside drawing attention just to hide from the "Ai" crawlers. The internet seems dead except for the few pockets we each know existed away from the clankers
We all migrate to smaller websites try not to post outside drawing attention just to hide from the "Ai" crawlers. The internet seems dead except for the few pockets we each know existed away from the clankers
93 Comments
daniskarma@lemmy.dbzer0.com · 220 pts · 299d
I have a testing website. I have never gave the address to absolutely anyone, ever. It's not linked with anything. It's just a silly html site living in a domain.
It's still being ping and probed to death by bad actors. No necessarily AI scrappers. But it's dozens or hundreds of http petitions a day for random places all over the world.
There's no black forest. It's all light up and under constant attack, every tree is already on fire.
dual_sport_dork@lemmy.world · 110 pts · 299d
That's because it's numerically possible to sweep through the entire IPv4 address range fairly trivially, especially if you do it in parallel with some kind of botnet, proverbially jiggling the digital door handles of every server in the world to see if any of them happen to be unlocked.
One wonders if switching to purely IPv6 will forestall this somewhat, as the number space is multiple orders of magnitude larger. That's only security through obscurity, though, and it's certain the bots will still find you eventually. Plus, if you have a doman name the attackers already know where you are — they can just look up your DNS record, which is what DNS records are for.
MyNameIsIgglePiggle@sh.itjust.works · 19 pts · 299d
I like seeing them try and then thinking "begone thot! There is no entry for you"
In fact, I might make a honeypot that issues exactly that
WhyJiffie@sh.itjust.works · 5 pts · 299d
nepenthes is the tool for that
early_riser@lemmy.world · 1 pts · 285d
kossa@feddit.org · 13 pts · 299d
But an IP can have multiple websites and even not return anything on plain IP access. How do crawlers find out about domains and unlinked subdomains? Do they even?
chamomile@furry.engineer · 19 pts · 299d
@kossa @dual_sport_dork If you're using HTTPS, which is by and large the norm nowadays, then every domain is going to be trivially discoverable via certificate transparency logs: https://social.cryptography.dog/@ansuz/115592837662781553
taaz@biglemmowski.win · 13 pts · 299d
thinking about this, wouldn't the best way to hide a modern websie be something along getting a wildcard domain cert (can be done with LE with DNS challenge), cnaming the wildcard to the root domain and then hosting the website on a random subdomain string ? am I missing something
confusedpuppy@lemmy.dbzer0.com · 10 pts · 299d
I do something something like this using wildcard certs with Let's Encrypt. Except I go one step further because my ISP blocks incoming data on common ports so I end up using an uncommon port as well.
I'm not hosting anything important and I don't need to always access to it, it's mostly just for fun for myself.
Accessing my site ends up looking like
https://randomsubdomain.registered-domain-name.com:4444/My logs only ever show my own activity. I'm sure there are downsides to using uncommon ports but I mitigate that by adjusting my personal life to not caring about being connected to my stuff at all times.
I get to have my little hobby in my own corner of the internet without the worry of bots or AI.
MrPoopyButthole@lemmy.world · 1 pts · 299d
Thanks for the link!
simeon@reddthat.com · 4 pts · 299d
Every SSL certificate is publicly logged(you can see these logs e. g. under crt.sh) and you might be able to read DNS records to find new (sub)domains. The modern internet is too focused on being discoverable and transparent to make hiding an entire service(domain + servers) feasible. But things like example.com/dhusvsuahavag8wjwhsusiajaosbsh are entirely unfindable as long as they are not linked to
kossa@feddit.org · 5 pts · 299d
Random subdomain on wildcard certificate, IP written in the host file to mitigate DNS records, only given by word-to-mouth 😅.
Nobody said the uncrawled dark forest would be comfortable.
kazaika@lemmy.world · 9 pts · 299d
Servers which are meant to be secure usually are configured to not react to pings and do not give out failure responses to unauthenticated requests. This should be viable for a authenticated only walled garden type website op is suggesting, no?
dual_sport_dork@lemmy.world · 1 pts · 299d
There are several things you could do in that regard, I'm sure. Configure your services to listen only on weird ports, disable ICMP pings, jigger your scripts to return timeouts instead of error messages... Many of which might make your own life difficult, as well.
All of these are also completely counterproductive if you want your hosted service, whatever it is, to be accessible to others. Or maybe not, if you don't. The point is, the bots don't have to find every single web service and site with 100% accuracy. The hackers only have to get lucky once and stumble their way into e.g. someone's unsecured web host where they can push more malware, or a pile of files they can encrypt and demand a ransom, or personal information they can steal, or content they can scrape with their dumb AI, or whatever. But they can keep on trying until the sun burns out basically for free, and you have to stay lucky and under the radar forever.
In my case just to name an example I kind of need my site to be accessible to the public at large if I want to, er, actually make any sales.
Cooper8@feddit.online · 1 pts · 299d
I have suggested a couple of times now that ActivityPub should implement an encryption layer for user authentication of requests and pings. It already has a system for instances vauching for each other. The situation is that users of "walled garden" instances in ActivityPub lack means of interfacing with public facing instances that doesnt leave the network open for scraping. I believe a pivot towards default registered users only content service built on encrypted handshakes, with the ability for servers to opt-in to serving content to unregistered users would make the whole network much more robust and less dependent on third party contingencies like CloudFlare.
Then again, maybe I should just be looking for a different network, I'm sure there are services in the blockchain/cryptosphere that take that approach, I just would rather participate in a network built on commons rather than financialization at it's core. Where is the protocol doing both hardened network and distributed volunteer instances?
SkyeStarfall@lemmy.blahaj.zone · 5 pts · 299d
It's not as simple as "only security through obscurity". You could say the same thing for an encryption key of a certain length. The private key to a public key is still technically just an obscurity, but it's still impractical to actually go through the entire range
IPv6 is big enough where this obscurity becomes impractical to sweep. But of course, as you said, there may be other methods of finding your address
lauha@lemmy.world · 2 pts · 299d
I love your "multiple orders of magnitude". I don't think you appreciate or realise how much larger ipv6 address space is :)
dual_sport_dork@lemmy.world · 8 pts · 299d
I wasn't going to type that many commas for the sake of brevity, but it's 340,282,366,920,938,463,463,374,607,431,768,211,456 possible addresses. I.e. 2^128^. So yes, I do.
I consider 96 orders (in binary, anyway) as "multiple." Wouldn't you?
lauha@lemmy.world · 0 pts · 299d
No need to be defensive. I'm not insulting, I just find it funny :) usually people call that "dozens". But dozens of orders of magnitude really doesn't give the sense of scale.
You could have 8 billion in habitants in every 10^24 stars in the universe and everyone could still have 42k addresses.
muntedcrocodile@hilariouschaos.com · 8 pts · 299d
It it had a DNS its obviously lit up like a Christmas tree. If its ipv4 its on shodan. The only way to hide is to have no DNS and an ipv6.
Aussiemandeus@aussie.zone · 7 pts · 299d
Do you know how they find it? Is it just random input of address over and over?
dual_sport_dork@lemmy.world · 43 pts · 299d
Almost certainly. There are only 4,294,967,296 possible IPv4 addresses, i.e. 4.3ish billion, which sounds like a lot but in computer terms really isn't. You can scan them in parallel, and if you're an advanced script kiddie you could even exclude ranges that you know belong to unexciting organizations like Google and Microsoft, which are probably not worth spending your time messing with.
If you had a botnet of 8,000 or so devices and employed a probably unrealistically generous timeout of 15 seconds, i.e. four attempts per minute per device, you could scan the entire IPv4 range in just a hair over 93 days and that's before excluding any known pointless address blocks. If you only spent a second on each ping you could do it in about six days.
For the sake of argument, cybercriminals are already operating botnets with upwards of 100,000 compromised machines doing their bidding. That bidding could well be (and probably is) probing random web servers for vulnerabilities. The largest confirmed botnet was the 911 S5 which contained about 19 million devices.
Melobol@lemmy.ml · 16 pts · 299d
That's amazing and scary at the same time. Thanks for putting it into perspective!
pipe01@programming.dev · 11 pts · 299d
I don't know exactly how they do it, but probing every ipv4 address isn't that hard
kossa@feddit.org · 3 pts · 299d
But there can be multiple websites behind one IP address?! They would not show when onhy accessing the IP. They would need to know about the domains somehow.
friend_of_satan@lemmy.world · 7 pts · 299d
RememberTheApollo_@lemmy.world · 6 pts · 299d
I have a DDNS setup. Pretty random site name. Nonetheless, it’s been found and constantly probed. Lots of stuff from Russia, China, a few countries in Africa, and India. A smattering of others, but those are the constant IPs that are probing or attempting logins.
Croquette@sh.itjust.works · 4 pts · 299d
DNS only translate a string address (www.mywebsite.com) to its IP address (xxx.xxx.xxx.xxx) so that it is easier to remember.
Bots just try a range of address and they don't need to know your domain name. You could have the most unintelligible domain name in the world, bots would still ping your website because they use direct IP addresses.
RememberTheApollo_@lemmy.world · 2 pts · 299d
Yeah, that’s probably it. Just them spamming different numeric IP addresses to see if any get a hit.
taaz@biglemmowski.win · 4 pts · 299d
crt.sh and certificate transparency
Taleya@aussie.zone · 3 pts · 297d
I have domain rather the same and it's frankly hilarious watching people try IIS and PHP exploits over and over. That box technically doesn't even qualify as a LAMP, it's a LA
crandlecan@mander.xyz · 48 pts · 299d
Fabulous insight. I think that would make me very happy. Bring back the forests! Burn down the Nazi trees!
Carnelian@lemmy.world · 6 pts · 299d
That’s not just a fabulous insight, it’s a powerful revelation!
SaharaMaleikuhm@feddit.org · 40 pts · 299d
How about just living in the actual woods with no internet? Gets more tempting by the day.
Aussiemandeus@aussie.zone · 12 pts · 299d
Yeah but where i live its to damn hot
PokerChips@programming.dev · 3 pts · 298d
Actually, there's no woods left.
TronBronson@lemmy.world · 4 pts · 298d
Seriously, I live in the woods and people keep moving in and clear cutting. The woods are just suburbia after covid lol
Taleya@aussie.zone · 1 pts · 297d
It's all fun and games until bushfire season
TronBronson@lemmy.world · 2 pts · 297d
I’m on a hill so I’m waiting for the rains to pick up and carve their house into a gully. It’s basically a future stream now
TheBat@lemmy.world · 0 pts · 298d
kazaika@lemmy.world · 31 pts · 299d
Is this hopeposting ?
Aussiemandeus@aussie.zone · 7 pts · 299d
Kinda yeah, it's what I thought lemmy would be, but more and more it isn't
JoMiran@lemmy.ml · 27 pts · 299d
Cyberpunk as a literary genre, and the Cyberpunk TTRPG in specific, are incredibly prophetic. In the Cyberpunk TTRPG (which predates the WWW), "the net" is eventually condemned (as in boarded up) because of AI and ia replaced by silo'd networks (think extended intranets).
Cooper8@feddit.online · 11 pts · 299d
And of course in Cyberpunk the ttrpg setting much of the o0en internet was rendered useless by self replicating AI malware hijacking storage, processing, and bandwidth due to a zero day exploit discovered by one egomaniacal hacker.
Geobloke@aussie.zone · 5 pts · 299d
Sounds a bit like computers in Dune as well
ipkpjersi@lemmy.ml · 23 pts · 299d
Well I mean that's kind of what Lemmy is like since it's far more niche than something like reddit, but AI crawlers will find it anyway.
mic_check_one_two@lemmy.dbzer0.com · 14 pts · 299d
AI crawlers don’t even need to crawl individual instances. If someone wanted to scrape Lemmy, it would be way more efficient to simply spin up their own instance and let federation do its thing. Federation is literally a built in way to mass distribute content to a bunch of different servers. So just spin up an instance, set it to not respect delete requests, (so you still get the deleted posts and comments), and scrape it locally. The entire thing could be set up in like 20 minutes, and it would allow for passive data collection instead of requiring active scrapers that run constantly.
phoenixz@lemmy.ca · 5 pts · 298d
I'm sure they'll use instance killing AI scrapers anyway because they don't give a shit.
pressanykeynow@lemmy.world · 3 pts · 298d
Efficiency is not what AI is known for.
MarriedCavelady50@lemmy.ml · 13 pts · 299d
tomcatt360@lemmy.zip · 4 pts · 299d
And don't forget freenet!
Hackworth@piefed.ca · 12 pts · 299d
AI-orchestrated cyber espionage campaign
Cooper8@feddit.online · 4 pts · 299d
Damn, how was this not big headline news?
SaharaMaleikuhm@feddit.org · 6 pts · 299d
China and Russia do this shit all the time. Now they used a new tool. It's really not big headline news. I'd love for it to be and burst that AI bubble tho.
DeathByBigSad@sh.itjust.works · 11 pts · 299d
shhhthey'llhearyou!FUCK WE'RE TOO LATE, YOU ACTIVATED THE BOTS! YOU DOOMED US!
Aussiemandeus@aussie.zone · 5 pts · 299d
My bad, I'm sorry
Brkdncr@lemmy.world · 11 pts · 299d
Back in the days of dial up and bbs this was a problem but you would still get robots trying to connect to modems by dialing every phone number possible.
friend_of_satan@lemmy.world · 3 pts · 299d
Soggy@lemmy.world · 9 pts · 298d
It's almost time, we're almost back to web-rings.
coronach@lemmy.sdf.org · 2 pts · 298d
https://sr.ht/~sircmpwn/openring/
Already there!
SanctimoniousApe@lemmings.world · 8 pts · 299d
Isn't that what TOR-based .onion sites are for?
Aussiemandeus@aussie.zone · 1 pts · 299d
Unfortunately I've no idea
sukhmel@programming.dev · 2 pts · 298d
That's the point
Aussiemandeus@aussie.zone · 2 pts · 298d
Perfect that's what I need
GuyFawkes@sh.itjust.works · 7 pts · 298d
Morpheus, that you?
TheWeirdestCunt@lemmy.today · 7 pts · 299d
Ah a fellow spacetime enjoyer
Aussiemandeus@aussie.zone · 1 pts · 299d
Space time?
Someone else had this idea before me? Like they say no new ideas haha
TheWeirdestCunt@lemmy.today · 3 pts · 299d
Tbh I mixed up a couple videos I saw in my feed, I was watching a new PBS space time video and saw an old cool world's lab video on the dark forest hypothesis in the side bar.
JohnnyCanuck@lemmy.ca · 7 pts · 299d
Too late, it's dead!
Aussiemandeus@aussie.zone · 4 pts · 299d
Yeah it certainly is
MidsizedSedan@lemmy.world · 6 pts · 298d
I would prefer a smaller HUMAN internet, over a bigger AI internet.
Aussiemandeus@aussie.zone · 2 pts · 298d
Absolutely
BodePlotHole@lemmy.world · 6 pts · 299d
I was thinking the other week about how it's getting to a point that I would consider a membership fee to access something like lemmy but guaranteed no AI or bots or bullshit advertising.
I know it isn't possible, but if it was, I'd pay a small fee to have it.
NihilsineNefas@slrpnk.net · 6 pts · 299d
Do you think there will be safe places on the internet?
If it's connected, it's accessible. Won't matter what human level security we put in place when the datacenters these clankers run on have enough GPUs to brute force their way through.
Offline communication will make a resurgence, and will become indespensible when the resource wars the billionaires are funding reach the rest of the world.
Quadrexium@sopuli.xyz · 12 pts · 299d
If i had to guess, maybe everything would become invite-only
pcr3@lemmy.world · 3 pts · 299d
So like past torrent search providers, ike demoniod?
Lag@lemmy.world · 6 pts · 299d
If the person who got invited by the person you invited gets banned, your whole family dies. It's the only way to keep people honest.
Aussiemandeus@aussie.zone · 3 pts · 298d
Internet vampires
bampop@lemmy.world · 1 pts · 298d
You know, maybe that's not so bad, there would be real world links between users rather than the random collection of absolute strangers you get now. Oh wait no that's just facebook again
Siegfried@lemmy.world · 5 pts · 299d
The last reduct of mankind against the machines? Let's call it Sion
Aussiemandeus@aussie.zone · 4 pts · 299d
I think you're on to something maybe make a movie
TropicalDingdong@lemmy.world · 5 pts · 299d
peaks out from behind tree
Aussiemandeus@aussie.zone · 1 pts · 299d
PearOfJudes@lemmy.ml · 5 pts · 299d
I like this metaphor
Aussiemandeus@aussie.zone · 1 pts · 298d
Thank you
minorkeys@lemmy.world · 4 pts · 299d
Disconnection is the only solution, walled gardens, paid or by invite, that prevent all the shit corporate America fills the commons with.
Aussiemandeus@aussie.zone · 2 pts · 299d
Paid subs is no good but invite only is a good idea but how do you distinguish invite to a person compared to invite to a bot
vatlark@lemmy.world · 2 pts · 299d
This is something I'm very curious about. It seems like a really necessary utility in the future.
A way for people to validate other people but not totally blow away all privacy. Large group chats, email providers, etc already try to solve it. It would be cool to see some powerful open source tooling. Like what Signal is to E2E encrypted chats.
minorkeys@lemmy.world · 1 pts · 299d
You only invite those you know, personally.
Aussiemandeus@aussie.zone · 1 pts · 298d
Just go outside at that point
minorkeys@lemmy.world · 1 pts · 298d
I think you can know them personally from online. Ppl you game with or fellow content creators or people in the same hobby spaces etc.
Cooper8@feddit.online · 4 pts · 299d
How is Gemini fairing in the existing bot landscape? Usenet?
Fmstrat@lemmy.world · 3 pts · 296d
There are plenty of alternative Internets. And even just skipping the HTTP protocol is a start. IRC is a great example.
sem@piefed.blahaj.zone · 2 pts · 285d
abbiistabbii@lemmy.blahaj.zone · 2 pts · 298d
Already happened, but the AI companies will try their best to kill that off too.
Taleya@aussie.zone · 2 pts · 297d
They don't want to kill it...they're just incapable of invention and innovation, so they push and crowd like parasite universes on the edge of humanity. Like the sea trying to warm itself around a candle.
early_riser@lemmy.world · 2 pts · 285d
Quazatron@lemmy.world · 1 pts · 296d
I'm ready for the return to webrings.
noisypine@infosec.pub · 0 pts · 298d
Some sort of F2F network like Retroshare would solve the problem.