Is it possible to de-duplicate lemmy posts from other instances?

It’s annoying seeing the same headlines numerous times linking to same stuff because it’s cross posted to 3+ instances.

Is there a setting to reduce that or app that handles that well?

69 points · 14 comments · view on lemmy.world

14 Comments

MentalEdge@sopuli.xyz · 26 pts · 2y (11 replies)

The default webUI de-duplicates cross-posts. I don't know if any apps do.

sbv@sh.itjust.works · 13 pts · 2y (8 replies)

They have to be actual cross posts though, don't they? Would that catch the same link spammed across multiple instances and communities?

MentalEdge@sopuli.xyz · 9 pts · 2y (4 replies)

There are no "actual" cross posts. Cross-posts on Lemmy are just all posts that have the exact same URL in the URL field.

muntedcrocodile@lemm.ee · 2 pts · 2y (3 replies)

So if I post a URL to one community and post a URL of that post in another what's that?

MentalEdge@sopuli.xyz · 2 pts · 2y (2 replies)

Not a cross post. It just creates a post that links to the first post.

If you made a third post that also linked to the first post using the same url as the second post, that would be detected and get linked up as a cross-post to the second post.

Actual cross-posts work in both directions. Lemmy detects them as the same thing, both posts link to the other.

Not in the post body, the post body quote and "cross-posted from" text is completely meaningless and just gets added by most clients, adding it doesn't make something a cross-post, and removing it doesn't break the actual cross-post feature, which shows up as a little "cross-posted to: list of communities" thing on the post page.

Thunder has it, Photon does it, and the default webUI does it.

muntedcrocodile@lemm.ee · 2 pts · 2y (1 reply)

So does that mean shared comments? Or are they separate? And yeah I'm on eternity hangover from the Reddit days.

MentalEdge@sopuli.xyz · 1 pts · 2y

No, the posts are separate. But each post will list every other post so you can visit the other comment sections.

andrew_s@piefed.social · 4 pts · 2y (2 replies)

Yeah - I think anything the UI is doing, it's getting the info from the API, so the poster would've had to use the 'cross-post' feature. There are some apps (e.g. Voyager) that try to wrangle cross-posts by title or URL, but title-matching can give false postives, and URL-matching usually assumes that one link hasn't picked up some cruft, and it can't do much for uploaded images if the poster didn't cross-post (because it'll be 2 different files with different URLs)

MentalEdge@sopuli.xyz · 5 pts · 2y (1 reply)

Different people posting the same URL get their posts linked up as cross posts.

The cross post button just copies the post text into a new post. It doesn't do anything special.

andrew_s@piefed.social · 2 pts · 2y

Oh, right. Sorry. I've gone back and checked the post I was thinking of when I made that comment, and - yeah - it turns out I was misremembering / didn't properly investigate the first time.

Aquila@sh.itjust.works · 7 pts · 2y (1 reply)

News to me! Good to know. Will have to see how that’s handled and maybe make pr to my favorite apps

can@sh.itjust.works · 2 pts · 2y

If you find any with it let me know. Not hopeful for Sync.

thesporkeffect@lemmy.world · 9 pts · 2y

There ought to be a way to tag a single post for multiple communities

Crackhappy@lemmy.world · 1 pts · 2y

I just block the users who do it. Requires a few seconds and then I don't worry about it again.