This is very concerning. DOJ has stated explicitly that any CSAM was removed before releasing the files. Should I remove the magnet link to the merged Data Set 9 torrent?
I haven't looked inside any of these sets myself. My primary goal has been to get the DOJ data distributed.
Amazing, thank you. That was my thought, check hashes while merging the files to keep any copies that might have been modified by DOJ and discard duplicates even if the duplicates have different metadata, e.g. timestamps.
I've added all magnet links for sets 1-8 to the original post. Magnet links for 9-11 match OP. Magnet link for 12 is different, but we've identified that there are at least two versions. DOJ removed files before the second version was downloaded. OP contains the early version of data set 12.
The flashing banner at the top says that it includes 101GB of Data Set 9. Unfortunately, I think they just grabbed the larger of the two torrents.
This is very concerning. DOJ has stated explicitly that any CSAM was removed before releasing the files. Should I remove the magnet link to the merged Data Set 9 torrent?
I haven't looked inside any of these sets myself. My primary goal has been to get the DOJ data distributed.
Amazing, thank you. That was my thought, check hashes while merging the files to keep any copies that might have been modified by DOJ and discard duplicates even if the duplicates have different metadata, e.g. timestamps.
Thank you for this!
I've added all magnet links for sets 1-8 to the original post. Magnet links for 9-11 match OP. Magnet link for 12 is different, but we've identified that there are at least two versions. DOJ removed files before the second version was downloaded. OP contains the early version of data set 12.
When merging versions of Data Set 9, is there any risk of loss with simply using
rsync --checksumto dump all files into one directory?