Hello [email protected]

I intend to upload an archive of a website I scraped for the last few weeks. The torrent is about 4 TB large and contains 400000 folders which in turn contain 40 .jpg’s on average (plus some metadata).

Should I just create the torrent with the files as they are right now, or should I put the individual folders in archives (or maybe even the entire torrent?)?

Thanks in advance

  • cesium@sh.itjust.worksOP
    link
    fedilink
    English
    arrow-up
    6
    ·
    1 year ago

    Makes sense. What about individual folders? I’m concerned that the massive amount of small files could slow down the download quite a bit. Unfortunately, I’m not too familiar with the BitTorrent protocol, so I don’t know if this would have an impact.

    • zodo123@lemmy.world
      link
      fedilink
      English
      arrow-up
      14
      ·
      1 year ago

      Small files don’t impact BitTorrent transfers. They’re sent in pieces of a fixed size, and those pieces can contain multiple files.

    • fkn@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 year ago

      Depends on final size of the folder zips. 20-30mb? Sure. 200-300? Nah.