• ClamDrinker@lemmy.world
      link
      fedilink
      arrow-up
      3
      ·
      edit-2
      14 days ago

      I hate to ruin this for you, but if you post nonsense, it will get downvoted by humans and excluded from any data set (or included as examples of what to avoid). If it’s not nonsensical enough to be downvoted, it still won’t do well vote wise, and will not realistically poison any data. And if it’s upvoted… it just might be good data. That is why Reddit’s data is valuable to Google. It basically has a built in system for identifying ‘bad’ data.

      • leftzero@lemmynsfw.com
        link
        fedilink
        arrow-up
        1
        ·
        13 days ago

        No, you’re missing the point. You make up some credible misinformation to poison AI training with, but you don’t stop there: you get an LLM to rewrite it for you. Retry until you get a text that sounds credible, doesn’t particularly look written by AI, and people will upvote, and post that.

        With this, even if the text looks good, you’re not only poisoning future models with the misinformation you started with; by feeding them a text generated by an LLM (even if you can’t tell the difference at first glance) you’re introducing feedback into the model that will also poison it, not with misinformation, but by reinforcing its biases and errors and reducing the variety of its training data.