Warning: Some posts on this platform may contain adult material intended for mature audiences only. Viewer discretion is advised. By clicking ‘Continue’, you confirm that you are 18 years or older and consent to viewing explicit content.
This makes me so happy. I’ve been wanting to start a project to scrape fandom for a while so I can just have the info without accessing their godawful website. Any time I need info on something in a game I get hit with 5 results from fandom and some half-related articles in the search results.
As someone who has tried that effort to do scraping from Fandom, it’s certainly an experience - the HTML is truly nightmarish and definitely written in a way to make scraping as hard as possible (and potentially even programmatically obfuscated, too)
This makes me so happy. I’ve been wanting to start a project to scrape fandom for a while so I can just have the info without accessing their godawful website. Any time I need info on something in a game I get hit with 5 results from fandom and some half-related articles in the search results.
As someone who has tried that effort to do scraping from Fandom, it’s certainly an experience - the HTML is truly nightmarish and definitely written in a way to make scraping as hard as possible (and potentially even programmatically obfuscated, too)
Thanks for the heads up! I was worried it would be pretty far out of my skillset, but still thought it would be fun to give it a shot.