Reviewing the question How to scrape rugby union data from espn.co.uk, should we allow web scraping question. This is a wide topic ranging from:
- technology used for scraping
- navigation and selection technique to get the data
- how to avoid anti-bot (scraping) defense from the website
Often web scraping is the last resort when open data is not available. It might be good too to discuss the legality of the technique.
This question is related to Are data extraction techniques on-topic? which was more focused on PDF format.