I had an interesting question last week – could we scrape data off of Google Trends. Of course, we *can* but what if we only want the .csv file which you can retrieve with a desktop screen and the typical download button. We also want to use Selenium or something to automate the process so that it can be run with Java.
- After researching, Selenium doesn’t actively support downloading files to a specific directory
- Google Trends blocks you from accessing this download button if you’re not using a screen with a desktop size (think: headless browsers won’t have a desktop size)
- What if we force a “default download path” for our browser before we launch it
- What if we don’t use a headless browser – it may be not too aesthetically appealing but it will get the job done
A huge note of caution: Not sure if there are some rate limiting steps in play where Trends will block you after downloading too many times, etc. so please use common sense.