Agenda

  1. What is Web scraping?
    1. http://blog.screen-scraper.com/2008/04/21/screening-scraping-ethics/
    2. http://onlinejournalismblog.com/2013/09/18/ethics-in-data-journalism-mass-data-gathering-scraping-foi-and-deception/
  2. Scrape a website w/out programming
    1. Download data set and Scraper extension: Dataset | Scraper
      1. Learn how to make a list into a spreadsheet
    2. Go over parts of a webpage, how they work together.
    3. Introduce import.io with above dataset, a more flexible option.
    4. More advanced techniques with Outwit Hub, using same data set
      1. Pull specific links/images
      2. Use knowledge of how the Web works to write more complex before/after scraper. – http://michelleminkoff.com/web-scraping-without-programming-nicar-2012-hands-on-tutorial/
  3. Scrape a website with programming: Dataset (We may hit all/part of this section on Thursday)
    1. Download Ruby (If you are on a Mac, just go to the Terminal and hit irb. If Windows, download this: http://dl.bintray.com/oneclick/rubyinstaller/rubyinstaller-2.2.2.exe)
    2. Download Nokogiri. In the command prompt, “gem install nokogiri”
    3. Grab the file I created, and run it sucessfully: https://gist.github.com/michelleminkoff/7e934e88bf958496c10e
    4. Comment out/remove various parts of the file and talk about what makes it work.

Homework

  1. For this week, basically readings.
    1. Ethics of scraping – http://onlinejournalismblog.com/2013/09/18/ethics-in-data-journalism-mass-data-gathering-scraping-foi-and-deception/
    2. Getting data from the Web – challenges – http://datajournalismhandbook.org/1.0/en/getting_data_3.html
    3. Role of visualization in finding story in data –http://datajournalismhandbook.org/1.0/en/understanding_data_7.html
    4. Directory of visualization types – http://guides.library.duke.edu/vis_types
    5. Visual math mistakes – https://eagereyes.org/criticism/visual-math-wrong
    6. Stacked area chart vs. line chart – http://vizwiz.blogspot.com/2012/10/stacked-area-chart-vs-line-chart-great.html
Advertisements