Colleagues, Might one, two, or more of you have advice on software for web scraping? My particular and immediate interest is in one-time scraping of modest amounts of text data from government websites in the United States. Of course, if I like this hammer.. I have found only brief anecdotal comments about how to do it in political science methods papers, and even in papers that obviously have carried out such a task. Much software of this ilk is touted on the web, but rare is the online paper that reviews more than one piece of software. And the most useful, if still limited, papers of the latter sort that I have found are at: http://lethain.com/an-introduction-to-compassionate-screenscraping/ and http://www.notprovided.eu/six-tools-web-scraping-use-data-journalism-creatin g-insightful-content/ From having read those sources and the text on a number of different software options, I am led to believe that such software varies by how "user friendly" it is, how technically sophisticated it can be, and how much user specified programming is necessary to employ a given software package. Thus I, and perhaps other readers of this list might appreciate advice from those of you who have used such software about several options across the range of those attributes. Kim Kim Quaile Hill Cullen-McFadden Professor of Political Science, Presidential Professor for Teaching Excellence, and Eppright Professor in UndergraduateTeaching Excellence Department of Political Science 4348 TAMU Texas A&M University College Station, TX 77843-4348 ph. 979/845-8235 fax 979/847-8924 e-mail: <mailto:[log in to unmask]> [log in to unmask] ********************************************************** Political Methodology E-Mail List Editors: Ethan Porter <[log in to unmask]> Gregory Whitfield <[log in to unmask]> ********************************************************** Send messages to [log in to unmask] To join the list, cancel your subscription, or modify your subscription settings visit: http://polmeth.wustl.edu/polmeth.php **********************************************************