WB still excellent web scraper

Started by stanl, May 20, 2017, 10:15:42 AM

Previous topic - Next topic

stanl

Need to put Tony in a better mood >:(

I took on a task to get a data field's (numeric code) description ( multi line text) from a web url, where you must input 1 numeric code and the results pop out as a javascript. Having not done this in 4 years javascript urls get pretty sophisticated...  this one blocked forward and backward caching, and all of the hrefs pointed back to the same menu. I was looking at a Python add-in and script or working with Firefox, or trying to figure out the jquery/AJAX stuff in the site. I then noticed that in spite of all the sophistication, the basic navigation was BaseURL: \Search\ : input number.  This added the secret sauce to the hrefs and by re-navigating to the results from the search I got what I needed.  Basically 2 navigates with basic WB code - in 20 lines of code.

I have been with WB since 1998. It never fails in a pinch. I'm too old to be trying to replace it with Python, Ruby or Visual Studio - I have worked with them but I don't have the flow and would not be competitive. Scripting and WB in particular helped pay off our small house and pay off our daughter's Duke loans.