WB still excellent web scraper

Started by stanl, May 20, 2017, 10:15:42 AM

Previous topic - Next topic

stanl

Need to put Tony in a better mood >:(

I took on a task to get a data field's (numeric code) description ( multi line text) from a web url, where you must input 1 numeric code and the results pop out as a javascript. Having not done this in 4 years javascript urls get pretty sophisticated...  this one blocked forward and backward caching, and all of the hrefs pointed back to the same menu. I was looking at a Python add-in and script or working with Firefox, or trying to figure out the jquery/AJAX stuff in the site. I then noticed that in spite of all the sophistication, the basic navigation was BaseURL: \Search\ : input number.  This added the secret sauce to the hrefs and by re-navigating to the results from the search I got what I needed.  Basically 2 navigates with basic WB code - in 20 lines of code.

I have been with WB since 1998. It never fails in a pinch. I'm too old to be trying to replace it with Python, Ruby or Visual Studio - I have worked with them but I don't have the flow and would not be competitive. Scripting and WB in particular helped pay off our small house and pay off our daughter's Duke loans.

SMF spam blocked by CleanTalk