WinBatch® Technical Support Forum

All Things WinBatch => WinBatch => Topic started by: John Sikora on July 12, 2023, 08:30:31 AM

Title: Best way to get a page from Edge into a DOM
Post by: John Sikora on July 12, 2023, 08:30:31 AM
I've been searching through the forum and other sources and am, to put it bluntly, confused. What is the best way (if there is one), to request a page from the Edge browser and then make that page a DOM object so that I can pull items from the page?
Title: Re: Best way to get a page from Edge into a DOM
Post by: td on July 12, 2023, 08:50:40 AM
Why do you want to use Edge? There are several techniques for Web scraping including the WinBatch WinInet extender, WinBatch COM automation subsystem, or WinBatch .Net subsystem. But which one you use depends greatly on the nature of the Webpage you are scraping. Keep in mind that some "dynamic" Webpages may require third-party software to use with the above WinBatch functionality.

Most of this is covered in the Tech Database and on this forum.
Title: Re: Best way to get a page from Edge into a DOM
Post by: td on July 12, 2023, 10:24:49 AM
It may be a bit dated but here is an about 5-year-old discussion about using a third-party tool for Web scraping:

https://forum.winbatch.com/index.php?topic=2010.msg10318#msg10318 (https://forum.winbatch.com/index.php?topic=2010.msg10318#msg10318)
Title: Re: Best way to get a page from Edge into a DOM
Post by: stanl on July 13, 2023, 02:28:18 AM
Quote from: td on July 12, 2023, 10:24:49 AM
It may be a bit dated but here is an about 5-year-old discussion about using using a third-party tool for Web scraping:

https://forum.winbatch.com/index.php?topic=2010.msg10318#msg10318 (https://forum.winbatch.com/index.php?topic=2010.msg10318#msg10318)


and here is a more recent discussion
https://learn.microsoft.com/en-us/microsoft-edge/webdriver-chromium/?tabs=c-sharp

Title: Re: Best way to get a page from Edge into a DOM
Post by: td on July 14, 2023, 06:09:21 AM
With Edge thrown in as a bonus.