WinBatch® Technical Support Forum

All Things WinBatch => WinBatch => Topic started by: mathia on July 25, 2014, 09:11:40 AM

Title: Need help navigating a page
Post by: mathia on July 25, 2014, 09:11:40 AM
I have gotten most of the way though it, but ran into trouble with jquery.  Not a clue where to start. I need to click/navigate to the Management Center link.  I'm attaching the page source.  Any help would be greatly appreciated.

Title: Re: Need help navigating a page
Post by: Deana on July 25, 2014, 10:50:24 AM
Please post the code you have so far.
Title: Re: Need help navigating a page
Post by: mathia on July 25, 2014, 10:51:59 AM
So far, the only code I have is what got me to the page I am trying to navigate.
Title: Re: Need help navigating a page
Post by: mathia on July 25, 2014, 10:56:08 AM
Browser = ObjectCreate("InternetExplorer.Application")
browser.visible = @true
url = "http://bvsperformance.bvsinc.com/registrationbank.asp"
browser.navigate(url)
gosub loading
;submitbutton=doc.Body.GetElementsByTagName("submit")
;submitbutton.click()

doc = browser.document
user=doc.Body.GetElementsByTagName("input").Item(0) ;members only
user.value="ID"
user.post
submitbutton=doc.Body.GetElementsByTagName("input").item(1)
submitbutton.click()
gosub loading
submitbutton1=doc.Body.GetElementsByTagName("INPUT").Item(0) ;attention
submitbutton1.click()
gosub loading
user=doc.Body.GetElementsByTagName("input").Item(0) ;username
user.value="username"
pass=doc.Body.GetElementsByTagName("input").Item(1) ;password
pass.value="password"
gosub loading
submitbutton=doc.Body.GetElementsByTagName("INPUT").Item(2)  ;welcome screen
submitbutton.click()
gosub loading
submitbutton=doc.Body.GetElementsByTagName("Input").Item(0)
submitbutton.click()
gosub loading
Title: Re: Need help navigating a page
Post by: Deana on July 25, 2014, 11:12:30 AM
The following article looks like an interesting start to executing jQuery from a script: http://www.s-anand.net/blog/automating-internet-explorer-with-jquery/
Title: Re: Need help navigating a page
Post by: mathia on July 25, 2014, 11:39:31 AM
Thanks. I will check it out to see if I can make heads or tails of this thing.
Title: Re: Need help navigating a page
Post by: mathia on July 28, 2014, 07:58:38 AM
I got a little ways into it this morning, but I am not able to get a reference to return.  Everything seems to execute properly, but when I try to use JQuery, there is no object reference so I get an error.  I have little to no experience with UDF's thus far, but had to use one in this instance.  Am I doing something wrong that you can see?  Here is my code:

#DefineFunction addJQuery(jquery,browser,url) 
    document = browser.document
    window = document.parentWindow
    head = document.getElementsByTagName("head");
Title: Re: Need help navigating a page
Post by: stanl on July 28, 2014, 09:08:21 AM
I posted a jquery test back in 2011 (I attached it).  Back then I had IE 9 and it returned an error, but I just tested it with IE 11 and it returns DISPATCH [for the jquery object).

Maybe I am misunderstanding the original post but jquery must be returned as an object and should reference a library of js methods and properties (the script uses Google's).

Again, I haven't messed with jQuery in years, and usually when I see DISPATCH, I rely on Tony or Deana to do the heavy lifting. Another option is to use the F12 Developer's Tools to track the actual jQuery process.

[EDIT]:  I did see the reference to the google library.... guess I was looking for exactly where your script failed. However, if you can get you script to return a jquery object, I would assume it could be parsed as a json array, or maybe some sort of AJAX method to parse it....  Sorry for any confusion from the initial part of this response.
Title: Re: Need help navigating a page
Post by: mathia on July 28, 2014, 02:59:08 PM
I looked at your code. I based mine on the same article.  I am stuck at the point of it returning a reference to the jquery object.  Not sure where to go from here.  I thought maybe I had messed up in putting together the UDF.  Haven't done anything with anything like that since college, which was a loooooong time ago.  Since videos.google.com is not the same as it was at the time the article was written, I am not even sure that youtube would return similar results or not.  If you have further guidance to offer, it would be greatly appreciated.
Title: Re: Need help navigating a page
Post by: stanl on July 29, 2014, 03:39:15 AM
In looking at the text file you posted, I'm not sure you really need a jquery object. Seems the references are there to support several of the functions in the code. Since the actual site is password protected cannot actually see the display.  But it appears you need to access a menu option then execute jscript code to proceed.
Title: Re: Need help navigating a page
Post by: mathia on July 29, 2014, 06:04:57 AM
You are correct. I am trying to access a menu item.  I am automating a process that involves logging into the site, then uploading a file.  I saw the jquery thing in the code of the site and thought that was the way to go to get there.
Title: Re: Need help navigating a page
Post by: stanl on July 29, 2014, 09:36:36 AM
Quote from: mathia on July 29, 2014, 06:04:57 AM
You are correct. I am trying to access a menu item.  I am automating a process that involves logging into the site, then uploading a file.  I saw the jquery thing in the code of the site and thought that was the way to go to get there.

OK. From what I see, you need to iterate the pages listitems.  Each has an ID and an Href.  For example,

id='courses24'   has an href \"menuNL/continuingEducationCredit.asp\" 

and I would think you could code the click() event of the listitem which would launch the href.  There may be an example in the tech db associated with this. I can look in my archives after work.
Title: Re: Need help navigating a page
Post by: Deana on July 29, 2014, 10:13:11 AM
You might try this based on Stans suggestion of locating the item by its id then click on it:

Code (winbatch) Select
oIE = ObjectCreate("InternetExplorer.Application")
oIE.visible = @true
url = "C:\TEMP\dynamic learning.html"
oIE.navigate(url)

while oIE.readystate <> 4
timedelay(0.5)
endwhile

oElement = oIE.Document.getElementByID("td_perf")

;Click on Element
;oElement.click()
;OR
oEvent = oIE.document.createEvent("HTMLEvents")
oEvent.initEvent("click", @TRUE, @TRUE)
oElement.dispatchEvent(oEvent)

while oIE.readystate <> 4
        timedelay(0.5)
endwhile
exit
Title: Re: Need help navigating a page
Post by: mathia on July 29, 2014, 10:52:44 AM
Tried this with slight changes to fit into my existing script as you cannot get to the particualr page I am on without logging in and going through the preceding pages.

;oIE = ObjectCreate("InternetExplorer.Application")
;oIE.visible = @true
;url = "C:\TEMP\dynamic learning.html"
;oIE.navigate(url)

while browser.readystate <> 4
        timedelay(0.5)
endwhile

oElement = doc.getElementByID("td_perf")

;Click on Element
;oElement.click()
;OR
oEvent = doc.createEvent("HTMLEvents")
oEvent.initEvent("click", @TRUE, @TRUE)
oElement.dispatchEvent(oEvent)

while browser.readystate <> 4
        timedelay(0.5)
endwhile

oEvent returns a null.
Title: Re: Need help navigating a page
Post by: mathia on July 29, 2014, 11:03:24 AM
Probably speaking too soon, but I think this method will work on the next page, but not the one I am currently stuck on.   I am trying to click on the Management Center item.  Is it different from a regular item you would click on?
Title: Re: Need help navigating a page
Post by: Deana on July 29, 2014, 11:28:55 AM
Did you try oElement.click()?
Title: Re: Need help navigating a page
Post by: stanl on July 29, 2014, 01:40:55 PM
Quote from: mathia on July 29, 2014, 11:03:24 AM
Probably speaking too soon, but I think this method will work on the next page, but not the one I am currently stuck on.   I am trying to click on the Management Center item.  Is it different from a regular item you would click on?

For that you need to run -  constructMenu("perf") which then creates the listItems you can select from. I believe you can execute it from the document object.  Not sure, but something like.....

oIE.document.parentWindow.execScript(""javascript:constructMenu("perf") ;"")

or

oIE.document.parentWindow.execScript('javascript:constructMenu("perf") ;')
Title: Re: Need help navigating a page
Post by: mathia on July 29, 2014, 01:49:14 PM
Quote from: Deana on July 29, 2014, 11:28:55 AM
Did you try oElement.click()?

You, my dear, are a genius.
I saw that commented out and didn't know whether to try it or not.  I did and made it though the next 2 pages. I am now working to see how to select an item that I think is in a frame. I am looking at the source for the page, but am not seeing the text that is displayed for the link.  I've even tried Stan's ie analyze tool with an htm file. 
Title: Re: Need help navigating a page
Post by: Deana on July 29, 2014, 01:59:29 PM
Quote from: mathia on July 29, 2014, 01:49:14 PM
Quote from: Deana on July 29, 2014, 11:28:55 AM
Did you try oElement.click()?

You, my dear, are a genius.
I saw that commented out and didn't know whether to try it or not.  I did and made it though the next 2 pages. I am now working to see how to select an item that I think is in a frame. I am looking at the source for the page, but am not seeing the text that is displayed for the link.  I've even tried Stan's ie analyze tool with an htm file.

Excellent. Glad to hear that worked.
HINT: F12 option in your browser can be helpful in identifying webpage elements.
Title: Re: Need help navigating a page
Post by: stanl on July 30, 2014, 06:15:13 AM
Quote from: mathia on July 29, 2014, 01:49:14 PM
I've even tried Stan's ie analyze tool with an htm file.

I think when I first posted that, iframes and divs were not part of the options. You never mentioned what version of IE you were using which might explain why the object.click() worked and Deana's other suggestion didn't. Out of curiosity, what if you manually navigate all the way to a desired page, copy/paste the url into notepad, then modify your base script to login to the site with credentials then navigate to that url - would it work. If so, you might be able to shortcut your navigation on the site.
Title: Re: Need help navigating a page
Post by: mathia on July 30, 2014, 06:25:31 AM
My first approach was just to look for links in the code of the site as I figured that was the easiest way to get there.  I found out very quickly it wasn't going to be that easy.  Even once you are a little deeper into the site, there are no link, just functions that build the next page or menu.  Well, as far as I can see, that's how it is.  Currently, I am on IE 8, but that will probably change very soon.  From what i've read in my quest to complete this task, I will probably be revisiting this again once browser upgrades are pushed out.
Title: Re: Need help navigating a page
Post by: stanl on July 30, 2014, 07:27:36 AM
Quote from: mathia on July 30, 2014, 06:25:31 AM
Currently, I am on IE 8

Funny, a lot of corporations are still on 8, and use chrome for other browsing. Just to clarify what I was talking about in terms of navigation, in my last job we had to work with a protected site that was all Jquery, used json to activate hidden divs, and made typical page navigation near impossible (using things like document.readystate were inaccurate, as you couldn't tell when a div or iframe was actually built, and previous data was stored as a _VIEWSTATE)

However, once I played around and determined where I needed to be, the following constants proved invaluable..


navOpenInNewWindow=1
navNoHistory=2
navNoReadFromCache=4
navNoWriteToCache=8
navAllowAutosearch=16
navBrowserBar=32
navHyperlink=64
navEnforceRestricted=128
navNewWindowsManaged=256
navUntrustedForDownload=512
navTrustedForActiveX=1024
navOpenInNewTab=2048
navOpenInBackgroundTab=4096
navKeepWordWheelText=8192
navVirtualTab=16384
navBlockRedirectsXDomain=32768
navOpenNewForegroundTab=65536


and I used IE's navigate2() rather than navigate()


oIE.Navigate2(::url=cURL,Flags=navOpenInNewTab)

or

oIE.Navigate2(::url=cURL,Flags=navNoReadFromCache)


so my script would have a number of url entries, many just the base url with a call to a .asp page but I was able to handle them without relying on the cache or opening in a new tab.
Title: Re: Need help navigating a page
Post by: mathia on July 30, 2014, 12:51:33 PM
Well, I'm pretty new to all this web stuff, which is why I'm stumbling through it.  I am currently trying to click a link that is contained within an iFrame.  Can't get the code, so I can't post it.  I can return a reference to the iFrame itself, but can't seem to get to the onclick item I'm trying to get to within the iFrame.  I'm not even sure that's what I'm supposed to be looking at. 
Title: Re: Need help navigating a page
Post by: Deana on July 30, 2014, 01:06:27 PM
F12 web developer tools built into many browsers will be your friend. ;)
Title: Re: Need help navigating a page
Post by: Deana on July 30, 2014, 01:09:22 PM
Quote from: stanl on July 30, 2014, 06:15:13 AM
Quote from: mathia on July 29, 2014, 01:49:14 PM
I've even tried Stan's ie analyze tool with an htm file.

I think when I first posted that, iframes and divs were not part of the options. You never mentioned what version of IE you were using which might explain why the object.click() worked and Deana's other suggestion didn't. Out of curiosity, what if you manually navigate all the way to a desired page, copy/paste the url into notepad, then modify your base script to login to the site with credentials then navigate to that url - would it work. If so, you might be able to shortcut your navigation on the site.

Here is a function that can be used to click using either method if available:

Code (winbatch) Select
#DefineFunction udfIEClick( objIE, objElement )
    objEvent = objIE.document.createEvent( "HTMLEvents" )
    If objEvent == "" Then
      objElement.Click()
    Else
      objEvent.initEvent("click", @TRUE, @TRUE)
      objElement.dispatchEvent(objEvent)
    EndIf
    Return 1
#EndFunction

Title: Re: Need help navigating a page
Post by: mathia on July 30, 2014, 04:16:56 PM
I've googled and read and tried and tried.  Finally found the links collection.  I can get a list of all of the anchor tags in the section I need.  None of them have names, though.  I can click every link in the collection using a foreach loop and was able to count down through the code to figure out which one I need to click.  After that, I can simply reference it using the item property of the links collection.  This all seems a bit kludgey to me. Is there a way to identify an anchor tag that doesn't have a name property? It only has text that identifies it as far as I can see.   I've tried all kinds of properties, but they all seem to return a NULL.  Except for the href property which returns the same thing for each item.  Hope some of that makes sense.
Title: Re: Need help navigating a page
Post by: mathia on July 31, 2014, 02:47:03 PM
When a new window opens, how do you get a reference to the new browser object?
Title: Re: Need help navigating a page
Post by: td on July 31, 2014, 03:15:13 PM
One approach is to use the "Shell.Application" object.  I think there is a better way but can't recall it at the moment.

The "Shell.Application" approach can be found on this page.  You will need to scroll down a bit to find it.

http://techsupt.winbatch.com/webcgi/webbatch.exe?techsupt/nftechsrch.web+~+TechHome (http://techsupt.winbatch.com/webcgi/webbatch.exe?techsupt/nftechsrch.web+~+TechHome)
Title: Re: Need help navigating a page
Post by: mathia on August 01, 2014, 07:54:51 AM
If I get a window handle using this method, will I still be able to use the IE object model?
Title: Re: Need help navigating a page
Post by: td on August 01, 2014, 08:16:19 AM
The UDS in the tech db returns an IE application object which means you have access to the document object.
Title: Re: Need help navigating a page
Post by: mathia on August 01, 2014, 08:29:42 AM
Excellent.  By the way, that link only takes me to a seach page.
Title: Re: Need help navigating a page
Post by: td on August 01, 2014, 08:37:23 AM
Yes the changes to the tech db require a different approach to getting links to search results.  You can find the article and others by executing a search on the mentioned progid.
Title: Re: Need help navigating a page
Post by: mathia on August 01, 2014, 09:57:15 AM
I'm assuming you're referring to this piece of code:

#definefunction BrowserGetObject()
oShellApp=Objectaccess("Shell.Application",1)
oWindows=oShellApp.Windows()
count=oWindows.Count   
obrowser=oWindows.Item(count-1)
objectclose(oShellApp)
return obrowser
#endfunction

When I add this to my existing code, it returns an object reference, but it is a reference to the window that spawned the second browser instance.  I have verified this, I think, by getting the title of the window.  If it helps, here is my code leading up to all of this

link=iframe.contentwindow.document.GetElementsByTagName('div').id('s29')
link.length
count=0
foreach link in elementlist
   test2=link.id
   if test2=='s29' then index=count
   count=count+1
next
test3=elementlist.item(index).getelementsbytagname('input')
len=test3.length
test4=test3.item(0)
test4.click()  ;This line opens a new browser window
gosub loading ;wait 4 seconds for page to load
test7=browserGetObject()

title=test7.document.title
oelement= test7.document.getElementBytagname("input") ;returns 0 elements
test=oelement.length ;error, presumably because there aren't any elements
Title: Re: Need help navigating a page
Post by: td on August 01, 2014, 10:28:10 AM
Actually, I was refer to the article entitled "Working With Web Pages" - script Q10.WBT.  It is the very first article link when you search on "shell.application" sans quotes. Note this script returns an 'objWindows' object but that is a meaningless distinction in this case.




Title: Re: Need help navigating a page
Post by: stanl on August 01, 2014, 01:23:03 PM
here is some WB code specific to iFrames

http://techsupt.winbatch.com/webcgi/webbatch.exe?techsupt/tsleft.web+WinBatch/OLE~COM~ADO~CDO~ADSI~LDAP/OLE~with~MSIE+Dealing~with~iFrames.txt (http://techsupt.winbatch.com/webcgi/webbatch.exe?techsupt/tsleft.web+WinBatch/OLE~COM~ADO~CDO~ADSI~LDAP/OLE~with~MSIE+Dealing~with~iFrames.txt)

Title: Re: Need help navigating a page
Post by: mathia on August 01, 2014, 02:10:55 PM
I can probably make it work, but it's a bit inconsistent when trying to find the IE window.  I initially had to open another window just to get it to detect the browser.
Title: Re: Need help navigating a page
Post by: td on August 01, 2014, 03:48:48 PM
The whole process of using IE's COM Automation interfaces to manipulate websites is a first-order kludge.  The constantly changing and inconsistently followed standards only add to fun.   

Here is the link to Jay's afore mentions tutorial in case you haven't found it yet.

http://techsupt.winbatch.com/webcgi/webbatch.exe?techsupt/nftechsupt.web+Tutorials+Working~With~Web~Pages.txt (http://techsupt.winbatch.com/webcgi/webbatch.exe?techsupt/nftechsupt.web+Tutorials+Working~With~Web~Pages.txt)
Title: Re: Need help navigating a page
Post by: mathia on August 04, 2014, 09:44:41 AM
I'm still wrestling with getting the UDF to recognize the windows I have open.  Not sure what the issue is. I have only gotten it to work once, so far.
I'm wondering if I can modify and execute the code attached to the button I am clicking.  The line in question is this:

<input class="btn100" onclick="openWindow('docUploadAdhocFile.asp',800,550)" type="button" value="Display Form"/>

Is there a way to get this to display in the current window rather than opening a new one.  Site code is totally outside my control, but I though it might be possible to send this directly to the page to get it to do my bidding. 
Title: Re: Need help navigating a page
Post by: td on August 04, 2014, 11:32:44 AM
Wild guess is that your "Shell.Application" problem is the result of some kind of timing issue.  The timing problem could be in one of several places during the process. The problem could even be creating the "Shell.Application" object prematurely.

I am not aware of any generic something or another you could send to the server to change window creation behavior.  If there is such a thing, best guess is that it would be site specific.   

The 'openWindow' method likely is a javascript method.  In theory it is possible to execute a client side script that would override one or more javascript functions sent by the site with your own functions.  Your own functions could modify site behavior as you see fit.   I have never tried this using IE COM Automation interfaces so I have no idea if it will work in that context.

Some browsers allow you to maintain a library of javascipt overrides that can be associated with specific web pages.  My browser of choice supports site specific javascript libraries and I have used this functionality successfully.     
Title: Re: Need help navigating a page
Post by: stanl on August 05, 2014, 07:27:03 AM
Quote from: td on August 04, 2014, 11:32:44 AM
The 'openWindow' method likely is a javascript method.  In theory it is possible to execute a client side script that would override one or more javascript functions sent by the site with your own functions.  Your own functions could modify site behavior as you see fit.   

I've seen this done, in FireFox with Ruby. Ruby (and I guess Python) can harvest Ajax and PHP to manipulate web sites.
Title: Re: Need help navigating a page
Post by: mathia on August 05, 2014, 07:38:00 AM
I assumed it was possible, but I don't know how to do it.
Title: Re: Need help navigating a page
Post by: mathia on August 05, 2014, 07:45:13 AM
I found this article yesterday, which might shed some light on problems getting a handle to a new browser window.  I have modified my code to launch the browser with a Run command, then get a handle.  This works for most of the script, but once I get to the part that opens a new window, the shell object is not seeing the new window, just the one I launched initially.

http://blogs.msdn.com/b/ieinternals/archive/2012/02/07/10192483.aspx
Title: Re: Need help navigating a page
Post by: mathia on August 05, 2014, 08:11:12 PM
Since I can't get a handle using the shell object, is it possible to get a handle if you have the process id?
Title: Re: Need help navigating a page
Post by: JTaylor on August 05, 2014, 10:06:04 PM
Did you try to "navigate" to that asp page you wanted?  Not sure what you are trying to do overall but if it works it would eliminate your stated problem, as I understand it anyway.

Jim
Title: Re: Need help navigating a page
Post by: mathia on August 06, 2014, 06:22:32 AM
That definitely would have eased my troubles, but I have not been able to get it to work.