Changes to Internet Explorer DOM

[George Vagenas]

I've moved from XPx32 to Win7x64 and I'm trying to understand why some code that used to work with IE 8 does not work with IE 11.  At first I thought the problem was changes to the site I'm trying to access and while there were some changes I couldn't see how they would account for the code no longer working.
Finally (after two days of head scratching) I was able to get the script working again.  One of the problems came down to how the innerHTML property was being returned which I was able to work around with a very minor change to the script but the only solution to the other problem was to resort to using SendKeysTo.

I had archived the original pages that I based the code on and I'm stumped as to what caused the original script to stop working.  Here's the original HTML of the login page:

Code (html5) Select Expand

<div class="formActions">
<ul>
<li><input type="submit" title="Login to Online Banking" name="Continue" id="Continue" tabindex='1004' value="Login" class="submit"/></li>

</ul>
</div>

and here's the current page:

Code (html5) Select Expand

<div class="formActions">
<ul>
<li><input type="submit" title="Login to Online Banking" name="Continue" id="Continue" value="Login" class="submit"/></li>





</ul>
</div>
Apart from whitespace they are identical but this code no longer works:

Code (winbatch) Select Expand

   prqDoc = prqIE.Document                                            ; Safety refresh.
   prqAll = prqDoc.all

   prqAction = inireadpvt('Internet', prqActionRqst, '', gIniPvt)     ; Get the current action.
   prqType = prqAll.%prqAction%
   prqType.Click

After substitution the line reads:

Code (winbatch) Select Expand

prqType = prqAll.Continue and prqIE is defined earlier in the script as

Code (winbatch) Select Expand

prqIE = objectopen("InternetExplorer.Application") Only now with IE11 it doesn't work and just in case I have designated the site as TRUSTED with LOW security and UAC is turned off.  The only work around I could come up with was resorting to SendKeysTo.  I'd really to get the click action working again but I don't have a clue as to what I need to change.

[JTaylor]

For the "click" you will need to do something like (I have not altered this for your object names or situation) the following.  It will handle older and newer versions of IE.

Code (winbatch) Select Expand

 
    objEvent = yobrowser.document.createEvent("HTMLEvents")
    i = yobrowser.document.GetElementById("olp_page_next")
    If objEvent == "" Then
      i.click
    Else
      objEvent.initEvent("click", @true, @true)
      i.dispatchEvent(objEvent)
    EndIf

[Deana]

Welcome to the wonderful world of IE. Microsoft has made some significant changes to IE between version 8 and 11. Also now with Windows 7, UAC is also a new factor.

For example:
Since Internet Explorer 4, document.all has been supported in Internet Explorer. Prior to the implementation of document.getElementById(), document.all was the best way of getting an element reference. document.all has remained in Internet Explorer through version IE10. As of IE11, document.all is no longer...Therefore you will need to modify your code accordingly and call document.getElementById instead.

Please read Microsoft's articles
Compatibility changes in IE11 : http://msdn.microsoft.com/en-us/library/ie/bg182625(v=vs.85).aspx
Internet Explorer 11 guide for developers : http://msdn.microsoft.com/library/ie/bg182636(v=vs.85).aspx

I have also found that the Click and Focus methods seem to no longer work starting with IE9. Often adding the webpage to Compatibility View resolves the Focus and Click issue. However, here is some code that can often be used in place of the Click method:

Code (winbatch) Select Expand


; WORKAROUND - createEvent initEvent DispatchEvent
objEvent = objBrowser.document.createEvent("HTMLEvents")
objEvent.initEvent("click", @TRUE, @TRUE)
objElement.dispatchEvent(objEvent) ; objElement is the handle to the object to click (i.e. button)


Check out my Webpage Element Explorer script which can be used for Discovering form elements on a webpage: http://techsupt.winbatch.com/webcgi/webbatch.exe?techsupt/tsleft.web+WinBatch/OLE~COM~ADO~CDO~ADSI~LDAP/OLE~with~MSIE+!!~Webpage~Element~Explorer~!!.txt

[George Vagenas]

Sorry for not responding immediately but other lives do look in from time to time.  Thank you both for the input and I've used JTaylor's code to fix the script (it would appear that IE11 is not consistent in whether it will honor the request to create an event).
I followed Deana's links and learned that "Document modes are deprecated", the current version of IE11 still supports "document.all", but if future versions no longer support it then my script will be seriously broken.
Currently I'm using "document.all" like this

Code (winbatch) Select Expand

   prqObj = prqAll.item(prqObjNum)
   prqTag = prqObj.tagname
   prqInner =  prqObj.InnerHtml
   if strindexnc(prqInner, '<INPUT', 0, @fwdscan)==1 && strindexnc(prqInner, 'class="checkbox eventEnabledFields2"', 0, @fwdscan)!=0 && strupper(prqTag)=='LABEL'
      return prqObj
   else
      prqCnts = prqAll.length-1
      for prqCnt = 0 to prqCnts
         prqObj = prqAll.item(prqCnt)
         prqTag = prqObj.tagname
         prqInner =  prqObj.InnerHtml
         if strindexnc(prqInner, '<INPUT', 0, @fwdscan)==1 && strindexnc(prqInner, 'class="checkbox eventEnabledFields2"', 0, @fwdscan)!=0 && strupper(prqTag)=='LABEL'
            iniwritepvt(Section, KeyName, prqCnt, gIniPvt)
            prqChk = @true
            ; prqObj = prqAll.item(prqCnt)
            ; prqTable =  prqObj.InnerHtml
            display(3, 'Page Change', 'Ini file has been updated with new table object number.')
            return prqObj
         endif
      next
   endif

Later in the script the same code is called again to verify the value in prqInner and if it checks out then it is  is written to a file for further processing.  This code is part of Winbatch app that I've patched and prodded over the last sixteen years to retrieve bank statements and feed them to a bookkeeping program.
As usual and help is greatly appreciated.

[Deana]

Instead of using document.all use one of the following to access the individual page elements:

document.getElementById() Returns the element that has the ID attribute with the specified value
document.getElementsByName() Accesses all elements with a specified name
document.getElementsByTagName() Returns a NodeList containing all elements with the specified tagname
Reference: http://www.w3schools.com/jsref/dom_obj_document.asp

Check out my Webpage Element Explorer script which can be used for Discovering form elements on a webpage: http://techsupt.winbatch.com/webcgi/webbatch.exe?techsupt/tsleft.web+WinBatch/OLE~COM~ADO~CDO~ADSI~LDAP/OLE~with~MSIE+!!~Webpage~Element~Explorer~!!.txt

NOTE: This script can also be used as an example of how to access all off the elements on a webpage.

[Deana]

If you need assistance getting an object handle to the webpage element...Post the HTML source of the element that you are trying to query with your code.

[George Vagenas]

It turns out that the innerHtml IE is returning is not the same as IE8's which means I have to revise the script to handle this.  The page has a search button which refreshes the page with the requested transactions, after the refresh the page has a DIV structure containing the transactions.

Code (html5) Select Expand

<div class="summarygroup">
.
</div>

Now, if there was a getElementByClass function I could actually simplify the code.

[Deana]

It turns out that the innerHtml IE is returning is not the same as IE8's which means I have to revise the script to handle this.  The page has a search button which refreshes the page with the requested transactions, after the refresh the page has a DIV structure containing the transactions.

Code (html5) Select Expand

<div class="summarygroup">
.
</div>

Now, if there was a getElementByClass function I could actually simplify the code.

See the getElementByClassName Method:

http://msdn.microsoft.com/en-us/library/ie/ff975198(v=vs.85).aspx

Code (winbatch) Select Expand


#DefineFunction udfIEPageLoadWait( objIE )
    ; Wait for webpage to load
    While !(objIE.readyState == 'complete' || objIE.readyState == 4 )
       TimeDelay(0.1)
    EndWhile
    While !(objIE.document.readyState == 'complete' || objIE.document.readyState == 4 )
       TimeDelay(0.1)
    EndWhile
    Return 1
#EndFunction


url = ????
classname = ????

; Initialize MSIE object
oIE   = ObjectCreate("InternetExplorer.Application")
oIE.Visible = @TRUE ; Change to @FALSE to hide the process from the user
oIE.Navigate(url)

; Wait for webpage to load
udfIEPageLoadWait( oIE )

; Get document object
oDoc = oIE.Document

cElements = oDoc.getElementsByClassName(classname)

ForEach oElement In cElements
      ErrorMode(@off)
      prqType  = oElement.type
      ErrorMode(@cancel)
      prqTag = oElement.tagname ;The value of tagName is the same as that of nodeName.
      prqInner =  oElement.InnerHtml
      if prqtype=='INPUT' && strindexnc(prqInner, 'class="checkbox eventEnabledFields2"', 0, @fwdscan)!=0 && strupper(prqTag)=='LABEL'
         Pause('',prqInner)
         ;return oElement
      endif
Next



:CleanUp

; Quit
oIE.Quit

; Close open COM/OLE handles
oElement = 0
oDoc = 0
oIE = 0
Exit

[George Vagenas]

Deana thanks but I'm not sure what object I should be using with this method.

Code (winbatch) Select Expand

prqType = prqDoc.getElementByClassName("summarygroup")
returns "VT NULL". 
I've successfully invoked the getElementById method in other parts of the script with:

Code (winbatch) Select Expand

prqType = prqDoc.getElementById("acctnum")
Because there was a page change I used this code

Code (winbatch) Select Expand

prqDoc = prqIE.Document prior to invoking the method.  As usual with IE and OOP I'm stumped and feeling like a n00b.

[Deana]

Deana thanks but I'm not sure what object I should be using with this method.

Code (winbatch) Select Expand

prqType = prqDoc.getElementByClassName("summarygroup")
returns "VT NULL". 

Can you confirm the source HTML that contains the class attribute 'summarygroup'?

[George Vagenas]

If by confirm you mean can I see the updated page in IE, yes (I've set IE.visible to @true while I'm debugging).  Also the DIV structure I posted was from the source Html which I downloaded from the website after the transaction request.

[Deana]

If by confirm you mean can I see the updated page in IE, yes (I've set IE.visible to @true while I'm debugging).  Also the DIV structure I posted was from the source Html which I downloaded from the website after the transaction request.

Actually I was asking you to share the HTML source code of the page, if you need further assistance.

[George Vagenas]

Here's the source, I've removed any personal info and left two modified transactions in the DIV structure.  If you search for "summarygroup" that will take you to the content that is added to the page after the script clicks the Search input.

[Deana]

The code I posted previously Can get the innerHTML of that DIV! Here is it is again with a slight modification:

Code (winbatch) Select Expand

#DefineFunction udfIEPageLoadWait( objIE )
    ; Wait for webpage to load
    While !(objIE.readyState == 'complete' || objIE.readyState == 4 )
       TimeDelay(0.1)
    EndWhile
    While !(objIE.document.readyState == 'complete' || objIE.document.readyState == 4 )
       TimeDelay(0.1)
    EndWhile
    Return 1
#EndFunction


url = 'C:\TEMP\Results.htm'
classname = 'summarygroup'

; Initialize MSIE object
oIE   = ObjectCreate("InternetExplorer.Application")
oIE.Visible = @TRUE ; Change to @FALSE to hide the process from the user
oIE.Navigate(url)

; Wait for webpage to load
udfIEPageLoadWait( oIE )

; Get document object
oDoc = oIE.Document

cElements = oDoc.getElementsByClassName(classname)

ForEach oElement In cElements
      ErrorMode(@off)
      prqType  = oElement.type
      ErrorMode(@cancel)
      prqTag = oElement.tagname ;The value of tagName is the same as that of nodeName.
      prqInner =  oElement.InnerHtml
      Pause('InnerHtml',prqInner)
      ;if prqtype=='INPUT' && strindexnc(prqInner, 'class="checkbox eventEnabledFields2"', 0, @fwdscan)!=0 && strupper(prqTag)=='LABEL'
      ;   Pause('',prqInner)
         ;return oElement
      ;endif
Next


:CleanUp

; Quit
oIE.Quit

; Close open COM/OLE handles
oElement = 0
oDoc = 0
oIE = 0
Exit

[George Vagenas]

Code (winbatch) Select Expand

   ; Get document object
   prqDoc = prqIE.Document
   cElements = prqDoc.getElementsByClassName(classname)

returns VT Null. 
I think its major revision time, I've got an idea for a different approach and I can take this opportunity to lose the called batch files and convert them to UDFs.
As you said Deana "Welcome to the wonderful world of IE".  The click method would not work on the login page but worked in two other parts of the script, one to select a radio button and another to submit the transaction query.  Maybe there's a "Hmm" in there somewhere for the Html gurus because in these instances the "createEvent("HTMLEvents")" did not succeed but on the login page it did.
At any rate if my idea pans out hopefully I'll have a more bullet proof script against IE's search for browser legitimacy.  And I know I've said it before but if this was the movie industry your team would be walking off with all the oscars.

[Deana]

Previously you mentioned ' If you search for "summarygroup" that will take you to the content that is added to the page after the script clicks the Search input.' Have you confirmed that the click is successfully being sent. Based on the NULL result returned from getElementsByClassName, I suspect the click was not successfully sent.

To debug, try adding a Pause(0,0) or a breakpoint just before the getElementsByClassName. Do you visually see the data you are trying to query? Look at the webpage source at this point to see if the class appears. IF not, then address the Search input click problem. Might just need a TimeDelay after the click to give the page time to load the new data.

[George Vagenas]

As you could see from the page I emailed you the transactions are definitely showing up in IE which means the click was received and processed. Also, I ran your udfIEWait4Page before trying to access the class.

[George Vagenas]

This stuck in my head from another topic

Progress...it's 2014 and I'm installing 2003...because it works LOL

and seemed apropos.
Its 2014 and after using OLE to get to a page I'm using basic page scrapping, as per your post,

Code (winbatch) Select Expand

objBrowserDoc = objBrowser.Document
objBrowserBody = objBrowserDoc.Body
Message("InnerText", objBrowserBody.InnerText )
and modifying it to suit my needs.

Code (winbatch) Select Expand

   prqDoc = prqIE.Document
   prqBody = prqDoc.Body
   prqInner = prqBody.InnerText

   prqIE.GoBack()    ; Back to the request page.
   IEWait4Page(prqIE)

   ; Test prqInner for expected content.
   gErr = strindexnc(prqInner, 'Skip to Content', 0, @fwdscan)-1
   if gErr==0
      fileput(prqParam2, prqInner)
      return ; to CUrAcct.wbt - ;: Call OlePostReq
   endif


All I have to do is write a simple parsing routine to extract the transactions.