WinBatch® Technical Support Forum

All Things WinBatch => WinBatch => Topic started by: stanl on September 28, 2021, 04:27:26 PM

Title: CLR: WebRequest / headers only
Post by: stanl on September 28, 2021, 04:27:26 PM
Just wondering is there is a way to get around the attached error the script produces on line headers = oWebRequest.GetResponse().Headers


Code (WINBATCH) Select


;CLR - attempt Web Request to get headers only
Url = 'https://community.idera.com/'
ObjectClrOption('useany', 'System')
oWebUtil = ObjectClrNew('System.Net.WebUtility')


oUri = ObjectClrNew('System.Uri', Url)
oSvcManager = ObjectClrNew('System.Net.ServicePointManager')
protocols = ObjectClrType("System.Net.SecurityProtocolType",3072|768) 
oSvcManager.SecurityProtocol = protocols
oSvcPoint = oSvcManager.FindServicePoint(oUri)


oWebRequest = ObjectClrNew('System.Net.WebRequest')
oRequest = oWebRequest.Create(oUri)
oRequest.Timeout = oRequest.Timeout * 6
headers = oRequest.GetResponse().Headers
;gives error here


keys = headers.AllKeys
output="key":@tab:"Value":@lf


ForEach k in keys
    output:= k:@tab:headers.GetValues(k):@lf
Next


oResponse.Close()


Message(Url,output)


Exit

Title: Re: CLR: WebRequest / headers only
Post by: stanl on September 29, 2021, 10:47:45 AM
Oh, and before the put-downs or other arguments start: yes, this does work in powershell


$url = 'https://community.idera.com/'
$request = [System.Net.WebRequest]::Create( $Url )
$headers = $request.GetResponse().Headers
$headers.AllKeys |
     Select-Object @{ Name = "Key"; Expression = { $_ }},
     @{ Name = "Value"; Expression = { $headers.GetValues( $_ ) } }



with 2 points:

no harm / no foul
Title: Re: CLR: WebRequest / headers only
Post by: td on September 29, 2021, 10:57:48 AM
That is a bizarre error message. I have no idea why the CLR is regurgitating that exception because the WIL DLL is not attempting to use an IDispatch interface to access anything in your script. The response class is abstract so the behavior of the object returned by GetResponse can vary based on the concreate class returned but the SupportsHeaders property returns true. For example,

Code (winbatch) Select
objResponse = oRequest.GetResponse()
if objResponse.SupportsHeaders then strText = 'Headers supported'
else strText = 'Headers not supported'
Message('Are Headers supported', strText)


Also, the WebHeaderCollection class returned by the Headers property is marked as ComVisibleAttribute so it shouldn't make any difference anyway.  Here's a link to the class:

https://docs.microsoft.com/en-us/dotnet/api/system.net.webheadercollection?view=netframework-4.8 (https://docs.microsoft.com/en-us/dotnet/api/system.net.webheadercollection?view=netframework-4.8)

I will have to do a little bit more investigation in an attempt to at least come up with a better explanation of what is going on.

You might be able to use the GetResponseHeader method as an alternative.

https://docs.microsoft.com/en-us/dotnet/api/system.net.httpwebresponse.getresponseheader?view=netframework-4.8 (https://docs.microsoft.com/en-us/dotnet/api/system.net.httpwebresponse.getresponseheader?view=netframework-4.8)
Title: Re: CLR: WebRequest / headers only
Post by: td on September 29, 2021, 11:01:47 AM
Quote from: stanl on September 29, 2021, 10:47:45 AM
Oh, and before the put-downs or other arguments start

I was under the impression that we have gotten past that point...
Title: Re: CLR: WebRequest / headers only
Post by: stanl on September 29, 2021, 12:07:08 PM
Quote from: td on September 29, 2021, 11:01:47 AM
Quote from: stanl on September 29, 2021, 10:47:45 AM
Oh, and before the put-downs or other arguments start

I was under the impression that we have gotten past that point...


like I wrote... no harm / no foul..


but, if you execute the PS code against 'https://forum.winbatch.com/' you will get 403 forbidden error, which is useful info [I'm doing this as a favor for an old client, not for money]...


Just turned out an interesting error.... I think there is a javascript getresponseheaders() function... dunno
Title: Re: CLR: WebRequest / headers only
Post by: td on September 29, 2021, 12:37:04 PM
The GetResponseHeader method appears to work. Using your script with the method call GetResponseHeader('Content-Type') I received the header content "text/html: charset=utf-8" which is what you would expect.
Title: Re: CLR: WebRequest / headers only
Post by: stanl on September 29, 2021, 03:01:09 PM
Quote from: td on September 29, 2021, 12:37:04 PM
The GetResponseHeader method appears to work. Using your script with the method call GetResponseHeader('Content-Type') I received the header content "text/html: charset=utf-8" which is what you would expect.


Understood. But the object was to iterate all header keys not just a single one as web request will return different or different amount of keys. And, in testing I missed your use of the function



oResponse = oRequest.GetResponse()
test = oResponse.GetResponseHeader('Content-Type')
Message("",test)          FAILS
Title: Re: CLR: WebRequest / headers only
Post by: td on September 29, 2021, 04:58:02 PM
There is a defined list of headers that are supported by the response object so those would be the only ones you need to test for in a straightforward loop. Or just use a list of common header names.  I thought that would be self-evident.  I can't remember where the list is on MSFT's website or maybe I just imaged it. Anyway, all I can tell you is that I tried the method on the site in your script and it worked repeatedly.
Title: Re: CLR: WebRequest / headers only
Post by: td on September 29, 2021, 05:06:12 PM
This hack of your script works in repeated tests.

Code (winbatch) Select
Url = 'https://community.idera.com/'
ObjectClrOption('useany', 'System')
oWebUtil = ObjectClrNew('System.Net.WebUtility')


oUri = ObjectClrNew('System.Uri', Url)
oSvcManager = ObjectClrNew('System.Net.ServicePointManager')
protocols = ObjectClrType("System.Net.SecurityProtocolType",3072|768) 
oSvcManager.SecurityProtocol = protocols
oSvcPoint = oSvcManager.FindServicePoint(oUri)


oWebRequest = ObjectClrNew('System.Net.WebRequest')
oRequest = oWebRequest.Create(oUri)
oRequest.Timeout = oRequest.Timeout * 6
;;headers = oRequest.GetResponse().Headers
;gives error here
objResponse = oRequest.GetResponse()
if objResponse.SupportsHeaders then strText = 'Headers supported'
else strText = 'Headers not supported'
Message('Are Headers supported', strText)

header = objResponse.GetResponseHeader('Content-Type')
message('Content-Type Header', header)

Title: Re: CLR: WebRequest / headers only
Post by: stanl on September 30, 2021, 03:00:11 AM
Your hack worked. I probably had a typo error when I tried GetResponseHeader(). I attached the header list for the Idera url. I checked with the person I will be compiling this for and he is only interested in the set-cookie header. That was easy to test with both the Idera url and WB tech db url.  I can adapt the code to a UDF chkheader(url,header). Didn't mean to come off so contentious in my replies. My hack to your hack can test for Set-Cookie
Code (WINBATCH) Select


;CLR - attempt Web Request to get headers only
hdr = 'Set-Cookie'
;comment/uncomment url to test
Url = 'https://community.idera.com/'   ;header should exist
;Url = 'https://techsupt.winbatch.com/'  ;header should not exist
;==============================================================
ObjectClrOption('useany', 'System')
oWebUtil = ObjectClrNew('System.Net.WebUtility')
oUri = ObjectClrNew('System.Uri', Url)
oSvcManager = ObjectClrNew('System.Net.ServicePointManager')
protocols = ObjectClrType("System.Net.SecurityProtocolType",3072|768) 
oSvcManager.SecurityProtocol = protocols
oSvcPoint = oSvcManager.FindServicePoint(oUri)


oWebRequest = ObjectClrNew('System.Net.WebRequest')
oRequest = oWebRequest.Create(Url)
oRequest.Timeout = oRequest.Timeout * 6
objResponse = oRequest.GetResponse()
if objResponse.SupportsHeaders
   header = objResponse.GetResponseHeader(hdr)
   if Strlen(header)==0
      message('Set-Cookie Header', "Does Not Exist")
   else
      message(hdr:' Header', header)
   endif
else
   Message('Are Headers supported','Headers not supported')
endif
objResponse.Close()
Exit
Title: Re: CLR: WebRequest / headers only
Post by: td on September 30, 2021, 09:03:41 AM
For verification purposes, I had gotten a dump of all headers the old-fashioned way.

Code (winbatch) Select
Url = 'https://community.idera.com/'
ObjHttp = ObjectCreate("WinHttp.WinHttpRequest.5.1")

TimeoutVal = 59000
ObjHttp.SetTimeouts(TimeoutVal, TimeoutVal, TimeoutVal, TimeoutVal)   
ObjHttp.Open("GET", Url)
ObjHttp.Send()

strHeaders = ObjHttp.GetAllResponseHeaders()

;; More stuff here.


And yes, it still works on Windows 11.
Title: Re: CLR: WebRequest / headers only
Post by: stanl on September 30, 2021, 03:01:30 PM
touché
Title: Re: CLR: WebRequest / headers only
Post by: td on October 01, 2021, 01:33:05 PM
I confess to an ulterior but benign motive. Our discussions seem to attract a few eyeballs so I thought I would drop a nugget from the past to maybe remind other users that there is more than one way to perform the task.
Title: Re: CLR: WebRequest / headers only
Post by: stanl on October 02, 2021, 02:41:24 AM
Quote from: td on October 01, 2021, 01:33:05 PM
I confess to an ulterior but benign motive. Our discussions seem to attract a few eyeballs so I thought I would drop a nugget from the past to maybe remind other users that there is more than one way to perform the task.


I alluded to 'another' method in my early posts, but WinHttp had slipped my mind although it has served as a swiss army knife for many scripts I have written.  I did end up sending a compiled exe to my friend via the CLR route and I'm sure I will have other CLR questions coming up.