CLR: WebRequest / headers only

[stanl]

Just wondering is there is a way to get around the attached error the script produces on line headers = oWebRequest.GetResponse().Headers

Code: Winbatch

;CLR - attempt Web Request to get headers only
Url = 'https://community.idera.com/'
ObjectClrOption('useany', 'System')
oWebUtil = ObjectClrNew('System.Net.WebUtility')


oUri = ObjectClrNew('System.Uri', Url)
oSvcManager = ObjectClrNew('System.Net.ServicePointManager')
protocols = ObjectClrType("System.Net.SecurityProtocolType",3072|768)  
oSvcManager.SecurityProtocol = protocols
oSvcPoint = oSvcManager.FindServicePoint(oUri)


oWebRequest = ObjectClrNew('System.Net.WebRequest')
oRequest = oWebRequest.Create(oUri)
oRequest.Timeout = oRequest.Timeout * 6
headers = oRequest.GetResponse().Headers
;gives error here


keys = headers.AllKeys
output="key":@tab:"Value":@lf


ForEach k in keys
    output:= k:@tab:headers.GetValues(k):@lf
Next


oResponse.Close()


Message(Url,output)


Exit

 

[stanl]

Oh, and before the put-downs or other arguments start: yes, this does work in powershell

Code: [Select]


$url = 'https://community.idera.com/'
$request = [System.Net.WebRequest]::Create( $Url )
$headers = $request.GetResponse().Headers
$headers.AllKeys |
     Select-Object @{ Name = "Key"; Expression = { $_ }},
     @{ Name = "Value"; Expression = { $headers.GetValues( $_ ) } }


with 2 points:

• doing as a favor for old client who wanted to know urls that contained cookies in headers
• GetResponse() is defined as an 'abstract class' so would be interested in if / how they may work with WB's CDL

no harm / no foul

[td]

That is a bizarre error message. I have no idea why the CLR is regurgitating that exception because the WIL DLL is not attempting to use an IDispatch interface to access anything in your script. The response class is abstract so the behavior of the object returned by GetResponse can vary based on the concreate class returned but the SupportsHeaders property returns true. For example,

Code: Winbatch

objResponse = oRequest.GetResponse()
if objResponse.SupportsHeaders then strText = 'Headers supported'
else strText = 'Headers not supported'
Message('Are Headers supported', strText)
 

Also, the WebHeaderCollection class returned by the Headers property is marked as ComVisibleAttribute so it shouldn't make any difference anyway.  Here's a link to the class:

https://docs.microsoft.com/en-us/dotnet/api/system.net.webheadercollection?view=netframework-4.8

I will have to do a little bit more investigation in an attempt to at least come up with a better explanation of what is going on.

You might be able to use the GetResponseHeader method as an alternative.

https://docs.microsoft.com/en-us/dotnet/api/system.net.httpwebresponse.getresponseheader?view=netframework-4.8

[td]

Oh, and before the put-downs or other arguments start

I was under the impression that we have gotten past that point...

[stanl]

Oh, and before the put-downs or other arguments start

I was under the impression that we have gotten past that point...

like I wrote... no harm / no foul..


but, if you execute the PS code against 'https://forum.winbatch.com/' you will get 403 forbidden error, which is useful info [I'm doing this as a favor for an old client, not for money]...


Just turned out an interesting error.... I think there is a javascript getresponseheaders() function... dunno

[td]

The GetResponseHeader method appears to work. Using your script with the method call GetResponseHeader('Content-Type') I received the header content "text/html: charset=utf-8" which is what you would expect.

[stanl]

The GetResponseHeader method appears to work. Using your script with the method call GetResponseHeader('Content-Type') I received the header content "text/html: charset=utf-8" which is what you would expect.

Understood. But the object was to iterate all header keys not just a single one as web request will return different or different amount of keys. And, in testing I missed your use of the function



oResponse = oRequest.GetResponse()
test = oResponse.GetResponseHeader('Content-Type')
Message("",test)          FAILS

[td]

There is a defined list of headers that are supported by the response object so those would be the only ones you need to test for in a straightforward loop. Or just use a list of common header names.  I thought that would be self-evident.  I can't remember where the list is on MSFT's website or maybe I just imaged it. Anyway, all I can tell you is that I tried the method on the site in your script and it worked repeatedly.

[td]

This hack of your script works in repeated tests.

Code: Winbatch

Url = 'https://community.idera.com/'
ObjectClrOption('useany', 'System')
oWebUtil = ObjectClrNew('System.Net.WebUtility')


oUri = ObjectClrNew('System.Uri', Url)
oSvcManager = ObjectClrNew('System.Net.ServicePointManager')
protocols = ObjectClrType("System.Net.SecurityProtocolType",3072|768)  
oSvcManager.SecurityProtocol = protocols
oSvcPoint = oSvcManager.FindServicePoint(oUri)


oWebRequest = ObjectClrNew('System.Net.WebRequest')
oRequest = oWebRequest.Create(oUri)
oRequest.Timeout = oRequest.Timeout * 6
;;headers = oRequest.GetResponse().Headers
;gives error here
objResponse = oRequest.GetResponse()
if objResponse.SupportsHeaders then strText = 'Headers supported'
else strText = 'Headers not supported'
Message('Are Headers supported', strText)

header = objResponse.GetResponseHeader('Content-Type')
message('Content-Type Header', header)
 

[stanl]

Your hack worked. I probably had a typo error when I tried GetResponseHeader(). I attached the header list for the Idera url. I checked with the person I will be compiling this for and he is only interested in the set-cookie header. That was easy to test with both the Idera url and WB tech db url.  I can adapt the code to a UDF chkheader(url,header). Didn't mean to come off so contentious in my replies. My hack to your hack can test for Set-Cookie

Code: Winbatch

;CLR - attempt Web Request to get headers only
hdr = 'Set-Cookie'
;comment/uncomment url to test
Url = 'https://community.idera.com/'   ;header should exist
;Url = 'https://techsupt.winbatch.com/'  ;header should not exist
;==============================================================
ObjectClrOption('useany', 'System')
oWebUtil = ObjectClrNew('System.Net.WebUtility')
oUri = ObjectClrNew('System.Uri', Url)
oSvcManager = ObjectClrNew('System.Net.ServicePointManager')
protocols = ObjectClrType("System.Net.SecurityProtocolType",3072|768)  
oSvcManager.SecurityProtocol = protocols
oSvcPoint = oSvcManager.FindServicePoint(oUri)


oWebRequest = ObjectClrNew('System.Net.WebRequest')
oRequest = oWebRequest.Create(Url)
oRequest.Timeout = oRequest.Timeout * 6
objResponse = oRequest.GetResponse()
if objResponse.SupportsHeaders
   header = objResponse.GetResponseHeader(hdr)
   if Strlen(header)==0
      message('Set-Cookie Header', "Does Not Exist")
   else
      message(hdr:' Header', header)
   endif
else
   Message('Are Headers supported','Headers not supported')
endif
objResponse.Close()
Exit
 

[td]

For verification purposes, I had gotten a dump of all headers the old-fashioned way.

Code: Winbatch

Url = 'https://community.idera.com/'
ObjHttp = ObjectCreate("WinHttp.WinHttpRequest.5.1")

TimeoutVal = 59000
ObjHttp.SetTimeouts(TimeoutVal, TimeoutVal, TimeoutVal, TimeoutVal)  
ObjHttp.Open("GET", Url)
ObjHttp.Send()

strHeaders = ObjHttp.GetAllResponseHeaders()

;; More stuff here.
 

And yes, it still works on Windows 11.

[stanl]

touché

[td]

I confess to an ulterior but benign motive. Our discussions seem to attract a few eyeballs so I thought I would drop a nugget from the past to maybe remind other users that there is more than one way to perform the task.

[stanl]

I confess to an ulterior but benign motive. Our discussions seem to attract a few eyeballs so I thought I would drop a nugget from the past to maybe remind other users that there is more than one way to perform the task.

I alluded to 'another' method in my early posts, but WinHttp had slipped my mind although it has served as a swiss army knife for many scripts I have written.  I did end up sending a compiled exe to my friend via the CLR route and I'm sure I will have other CLR questions coming up.