WinBatch® Technical Support Forum

All Things WinBatch => WinBatch => Topic started by: td on October 12, 2021, 02:48:45 PM

Title: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 12, 2021, 02:48:45 PM
The makeover version?

Version 44004  Oct 12, 2021 (Experimental release.)
     
      Changed the names of the jsObjNew, jsObjMap,
      jsObjClose to jsConNew, jsConMap, and
      jsConClose respectively.

      Modified the jsParse to place top-level
      JSON arrays in the extender's container
      table along with JSON objects.
     
      Modified the jsKeyPaths, jsValueAdd,
      jsValueGet and jsValueType to handle
      JSON arrays as JSON containers.

This pre-release will break most existing scripts written to use this extender.

Unleashing half-baked software on unsuspecting users has been an interesting test of the latest fad in software development but not one that necessarily warrants repeating. But at least I have a better insight into why Windows 11 is so buggy.

It is available at the usual link:

https://files.winbatch.com/downloads/wb/ilcjs44i.zip
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: JTaylor on October 12, 2021, 05:59:27 PM
Should the following work?

Code (winbatch) Select


xtxt = '{"returnCode":0,"searchResultsKey":-2100454365,"numberOfResults":0}' 


jnode = jsParse(xtxt)


  If jsValueGet(jnode,"numberOfResults") == 0  Then Return

Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 12, 2021, 10:59:48 PM
It does for me. The jsValueGet function returns the value of the "numberOfResults" key which is "0" so the "Return" statement is processed.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: stanl on October 13, 2021, 02:51:25 AM
I left off part 2 with a url that returned a Json array - if I get time this weekend I'll try the latest as the script associated with that post failed
Code (Winbatch) Select


;Winbatch 2021C -
;uses  WinHttp.WinHttpRequest.5.1 for JSON return
;Stan Littlefield, September 14, 2021
;Updated: October 10, 2021 - based on new Extender using array rather than delimited list
;======================================================================================================
IntControl(73,1,0,0,0)
Gosub udfs
AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')
cUrl = "https://www.colourlovers.com/api/colors/top?format=json&numResults=100"
BoxOpen("Parsing:":cUrl,"Please Wait...")
cFile = Dirscript():"ColorsAPI.txt"
cJson = Dirscript():"ColorsAPI.json"
If FileExist(cFile) Then FileDelete(cFile)
If FileExist(cJson) Then FileDelete(cJson)       
request = Createobject("WinHttp.WinHttpRequest.5.1")
request.Open("GET", cUrl, @False )
request.SetRequestHeader("Content-Type", "application/x-www-form-urlencoded")
request.SetRequestHeader("Accept", "application/json")
request.Send()
jdata = request.ResponseText
request = 0


;persist .json file
FilePut(cJson,jdata) ;uncomment if needed
;=================================
If jsValid(jdata)
   ;jstypes = crtypes()  ;no need for map, not persisting node types
   Object = jsParse(jdata)
   Drop(jdata)
   BoxTitle("Creating text Output ":cFile )
   crTree()   ;script will fail as array canot be created with JsParse()
   ;crObj()
   jsObjClose()
   BoxShut()
   Message("Json Tree Created",cFile)
Else 
   jsObjClose()
   Drop(jdata)
   BoxShut()
   Message(cJson,"Not Valid Json")
Endif


Exit


:WBERRORHANDLER
geterror()
Terminate(@TRUE,"Error Encountered",errmsg)
Exit


:udfs
#DefineSubRoutine geterror()
   wberroradditionalinfo = wberrorarray[6]
   lasterr = wberrorarray[0]
   handlerline = wberrorarray[1]
   textstring = wberrorarray[5]
   linenumber = wberrorarray[8]
   errmsg = "Error: ":lasterr:@LF:textstring:@LF:"Line (":linenumber:")":@LF:wberroradditionalinfo
   Return(errmsg)
#EndSubRoutine


#DefineSubRoutine crTree()
   Fx = FileOpen(cFile,"Write")
   output = "node":@tab:"value"
   FileWrite(Fx,output)
   tree =jsKeyPaths(Object,"")
   ;tree =jsKeyPaths(Object,"variables")
   ;just for test
   ;Message("Variable Type for tree",VarType(tree)) ;would not expect blank return
   Message("tree",ObjectTypeGet(tree))


   Cnt = Arrinfo(tree,1)
   nMod = Int(Cnt/10)
   i=1
   ForEach item in tree
      If jsValueType(Object,item) <> @JsonObj
         output = item:@tab:jsValueGet(Object,item)
         If (i mod nMod == 0) Then BoxText("Processing ":i:" of ":Cnt)
         FileWrite(Fx,output)
         i +=1
         If i>10000 Then Break
      Endif
   Next
   FileClose(Fx)
   Drop(Fx)
   Drop(tree)
   Return(1)
#EndSubRoutine


#DefineSubRoutine crObj()
   Fx = FileOpen(cFile,"Write")
   output = "node":@tab:"value"
   FileWrite(Fx,output)
   tree =jsKeyPaths(Object,"")
   Cnt = Arrinfo(tree,1)
   nMod = Int(Cnt/10)
   i=1
   brk=@False
   ForEach item in tree
      If jsValueType(Object,item) == @JsonObj
         subtree =jsKeyPaths(Object,item)
         ForEach subitem in subtree
            output = item:@tab:jsValueGet(Object,subitem)
            FileWrite(Fx,output)
            brk=@True
         Next
      Endif
      If brk Then Break
   Next
   FileClose(Fx)
   Drop(Fx)
   Drop(tree)
   Return(1)
#EndSubRoutine




#DefineSubRoutine crtypes()
maps = $"1,NULL
2,BOOL
4,NUMBER
8,STRING
16,ARRAY
32,OBJECT
$"
jstypes = MapCreate(maps,',',@lf)
Return(jstypes)
#EndSubRoutine


Return
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: JTaylor on October 13, 2021, 05:12:29 AM
Well....of course now it works.   Not sure what was going on.   Probably spent an hour trying this and variations but kept get errors about not being able to convert to a number or a string, depending on what I was doing.

Thanks.

Jim
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 13, 2021, 07:54:28 AM
Quote from: JTaylor on October 13, 2021, 05:12:29 AM
Well....of course now it works.   Not sure what was going on.   Probably spent an hour trying this and variations but kept get errors about not being able to convert to a number or a string, depending on what I was doing.

Thanks.

Jim

WinBatch is and always has been negatively typed. That means that relational operands are type-matched before the operation is applied. In this case the result of the call the jsValueGet function is converted from a string to a number and then compared to the integer literal on the right-hand side of the equals comparison operator.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 13, 2021, 08:02:14 AM
Quote from: stanl on October 13, 2021, 02:51:25 AM
I left off part 2 with a url that returned a Json array - if I get time this weekend I'll try the latest as the script associated with that post failed
...

You need the change the name of the jsObjClose function to jsConClose but outside of that, your script works just fine.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: stanl on October 14, 2021, 02:53:43 AM
Quote from: td on October 13, 2021, 08:02:14 AM
You need the change the name of the jsObjClose function to jsConClose but outside of that, your script works just fine.


Yes, it did. Got to do some thinking about handling the node values.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 16, 2021, 09:42:47 AM
Quote from: JTaylor on October 13, 2021, 05:12:29 AM
Well....of course now it works.   Not sure what was going on.   Probably spent an hour trying this and variations but kept get errors about not being able to convert to a number or a string, depending on what I was doing.

Thanks.

Jim

Neglected to mention that the type of error you encounter likely has very little to do with the extender other than possiblely placing parameters into the extender's functions in the wrong order or with incorrect content.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: JTaylor on October 16, 2021, 09:47:42 AM
Possibly.  As far as I know I didn't change what I posted and it worked after posting.  Obviously something changed but no idea what.   Don't recall changing anything in my script.

Jim
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 16, 2021, 11:02:31 PM
We didn't magically reach out and touch your system so something on your end changed.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: JTaylor on October 27, 2021, 10:22:20 AM
Haven't embarrassed myself lately so thought I would give it a try here...

Would someone be so kind as to help me figure out how to extract data from this JSON response?   I have tried many things and this is one of my latest attempts.   Originally the problem was the extender,  I think, but this latest release supposedly resolved that issue but still no joy.   Feels like one of those things where I am missing the obvious.

Thanks.

jim


Code (winbatch) Select


  AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')
xtxt = `{"returnCode":0,"searchResultsKey":1457977769,"numberOfResults":1,"blockOfResults":[{"numberCopiesTotal":1,"numberCopiesAvailable":0,"numberCopiesReference":0,"numberCopiesReserved":0,"resultsIndex":1,"baseResourceSn":76449,"baseResourceKey":409560567,"title":"Artificial unintelligence : how computers misunderstand the world","edition":"First MIT Press paperback edition","author":"Broussard, Meredith","callNumber":"303.4834 B876","isbn":"9780262537018","hasLocalImage":false,"mediaDescription":"Book","mediaCategory":"Non-Fiction","location":"General Collection","notes":"First published in hardback in 2018.","series":[{"sn":73323,"seriesName":"Cambridge, Massachusetts"},{"sn":268476,"seriesName":"The MIT Press"},{"sn":274366,"seriesName":"Informational works"}],"rating":0,"reviewCount":0,"publisherName":"The MIT Press","publisherPlace":"Cambridge, Massachusetts","publisherYear":"2019","subjects":[{"value1":274365,"value2":"Computer programs - Correctness","sortByIntFirst":false},{"value1":274364,"value2":"Electronic data processing - Social aspects","sortByIntFirst":false},{"value1":58603,"value2":"Errors","sortByIntFirst":false}],"summaryText":"\"In Artificial Unintelligence, Meredith Broussard argues that our collective enthusiasm for applying computer technology to every aspect of life has resulted in a tremendous amount of poorly designed systems. We are so eager to do everything digitally--hiring, driving, paying bills, even choosing romantic partners--that we have stopped demanding that our technology actually work...","readingLists":[],"genre":"Informational works","physicalDesc":"237 pages : illustrations ;23 cm","hasPRC":false,"hasAR":false,"collIconSN":15,"AvailText":"0","AvailColour":"#FF0000"}]}`

  jnode = jsParse(xtxt)

  If jsValueGet(jnode,"numberOfResults") == 0  Then Return

  jitems        = jsKeyPaths(jnode,"blockOfResults")
message("CNT",ArrInfo(jitems,1))
  Foreach jitem in jitems
    bib          = jsValueGet(jitem,"baseResourceKey")
message(" BIB",bib)
  Next


Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: stanl on October 27, 2021, 01:20:35 PM
On the surface nothing is wrong and it is valid Json.... but a strange error for sure. I'll do some more playing around.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 27, 2021, 01:30:08 PM
 
Code (winbatch) Select
AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')
xtxt = `{"returnCode":0,"searchResultsKey":1457977769,"numberOfResults":1,"blockOfResults":[{"numberCopiesTotal":1,"numberCopiesAvailable":0,"numberCopiesReference":0,"numberCopiesReserved":0,"resultsIndex":1,"baseResourceSn":76449,"baseResourceKey":409560567,"title":"Artificial unintelligence : how computers misunderstand the world","edition":"First MIT Press paperback edition","author":"Broussard, Meredith","callNumber":"303.4834 B876","isbn":"9780262537018","hasLocalImage":false,"mediaDescription":"Book","mediaCategory":"Non-Fiction","location":"General Collection","notes":"First published in hardback in 2018.","series":[{"sn":73323,"seriesName":"Cambridge, Massachusetts"},{"sn":268476,"seriesName":"The MIT Press"},{"sn":274366,"seriesName":"Informational works"}],"rating":0,"reviewCount":0,"publisherName":"The MIT Press","publisherPlace":"Cambridge, Massachusetts","publisherYear":"2019","subjects":[{"value1":274365,"value2":"Computer programs - Correctness","sortByIntFirst":false},{"value1":274364,"value2":"Electronic data processing - Social aspects","sortByIntFirst":false},{"value1":58603,"value2":"Errors","sortByIntFirst":false}],"summaryText":"\"In Artificial Unintelligence, Meredith Broussard argues that our collective enthusiasm for applying computer technology to every aspect of life has resulted in a tremendous amount of poorly designed systems. We are so eager to do everything digitally--hiring, driving, paying bills, even choosing romantic partners--that we have stopped demanding that our technology actually work...","readingLists":[],"genre":"Informational works","physicalDesc":"237 pages : illustrations ;23 cm","hasPRC":false,"hasAR":false,"collIconSN":15,"AvailText":"0","AvailColour":"#FF0000"}]}`

jnode = jsParse(xtxt)

If jsValueGet(jnode,"numberOfResults") == 0  Then Return

jitems        = jsKeyPaths(jnode,"baseResourceKey")
message("CNT",ArrInfo(jitems,1))
Foreach jitem in jitems
    bib          = jsValueGet(jnode, jitem)
    message(" BIB",bib)
Next


Have either of you downloaded the release version and read the documentation yet?

[edit] The extender release version does not have any coding changes from this experimental version. The only changes are to the documentation; the discussion of JSON paths was broken out into its own help file topic.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: JTaylor on October 27, 2021, 01:44:46 PM
Hmmmmmm...thought I had the latest.   Looking but don't see a post for an official release, although that sounds vaguely familiar.   I have read through the documentation I do have.

This works.  Thanks.   

Jim
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 27, 2021, 02:30:22 PM
Didn't post a release notice because I hadn't gotten around to it yet. As mentioned the only changes were minor improvements to the help file (and the extender version number, of course.)
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 27, 2021, 05:24:57 PM
The extender is fairly flexible. If you like doing things the hard way here you go.

Code (winbatch) Select
AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')
xtxt = `{"returnCode":0,"searchResultsKey":1457977769,"numberOfResults":1,"blockOfResults":[{"numberCopiesTotal":1,"numberCopiesAvailable":0,"numberCopiesReference":0,"numberCopiesReserved":0,"resultsIndex":1,"baseResourceSn":76449,"baseResourceKey":409560567,"title":"Artificial unintelligence : how computers misunderstand the world","edition":"First MIT Press paperback edition","author":"Broussard, Meredith","callNumber":"303.4834 B876","isbn":"9780262537018","hasLocalImage":false,"mediaDescription":"Book","mediaCategory":"Non-Fiction","location":"General Collection","notes":"First published in hardback in 2018.","series":[{"sn":73323,"seriesName":"Cambridge, Massachusetts"},{"sn":268476,"seriesName":"The MIT Press"},{"sn":274366,"seriesName":"Informational works"}],"rating":0,"reviewCount":0,"publisherName":"The MIT Press","publisherPlace":"Cambridge, Massachusetts","publisherYear":"2019","subjects":[{"value1":274365,"value2":"Computer programs - Correctness","sortByIntFirst":false},{"value1":274364,"value2":"Electronic data processing - Social aspects","sortByIntFirst":false},{"value1":58603,"value2":"Errors","sortByIntFirst":false}],"summaryText":"\"In Artificial Unintelligence, Meredith Broussard argues that our collective enthusiasm for applying computer technology to every aspect of life has resulted in a tremendous amount of poorly designed systems. We are so eager to do everything digitally--hiring, driving, paying bills, even choosing romantic partners--that we have stopped demanding that our technology actually work...","readingLists":[],"genre":"Informational works","physicalDesc":"237 pages : illustrations ;23 cm","hasPRC":false,"hasAR":false,"collIconSN":15,"AvailText":"0","AvailColour":"#FF0000"}]}`

jnode = jsParse(xtxt)

If jsValueGet(jnode,"numberOfResults") == 0  Then Return

jsPaths = jsKeyPaths(jnode,"blockOfResults")
Foreach jsPath in jsPaths
   jsObjElem = jsValueGet(jnode, jsPath ) 
   jsArrPaths = jsKeyPaths(jsObjElem,"baseResourceKey")
   Foreach jsElemPath in jsArrPaths
      bib       = jsValueGet(jsObjElem, jsElemPath)
      message(" BIB",bib)
  Next
Next


Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: JTaylor on October 27, 2021, 06:33:28 PM
Thanks again.   I didn't think it was too hard but just wasn't getting any traction and needed a bit of a push. 

Jim
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: stanl on October 28, 2021, 02:39:42 AM
Quote from: td on October 27, 2021, 01:30:08 PM
Have either of you downloaded the release version and read the documentation yet?



I pasted your code over what I originally cut out of Jim's post - worked fine.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: JTaylor on October 28, 2021, 09:14:41 AM
What is the most efficient way to determine if an element exists?    In the JSON I already posted some elements do not always exist.   

Do I need to use jsKeyPaths() and then check for an empty array?   Thought there they was something like jsKeyExist() but not seeing it now.

Depending on answer, a couple of requests would be for a jsKeyExist() function, as well as a default value for jsValueGet(), for when there is no path/key.   I have found that VERY useful in my work.   I deal with a lot of data and very often if there is no value I just want a blank value returned without having to check whether that path/key exists or not.  So in the following, rather than an error when edition doesn't exist, it would return a blank string.  If that final default parameter doesn't exist it would return an error, as it does now.


edition      = jsValueGet(jnode,"blockOfResults[0].edition",".","")


Thanks.

Jim
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 28, 2021, 01:21:51 PM
The function jsKeyPaths is very efficient at it uses a hash algorithm to identify JSON data content. You could wrap it in a subroutine if you like the name jsKeyExist.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 28, 2021, 01:40:02 PM
Code (winbatch) Select
#definesubroutine jsKeyExist(_handle,_Key)
   return Arrinfo(jsKeyPaths(_handle, _key), 1)
#endsubroutine

AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')

;; From JSON extender help file.
jsHandle = jsParse('{ "color": "red", "rgba": [255,0,0,1]}')

;; This can still error if the key contains path information and
;; it is syntactically malformed.
Exists = jsKeyExist(jsHandle, "bob")

exit

Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: JTaylor on October 28, 2021, 01:44:23 PM
Just seems like a useful and relevant function that many would appreciate having.   Having implemented the default GetValue idea in my own stuff I can also say having that option would be REALLY REALLY appreciated and much used, at least by me :)    Always nice not having to maintain such subroutines for things that make sense being native as it keeps a 1,000 people from writing duplicate code by having it done once in the Extender.

In any event,  just trying to make things better and more useful.  Thanks.

Jim
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: JTaylor on October 28, 2021, 02:08:31 PM
I am sure it is probably just me but if do something like:

Message("BIB",jsKeyExist(jnode,"[blockOfResults][%x%].[baseResourceKey]"))


I get 0 as the response.   If I use those some values in jsValueGet() I get a response.  Should it work for that path?

Jim
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: stanl on October 29, 2021, 04:35:36 AM
I'm a little confused with the response I get back here, building on the jsKeyExist() udf.
Code (WINBATCH) Select


#definesubroutine jsKeyExist(_handle,_Key)
   return Arrinfo(jsKeyPaths(_handle, _key), 1)
#endsubroutine


AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')


;; From JSON extender help file.
jsHandle = jsParse('{ "color": "red", "rgba": [255,0,0,1]}')


;; This can still error if the key contains path information and
;; it is syntactically malformed.
cKey = "rgba"
If jsKeyExist(jsHandle,cKey)
   Paths = jsKeyPaths(jsHandle,cKey)
   Values = ''
   ForEach element In Paths
     Values := jsValueGet(jsHandle,element):@lf
   Next
   Message(cKey,Values)
Else
   Message(cKey,"Key Does Not Exist")
Endif
jsConClose(jsHandle)


exit
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 29, 2021, 06:56:19 AM
"Rgba" is a JSON array so it is treated as a container by the extender. All containers have an extender JSON handle for a value. Extender JSON handles can use with any extender function with a "container-handle" parameter.

FWIW, "container" is a computer science term for a type of abstract data structure:

https://en.wikipedia.org/wiki/Container_%28abstract_data_type%29 (https://en.wikipedia.org/wiki/Container_%28abstract_data_type%29)

Also since you are using a "foreach" loop to cycle through an array of paths you don't need to check for the existence of the key before doing it. The loop will simply not iterate if the array is empty.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: JTaylor on October 29, 2021, 07:20:26 AM
So you are saying that it won't work with what I am needing to do? 

Jim
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 29, 2021, 07:32:57 AM
Not sure what you mean by "it" but no, not saying anything about something not working. Quite the opposite.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: JTaylor on October 29, 2021, 07:56:44 AM
I would expect the following to return a 1 but apparently I am not understanding.

Jim


Code (winbatch) Select


AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')

#definesubroutine jsKeyExist(_handle,_Key)
   Return Arrinfo(jsKeyPaths(_handle, _key), 1)
#endsubroutine

#DefineSubroutine jsValueGetDefault(_handle,_Key,_Delimiter,_Default)
    Message(Arrinfo(jsKeyPaths(_handle, _key), 1),jsValueGetDefault(_handle, _key, _Delimiter,""))
   If Arrinfo(jsKeyPaths(_handle, _key), 1) > 0 Then
     Return jsValueGet(_handle, _key, _Delimiter)
   Else
     Message("DEFAULT","")
     Return _Default
   EndIf
#EndSubroutine

xtxt = `{"returnCode":0,"searchResultsKey":1457977769,"numberOfResults":1,"blockOfResults":[{"numberCopiesTotal":1,"numberCopiesAvailable":0,"numberCopiesReference":0,"numberCopiesReserved":0,"resultsIndex":1,"baseResourceSn":76449,"baseResourceKey":409560567,"title":"Artificial unintelligence : how computers misunderstand the world","edition":"First MIT Press paperback edition","author":"Broussard, Meredith","callNumber":"303.4834 B876","isbn":"9780262537018","hasLocalImage":false,"mediaDescription":"Book","mediaCategory":"Non-Fiction","location":"General Collection","notes":"First published in hardback in 2018.","series":[{"sn":73323,"seriesName":"Cambridge, Massachusetts"},{"sn":268476,"seriesName":"The MIT Press"},{"sn":274366,"seriesName":"Informational works"}],"rating":0,"reviewCount":0,"publisherName":"The MIT Press","publisherPlace":"Cambridge, Massachusetts","publisherYear":"2019","subjects":[{"value1":274365,"value2":"Computer programs - Correctness","sortByIntFirst":false},{"value1":274364,"value2":"Electronic data processing - Social aspects","sortByIntFirst":false},{"value1":58603,"value2":"Errors","sortByIntFirst":false}],"summaryText":"\"In Artificial Unintelligence, Meredith Broussard argues that our collective enthusiasm for applying computer technology to every aspect of life has resulted in a tremendous amount of poorly designed systems. We are so eager to do everything digitally--hiring, driving, paying bills, even choosing romantic partners--that we have stopped demanding that our technology actually work...","readingLists":[],"genre":"Informational works","physicalDesc":"237 pages : illustrations ;23 cm","hasPRC":false,"hasAR":false,"collIconSN":15,"AvailText":"0","AvailColour":"#FF0000"}]}`

jnode = jsParse(xtxt)

If jsValueGet(jnode,"numberOfResults") == 0  Then Return
   bib       = jsValueGet(jnode,"blockOfResults[0].baseResourceKey")
Message(" BIB",bib)

Message("Bib Key Exist?",jsKeyExist(jnode,"[blockOfResults][0].[baseResourceKey]"))

jsConClose(jnode)



Jim
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 29, 2021, 08:19:58 AM
The output would depend on the input...


[edit] Also note the what is expected in the jsGetValue parameters second parameter.

https://docs.winbatch.com/mergedProjects/Json/jsValueGet.htm (https://docs.winbatch.com/mergedProjects/Json/jsValueGet.htm)
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: JTaylor on October 29, 2021, 08:59:39 AM
Any chance of getting a clear answer on this one?   Should the jsKeyExist() function return a 1 or 0 in this instance?   Not sure how many different ways I can ask the same question and we probably both have better things to do with our time.  Thanks.

Jim
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 29, 2021, 09:08:37 AM
Patience. You have found a bug in the jsKeyPaths function. Will try to get an updated extender out today.

Nice data set by the way. Will have to include that in the extender test suite.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: stanl on October 29, 2021, 09:17:21 AM
Quote from: td on October 29, 2021, 06:56:19 AM
"Rgba" is a JSON array so it is treated as a container by the extender. All containers have an extender JSON handle for a value. Extender JSON handles can use with any extender function with a "container-handle" parameter.

FWIW, "container" is a computer science term for a type of abstract data structure:

https://en.wikipedia.org/wiki/Container_%28abstract_data_type%29 (https://en.wikipedia.org/wiki/Container_%28abstract_data_type%29)

Also since you are using a "foreach" loop to cycle through an array of paths you don't need to check for the existence of the key before doing it. The loop will simply not iterate if the array is empty.


No offense, but I thought rgba was a key, i..e jsKeyExist().... and the foreach loop is taken directly from the help file for jsValueGet(). I was thinking the Extender could pull back the map value, i.e. key:value if jsKeyExists() returned @TRUE.... Point is: If keyexistat => return key value(s).... so maybe a SearchKey() function is worth asking about.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 29, 2021, 09:42:17 AM
Quote from: stanl on October 29, 2021, 09:17:21 AM

No offense, but I thought rgba was a key, i..e jsKeyExist().... and the foreach loop is taken directly from the help file for jsValueGet(). I was thinking the Extender could pull back the map value, i.e. key:value if jsKeyExists() returned @TRUE.... Point is: If keyexistat => return key value(s).... so maybe a SearchKey() function is worth asking about.

Rather induce more typos... a quote from the help file:

"This function retrieves the value of a key/value pair contained within the passed in JSON object, one of its child objects, in a JSON array or a JSON array element contained within the container or sub container. If the key's value is a JSON object (@JsonObj), the returned value will be a JSON extender object handle. If the returned key's value is a JSON array (@JsonArr), and a bracketed array index is not a postfix to the key name, the return value is a container-handle representing the array."

Also for the JSON path discussion in the help file:

"Important: JSON objects and arrays are collectively referred to as containers in the extender's documentation."
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 29, 2021, 09:49:01 AM
Quote from: JTaylor on October 29, 2021, 08:59:39 AM
Any chance of getting a clear answer on this one?   Should the jsKeyExist() function return a 1 or 0 in this instance?   Not sure how many different ways I can ask the same question and we probably both have better things to do with our time.  Thanks.

I guess this is more a documentation problem than a bug. The idea of jsKeyPaths is to specify the key name without any path information other than an index for an array and name brackets. And the way to narrow the search is to use a specific container handle. However, being able to narrow the search using the key name parameter seems like a worthy modification.  Will try to get that change made.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: stanl on October 29, 2021, 10:15:23 AM
Quote from: td on October 29, 2021, 09:42:17 AM
Quote from: stanl on October 29, 2021, 09:17:21 AM


No offense, but I thought rgba was a key, i..e jsKeyExist().... and the foreach loop is taken directly from the help file for jsValueGet(). I was thinking the Extender could pull back the map value, i.e. key:value if jsKeyExists() returned @TRUE.... Point is: If keyexistat => return key value(s).... so maybe a SearchKey() function is worth asking about.

Rather induce more typos... a quote from the help file:

"This function retrieves the value of a key/value pair contained within the passed in JSON object, one of its child objects, in a JSON array or a JSON array element contained within the container or sub container. If the key's value is a JSON object (@JsonObj), the returned value will be a JSON extender object handle. If the returned key's value is a JSON array (@JsonArr), and a bracketed array index is not a postfix to the key name, the return value is a container-handle representing the array."

Also for the JSON path discussion in the help file:

"Important: JSON objects and arrays are collectively referred to as containers in the extender's documentation."


So.. if jsKeyExist() there is no way to immediately return the key value(s)...  This is not an immediate issue as I am working with a Json extract with defined keys... but for testing with any nuances of the data I receive, I'll stick with a PS alternative.


[EDIT}


$json = '{ "color": "red", "rgba": "255,0,0,1"}'
$value = ($json | ConvertFrom-Json).rgba
$value
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 29, 2021, 01:22:39 PM
Not sure what you mean by "if jsKeyExist() there is no way to immediately return the key value(s)...". The purpose of the subroutine isn't to return values.

If you are referring to jsValueGet, you can return any value you want, including arrays and objects. 
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: JTaylor on October 29, 2021, 11:18:29 PM
Thank you.   Since you knew what I was doing and offered a solution I just assumed it should work so was trying to figure out what I might be doing wrong.   I figured the problem was what it turned out to be but wanted to make sure.    This would be helpful but, again, what would be REALLY helpful is the default value option for jsValueGet() and adding a jsKeyExist() function so we all don't have to maintain a subroutine for something I would think most everyone would find very useful would be great :)

Yes. I know I can write a subroutine and already have and plan to post it but can't use it until this other is resolved anyway.  Guess I could if I wanted to do it via error trapping but if you are working on it I will wait.

Thanks again.

Jim
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: stanl on October 30, 2021, 04:17:10 AM
Quote from: td on October 29, 2021, 01:22:39 PM
Not sure what you mean by "if jsKeyExist() there is no way to immediately return the key value(s)...". The purpose of the subroutine isn't to return values.

If you are referring to jsValueGet, you can return any value you want, including arrays and objects.


Understood. The Json I work with is more or less database records, so I use "parent" and "elements"  where the parent can be iterated to it's keys and each key can return one or more elements which are values, skipping a "container" type, i.e. object or array when applicable. Of course, I need to fit my way of thinking to the Extender rather than the other way around.  ;D


.... but for chuckles


$json = '{ "color": "red", "rgba": [255,0,0,1]}'
$keys = ($json | ConvertFrom-Json).psobject.properties.name
$key = "rgba"
if ($keys -match $key) {
   Write-Host $key" exists"
   Write-Host ($json | ConvertFrom-Json).$key
   }
else {
   Write-Host $key" does not exists" }


#should return
#rgba exists
#255 0 0 1


($json | ConvertFrom-Json)|get-member


#should return
#  TypeName: System.Management.Automation.PSCustomObject


#Name        MemberType   Definition                   
#----        ----------   ----------                   
#Equals      Method       bool Equals(System.Object obj)
#GetHashCode Method       int GetHashCode()             
#GetType     Method       type GetType()               
#ToString    Method       string ToString()             
#color       NoteProperty string color=red             
#rgba        NoteProperty Object[] rgba=System.Object[]  <= consistent with Extender


Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 30, 2021, 12:05:03 PM
Just for fun and not trying to prove anything but here is a WinBatch somewhat equivalent script. It is an oversimplification but it's Saturday and I have an outdoor project to get to because the sun is actually shining today. 

Code (winbatch) Select
AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')

values = ''
json = '{ "color": "red", "rgba": [255,0,0,1]}'
jshandle = jsParse(json)
key = 'rgba'
path = jsKeyPaths(jshandle, key)
if !!ArrInfo(path, 1) 
   jsArray = jsValueGet(jshandle, path[0])
   paths = jsKeyPaths(jsArray, '')
   foreach elem in paths
      values := jsValueGet(jsArray, elem):','
   next
endif

if values == '' then text = 'Does not exit'
else text = 'Exists':@lf: values

message (key, text)
exit
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: stanl on October 30, 2021, 12:53:39 PM
yes.. somewhat equivalent is an operative term.... My main point was going directly from if key exists to element value [not its quasi-datatype]. I never said the extender was incapable of that, just more hoops than I cared to jump.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 30, 2021, 04:56:57 PM
Not sure that I would agree with the "extra hoops" statement as it is more or less a trivial task. But that is just my subjective opinion.  The extender was originally written to return delimited lists of elements but that presents some issues with regard to nested arrays and embedded delimiters. It would be nice to return a WIL array but WIL arrays cannot have arrays as elements which leads to an inconsistent approach to arrays. I suppose it would be possible to add a function to the extender that returns JSON arrays as delimited lists.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: stanl on October 31, 2021, 03:00:01 AM
Quote from: td on October 30, 2021, 04:56:57 PM
Not sure that I would agree with the "extra hoops" statement as it is more or less a trivial task. But that is just my subjective opinion.  The extender was originally written to return delimited lists of elements but that presents some issues with regard to nested arrays and embedded delimiters. It would be nice to return a WIL array but WIL arrays cannot have arrays as elements which leads to an inconsistent approach to arrays. I suppose it would be possible to add a function to the extender that returns JSON arrays as delimited lists.


Again agreed. To be clear, I already have a working compiled script with the previous release of the Extender. It works against a known Json structure from an API query. It is fast and moves the Json into Excel.  My questions here are moot in terms of my actual need... more just exploring other Json returns for Extender possibilities. It is unfair to set up what looks like a PS<>WB competition. In PS everything is an Object, so the dot notation ($json | ConvertFrom-Json).$key once understood is pretty intuitive.


But in your example - if your jskeyexist() for "color" returns "red", but for "rgba" returns a nested array which has to be processed with a subsequent call to jskeypaths().  Maybe a dumb question - but what if jskeyexist() would still return a 0 that the key didn't exist but return the jsvaluetype() if it did?


[EDIT]
Code (WINBATCH) Select


#definesubroutine jsKeyExist(_handle,_Key)
   retval = Arrinfo(jsKeyPaths(_handle, _key), 1)
   If retval > 0 Then retval = jsValueType(_handle, _key)
   return retval
#endsubroutine
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on October 31, 2021, 08:36:19 AM
The idea has merit because knowing that a data item exists and its type is connected from the usage perspective.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: stanl on November 01, 2021, 03:11:23 AM
Quote from: td on October 31, 2021, 08:36:19 AM
The idea has merit because knowing that a data item exists and its type is connected from the usage perspective.


Here is a shot:
Not sure why some test keys error as they are valid keys.
Code (WINBATCH) Select


IntControl(73,1,0,0,0)
Gosub udfs
AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')
JsonData = $"{
    "version":: "1.0",
    "dependencies":: {
      "Microsoft.NETCore.UniversalWindowsPlatform":: "5.2.2",
      "Microsoft.Toolkit.Uwp.Notifications":: "1.1.0",
      "QueryString.NET":: "1.0.0"
    },
    "frameworks":: {
      "uap10.0":: {}
    },
    "runtimes":: {
      "win10-arm":: {},
      "win10-arm-aot":: {},
      "win10-x86":: {},
      "win10-x86-aot":: {},
      "win10-x64":: {},
      "win10-x64-aot":: {}
    }
  }$"


If ! jsValid(JsonData) Then Terminate(@TRUE,"Cannot Continue","Invalid Json")
jsHandle = jsParse(JsonData)
;; This can still error if the key contains path information and
;; it is syntactically malformed.
;try these
;cKey = "version"   ;should return 1.0
;cKey = "runtimes"  ;should return Json Object
;cKey = "QueryString.NET" ;should error
cKey = "uap10.0"   ;should error
result = jsKeyExist(jsHandle,cKey)
If result == 0 Then Message(cKey,"Does Not Exist")
If result >1 && result <=8
   Message(cKey,jsValueGet(jsHandle,cKey))
Endif
If result == 16
   path = jsKeyPaths(jsHandle,cKey)
   values = ''
   jsArray = jsValueGet(jshandle, path[0])
   paths = jsKeyPaths(jsArray, '')
   foreach elem in paths
      values := jsValueGet(jsArray, elem):','
   next
   Message(cKey,values)
Endif
If result == 32 Then Message(cKey,"is a Json Object")
jsConClose(jsHandle)
exit


:udfs
#definesubroutine jsKeyExist(_handle,_Key)
   IntControl(73,1,0,0,0)
   retval = Arrinfo(jsKeyPaths(_handle, _key), 1)
   If retval > 0 Then retval = jsValueType(_handle, _key)
   return retval
   :WBERRORHANDLER
   geterror()
   jsConClose(jsHandle)
   Terminate(@TRUE,"Error Encountered",errmsg)
Exit
#endsubroutine


#DefineSubRoutine geterror()
   wberroradditionalinfo = wberrorarray[6]
   lasterr = wberrorarray[0]
   handlerline = wberrorarray[1]
   textstring = wberrorarray[5]
   linenumber = wberrorarray[8]
   errmsg = "Error: ":lasterr:@LF:textstring:@LF:"Line (":linenumber:")":@LF:wberroradditionalinfo
   Return(errmsg)
#EndSubRoutine
Return

Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on November 01, 2021, 08:02:27 AM
You get errors because you don't have square brackets around the keys with embedded key delimiters (periods) in their names.

From the help file, "Each key name can be surrounded by square brackets ( [key-name]) to avoid conflicts between JSON names and name delimiter".
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: stanl on November 01, 2021, 02:51:44 PM
Quote from: td on November 01, 2021, 08:02:27 AM
You get errors because you don't have square brackets around the keys with embedded key delimiters (periods) in their names.

From the help file, "Each key name can be surrounded by square brackets ( [key-name]) to avoid conflicts between JSON names and name delimiter".


Yes, then what would the point be with a function to see if a key exists... If you have to surround key with brackets and . comment after that.. there is no purpose for a key search, you are probably sure it already exists, or just do a text search.


Point is I cannot search for "QueryString.NET" and return "1.0.0" without prior knowledge of brackets and . separators.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on November 01, 2021, 02:58:44 PM
If you are searching for "key" then make it "[key]". You don't need to know anything special. Presumably, you have read the extender documentation before you use it so you already know what the default key delimiter is, a period. You have three choices: enclose all key names in brackets, enclose key names in brackets only when you detect a delimiter in the key name or change the delimiter used by the extender's functions.  With any choice, the point of the function is to return paths that allow you to obtain the value and data types of any and all instances of the key.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: stanl on November 02, 2021, 02:45:35 AM
So what is the correct search key for [QueryString.NET] or does the jskeyexist() need tweaking?
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on November 02, 2021, 07:38:05 AM
The function does not need "tweaking" because brackets are sometimes required.  It is changing a bit in the next release but brackets will still be needed brackets when the key name contains a key delimiter.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: stanl on November 02, 2021, 01:25:17 PM
Quote from: td on November 02, 2021, 07:38:05 AM
The function does not need "tweaking" because brackets are sometimes required.  It is changing a bit in the next release but brackets will still be needed brackets when the key name contains a key delimiter.


Don't need to get too Existential about this... but you send a string value to a function and it either exists or it doesn't, and if it does the reply from the function indicates how to deal with it.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on November 02, 2021, 01:58:52 PM
You pass in a nonexistent key your get an empty array. If you want to know why read the documentation. The function is designed to minimize errors although there are a few that have to be reported because of system-related issues.  The next release would break existing scripts without the bracket requirement. We do plan ahead...
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: stanl on November 03, 2021, 02:37:06 AM
Code (WINBATCH) Select


IntControl(73,1,0,0,0)
Gosub udfs
AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')
JsonData = $"{
    "version":: "1.0",
    "dependencies":: {
      "Microsoft.NETCore.UniversalWindowsPlatform":: "5.2.2",
      "Microsoft.Toolkit.Uwp.Notifications":: "1.1.0",
      "QueryString.NET":: "1.0.0"
    },
    "frameworks":: {
      "uap10.0":: {}
    },
    "runtimes":: {
      "win10-arm":: {},
      "win10-arm-aot":: {},
      "win10-x86":: {},
      "win10-x86-aot":: {},
      "win10-x64":: {},
      "win10-x64-aot":: {}
    }
  }$"


If ! jsValid(JsonData) Then Terminate(@TRUE,"Cannot Continue","Invalid Json")
jsHandle = jsParse(JsonData)
;; This can still error if the key contains path information and
;; it is syntactically malformed.
;try these
;cKey = "version"   ;should return 1.0
;cKey = "runtimes"  ;should return Json Object
;cKey = "QueryString.NET"
cKey = "uap10.0"   
result = jsKeyExist(jsHandle,cKey)                 
If result == 0 Then Message(cKey,"Does Not Exist")
If result >1 && result <=8
   Message(cKey,jsValueGet(jsHandle,cKey))
Endif
If result == 16
   path = jsKeyPaths(jsHandle,cKey)
   values = ''
   jsArray = jsValueGet(jshandle, path[0])
   paths = jsKeyPaths(jsArray, '')
   foreach elem in paths
      values := jsValueGet(jsArray, elem):','
   next
   Message(cKey,values)
Endif
If result == 32 Then Message(cKey,"is a Json Object")
If result == 64 Then Message(cKey,"key exists but value cannot be determined")
jsConClose(jsHandle)
exit


:udfs
#definesubroutine jsKeyExist(_handle,_Key)
   IntControl(73,1,0,0,0)
   retval = Arrinfo(jsKeyPaths(_handle, _key), 1)
   If retval > 0 Then retval = jsValueType(_handle, _key)
   return retval
   :WBERRORHANDLER
   IntControl(73,1,0,0,0)
   return 64
Exit
#endsubroutine


#DefineSubRoutine geterror()
   wberroradditionalinfo = wberrorarray[6]
   lasterr = wberrorarray[0]
   handlerline = wberrorarray[1]
   textstring = wberrorarray[5]
   linenumber = wberrorarray[8]
   errmsg = "Error: ":lasterr:@LF:textstring:@LF:"Line (":linenumber:")":@LF:wberroradditionalinfo
   Return(errmsg)
#EndSubRoutine
Return


Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: td on November 03, 2021, 08:31:00 AM
Your script contains a bug so it should error. You use the jsValueGet function incorrectly. The second parameter needs to be a JSON path, not just a key name. The help file is clear on that. The design of the extender is based on the idea of using jsKeyPath to establish a valid path to a JSON key/value pair. Once you have the path you can use the path instead of wasting processing time repeating the key name search. The jsKeyPath function is intended to not produce errors so it can be used for existential testing of key names without error suppression. JsKeyPaths may produce a few syntax-based errors because we missed one or two in called functions within the extender. The purpose of the other functions does not include existential testing so they are implemented to produce more error information.

The jsKeyPaths function does need better documentation but documentation needs to change for the next release anyway. That makes the documentation problem irrelevant at this point.

This thread has already lived too long so it is time for me to bow out.
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: stanl on November 07, 2021, 08:04:57 AM
Quote from: td on November 03, 2021, 08:31:00 AM
This thread has already lived too long so it is time for me to bow out.


Agreed. Good luck with Part Quatre. I'll stay away from any pseudo-issues. Your JsonData was a good base for exploring stuff. I poked at it in PS enough to come up with code to walk down nested Json keys/elements and obtain value(s). But, as I previously mentioned the Json I will parse out to Excel at work is well-defined and all elements are documented. So, questions like "does key exist" are theoretical: though I learned from the rabbit-hole I went down. :-[
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: JTaylor on November 07, 2021, 10:57:37 AM
Hopefully the next version will allow the jsKeyExist() function to actually work because it isn't theoretical for me :)

Jim
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: stanl on November 07, 2021, 12:36:05 PM
Quote from: JTaylor on November 07, 2021, 10:57:37 AM
Hopefully the next version will allow the jsKeyExist() function to actually work because it isn't theoretical for me :)
Jim


Probably any confusion is.... given key = "mykey' , simple element in Json
In either case, if the key exists could the value be retuned [actual value/stringtype/numerictype/arraytype/objecttype], or is that a separate function?
Title: Re: **************** New JSON Extender (Experimental) * Part Trois***********
Post by: JTaylor on November 07, 2021, 03:15:38 PM
If I do a jsValueGet() for the same path I get data.  If I use that same path in the jsKeyExist() function it tells me it doesn't exist.

Jim