**************** New JSON Extender (Experimental) * Part Trois***********

[td]

The makeover version?

Version 44004  Oct 12, 2021 (Experimental release.)
     
      Changed the names of the jsObjNew, jsObjMap,
      jsObjClose to jsConNew, jsConMap, and
      jsConClose respectively.

      Modified the jsParse to place top-level
      JSON arrays in the extender's container
      table along with JSON objects.
     
      Modified the jsKeyPaths, jsValueAdd,
      jsValueGet and jsValueType to handle
      JSON arrays as JSON containers.

This pre-release will break most existing scripts written to use this extender.

Unleashing half-baked software on unsuspecting users has been an interesting test of the latest fad in software development but not one that necessarily warrants repeating. But at least I have a better insight into why Windows 11 is so buggy.

It is available at the usual link:

https://files.winbatch.com/downloads/wb/ilcjs44i.zip

[JTaylor]

Should the following work?

Code (winbatch) Select Expand



xtxt = '{"returnCode":0,"searchResultsKey":-2100454365,"numberOfResults":0}' 


jnode = jsParse(xtxt)


  If jsValueGet(jnode,"numberOfResults") == 0  Then Return



[td]

It does for me. The jsValueGet function returns the value of the "numberOfResults" key which is "0" so the "Return" statement is processed.

[stanl]

I left off part 2 with a url that returned a Json array - if I get time this weekend I'll try the latest as the script associated with that post failed

Code (Winbatch) Select Expand



;Winbatch 2021C -
;uses  WinHttp.WinHttpRequest.5.1 for JSON return
;Stan Littlefield, September 14, 2021
;Updated: October 10, 2021 - based on new Extender using array rather than delimited list
;======================================================================================================
IntControl(73,1,0,0,0)
Gosub udfs
AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')
cUrl = "https://www.colourlovers.com/api/colors/top?format=json&numResults=100"
BoxOpen("Parsing:":cUrl,"Please Wait...")
cFile = Dirscript():"ColorsAPI.txt"
cJson = Dirscript():"ColorsAPI.json"
If FileExist(cFile) Then FileDelete(cFile)
If FileExist(cJson) Then FileDelete(cJson)       
request = Createobject("WinHttp.WinHttpRequest.5.1")
request.Open("GET", cUrl, @False )
request.SetRequestHeader("Content-Type", "application/x-www-form-urlencoded")
request.SetRequestHeader("Accept", "application/json")
request.Send()
jdata = request.ResponseText
request = 0


;persist .json file
FilePut(cJson,jdata) ;uncomment if needed
;=================================
If jsValid(jdata)
   ;jstypes = crtypes()  ;no need for map, not persisting node types
   Object = jsParse(jdata)
   Drop(jdata)
   BoxTitle("Creating text Output ":cFile )
   crTree()   ;script will fail as array canot be created with JsParse()
   ;crObj()
   jsObjClose()
   BoxShut()
   Message("Json Tree Created",cFile)
Else 
   jsObjClose()
   Drop(jdata)
   BoxShut()
   Message(cJson,"Not Valid Json")
Endif


Exit


:WBERRORHANDLER
geterror()
Terminate(@TRUE,"Error Encountered",errmsg)
Exit


:udfs
#DefineSubRoutine geterror()
   wberroradditionalinfo = wberrorarray[6]
   lasterr = wberrorarray[0]
   handlerline = wberrorarray[1]
   textstring = wberrorarray[5]
   linenumber = wberrorarray[8]
   errmsg = "Error: ":lasterr:@LF:textstring:@LF:"Line (":linenumber:")":@LF:wberroradditionalinfo
   Return(errmsg)
#EndSubRoutine


#DefineSubRoutine crTree()
   Fx = FileOpen(cFile,"Write")
   output = "node":@tab:"value"
   FileWrite(Fx,output)
   tree =jsKeyPaths(Object,"")
   ;tree =jsKeyPaths(Object,"variables")
   ;just for test
   ;Message("Variable Type for tree",VarType(tree)) ;would not expect blank return
   Message("tree",ObjectTypeGet(tree))


   Cnt = Arrinfo(tree,1)
   nMod = Int(Cnt/10)
   i=1
   ForEach item in tree
      If jsValueType(Object,item) <> @JsonObj
         output = item:@tab:jsValueGet(Object,item)
         If (i mod nMod == 0) Then BoxText("Processing ":i:" of ":Cnt)
         FileWrite(Fx,output)
         i +=1
         If i>10000 Then Break
      Endif
   Next
   FileClose(Fx)
   Drop(Fx)
   Drop(tree)
   Return(1)
#EndSubRoutine


#DefineSubRoutine crObj()
   Fx = FileOpen(cFile,"Write")
   output = "node":@tab:"value"
   FileWrite(Fx,output)
   tree =jsKeyPaths(Object,"")
   Cnt = Arrinfo(tree,1)
   nMod = Int(Cnt/10)
   i=1
   brk=@False
   ForEach item in tree
      If jsValueType(Object,item) == @JsonObj
         subtree =jsKeyPaths(Object,item)
         ForEach subitem in subtree
            output = item:@tab:jsValueGet(Object,subitem)
            FileWrite(Fx,output)
            brk=@True
         Next
      Endif
      If brk Then Break
   Next
   FileClose(Fx)
   Drop(Fx)
   Drop(tree)
   Return(1)
#EndSubRoutine




#DefineSubRoutine crtypes()
maps = $"1,NULL
2,BOOL
4,NUMBER
8,STRING
16,ARRAY
32,OBJECT
$"
jstypes = MapCreate(maps,',',@lf)
Return(jstypes)
#EndSubRoutine


Return


[JTaylor]

Well....of course now it works.   Not sure what was going on.   Probably spent an hour trying this and variations but kept get errors about not being able to convert to a number or a string, depending on what I was doing.

Thanks.

Jim

[td]

Well....of course now it works.   Not sure what was going on.   Probably spent an hour trying this and variations but kept get errors about not being able to convert to a number or a string, depending on what I was doing.

Thanks.

Jim

WinBatch is and always has been negatively typed. That means that relational operands are type-matched before the operation is applied. In this case the result of the call the jsValueGet function is converted from a string to a number and then compared to the integer literal on the right-hand side of the equals comparison operator.

[td]

I left off part 2 with a url that returned a Json array - if I get time this weekend I'll try the latest as the script associated with that post failed
...

You need the change the name of the jsObjClose function to jsConClose but outside of that, your script works just fine.

[stanl]

You need the change the name of the jsObjClose function to jsConClose but outside of that, your script works just fine.

Yes, it did. Got to do some thinking about handling the node values.

[td]

Well....of course now it works.   Not sure what was going on.   Probably spent an hour trying this and variations but kept get errors about not being able to convert to a number or a string, depending on what I was doing.

Thanks.

Jim

Neglected to mention that the type of error you encounter likely has very little to do with the extender other than possiblely placing parameters into the extender's functions in the wrong order or with incorrect content.

[JTaylor]

Possibly.  As far as I know I didn't change what I posted and it worked after posting.  Obviously something changed but no idea what.   Don't recall changing anything in my script.

Jim

[td]

We didn't magically reach out and touch your system so something on your end changed.

[JTaylor]

Haven't embarrassed myself lately so thought I would give it a try here...

Would someone be so kind as to help me figure out how to extract data from this JSON response?   I have tried many things and this is one of my latest attempts.   Originally the problem was the extender,  I think, but this latest release supposedly resolved that issue but still no joy.   Feels like one of those things where I am missing the obvious.

Thanks.

jim

Code (winbatch) Select Expand



  AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')
xtxt = `{"returnCode":0,"searchResultsKey":1457977769,"numberOfResults":1,"blockOfResults":[{"numberCopiesTotal":1,"numberCopiesAvailable":0,"numberCopiesReference":0,"numberCopiesReserved":0,"resultsIndex":1,"baseResourceSn":76449,"baseResourceKey":409560567,"title":"Artificial unintelligence : how computers misunderstand the world","edition":"First MIT Press paperback edition","author":"Broussard, Meredith","callNumber":"303.4834 B876","isbn":"9780262537018","hasLocalImage":false,"mediaDescription":"Book","mediaCategory":"Non-Fiction","location":"General Collection","notes":"First published in hardback in 2018.","series":[{"sn":73323,"seriesName":"Cambridge, Massachusetts"},{"sn":268476,"seriesName":"The MIT Press"},{"sn":274366,"seriesName":"Informational works"}],"rating":0,"reviewCount":0,"publisherName":"The MIT Press","publisherPlace":"Cambridge, Massachusetts","publisherYear":"2019","subjects":[{"value1":274365,"value2":"Computer programs - Correctness","sortByIntFirst":false},{"value1":274364,"value2":"Electronic data processing - Social aspects","sortByIntFirst":false},{"value1":58603,"value2":"Errors","sortByIntFirst":false}],"summaryText":"\"In Artificial Unintelligence, Meredith Broussard argues that our collective enthusiasm for applying computer technology to every aspect of life has resulted in a tremendous amount of poorly designed systems. We are so eager to do everything digitally--hiring, driving, paying bills, even choosing romantic partners--that we have stopped demanding that our technology actually work...","readingLists":[],"genre":"Informational works","physicalDesc":"237 pages : illustrations ;23 cm","hasPRC":false,"hasAR":false,"collIconSN":15,"AvailText":"0","AvailColour":"#FF0000"}]}`

  jnode = jsParse(xtxt)

  If jsValueGet(jnode,"numberOfResults") == 0  Then Return

  jitems        = jsKeyPaths(jnode,"blockOfResults")
message("CNT",ArrInfo(jitems,1))
  Foreach jitem in jitems
    bib          = jsValueGet(jitem,"baseResourceKey")
message(" BIB",bib)
  Next




[stanl]

On the surface nothing is wrong and it is valid Json.... but a strange error for sure. I'll do some more playing around.

[td]

 

Code (winbatch) Select Expand

AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')
xtxt = `{"returnCode":0,"searchResultsKey":1457977769,"numberOfResults":1,"blockOfResults":[{"numberCopiesTotal":1,"numberCopiesAvailable":0,"numberCopiesReference":0,"numberCopiesReserved":0,"resultsIndex":1,"baseResourceSn":76449,"baseResourceKey":409560567,"title":"Artificial unintelligence : how computers misunderstand the world","edition":"First MIT Press paperback edition","author":"Broussard, Meredith","callNumber":"303.4834 B876","isbn":"9780262537018","hasLocalImage":false,"mediaDescription":"Book","mediaCategory":"Non-Fiction","location":"General Collection","notes":"First published in hardback in 2018.","series":[{"sn":73323,"seriesName":"Cambridge, Massachusetts"},{"sn":268476,"seriesName":"The MIT Press"},{"sn":274366,"seriesName":"Informational works"}],"rating":0,"reviewCount":0,"publisherName":"The MIT Press","publisherPlace":"Cambridge, Massachusetts","publisherYear":"2019","subjects":[{"value1":274365,"value2":"Computer programs - Correctness","sortByIntFirst":false},{"value1":274364,"value2":"Electronic data processing - Social aspects","sortByIntFirst":false},{"value1":58603,"value2":"Errors","sortByIntFirst":false}],"summaryText":"\"In Artificial Unintelligence, Meredith Broussard argues that our collective enthusiasm for applying computer technology to every aspect of life has resulted in a tremendous amount of poorly designed systems. We are so eager to do everything digitally--hiring, driving, paying bills, even choosing romantic partners--that we have stopped demanding that our technology actually work...","readingLists":[],"genre":"Informational works","physicalDesc":"237 pages : illustrations ;23 cm","hasPRC":false,"hasAR":false,"collIconSN":15,"AvailText":"0","AvailColour":"#FF0000"}]}`

jnode = jsParse(xtxt)

If jsValueGet(jnode,"numberOfResults") == 0  Then Return

jitems        = jsKeyPaths(jnode,"baseResourceKey")
message("CNT",ArrInfo(jitems,1))
Foreach jitem in jitems
    bib          = jsValueGet(jnode, jitem)
    message(" BIB",bib)
Next

Have either of you downloaded the release version and read the documentation yet?

[edit] The extender release version does not have any coding changes from this experimental version. The only changes are to the documentation; the discussion of JSON paths was broken out into its own help file topic.

[JTaylor]

Hmmmmmm...thought I had the latest.   Looking but don't see a post for an official release, although that sounds vaguely familiar.   I have read through the documentation I do have.

This works.  Thanks.   

Jim

[td]

Didn't post a release notice because I hadn't gotten around to it yet. As mentioned the only changes were minor improvements to the help file (and the extender version number, of course.)

[td]

The extender is fairly flexible. If you like doing things the hard way here you go.

Code (winbatch) Select Expand

AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')
xtxt = `{"returnCode":0,"searchResultsKey":1457977769,"numberOfResults":1,"blockOfResults":[{"numberCopiesTotal":1,"numberCopiesAvailable":0,"numberCopiesReference":0,"numberCopiesReserved":0,"resultsIndex":1,"baseResourceSn":76449,"baseResourceKey":409560567,"title":"Artificial unintelligence : how computers misunderstand the world","edition":"First MIT Press paperback edition","author":"Broussard, Meredith","callNumber":"303.4834 B876","isbn":"9780262537018","hasLocalImage":false,"mediaDescription":"Book","mediaCategory":"Non-Fiction","location":"General Collection","notes":"First published in hardback in 2018.","series":[{"sn":73323,"seriesName":"Cambridge, Massachusetts"},{"sn":268476,"seriesName":"The MIT Press"},{"sn":274366,"seriesName":"Informational works"}],"rating":0,"reviewCount":0,"publisherName":"The MIT Press","publisherPlace":"Cambridge, Massachusetts","publisherYear":"2019","subjects":[{"value1":274365,"value2":"Computer programs - Correctness","sortByIntFirst":false},{"value1":274364,"value2":"Electronic data processing - Social aspects","sortByIntFirst":false},{"value1":58603,"value2":"Errors","sortByIntFirst":false}],"summaryText":"\"In Artificial Unintelligence, Meredith Broussard argues that our collective enthusiasm for applying computer technology to every aspect of life has resulted in a tremendous amount of poorly designed systems. We are so eager to do everything digitally--hiring, driving, paying bills, even choosing romantic partners--that we have stopped demanding that our technology actually work...","readingLists":[],"genre":"Informational works","physicalDesc":"237 pages : illustrations ;23 cm","hasPRC":false,"hasAR":false,"collIconSN":15,"AvailText":"0","AvailColour":"#FF0000"}]}`

jnode = jsParse(xtxt)

If jsValueGet(jnode,"numberOfResults") == 0  Then Return

jsPaths = jsKeyPaths(jnode,"blockOfResults")
Foreach jsPath in jsPaths
   jsObjElem = jsValueGet(jnode, jsPath ) 
   jsArrPaths = jsKeyPaths(jsObjElem,"baseResourceKey")
   Foreach jsElemPath in jsArrPaths
      bib       = jsValueGet(jsObjElem, jsElemPath)
      message(" BIB",bib)
  Next
Next


[JTaylor]

Thanks again.   I didn't think it was too hard but just wasn't getting any traction and needed a bit of a push. 

Jim

[stanl]

Have either of you downloaded the release version and read the documentation yet?

I pasted your code over what I originally cut out of Jim's post - worked fine.

[JTaylor]

What is the most efficient way to determine if an element exists?    In the JSON I already posted some elements do not always exist.   

Do I need to use jsKeyPaths() and then check for an empty array?   Thought there they was something like jsKeyExist() but not seeing it now.

Depending on answer, a couple of requests would be for a jsKeyExist() function, as well as a default value for jsValueGet(), for when there is no path/key.   I have found that VERY useful in my work.   I deal with a lot of data and very often if there is no value I just want a blank value returned without having to check whether that path/key exists or not.  So in the following, rather than an error when edition doesn't exist, it would return a blank string.  If that final default parameter doesn't exist it would return an error, as it does now.


edition      = jsValueGet(jnode,"blockOfResults[0].edition",".","")


Thanks.

Jim

[td]

The function jsKeyPaths is very efficient at it uses a hash algorithm to identify JSON data content. You could wrap it in a subroutine if you like the name jsKeyExist.

[td]

Code (winbatch) Select Expand

#definesubroutine jsKeyExist(_handle,_Key)
   return Arrinfo(jsKeyPaths(_handle, _key), 1)
#endsubroutine

AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')

;; From JSON extender help file.
jsHandle = jsParse('{ "color": "red", "rgba": [255,0,0,1]}')

;; This can still error if the key contains path information and
;; it is syntactically malformed.
Exists = jsKeyExist(jsHandle, "bob")

exit



[JTaylor]

Just seems like a useful and relevant function that many would appreciate having.   Having implemented the default GetValue idea in my own stuff I can also say having that option would be REALLY REALLY appreciated and much used, at least by me :)    Always nice not having to maintain such subroutines for things that make sense being native as it keeps a 1,000 people from writing duplicate code by having it done once in the Extender.

In any event,  just trying to make things better and more useful.  Thanks.

Jim

[JTaylor]

I am sure it is probably just me but if do something like:

Message("BIB",jsKeyExist(jnode,"[blockOfResults][%x%].[baseResourceKey]"))


I get 0 as the response.   If I use those some values in jsValueGet() I get a response.  Should it work for that path?

Jim

[stanl]

I'm a little confused with the response I get back here, building on the jsKeyExist() udf.

Code (WINBATCH) Select Expand



#definesubroutine jsKeyExist(_handle,_Key)
   return Arrinfo(jsKeyPaths(_handle, _key), 1)
#endsubroutine


AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')


;; From JSON extender help file.
jsHandle = jsParse('{ "color": "red", "rgba": [255,0,0,1]}')


;; This can still error if the key contains path information and
;; it is syntactically malformed.
cKey = "rgba"
If jsKeyExist(jsHandle,cKey)
   Paths = jsKeyPaths(jsHandle,cKey)
   Values = ''
   ForEach element In Paths
     Values := jsValueGet(jsHandle,element):@lf
   Next
   Message(cKey,Values)
Else
   Message(cKey,"Key Does Not Exist")
Endif
jsConClose(jsHandle)


exit


[td]

"Rgba" is a JSON array so it is treated as a container by the extender. All containers have an extender JSON handle for a value. Extender JSON handles can use with any extender function with a "container-handle" parameter.

FWIW, "container" is a computer science term for a type of abstract data structure:

https://en.wikipedia.org/wiki/Container_%28abstract_data_type%29

Also since you are using a "foreach" loop to cycle through an array of paths you don't need to check for the existence of the key before doing it. The loop will simply not iterate if the array is empty.

[JTaylor]

So you are saying that it won't work with what I am needing to do? 

Jim

[td]

Not sure what you mean by "it" but no, not saying anything about something not working. Quite the opposite.

[JTaylor]

I would expect the following to return a 1 but apparently I am not understanding.

Jim

Code (winbatch) Select Expand



AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')

#definesubroutine jsKeyExist(_handle,_Key)
   Return Arrinfo(jsKeyPaths(_handle, _key), 1)
#endsubroutine

#DefineSubroutine jsValueGetDefault(_handle,_Key,_Delimiter,_Default)
    Message(Arrinfo(jsKeyPaths(_handle, _key), 1),jsValueGetDefault(_handle, _key, _Delimiter,""))
   If Arrinfo(jsKeyPaths(_handle, _key), 1) > 0 Then
     Return jsValueGet(_handle, _key, _Delimiter)
   Else
     Message("DEFAULT","")
     Return _Default
   EndIf
#EndSubroutine

xtxt = `{"returnCode":0,"searchResultsKey":1457977769,"numberOfResults":1,"blockOfResults":[{"numberCopiesTotal":1,"numberCopiesAvailable":0,"numberCopiesReference":0,"numberCopiesReserved":0,"resultsIndex":1,"baseResourceSn":76449,"baseResourceKey":409560567,"title":"Artificial unintelligence : how computers misunderstand the world","edition":"First MIT Press paperback edition","author":"Broussard, Meredith","callNumber":"303.4834 B876","isbn":"9780262537018","hasLocalImage":false,"mediaDescription":"Book","mediaCategory":"Non-Fiction","location":"General Collection","notes":"First published in hardback in 2018.","series":[{"sn":73323,"seriesName":"Cambridge, Massachusetts"},{"sn":268476,"seriesName":"The MIT Press"},{"sn":274366,"seriesName":"Informational works"}],"rating":0,"reviewCount":0,"publisherName":"The MIT Press","publisherPlace":"Cambridge, Massachusetts","publisherYear":"2019","subjects":[{"value1":274365,"value2":"Computer programs - Correctness","sortByIntFirst":false},{"value1":274364,"value2":"Electronic data processing - Social aspects","sortByIntFirst":false},{"value1":58603,"value2":"Errors","sortByIntFirst":false}],"summaryText":"\"In Artificial Unintelligence, Meredith Broussard argues that our collective enthusiasm for applying computer technology to every aspect of life has resulted in a tremendous amount of poorly designed systems. We are so eager to do everything digitally--hiring, driving, paying bills, even choosing romantic partners--that we have stopped demanding that our technology actually work...","readingLists":[],"genre":"Informational works","physicalDesc":"237 pages : illustrations ;23 cm","hasPRC":false,"hasAR":false,"collIconSN":15,"AvailText":"0","AvailColour":"#FF0000"}]}`

jnode = jsParse(xtxt)

If jsValueGet(jnode,"numberOfResults") == 0  Then Return
   bib       = jsValueGet(jnode,"blockOfResults[0].baseResourceKey")
Message(" BIB",bib)

Message("Bib Key Exist?",jsKeyExist(jnode,"[blockOfResults][0].[baseResourceKey]"))

jsConClose(jnode)



Jim

[td]

The output would depend on the input...


[edit] Also note the what is expected in the jsGetValue parameters second parameter.

https://docs.winbatch.com/mergedProjects/Json/jsValueGet.htm

[JTaylor]

Any chance of getting a clear answer on this one?   Should the jsKeyExist() function return a 1 or 0 in this instance?   Not sure how many different ways I can ask the same question and we probably both have better things to do with our time.  Thanks.

Jim

[td]

Patience. You have found a bug in the jsKeyPaths function. Will try to get an updated extender out today.

Nice data set by the way. Will have to include that in the extender test suite.

[stanl]

"Rgba" is a JSON array so it is treated as a container by the extender. All containers have an extender JSON handle for a value. Extender JSON handles can use with any extender function with a "container-handle" parameter.

FWIW, "container" is a computer science term for a type of abstract data structure:

https://en.wikipedia.org/wiki/Container_%28abstract_data_type%29

Also since you are using a "foreach" loop to cycle through an array of paths you don't need to check for the existence of the key before doing it. The loop will simply not iterate if the array is empty.

No offense, but I thought rgba was a key, i..e jsKeyExist().... and the foreach loop is taken directly from the help file for jsValueGet(). I was thinking the Extender could pull back the map value, i.e. key:value if jsKeyExists() returned @TRUE.... Point is: If keyexistat => return key value(s).... so maybe a SearchKey() function is worth asking about.

[td]

No offense, but I thought rgba was a key, i..e jsKeyExist().... and the foreach loop is taken directly from the help file for jsValueGet(). I was thinking the Extender could pull back the map value, i.e. key:value if jsKeyExists() returned @TRUE.... Point is: If keyexistat => return key value(s).... so maybe a SearchKey() function is worth asking about.

Rather induce more typos... a quote from the help file:

"This function retrieves the value of a key/value pair contained within the passed in JSON object, one of its child objects, in a JSON array or a JSON array element contained within the container or sub container. If the key's value is a JSON object (@JsonObj), the returned value will be a JSON extender object handle. If the returned key's value is a JSON array (@JsonArr), and a bracketed array index is not a postfix to the key name, the return value is a container-handle representing the array."

Also for the JSON path discussion in the help file:

"Important: JSON objects and arrays are collectively referred to as containers in the extender's documentation."

[td]

Any chance of getting a clear answer on this one?   Should the jsKeyExist() function return a 1 or 0 in this instance?   Not sure how many different ways I can ask the same question and we probably both have better things to do with our time.  Thanks.

I guess this is more a documentation problem than a bug. The idea of jsKeyPaths is to specify the key name without any path information other than an index for an array and name brackets. And the way to narrow the search is to use a specific container handle. However, being able to narrow the search using the key name parameter seems like a worthy modification.  Will try to get that change made.