Maybe this should be the embellished version?
Version 44003 Sep 17, 2021 (Experimental release.)
Modified the jsKeyPaths function to return
a single dimension array instead of an item
list.
Improved performance of the jsValueGet and
jsValueType functions.
The performance improvements are significant but this version also breaks any script using jsKeyPath because the function now returns an array instead of an item list. It is experimental.
The documentation has been gussy up some as well.
It is available at the usual place:
https://files.winbatch.com/downloads/wb/ilcjs44i.zip (https://files.winbatch.com/downloads/wb/ilcjs44i.zip)
Quote from: td on September 17, 2021, 10:55:37 AM
Maybe this should be the embellished version?
A lot faster.... I adjusted my script for array rather than list; eliminated the map lookup; modified the Boxtext to not display each node;only wrote out the node and value... tested on first 10,000 nodes. There is one test line where I thought the ObjectType for the jsKeyPaths would return ARRAY, but might be using the function wrong.
[EDIT]: my bad, use VarType() not ObjectTypeGet()... the use of "Object" in the Extender docs confused me.
Anyway, this is an excellent Extender and folks.... Json ain't going away :D
;Winbatch 2021C - Census Data - tests large json return
;uses WinHttp.WinHttpRequest.5.1 for JSON return
;Stan Littlefield, September 14, 2021
;Updated: September 18, 2021 - based on new Extender using array rather than delimited list
;======================================================================================================
IntControl(73,1,0,0,0)
Gosub udfs
AddExtender('ilcjs44i.dll', 0, 'ilcjs64i.dll')
cUrl = "https://api.census.gov/data/2019/acs/acs1/variables.json"
BoxOpen("Parsing:":cUrl,"Please Wait...")
cFile = Dirscript():"CensusAPI.txt"
cJson = Dirscript():"CensusAPI.json"
If FileExist(cFile) Then FileDelete(cFile)
If FileExist(cJson) Then FileDelete(cJson)
request = Createobject("WinHttp.WinHttpRequest.5.1")
request.Open("GET", cUrl, @False )
request.SetRequestHeader("Content-Type", "application/x-www-form-urlencoded")
request.SetRequestHeader("Accept", "application/json")
request.Send()
jdata = request.ResponseText
request = 0
;persist .json file
;FilePut(cJson,jdata) ;uncomment if needed
;=================================
If jsValid(jdata)
;jstypes = crtypes() ;no need for map, not persisting node types
Object = jsParse(jdata)
BoxTitle("Creating text Output ":cFile )
crTree()
jsObjClose()
BoxShut()
Message("Json Tree Created",cFile)
Else
jsObjClose()
BoxShut()
Message(cJson,"Not Valid Json")
Endif
Exit
:WBERRORHANDLER
geterror()
Terminate(@TRUE,"Error Encountered",errmsg)
Exit
:udfs
#DefineSubRoutine geterror()
wberroradditionalinfo = wberrorarray[6]
lasterr = wberrorarray[0]
handlerline = wberrorarray[1]
textstring = wberrorarray[5]
linenumber = wberrorarray[8]
errmsg = "Error: ":lasterr:@LF:textstring:@LF:"Line (":linenumber:")":@LF:wberroradditionalinfo
Return(errmsg)
#EndSubRoutine
#DefineSubRoutine crTree()
Fx = FileOpen(cFile,"Write")
output = "node":@tab:"value"
FileWrite(Fx,output)
tree =jsKeyPaths(Object,"")
;just for test
Message("Object Type for tree",ObjectTypeGet(tree)) ;would not expect blank return
Cnt = Arrinfo(tree,1)
i=1
ForEach item in tree
iVal = jsValueGet(Object,item)
If jsValueType(Object,item) <> @JsonObj
output = item:@tab:iVal
If (i mod 100 == 0) Then BoxText("Processing ":i:" of ":Cnt)
FileWrite(Fx,output)
i +=1
If i>10000 Then Break
Endif
Next
FileClose(Fx)
Drop(Fx)
Return(1)
#EndSubRoutine
#DefineSubRoutine crtypes()
maps = $"1,NULL
2,BOOL
4,NUMBER
8,STRING
16,ARRAY
32,OBJECT
$"
jstypes = MapCreate(maps,',',@lf)
Return(jstypes)
#EndSubRoutine
Return
Thanks for taking the time to check the latest version. If you place the "iVal = jsValueGet(Object,item)" inside the "if" block under the line "If jsValueType(Object,item) <> @JsonObj", your script will execute even faster and use less memory. You don't use the value for JSON objects so no reason to waste CPU cycles obtaining them. JSON objects are the most expensive JSON type the jsValueGet function processes.
For the curious, the term "object" is a general computer science concept and as such is used in many different contexts. In this case, the term is part of the formal description of JSON.
https://datatracker.ietf.org/doc/html/rfc7159 (https://datatracker.ietf.org/doc/html/rfc7159)
Well I'm having trouble (see attached). Would appreciate any help people can offer. Would also be nice to be able to see the paths variable when debugging.
Thanks for the reply Jim. The JSON is valid. I can use javascript to parse it.
Quote from: bottomleypotts on September 19, 2021, 01:18:19 PM
Well I'm having trouble (see attached). Would appreciate any help people can offer. Would also be nice to be able to see the paths variable when debugging.
First, you have a variable error in your code jsValueGet(Object, path) should be jsValueGet(obj, path)
I tried paths = jsKeyPaths(obj,"") instead of paths = jsKeyPaths(obj,"id")... didn't help much
Bur, see what happens with prefacing the array and using obj,""
{ "items":
[
{
"id": "F2EVUSJVLDUCI",
"generated": "2021-09-19T14:45:46.000Z"
},
{
"id": "PFEPH53XKWDHQ",
"generated": "2021-09-19T14:45:46.000Z"
},
{
"id": "YLCDX6BKM5FB6",
"generated": "2021-09-19T14:45:45.000Z"
},
{
"id": "6OH7YI5GVDNI4",
"generated": "2021-09-19T14:45:45.000Z"
},
{
"id": "SOEQCKWWQ2UNQ",
"generated": "2021-09-19T14:45:45.000Z"
},
{
"id": "GUIUJV2HXCT6W",
"generated": "2021-09-19T14:45:45.000Z"
},
{
"id": "DKYFE4GOFF73A",
"generated": "2021-09-19T14:45:44.000Z"
},
{
"id": "CGG6B2GY5PH2M",
"generated": "2021-09-19T14:45:44.000Z"
},
{
"id": "5IVIANXJBG7FA",
"generated": "2021-09-19T14:45:44.000Z"
},
{
"id": "YKO537IUD2QKE",
"generated": "2021-09-19T14:45:44.000Z"
},
{
"id": "7MG2VEF5TNSDI",
"generated": "2021-09-19T14:45:44.000Z"
},
{
"id": "NUTFET4Y63D4I",
"generated": "2021-09-19T14:45:44.000Z"
},
{
"id": "PYGUPZV3KSQG6",
"generated": "2021-09-19T14:45:44.000Z"
},
{
"id": "K7YOX3V7PCL62",
"generated": "2021-09-18T23:46:22.000Z"
},
{
"id": "ST2G5JLV3TLRO",
"generated": "2021-09-18T23:46:22.000Z"
}
]}
Then try again with jsKeyPaths(obj,"items").... the extender will kick out @jsonobj references and the numbers change each iterations. Main reason why I put If jsValueType(Object,item) <> @JsonObj in my code. Initially I thought if I referenced any of those I would get a path, i.e. a sub-array....
[EDIT] .... Oh, and use displaythis:=path:@TAB:jsValueGet(obj, path):@CR for a better view of data
Good to see more users playing with Json.
There is definitely a problem with the jsParse function and how it handles nameless array containers so it will require modification.
As I have mentioned before, constructive user feedback is invaluable.
Quote from: JTaylor on September 20, 2021, 06:37:36 AM
Odd....I didn't have the "items" preceding the data. Must have been why nothing I tried worked.
Jim
No, I added it to make a point.
- the Op had valid Json
- jskeypaths() seemed to fail
- they would work with added "items" and jsKeyPaths(obj,"") but not with jsKeyPaths(obj,"items")
- The Extender @jsonObj needs more explanation...
and, of course, this leads to more semanatic arguments over the term "Object". But, let's assume the OP might ultimately want the Json parsed as
item,id,generated => basic .csv output
so something like
output=""
ForEach i in sKeyPaths(obj,"items")
output:=i.id:",":i.generated:@CR
Next
would, in my opinion, be adequate Json parsing.
Glad I diagnosed that one correctly. :) Removed my other posts before I saw this one as I thought I messed up and didn't want to confuse things.
Jim
Quote from: td on September 20, 2021, 07:45:32 AM
There is definitely a problem with the jsParse function and how it handles nameless array containers so it will require modification.
As I have mentioned before, constructive user feedback is invaluable.
Quote from: td on September 20, 2021, 07:45:32 AM
There is definitely a problem with the jsParse function and how it handles nameless array containers so it will require modification.
As I have mentioned before, constructive user feedback is invaluable.
sorry, Tony... I was typing while you were typing. :-\ :-\ :-\
Quote from: JTaylor on September 20, 2021, 07:55:04 AM
Glad I diagnosed that one correctly.
Jim;
although I didn't diagnose anything I did bring up that jsparse() returns a vartype() array, not an ObjectTypeGet() array... which is where we digress to semantic quibbling. Of course any nuances can be handled internally but any additional code would be specific to certain Json formats.
...and it just dawned on me that this issue is why all my attempts at using the Extender keep failing in my current project. Been beating my head against a wall thinking I was doing something wrong :)
Jim
Yeah...JSON seems to be a topic where I see many variations in the semantics. I spent quite a bit of time trying to decide what terms to use in my Extender. Even googling for the "correct" terminology brought up differences.
Jim
Quote from: stanl on September 20, 2021, 09:14:10 AM
Quote from: JTaylor on September 20, 2021, 07:55:04 AM
Glad I diagnosed that one correctly.
Jim;
although I didn't diagnose anything I did bring up that jsparse() returns a vartype() array, not an ObjectTypeGet() array... which is where we digress to semantic quibbling. Of course any nuances can be handled internally but any additional code would be specific to certain Json formats.
Here is another interesting API to play with: https://www.colourlovers.com/api/colors/top?format=json&numResults=100