Omnibus Extender - CSV Update

[JTaylor]

For any interested parties...I have updated/added some CSV functions in this Extender.

Added Load using CSV String.
Added Save to String.
Some feature updates to InsertRow/Column.
Some other unrelated tweaks.

http://www.jtdata.com/anonymous/wbomnibus.zip


Jim

[kdmoyers]

Thanks Jim!

[JTaylor]

No problem.  Only took me a year or so to figure out why the Load from String wasn't working  :)

Jim

[JTaylor]

I have posted another update.  I discovered a bug in the CSV library and the author just fixed it.   It was related to Removing columns when using column headers as index.

Also, I did a major revamp of the HTML Parsing.  I operated under the assumption that no one was using it so also made some breaking changes.  Mostly function name changes but any function that had a "Count" parameter no longer has that parameter.    I primarily made this assumption on the fact that it didn't really seem to work very well so figured I would have heard something.   If this is not true and this will create major problems for you, let me know.    I think it would be worth making the changes as this seems to work much better and, as a bonus, I believe works correctly.   Can't say that was true of the previous version.   Always surprised at how much testing I can do and then go to use something in production and it has glaring problems.

http://www.jtdata.com/anonymous/wbomnibus.zip


Jim

[stanl]

Jim;


I'll try to test this out during long weekend. For .csv/.txt/.json I have been using Power Query and M-code which WB works with well. Just finished a script where PQ can calculate difference between 2 text Unix Timestamps and create an extra column for the duration in minutes from raw .csv data returned from web query. I'll see how the extender works with raw data.

[JTaylor]

Thanks.

[JTaylor]

I added a few more functions to the Dialog section of the Extender.

   New - cbShowList()   Opens/Closes DropListBox.
   New - cbGetCount()  Count of Items in DropListBox.
   New - cbGetIndex()   Index of Specified Item in DropListBox.
   New - dbScrollTo()    Scroll to Item in DropListBox by Index.


I do take requests so if there is something that would be useful let  me know and I will see what I can do.

Jim

[jmburton2001]

Hi Jim!

The new dialog functions look very interesting. Are they included in the download at Reply #3?

[JTaylor]

I just posted that today so you will need to download again.   Sorry.  Should have been a bit clearer.

http://www.jtdata.com/anonymous/wbomnibus.zip


Jim

[JTaylor]

At the risk of being annoying...I just discovered that passing by Reference has once again come back to bite me.   Hopefully I covered all the bases this time.   This is one upside to me using what I write :)

http://www.jtdata.com/anonymous/wbomnibus.zip


Jim

[stanl]

Jim,


I ran a few of the .csv functions and all worked as expected. However, being the ultimate goal of a .csv or .txt is Excel I find myself relying heavily on Power Query and it's transform functionality. For example, I have a an API call that returns unix dates which I transform to datetime and recalculate from PST to EST adding a duration field for minutes between times. This is accomplished with 6 lines of m-code [run as a query from WB]. But I keep your extender in the back-pocket as you never know. I think Tony needs to move your title up to Developer.

[JTaylor]

Appreciate the time and knowing that things are working.   

Yeah...no doubt there will always be some things done more easily other ways.

Thanks again.   The feedback is very helpful.

Jim

[stanl]

2 more ++'s for your .csv functions. Both the get column name / column index have significance for some Hana processing I am doing [Hana is column-wise rather than row-wise, so order is important not just column name]. Also, since server Hana inputs can be set to NoHeader, I'm going to see if I can tweak your existing functions to recognize a .csv with or w/out header row.

[JTaylor]

Sounds good.   The CSV is quickly becoming one of my favorite parts of the extender.   As you are discovering, it makes manipulating data much easier and I often go this route now rather than messing with arrays.

Jim

[JTaylor]

I have added a csvFilter() function to my Extender.   See Help file for details.   I think all is solid.  Did lots of testing but I am sure I didn't try every single combination and probably even missed something obvious but hopefully not.


http://www.jtdata.com/anonymous/wbomnibus.zip

Jim

[bottomleypotts]

Thanks Jim, as always much appreciated!

Agree that the CSV is a much more elegant way of handling data.

Quick question - have you made any progress on your SECURE Websocket funtionality?

[JTaylor]

I worked on that but, oddly enough, I cannot find ANY websocket service but my own that I can connect with for testing.  I just installed the standard stuff on a Windows Server and it worked fine.  Every test site I try fails.   Seems like it should work with either secure or unsecure, as it is but no way for me to test.   I think I had someone tell me it worked for them but not sure if that was secure or not and if they were connecting to my server or not.

Have you tried connecting either way?  Results?

Sorry I don't have anything more definite.

Plus, I keep hoping Tony will announce that as a new release for WinBatch and I can remove it from my Extender :)

Jim

[bottomleypotts]

I have got it working with echo servers. And it works just as expected - perfectly! But that was an unsecured service!

I would like to use it to interact with Discord and Slack, rather than just using it as a messaging service. However their websockets service is only secure. I did not try connecting as you had said in the past it wasn't ready. (Just tried connecting to a secure echo service and it failed to connect.)

As always, thanks!

Agree, it would be nice if this was integrated into winbatch. I believe I asked for this in 2017. So I'm not holding my breath!

[JTaylor]

Do you have a link to a service that worked for you?   I have tried a few but no luck.  Not sure why.

Jim

[bottomleypotts]

Here's the last code I played around with. One observation - no error message if something went wrong, the process just crashed. This would be a problem in real life, as I'd be expecting this to continuously poll.

Code (winbatch) Select Expand

AddExtender(`wbOmnibus.dll`)

BoxOpen(`test`,``)

zUrl=`ws://echo.websocket.events/.ws`
zBT=``

zMessages=`this is a test,this works,wow,hello`

zWs=wsConnect(zUrl)

c=0

While @TRUE

c=c+1
BoxTitle(c)

Ret=wsPoll()
While Ret!=``
zBT:=@CR:Ret
BoxText(zBT)

Ret=wsPoll()
EndWhile

r=Random(3)+1

e=ItemExtract(r,zMessages,`,`)
wsSend(c:` `:e)

TimeDelay(c)

EndWhile

wsDisConnect()


[JTaylor]

Thank you.

Jim

[JTaylor]

Just to follow up on the secure WebSocket stuff.  Don't be holding your breath.   For obvious reasons this utilizes events.   That is not something that I can leverage in an Extender.   Add to that the need for openssl and asio libraries and it complicates things to a point where I don't think this will be feasible in an Extender, at least not utilizing the SDK.

Jim

[JTaylor]

I was inspired by a recent discussion about sorting and I have added a csvSort() function to my Extender.   See Help file for details.     I allowed up to three column sorting.  If you need more let me know.  I think i can easily do more.  When I first started more seemed like it would be a hassle but once I got things sorted ;), I realized I was thinking wrong.

http://www.jtdata.com/anonymous/wbomnibus.zip

Jim

[stanl]

Nice

[JTaylor]

I have added a csvReplace() function to my Extender.   See Help file for details.   

http://www.jtdata.com/anonymous/wbomnibus.zip

Jim

[bottomleypotts]

Thanks Jim. Always look forward to these updates.

[JTaylor]

Added a csvSum() function.

Be aware that I have COMPLETELY reworked the XML functions.  This version breaks all the previous XML functionality, which I consider a good thing.  I don't think anyone was using it anyway.  Some of the XML functions require Winbatch 2021B or newer.   Started to simply deprecate all of it since I have never liked what I did but finally found a library that is as elegant as the CSV library that I use, which is saying something.   I am not suggesting that this is superior to the other options that are available but is now something that doesn't embarrass me.

See the documentation for further information.

http://www.jtdata.com/anonymous/wbomnibus.zip


Jim

[snowsnowsnow]

(This is me basically saying I'm too lazy to figure out what it does without some help)

I just heard about this extender today (on this board) and so downloaded it from the link above.  The ZIP file has 4 files in it.  I have not actually looked at any of them.

However, it seems to me that two things are missing from the ZIP file.  Would it be possible to add them?  They are:

1) Source code.  I assume source is in C (or C++).  I'd be more interested in reading the source than in trying to figure out what it does from the help file (The .CHM file)  Is the source proprietary in some way?

2) Some sort of "README" that contains a quick blurp as to what the overall point of the extender is.  N.B.  I'm not in any way saying that there is no point to the extender (or anything like that).  It's just that at this point, I have no idea what that point is.

(Note: I'm aware some people will consider both of these points of mine to be nitpicking, and, I suppose, in some way they are, but they are important to me.  If you think I've overstepped, don't bother writing in to tell me so...)

[JTaylor]

From the Help file:

"This Extender provides parsing for JSON, XML, HTML and CSV.  It also includes the ability to interact with SQLite databases, Image manipulation, Toast Notifications, WebSockets as well as a number of other data manipulation functions.  The DialogEx Extender has also been folded into this Extender."

I had a number of separate Extenders and decided to put them all into one, hence the name.  I still haven't determined whether that has made maintenance easier or harder.

Tony has since released Extenders for JSON and SQLite so I list those as deprecated but haven't removed them since some people might be using them.   Plus I have no interest in trying to maintain duplicate functionality.

While not necessarily proprietary I am not sure about posting the source code for the reason requested.  I would hope the Help file would be easier to use for determining what a function does than the source code.  If not, I fear I have failed in a major way.

I had three primary motivations and goals for developing this extender.   Learn C/C++, Help WinBatch Users and develop tools I felt were missing in WinBatch to use in my own projects.  I like to think I have accomplished all three to varying degrees.  My beginnings with the Dialog Extender mostly involved me hacking away at stuff until it worked and the code reflects that.   Newer stuff like the CSV functionality was accomplished with a number of moments of me feeling like I knew what I was doing and even contributed a bit of code to the library project.  As far as how much it has helped others, I do not know but a number of people have expressed their thanks.   As for how much it has helped me, it would be hard to gauge but it has been SIGNIFICANT. 

Hope the above helps answer some of your questions.  Always happy to help if you have further questions.

Jim

[JTaylor]

Fixed some issues with CSV and added some sample code for CSV and Dialog functions.

    http://www.jtdata.com/anonymous/wbomnibus.zip

Jim