TCP messaging speed through WWWSK64I connection

[DickT]

I have need to communicate short messages (4 characters or less) from one WinBatch application to WB applications running on 9 other computers.  Although spaced about 6 seconds apart, I need to send to all 9 computers at relatively the same time since these activate either video or displayed graphics that need to be synchronized within a 30 milliseconds or less.  Each computer receives a unique message, so any broadcast method (ie UDP) would not work for me.

I have been successful communicating with multiple computers, but there is a significant enough delay from one receiving computer to the next that the video or graphics trigger a few hundred milliseconds apart.  I know that my messages are being sent in rapid succession since by using a terminal program to receive, I can see the messages arrive fairly synchronously.

I suspect the delay is because I am using sAccept with blocking set to @true on the receiving computers.  But I have had no success using @false even though I am able to establish an open TCP connection.

My first question is: Am I just asking too much to make this possible with WinBatch?

And of course if someone knows how to make this happen, can you point me to an example?  I've searched the archives and have come up empty.

[td]

Interesting problem but unfortunately, I don't have a good solution to offer.  Based on your description, it would appear that you have some a built-in delay because you are sending out your messages in sequence.  Then you have to factor in a second delay because the sAccept function pauses 20 milliseconds between checks and your message could arrive anytime during that 20 ms pause.   In a worst-case scenario, the sAccept delay could add up close to a 180 ms by itself over nine targets.  Then there is the issue of the resolution of the system clock which could add a bit more uncertainty to the length of sAccept function delays.

You did not indicate how you attempted to use sAccept in non-blocking mode but it would require that the function be called in a very tight loop with minimal time delays - maybe 5 or 10 ms maximum.  You would also need to run the script on a multi-core or multi-processor system so that the thread running the script had ample processor time. Even then it is not clear that it would provide you with what you want as this does not address the issue of the system clock resolution. 

There are several high-resolution timers available on Windows-based systems that could provide more accuracy but the overhead of calling the APIs via DllCall may or may not make them useful.

[DickT]

Thanks, TD. 

I didn't explain how I was attempting to use non-blocking because I don't know how and my futile attempts went nowhere.  I couldn't find any examples of that.  I believe I did try a tight loop as you suggest with no success, but didn't know if my coding was otherwise valid.  I might try again.  I do need to exit the loop to perform other functions, but to a certain extent, I can control the pacing of incoming messages so that it would always be finished in time to return for the next message.  My testing is being done on an i7-6700K system and I am compiling to 64 bit.

Dick

[DickT]

FWIW, I was hoping there was a way to keep the connection open.  I'm more used to working with AV control systems and devices  where that is possible.

Dick

[DickT]

As I'm under pressure to make something work, unless someone jumps in with a different solution, I am going to switch to using serial communications.  That means adding some hardware and it won't be as fast as TCP could have been.  But I'm under contract and didn't expect this problem.  One of the other parts of the application is having the 9 client applications communicate via TCP with VLC media player.  That is working perfectly since VLC rc keeps the host open full time.

Dick

[JTaylor]

Could you use a broadcast method but some type of station identifier so each station knows what message is for them?

Jim

[JTaylor]

Been a while since I messed with them but what about Pipes?  Is that an option?

Jim

[JTaylor]

No idea if WebSockets make sense in this situation or not but for what it is worth...

Some related light reading

    https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_servers


Jim

[td]

FWIW, I was hoping there was a way to keep the connection open.  I'm more used to working with AV control systems and devices  where that is possible.

Dick

You do keep a connection open until you call the sClose function to close it. Have you looked using one of the sReceive* functions They are based on the presents of data instead of a time delay.

[JTaylor]

If you search for "sAccept kirby snapshot server" you will find an example of using sAccept & @FALSE.

Jim

[td]

I don't believe named pipes would accommodate the timing problem but using a broadcast message with the data for all listeners is interesting.  Of course, your listener scripts would need to parse out their individual bits but that shouldn't be to difficult to code or cause timing problems.

[td]

If you search for "sAccept kirby snapshot server" you will find an example of using sAccept & @FALSE.

Jim

The example certainly looks like it would serve the OP's needs if the time delay was something like .01 or .005 instead of 1.0.   

[DickT]

I've been "away" for a bit working on my serial solution.  So a bunch of messages have accumulated in the last few hours.  Thank you all for the suggestions.

The serial method is working for the single client at this point and should be easy to scale to the 9 clients by using a "broadcast" method as suggested by JTaylor.  I figure a complete packet sent to all 9 clients through a serial repeater (see Digital Loggers) at 115200 will only take about 4 ms.  Each client would parse the data intended for it from the full packet.

For td, yes, I was using sRecvLine in my tests of TCP. 

Again to td, yes, I had looked over the snapshot server example and thought I had tested it, but with no success.  Maybe I should revisit that.  I'll probably clean up my serial based code, then try to implement Kirby's example again.

Although I mentioned being under pressure, I do have a few weeks and being semi-retired, my time is my own on this, so I'll put the time in needed to make it right.

As a side note, I hadn't looked for WinBatch releases since I received notice several years ago that Wilson WindowWare had ceased operation.  I was very pleased to see ongoing development.  The other development environment I'm using in this project has been out of publication for nearly 20 years now, but still very useful to me.  The old adage of "You can't teach old dogs new tricks" seems to apply here.

Thanks again for all the suggestions.

Dick

[DickT]

I just noticed my status as "newbie".   Indeed, I am new to the forum but was surprised that I had never registered, apparently even in the old days of WilsonWare.  In fact, I've been using WinBatch Compiler for 15 years, give or take.  I have TCP and/or serial based communications applications running in museums and visitor centers around the country.

[DickT]

using a broadcast message with the data for all listeners is interesting.  Of course, your listener scripts would need to  parse out their individual bits but that should be to difficult to code or cause timing problems.

BTW, unless I'm mistaken, broadcasting isn't possible with TCP connections.  UDP, yes, and in different environments, I use UDP to broadcast to multiple devices.  As I understand it, there is no way to implement UDP in WinBatch.

[ChuckC]

You have access to the entire .Net framework from within WinBatch, so you can make use of the functionality that the framework has for sending UDP [broadcast] messages and listening for incoming UDP messages on a specific port.

Take a look at the following:

https://stackoverflow.com/questions/40616911/c-sharp-udp-broadcast-and-receive-example

https://gist.github.com/zmilojko/5055246

Yes, it means stepping beyond using the Winsock extender, but there's all kinds of capabilities from the .Net framework that you can utilize in a WinBatch script.

[kdmoyers]

<< Take a look at the following:....>>

It's a big ask, but it would be a great blessing for a smart person with free time to translate one of these into wbt for the aging winbatch hackers in the crowd.  I feel like we could always use more examples of using .Net from winbatch.  It looks a little like COM objects, but different.  And where does "IPAddress.Any" come from?  Very Mysterious.

$0.02
-Kirby

[td]

Code (winbatch) Select Expand

ObjectClrOption('useany', 'System.Net')

;; Need an int64 for the constructor so...
vtIP = ObjectType('I8', '4294967295') ; 255.255.255.255 address as int64.
objIP = ObjectClrNew('System.Net.IPAddress',vtIP)


objAny = objIP.Any
Message('Any IP Address is 0.0.0.0 or just 0 as an int64', objAny.Address)


MSFT does a supprisingly good job of documenting the dotNet Framework and dotNet Core.  It is the goto place to find answers to dotNet usage in WinBatch.

https://docs.microsoft.com/en-us/dotnet/api/system.net.ipaddress?view=netframework-4.8

[ChuckC]

A quick Google Search for "IPAddress.Any" turns up multiple results... the first of which is the following:

https://docs.microsoft.com/en-us/dotnet/api/system.net.ipaddress.any?view=netframework-4.8

And, from that documentation, you can see that it's a static readonly field in a class, and has a value equivalent to "0.0.0.0".  Then again, if you've had to write an app that listens for an incoming TCP connection on a port, the concept of listening on all available interfaces [e.g. on address "0.0.0.0"] should already be familiar and works the same when listening for incoming UDP packets.

By far, the most numerous examples for using the .Net Framework will be c# code, with some VB.Net in the mix as well.  Microsoft's makes Visual Studio Code available for free, so you have a ready to use development tool that will allow you to build c# code.  A very beneficial learning exercise that you can engage in is to use the freely available tools, sample code, code snippets and Microsoft's online documentation for the .Net Framework to develop some basic proficiency with c# and the .Net Framework to facilitate making use of the various classes in the .Net Framework from within WinBatch.

Another approach that could work well is to look for examples on how to use any particular part of the .Net Framework from within PowerShell.  Those types of PowerShell code samples would show how the usage of the .Net Framework was adapted for use in PowerShell and would likely give you some clues in translating them to WinBatch if you happen to already have a bit of PowerShell knowledge & proficiency.

[td]

Here is a simple example that isn't particularly useful but does work.

You need two scripts because we can't call the async methods in WIL.  The receiving script:

Code (winbatch) Select Expand

ObjectClrOption('useany', 'System.Net')
ObjectClrOption('useany', 'System.Net.Sockets')

;; Constuctor needs an IP address.
vtIP = ObjectType('I8', '4294967295') ; 255.255.255.255 address as int64.
objIP = ObjectClrNew('System.Net.IPAddress',vtIP)

objAny = objIP.Any

vtPort = ObjectType('I4', 9876)
objClient = ObjectClrNew('System.Net.Sockets.UdpClient', vtPort) ; Bind to Port 9876
objEndPt = ObjectClrNew('System.Net.IPEndPoint', objAny, vtPort)
Encoding = ObjectClrNew('System.Text.Encoding')
 
while 1
    aBuf = objClient.Receive(objEndPt)
    strMess = Encoding.UTF8.GetString( aBuf )
    Message('Recieved this', strMess)
    break
endwhile


And the handy dandy sending script:

Code (winbatch) Select Expand

ObjectClrOption('useany', 'System.Net')
ObjectClrOption('useany', 'System.Net.Sockets')

Encoding = ObjectClrNew('System.Text.Encoding')
aBuf = Encoding.UTF8.GetBytes("ABCD");
objClient = ObjectClrNew('System.Net.Sockets.UdpClient')
nLen = ArrInfo(aBuf, 1)
objClient.Send(aBuf, nLen, "255.255.255.255", 9876);

[td]

There are two features of C#/.Net/object oriented programming that seem to be challenging for WinBatch users to master when looking at C# examples.

The first is the concept of constructors.  The easiest way for WinBatch users to think about constructors is to substitute the ObjectClrNew function for places in C# where an object reference is created using the new operator or by declaration. 

The second is the concept of method overloading and the related use of method signatures.  The first thing to remember here is that you often need to specify parameter type information for any method that has multiple versions in the documentation (this is, is overloaded.)  Framework classes can be a bit quirky so some object methods may require parameter typing even when the method is not overloaded.  It is fairly easy to know when you need to supply type information because you will get an error that mentions something about type or method not found when you call the method.    WinBatch tries to guess the dotNet type but when you see the aforementioned errors you will need to provide type information for some or all of your parameters.  Generally,  using a COM Automation type is all that is needed but occasionally you will need to use the ObjectClrType function to directly cast the parameter to a Framework type.

[kdmoyers]

Thanks Tony, I'm off and running. I just need time to do this more and I'll get used to it.

And thanks to whole thread for getting me thinking about UDP broadcast.  This could really be handy for something I have coming up.

-Kirby

[DickT]

I found an inefficiency in my code this morning that is making the delay more predictable at about 200 ms on TCP connection from system 1 to system 2.  By putting a 200 ms delay on slave1, I've been unable to detect any noticeable difference in display time when the content is identical.  Since in reality, content on adjacent displays will be different, it might be even harder to detect.

Since my two slaves are dissimilar (an i7 desktop and i5 laptop), I swapped their positions to make sure my correction stayed consistent.  It did.  In actual usage, the systems will be identical.

So I'm considering sticking with TCP and adjusting the delay on the systems to compensate for the cascading delays.  Once I get 9 systems on line, matching system 9 with system 1 should give me a more precise delay to put in each system.

I had started this reply earlier and now see that there are 3 new replies with some code from TD that could help in a UDP solution.  I'll review before settling on what I have working.

Thanks again for everyone who has weighed in on this.

[kdmoyers]

Follow up detail: How to support the idea of timeout on the Receive?

You can put

Code (winbatch) Select Expand

UdpClient.Client.ReceiveTimeout = 5000; 5 seconds
right after creating the UdpClient to set the timeout value.  Easy peasy.

However, when that happens, winbatch will treat it as a legit error, so some handling is required.
Here's my clumsy solution:

Code (winbatch) Select Expand

        errormode(@off)    ; turn errors off for one line only
          aBuf = UdpClient.Receive(objEndPt)
        errormode(@cancel) ; turn errors right back on because im smart
        if vartype(aBuf) != 512 ; detect the error: it's not a variant array
          strMess = ""
        else
          strMess = Encoding.UTF8.GetString( aBuf )
        endif


[td]

MSFT likes to through exceptions and WinBatch likes to catch them so that it can report useful error information instead of "An error has occurred."  So I am guessing that the Receive method is handling the timeout by throwing an exception which WinBatch is dutifully catching.

[kdmoyers]

Yeah, it actually works fine, just caught me off guard at first.
Thanks for the pointers!

[DickT]

One last question from me (the OP).  I am seeing some functions in the examples that don't appear in my help file or documentation.
I assume that ObjectClrOption and ObjectClrNew are recent additions. 

I'm working with Compiler 2012A.  I'm absolutely willing to pony up to come current if there are some benefits.  I updated regularly
until I was doing so little work with it that it wasn't making sense.

[kdmoyers]

Check this page: https://www.winbatch.com/whatsnew.html?
for everything that's been added since 2012.  SO MUCH!!