Another CLR Investigation

[stanl]

Pretty silly code attached but was wondering if there might be any practical use for .NET Queue Class

Code (WINBATCH) Select Expand



;Winbatch 2018B - CLR Queue
;=================================================================================
ObjectClrOption("useany","System")
oQueue = ObjectClrNew('System.Collections.Queue')
Message("CLR Queue Object",oQueue)
oQueue.Enqueue("Hello")
oQueue.Enqueue("World")
Message("Queue Items",oQueue.Count())
A = oQueue.ToArray()
Message("Queue Contents",ArrayToStr( A ))
oQueue = 0
Exit


[ChuckC]

What's your definition of "practical" in this context? :P

I use queue containers in asynchronous producer/consumer patterns all the time.  In particular, at this moment, I'm working on code that monitors the file system for changes and queues them up for subsequent processing by other threads in the same process.  Then the consumer of those events invokes some developer-supplied callback methods to filter the events and put a subset of them into another queue.  Finally, another developer-supplied callback method consumes the content of that second queue to forwards the filtered events on to a cloud-based message bus where various subscribers receive the events.

[stanl]

What's your definition of "practical" in this context? :P

More specific to something that cannot be easily done with current WB tools like lists, arrays, pointers etc...

[JTaylor]

That is what I was just trying to decide...

Jim

[ChuckC]

Many of the generic collections available in .NET are actually quite similar, but with meaningful differences that facilitate or even enforce certain usage patterns.

The type in question, a Queue, simply enforces FIFO access behavior.  The Stack type enforces LIFO access behavior.  The Set type ensures that there is only ever a single instance of a value in the set, whereas a list allows duplicate values to exist; both Set & List types can be unsorted or sorted.

So take a look at what you would use a Queue for and ask yourself how readily can you perform queue-like functions with the language features present in WIL.  Certainly, with an array and some UDF code, you could emulate a queue.  However, if you wanted your "queue" to limit the access semantics to FIFO, then using the CLR to use the .NET Queue<> generic collection type is a good idea.

[td]

Data structures and algorithms are tools in the software developer's toolbox and you use one or more of those tools when its function best matches the solution to the problem.  Once you decide on the tool you have to decide whether to make it yourself or use a "canned" solution when that is an option.  There are too many factors in making that decision to discuss here but some of the relevant factors include time, cost, performance, reliability, and adaptability.

For example, I recently had a project where I needed to store and return user input in the order that it was typed in.  Of course, that is almost the definition of a queue.  The only decision was whether to roll my own or use a canned solution.  In this case, I chose to create my own custom implementation because I need something fast and lightweight but I could have just as easily chosen a canned solution.

The point is that asking what a data structure/algorithm or even an implementation of a structure/algorithm is good for is more or less asking the wrong question.  The better question is I have this problem so which data structure/algorithm is the best solution?  Or to put it another way, "Prophecy is difficult. Particularly when it is about the future."

[stanl]

That's why I don't plant tobacco.

[JTaylor]

Being made curious I thought I would try the HashSet() as it sounded like it might have some use but I cannot get past trying to initialize an object for the referenced library.   This is simply one of many variations.   I am sure it is something obvious but all the googling I do shows a straightforward reference and can't find any posts concerning problems apart from it needing to be 4.0 or higher but that could be specific to the project they were referencing.   Tried not using the "useany" and referencing 4.0 but didn't help.  Suggestions?

ObjectClrOption("useany","System")
oSCG = ObjectClrNew('System.Collections.Generic')

Here is some sample code for reference.  Thanks.

Code (csharp) Select Expand



using System;
using System.Collections.Generic;

class Program
{
    static void Main()
    {
        HashSet<int> evenNumbers = new HashSet<int>();
        HashSet<int> oddNumbers = new HashSet<int>();

        for (int i = 0; i < 5; i++)
        {
            // Populate numbers with just even numbers.
            evenNumbers.Add(i * 2);

            // Populate oddNumbers with just odd numbers.
            oddNumbers.Add((i * 2) + 1);
        }

        Console.Write("evenNumbers contains {0} elements: ", evenNumbers.Count);
        DisplaySet(evenNumbers);

        Console.Write("oddNumbers contains {0} elements: ", oddNumbers.Count);
        DisplaySet(oddNumbers);

        // Create a new HashSet populated with even numbers.
        HashSet<int> numbers = new HashSet<int>(evenNumbers);
        Console.WriteLine("numbers UnionWith oddNumbers...");
        numbers.UnionWith(oddNumbers);

        Console.Write("numbers contains {0} elements: ", numbers.Count);
        DisplaySet(numbers);

    }

    private static void DisplaySet(HashSet<int> set)
    {
        Console.Write("{");
        foreach (int i in set)
        {
            Console.Write(" {0}", i);
        }
        Console.WriteLine(" }");
    }
}
/* This example produces output similar to the following:
* evenNumbers contains 5 elements: { 0 2 4 6 8 }
* oddNumbers contains 5 elements: { 1 3 5 7 9 }
* numbers UnionWith oddNumbers...
* numbers contains 10 elements: { 0 2 4 6 8 1 3 5 7 9 }
*/




Jim

[td]

You cannot use generics directly with WinBatch CLR hosting.  There isn't a mechanism that allows it.  But as MSFT states, "Many of the generic collection types are direct analogs of nongeneric types." And WIL maps are a form of "hashset". 

[JTaylor]

Okay.   Glad to know it wasn't just me.  Thanks.   

Was mostly interested in the unique aspects as a deduping tool.  May have been overkill but was curious.  Guess I will continue to look forward to you adding that option in WinBatch.

Jim

[td]

What option are you referring to?  If by option you mean, supporting FCL generics in WinBatch, you will be waiting a very long time because of how generic programming works.  If you ever really want to use an FCL generic in WinBatch, just wrap it in a non-generic class and call that class using WinBatch CLR hosting.   

[stanl]

Something like.....??

Code (WINBATCH) Select Expand



ObjectClrOption("useany","System")
oSCG = ObjectClrNew('System.Collections.Hashtable')
Message("Hashtable",oSCG)
oSCG=0
Exit



It's our 46th anniversary coming up. But by June I will update to the latest and check out the WB maps. [Nancy is getting a little tired of the Waffle House]

[JTaylor]

I meant a deduping function.

Jim

[stanl]

I meant a deduping function.

Jim

Generic threads like this often get confusing. I was responding to your initial code failure with Hashsets, Tony's response suggesting using non-generic, then adding WB maps emulate... etc... assuming you thought Hashsets might be used for deduping.

[ChuckC]

I'm wondering if maybe the meaning of what Tony suggested was lost in the overall discussion related to why direct use of FCL generics from within WinBatch is a significantly non-trivial problem that won't be solved anytime soon.

To the point, using a generic class like System.Collections.Generic.SortedSet<T> is problematic as the CLR expects that a compiler will have already compiled the generic class code into a specialized instance where a data type was specified for "T".

The work-around that Tony recommended would be to do something like the following:

In a c# module, define a class as follows:

namespace myclasses.collections
{
    public class StringSortedSet : System.Collections.Generic.SortedSet<string>
    {
    }
}


Compile that to an assembly DLL.


From your WinBatch script, create an instance of "mycollections.generics.StringSortedSet" from your assembly using the CLR support features in WinBatch.

The c# compiler took care of all the details of generating the IL code that the CLR consumes where "T" has been replaced with the "string" type and all of the generic class "SortedSet" code has been used to generate a fully usable class "StringSortedSet".

I chose "SortedSet<>" as a base class since it is a container type that provides sorting and dedup features.

[JTaylor]

I understood.   When Chuck mentioned different types with one only accepting unique values I was intrigued as I often have need of retrieving unique values and, I believe, the only native way in WinBatch is a loop with the appropriate checks.  Not sure a C# solution will be better but always open to learning something new.   Hoping Tony & Crew will provide something more straightforward, hence my misunderstood comment :)

Thanks.   I do intend to take a look at what you posted.

Jim

Generic threads like this often get confusing. I was responding to your initial code failure with Hashsets, Tony's response suggesting using non-generic, then adding WB maps emulate... etc... assuming you thought Hashsets might be used for deduping.

[JTaylor]

Thanks Chuck.   Not sure I quite followed what Tony suggested so this was very helpful.  I had been wondering if doing an in-memory compile would work.  I use that occasionally to great effect and I assume it would work here as well.

Thanks again.

Jim

I'm wondering if maybe the meaning of what Tony suggested ....

[td]

I'm wondering if maybe the meaning of what Tony suggested was lost in the overall discussion related to why direct use of FCL generics from within WinBatch is a significantly non-trivial problem that won't be solved anytime soon.

To the point, using a generic class like System.Collections.Generic.SortedSet<T> is problematic as the CLR expects that a compiler will have already compiled the generic class code into a specialized instance where a data type was specified for "T".

The work-around that Tony recommended would be to do something like the following:

In a c# module, define a class as follows:

namespace myclasses.collections
{
    public class StringSortedSet : System.Collections.Generic.SortedSet<string>
    {
    }
}


Compile that to an assembly DLL.


From your WinBatch script, create an instance of "mycollections.generics.StringSortedSet" from your assembly using the CLR support features in WinBatch.

The c# compiler took care of all the details of generating the IL code that the CLR consumes where "T" has been replaced with the "string" type and all of the generic class "SortedSet" code has been used to generate a fully usable class "StringSortedSet".

I chose "SortedSet<>" as a base class since it is a container type that provides sorting and dedup features.

Thanks for the detailed explanation.   I had made the mistake of assuming more knowledge of generic programming.

[ChuckC]

YW :)

The use of Generics in c# [.NET managed code] is conceptually similar to Templates in C++ [native code], but the process of specialization and instantiation has some significant differences in that c# does it all via compilation to IL code while C++ uses a pre-compiler phase to expand & specialize the template followed by compilation to turn the code into something that can be executed.  I, too, would expect that not everybody knows the bloody gory awful details of how this all gets done.

In-memory compilation should work OK, too, with WinBatch.  I use that technique with PowerShell scripts where an in-memory compilation of a little bit of c# code drastically simplifies usage of native API functions by wrapping a PInvoke up in some c# code that provides a more friendly interface that PowerShell can consume in a trivial manner.

Some tested & vetted sample code involving WinBatch using the CLR to perform some in-memory compilation of c# for a specialized derivative of a generic collection class would be a nice addition to the tech db.

[td]

FWIW, There are already several examples of in-memory compilation in the Tech Database and on this forum. Here is a link to one example in the Tech Database. In case anyone is interested.

http://techsupt.winbatch.com/webcgi/webbatch.exe?techsupt/nftechsupt.web+WinBatch/dotNet/System_CodeDom+Execute~CSharp~in~Memory.txt

Have used the technique for fun and profit a few times myself and of course, WinBatch software products make extensive use of generics in their implementations.

[td]

Also, the reason I mentioned WIL maps is that they are hash table based and can be used as part of a very efficient dup detector.   This is because a map automatically removed duplicates because all keys must be unique.  The catch is that the data must be in a format that can be loaded directly into a map or at least can be indexed by a map.