Do I need a 64-bit version of wbOmnibus.dll ?

[kdmoyers]

I recently started using the 64-bit version of the run button in Winbatch Studio, and the 64 bit compile button and that's been working OK.
I can't remember why, something to do with recent windows updates?

However, Jim's wbOmnibus.dll needs to run 32 bit apparently. AddExtender("wbOmnibus.dll") fails in 64 bit mode.
Is it logical for Jim to make a 64 bit version?

Not sure the right way to go here... I don't fully understand the 64/32 bit issue.

-Kirby

[ChuckC]

Yes, a 64-bit build of the extender is needed.

It would also be helpful if the 32-bit & 64-bit versions of the extender had appropriate differences in their names so that a script could test for the run-time architecture and call AddExtender() with parameter values that are appropriate for the architecture.

32-bit vs. 64-bit isn't hard to understand.  CPUs are made with a 64-bit or 32-bit architecture.  The operating system is then built to utilize the CPU's architecture.  In the case of AMD's 64-bit extensions to the x86 processor family, you typically see references to AMD64 or x86_64, both of which are the same thing.  Thus, a pure 32-bit x86 processor can only ever run a 32-bit build of an operating system on it.  However, a 64-bit processor could, in most cases, run a 32-bit build of an operating system but would be limited in the maximum amount of memory that it can support and the peripherals that it can support.  Typically, though, a 64-bit processor is paired with a 64-bit build of an operating system.

On a 32-bit operating system, you can only make use of binaries [executables and libraries] built for a 32-bit architecture.  Extender DLLs are in the binary library category and are subject to the same constraint.

On a 64-bit operating system, there is typically a 32-bit emulation subsystem present that allows 32-bit applications to run under the 64-bit operating system.  On Windows, this is referred to as WOW [Windows on Windows] where a 32-bit emulation layer brokers the interactions between 32-bit processes and the surrounding 64-bit operating system.  Given that both 32-bit and 64-bit binaries may be executed, it is important to note that any DLLs that an executable relies on but be of the same architecture as the executable.

There are both 32-bit and 64-bit builds of WinBatch.  Even though any given script is relatively platform-neutral since there's nothing specific to WIL that makes it prefer 32-bit vs. 64-bit, the script is interpreted by either a 32-bit or 64-bit build of WinBatch.  This means that any calls to AddExtender(), DllLoad() or DllCall() must specify a DLL that matches the 32-bit/64-bit nature of the instance of WinBatch that is processing a script.  The same applies to compiled scripts where the runtime DLLs for the WIL interpreter are used on combination with a boot-strap executable to process/interpret a "compiled" script that is embedded within the boot-strap executable.

[JTaylor]

...and if I knew how to convert the Extender to 64-bit I would happily do so :)

Just gave it another try and like all other attempts haven't gotten very far.   I do know a little bit more now than I did a few years ago when trying to convert the Dialog Extender to 64-bit so maybe I can make more progress before ending in complete and utter failure  ;)

That said, if you are running a standard Windows install, you probably do not need to compile Winbatch as 64-bit.  Obviously do not know your situation so may be completely wrong.

Jim

[td]

Yes, a 64-bit build of the extender is needed.

The one big advantage of using 64-bit WinBatch is that 64-bit WinBatch is compiled large address aware. That means any 64-bit WinBatch process can access more memory than the 2GBs available to 32-bit processes. The 64-bit process memory limit is more or less the amount of virtual memory on the system running the process. The other smaller advantage of 64-bit processes is that 64-bit Windows uses a different calling convention for stack-based subroutines. The calling convention is called "fastcall" and as the name implies it is faster than the 32-bit "cdecl" and "stdcall" calling conventions. For WinBatch this can mean anywhere from no script performance difference to about a 25% performance improvement for 64-bit executed scripts. Performance increases depend on what the script is doing and how it is doing it along with the version of the processor hardware. Performance increases are very difficult to predict because of processor caching and other factors. It most cases running scripts in 64-bit WinBatch to obtain better performance will lead to disappointment.

One consequence of 64-bit WinBatch being large address aware is that all 64-bit extenders must be coded to be large address aware. This means that memory addresses and kernel objects - like handles - do not necessarily fit into 32-bit integer sized variables.

[ChuckC]

...and if I knew how to convert the Extender to 64-bit I would happily do so :)

You control the target architecture when you set up the build configurations for the Visual C/C++ project that is used to compile & link your extender's source code into a DLL.  Much like setting up Debug and Release build targets you can introduce a "platform" element into the project's configuration that allows separate 32-bit and 64-bit targets to be produced.

For example, right now, your project most likely has an output path for the targets [DLLs] that it products that looks something like "bin\Debug" and "bin\Release".  When you introduce the platform [architecture], the paths typically change to something like "bin\Win32\Debug", "bin\Win32\Release", "bin\x64\Debug" and "bin\x64\Release".

Do some reading on Visual C/C++ and setting up multiple target configurations and you'll get it all sorted out.

[td]

The SDK examples using templates are set up to compile both 32-bit and 64-bit versions of the extenders.  The simplest way to create a 64-bit version of an extender is to imitate the example project properties in your extender.

[JTaylor]

Yes.   Have been working on that and bringing the x64 profile in line with x86...keep finding and fixing compile errors.  Have a number of libraries involved which isn't helping.   I understand the concept and, generally, what is needed.   If I can get to where it compiles then I assume I will then need to address variable types, especially ints, as mentioned.

Jim

[ChuckC]

Visual C/C++ provides macros for data types where the definition of the macro changes based on the target platform/architecture.  As long as you use types like HANDLE, INT_PTR, UINT_PTR, etc..., they will equate to either a 4 byte [32-bit] or 8 byte [64-bit] integer that is suitable for containing the integer value representation of a native/raw pointer value.

Never assume that you can take the address of a memory location and cast it to an "int", "long", "unsigned int", "unsigned long", etc..., as this will fail when compiled for a 64-bit target.

[JTaylor]

Okay.   Sounds like my earlier analysis of this attempt ending in failure was probably on track :)

Thanks for all the info though.  It does help as I learn a bit more every time I make the attempt.

Jim

[td]

The early versions of 64-bit WinBatch were not large address aware. This made it much easier to port the hundreds of thousands of lines of code from 32-bit to 64-bit. You can cast away compiler whining when you don't have to worry about 64-bit addresses using the most significant double word. Fortunately, a lot of the data types declarations were changed even though it was not completely necessary at the time.  That made it a lot easier to finish the job of a full port when the WinBatch image changed to support the large address aware memory model. One very useful testing method is to throw a register switch that causes the OS to start process memory from the highest available address. Any memory truncation problems will reveal themselves very quickly in testing.

[Edit] The latest versions of Windows 10 may not support changing the initial process memory starting locations.  I have some notion that MSFT took that option away from developers at some point in the recent past but am not 100% sure about it.

[JTaylor]

Thanks again.

Jim

[td]

Forgot to mention the registry setting...

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management[AllocationPreference] = 0x100000

If you have an older Windows 10 system/VM or a Windows 7 version, it may still be useful.

[JTaylor]

Will keep in mind.  If I get to a point it will be useful I will switch to Win7 for continuing the conversion.   So far the libraries I am using haven't been very cooperative so not sure I will ever get this done.    If anyone out there who knows what needs done wants to tackle it, let me know.

Thanks.

Jim

[ChuckC]

For starters, it would help to know what libraries you are working with that you are encountering problems with as it relates to getting a your extender project successfully built for a 64-bit target platform.

Are these libraries DLLs?  Are the code libraries pre-compiled object files?  Are the just C/C++ source code under some sort of open-source public license?

[JTaylor]

The current block is C/C++ code.  I think I only have one "lib" file I compiled.    I need to look a little more at the Properties between the two versions.    It is the dreaded, for me anyway, unresolved external symbols error so may have missed some include path.    Part of the issue is simple motivation as well.   I am having trouble deciding if the effort is worth it, considering everything else I have going on.  Plus, if this ends up requiring me to maintain two copies of the Extender, as opposed to just compiling it twice that lessens my enthusiasm as well.

Might be better to go back to a version where the projects are separate, like the Dialog Extender, and work through that as a learning exercise.  It doesn't have all the extra libraries.  Would be good to figure out how this all works though.

Thanks.


Jim

[kdmoyers]

I think I see why I was trending to 64 bit.  The 2021B version was not compiling make working 32bit programs!  They would fail saying that wbdub44i.dll was not there. Indeed, it was nowhere at all.  I guess I figured this was the nudge to go 64 bit, because compiling for 64 bit produced no error.

Upon closer investigation, it seems the wbdub44i.dll file was included in the distribution zip, but as a dll, not as a DL_ like all the rest. Consequently, it never gets unpacked by the install script?  Maybe that's why I don't apparently have the file?

Anyway, that's why I was attempting to shift to 64 bit.

-Kirby

[ChuckC]

Mixing replies to 2 previous messages in this thread...


1)  If you are building C/C++ code that references a ".lib" file, then what is happening is that your code is making use of functions, data types and/or classes defined in an external DLL file.  The ".lib" file is just a bit of "glue" that allows the linker to properly build a .exe or .dlll that contains the proper early binding that instructs the operating system to load a .dll file containing that external code before it completes the loading of your .exe/.dll.  The key terminology involved here would be "external reference" and "early binding".  When you fail to provide the proper .lib & .dll for external references, then you'll get either a linker error regarding being unable to resolve external symbols or a run-time error indicating that a dependent .dll file could not be found on the path.

2)  Regarding 32-bit vs. 64-bit extenders... they have different suffixes in the names.  If you are trying to build a 32-bit compiled script on a 64-bit system, you have to make sure that you are using the correct DLL name for the extender that matches the target 32-bit/64-bit architecture that you are compiling the script for.  This includes making sure that the extenders have been properly installed to begin with before the WinBatch+Compiler can include them in a Large EXE build of a compiled script.

[td]

I think I see why I was trending to 64 bit.  The 2021B version was not compiling make working 32bit programs!  They would fail saying that wbdub44i.dll was not there. Indeed, it was nowhere at all.  I guess I figured this was the nudge to go 64 bit, because compiling for 64 bit produced no error.

Upon closer investigation, it seems the wbdub44i.dll file was included in the distribution zip, but as a dll, not as a DL_ like all the rest. Consequently, it never gets unpacked by the install script?  Maybe that's why I don't apparently have the file?

Anyway, that's why I was attempting to shift to 64 bit.

-Kirby

The 32-bit interpreter dll has never had the .DL_ extension in the installation zip file. The setup program cannot execute without it so it is always distributed with its final file name. The  2021B version has been available for some time now and we would have heard about any problems installing or executing scripts before now.

As to why you are missing the file is more or less impossible to even speculate about.  An installation is tested on a wide range of systems before it is released and it must work properly on all of them.

[td]

Of course, after saying it is impossible to speculate about what do I do? Speculate. You wouldn't by chance be compiling your script with the Small Exe option, would you? That will cause the interpreter missing error message you mention.

[kdmoyers]

Why, yes, All of my programs are compiled small exe
-Kirby

[kdmoyers]

I solved the problem by extracting the dll from the zip file myself and placing it into \winbatch\system and into my network shared dlls folder.  The problem was, It somehow did not get extracted by the install script, and I misinterpreted that as being intentional.  my bad really, that was a bad conclusion to jump to!

[td]

If the dll were actually missing nothing would work. WinBatch Studion wouldn't start, FileMenu wouldn't launch nor would PopMenu ever start. In fact, the compiler wouldn't run either.

[kdmoyers]

hmm... I see.  Well, I'll keep digging.  Thanks Tony!

[td]

I overstated a bit. FileMenu is a 64-bit DLL on 64-bit systems so missing the 32-bit interpreter DLL does not affect it on 64-bit systems.  This is because 64-bit FileMenu uses the 64-bit interpreter DLL (wbdub64i.dll) which usually in Program Files/WinBatch/System instead of Program Files(X86)/WinBatch/System where wbdub44i.dll  is located.