FileItemize & Variable Issues

[vulture]

We're trying to transition our environment off of a very old version of Winbatch (99a) to the newest version of Winbatch. The person over in our testing environment didn't run into any issues, but when I deployed it we immediately ran into a FileCopy error.

After some testing, it appears that creating a variable and using it like so;

exampleVariable=FileItemize("\\examplehost\ehc\example\IITR99*")
FileCopy("\\examplehost\ehc\example\%exampleVariable%", "\\examplehost\ehc\example\example.001",@false)

results in the file copy error but if I change the wildcarding to this;

exampleVariable=FileItemize("\\examplehost\ehc\example\IITR99*.*")
FileCopy("\\examplehost\ehc\example\%exampleVariable%", "\\examplehost\ehc\example\example.001",@false)

It runs without issue. We have a vast number of scripts that are used throughout the environment and I'm wondering if there are any settings that I could use to bypass this behavior rather than going through all the scripts in the system.

[ChuckC]

Uh... I'd recommend that you re-read the documentation for FileItemize() and FileCopy(), paying particular attention to the 1st input parameter to FileCopy(), and then look again at what your script is doing.  Also, you might find some enlightenment by displaying the value of "exampleVariable" in a message box or by inspecting it in the debugger by running the script under WinBatch Studio.

Also, ditch the "%%" variable substitution... it will only ever cause trouble & confusion that is in many cases counter intuitive to properly diagnose.  Instead, switch to using the newer ":" string concatenation operator.

[vulture]

I'm aware that the original script is not exactly the most efficient way to copy a single file. They were all written years ago by someone else. Using the message function I've determined that the newest version of Winbatch will return nothing when the single wildcard is used at the end of a file name in FileItemize but returns the filename when *.* is used at the end. The older version picks up the file name with either the single * or the *.* I was hoping that I would be able to avoid combing through hundreds of scripts to change all the wildcards in all the lines where FileItemize is found. The deployed environment doesn't handle disruptions to its routine very well at all. I'm also aware that variable substitution is not desirable in the way that it's being used, but the powers that be are comfortable with the scripts being setup that way and do not want that changed.

[td]

In testing the newest version of WinBatch, FileItemize behaves as it is documented, i.e.,

FileItemize("*.*") ; a list of all files in the current directory.
FileItemize("*") ; a list of all files in the current directory.

In other words, the two file masks return all the files in the targeted directory (folder.) 

You may want to reconsider your diagnostic approach is it appears you have misdiagnosed the root cause of your problem.

[vulture]

In testing the newest version of WinBatch, FileItemize behaves as it is documented, i.e.,

FileItemize("*.*") ; a list of all files in the current directory.
FileItemize("*") ; a list of all files in the current directory.

In other words, the two file masks return all the files in the targeted directory (folder.) 

You may want to reconsider your diagnostic approach is it appears you have misdiagnosed the root cause of your problem.

I do have a test script with the same FileItemize line with the only difference being * vs *.* and a message displays the result of each line and the one using the * only returns as blank whereas the line with *.* returns the filename.

[vulture]

In testing the newest version of WinBatch, FileItemize behaves as it is documented, i.e.,

FileItemize("*.*") ; a list of all files in the current directory.
FileItemize("*") ; a list of all files in the current directory.

In other words, the two file masks return all the files in the targeted directory (folder.) 

You may want to reconsider your diagnostic approach is it appears you have misdiagnosed the root cause of your problem.

If you don't believe me, create a directoy, say c:\test or something and place within it a file named test.2018.txt and run this code;

testVar=FileItemize("c:\test\test*")
Message("Results","%testVar%")
testVar=FileItemize("c:\test\test*.*")
Message("Results","%testVar%")

You will see that the variable is simply blank in the first message window.

[stanl]

Worked fine for me - no blanks

[ChuckC]

Interesting...

Using the outlined steps, I can reproduce the problem and get back an empty result from FileItemize() on a Windows 10 Professional system running WinBatch 2017B when the wildcard pattern used with FileItemize() is "C:\Temp\Test*", while getting back a non-empty result when using "C:\Temp\Test*.*".

[stanl]

Interesting...

Using the outlined steps, I can reproduce the problem and get back an empty result from FileItemize() on a Windows 10 Professional system running WinBatch 2017B when the wildcard pattern used with FileItemize() is "C:\Temp\Test*", while getting back a non-empty result when using "C:\Temp\Test*.*".

All apologies....  I used C:\temp\test\* and C:\temp\test\*.*  which returned all folder files, not a specific file

[vulture]

Well, I see  there will be no solution for this. We'll have to go through and change code in potentially hundreds of scripts.

[ChuckC]

I would tend to think that it is very much premature to make a pronouncement that there will be no solution for this problem.  It may be valid to say that there will not be a solution available within the time frame that you're working in, depending the time frame itself.  However, I'd hazard to guess that once the WinBatch product engineers are also able to perform a test that reproduces the problem, we'll be provided with something more informative regarding whether it's determined to be an easily fixable bug in WinBatch itself or if it's something related to misbehavior of an underlying API function within Windows itself that could complicate the process of finding a work-around for the problem.

[JTaylor]

Assuming I am reading this thread correctly, which is always in doubt, I brought up a similar issue sometime back, after much head-to-wall contact, with "*" vs. "*.*" and was told that this is how Windows works and has nothing to do with WinBatch, per se.

Jim

[vulture]

Assuming I am reading this thread correctly, which is always in doubt, I brought up a similar issue sometime back, after much head-to-wall contact, with "*" vs. "*.*" and was told that this is how Windows works and has nothing to do with WinBatch, per se.

Jim

That doesn't make much sense. Our old version doesn't do this when running on the same version of Windows as the new version.

[JTaylor]

May be a slight variation of your issue but in my experience * and *.* have always acted differently, at least what I remember, and has caused no end of headaches because I keep forgetting that fact.   I will see if I can find my earlier post and see what I was complaining about.

Jim

[JTaylor]

Sorry, can't find it.

Jim

[ChuckC]

I've done a bit more testing with this.  I've used both WinBatch 2016B and 2017B, both running on the same Windows 10 Professional system.

The test data is as follows:

I have a NTFS volume with a test directory "C:\Temp\Test01", and in that directory are 2 files, "Test.2018.txt" and "Test.txt".

If I call FileItemize() with the following parameter values of "C:\Temp\Test01\*" or "C:\Temp\Test01\*.*", I get results back that are identical in both cases, and both files are in the results.

If I call FileItemize() with "C:\Temp\Test01\Test*", I get back empty results.

If I call FileItemize() with "C:\Temp\Test01\Test*.*", I get back results that contain both files.


It's not clear to me whether there might be some factor in the O.S. version or configuration that may have any impact on this behavior.

[td]

I need to apologize to the OP for neglecting to notice the partial file name in the original post.  Bad form on my part.

Chuck alluded to the explanation for the way star (*) works with partial file names.  Basically, the answer is that this is how the underlying Windows APIs used by the FileItemize function work.  And how the Windows APIs work with regard to wildcard matching is not identical to the way it works at the Windows Command Shell prompt which attempts to emulate the old DOS dir command with respect to wildcard matching.

With the Windows APIs, the "*" does not exactly mean match anything, and both the last  "." in a file name and the "*.*" have added meanings.  There is a long explanation for why MSFT software engineers did it this way.  However, the gist is that specify something like "foo*" to the Windows APIs used by FileItemize means that the user requests all files that have "foo" at the beginning of their name but do not have a file extension - a ".xyz" part.  This seems confusing for several reasons but mostly because using just "*" does work to give you all the files in a directory and the command prompt's "Dir" command behaves differently.

WinBatch does have an alternative method for matching file names.  The optional method can be set using IntControl 91.  If you set the IntControl's p1 parameter to 1, the underlying Windows API matches file names using short file names just like the Command Shell "Dir" command.  So this could be used to kludge a solution that might make your scripts work.  But that would be contingent on the targeted system's file system being configured to support short file names and this is not guaranteed because NTFS can be configured to either support or not support short file names.  Using short file names can also lead to some unexpected and undesirable matching results in some cases.

So yes there is a setting that may or may not change the file matching behavior of the FileItemize function but it is not the recommended solution.  In the long run, you would be better off either manually or through a script changing the file masks.

Update:  After a little more checking it appears that IntControl 91 will work to produce "Dir" command style results on a system without short file names enable.  However, there still is a possibility for unexpected and undesired matching results when short names are enabled on a file system.

[vulture]

Thank you for the thorough explanation. I intend to do it all edits manually, as our scripts get used every day on a time sensitive schedule. If I had a script changing strings and messed something up that we had a hard time identifying, it would cause chaos.

[td]

Writing a script to modify scripts isn't as hard to do or test as it may seem but I may think that way only because I have done it so frequently. 

[vulture]

That's a little more difficult when many of the files specified across hundreds of scripts are grabbed from an external source by the script and deleted by the script. It's very laborious to go through hundreds of scripts across different machines and insert code to capture the full filenames that the scripts are grabbing so that the wildcard can be changed to accommodate the presence or lack of an extension else a script will bring everything to a halt.

[td]

I guess the perceived level of difficulty depends on your background and experience.  I would take the approach of first pulling all the file functions used by the scripts out using either binary functions, ArrayFile functions,  or a third party tool. It could be done with a relatively short script.  The next step would be to distill that information down to a set of patterns. The strategy to modify the files would then depend on that analysis.

[td]

Come to think of it.  You could always just use IntControl 91 and do some testing for unwanted file matches.  It is not guaranteed that matching both short and long names produces untoward results.

[vulture]

I forgot to mention it, but IntControl (91, 1, 0, 0, 0) completely resolved the issue. Thank you.

[td]

Just keep in mind that by including IntControl 91 you are creating a subtle data and environmental dependency that may break your scripts without warning at some point in the future.