"Here Documents" in WB

[snowsnowsnow]

This is a spinoff from the the other thread about "Feature Requests".

The feature being requested - the ability to include arbitrary text inside a source code file (and be able to access that text from the running program) - comes originally from Unix shell, where it is known as a "here document".  Perl inherited the idea from shell.  Quite often, as was alluded to by the OP, the desired text to be included is source code for another language.  Over the years, I've implemented this idea many times, for many combinations of languages.  I currently have a system that allows me to embed WinBatch source inside an ordinary DOS/Windows batch file.  In effect, this system is a workaround for the fact that WinBatch is missing the "execute from command line" functionality that is found in most Unix interpreted languages (e.g., sh, AWK, Perl, etc).

Explanatory note: In case, the previous sentence was not clear - for people unfamiliar with Unix - I am referring to, e.g., the ability to do: awk 'awk program here'
or: perl -e 'perl program here'

Now, having said all that, let me say two things:

1) I fully support the idea of including this functionality into the WB language.  It is very useful.  Of course, it will never do me any good, but that's just me.

2) One starts thinking in terms of workarounds - that is, ways to get the functionality using existing versions of WB.

One immediately realizes that is is quite easy to implement, provided one is working in .WBT - that is in uncompiled scripts.  As 99% of my work is in this mode, I would consider the problem solved for myself.
Note that although I haven't actually implemented it yet, I had designed a method that I am 97% sure would work without any glitches.  It took me about 30 seconds to do the design.

However, as I realize that many of the readers of this forum work in .WBC (aka, EXE) mode, the problem becomes a bit thornier.   Here, I think we would need to use one of the methods posted here over the years that allows you to include your source code into your source code into your compiled programs (these methods are marketed as solutions to the all-too-common "OMIGOD!  I've lost my source code" panics).  So, if you use one of these "include-your-source-code" methods, then it would be relatively easy to allow that source code to be extracted at run-time (note that these methods encrypt the source code, so end-users can't get at it, but the program itself could), and then extract the needed text thereby.

This later all sounds do-able, but again, it hasn't reached sufficient a level of interest for me to actually do the work.

[td]

Interestingly suggestions.  Prior art kinda makes the point in this case and you can add Python, PHP, Ruby, and Powershell, to the list of scripting languages that support some form of multiline text string literals.   

[kdmoyers]

I've hacked this two different ways myself, but it has always felt unsatisfying: slow and/or kludgy.  It's certainly a superb request, but it would seem to require some serious mods to the way the interpreter works.  Also, there's the question of whether substitution is supported within the text.
-Kirby

[td]

The WIL interpreter isn't altogether friendly to line spanning string literals but it certainly doesn't completely preclude them either.   And if something along  these lines were implemented, substitution would definitely occur inside muiltline strings for consistency's sake.*  Specifying variables that contain line spanning strings in substitution would have the same often untoward result as is currently experiences when using variables containing carriage returns and or line feeds in substitution.

*Note the 'if'.

[td]

One curious and potentially severe problem involves backward compatibility and WWW's commitment to it.   The WIL interpreter sometimes lets you get by with syntactically incorrect string literals.  Any working script that happens to contain one or more of these syntactic errors could potentially stop working when executed using a new version of WinBatch that implemented multiline literals.

[kdmoyers]

<< potentially severe problems involves backward compatibility >>

interesting.  I think I've left off quote marks myself, now that I think about it.  sloppy.

Well, speaking for myself, I'd gladly fix a PILE of code in exchange for this feature, IF it were possible.

-K

[....IFICantBYTE]

I personally think that WWW shouldn't worry so much about backward compatibility if it means that you can't implement very valuable new features.

[JTaylor]

I agree.  Obviously you don't want to be breaking things year-to-year but there has to be a reasonable point where you can move ahead.   Actually I think you would be doing people a favor if you forced them to take a look at their scripts once in a great while.  Plus, discarding desirable features so you can support incorrect coding doesn't really make sense to me.

Jim

[snowsnowsnow]

Incidentally, here's my basic idea for how to implement this in current WB (for .WBT mode).

Is there any particular reason this couldn't be the way it gets implemented "for real" ?  Would that have the issues mentioned here (backwards compatibility) ?

Code (winbatch) Select Expand


#DefineFunction udfGetHereDocument(tag)
A = Arrayize(StrReplace(FileGet(IntControl(1004,0,0,0,0)),@CRLF,@LF),@LF)
n0 = ArrInfo(A,1) - 1
tag = StrCat("; ",tag)
p = 0
str = ""
FOR I = 0 TO n0
    SWITCH 1
    CASE p
p = StrSub(A[I],1,1) == ";"
Continue
    CASE p
str = StrCat(str,StrSub(A[I],3,-1),@CRLF)
Break
    CASE 1
p = A[I] == tag
    ENDSWITCH
NEXT
Return str
#EndFunction

; Tag1
; This is my text
; It goes on and on
; Then it ends

Pause("udfGetHereDocument(Tag1)",udfGetHereDocument("Tag1"))




[td]

Is there any particular reason this couldn't be the way it gets implemented "for real" ?  Would that have the issues mentioned here (backwards compatibility) ?

I mean no criticism of your idea and I can't take the time to go into WIL interpreter internals but its design is not anything like a linear script.  Implementation and backward compatibility problems are not insurmountable but neither are they trivial.   Backward compatibility in this case goes beyond simply forgetting a quote character and may not be much of an issue for  'lone wolf' shops but it is a huge issue for many large institutions for reasons that I hope are obvious.   

[snowsnowsnow]

but it is a huge issue for many large institutions for reasons that I hope are obvious.

Yes, I get it.

I also get that comment stripping is probably among the first steps in compilation (it usually is in most "conventional" compiled languages), so my idea of embedding the text in comments may be problematic for that reason.

Anyway, just grist for the mill...

[td]

The idea of some kind of multi-character token to indicate the beginning and end of a multiline text string is both common and  sound.  If implemented in the WIL interpreter, these tokens could be classified as reserved 'delimiters' and perhaps even 'operators'.*  It would all depend on were the performance sweet spot lies. 

* Usual disclaimer again...

[stanl]

* Usual disclaimer again...

Agreed.  I know in Powershell, where I have written scriptlets with very complicated SQL Server queries, and stored procs I would use

Code Select Expand


$sql = @"
blah
blah
blah
and 100 more lines of blah
"@


Probably a reason for the markers as SQL can contain * and or % or /, but if the SQL contains " there is no conflict with the marker. Also, the interpretation of the text block is up to the .NET connection and command. 

And Jim's point brings up an excellent use of WB's Execute.

just musing....

[DAG_P6]

The idea of some kind of multi-character token to indicate the beginning and end of a multiline text string is both common and  sound.  If implemented in the WIL interpreter, these tokens could be classified as reserved 'delimiters' and perhaps even 'operators'.*  It would all depend on were the performance sweet spot lies. 

I have considered many times the idea of writing a WIL preprocessor, along the lines of the well established C/C++ preprocessors,  both of which use a reserved token (albeit a one character token) to differentiate instructions for the preprocessor from the rest of the source code. If I wrote one, its goal would be to replace the scores of symbolic constants that I currently define as variables with literals in the emitted script.

[snowsnowsnow]

If you go the pre-processor route, there's always 'm4'.

m4 is pretty cool, once you get used to it.

[td]

Reminds me of the time someone mentioned Lex, Yacc, and WinBatch in the same sentence.

[....IFICantBYTE]

Reminds me of the time someone mentioned Lex, Yacc, and WinBatch in the same sentence.

I don't want to get myself into a complete rant (because I easily could), but you know Tony, I think WWW have done WinBatch a bit of a dis-service for years now, by the very thing that may have once (a long time ago) been a good thing...
... and that is the actual name of the product .. "WinBatch" ... I wish for it's own sake, it was called something else, because I think most people assume it is MS DOS Batch files when you mention WinBatch, and acquaint it with yesterday's technology. That could be part of the reason (if you weren't fully joking) of the Lex and Yacc era association.

Seriously, I think WinBatch is a wonderful product and should be much more widely publicised, used and respected out there...
radical idea: ... maybe it's time for a big re-write - many new features added, updated IDE, legacy support for old code and old OS's (pre Win XP or 7?) dropped and a new name and marketing campaign ... WILScript?, WinCode? .. I dunno.. even an introductory low-cost upgrade or something to entice folks to use the latest and greatest and get it out there?

Whoops... almost did a rant didn't I  :-\

[stanl]

and that is the actual name of the product .. "WinBatch" ... I wish for it's own sake, it was called something else,

Darn! AutoIT is already taken >:(

[td]

"badum-CHING"

[stanl]

I don't think UberScript is taken.

[JTaylor]

Probably for good reason  :)

Jim

[DAG_P6]

If you go the pre-processor route, there's always 'm4'.

m4 is pretty cool, once you get used to it.

I wasn't aware of m4, and it looks interesting, but my experience with anything GNU has been lackluster, especially in terms of documentation (or sparseness thereof), and I an not at all a fan of the GPL, which I think is too restrictive, in that it insists that if you make improvements, you are encouraged to hand them over, for free, and that you are not allowed to make any derivative work a commercial product. I much prefer the two and three clause BSD licenses.