First, want to say thanks again for this feature. It is awesome!!!
My question though is, is there a limit to the size or number of lines? I get an Out of Memory for Strings error at 2,286 lines and 80,577 characters. I am iterating through a text file, line by line. Using @LF as the delimiter. I did a replace on @CRLF.
I tried just repeating the word hello instead of using long lines of text and I could have 5457 lines and 38,204 characters. Same error if I added another line.
I do know I could use FileRead but would be good to know if there is a limit or a bug.
Thanks.
Jim
I don't know about this case, but I will say that there is something strange with this feature when used in large programs with big arrays and maps and stuff. So far, my response has been to replace the Foreach-with-delimiter loop with an old ItemExtract loop, and the problem goes away.
It has resisted my attempts to create a "smoking gun" sample -- when I try to create a simple case, it always works perfectly. Which just means my case was too simple. I haven't given up, I just haven't got it yet.
Thing is, you can't conclude much from this report: until I find a smoking gun, the most likely explanation is simply an unexpected delimiter in the string.
That is why I tried the lines of "Hello" as I thought the same thing. Appreciate the reply.
Quote from: kdmoyers on March 16, 2026, 07:34:29 AMI don't know about this case, but I will say that there is something strange with this feature when used in large programs with big arrays and maps and stuff. So far, my response has been to replace the Foreach-with-delimiter loop with an old ItemExtract loop, and the problem goes away.
It has resisted my attempts to create a "smoking gun" sample -- when I try to create a simple case, it always works perfectly. Which just means my case was too simple. I haven't given up, I just haven't got it yet.
Thing is, you can't conclude much from this report: until I find a smoking gun, the most likely explanation is simply an unexpected delimiter in the string.
I agree with you though -- I dearly love this foreach delimiter feature, so I will continue to lean on it whenever time allows.
Quote from: kdmoyers on March 16, 2026, 07:34:29 AMI don't know about this case, but I will say that there is something strange with this feature when used in large programs with big arrays and maps and stuff. So far, my response has been to replace the Foreach-with-delimiter loop with an old ItemExtract loop, and the problem goes away.
It has resisted my attempts to create a "smoking gun" sample -- when I try to create a simple case, it always works perfectly. Which just means my case was too simple. I haven't given up, I just haven't got it yet.
Thing is, you can't conclude much from this report: until I find a smoking gun, the most likely explanation is simply an unexpected delimiter in the string.
Putting in the effort to create an example is much appreciated. The script below does illustrate the problem on my system.
; Build a test string
Text = 'Hello world':@lf
strContent = StrFill(Text, 42000)
; This errors at about line 3367.
count = 0
Foreach line in strContent by @lf
dummy = line
count += 1
next
exit
Sorry about that. Wish I was as smart as you, as your approach is much easier to figure out the fail point :-) I spent so much time narrowing down the file looking to see if there was a some character causing the problem and then narrowing down the Hello list it didn't cross my mind. I was just happy to confirm I wasn't doing anything stupid to cause the problem.
So, is it a bug or a feature? I guess it will depend on whether you fix it or document it???
Thanks.
Jim
Something I asked Jim about, but also fails in WB 64. Not sure CLR can pull into a .Net Arraylist (I think PS can)
Or just use the itemlist functions or the Arrayize function on any very large lists until the next release.
This works as well, but it is a bit slower.
objAList = ObjectClrNew( 'System.Collections.ArrayList')
for x = 1 to 4000
objAList.Add('Hello world')
next
y = 0
ForEach objitem in objAList
y += 1
Next
exit
Sounds good. Thanks.
Jim
Quote from: td on March 17, 2026, 01:49:38 PMOr just use the itemlist functions or the Arrayize function on any very large lists until the next release.
Works for me!
This works if you want to use a ForEach structure.
; Build a test string
Text = 'Hello world':@lf
strContent = StrFill(Text, 42000)
aContent = Arrayize(strContent, @lf)
; This does not error.
count = 0
Foreach line in aContent
dummy = line
count += 1
next
Got it, nice.
Hmmm... would this work too?
Foreach line in Arrayize(strContent, @lf)
dummy = line
count += 1
next
Nice.
Thanks.
Jim
Quote from: kdmoyers on March 19, 2026, 06:54:55 AMGot it, nice.
Hmmm... would this work too?
Foreach line in Arrayize(strContent, @lf)
dummy = line
count += 1
next
WIL has an optimization that prevents recalling the function on each iteration, as long as the physical line does not change. However, making changes to the
strContent variable from within the foreach next loop are not reflected in the contents of each
line variable in the loop. This illustrates the point:
; This does not error.
count = 0
Foreach line in Arrayize(strContent, @lf)
dummy = line
count += 1
if IsDefined(strContent) then drop(strContent)
next
exit[edit] Also, using substitution for the parameters to the Arrayize function can cause a physical change to the line and result in reevaluation of the expression containing the Arrayize function call.
Super interesting Tony, thanks!
Just curious. But while on the subject of Foreach and Arrays. Could ForEach with ArrayFileGet be preferable to a FileRead() loop - dependent on any maxsize for ArrayFileGet and accounting for the 4096 char line length.
I haven't tested, but I don't think you would get much of a performance boost. It does save some typing, e.g., not using an integer array index. I tend to use ForEach because I am lazy.