The dreaded VMalloc error

[kdmoyers]

I'm getting a VMalloc error (1st picture) followed by error 3096 (2nd picture).

This is invariably some sort of bug my own doing, which is easy for me to imagine in this rather complex program.

Question: is there some value or statistic that I can watch to see this resource getting progressively consumed?  It would be much easier to find if I could install little watchers at various places that could tell me when the number creeps up (or down as the case may be).

Thanks!!
-Kirby

[Deana]

Looks like you are running out of 'string memory'. Make sure you Drop any large unused string variables. you may need to modify the script to use binary operation to work with large amounts of data. Keep in mind that there is a hard maximum of 250MB of total string memory available. 

IntControl 77 request 0 can be used to determine total memory allocated for strings, in bytes.

[td]

Of course there is always IntControl (77, 0...)  which will tell you how much memory is currently allocated to the WinBatch string heap.

And the snarky  Computer Science guy would say, "Fix our algorithm."  You have ~ 250MB of process address space to play with for storing strings but part of that is used as scratch space by WinBatch.   You can store strings as Variant BSTRs but they get moved back into the string heap when ever they are used in any kind of operation.  This all means that predicting when you are going to run out of string memory is not an exact science.  The danger point will vary greatly depending on the size and number of strings, and the operations being performed

You could consider using external process monitoring (performance counters, process extender, dotNet, e.t.c) to  track memory usage but I don't know how useful that would be.

[kdmoyers]

Thanks guys! sounds like ic(77,0 is what I need.

It takes a couple weeks of run time for the error to hit, so it's a slow leak!

I'll report back when I find it.

-Kirby

[mtwain]

...

It takes a couple weeks of run time for the error to hit, so it's a slow leak!

Or it could be data driven.

[kdmoyers]

Could it be that this

Code (winbatch) Select Expand

DllCall(StrCat(DirWindows(1),"user32.dll"),long:"GetWindowRect", long:hwnd, lpbinary:lpRect)

bleeds a little strings memory every time around because I didn't use a DLL handle?

I know there's the DLLLoad function, but I never use it because there didn't seem to be any reason to.  Could this be the reason?

TIA,
Kirby

[Deana]

Could it be that this

Code (winbatch) Select Expand

DllCall(StrCat(DirWindows(1),"user32.dll"),long:"GetWindowRect", long:hwnd, lpbinary:lpRect)

bleeds a little strings memory every time around because I didn't use a DLL handle?

I know there's the DLLLoad function, but I never use it because there didn't seem to be any reason to.  Could this be the reason?

TIA,
Kirby

There are NO known memory leaks in the current version of WinBatch. However this particular API is passed a binary buffer lpRect. Do you have code that frees that buffer?

Code (winbatch) Select Expand

BinaryFree(lpRect)

[Deana]

I just did some testing using WinBatch 2013B on Windows 7 using the following code:

Code (winbatch) Select Expand

AddExtender("wwctl44i.dll")
window1=cWndByWndSpec("Shell_TrayWnd","explorer",2,303,40965)
window2=cWndByClass(window1,`TrayNotifyWnd`)
ControlHandle=cWndByClass(window2,`TrayClockWClass`)

BoxOpen('Infinite loop calling GetWindowRect','Open Resource Monitor or task Manager to see if there is any memory leaks in WinBatch.exe':@LF:@LF:'Press Ctrl+Break to end script')
While @True
  ; Fill buffer with control coordinates
  rect = BinaryAlloc(16)
  DllCall(StrCat(DirWindows(1), "User32.DLL"), long:"GetWindowRect", long:ControlHandle, lpbinary:rect)
  xx = BinaryPeek4(rect, 0)
  yy = BinaryPeek4(rect, 4)
  BinaryFree(rect)
EndWhile
Exit

I was unable to reproduce any memory leak using the DllCall to GetWindowRect.

Can you confirm that you are Freeing all binary buffers using BinaryFree? Missing BinaryFree's can cause memory leaks.

Also I do not expect that this has anything to do with string memory being chewed up...Are you using very large strings or arrays that you are not 'Dropping' in your script?

[kdmoyers]

Yep, I'm doin the BinaryFree

I'll keep lookin for big strings or arrays or other Binary buffers or whatever.

Because the intcontrol(77,0,0,0,0) number goes up in allocation chunks, watching it is not as useful as I had hoped.

Could it also be OLE objects? or would that come out of some other pool of memory?  I do a fair amount of ADO SQL calls.

-Kirby

[Deana]

Yep, I'm doin the BinaryFree

I'll keep lookin for big strings or arrays or other Binary buffers or whatever.

Because the intcontrol(77,0,0,0,0) number goes up in allocation chunks, watching it is not as useful as I had hoped.

Could it also be OLE objects? or would that come out of some other pool of memory?  I do a fair amount of ADO SQL calls.

-Kirby

I would suspect it would only happen with strings and arrays. However using IntControl 77 you can get information about the number of binary buffers and OLE objects that are open. Take a look at IntControl(77, p1, 0, 0, 0) request 30 which can return the number of binary buffers open and request 60 which can return the number of COM/OLE objects open.

[DAG_P6]

Kirby,

Could it be that this

Code (winbatch) Select Expand

DllCall(StrCat(DirWindows(1),"user32.dll"),long:"GetWindowRect", long:hwnd, lpbinary:lpRect)

bleeds a little strings memory every time around because I didn't use a DLL handle?

These string literals are temporary objects. I would expect their memory to be freed as soon as the call returns.  Regardless, when I need a string repeatedly, I define a script variable, named in upper case (following the style used for the C preprocessor constants defined in the headers that define much of the Windows API.

I know there's the DLLLoad function, but I never use it because there didn't seem to be any reason to.  Could this be the reason?

Since user32.dll and kernel32.dll are auto-loaded into practically every process (certainly any GUI process, I've never called DllLoad on either of them. The only time I've needed DLLLoad is when I use my ancient (and deprecated) MD5 digest library, which requires a pair of calls in certain circumstances, the second of which reads data stored in a DWORD that lives in the address space of the DLL. Unless the DLL is loaded before the first call, and called through the handle both times, the second call always returns zero.

[kdmoyers]

OK, further developments on this problem:

I have (exhaustively) reduced this program, eliminating functions, and watching the memory usage.  I fully expected that I would remove some block of code or other and the leak would stop.  Then I would find my mistake. But the leak did not stop. 

Below is a forty-something line program that is, I believe, about as short as it can be and still do a callback dialog with timer.  It exhibits the memory leak.  According to my inexpert calculations, it leaks around 15 bytes per timer event.

I use a program called DebugView http://technet.microsoft.com/en-us/sysinternals/bb896647 to display my results using the DebugData statement, but you can replace that line with a Message statement to get the same results.

Question: what am I doing wrong?
-Kirby

Code (winbatch) Select Expand

    Tick = 100                ; timer tick interval in milliseconds
    memory = ""               ; last memory status
    #definesubroutine dlgcb(DH, DE, DC, undef1, Dextra)
        switch 1
        case DE == 0
          DialogProcOptions(DH,1,Tick)      ; timer ticks in milliseconds
          DialogProcOptions(DH,2,@TRUE)    ; accept push button
        case DE == 1
          z = intcontrol(77,0,0,0,0):'/':intcontrol(77,10,0,0,0):'/':intcontrol(77,30,0,0,0):'/':intcontrol(77,60,0,0,0)
          if z != memory
            DebugData('',"dbg: str/var/bb/com:": z )
            memory = z
          endif
          break
        case DE == 2
          return 1
        endswitch
        return -1
    #endsubroutine

MyDialogFormat=`WWWDLGED,6.2`

MyDialogCaption=`Menu`
MyDialogX=013
MyDialogY=066
MyDialogWidth=200
MyDialogHeight=355
MyDialogMinWidth=100
MyDialogMinHeight=200
MyDialogNumControls=002
MyDialogProcedure=`dlgcb`
MyDialogFont=`Microsoft Sans Serif|8192|40|34`
MyDialogTextColor=`0|0|0`
MyDialogBackground=`DEFAULT,DEFAULT`
MyDialogConfig=2

MyDialog001=`003,001,192,012,STATICTEXT,"st",DEFAULT,"Sales Menu",DEFAULT,20,1024,DEFAULT,DEFAULT,DEFAULT`
MyDialog002=`027,023,036,012,PUSHBUTTON,"push",DEFAULT,"OK",1,10,DEFAULT,DEFAULT,DEFAULT,DEFAULT`

ButtonPushed=Dialog("MyDialog",1)

EXIT

[kdmoyers]

I should add that, on my system, it takes about 4 minutes for winbatch to allocate more strings memory and thereby make the value returned by intcontrol(77,0,0,0,0) change.  On my system, it changes from 65536 to 102400.  2 minutes later, it  changes to 139264.  Thereafter it grows by 36864 every two minutes.
-Kirby

[kdmoyers]

Alternate ending to 2001 A Space Odyssey:

We pick up the action with Dave Bowman pulling boards out of the homicidal HAL computer.

HAL pleads, "Stop Dave, take a pill and we'll talk it over."

Bowman continues to pull boards, one after another, until they are all gone! 

Still, HAL continues, "Dave this is silly, let's play a game of chess.  I'll let you win!"

Bowman, alarmed, cuts power to the entire IT system of the ship.

"Dave, I don't think you're getting enough sleep."

Bowman, in a panic, blows the explosive bolts connecting the two segments of the ship, sending the entire computing core and associated circuits spinning into a passing asteroid.

"Dave, look at the bright side.  There's plenty of your favorite salsbury steak entre now.  No more sharing with Frank!"

Bowman, terrified, suspecting anything with a flashing light, blows his ejection seat and two minutes later triggers the ships self destruct.  From his now distant seat, he sees the ship detonate in a blinding flash.

Finally, he is alone, floating in space, speeding towards his destiny at planet Jupiter.

"Yep, it's just you and me now Dave, best buddies!"

A scream is heard

-- fade to black --

[Deana]

Thank you for providing a simplified code sample. I will have the developers look into the issue.

[td]

Alternate ending to 2001 A Space Odyssey:

We pick up the action with Dave Bowman pulling boards out of the homicidal HAL computer.

HAL pleads, "Stop Dave, take a pill and we'll talk it over."

Bowman continues to pull boards, one after another, until they are all gone! 

Still, HAL continues, "Dave this is silly, let's play a game of chess.  I'll let you win!"

Bowman, alarmed, cuts power to the entire IT system of the ship.

"Dave, I don't think you're getting enough sleep."

Bowman, in a panic, blows the explosive bolts connecting the two segments of the ship, sending the entire computing core and associated circuits spinning into a passing asteroid.

"Dave, look at the bright side.  There's plenty of your favorite salsbury steak entre now.  No more sharing with Frank!"

Bowman, terrified, suspecting anything with a flashing light, blows his ejection seat and two minutes later triggers the ships self destruct.  From his now distant seat, he sees the ship detonate in a blinding flash.

Finally, he is alone, floating in space, speeding towards his destiny at planet Jupiter.

"Yep, it's just you and me now Dave, best buddies!"

A scream is heard

-- fade to black --

I'd like to thank The Academy... Oh wait....   The developers have already looked into it.  It will be fixed in the next release.  I know this may not be an option but  switching to 6.1 dialogs should work as a temporary workaround.

Very nice bit of debugging, BTW.  Many thanx.

[kdmoyers]

It will be fixed in the next release.

Sounds fine. It is a very slow leak.  Not many "call back dialog with timer" programs are asked to run for weeks on end. 
In this case, I'll just throw in a daily restart.

Very nice bit of debugging, BTW.  Many thanx.

You are most welcome!
-Kirby