iFTPFileSize Error 300: FileOpen failed when Firewall active?

[IJRobson]

I have written a Winbatch application that logs into an FTP Server and downloads any new Files found on this Server.

This system has been working for years, but last week this system stopped working.  It still successfully logs onto the FTP server and obtains the list of Files available.  It then uses iFTPFileSize to obtain the Size of the file it wants to download but instead of the size being returned I get a 300 Error: FileOpen failed.

I have investigated and found that I get this error if the Windows 7 Firewall is active.  With the Firewall switched off the System works perfectly.  We have not changed any Firewall Settings but I guess Microsoft may have patched something in a Windows Update?

I have rebuilt the system using the latest version of WinBatch and Extenders and get the same error.  I have also checked the application on other computers and they work so the problem does relate to this Computer and the Firewall.

Any ideas what Windows 7 Firewall rule or Ports I should open to allow iFTPFileSize to work with the Firewall active?

Thanks

[Deana]

What SERVICE value (@FTP or @FTPPASSIVE) are you specifying in your iHostConnect statement?

If specifying @FTP, maybe try using @FTPPASSIVE instead...

Also, Are you able to FTP the file using Windows built in FTP.exe or any other FTP program? Check the passive transfer settings in that application.

[IJRobson]

I am already using @FTPPASSIVE:

conhandle = iHostConnect(tophandle, Host, @FTPPASSIVE, UserID, Pswd)

[Deana]

Are you able to FTP the file using Windows built in FTP.exe or any other FTP program?

[IJRobson]

I have not tried any other FTP tools, I can try that next week when I get access to the Machine again.

My FTP system has been tested on 10 different machines (nine Windows 7 and one Windows XP) and 8 of the 10 work fine with the Firewall enabled.  The two that fail always fail at iFTPFileSize command with the "FileOpen failed" Error.  And both work fine with the Firewall Switched off.

I have checked the file names and path information of these files with and without the Firewall ON and the same information is returned.  All the previous FTP initialization and FTP file search commands are working fine and don't report any errors it just gets to the iFTPFileSize command and fails?

I have used the "NETSH ADVFIREWALL" command to extract the Firewall settings to try and compare them between the working and non-working machines but that will take a while to sort through.

They are all accessing the same files on the FTP server and looking at the Logs on the FTP server there is no FTP Errors reported related to these failing machines.

Thanks

[Deana]

Maybe take a look at the wwwbatch.ini file on the failing machines. It might contain some extended error information that might be useful. The extended error info will be located under the heading [wwwint44i].

IE is responsible for WinInet. You might also want to check the version of IE installed on the failing systems and compare them to the working systems.

Reference:
https://connect.microsoft.com/IE/feedback/details/809080/problem-with-ftp-wininet-after-installing-ie11
https://connect.microsoft.com/IE/feedback/details/809393/windows-8-1-upgrade-ie11-ftp-problems#
http://stackoverflow.com/questions/19683291/wininet-from-ie-11-randomly-returns-error-12003-for-most-ftp-functions 

[IJRobson]

Thanks for the information, I will take a look at the wwwbatch.ini file next week.

[Deana]

After some research it appears lots of tools that depend on WinInet functionally began to have issues when IE 11 is installed:
http://stackoverflow.com/questions/19683291/wininet-from-ie-11-randomly-returns-error-12003-for-most-ftp-functions

Does the FTP server support active connections? If so try changing @FTPPASSIVE to @FTP.

[IJRobson]

I will check with the IT department if Active connection is supported.

The only thing that concerns me about this is the Firewall appears to the be problem?  Nine of the Machines are running Windows 7 with IE 11 only two have this problem?  And I can fix this problem by switching off the Firewall.  If this is related to IE then why does the Firewall have an effect.

All the Machines have been updated with the latest Windows (and IE) Updates so surely they should all fail, or I should get Random failures.  On these two machines I get 100% failure rate with Firewall and 100% success without Firewall.

My Software also has the ability to upload files back to the FTP Server.  This functionality works on all of the Machines.  Which implies this problem relates to In Bound FTP traffic through the Firewall.

I believe FTP uses two ports one for Commands and one for Data.  Because the FTP connection can be established and Folder / File names obtained I was wondering if these are using the one port but the iFTPFileSize is using the other Port?  If this second port was blocked then I guess that would cause this issue?

[Deana]

The iFtpFindSize function is attempting to open the file in READ mode, using the Wininet Api function FtpOpenFile: http://msdn.microsoft.com/en-us/library/windows/desktop/aa384166%28v=vs.85%29.aspx

The error code from the call to FtpOpenFile is written to the wwwbatch.ini file and stored in the IntControl 73 error handlers wberroradditionalinfo variable. You might consider adding error handling to capture the error on the faulty machines. for more on error handling check out: http://techsupt.winbatch.com/webcgi/webbatch.exe?techsupt/tsleft.web+Tutorials+Trap~Errors.txt

[IJRobson]

I already have an Error Handler within the application that writes information to a LOG file.  I will add wberroradditionalinfo to this LOG to see if it tells me some more information.

I will investigate and update the Post once I have some more details.

Thanks for your help.

[Deana]

Currently I suspect you might be dealing with this IE11/WinInet bug: https://connect.microsoft.com/IE/feedback/details/808279/ftpopenfile-and-internetwritefile-broken-changed-in-ie11

[IJRobson]

Thanks for the Link I will investigate further but if this problem is Microsoft, then I would be surprised if a fix is released soon especially as they are scrapping IE11 ready to release IE12?

I would like to blame Microsoft for this but I still not convinced?

• All the Windows 7 Machines are running the same Patches and Version of IE11 but only two fail FTP?
• The users on the Microsoft Forum report problems Uploading files I don't have any problems Uploading only Downloading.
• Other FTP tools like Firefox FTP client and CuteFTP also have problems downloading Files.
• Disabling the Firewall works 100% of the time.

I am running some more tests and I will let you know what I find.

[Deana]

So other FTP Tools are also failing?

Don't forget to share the WWWBATCH.INI from a failing system. It may provide a clue.

[IJRobson]

Yes, other FTP tools are also failing.

I am in the process of getting held of the WWWBATCH.INI off these machines.  But as machines are on another site and we use FTP to Access them that is proving to be difficult.

[IJRobson]

OK WWWBATCH.INI reports:
      [WWINT44I]
      LastError=12002 (iFtpFileSize(FtpOpenFile))

[Deana]

Yes, other FTP tools are also failing.

Okay so this confirms the problem is not limited to the FTP functions in the Wininet Extender (as I suspected)...the problem is a bigger issue on those systems that will need to be addressed.

I am in the process of getting held of the WWWBATCH.INI off these machines.  But as machines are on another site and we use FTP to Access them that is proving to be difficult.

Yes please. I am still curious to see what the underlying WinInet API is returning for an error ( if anything ). You could ultimately add code to your script to capture for this error and skip the iFtpFileSize calls.

Also does the ftp server support active connections? If it does have you tried using @FTP? Does that have any effect on the problem systems?

[Deana]

OK WWWBATCH.INI reports:
      [WWINT44I]
      LastError=12002 (iFtpFileSize(FtpOpenFile))

That error indicates: ERROR_INTERNET_TIMEOUT. 12002 .

You might try adding code to your script to increase the timeout using iOptionSet.

control_send_timeout ( FTP Control channel send timeout ) by default is 300000.
control_receive_timeout ( FTP Control channel receive timeout ) by default is 300000.


Notes regarding the firewall: In Passive mode, the FTP server opens a port on its end and tells the FTP client the Port it needs to connect to. On the client end, it creates a new socket, binds it to a local Port and connects to the server's specified Port. Without seeing a trace log from a network packet sniffer,  I would guess that the firewall on the client end is possibly blocking the outbound connection.  Have you tried and active connection (@FTP) instead?

[IJRobson]

Interesting, a Timeout error when the connection is less then 1 Second old?

The Timeout has not been changed so it should still be 300000?

I am still waiting to see if the FTP server supports an Active Connection.  The FTP Mode is hard coded in the EXE so will require a new release of the EXE, which will have to be done via FTP so I don't want to do that until I know it will work with the FTP Server!

[Deana]

Interesting, a Timeout error when the connection is less then 1 Second old?

Yes that is what the SYSTEM is returning to WinBatch.

The Timeout has not been changed so it should still be 300000?

Yes if not specified the default timeout is 300000 milliseconds. I recommend using iOptionSet to set the value higher to see if you can work around the timeout.

[td]

The timeout error could very well be a red-herring.  Windows errors are notorious for requiring a good deal of imagination to make a connection between the error and the actual cause of the problem.   Also, the OP's analysis is sound and certainly points to a firewall setting or lack thereof being the cause of the problem. 

The firewall on Windows 7 has a lot of configuration possibilities for any given protocol so it is not to hard to image that variations between machines exist.     

[wagert]

Hmmm..

I just was researching an issue i started having with iURLOpen when i came across this thread.

I have some code where when i 'run' it in Studio, works ok.. but when i 'debug' in Studio, fails..   iURLOpen completes ok when run, but fails when debugging.

Is this issue related to the errors being seen with IE11 and FTP discussed in this thread?

In any case,

I don't know why it's working when run and fails when debugged. 

the error i get in debug is 12029 ..  but no when run

Any suggestions?

Thanks, Scott

[IJRobson]

Scott,

I would be surprised if your Problem is related to this thread.

I would guess the Problem relates to "Paths"?

When you run a WinBatch Script in the Studio in normal mode it runs relative to the folder were the WBT file is located, however in DEBUG Mode it always appears to start in the Windows Folder.  This means that any commands expecting to find Files in locations relative to the Working Folder don't work.

Check your Paths and were possible always use Full Path and File name definition.

[IJRobson]

I have checked with IT and the FTP server does support Active Connections but the Network Firewall does not.  So in theory I could use an Active Connection within the local Network but as the problem Machines are external to that Network the Firewall will block the traffic so would not help the investigation.

I still think this is Firewall Rule related so I am setting up new Rules for FTP and my Application to see if this fixes the Problem.

[Deana]

Hmmm..

I just was researching an issue i started having with iURLOpen when i came across this thread.

I have some code where when i 'run' it in Studio, works ok.. but when i 'debug' in Studio, fails..   iURLOpen completes ok when run, but fails when debugging.

Is this issue related to the errors being seen with IE11 and FTP discussed in this thread?

In any case,

I don't know why it's working when run and fails when debugged. 

the error i get in debug is 12029 ..  but no when run

Any suggestions?

Thanks, Scott

It does not sound like a related issue. Please see: http://techsupt.winbatch.com/webcgi/webbatch.exe?techsupt/tsleft.web+WIL~Extenders/WinInet+iGetConState~Returns~False.txt

If you continue to have an issue create a new forum post with details.

[Deana]

This looks interesting, so I thought I would share: http://support.microsoft.com/kb/2754804

See also: http://stackoverflow.com/questions/6990663/java-7-prevents-ftp-transfers-on-windows-vista-and-7-if-firewall-is-on-any-idea

[td]

Hopefully, the OP isn't working with systems that haven't been patched in a couple of years but you never know... 

[IJRobson]

I ensured all the Machines (working and failing) had been updated with the latest Windows Updates before raising this question on the forum.

Looking at the Hot fix Support Page the Version of the Files listed are the ones installed on these Machines which implies they have not been patched?

I have obtained the Hot fix and I plan to install it on one of the two Machines to see if it fixes anything.

This test will not happen until next week, but I will update you once I have some results.

Thanks

[td]

I ensured all the Machines (working and failing) had been updated with the latest Windows Updates before raising this question on the forum.

Looking at the Hot fix Support Page the Version of the Files listed are the ones installed on these Machines which implies they have not been patched?

Actually, it implies that the hotfix has been installed.  However, I am surprised that your files aren't newer than the ones listed for the hotfix, if you keep your systems up-to-date.  MSFT has patched some of those files several times since this older hotfix was released. I keep my Win7 workstation up-to-date and I have never installed a Windows hotfix other than those provided by Windows Update but all of the hotfix related files on my workstation are more resent version than the files listed for the hotfix.

[IJRobson]

That's odd because if all our Machines are up to date with Microsoft Updates they they should all have the same Versions of the Files?

It is possible someone has hidden non-installed updates so we will check.

The other possibility is that the Updates you have installed are part of other Systems you have installed on your Machine?  Our machines basically just have a standard Windows 7 installation and my WinBatch Applications.  They do not have any third party software, IE extensions or things like flash, java .net installed.

[td]

The files on my 64-bit workstation are directly from an MSFT Windows Update released as part of the September/October of last year patch Tuesday cycle.  Since these files are MSFT system drivers (as apposed to OEM device drivers), their installation is likely not dependent any software or hardware installed on the system other than the version of the OS and the hardware architecture i.e., x64, x86, Itanium, or ARM.

Of course, this issue is moot since you said your files match the versions listed on the hotfix page, which means you already have the fix.

[IJRobson]

Yes, the patch is done but I am still concerned why the patched files have not been updated again with newer Updates as yours have?

[td]

MSFT's patch system works in mysterious ways.  The files on my system may be from a recommended but not critical MSFT patch.  Some companies vet each patch before distributing them to individual systems.  Other companies use custom PE images.   Any of these things as well as many others could affect which patches are installed on a system.

For critical systems, I do not allow automatic updates.  Instead I wait until the monthly patches have been vetted before manually installing them.  This means, for example, that on my workstation some patches don't get installed until a patch for the patch has been released and others never get installed. 

[IJRobson]

OK, that makes sense and may also explain how a set of Machines on one site could have different versions of the Files to another Site.  I do know our IT department vet the Patches before distribution, other sites may have different rules on patches.

We are running some more tests on these Machines this week so hopefully we will have narrowed down the problem / cause by the end of the week.

Thanks.

[IJRobson]

I am still investigating this but I have found if I use iFTPFileExist before the iFTPFileSize command then this appears to report the File does not exist and does not crash unlike iFTPFileSize.

It does not solve the problem but at least now I know when it will not work.  So whatever the cause it makes the File display as far as the iFTPFileExist and the iFTPFileSize  commands are concerned.