Winsock FTP problem

[alruma]

Hello,
i have troubles with Winsock FTP list a Directory.
i must read  a directory listing from a remote FTP server with command ftpList.
if the server has only several 100 files in this directory, all is O.K.
but if the server has 6000 files in this directory, then i get error-code 425.
is there a other way to make a dir listing?

best regards
ruedi

[td]

Your description is implying that the problem is the result of the number of files in the target directory. However, the FTP error code 425 means "Can't open data connection" so it is unclear if the problem has the cause you think it has. You might first try to get the listing by using the FTP command from a command prompt. This will allow you to determine if there is a different issue that can be correct by making some changes to your script.  You could also consider other FTP client implementations like the WinBatch WinInet extender's iFtpCmd function.

[alruma]

Thanks for your fast answer.
i tried to access to the server with WinSCP (port 21), and it works well.
but it takes time, to reading the remote directory (ca. 11 sec.)
in a earlier time, i used iFTP from WinInet, but it was too slow (unusable).

regards
Ruedi

[stanl]

Thanks for your fast answer.
i tried to access to the server with WinSCP (port 21), and it works well.
but it takes time, to reading the remote directory (ca. 11 sec.)

There is a sample in the Tech Database using WB's CLR to automate WinSCP but if you feel Session.ListDirectory would be too slow probably not worth the effort.


[EDIT}: there is also System.Net.FtpWebRequest which you can try.

[td]

Good point. Using WinBatch CLR hosting to access MSFT's latest and greatest FTP implementation might provide some improved performance.  Of course that all depends on what is causing the perceived slowness in the first place.

[stanl]

Good point. Using WinBatch CLR hosting to access MSFT's latest and greatest FTP implementation might provide some improved performance.  Of course that all depends on what is causing the perceived slowness in the first place.

The Op asked for alternatives. Below are snippets, speed not important  (1) PS script to automate WinSCP, callable via WB CLR; I've used it for 3-4 years (2) starter code for .NET FtpWebRequest also callable via CLR

Code Select Expand



try
{
    # Load WinSCP .NET assembly
    Add-Type -Path "C:\scripts\wspc\WinSCPnet.dll"
    $listdir = "/home/www/" # could be variable i.e. $listdir = "|DIR|" replaced from script
    # Setup session options
    $sessionOptions = New-Object WinSCP.SessionOptions -Property @{
    Protocol = [WinSCP.Protocol]::Ftp
    HostName = "ftp.myhost"
    UserName = "user"
    Password = "pw"
    }

    $session = New-Object WinSCP.Session

    try
    {
        # Connect
        $session.Open($sessionOptions)

        $directory = $session.ListDirectory($listdir)
        $csv = "FileName," + "Size," + "Permissions," + "LastModified" + "`r`n"
        foreach ($fileInfo in $directory.Files)
        {
            $var = $fileInfo.Name +"," + $fileInfo.Length + "," + $fileInfo.FilePermissions + "," + $fileInfo.LastWriteTime + "`r`n"
            $csv = $csv + $var
        }
    }
    finally
    {
        # Disconnect, clean up
        $csv | Out-File c:\temp\ftpfiles.txt
        $session.Dispose()
    }

    exit 0
}
catch
{
    Write-Host "Error: $($_.Exception.Message)"
    exit 1
}




Code (WINBATCH) Select Expand



;WB - CLR FtpWebRequest
ObjectClrOption("useany", "System")
ObjectClrOption("useany", "System.Net")
request = ObjectClrNew("System.Net.FtpWebRequest")
creds =  ObjectClrNew("System.Net.NetworkCredential","user","pw")
request.Create("ftp://myhost")
methods = ObjectClrNew("System.Net.WebRequestMethods")


;These would take some special handling
;request.Method = methods.Ftp.ListDirectory
;request.Timeout = 10000
;request.ReadWriteTimeout = 10000
;request.KeepAlive = @False
;request.UseBinary = @True
;Response = request.GetResponse()
;Stream = request.GetResponseStream()




[alruma]

Hello,
Thanks for your tips ant explanations.
I have now this question:
How does Winsock ftpOpen works, in active-mode or in passive-mode?
And is there a option, to select the mode?
best regards
ruedi

[td]

I believe that the WinSock extender connects using active mode. I know of no practical way to set up a lazy socket to change that.

You could send a "PASV" command to the server using the FtpQuote function but that doesn't mean that the extender will use the server specified port for data transfers.

[td]

My apologies, I forgot about the ftpFirewall function's ability to set up "passive" mode and I think you can use it without having a firewall.  Check out the function's Proxy Type 16.

[alruma]

Hello,
The problem is now solved by two steps.
1. The main problem was, that port 20 was not open in our filewall, and now Winsock/ftpList works.
    The strange situation was, that the firewall-rules for the servers with several 100 files (Testing) had port 20 open and work well.
    But he firewall-rules for the servers whit several 6000 files (Production) had port 20 blocked and therefor not working.

2. Anyway, i changed the FTP-part to sFTP by using WinSCP, as you described earlier, and it seems to work well.

So now i say MANI MANI thanks for your great and fast support
and best regards from switzerland

ruedi 

[td]

Are you using WIL CLR hosting or PowerShell or neither or both to use WinSCP?

[td]

In case you missed it, here is a link to the WinSCP + dotNet example in the Tech Database.

https://techsupt.winbatch.com/webcgi/webbatch.exe?techsupt/nftechsupt.web+WinBatch/dotNet/Third~Party~Components/WinScp+SFTP~using~WinSCP~dotNet~.txt

[alruma]

Just now, for a fast solution, i call WinSCP as a external programm by using sessions:

WinSCP.com ' %SFTP_LogLevel% /command "open %SFTP_session%" "option batch on" "option confirm off" ls %SFTP_Source%" "exit" >%SFTP_Dest%'

But i am developping a solution with CLR in that way:

WinSCP_Location = StrCat(act_dir, "WinSCP")
ObjectClrOption( 'appbase', WinSCP_Location )
ObjectClrOption('use','WinSCPnet, Version=1.7.2.11087, Culture=neutral') ; Confirm version matches the dll

;Create an instance of the WinSCP.SessionOptions class and fill in all necessary information to allow an automatic connection and authentication of your session.
objSessionOptions = ObjectClrNew( 'WinSCP.SessionOptions' )

; Define Protocol
;  Possible values are Protocol.Sftp , Protocol.Scp and Protocol.Ftp.
objProtocol = ObjectClrNew( 'WinSCP.Protocol' )

If StrUpper(WinSCP_prot) == "SFTP"
  Sftp = ObjectClrType('WinSCP.Protocol', objProtocol.Sftp ) ; Sftp
  objSessionOptions.Protocol = Sftp
Elseif StrUpper(WinSCP_prot) == "FTP"
  Ftp = ObjectClrType('WinSCP.Protocol', objProtocol.Ftp )   ; Ftp
  objSessionOptions.Protocol = Ftp
Elseif StrUpper(WinSCP_prot) == "SCP"
  Scp = ObjectClrType('WinSCP.Protocol', objProtocol.Scp )   ; Scp
  objSessionOptions.Protocol = Scp
Else
  IntControl (1000, 13, 0, 0, 0)                           ; Sets the exit code returned by WinBatch  0 -> no error  >0 -> several errors
  WinSCP_error_text = StrCat("PRG_ERROR  2 %WinSCP_CallExch% from: %PRG_Pname% at:  ", TimeYmdHms(), " Reason: WinSCP_error_text: Protocoll wrong", " WinSCP_prot= ", WinSCP_prot)
  Display(10, "ERROR", WinSCP_error_text)
  Write_File(WinSCP_Logf, WinSCP_error_text, "APPEND")
Endif   

objSessionOptions.HostName = WinSCP_host                               ; MODIFY TO FIT YOUR NEEDS
objSessionOptions.UserName = WinSCP_user                               ; MODIFY TO FIT YOUR NEEDS
objSessionOptions.Password = WinSCP_pw                                 ; MODIFY TO FIT YOUR NEEDS

If IsDefined(Sftp)
  objSessionOptions.SshHostKeyFingerprint = "ssh-rsa 1024 xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx" ;REQUIRED by SFTP     ; MODIFY TO FIT YOUR NEEDS
Endif

;Create an instance of the WinSCP.Session class. Optionally you can hook handlers of some events of the class.
objSession = ObjectClrNew("WinSCP.Session" )

;.Open the session using Session.Open method, passing instance of your WinSCP.SessionOptions.
objSession.Open(objSessionOptions)

;WinSCP-LIST a file or directoty
If StrUpper(WinSCP_Choice) == "L"
  directoryList = objSession.ListDirectory(WinSCP_Source)
  ForEach fileInfo in directoryList.Files
    name = fileInfo.Name
    type = Num2Char(fileInfo.FileType)
    length = fileInfo.Length
    perms =  fileInfo.FilePermissions
    modified =  fileInfo.LastWriteTime
    Pause( name, 'type ':type:', with size ':length:', permissions ':perms:' and last modification at ': modified )
  Next
Endif