Friday, October 23, 2009

Why Secure Managed File Transfer instead of FTP?

First of all, I think we need to understand why FTP? 

FTP is an old legacy network protocol.  The first FTP standard was RFC 114, published in April 1971, before TCP and IP even existed.

Objectives of FTP, as outlined by its RFC, are:  
To promote sharing of files (computer programs and/or data).
To encourage indirect or implicit use of remote computers.
To shield a user from variations in file storage systems among different hosts.
To transfer data reliably, and efficiently.

FTP does work and works decently well for users who know how to utilize the system. To learn how to use FTP is a different matter.  There are hundreds of different FTP clients, different ways to access FTP, and even through the web browser.  One major challenge to users is that FTP is dependent on the FTP server itself and how it is configured and whether it supports more advanced features.  Not all FTP clients are the same, thus this causes confusion with users.  Once a user gets used to that client they pretty much stick with it, unless it does not work with specific FTP servers.  FTP is used because it has been the default standard in transferring files as there were not many other ways to transfer files efficiently.   Most other legacy file sharing protocols were designed for the LAN or over modems, or other propriety devices.   To this day only a small percentage of people who transfer files know about Managed File Transfer.  I believe this was  a term developed by the Gartner Group so that they could focus a consulting practice around this area.  

File transfers have been typically performed through several methods:
1) FTP

2) LAN Protocols with File sharing: SMB, SAMBA, NFS, CIFS, AFP.

3) Email

4) Sneakernet:  Burn to CD, DVD, USB Memory stick, etc., then send by courier, Fedex, Postal service, etc. 

5) HTTP

6) EDI

7) Message Queuing/Application and Business Integration Systems, however most of these systems default to FTP when integrating externally with business partners. 

Managed File Transfer is a term now mixed up with many different technologies, protocols, and solutions.  The market space now is confused.  How do you know which system or vendor to go with? 

Managed file transfer market was defined as a system to facilitate the secure transfer of data from one computer to another through a network.  It has evolved to a much more expanded view by many vendors as the ability to guarantee transfer of files securely, efficiently, with visibility, logging, audit-ability,  supporting multi-protocols, alerting and automated processes.   What a mouth full.  Do we as file transfer users care if we have all of these features?  Probably not.  We probably need many of these features which will make our lives easier. 

When assessing your file transfer solution, you need to examine a two things:
Cost of ownership and and return on investment. 

Cost of ownership:

How much is it costing your organization by utilizing FTP? 

How much time does it take to provision new accounts, properly secure and provide the proper rights to folders and sub-folders? 

Can you integrate the provisioning and decommissioning of users through integration with Active Directory and/or LDAP or other databases? 

How much time does it take to clean up old, stale files? 

Do your users store files on the FTP server because they believe it's a permanent storage device? 

Do you remove users who don't access the system on a periodic period?  

What is the risk of having your login and password credentials compromised? How would that impact your organization? 

How many calls are received by your help desk because someone couldn't use the FTP server or their FTP client because they didn't know how to?

How much storage do you have dedicated to your FTP Server? 

Cost of implementation.  How complex is the solution?  Is it out of the box or do you need customization, custom code written? 

How flexible is the system to mold around your specific workflow process?  Do you have to take the system down to make the proper changes?   How much downtime can you afford and risk? 

Return on Investment:

Gains in productivity: 

How much time is actually spent babysitting/watching the file transfer progress? 

How often does the file transfer fail? 

How do you know that the file sent is exactly the file received by the other party? 

Have you ever received a corrupted file? 

How many times have you had to resend a file?

Do you wonder if the file has actually been received? How many times have you picked up the phone to alert the other party that files are ready to be picked up? Let them know where it its on the folder structure by sending emails or hand holding them through the phone? 

How many times have you had to check the server to see if files have arrived or sent? 

Do you ever zip or archive the files to protect the file integrity, file names, etc., before sending? and after you've received the files? How long does that process take for both users? 

How much time is spent manually shipping files? Burning to CD, DVD, USB Memory, then getting someone to package and ship them for you?  How much does each package cost to ship?

How easy is it to do business with you now? Does that help you gain more future business?

Can you bring to market a product quicker because you are reducing your production cycle times? 

As you can see, it's probably very easy to justify a low cost product to replace FTP.  

The reduction of cost of ownership with the benefits and the return on investment should easily be justified.  

When you get into more complex file transfer workflows then you need to be responsible on understanding the specific workflow interaction by the human factor and what can be improved by taking out manual processes or human intervention.  If the workflow tends to be a high volume workflow then it is best to deploy a system to system automated solution.  With this type of system you still need the visibility into the transactions through a logging mechanism.  Some MFT systems like  Group Logic's MassTransit has the ability to perform valuable business intelligence on the file transfer transactions through trending and charting the volume of file transfers through, specific time frames or specific users.    

There of course are several features you will need as standard features on any file transfer system.  
1) Secure file transfers with encryption both on the login credentials as well as the file transfers. 

2) Logging for governance, compliance and security.

3) Custom alerting through email, SMS Text, and other means most appropriate for your business needs.

4) Worry free guaranteed speedy transfers, through automatic re-queuing of files, file integrity validation, utilization of the appropriate optimized network protocols, proven track record of scalability and reliability. 

5) Easy to use by all and easy to manage.