Transferring Large Files

Posted by: Scott on Writing, on 19 Sep 2008 | View original | Bookmarked: 0 time(s)

Every couple of weeks or so I seem to have a need to transfer (or receive) a relatively large file to another colleague or client. It might be a database zipped up into a 50 MB file, artwork totalling hundreds of megabytes, or various DLLs and source code. Over the course of my career I have used the following techniques for transferring data:

  • E-mail - the most straightforward technique for sharing data. The major con is that many email providers reject attachments over a certain threshold, usually in the 2-10 MB range. Also, e-mail is inherently insecure, making it a non-ideal medium for transferring sensitive data like passwords, credit card numbers, social security numbers, and so on.
  • Upload/Download from a Website - if you have write access to a website you can always upload the data to share and then send a URL to the file(s). The downside here is that you need a website. Security also requires a few extra steps - you need to have some mechanism to only allow the intended recepient from downloading the article.

    Side story: Back in 2002 Microsoft released Web Matrix, which later became Visual Web Developer Express Edition. Prior to announcing the release Microsoft and asp.netPRO Magazine asked me to write an overview of the tool to be included in the asp.netPRO edition that came out right when the product was released officially. There were a group of other ASP.NET folks at the time who also knew about this project and were invited to review my article. I distributed it by putting the article up on a website and provided this group a link to download the article. A couple days later I left for a short vacation during which I did not check e-mail or phone messages.

    The day I left town one of the reviewers leaked the article to Neowin.net, which linked directly to the URL on my website. Shortly after the link was published on Neowin.net I started getting e-mails and phone calls from Scott Guthrie asking me to take down the link. Of course, I did not get these messages until a few days later upon return from my vacation.

    I never did find out who leaked the link/article. I guess it didn't really matter in the end, and it wasn't the article on my server that caused the leak as someone else sent in the link to Neowin.net and they could have just as easily sent the Microsoft Word document. But still, I was rather red-faced when having to explain to the guys at Microsoft why the article was accessible to anyone on the Net and why it took so long to get the article down once it had been compromised.
  • Use a Third-Party Service - there are a plethora of web-based services that are built to allow large files to be transferred between two individuals. Most work by you uploading a file to a site, where it is stored temporarily. You then e-mail the recipient a link to download the attachment. I have used the free service TransferBigFiles.com many times and heartily recommend them. You could also use DropBox and place the file to share in the Public folder. The benefit of this approach is that you don't need a website; however, the security concerns remain. In fact, you might consider the security concerns even (slightly) greater because you are now adding an intermediary into the equation (the folks at TransferBigFiles.com or at DropBox).
  • Physically Deliver the Data - the most secure option is the physically deliver the data - put it on a thumb drive or burn it to a DVD and then mail or hand deliver the data. This approach takes the most work, but is the most secure, especially if you hand deliver the data. And if you are transferring extremely large amounts of data - terrabytes, let's say - this approach would likely be the fastest.

    I used this approach 8 years ago when transferring large amounts of information between myself and a publisher. At the time I only had dial-up access. When the publisher needed to send me several hundred megabytes worth of materials - mostly artwork from previous editions - it was more efficient to mail me a CD. I also had a student in one of my Web Services class, must have been four years ago or so, whose company was running computer simulations that would output several GBs worth of data per test run. To send the day's test data to another facility they would plug in an external hard drive, copy over the data, and then use a mail carrier to overnight deliver the drive. I wonder what techniques they use today.

How do you share large files with others?

Category: Security | Other Posts: View all posts by this blogger | Report as irrelevant | View bloggers stats | Views: 1463 | Hits: 18

Similar Posts

  • ASP.NET 4 Web Server Here Shell Extension more
  • Announcing Microsoft Ajax Library (Preview 6) and the Microsoft Ajax Minifier more
  • ZipBuilder - A zip package creation utility for collections of large files more
  • Your idea will probably change more
  • Agile Architecture Up Front (ADUF) more
  • Creating a folder inside the ZIP file with System.IO.Packaging more
  • The State of Things A Brief Review Before Tech Ed. more
  • Store images efficiently in SQL Server 2008 more
  • Hey, nice package! more
  • IIS 7 Compression. Good? Bad? How much? more

News Categories

.NET | Agile | Ajax | Architecture | ASP.NET | BizTalk | C# | Certification | Data | DataGrid | DataSet | Debugger | DotNetNuke | Events | GridView | IIS | Indigo | JavaScript | Mobile | Mono | Patterns and Practices | Performance | Podcast | Refactor | Regex | Security | Sharepoint | Silverlight | Smart Client Applications | Software | SQL | VB.NET | Visual Studio | W3 | WCF | WinFx | WPF | WSE | XAML | XLinq | XML | XSD