FileSender 1.5 released!

Today Xander announced the release of FileSender 1.5.  No beta, no release candidate, this is it: 1.5 release.  The 1.5 release is our most stable release and we recommend it for running production.

For details please read the announcement made by our release manager Xander Jansen to the filesender-dev mailinglist about an hour ago.  Please report any issues you find to the FileSender support mailinglist.  Like Xander I want to express my thanks to the FileSender team and all those who contributed to this important milestone for our FileSender services.

The 1.5 Release Candidate 1 has been running in production on the AARNet, UNINETT and SURFnet services for more than 10 weeks and has proven to run stable and without problems.

The FileSender team is now gearing up for its next targets:

  1. version 1.6 with the results of the FileSender Terabyte Challenge as an experimental feature, to allow field testing ASAP
  2. version 1.7 with the Quick Wins A features
  3. a version 1.8-beta with the multi-file upload feature

Xander’s announcement follows verbatim:

Date: Sun, 03 Mar 2013 23:08:27 +0100
From: Xander Jansen <Xander.Jansen@surfnet.nl>
To: filesender-dev@filesender.org
Subject: [filesender-dev] FileSender 1.5 release now available

Hi,

We’re happy to announce that FileSender 1.5 release is now available for download and in the FileSender package repositories.

Version 1.5 is our latest production release. Apart from a few minor patches, it is essentially a re-release of the 1.5-rc1 release candidate that has been happily running on at least four production sites over the past few months.

Documentation is available at:

https://www.assembla.com/spaces/file_sender/wiki/Documentation_v1-5

If you are upgrading from a previous release be sure to read:

https://www.assembla.com/wiki/show/file_sender/Upgrade_notes

which contains *important* installation and upgrade notes.

Download details are at:

https://www.assembla.com/spaces/file_sender/wiki/Release_1-5

For those of you using the Debian or RPM package repositories: 1.5 packages are available in the *testing* repository and will be moved to the *stable* repositories as soon as 1.5 will become the main supported release. If you want to upgrade to 1.5 now please switch your apt/rpm configuration from ‘stable’ (or ‘unstable’ if you are running the 1.5 development packages) to ‘testing’.

The noteworthy changes since the previous major and minor releases (ticket numbers listed in parentheses) are:

Major changes since 1.1.1:

  • User interface is now HTML based, no Flash required except when uploading files with a non-HTML5-capable browser
  • Multilanguage support in web interface
  • Support for multiple database backends (PDO)
  • Changes in configuration (see Upgrade Notes)

Changes since 1.5-rc1:

  • Added Serbian language file (#865) (ok, that’s a major change 😉
  • Updated language definitions (#666, #706, #720, #880)
  • Consolidated filename validation (#863)
  • Don’t insert guest voucher(s) in case of validation errors (#862)
  • Allow spaces in configurable setting for admin users (#869)
  • Restrict access to admin page skeleton (#881)
  • Include version in call to default.css to prevent upgrade issues (#834)
  • Add color style for links in default.css (#837)
  • Debian packaging: ‘default-mta’ added to dependencies (#710)
  • Documentation updated

We would like to thank you all for your patience and of course your input and contributions. A long list of people and organisations that made this release possible can be found at:

https://www.assembla.com/wiki/show/file_sender/Acknowledgements_for_v1-5

And last but not least, please keep the patches, comments and translations and all other useful input coming!

On behalf of the FileSender team,

Xander

Terabyte challenge: preliminary results

We’ve got some exciting preliminary results from the work on the FileSender Terabyte Challenge! With the report only a couple of weeks away I won’t write too much for now and let the screenshots speak for themselves.   It looks like UvA System and Network Engineering students René Klomp and Edwin Schaap cracked the problem.

After confirming their hypothesis on the FileSender performance bottlenecks the students wrote a small library to demonstrate a solution.  A standard FileSender 1.5 trunk code installation was modified to use this library.  This installation was used today for a first full 1TB upload test over a 0 ms latency path.   The file took a little under 3.5 hours to transfer from laptop to server.  More testing with longer latency paths will follow.  Congratulations René and Edwin!

Update 25.1.2013: I forgot to mention that during this test the external disk wouldn’t go faster than 800Mbit/s despite it being eSATA.  One might imagine this has an impact on the transfer speed of the file.

Update 25.1.2013, #2: AARNet installed the prototype code on their standard service platform. and ran a quick test from Perth to Brisbane, a 67 ms rtt path, using a standard off the shelf employee’s laptop.  Standard filesender code:162 MB file in 220 seconds.  New webworker filesender code: 162MB file in 16 seconds.

Update 28.1.2013: we performed tests between Trondheim and Utrecht on a 40ms rtt path using the test installation set up by René and Edwin’s test installation in Utrecht and my Macbook Pro in Trondheim.  The laptop is a November 2012 model with 8GB RAM, OS-X 10.8 and the latest FireFox.  The network path has not been baseline-tested.  A 2GB file took about 68 seconds to upload when the disk cache was purged first.  If the entire file was cached, or if the file was read from a RAM disk, it took about 45 seconds to upload.  René and Edwin are considering the feasibility of a read-ahead buffer.  To compare: from the same desktop, disk cache purged,  to Xander’s stock 1.5-rc1 FileSender installation in Amsterdam, on a 37 ms rtt path, it took a 2GB file ca. 340 seconds.  The same upload repeated immediately afterwards without purging the disk cache, implying a read from disk cache, took about 330 seconds.

filesender-tb-challenge-1tbtransfer-Screenshot from 2013-01-24 11_15_01

Transfer of 1TB file starts at 11:15

1TB transfer at 83% around 14:00 (2 hours 45 minutes later)
1TB transfer at 83% around 14:00 (2 hours 45 minutes later)

1TB transfer at 83% around 14:00 (2 hours 45 minutes later)

Email received after succesful upload first 1TB file

Upload completed and email sent at 14:34.  Note the email says 14:44 due to graylisting at recipient

Unlimited file size now also with IE10 preview for Win7 and Safari 6.0.1 on Mountain Lion

The FileSender project made a decision some years ago to use HTML5 FileAPI functionality to support file uploads of any size.  Although only supported by FireFox at that time we were confident that the rest of the browser vendors would catch up and Google’s Chrome did so pretty fast.  Intenet Explorer and Safari users unfortunately had to wait and could either switch browser when using FileSender, or saw their uploads limited in size to 2 Gigabytes.  But now the wait appears to be over.

Safari 6.0.1 on Mountain Lion supports HTML5 uploads and has been tested with a 100GB upload:

Screen Shot 2012-12-03 at 2.57.20 PM

Testing of the Internet Explorer 10 preview on Windows7 also looks positive, with 25 GB uploads working like a charm.

It will take a while before the new Internet Explorer and Safari versions will have filtered through to all relevant users.  Even so, we think it’s great news the browsers used by the vast majority of people in our community -Firefox, Chrome, Safari and IE- now support uploads without size limitations.  Thank you open standards, thank you HTML5 FileAPI.

 

FileSender Terabyte Challenge

The University of Amsterdam offers a master programme in System and Network Engineering, SNE.  As part of the curriculum, SNE students execute two research projects (RP) of 1 month each, one in January and the other in June.  Each RP stands for one month of very intensive work by two students collaborating to achieve a lot in a short amount of time.

We submitted a proposal to this year’s Research Project 1 (RP), to be executed in January 2013: the FileSender Terabyte Challenge:

“Current upload speeds with FileSender are nice for files up to several GBs.  We want to enable use of FileSender to transfer a 1 TB file in a reasonable amount of time (5 hours on a low latency path) using a standard web browser on a standard Windows or Mac desktop.  Identify current performance bottlenecks and design possible solution strategies which hold as latency increases.”

The assignment piqued the interest of 3 different student groups and following an interview round we selected René Klomp and Edwin Schaap for the assignment. I’d like to thank them for their interest and look forward to their results!

René and Edwin chose to work from the Netherlands and will present their results on Wednesday February 6th 2013 in Amsterdam, their report is due Monday February 11th.

FileSender 1.5-rc1 released!

It is with great pleasure I write this blog entry regarding the availability of FileSender 1.5-rc1  (release candidate 1)! For details please read the announcement made by our release manager Xander Jansen to the filesender-dev list about an hour ago.  Please report any issues you find to the FileSender support mailinglist.  As Xander I want to express my thanks to the FileSender team and all those who contributed to this important milestone for our FileSender services.

Xander’s announcement:

“Hi,

We’re happy to announce that the first release candidate for the
upcoming 1.5 release of FileSender is now available for download and in
the FileSender package repositories.

Version 1.5-rc1 has been very thoroughly tested (thanks Wendy!) and
contains a few fixes for bugs found during testing and in the beta4
cycle (thanks Chris for fixing and all reporters for reporting and
sending patches). Also a big thank you to all translators for their
contributions.

We expect that this will be the only release candidate but to verify the
release readiness the rc1 code will be run on at least two ‘real use’
installations for at least two weeks. During this period we also will
finalise the documentation for the final release.

As usual all input on this release candidate is welcome.

When installing or upgrading be sure to read:

http://www.assembla.com/spaces/file_sender/wiki/Installation_notes_for_1-5_development_code

for installation and upgrade notes.

Download details are at:

http://www.assembla.com/spaces/file_sender/wiki/Release_1-5-rc1

The changes since 1.5-beta4 are:

– Update of default Help text (English, Dutch, Norwegian) (#650, #651)
– Updated language definitions
– Fixed links to caniuse.com (all languages)
– Disable Send button after first press (#805, #811, #812)
– Do not mark Guest Voucher as used when upload fails (#807)
– Prevent erroneous showing of DatePicker in recent Chrome versions (#823)
– Debug setting set to false in config-dist.php (#804)
– Only log user information when debug is true (#808)
– Fixes in error logging (#806)
– Remove dependency on php-gmp (#813)
– Fixed include files in emailbouncehandler.php
– Various small changes and bug fixes (#709, #794, #809, #810, #828)

As usual many thanks to all contributors to this release!

On behalf of the FileSender team,

Xander

Status FileSender 1.5-rc1

After a couple of weeks of absence our release manager and code tree guardian Xander is back on his post, welcome back Xander!   He’s started ticking off the remaining RC1 work, for details check our activity stream.  We expect to come with more and probably good news later this week.  If you’re in a hurry and need more details, please drop me an email.

QA testing FileSender 1.5-RC1 started

We started quality assurance testing of FileSender 1.5-Release Candidate 1 this week and target the RC1 release for next week.

The first round of QA tests uncovered some small corner case bugs (thanks Wendy!).  Thanks to a well coordinated team effort those were quickly fixed and verified (check #804-#808 for those interested) and all looks well.  We’re currently conducting a 2nd round of tests and assuming all goes well will be making final preparations for the RC1, consisting of updating documentation.

For those who can’t wait for the RC1 changelog, here are the expected combined changes since 1.5-beta4:

  • Update of default Help text (English, Dutch, Norwegian) (#650, #651)
  • Updated language definitions
  • Fixed links to caniuse.com (all languages)
  • Debug setting set to false in config-dist.php (#804)
  • Only log user information when debug is true (#808)
  • Fixes in error logging (#806)
  • Disable Send button after first press (#805)
  • Do not mark Guest Voucher as used when upload fails (#807)
  • Fixed include files in emailbouncehandler.php