Like many video content creators, I have a local server running 24/7 dealing with media transfer, sync and backup duties.
My weapons of choice are ownCloud for file sync (think: unlimited dropbox) and CrushFTP for HTTP and FTP. Both applications are cross platform, and both have a great user base always contributing to the forums and helping each other out.
I decided to move the setup over to Linux. I was relatively new to Linux at this point but I’ll admit I’m a bit of a [massive] nerd, so I don’t shy away from any opportunities to tinker, especially in the Terminal!
Anyway, this post is about the single biggest problem I faced when everything was up and running – terrible, I mean terrible write speeds on an external USB hard drive. Writes were well under 1MB/s, which is frustrating if someone is trying to send you a 10gb video, and it’s not the internet speed that’s bottlenecking the transfer but the drive interface.
Many people have reported issues like this in the Linux community, some to do with Samba, some to do with drives being mounted in USB1 speed, etc.
Mine was down to Synchronous mounting of the USB drive. I’m not sure what the super technical description of this is, but essentially your data is safer when mounted synchronously and pulling the drive out or powering it down won’t result in data loss in most cases. It’s the same as the ‘optimize for quick removal’ option in Windows. Drives mounted using sync require more write operations, therefore bogging down the drive and costing performance. Since my drive is in a I/O intensive environment I need all the speed USB2 can give me, and I plan on leaving the server running an not touching it for months (hopefully!)
So to rewind, the checks I did were the following:
1- check network connection:
will show you your network interfaces, where you can find the name of your network adapter. Mine is eth0
will show you the details of the connection. You want to verify that you’re running full duplex at whatever speed your network is set up.
2- check the performance of your media locally
I use df (DiskFree) to find the names of the mounted media and their mount points.
sudo hdparm -Tt /dev/sdb
(my drive was sdb – replace sdb with your drive name you found in the previous step).
This tests the read speed of the device
sudo dd count=1k bs=1M if=/dev/zero of=/dev/sdb
(again, replace sdb with your device)
This tests the write speed of your device. This is where my drive was lagging. A modern 7200rpm USB3 drive should read and write faster than 100MB/s. Over USB2 this will be capped at around 30MB/s
Mine showed around 10MB/s for the write speed. And transferring the file over the network (SMB) sat at about 700KB/s.
3- find the mount options of the device:
this lists all the mount points and their options.
This is where I found ‘sync’ listed in the mount options for my drive.
4- find the process/daemon responsible for mounting the drive:
mine was usbmount – which is an auto-mounter for Ubuntu Server. I remembered that I installed it, and set off looking for it’s config file. One of the beautiful things about Linux is that most things in it are text files! So options for programs will also be inside text files.
After finding this blog: http://sudocode.blogspot.fr/2009/02/automount-usb-disks-in-ubuntu-server.html
I was able to edit the config file, and find the sync option to remove:
sudo nano /etc/usbmount/usbmount.conf
rebooted and it was fixed! 🙂 Now my writes are ~30MB/s what they should be.
I hope this will be useful for somebody!