Here at Heart Internet we love Linux, we love open source, and we love building things.

One of the reasons Heart Internet is so successful is that almost 100% of our platform is built in-house: we use technologies we appreciate and understand, and we’re genuine enthusiasts when it comes to our beloved Linux.

Earlier this year, our Product Ideas team decided that it was time to implement master FTP functionality into our reseller package (and other areas too). It’s a great idea; it allows resellers to easily update some or all of their websites using one master FTP login. It also allows them to hand-down semi-master FTP logins to their staff, so that they can update a limited set of sites.

It’s a very simple idea (the best ideas always are), but some thought was required for it to fit neatly into two very important areas: our security and monitoring model. I gave the implementation some initial thought, but nothing really excited me, and nothing really felt like “the Heart Internet way”. Then it came to me:

FUSE!

What’s FUSE I hear you say? FUSE is a Filesystem in USErspace. Basically, what this means is that, in Linux, you can write a program (I opted for Perl) which will allow me to implement a file and directory structure of my own creation, which works just as if it were local.

FUSE has been used for many things, and forms the basis of many of the ways of accessing non-local-files-as-if-they-were-local fancyness in the modern Fedora and Ubuntu desktops. You can mount a file from the web (httpfs) with it. You can mount Wikipedia (wikipediafs) and edit articles as if they were real files with it. You can even mount your Amazon S3 space locally (s3fs) with it, and play with your data as if it were right there on your computer!

So how did we use it?

We wanted to keep the master FTP idea simple. We didn’t want to force our customers to use separate FTP servers for master FTP. We just wanted a separate master FTP to “just work” on whichever FTP server you use. Fortunately, our FTP servers all run Linux – which meant that FUSE was available to me.

The basic idea behind our master FTP is that we would have a virtual filesystem which (after looking all the details from a MySQL database) presents the files from some/all of a given customer’s websites each in a separate directory. You login using your regular FTP client with your master FTP login and you can see each site’s files right there.

The problems here are that (a) each website runs as a separate user (for security and monitoring reasons) and (b) the website data can be on any number of backend storage devices (for redundancy and fault tolerance).

Enter FUSE: it forms a layer of ownership rewriting (for problem a) and locating the data (problem b).

Obviously, there is a little more complexity to it than that: for example, how do cross-site symlinks work? They remain absolute. Can you accidentally delete a whole site? No! Can you move files between sites? Sure you can. Would it be suitable for moving a site from staging to development? Definitely. Does it allow easy downloads for all your sites in one go? Certainly does – but we already have a tool for that!

It even allows access to both the Linux and Windows packages (if you’re a reseller).

The main fuse code is written using perl which binds to the FUSE kernel module using the FUSE CPAN perl module. Each filesystem call is mapped to a subroutine in the perl code, which performs the necessary magic to present answers to the filesystem subsystem of the operating system. Because our FTP server software just sits on top of the filesystem layer, it doesn’t even know that the master FTP files aren’t real – it just works.

All in all, this allows for a very flexible, secure and elegant way of providing master FTP functionality to our customers without adding any extra complexity to how they use their hosting.

So I hope that this article has given you an insight into how we at Heart Internet think about and develop our technologies.

References:

FUSE – https://en.wikipedia.org/wiki/FUSE_(linux)

httpfs – https://httpfs.sourceforge.net/

wikipediafs – https://wikipediafs.sourceforge.net/

s3fs – https://github.com/tongwang/s3fs-c