developer documentation r21357
External storage & Content Delivery Networks
FileMobile provides storage by default on it's own infrastructure, through Amazon S3. If you'd like to use a different infrastructure for media delivery, it is possible to set this up.
How media delivery works
By default filemobile uses 2 domains for media delivery. The first domain gets all the http requests first, logs the relevant information and then redirects the user to the real storage url. This concept is further explained in the chapter about accessing the files.
Currently filemobile uses rstorage.filemobile.com as a redirector, and storage.filemobile.com as the storage domain. it should be noted that it is never recommended to hardcode these urls in your applications, and they will always be subject to change.
Content delivery networks
It is relatively easy to setup most CDNs for Filemobile media delivery. Keep the following information in mind:
- You'll need a dedicated (new) domainname for your media delivery. (Ex: media.example.org).
- You'll need to use storage.filemobile.com as the origin server.
After configuration of your CDN is complete, you can change the Storage domain name in the Project settings menu. After this, all requests going to rstorage.filemobile.com will no longer get redirected to storage.filemobile.com, but to media.example.org.
External storage
If you choose to completely host your media yourself, you have the choice to set this up using the External Storage settings interface. The item can be found under the dashboard navigation. If you cannot find it, it probably means it has been disabled for your account. Contact filemobile support for a resolution.
In this interface you can setup a new storage profile. You have a choice of 2 protocols:
Amazon S3
If you would like to use your own Amazon S3 account instead of Filemobile's, you can do this here. You need to specify an S3 Access key, your secret key and a (precreated) bucket.
The basePath is used to prefix all the file names. While you can specify your own basePath, the only basePath that is actually compatible with Filemobile is '/storage/'. So unless you have a special purpose for the external storage system, be sure to always use /storage/ here.
SFTP
If you choose SFTP, you must also make sure you are able to serve the media using HTTP. So in this case, you'll need at least a webserver with scripting support, as well as an SSH server.
After creation, the following information is requested (all of which is required).
- Hostname - domainname of your SSH server.
- TCP port - usually this is 22, but if your SSH server is running on a different port, change it here.
- Username - The username to your SSH server.
- Password - The password.
- Base path - If the files need to be in a subdirectory, you can enter it here. If you preceed the base path with a /, it will be relative to the root of your SSH server, otherwise it will be relative to the login directory.
- Directory structure - explained below
FTP
If you choose FTP, you must also make sure you are able to serve the media using HTTP. So in this case, you'll need at least a webserver with scripting support, as well as an FTP server.
After creation, the following information is requested (all of which is required).
- Hostname - domainname of your FTP server.
- TCP port - usually this is 21, but if your ftp server is running on a different port, change it here.
- Username - The username to your FTP server.
- Password - The password.
- Base path - If the files need to be in a subdirectory, you can enter it here. If you preceed the base path with a /, it will be relative to the root of your FTP server, otherwise it will be relative to the login directory.
- Directory structure - explained below
The directory structure
If 'flat' is chosen for a directory structure, all files will be thrown in 1 directory. While this is fine for a small amount of files, if you plan to exceed more than a few hundred files it is highly recommmended to choose 'hash4'. The 'hash4' structure ensures that your filesystem stays very fast. The only reason you would not want to use it, is if you use a btree-based filesystem, which is very uncommon as of 2010.
The hash4 uses the id of the file to get an md5 hash. (for example fcea920f7412b5da7be0cf42b8c93759 for file id 1234567). The first 8 characters are then used to generate a 4-level deep directory name. In the example of 1234567, we would store the file in a directory named fc/ea/92/0f. The ensures that no more than 256 subdirectories are created in every directory.
Now, if we assume that 1234567 was a jpeg, the following files will be created:
fc/ea/92/0f/1234567_11.jpg
fc/ea/92/0f/1234567_12.jpg
fc/ea/92/0f/1234567_13.jpg
fc/ea/92/0f/1234567_14.jpg
fc/ea/92/0f/1234567_15.jpg
Serving the files
The problem with serving media files, is that the urls used for accessing the files are completely different from the urls used to store the files. For this reason you are forced to host a script that maps the urls to their respective paths.
The urls expected by the media serving system look like the following:
media.example.org/storage/1234567/11
media.example.org/storage/1234567/12
media.example.org/storage/1234567/13
media.example.org/storage/1234567/14
media.example.org/storage/1234567/15
As you can see, it is also missing the extension, which requires you to actually do a directory scan to find the appropratiate file.
We have provided a PHP script that does all these things, and it's freely downloadable:
Last steps
If you verified your settings are entered correctly, you can change the Default external storage profile setting from Project settings. This will switch the transcoders to send the files to your infrastructure. The last step is to also change the Storage domain name to match your HTTP domainname.
Be adviced that any previously uploaded files to a different storage profile will now no longer be accessible. It is therefore wise to have everything setup before your project starts.