neosam's dev blog

Idea for a file hosting software

One thing I need is a file sync client/server software like OwnCloud or NextCloud but much more simple to host. Also I want a good encryption model. These are the core feature I need:

  • Syncing files between multiple devices while mobile and tables are nice to have.

  • Storing files End-To-End encrypted as long as they are not public. For public shared files, try to do the best to encrypt the files.

  • Allow multiple users on the server who can share their files with each other.

  • Store data only on the filesystem without the need of a database.

Here are my ideas on how to archive this but I'm not sure if this is actually secure since I'm not an expert. At least I wouldn't know how to bypass this.

File encryption

Every user generates a key-pair and the server holds the public key for every user. The private key is stored on the client only. Every file which is stored in the system is stored encrypted and contains its own symmetric encryption key. Another file stores this key encrypted by the public key of every user who can access the file. Only users who have access to the file can decrypt the key to access the file.

User registration

The client locally generates a key pair. The public key and a user name is sent to the server which stores this combination on the server. More is not required for registration. If another computer should get access, the private key must somehow be exchanged between the clients.

User verification

The client gets a session key from the server. In order to get it, it sends the username to the server, the server responds with a message and the client sends the signature of the message back to the server. The server verifies the signature and sends back the session key if it is valid. The communication between the server and the client should always be encrypted. I think of WebDav with a TLS layer on top if.

Public files

Sharing files publicly always has security drawbacks because they cannot be encrypted asymmetrically. But it should be save if these requirements are given:

  • The server doesn't the request URLs

  • The connection between client/browser and server is secured with TLS.

This would allow to pass a key to decrypt the file with the URL and the server sends the decrypted file via TLS to the client. This public-decryption-key is used to access the file-decryption-key from the file the users use to access the files as well. When the server doesn't log the URL, it will forget the decryption key as soon as data transfer is done. Of course this requires a lot of trust but this is the best conclusion which comes into my mind right now.

Another issue is that the file is public as soon as the URL gets public which can be on purpose or by accident. It would be possible to set up an URL which only works once by removing the key once it was used. Once the receiver downloaded the file, it cannot be downloaded anymore. If the download doesn't work, it means the data was already downloaded by someone else.

Or maybe the key for public shares should be separately stored to not share the path to the file.

Directory structure

The directory structure could look like this:

  • ./user-files/USER/... - Encrypted user files are stored here

  • ./shared-files/SHARE-ID-DATA/... - Shared folders are here.

  • ./shared-files/SHARE_ID - File which contains meta data for the share like the name of the share and who can access the share.

  • ./keys/users/USER.pub - Contains the public keys of the users.

  • ./keys/user-files/USER/... - Encrypted key to decrypt the files for each user

  • ./keys/shared-files/SHARE-ID-DATA/... - Encrypted keys to decrypt the files for each user.

  • ./public/SHARE-ID - Encrypted file which contains the path to a file and the decryption key. The key to decrypt the SHARE-ID file is passed via the URL (e.g. https://my.super-cool.share.ru/public/SHARE-ID/DECRYPTION-KEY/filename.txt).

Why no database backend?

For file hosting, I don't think that a database backend is required. Databases require more effort to set up and to backup/restore. We don't need any data aggregation for this use case and so I think storing the files directly on disk is fine.

With the solution I have in mind you can simply use a server somewhere in the internet, install the binary, add a small configuration file which points to the document root and you are good to go. Backups are simply done with rsync.