Watching Nextcloud with inotify
2021-07-25
Sometimes, I want to dump a file into Nextcloud and have it automatically appear on a website. Since Nextcloud is hosted on my NAS at home, while the web server runs on a VPS in a datacenter, the files will need to be copied over.
How could this be done?
External Storage
Nextcloud already supports external storage, meaning it can connect to SFTP and other servers. In theory, I could connect Nextcloud to the web server via SFTP and be done with it.
Unfortunately, this feature is extremely unreliable. The UI is buggy, rendering it difficult to add new servers. Additionally, when the connection is interrupted for whatever reason, the entire Nextcloud stops syncing with clients and shows cryptic error messages.
WebDAV & cron
Since Nextcloud supports (and utilizes) WebDAV, it is easy to connect a WebDAV client. I like using rclone
for this: It supports a multitude of protocols and is very easy to use. The command rclone sync
implements a one-way sync, which is what I need here. It can be automated by a cron job:
*/10 * * * * rclone sync nextcloud:public_files/ ~/web/public_files
This approach works, and it's reasonably reliable (I have been using it for some time). However, it has also got multiple disadvantages: Obviously it can take up to ten minutes for changes to be reflected on the web — not a big deal, but annoying. Also, it seems wasteful to sync that often regardless of whether anything has changed.
Inotify
The idea behind inotify
is that the kernel itself watches a file or directory and sends a notification when something happens. This is much more efficient than polling on regular intervals.
Inotify looks like exactly what I need. A daemon on my the Nextcloud server could initiate the synchronization whenever necessary.
Now, the two machines just need to connect. The simplest and most straight-forward approach seems to be SFTP. I can use rclone sync
again, but on the source side this time. After adding the server with rclone config
, inotify
and rclone
can be combined in a simple shell script:
#!/bin/sh source="/path/to/nextcloud/data/user/files/public_files" target="server:web/public_files" inotifywait -mr "$source" --event modify,move,delete,create,attrib --exclude "\.ocTransfer.*\.part" \ | \ while read r do rclone sync "$source" "$target" done
Every time inotify
detects a change in $source
, the while loop is executed, synchronizing the entire directory. Improvements are still possible, but it does the job.
Update (2024-04-12)
After functioning reliably for years, my script suddenly stopped working. It would still sync when a file was modified, but couldn't detect new files anymore. Some investigating revealed that uploading a new file to Nextcloud does not trigger a CREATE
event, but ATTRIB
as well as several ACCESS
and CLOSE
events instead.
I modified the script to include ATTRIB
in the list of events it watches for, which made it work properly again.