Uploading under windows / Linux using rsync

I've recently moved my blog across to dreamhost and was looking for the best way to upload files locally to a remote server. Ideally I only want to upload the changes not the whole site every time.

So this is how I did it.

If your using windows then one way to gain access to rsync which is typically a linux / unix tool is via the use of MSYS2.
Windows 10 does have an option for a bash shell now, but personally I tend to prefer MSYS2.

• https://www.msys2.org/

The first file we're going to create is rsync.rules. This file lists which content should be excluded from the upload to the live site. It's similar to a .gitignore file but for rsync instead.
Note that I'm excluding the .git directory (since the source of the blog is a git repository backed up to a private git server). I'm also excluding the rsync scripts themselves, the attic directories which are just the history of edits for dokuwiki. This also includes anything private I don't want to show up on the live site and anything within the cache which will be auto regenerated anyway.

- /.git/
- /Readme.md
- /php_errors.log
- /web.config
- /rsync.live.bat
- /rsync.rules
- /rsync.sh
- /data/attic/*
- /data/media_attic/*
- /data/cache/*
- /data/pages/private/

Next we're going to create a separate file for our password.
This way we can avoid it being checked into git or source control (if we're doing that)

PasswordHere

Since we're using rsync via a bash shell next I need a shell script to do the actual upload. We're using sshpass to pass in the password for ssh access to rsync.

• userid - should be replaced with the username you've setup for upload
• PasswordHere - should be replaced with the password for the username
• examplesite.com - represents whichever domain your using for your website

#!/bin/bash

# Create directory if it doesn't exist
sshpass -f "rsync.pass" ssh userid@ssh.examplesite.com mkdir -p examplesite.com

# sync from localdir to site
sshpass -f "rsync.pass" rsync -e "/usr/bin/ssh" -av --delete --filter "merge rsync.rules"  . userid@ssh.examplesite.com:examplesite.com

# mark cache as invalid
sshpass -f "rsync.pass" ssh userid@ssh.examplesite.com "cd examplesite.com; touch ./conf/local.php"

# wait for user input to leave window open
read -p "Press any key to continue" x

The first line creates a directory on the server if it doesn't already exist.
Next we do the job of actually uploading the site.
Next we update the modified date / time of the configuration file for dokuwiki which causes the cache to be invalidated.
The last line just waits for a key press to prevent the window from closing so you can see if there have been any errors.

Next because I'm launching this from windows I just need a simple batch file to call msys2 and run the shell script

C:\msys64\msys2_shell.cmd -mingw64 -here -c "./rsync.sh"

At this stage you should now have all three of the rsync scripts within the root of your web directory.
To upload the site it will look something like this.

cd D:\examplesite
rsync.live.bat

You can then log onto your site via ssh to confirm the files have been uploaded.
If your looking for a good ssh terminal for windows I'd suggest

• Putty
• MobaXterm