Running a Master/Slave replication with Redis is common, but has a few things you might not think of from a configuration management standpoint. One of these is synchronization of live configuration changes from master to slave. This article adresses the gap and how to close it.
When Redis replicates, it replicates the data - not the configuration. There is
good reason for this: one of the reasons you might be using Replication is to
provide different persistence options on different systems. So Redis correctly
does not replicate the results of
CONFIG SET commands.
That said, there are cases where you do need certain configuration directives
to be identical across the slaves. An example of this are the
*-values ziplist settings. Another example might be maxmemory, and in some
cases you do want the
save directive settings replicated.
Normally you’d have to communicate to the master, get all slaves, then issue
CONFIG SET command to every instance. I am, however, of the opinion that
this is not suitable for most cases. This is especially true in deployments of
many Redis Pods.
In order to provide a way to have these variables synchronized I’ve written a simple tool which runs on a Sentinel, reads in it’s configuration file, and synchronizes a list of settings. You can find it in it’s GitHub repo at ConfigSync.
Note this will synchronize all slaves to the master for every configuration directive in the list. Currently this is all of the memory-tuning settings and persistence settings. You can, however, use an environment variable to tweak these to a different list, but remember these will be applied to all pods under management on that sentinel.
As I said, it is a simple solution. I will be working on integrating this functionality into Red Skull, and I think Redis/Sentinel should have a way of doing this in the longer term. However, this will work in the meantime.