Hooray For Statically Typed Client Libraries
If you’re a programmer you’ve likely heard or read someone expound the benefits and advantages of typed code. Things like compile-time validation that you are passing an integer rather than a string, or avoiding variable type overloading, speed, and many more. However, sometimes this can lead to better code in programs other than the one you’re writing in a statically typed language.
The Background: Redis Info Command Results
When interacting with a Redis instance, you can get quite a bit of information
about it using the info
command, complete with several sections such memory,
server, and stats. Basically Redis returns a big multi-line string with the
information. For example the memory section response looks like this:
# Memory
used_memory:1007440
used_memory_human:983.83K
used_memory_rss:1826816
used_memory_rss_human:1.74M
used_memory_peak:1007440
used_memory_peak_human:983.83K
total_system_memory:8589934592
total_system_memory_human:8.00G
used_memory_lua:37888
used_memory_lua_human:37.00K
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
mem_fragmentation_ratio:1.81
mem_allocator:libc
lazyfree_pending_objects:0
If you run info all
or don’t specify a section you get multiple stanzas like
the above, one for each info section.
Updating Libredis,The Typed Go Client Library
Nearly all client libraries for Redis return this as a big string. If you want to know how much memory is available before Redis starts refusing more additions, you have to parse that out yourself. I’m not a fan of making the client library user do all of the parsing. So when I forked GoRedis and made “libredis”, one of the reasons was to enable the info call to return a typed structure.
Each info section is a Go struct
, and when parsing the entire output these
structs are members of a master struct. Each item, such as “maxmemory” is
parsed into the appropriate type; in the case of maxmemory that being an
int64
. In some cases the values are themselves nested data and need to be
parsed even further.
Now to be certain this meant a fair bit more work for me. I had to code up the parsing and conversion of all existing fields and values in the info command’s various sections. Fortunately Go makes this relatively easy. However, it also means that whenever Salvatore adds new fields I have to add them as well or they go unused. While this has not happened often, and the existing code simply ignores new stuff it doesn’t recognize, it has happened a few times, most recently today.
Today, while playing around with ObjectRocket’s 3.2 release candidate offering,
I noticed several new fields as of 3.2. I knew one of them,
total_system_memory
, was coming. However I hadn’t realized maxmemory was going
to be there, let alone maxmemory_policy
. So off to add them I went. And when I
did, I ran into a bit of a problem. The maxmemory_policy
was listed as unknown.
I thought, “now why is that?”. So I turned to the source code.
I discovered that Redis uses an enum
internally to record the eviction
policy, which is the same as maxmemory_policy
, and uses a function to convert
said enum’s value to the string. If it doesn’t find a string matching the
enum’s value it returns unknown. With a bit more digging I found that it was
actually passing to this function the current maxmemory value, rather than the
policy enum value. Since maxmemory was (much) larger than the enum the value
wasn’t found, “unknown” was returned. Thus, as of this writing, if you run a
stock configured Redis 3.2 (or git unstable) version it will report
‘volatile-lru” as the default maxmemory is zero, and the first item in the enum
is “volatile-lru”. If you set it to any useful maxmemory limit it will be
“unknown”.
With a bit of work I had a pull request ready to go to fix this. For more issue details, and the pull request, you can see Issue 3187. Hopefully this fix will hit 3.2 prior to it being marked stable.
Hurray For Static Typing On The Client
Because I, as a client library author choosing to define and use a static typed structure to parse out the info responses, had to go I an manually add these changes I was forced to confront the response given, find it was incorrect, and fix it. Had I taken the route of simply using one of the other Go Redis libraries, or taken the “let the use parse it” route, I’d not have noticed this change and thus not have fixed it.
Even were I not inclined to dig into the source code reporting it as a bug would still have led Salvatore (or another contributor) to fix it and Redis still would have been improved. Even though Redis is written in a statically typed language, the fact that it merely output strings for this command, regardless of actual data types, provided an “escape route” for that code path. With something on the other side checking it by being statically typed as well, it closed this escape route.
One could make the case this is a (another?) good reason why, at least with out-of-process services, your testing code should validate everything and arguably be in a different language. Perhaps forcing type conversions in your consumers would be a useful core service test addition to many language. Just make sure you aren’t using a shared library which could mean sharing a non-validation bug in the test as well. Yes, it is more work, but isn’t the effort for testing just as important as the effort for developing the code in the first place?