Novelty Seeking (This Old Pony #79)
We've been looking at why people erroneously think they've reached the limits of their software stack:
Both generally and with regard to Django, in most of the cases where I've seen people who think they've reached the limits of what their tools can do, they're simply wrong, often by significant margins. The reasons, though, vary a lot. They're interesting in and of themselves but identifying them is important unless you have piles of cash to turn and don't mind risking your entire software stack.
This week we're going to be looking at novelty seeking, or, how the pursuit of novelty disrupts existing applications.
But novelty is a good thing!
_ Novelty in and of itself isn't the enemy here. _ We're hard wired to seek some amount of novelty, to try new things. Most of the software developers I know - whether professional or amateur programmers - were initially motivated by experiences of novelty.
The problem isn’t novelty itself, it’s how it's injected into working software projects absent a good alternative outlet.
So let’s not suggest that the problem is having an interest in new technologies, of trying or adopting new architectures, or even replacing components in existing apps.
Novel vs new
For our purposes here I want to draw a distinction between "novel" and "new". Either could refer to something new or something "new to me", but the implicature of "novel" is similar to that of "clever" - eye catching and inspiring because it's new. New is the solution just discovered to a problem you've been tacking. Novel is the solution to a problem you didn't even imagine you had.
Downsides of novelty seeking
It’s that too often, when there’ no outlet for trying novelty elsewhere, or boredom with a work project in some way (e.g. a staid architecture) - usually due to separation between the developer and the consequences of the architecture - there’s a temptation to adopt the new thing they read about on “the orange site”.
Maybe it's Kubernetes or GraphQL or Single Page Applications - some technology novel to the developer and perfectly good in it's own right - with glorious promises (see: MongoDB). But out of a desire to solve an immediate problem and play with a new toy, the developer is wont to misapply their experiment in such a way that compromises the application, the business, or at least vexes subsequent generations of software engineers and architects.
What this looks like is choices appropriate for scale Z applied to systems of scale B (microservice clusters), solutions that spread complexity around (SPAs), or applications built with unsustainable engineering practices.
Solving the novelty conundrum
Solving for this is largely a team problem, a management problem. As a brief answer I'd suggest that developers who are divorced from the end business of what they work on - including the business problem and customers - are more susceptible to novelty traps.
Further, in many cases there are ways of testing out "novel" technologies in production or production-like environments that simultaneously satisfy the individual need for novelty, the organizational need to learn, and the business need to not break things. Build out an internal tool, an admin-facing feature, or even a small customer-facing feature - where the costs both of "living with it" and backing out are relatively small. For more progressive (or funded) teams side projects are the first answer - there's a significant benefit to the organization for software developers writing disposable code (with positive NPV) but not every organization can afford this.
Curiously, the solution to preventing novelty seeking from adding unnecessary costs to production systems turns out to be setting up more opportunities to cheaply explore it.
Originally published 2019-02-26