- Published on
Essentially bounded, Accidentally unlimited
- Authors
- Name
- Damian Płaza
- @raimeyuu
An engineer
Let's pretend that we jumped into the past where software engineering as a job title didn't exist, computer science was more of a fairy tale, full of abstract nonsense.
Imagine that there's a mechanical engineer that wants to calculate some integral which is immensely complicated to approach by means of pure analysis and algebraic thinking.
Countless trial & error moments, many piece of papers and hairs pulled from the head later - he decides that an advancement - a computer program - will be a relief to his tormented, mathematical soul.
Goes to the bookshelf, picks the numerical computation book and picks Runge-Kutta integration approach. Hours pass and he has his solution - validates it against the intuition and seems like he made it!
Later, he meets another engineer nearby the coffee machine, they exchange the experience and seems like another guy shares the problem our hero had before - "software" gets applied again.
How to find a solution?
If we take a naive reflection on how our fellow mechanical engineer walked through seeking the solution:
- Make sure the problem is well-recognized and well-understood.
- Look for a possible solution.
- Apply the solution.
- Verify results.
- If problem is solved then move to the next one else go to the first step.
It looks dull, obvious and straightforward. This is typically how we are working on our daily basis, isn't it?
A software engineer
We don't need to go back into the future to experience what typical, modern software engineering looks like.
Let's imagine there's a software engineer that just grabbed a new task - "calculating an integral".
Couple of StackOverflow pages later, seems like this is a well-known problem in the community - so a bunch of packages were already created. Some of them are actively maintained, other are feeling "complete", but untouched for three years (possibly dead, three years is almost like forever in the modern software engineering, isn't it?).
This is just a single integral, what if in the future there are more integrals? "Ah, extensibility!"
What if, the range, in which integral needs to be calculated, gets wider? "Ah, scalability!"
What if someone else wants to calculate the same integral? "Ah, reusability!"
Few hours pass and finally the solution is there - a draft of a small service that is able to calculate various integrals, with configurable ranges, is ready to be shipped as a serverless function to provide auto-scaling.
How to find a problem?
Intuitively, it seems like it's quite similar to what we saw in How to find a solution?, but is it really the same? Let's examine the steps:
- Look for a possible solution.
- Apply the solution.
- Verify results.
- If problem is solved then move to the next one else go to the first step.
Everything looks correct - seeking the solution, applying it, checking the outcomes, finally reworking the solution. But is it really? Did you notice something missing in the above process?
Oranges vs. apples
One can easily state that I am comparing oranges to apples - and in some sense I'm doing so. The context is the king - "things changed", conditions are not the same, etc. This comparison is a mean to see a contradiction, a paradox, so please bear with me.
A synonym
Out of vast possible synonymous phrases to "software engineering", one is accurately precise - "complexity control". We are taught (either by other individuals or ourself) about the language syntax, patterns, design, rules, etc. - implicitly each of those parts has an associated charge of complexity. Each decision increases or decreases the overall complexity.
But actually - what is the complexity? Seems like there is no single complexity and the famous paper "No Silver Bullet—Essence and Accident in Software Engineering" reveals clear and pragmatic divison between the main two: essential and accidental.
Ying-Yang of Engineering
I assume you can read more about the definitions, but I like to think of them as follows:
Essential complexity defines how complex your problem is.
and
Accidental complexity defines how complex your solution can be.
As with the integral tale, the problem is quite limited - the goal is to get the result for the given integral. Nothing more, nothing less.
There is a single answer to this problem - a number (or an algebraic result depending if we don't operate on numbers).
On the other hand, the solution seems unlimited - is there a single way of solving the problem? Nope.
An observation might be:
There is no single solution, but there is a single problem.
Typically in practicing DDD we ought to say that there are two types of space: problem and solution ones. And, as I understand DDD, "it's all about" driving solution space by the problem space.
An avid reader might draw a connection between types of space and types of complexity:
- a problem space looks like an essential complexity
- a solution space looks like an accidental complexity
If we could plot them, essential complexity (problem space) will be more a less bounded with upper bound:
It might be complex as hell, still according to our observation - provided that you are focusing on one problem at the time - there is maximum complexity involved.
Contrary to it, accidental complexity (solution space) is not bounded at all (unlimited):
A minimum viable complexity is neccessary to get required outcome - THE solution to THE given problem.
However, it can explode quite unimaginably.
When the solution becomes the problem
What happens when we focus on the wrong thing?
As in our integral tale, suddenly we are solving reusability/scalability/extensibility/"any other problem you can imagine"-bility.
When we don't think of "why" and "what" in terms of our problem, suddenly we can turn our solution space into problem space - meaning that we created "accidentally essential complexity" that seems like having an upper bound, although generates more "accidentally essential complexities" that are almost like a wolf in the sheep's skin, tricking us badly.
Not knowing the "why" and the "what" is a curse.
Suddenly, "accidentally essential complexity" gives birth to more "accidentally essential complexity" because each time the solution becomes the problem itself.
THE Domain
Back then, engineers, or "domain experts", were skilled in, wait for it, their domain. Software or computational methods were the means to achieve the solution.
Software engineers are domain experts in, wait for it, software engineering and development. Software is treated as the means to achieve the solution - which is software itself.
One can conclude that we, software developers, are doomed to fall into the trap of changing the focus from the essential problem by an accidental jump into the solution that becomes a new problem - the target itself.
When the transition of focus (from treating real problem as the problem to treating solution as a fake problem instead) changes at least once, it seems that this will lead to recursive focus changes. And, as we all know fellow Problem Solvers, a termination condition is so important in terms of recursive algos/functions.
The XY problem
There is a famous bias, easily observable in, e.g. StackOverflow: XY problem.
Briefly about the meaning:
The XY problem is asking about your attempted solution rather than your actual problem.
Fighting with a solution to a technical problem, suddenly the technical solution becomes the problem.
But don't get me wrong - sometimes "technical" problems are true problems too - but then question - is it an outcome of not solving the "right" problem?
Examplary questions:
- Do you have scalability issues, because you ingest so much data or maybe model is weak/inadequate?
- Do you have maintainability problems, because you need to keep your ultra generic React component updated or maybe two totally separate scenarios were mistakenly understood as similar or identical?
Somehow it feels corresponding to what we've been discussing since the beginning of this post.
A solution to the problem of changing the solution into a problem?
This is the biggest framework I have discovered and learned recently. It has a simple API, but requires a lot of computational power to provide the right answer. I've learned it while undergoing the course called Legacy Fighter (unfortunately only in Polish):
This framework is just a single question.
What problem am I currently solving?
Being honest is the key here.
It really drives the focus into the correct direction. Almost like bringing back the "zero" step (btw. did you notice it was missing in "How to find a problem"?), the prerequisite in the search for a solution.
Suddently, when incorporating this framework, we are going back to the roots of our engineering ancestors and we do mythical (Problem) Domain-Driven (Solution) Design.