The law of eventual composition

"Can you help me?"

One day I was asked by a colleague if I can help him.

Let's say his name was Kyle.

Kyle said that I have an engineering background (I am a control engineer by heart), which made me curious even more.

I followed up with a question - "Ok, but what do you want me to help with?"

"I want to get help with reading how much energy we have produced by solar panels".

That made be a bit suspicious - "You can read, right? So what's the real problem?", I laughed.

"Yeah, yeah, I can but you know - I thought I might use your engineering skills", Kyle replied almost instantly.

Weird.

Still, I love helping people so why not trying to jump into it?

"Ok, listen because I will not repeat"

Equipped with a pen and a piece of paper, we went into his basement to solve his problem - with reading the measurements of how much energy was produced.

There was an electric box with a tiny LCD that enabled viewing some information about controller, as well as measurements.

An electric box related to solar panels

"Well, what is the problem then?", I asked myself.

"Ok, I will read it aloud and you will note it down", Kyle continued.

And then he started.

"Again?"

I prepared and waited to get values.

"35.9, 42.4, 39.7, 12.1, 22.6", he said.

I noted everything with a great precision.

But then I started wondering - "what the hell did I just write down?"

"Job is done, let's go upstairs.", Kyle stated.

"Kyle, wait. I don't know what I put on the paper. You didn't say what are you reading.", I stopped him with this statement.

Oh gee, again.

"I was reading how the consecutive values appear. What's the problem?", he continued.

"What do they mean? I don't know how to label them!", I almost yelled back.

"Ah, I see", Kyle stated.

Then he started reading the next batch but this time putting the label before giving any number.

"T 1.8.1 35.9, T 2.8.2 42.4, T 3.1.3 39.7, G 1.1 12.1, G 2.1 22.6", he said. (those labels and values were different, it does not matter that much)

"All good. What now?", I asked.

"Yeah, now we're going back upstairs", Kyle replied.

"Analysis?"

As we finished reading measurements, we went back and Kyle pulled a notebook, where he kept all previous measurements.

We sat behind the table and Kyle transferred the latest measurements into the notebook.

"Wait, you have previous measurements - so you have been doing what we did today, right?!", I was amazed while I asked so.

Kyle looked at me with a humble face, took a deep breath and started opening.

"Yeah, I collected all of them by myself although I wasn't feeling skilled enough to analyze all the results. I also noted past weather measurements like temperature, humidity and if it was a cloudy day back then."

"So you didn't want me to help you gathering today's measurements but rather analyze the results with you?", I was shocked.

"Pretty much, yes", Kyle smiled innocently.

We took all the data that he collected and we opened Excel to analyze it, together.

What will be rendered?

Let's change a topic.

Imagine that we have a React component that is responsible for presenting a bigger piece of UI - couple of buttons, some block of text and some fields (form) - <AddNewDevice />.

Because we are decent engineers, we sliced and diced this big thingy into a smaller thingies.

Inside of this big component, there's an icon, telling what type of device the user is currently adding:

• a peripheral device,
• a distributed device,
• or a simulated device.

Turns out that the logic is quite complex for determining what type of the device we should use and each time we touch related code, specifications for this entire component break down.

// inside of `<AddNewDevice />` component specs
it("should render a simulated device icon when a test mode is turned on or a user belongs to support group", () => {
    // correct test here.
})

So even though we have a nice piece of React component encapsulating icon presentation, the consumer's specification is affected.

That looks easy fix - let's specify how NewDeviceIcon component should behave instead of making specs for the "main" component be responsible for that! (btw. do you remember, dear Reader, that Simple isn't easy?)

In fact, it's just moving the spec from AddNewDevice.spec.tsx to NewDeviceIcon.spec.tsx.

That was easy.

New feature, new bug

Let's imagine that someone else, from the team, jumped into this component to do some new development.

He was in a bit hurry mode so we didn't collaborate on it, so he worked solo.

Turns out that he took a part of <AddNewDevice /> component and surrounded it with a if statement.

By the sheer coincidence, he removed usage of NewDeviceIcon from the <AddNewDevice /> component.

PR, LGTM, CICD, ship it.

Hello bug my old friend.

Users reported there's no icon when adding a new device.

We are decent engineers, so we started with a test that would verify the hypothesis.

But wait, we have the test already, isn't it?!

Yup, it verifies the behavior of NewDeviceIcon but we decomposed the behavior into many components, along with their specs, without covering "if they integrate well with each other".

We test if NewDeviceIcon works as it should (and it works), so it must be something with <AddNewDevice /> main component.

What can we expect from <AddNewDevice /> that is not covered by <NewDeviceIcon />?

Let's try with a simple thing:

// inside of `<AddNewDevice />` component specs
it("should render any new device icon", () => {
    // here we just check if there's any icon available
})

Of course, the test does not pass. Using <NewDeviceIcon /> in a correct place, inside of <AddNewDevice /> fixes the problem.

The world is saved.

Again.

What is the expected contract?

Another topic switch.

Let's imagine we are working with a team mate on building a new feature - as we have web application written in TS with React and API written in TS using express, we need to coordinate.

We analyze the requirements, define a plan and rush into implementation.

Web app part is taken by our colleague and we take the backend.

Hours pass, commit number grows, increments happen.

Eventually, we are ready to "meet in the middle" and see if everything matches.

First attempt, fail.

FAIL - First Attempt In Learning, right?

Turns out that our team mate assumed that we will return a JSON object (yes, we do REST) having the following structure:

{
    type: "THIS" | "THAT",
    details: string[] // only when "type" is "THAT", if "type" is "THIS" then there should be no details - so empty collection
}

Unfortunately, our API returns null when type is THIS and the whole application breaks down with a crappy, broken UI.

Of course, we fixed it within minutes but we double checked that another endpoint's contract is correctly aligned with both sides understanding.

We saved the world.

Again.

Why all of that?

You might wonder, dear Reader: "what the hell has just happened?"

A mixture of cautionary tales, IT anecdotes and day-to-day struggles - what's the point?

It turns out that all there have something in common.

In the first one, it turned out that segregting the responsibilities so that one party reads and another one writes require them to agree on the contract.

Both need to align and understand what they are promising.

In the second one, even though we made original specifications less brittle, we broke the promise to the end users (by not presenting the icon).

Finally, in the third one, we divided the work among two people which required them to be in the good dialogue.

This commonality is the act of decomposition.

The act of decomposition

It does not matter if the reason behind decomposition is hidden agenda (a need to get help with the analysis), managing the complexity, or trying to speed things up when it comes to software development.

Any time we decompose one thing into smaller ones, one might say we trade one problem to another.

The need for composition

So what is a new problem?

Now we need to compose those things back.

Is it really a problem? We got some other qualities back, isn't it?

As we saw in our tiny tales, we need to ensure that decomposed parts are joining forces to provide the behavior we expect.

This brings other (interesting) challenges, e.g. in terms of satisfying the contract (and notifying about changes).

But wait, what happens when things are "far from each other"? (we explored this topic too, in Modularity Uncertainty Heuristic)

It seems as if we need to compose back the capabilities provided by the decomposed parts - even though they might be "physically" separated, the primary behavior, provided before decomposition, needs to be provided by integrating them back.

As in the scenario with my friend - he was capable of both reading and noting down the measurements.

He decided to distribute those responsibilities across two actors - me and himself - which required controlling the contract to provide exactly the same outcome as he previously delivered by himself alone.

(did he assigned responsibilities correctly?)

The law of eventual composition

What about a well-recognizable architectural pattern which are microservices?

We also decompose "stuff", physically, to achieve certain qualities (hopefully it is done purposefully).

But as we saw, eventually we need to compose things back (or "integrate" if you wish to use enterprise lingo) and that's the place (and time) where dragons live.

Whether we decompose a role into responsibilities or one giant service into smaller ones - beware the eventual composition!

One might say: "yeah, it's all about coupling, what's the point?".

Yeah, that sounds reasonable - coupling.

But wouldn't it mean that whenever we decouple something (in order to achieve something else? See The ambiguity of software architecture) we must follow up with some activity to ensure it provides the same set of functionality as before "decoupling"?

Yeah, it might sound a bit intimidating.

If we push it through "sloganify" process, it might get distilled version:

Obvious obviousness, right?

You can't touch gravity

Even though it might sound anecdotically, like "the iron, when turned on, is hot", it might help us make some decisions more intentionally.

Splitting the work among team members, slicing the process into steps, or making two actors collaboratively solve a task - aren't they leading to similar problems?

Of course, now it means we can't do anything until we carefully analyze decomposition-composition.

Not.

I like to employ this kind of perspective when I am designing something.

The urge to "slice and dice stuff" is strong but I try (and sometimes fail) to Slow down, take some time (and space) to let the boundaries emerge.

What might be helpful?

Outside-In TDD might counterbalance that act - decomposing too early.

Scratchpad-oriented development also might be useful - keeping things together, e.g. in a single file, until they "find their place" (or we feel the pain of cognitive overload).

Don't be afraid to write "ugly things" (isn't ugliness in the eye of the beholder?)

Encapsulation might give us options to reverse our decisions - "what you hide, you can change".

Nothing new, one might say.

Next time, dear Reader, when designing or developing something, try asking yourself a question: