The ambiguity of composition

Any experts here?

Imagine that we were challenged to organize a process (what do I mean by "organize"? you might be interested in The ambiguity of software architecture) in which we need to work on some documents, on behalf of our customers.

We have plenty of resources at our disposal but one of the main factors is a workspace in which the experts will do their best to deliver value.

An empty room waiting to be used by experts

They (experts) enter the room through "for staff only" door and get the document that is delivered through "main entrance".

Typically they have all the knowledge to finish their task - most of the results are delivered outside of the room by some experts, through "main exit".

"Handle that document, please!""

Imagine we've invited bunch of experts to deal with coming documents.

There's one coordinator, or "process expert", that knows what are the "coarse-grain" steps to be followed.

The rest of the crew are experts on providing some kind of services.

Experts waiting to get things done

Through the main entrance a document is going to be delivered and process expert will be responsible for guiding what to do.

Others, by their expertise, will know how to deal with a particular stage.

As outcomes of such collaborative work, multiple facts will happen.

For the purpose of this tiny tale, here are the steps that the coordinator need to ensure to happen:

1. Syntactic details verification
2. Formal tone verification
3. Customer importance evaluation
4. Content's legal aspects verification
5. Actual document processing (signing, etc.)
6. Customer notification or customer notification with a delay and processed document archiving

As you can notice, dear Reader, some of the responsibilities might be in hands of the coordinator too (check the colors).

Of course, we are not naive - no one is omnipotent and knows everything - sometimes experts on providing a particular service need to get some help from their colleagues.

They need to refer to them and call them for help.

The important part here is that their mates might be located in different room, different floor or even different building! (if you are more interested in this kind of metaphor, please check "Organization-Driven Design" and Modularity Uncertainty Heuristic)

Service provider getting help from his collaborators

For the coordinator, it doesn't really matter (or it does?) how a given expert delivers a service - job needs to be done.

One could imagine that even a simple task might involve various people, from various places.

"Why the hell did you call me?!"

Imagine a scenario when a document is delivered and the process expert takes (1) step - he verifies syntactic details of the document.

The process coordinator takes the lead and works on the first stage

It turns out that there are some mistakes - a document issuer filled some of the fields incorrectly.

Other experts were invited to the room already but their skills were not able to be used.

The coordinator yelled "we have an exception - syntactic verification failed!".

The rest of the crew just looked at each other - no expected outcome was delivered out through the main exit.

Further processing was rejected.

Initially, everyone felt busy - you know, getting invitation to the "war room" and this kind of things.

The more silly mistakes were present in the document and the more "syntactic verification failures" were communicated by the process expert, the more other experts got angry.

Even if exceptional "syntactic mistakes" were not present anymore, other kinds of scenarios happened - formal tone wasn't meeting the corporate requirements and so on.

Some of the experts were just sitting there and waiting until they're getting something to work on.

Eventually, the outcomes were delivered through the main exit by Delivery expert.

However, most of the times something wasn't right and only yelling could be heared from outside of the experts' room.

"I depend on you, guys!"

The coordinator needed to beg other experts to come to the room to get things done - he promised that their time will be respected.

Fortunately and unfortunately, changes never change - they happen and occur, even if we think everything is and will be stable.

Same happened to our process expert - turned out that he needed to get help from more "dependencies" - as corporate called this kind of relationship.

More experts need more space to collaborate

Process also get a "little" more complex - there were more expected outcomes, each expert was able to break the entire workflow just by shouting "I have an exception X, we need to stop here".

Experts yelling at each other, the room is full of anger and frustration

"I depend on you, guys!" said the coordinator with huge sadness.

Another way of organizing the work?

Imagine that our process expert decided to organize the collaboration in a slightly different manner.

Instead of finding one, big workspace, he thought of many, smaller rooms where each expert might get focused work, when everything is ready for his or her stage.

Smaller room for focused work, by coincidence it is rotated vertically

Another interesting policy that we introduced was that when a document will be pushed through the main exit to another room's main entrance, it needs to be "labeled" in a specific way.

Based on this labeling, him, the coordinator, will be able to decide if further processing is required or not.

Experts' time is worth much of gold and he didn't want to get anyone angry anymore.

After organizing the collaboration in this way, experts still were able to invite some other people to get help but at least they were pretty confident that it's right moment to do so.

Experts waiting to get their turn only when everything is checked

From now, yelling isn't that effective as each expert might be located in a different room.

They started communicating through well-defined contracts - it became very explicit and visual when things got incorrect - the result was immediately discarded by the process expert and others were not even bothered.

Was it easy? Oh no, believe me - they needed to change their habits and learn that instead of yelling they can return results in a slightly different form.

Those results stated if there's an error or if everything is ok.

And for others - when they did some evaluation or scored something, like Expert on customer importance, they were able to state a certain fact. Then, this fact was propagated further and someone was able to interpret it in his or her way.

Eventually, Delivery expert was able to decide what to do, based on the facts that happened throught the process.

What is also important, all the results pushed through the main exit, there were just for reading - no one was able to change them when they received their "copy" through main entrances.

When someone required getting help from his or her mates, the room got slightly larger - but only the experts required at that stage were invited.

Composition?

Some ages ago, I posted a little visualization that emerged in my head. Along with confusion and questions, it got plenty (in the scale of mine) attention within the community.

Visualizing various paradigms

I didn't purposefully write any legend or description, as I was very curious how others could project their mental representations, in a kind of Rorschach test.

Some months passed, I observed more code and codebases, I participated in various interesting discussions.

Somehow it become obvious that the well-known, fundamental attribute of our designs - composition - exists in many forms, even though we refer to it as just "composition".

What is even more interesting - different languages and paradigms encourage or discourage some of its forms.

Horizontal composition

One of the typical way of referring to "external service providers" is calling them dependencies.

It expresses the necessity of having them when the task is on the table (or in the room).

In our little tale, the process expert "I depend on you, guys!" was very explicit that they need to collaborate together.

I believe that there's another name for "people" participating in the joint effort on solving a particular problem - collaborators (I think it was established by invention of the CRC cards?)

Nevertheless, in my world, it totally changes the meaning, the semantics of treating those "experts".

Suddently, we might use different language - we might "invite", "ask for participation" or postpone their appearance.

Still, how does our little metaphor relate to the code?

I believe we all have seen such a scenarios:

public class TheCoordinator
{
    public TheCoordinator(
        IFormalToneVerificationService formalToneVerificationService,
        ICustomerImportanceEvaluationService customerImportanceEvaluationService,
        IContentLegalAspectsVerificationService contentLegalAspectsVerificationService,
        ICustomerNotificationService customerNotificationService,
        IDocumentArchivingService documentArchivingService
    )
    {
        //...details omitted for brevity
    }

    public async Task Process(Document document)
    {
        //...details omitted for brevity
    }
}

(an avid Reader might notice that I included vital and helpful *Service suffix, to make the code looking more familiar)

A class "orchestrating" a given process "invites" many participants to get help in solving the task.

We could think of TheCoordinator's constructor as "for staff only" door.

Then:

• the "main entrance" gets mapped into all that gets into the method
• expected input is all the arguments
• the "main exit" gets mapped into all that gets out from the method
• expected outcome is all the visible "return types"

That's probably pretty obvious.

For now, the most important question is: how many "people" have been invited through the "for staff only" door?

Here we have only 5 experts but I do believe that both me and you, dear Reader, we saw much more "people" invited for collaboration.

As the number grows, "the room" needs to get wider too.

One might say: "yeah but that's the reason why you are hiding some of the dependencies under the abstraction!".

I am all for it. This becomes pretty visible and transparent when working with Outside-In TDD, in my humble opinion.

Especially, when one writes all stubs, fakes and mocks by hand - the pain guides to reduce the number of collaborators, or "dependencies", if you will, dear Reader.

Let's be a little honest - in most scenarios "the room gets horizontally wider", instead of reducing number of "invited" collaborators.

In my observations, this form of composition appears mostly in OO languages.

"Just injecting dependencies" - one might say.

If one is skilled and disciplined enough, only specific roles (roles? you might be interested in I, interface) are getting invited and then, if they need, they refer to their collaborators to get help. (via calling them directly, or through sending them a message - you decide)

And of course, there's another way of composing the efforts.

Vertical composition

What if, just as a pure thought experiment, we would orgnize our code similarly to what the process expert did in Another way of organizing the work??

Then, by focusing on the responsibilities:

public interface IVerifySyntacticCorrectness
{
    public Result<SyntacticallyCorrectDocument, SyntacticMistake> Verify(Document document);
}

public interface IVerifyFormalTone
{
    public Result<DocumentWithFormalTone, FormalToneProblem> Verify(SyntacticallyCorrectDocument document);
}

public interface IEvaluateCustomerImportance
{
    public Task<OneOf<CustomerImportance_1, CustomerImportance_2, CustomerImportance_3>> Verify(DocumentWithFormalTone document);
}

// ...and so on...

(you might wonder why the hell I'm using "interfaces" in such a strange way - stop and read I, interface, then come back)

As it's a thought experiment, we don't want to "invite" all the collaborators to a single, large room - instead we should use try to organize their work in "tiny rooms", distributed sequentially, in a "vertical manner".

var process = new Process(
    new SyntacticCorrectnessVerificationStep(
        new FormalToneVerificationStep(
            new CustomerImportanceEvaluationStep(
                new ContentLegalAspectsVerificationStep(
                    new DocumentProcessingStep(
                        new CustomerNotificationStep(/* other collaborators */),
                        new DocumentArchivizationStep(/* other collaborators */)
                    )
                )
            )
        )
    )
)
// imagine that `document` is of a type `Document`
var finalResult = process.Process(document); // it is of a type `Result<DocumentProcessingResult, DocumentProcessingError>

Of course, one might say: "hey, no one writes the code like that!".

We could use well-known "polymorphism", abstract class with template methods to ensure sequential nature and correct order, probably we would reach the level where we would need to get "chain of command" design pattern.

Nevertheless, a question we're trying to get answer to might be: how we can chain operations to get the desired result?

If we follow this reasoning, turns out all the "steps" form a "pipeline". In an imaginary language it could look as follows:

document
processWith SyntacticCorrectnessVerificationStep
thenProcessWith FormalToneVerificationStep
thenProcessWith CustomerImportanceEvaluationStep
thenProcessWith ContentLegalAspectsVerificationStep
thenProcessWith DocumentProcessingStep
thenProcessInParallel [
    CustomerNotificationStep,
    DocumentArchivizationStep
]

Yes, with high probability you already got me - the sequential nature of document processing didn't disappear and we exchange one syntax to another.

An important aspect to the above, imaginary "pipeline", that I found interesting to observe it's how it affects thinking - writing small, focused and oriented "steps" becomes quite natural.

One focuses on behaviors instead of all the -ers and -Services.

Suddenly, what gets through the "pipeline" isn't just "data" - we might enclose very important facts about the process by focusing on the correct concepts.

In our little tale, we found out that there's a slight difference between SyntacticallyCorrectDocument and Document. Not just in their structure but how they can be used.

Eventually, a Document might become either a SyntacticallyCorrectDocument or a SyntacticMistake (you might be interested in Modeling Maturity Levels).

Horizontal and vertical composition

The sequential character of processing does not disappear, it manifests itself either we use one language or another.

When it comes to collaborators, or "dependencies", some languages encourage inviting them through "for staff only" doors, whereas discouraging other means of working with them.

In .NET space, this "imaginary language" from Vertical composition already exists - it's F# (I hope you know I am biased towards this language, sorry).

So our "vertical composition", when expressed in F#, becomes:

document
|> verifySyntacticCorrectness
|> verifyFormalTone
|> evaluateCustomerImportance
|> verifyContentLegalAspects
|> processDocument
|> finalizeWith(notifyingCustomer, documentArchiving)

(note that this example comes from imaginary land I can do what I want to shape it)

And yes, even if I could be accused of comparing "orange to apple" - I still found it useful to think of the composition in those two forms.

In my humble experience, people, using "OO paradigm" (or at least using languages with class keyword), tend to evaluate "vertical composition" and writing small, well-focused classes as "overengineering".

Hence "horizontal composition" becomes bottleneck as there are so many "collaborators" that eventually things get "underengineered".

(of course, one possible origin might be lack of behavior specification through tests - you might be interested in "The ambiguity" of TDD)

Note I am not saying that OO paradigm is wrong, broken and no one should use it - I am just trying to show various "forms" of composition.

Yet again, the sequential nature of processing won't cease to exist - it's about the way we are organizing it to manage the complexity and "focus on one thing at the single time".

Composition - what are next steps?

As with other aspects of human activities, the ambiguity creeps in when no one is looking.

The King of Kings is always ruling - The Context doesn't leave his throne.

Try to think about "collaborators" and not "dependencies".

It will change your life, believe me.

What's more, all those collaborators play various roles so that the overall goal might get solved.

There's short path from roles, to responsibilities, to functions - when it comes to functions, soon we're reaching the vertical composition.

Next time, dear Reader, ask yourself a question: