Rethinking "missingness"

"Missingness"

Imagine that we need to notify a customer (whatever it actually means). To achieve it, we are loading a notification template, from a place where it can be typically found, so that we can use it as a content for communication:

class CustomerNotificationService
{
    /* declarations of the references to the collaborators from the constructor */
    public CustomerNotificationService(
        INotifier nofitier,
        INotificationTemplate notificationTemplate,
        ICustomerNotificationDetails customerNotificationDetails,
        ILogger<CustomerNotificationService> logger
    )
    {
        _notifier = notifier;
        _notificationTemplate = notificationTemplate;
        _customerNotificationDetails = customerNotificationDetails;
        _logger = logger;
    }

    public async Task NotifyCustomer(Guid customerId)
    {
        var details = await _customerNotificationDetails.GetBy(customerId);
        var template = await _notificationTemplate.GetBy(customerId);
        var notification = template.CreateNotificationWith(details);
        await _notifier.Notify(notification);
        _logger.Information($"Notification sent to {customerId}");
    }
}

Of course, such a code isn't robust at all, since we know that details and template might be null.

And of course, it's pretty easy to improve the method NotifyCustomer by checking against nulls.

We will focus only on the NotifyCustomer so it will only be showed from now.

public async Task NotifyCustomer(Guid customerId)
{
    var details = await _customerNotificationDetails.GetBy(customerId);
    if(details is null)
    {
        _logger.Error($"Notification not sent to {customerId}. Missing details.");
        
        return;
    }
    var template = await _notificationTemplate.GetBy(customerId);
    if(template is null)
    {
        _logger.Error($"Notification not sent to {customerId}. Missing template.");
        
        return;
    }
    var notification = template.CreateNotificationWith(details);
    await _notifier.Notify(notification);
    _logger.Information($"Notification sent to {customerId}");
}

Now, we are safe. The operation we implemented is robust enough, in case of any exceptional situations.

The "missingness" of those pieces sounds like a probable fact to happen in any real-life scenario, so it might be not considered as an "exception", but let's move along.

Nevertheless, it looks pretty normal, isn't it?

What does it mean "it's not there"?

If we stop for a while and think - why are the details missing? Why notification template is not there?

This information might sound like a lower level, and we shouldn't be bothered by it. With high chance, the collaborators, in the form of INotifier and INotificationTemplate, will handle noting why they are missing.

But as a consumer of their services - what do we still see? A null.

We can say that they are missing and do something with that fact.

So for sure, null means we lack customer notification details.

Also, null expresses that we don't have notification template.

So somehow this four-letter beast magically communicates the intention of "missingness".

In various programming languages, it is a first-class citizen that encodes such intention.

Can we do better?

Maybe.

Maybe or null

To be more verbose and compiler-safer, we could introduce a well-known "pattern" - Maybe.

Imagine that we installed one of the packages, providing such "container" (for example CSharpFunctionalExtensions).

To utilize it, we needed to teach upstream collaborators, INotificationTemplate and ICustomerNotificationDetails, to use Maybe<T> in their contracts.

This is how the code looks after doing the changes.

public async Task NotifyCustomer(Guid customerId)
{
    var details = await _customerNotificationDetails.GetBy(customerId);
    if(details.HasNoValue)
    {
        _logger.Error($"Notification not sent to {customerId}. Missing details.");
        
        return;
    }
    var template = await _notificationTemplate.GetBy(customerId);
    if(template.HasNoValue)
    {
        _logger.Error($"Notification not sent to {customerId}. Missing template.");
        
        return;
    }
    var notification = template.Value.CreateNotificationWith(details.Value);
    await _notifier.Notify(notification);
    _logger.Information($"Notification sent to {customerId}");
}

Now, at least, our service providers do not deceive us with their contract of what they give.

That's a nice improvement.

Compiler effectively prevents us from using the underlying details and template, in case of their "missingness" status.

However, we altered the language we use - we now are talking about "value". Code got a bit cluttered with HasNoValue and Value.

Our domain experts do not know what does HasNoValue mean, but at least they get assured by us it's for having something like "safety".

Business, business always changes

We are creating impactful software so our business evolves too.

Experts pointed out that our customers are a bit angry because they weren't notified at the right moment.

To counteract this fact, our experts introduced a new concept - a default template. It will be always present. Its absence means an exceptional situation.

The rule is simple: in case of incomplete customer details, we need to use a default template.

Ha! That's a trivial change.

A boolean flag would do the job.

public async Task NotifyCustomer(Guid customerId)
{
    var details = await _customerNotificationDetails.GetBy(customerId);
    if(details.HasNoValue)
    {
        _logger.Error($"Notification not sent to {customerId}. Missing details.");
        
        return;
    }
    var template = await GetTemplateBy(details);
    if(template.HasNoValue)
    {
        _logger.Information($"Notification not sent to {customerId}. Missing template.");
        return;
    }
    var notification = template.Value.CreateNotificationWith(details.Value);
    await _notifier.Notify(notification);
    _logger.Information($"Notification sent to {customerId}");

    private async Task<Maybe<INotificationTemplate>> GetTemplateBy(
        ICustomerNotificationDetails details
    )
    {
        if(details.Incomplete)
        {
            _logger.Information($"Incomplete customer template details for {customerId}. Using default template.");
            var defaultTemplate = await _notificationTemplate.GetDefault();
            
            return Maybe.Some<INotificationTemplate>(defaultTemplate);
        } else {
            var template = await _notificationTemplate.GetBy(customerId);
            if(template.HasNoValue)
            {
                _logger.Information($"Missing template for {customerId}.");
            }

            return template;
        }
    }
}

Commit. Push. Deploy. Profit.

Business is happy with the value delivered.

We applied our experience to introduce that change!

What about our flag? Incomplete deals with a different kind of "missingness".

We have two ways of modeling "missingness": we model all missing details with Maybe and some missing details with Incomplete flag.

Technically correct, but incorrect, technically.

As the model ought to communicate in the clear way, we could ask ourselves a question - based on the language, what is possible?

What is the language, available in the current context?

Are we really using language of the problem?

Incomplete is pretty verbose, although the type, in compile-time, remains the same - CustomerNotificationDetails.

Maybe informs us about "missingness" - for professionals like us, it makes sense, isn't it?

It still gives the value we are looking for, but language-wise, is Maybe that verbose?

When in doubt, collaborate!

Imagine that we went to our experts and they were clear with their language: details might be available, incomplete or missing.

Clearly, their (mental) model includes such scenarios.

Our model, living in the code, covers them, but in a kind of convoluted fashion.

We can't measure it like we measure time or voltage, but the language, used in the code, isn't completely domain-centric.

What else could we do?

Language-Driven Model

Let's say, we are wearing the disciplined software monk mask and we are obliged to model the language.

How the NotifyCustomer method could look like?

public async Task NotifyCustomer(Guid customerId)
{
    var details = await _customerNotificationDetails.GetBy(customerId);
    details.Match(
        missing: () => {
            _logger.Error($"Notification not sent to {customerId}. Missing details.");    
        },
        incomplete: async (incompleteDetails) => {
            _logger.Information($"Incomplete customer template details for {customerId}. Using default template.");
            var defaultTemplate = await _notificationTemplate.GetDefault();
            var notification = defaultTemplate.CreateNotificationWith(incompleteDetails);
            await _notifier.Notify(notification);
            _logger.Information($"Notification sent to {customerId}");
        },
        available: async (details) => {
            var template = await _notificationTemplate.GetBy(customerId);
            if(template.HasNoValue)
            {
                _logger.Information($"Missing template for {customerId}.");
            } else {
                var notification = template.Value.CreateNotificationWith(details);
                await _notifier.Notify(notification);
                _logger.Information($"Notification sent to {customerId}");
            }
        }
    );

It looks differently, that's for sure. We decided to not use Maybe for modeling "missingness" of customer notification details, but rather model it explicitly.

Also, we captured the language used by our experts. Each of the cases is separately handled, so any time someone from domain personnel mentions "missing notification details", we know where to look at.

Some accidental complexity, related to the technical solution we provided, is inevitable, being hidden behind Match method.

This might look intimidating, but we emphasized the essential matter - how it is communicated between people.

The code now isn't idiomatic

It's highly dependable, but the last iteration of code might be considered as complex/non-idiomatic/smart.

Also, it's pretty known fact that the language we use, limits our cognition capabilities.

This phenomenon applies to any language - spoken or programming.

When the language we are currently using (or thinking with, to be precise), doesn't provide a way of expressively emphasizing the intentions, it might be considered as unnecessary cost (what am I talking about? Check The cost of modeling).

Of course, this doesn't mean that the essential complexity will disappear.

We can try to deceive ourselves by saying that we are complicating "simple things" (why do we consider them as "simple", really?)

Even if the language treats null as its first citizen for modeling "missingness" (various level of it too), this doesn't mean we shouldn't look for other means for expressing it.

Maybe<T> might shortsightedly look as the best and the final solution but this varies as the context changes.

Initially, it maybe won't bring the havoc, but as soon as we don't rethink our way of modeling, things might get implicit and intentions-hiding.

Either you "pay" now or later

I like the saying which states that the complexity doesn't disappear, it gets moved.

In the past, I thought it says only about the place/the space, but this also includes time.

So complexity can move in time too.

One must "pay" the modeling cost and re-evaluate the model across the time.

Everything in this beautiful world is a target of entropy, and so do models.

It might be easier to put a flag on it or use Maybe, but as long as we don't take modeling seriously, we're putting "payment" off in time.

In our little tale, we saw various "techniques" for working with "missingness":

• using null
• using Maybe
• using "discriminated union"-like construct

Of course, depending on the context, they both gave different set of traits and incurred various cost-like qualities, that might be considered as "trade-offs".

Additionally, we could, hypothetically, use enum to express what we achieved with Match, but will we be honest with the problem we are trying to solve?

Modeling levels

One could say we were modeling the solution to the problem on different levels. (levels? I wrote something about similar topic in Modeling Maturity Levels)

Inherently, problems we are facing might seem dull, superficial and simple. This is a matter of time when our solutions will get challenged by the world.

"Shorter" code might look simple and effective and "longer" code might seem as "complicated", but when we use readability as our driver, it might be quite opposite.

There is the value in explicit modeling and we might not be used to that.

In our tiny tale, we explored three alternatives. I don't know what you think, dear Reader, but I like options.

As an engineer, I want to have multiple possibilities at my disposal so that I am able to consciously make a decision. Different languages allow for different styles of expressiveness, that's why it's worth taking the effort of learning a language from a different paradigm.

Unlearning is a bliss.