Code, knowledge and "AI"

Code production

Imagine that two developers, Chris and Paul, are working on a feature together.

They are using recent advancements of "AI" assistants to help them with the task.

They more a less understand what problems they are trying to solve, but they don't have a clear picture of the solution.

So they start prompting "AI" to generate code snippets that they can use.

Couple of tabs later, they have "a working code" (as they referred during daily standup).

Product stakeholders are happy, as they can see progress pretty quickly.

Each time "AI" produces something that doesn't quite fit, they just re-prompt it, and get a new snippet.

Soon, each PR is consisting of growing number of files changed.

Tons and tons of code is getting produced.

Product stakeholders are happy, as they can see progress pretty quickly.

"AI" assistant is very good at exploring existing codebase, hence it reuses Tour and Customer classes, putting there more and more knowledge.

Chris and Paul are also happy as changes take really little time to happen.

Product owner suggested that maybe even he could try to prompt "AI" to produce code himself.

"I know what I want, right?", he said.

Developers were a bit skeptical, but agreed to give it a try, and they offered some guidance, when needed.

Knowledge production

Soon, both developers realize that they don't really understand what is really happening.

What's even worse, they weren't involved in the process of designing, so they lack the context needed to make informed decisions about the code.

They start asking questions to "AI" assistant, trying to understand the code.

Soon they are able to re-gain control, and start making changes again.

"AI" assistant uses what it has been trained on, and produces code that is similar to what it has seen.

Vertical Slices, Clean Architecture, CQRS - they caught all the pokemons.

Yet again, the real struggle happens as they don't really understand the code that got produced.

Of course, they have some notion, but it's shallow.

They see the "patterns", but they don't understand the "why".

They ask another person from the team to review a 121-file pull request, getting "LGTM" as a response.

At some point, even the "AI" assistant starts to struggle, as the codebase is getting more and more complex.

They start visualizing the code structure, sequence diagrams, and other artifacts to get a better understanding.

But it's too much.

Too much of information produced in the process.

Too much knowledge represented in the codebase.

Tour and Customer concepts are getting quite unwieldy, as they are used in many different places.

But they got all "best practices" in place.

Why is it at the same time easier and harder to introduce changes?

Problem?

Many people are excited about the possibilities that "AI" brings to the table.

I am myself, but probably in a different way.

No more programmers needed, as we are one step between "ready product" and "idea" - did you hear or think so, dear Reader?

"Look ma, no hands" - just prompt "AI" to do the job, "vibe code it".

Mark Seeman already pointed out in one of his great articles that typing is not a bottleneck.

But what is "typing"?

In its bare form, it is putting characters on the screen, using a keyboard.

Those characters form "a program", which can be regarded as "code".

One could then conclude that "typing is code production".

This code got produced to solve a particular problem, which should be studied and analyzed beforehand, at least in some extent.

We operate in VUCA world (see more in From concepts to architecture) - building up basic concepts is necessary, if not representing them in the code, at least for efficient communication (!).

So the real problem is understanding a problem.

Taiichi Ohno, the father of Toyota Production System, pointed out that:

Producing code that is not needed, or producing more code than needed, is a waste.

And now it's getting cheaper.

One can easily "generate" more code, just by prompting "AI" assistant.

New models, less tokens, more context - what a bargain!

Overproduction of code

Back then, it was hard and expensive to produce code - IDEs were not advanced, computers had limited resources, internet was not widely available.

People needed to chisel, tinker, and "sweat" to produce code.

We got source code generators, taking away boilerplate code and more.

We got ORMs and other tools, automating away stuff like writing migration scripts by hand - more code got produced.

Now, everything gets easier and easier - and don't get me wrong, dear Reader, I really do admire that it's getting cheaper to materialize ideas in the form of working solutions.

I am benefitting from it myself, each day.

But this easiness comes with a cost - as we tried to discuss it earlier - Easy will make you dizzy?.

Each time we advanced capabilities by using the tool, we delegated some part of the work to it too.

What do we want to delegate?

Software engineering professionals are knowledge workers - our "material" we work on and work with is knowledge, information.

Our sword is understanding.

"AI" machinery is very good at producing code, but can it that easily produce understanding?

When not done consciously, it can be really hazardous with this flashy effect of "getting tons of lines of code produced".

How is this code, that got produced, used?

What is our intention, dear Reader?

Do I want to have a working solution that will go straight to production as I want to "close a ticket"?

Or maybe, I want to understand the problem better?

Hard questions.

Underproduction of understanding

Turns out that it is still "an operator" who must understand what is getting produced.

Understand: what is expected to be produced, what is the reason for production and why it is important.

As "material" is knowledge, and a product is also knowledge, that sometimes is materialized as code, other times as a diagram - "an operator" must build up the understanding.

Not "just code stuff".

And this problem really existed even before "code production" machines were invented.

No amount of efficient code production, low-code/no-code tools, and so forth, is needed when we work on and solve incorrectly understood problems.

Along with "code production" machinery, we are also getting bespoke "understanding production" machinery too - some vendors provide "assistants" that help us doing code reviews, that help us orienting in the maze of all the files changed in the process of development.

How modern is that, dear Reader?

As both Gerald Weinberg and John Gall concluded - "every solution becomes a new problem".

We overproduce code, and it becomes a problem.

So "fast typing" stops to be a bottleneck (was it ever a real problem?) and "fast understanding" of the "produced code" is.

To fight against that problem, we get more tools that enhance our ability to comprehend and manage the code we produce.

But what was a real problem in the first place?

What if tools make fools?

Cheap production upsides?

It's easy to moan and complain - "an old man yells at the cloud" - but is there a benefit of cheal code production?

In hands of an experienced professional, this becomes a tool for rapid ideas verification.

"Code production" machinery amplifies experimenting abilities - shaping and molding solutions to see if it makes sense.

Getting a feedback sooner and adjusting - the heart of agile software development (if you want more on that topic, please visit (Fr)Agile.

So when used pragmatically and consciously, for sure "code production" can increase option value, but it not necessarily needs to happen.

And yet again, it's the operator who steers everything - from concept to cash.

From analysis, through building understanding, to controlling "code production line", and verifying if it actually yields results we want to see.

Who is the biggest bottleneck - a machinery or an operator?

Do we want to remove an operator or use their time and energy, in an economically efficient way, to focus on building understanding and problem-solving?

In Toyota Production System, the core value is put on "utilizing the intelligence of ordinary people".

Think about that.

Code production downsides?

I once wrote a haiku, dedicated to yin-yang of problem solving using software:

Putting to much focus on code production and all the glitter around it might drag us away from the real problem - understanding the problem.

Even though "cheap code" should encourage escaping Inverse Matrix, it might lure into the lair of "you need better models".

We need to be aware that each tool teases and pleases our senses, as if a brave sailor was seduced by singing mermaids - you probably can guess how this typically ended up, dear Reader.

But what is even more concering is that more and more "operators" - software architects, software designers and software engineers - will get disconnected from "the material" they are working on and working with.

"Coding abilities" will decay, reasoning in "code" way will stop be natural, "mechanical sympathy" will become obsolute and oldfashion.

When I say (write?) "coding abilities", I think of representing knowledge and understanding as code - call it modelling, designing.

Clearly expressing what we know at the moment of enchanting it in the form of code.

Some say that "people will be able to reason on higher levels", "it's just another abstraction" - and I can agree on that, however, one must remember that when all works as we expect, you don't need to care.

But when things go in a way that we didn't expect - "you need to look under the hood".

Abstractions are necessary and useful, until you don't need to peel them, inlining covered concepts and elements.

Who will "look under the hood"?

Investing in details?

"You think you are smartass Damian? Then, what should we do?" - one might ask.

I would probably reply: I don't know.

Let's see what is the landscape and analyze the conditions we are operating in.

As we explored the topic of software engineering for busy parents, there are some easy gains one can get.

Maybe we could look at the investment domain and see what people recommend?

Allegedly said Jonathan Sacks (please correct me if I am wrong here).

Then, J. Paul Getty observed in a similar fashion:

Based on those observations, I believe we could forge a similar heuristic, when it comes to IT and software engineering domain:

"Fundamentals? Do you want me to learn design patterns?", you might be wondering, dear Reader.

Well, all the language patterns we use are concepts that we can use to effectively communicate with each other - LLMs can understand them too (there were trained on them, weren't they?).

For me, fundamentals are related to the core of what we are doing - methods, tools and recurring patterns for managing complexity, designing solutions in response to arising needs.

At some point, when dealing with "patterns" and "loading" them to one's head - they will "emerge" (or become visible) when interacting with a problem.

As with "code production", we still need to be doubtful, in a paranoid-esque way, trying to understand if our design choices and understanding we built are correct.

We can't escape that.

And if we get back to "AI" assistants - they can be a great tool for learning, for exploring and for building understanding - as this is the hardest part in our profession.

"Ask" them to explain concepts behind produced code (=knowledge represented).

So next time, dear Reader, when watching how LLMs are (over)producing code for you, think about formed understanding and ask yourself: