What is the root cause of problems in software engineering?
Assumptions cause most of the problems in software engineering (and in life, in general). Assumptions cause problems big and small. They cause problems with the technology and the people.
Assumption: people know what they want from IT support and understand the consequences
In my first job, I worked as an IT technician. One day, a person came with his laptop and told me to format the hard drive and reinstall the OS because the computer was getting slow. I asked if they had made a copy of the data because I would have to remove everything. The client agreed.
After returning the laptop to the client, he asked, “Where is the database?” “What? What database? You told me I can remove everything.” He never mentioned the database before!
Eventually, I managed to restore the database transaction log file. The transaction log allowed me to get the database content back.
It was a stressful day.
Assumption: people read instructions
Once, I assumed people read instructions, and I was wrong.
We had a failing ETL process which we had to replace with a new solution because the API we used got discontinued. We didn’t get a notification about the API shutdown. How come? I still don’t know. Perhaps the previous members of the team set the notifications to their addresses. But they all quit, so we didn’t know about the upcoming problem.
I created the new ETL to retrieve the data from the new API. I decided to run the ETL in a development environment because we still needed code review and testing. What’s the catch? Our production pipeline didn’t work, so I had to start using the new one right away. I was running the ETL in the development environment, but it was already processing production data.
I added a warning sentence in bold, red, and large font with lots of warning sign emoticons in the testing instructions. I warned the reviewer about the consequences of removing the data from the development environment. It was production data, do you remember?
Do you know what was the first thing the developer who tested the ETL did? He deleted all the data.
I could blame the developer. He was supposed to know about it. After all, we talked about the temporary solution in the meetings, and I wrote the warning message.
I could blame his lack of experience. But it isn’t fair. Usually, the inexperienced people follow the instructions carefully. He was an exception to the rule.
I could blame his cockiness and short attention span. It’s possible he read the instructions but forgot about the first sentence when he was reading the second paragraph.
However, the real problem was: that I was wrong. I assumed people would follow the instructions.
People don’t read (or don’t understand) instructions. Also, I should have setup a
prod-substitute environment. Perhaps, people would treat the tested pipeline as production code even during testing.
Unfortunately, we lost a few days of data.
What am I assuming?
Do you know what’s the worst? We can’t do anything about assumptions. We can’t live without them. We would starve, unable to decide what to eat for breakfast. We need a shortcut for our thoughts.
Yet, maybe we should be tactical about assumptions. In some cases, we should ask ourselves, “What am I assuming right now?” Especially when we can make an irreparable mistake.
What if we assume we don’t need to think about our assumptions? See? It never ends. Assumptions are a curse.
I couldn’t find reliable information about the original author of the words: “Assumptions are the mothers of all fuckups,” but the saying is true. It’s true in IT and every other part of life.
Assumptions kill people.
Assumptions are the cause of car accidents. Of course, nobody says, “I assumed nothing would happen when I drive at a highway speed in a city while overtaking cars at crossroads.” Why does nobody say it? They don’t say it only because those people are dead, not because it never happens.
People eat poor quality food in way too large quantities, sit all day, drink alcohol, and smoke, assuming that they will be the first person in the history of the world who won’t get a heart attack, stroke, cancer, diabetes, or any other serious medical condition because of their unhygienic lifestyle.
Wait. Some readers may misunderstand what I wrote. I didn’t say “hygienic lifestyle guarantees you good health.” It doesn’t. But an unhygienic lifestyle guarantees you health problems eventually.
The odds are against us.
Be wary of what you assume.
Did you enjoy reading this article?
Would you like to learn more about software craft in data engineering and MLOps?
Subscribe to the newsletter or add this blog to your RSS reader (does anyone still use them?) to get a notification when I publish a new essay!
You may also like
- Data/MLOps engineer by day
- DevRel/copywriter by night
- Python and data engineering trainer
- Conference speaker
- Contributed a chapter to the book "97 Things Every Data Engineer Should Know"
- Twitter: @mikulskibartosz