How to fail without fear?

Failing in production is bad. There have been many solutions, but non is guaranteeing a 100% success rate. There is always a chance things will go bad when we deploy to production. What can we do to mitigate this risk? Over the years there have been many solutions, albeit most of them are technical ones. Let's have a look at different solution that we have come up over the years and find out how we can minimize risk when we deploy.

Fix in production

The first thing we did when something fails is to fix it as soon as possible, most of the time this was in a production environment. Although we can bring our service back up real quickly, the uncertainty of the deployment makes us feel weary to actually do it more often than necessary or on any regular basis.

DTAP

In order to tackle all the problems we have encountered in a production fix, we introduce different environments to test every possible situation. I have seen up to seven different environments in one situation; just imagine the cost and the result is that a lot of environments are doing absolutely nothing for most of the time. 

Next thing we see here is that not all the environments are exactly the same. There is always a minor difference in those environments that can bite us in the last step to production. I've seen different size clusters, different machine configurations, different firewalls and even different data connections.

The strength of multiple environments that are dedicated to one phase gets nullified by the increase in maintenance, the extended process and the cost.

Cloud

In order to solve the DTAP-trap we can spin up environments on demand to test. This means we don't need idle DTAP stations. We can actually do with one instance of a cloud platform. Just spin up an environment, run the necessary tests and then kill the instance. We get billed by the time we use the system, the system is identical to production and our process is smaller.

Pipeline

In order to facilitate a solid process of testing and deploying it is mandatory to use an automated pipeline. A pipeline can be seen as all the steps that are needed from development to production within one automated process. When we hook this up to the cloud and we make it as fast as possible. We can almost mimic fixes in production this way. And because we can implement safe guards through the whole process we can be almost sure that our stuff works.

Experimentation

In order to fail without fear we need to move to pipelines and cloud solutions (either on or off premise) and we need to feel relaxed with experimentation. Experimentation early in the process gives us the opportunity to learn new methods, processes and technical solutions. The more experience we get, the less chances there are that anything will fail.

Conclusion

In order to be sure we deploy safely and with the lowest chance of failure we need to focus on cloud-like delivery through automated pipelines. 

Fight technical debt

Many software projects suffer from technical debt and there is not enough priority to solve them. With technical debt comes dependency and versioning problems, security issues and a non sustainable application. The longer technical debt is not treated it becomes exponential harder to solve.

When does technical debt occur?

Technical debt arises when the world around the application changes. For instance when dependencies are upgraded, supporting applications are upgraded, interfaces are renewed or new vulnerabilities are found. 

Another reason for technical debt is shortcutting, this means that we bring out a new feature that a client really needs but we take some shortcuts to make it work. The solution is there for the client, but it hasn't been implemented in a proper technical way. In this case we can reap the benefits of the new feature early, but we need to fix the implementation later on in the game. It gives us a short boost and an advantage over our competitors but it needs rework once released.

How does an application suffer from entropy?

Entropy is the aging of systems and the loss of relevance and compatibility of the application during this process. All systems are suffering from entropy, which is solved by updating those systems on a regular basis. When an application isn't updated while the surrounding infrastructure and application are, the application becomes seriously affected by entropy. Therefor refactoring of the application becomes mandatory.

How can we find technical debt?

Finding technical debt can be hard. Luckily there are applications that can help us spotting technical debt. One of those applications is SonarQube. With SonarQube you can find security flaws when new security insights are found. You can scan for CVE and OWASP vulnerabilities. You can also use this tool to spot deprecated dependencies. 

What can we do to fight technical debt?

Reserving time to refactor the application is important. When someone touches code, they should also solve reported problems in that piece of code. Writing unit tests on all the code we touch helps us in refactoring without breaking the correct working of the code.

Having a clear understanding of the dependencies  we use in the code is important. We need to keep ourselves updated on the versions of the dependencies and we need to do refactoring when a dependency is deprecated. 

By regularly building and testing our code we can get a grip of all the vulnerabilities in de the code. Even when we don't touch the code for a while, we need to regularly test for new vulnerabilities and implement fixes when we spot them. Refactoring and unit testing here is important as always.

Last but not least we need to give ourselves time and priority to do all the above on a regular basis. In other words, setup a build that runs automatically at least every week, preferably every day that inspects against entropy, dependencies and vulnerabilities and act on the results as soon as possible.

Conclusion

By inspecting our code on regular intervals and refactor it, we can keep our application safe and up to date. Keep technical debt as low as possible by acting on a changing world. Implement an automated build pipeline to make life easier and write unit tests so you can easily refactor.

Vision, mission and goals

Your team, department or company is just another fish in the pond. You produce stuff that people buy and that's it. When the world changes you might face a serious problem. Many companies have gone broke and ceased to exist. Out of business, for good. How can this happen? Why do some companies survive while others perish? 

Kodak and Fuji are good examples. The first one produced camera rolls, just like the second one. But when the digital camera disrupted the market, Kodak chose to stick to camera rolls, while Fuji focused on film; all kinds of film, layered plastic film. Kodak could not pivot by lack of vision, they where short sighted and only focused on the analog camera roll. Fuji on the other hand understood that it was making film that could be used in analogue cameras, but also understood that there are many other purposes for film. The first one went out of business while the latter one still produces film.

In this example we see that Fuji had a vision, a broad view of the world and its needs over a long period of time. Kodak did not have a vision for a longer term and they gambled and lost.

Another example. Elon Musk is successful because he has a vision that one day, men will be living among the stars. To make that happen he needs to be able to travel to space, so he makes rockets. But he knows that the planet will probably not survive long enough to reach to the stars due to pollution. So in order to achieve his vision he needs to act on that as well, hence he started Tesla.

Vision

Vision is one of the most important aspects of a company. Small startups have success because their vision is still alive and clear. Big companies that produce a lot of different stuff with a lack of vision might survive, but have a big chance to fail at any market disruption because they are not able to pivot based on their vision.

Developing a vision is imperative to give a company focus. A vision is about what a company wants to achieve in four years or even longer. We should ask ourselves: "What is the purpose of the company?", "Why do we exist?" A vision is about one sentence capturing the essence of the company.

Some examples:

Nike - "Bring inspiration and innovation to every athlete in the world"

Unilever -  "Make sustainable living commonplace" 

Netflix - "Helping content creators around the world to find a global audience."

As we can see from these examples, the companies could pivot, change their products and still fulfill their vision. Nike could go into gyms, food and stadiums. Unilever could go into housing and farmlands. And Netflix isn't bound to the internet. 

Mission

A mission is about how you are going to achieve the vision. Typical questions are: "What will we do?", "How will we do it?", "What do I need to do to reach the stars?"

A clear mission is made for a middle long term, about one to four years. It gives meaning to the vision, makes it less abstract. A mission should be revised as soon as the market is disrupted, while probably the vision will remain the same.

A mission for Nike could be: to start a company that creates composites for new soles or they could produce a movie about the origins of the Olympics.

Goals

Goals are the short term anchors on which to focus in order to achieve the said mission. Goals stretch from three months to about a year. They are clear, market focused and descriptive. Goals are something like a plan for the next quarter or year.

Cascading

This system of vision, mission and goals can be cascaded through the company from C-level to teams. A company will have a vision, mission and goals. Departments will have their own vision, mission and goals to cover the goals of the company. The same goes for teams. They will formulate their own vision, mission and goals based on the goals of the department. The further down the chain, the shorter lived the three elements are.

Just imagine a pivot of a company, this means all department and team elements must be flushed and remade in line with the new company goals.

Battleplan

In order to have your teams, departments and the whole company move in the right direction, you should formulate a goal. Mind you that this is a hard exercise and should not be taken lightly. You are going to be stuck with the vision for a long time, so spent enough time on it to get it right. Get the people involved, sketch, write, bin it and start over. Refactor the outcomes, right until it feels right, gives you meaning and energy. Writing a vision can take days or weeks. It is important to get the feeling right.

Once the vision is done, start with the mission. This is also no more then about a sentence, it describes how we are going to achieve the vision. It is important to stick to the how. Again sketch a lot, throw all away, start over and refactor until it is right.

Based on vision and mission the goals need to be created. There might be many goals, the rule of thumb is that there are about as many goals as there are departments, so every department can fulfill one goal. Do not base the goals on departments though, if there is no match, perhaps the department should be reworked and not the goal. Form the company about the vision, mission and goals, not the other way around.

Refactor

The world is constantly changing, we need to adapt to survive. The only way we can do that is to keep our eyes and ears open and pivot when we need. Pivoting is changing the course of the company to match the market demand. Pivot is what Fuji did and Kodak did not. 

In order to pivot, we need to refactor or rework our mission and goals often. We need to inspect them on a regular basis and we should adapt them when we see a disruption. Managing a company, department or team is hard work, these three mentioned elements should be at the core of that work. 

New world leaders

A new world leaders is someone who owns the vision, mission and goals. That leader can explain why a company is working the way it does. Why stuff needs to be done. A new world leader is always explaining why we are doing the things we do, because of the vision the leader wants to achieve. When you can explain the why as a leader, you build a successful company, department or team.