Working together

Once there where two kids next to each other, playing with toy cars in a sandbox. Their parents were in joy because they saw their kids playing together. But where they? The kids played with their own toys in their own patch of sand. They didn't play together, they just played next to each other.

A couple of years later, the kids sat in the sandbox again, but this time they shared their toy cars and build a road in the sandbox. Now they just didn't sit together, they were actually playing together and had a lot of fun.

Most teams work together as the little kids in the story, they don't actually work together, they just happen to be in the same team. They pull their work individually from the worklog. They are just individuals sharing a desk space. Each morning during standup they tell each other what they did, and no one has a clue about what it exactly is they have done or they don't care. 

Some engineers put their name on a couple of items to claim them as future work. When they have one item done, they move on to the next. Others finish their work and pull stuff from the backlog into the sprint because there are no free items to work on. See where this is going? Not everything the team committed to is done but extra stuff is delivered as well. This doesn't fall well with stakeholders.

What can we do about it? First of all no team member can have more than one story on their name. Second, in the daily scrum we talk about the work that is on the scrum board. From top right: the stuff that is almost done, to bottom left: the stuff that still needs to be done. We ask ourselves: "What can we do today to finish this story". Then we ask if anyone wants to help finish it, together. As we progress through the stories in the end we run out of engineers.

So now we have at least two engineers on one story. We need something to enable us to work together. The idea is that everybody participates in fixing the story. We use pair or mob programming to accomplish this.

With pair programming there is a single driver behind a computer and there is one or more navigator who sit close to the driver. The driver is just inputting in the computer, they put in their implementation. The navigator(s) talk about the problem and try finding a solution. Every once in a while the driver switches with a navigator.

When we use this setting we can skip peer review, we are peering the whole time, so there's no use for that. Because more brains have worked out the solution it is of better quality which result in less incidents. And most importantly we shared knowledge and learned from each other. 

Below are a couple of resources to have a look at, the first is Stacy who talks about pair programming, the second is an article about mob programming on medium.

Stacy Cashmore: How to really work together as a team

Sam Fare: I did mob programming every day for 5 months.

The cascading effect of the DevOps way of working

When you are going to work in a DevOps way it all starts with removing the first silo. This means removing the wall between Dev and Ops by putting them together in a team. The consequence is that the whole team is responsible for change and run. When the whole team becomes responsible, no individual can fall back on their role, they are all committed to the cause. Dev can do Ops, test can do Dev, Ops can do test and Dev and so on. This means more collaboration in the team and less push off of work to another role. As a DevOps team you are all in it together.

Working together is only possible if it is done in a small team, and with a small team I mean no more than five engineers (I like to call all team members engineers and not by their role). I state a team of five is the best, a team of three is minimum. With five it will still be a team, with six it will become two teams of three. Six or seven is still possible but communication and therefor collaboration will become increasingly harder. 

If you have smaller teams, you also need smaller software components and with smaller components you need more interfaces. These interfaces are twofold, one is the technical one with APIs, versioning, release cycles and so on. The other is the communication interface. How do you, as a team, communicate with other teams, how are your dependencies organised? You need to think about hard and soft dependencies and come up with a plan. Hard dependencies are those that make your team wait for work from another team or individual. Soft dependencies are those that interface with another team but don't require any work from another team. One of the goals for a DevOps team is to lessen hard dependencies and turn them into soft ones. This can only be done if it is clear who your teams customers are, which is one thing the team should know and investigate.

Working in a small team means a lot of collaboration, you need to share information and to improve sharing it is wise to minimise the work in progress by introducing a WIP limit. A WIP limit is a limit on the amount of work that can be in progress. If you need to learn a lot, make it a low number, so all team members need to work on the same thing together. One of the most errors I see is that every engineer works solely on items of the backlog not sharing knowledge and information. When two engineers work on the same item, they need to communicate and explain what they are doing, teaching the other one about their ways and maybe even learn something themselves.

It can happen that an engineer can't collaborate and that the WIP limit is reached. The purpose of DevOps is not to be as efficient as possible (max. resource utilisation) but to be as effective as possible (max. customer value). This means that when an engineer isn't working on items on the backlog, they should investigate improvements, refactor or learn.

Investigating the way of working and finding new ways to work leads to better processes and automation. When a DevOps team starts automating, many quirks in the process will be found. By solving these the quality of work and the speed of work will increase. 

So what are the steps to take?

1. Form mixed teams with the disciplines needed to convert customer requests in a usable product or service
2. Find out all the dependencies of the team
3. Make communication plans with other teams, technical and social
4. Introduce WIP limit to increase learning
5. Automate

Let's start doing DevOps!

Why we do things in Scrum

We do daily scrums by answering the three questions.
We give a only a demo in the review.
We skip retrospective.
Planning is distributing tasks
But we do Scrum.

That is not Scrum, that is doing what we do in another form. For me the basis are the three pillars: transparency, inspection and adaption. Every event in Scrum is based on these pillars. 

Take the standup for instance, it is transparent because everybody is welcome to attend and the work progress is made visible. It is a moment to inspect the work for that day and come up with a plan to deliver stuff. It is a moment to adapt the way we are organized to get things done. 
I can't withstand those annoying questions: what did you do, what are you going to to and do you have impediments. The result is somewhat in the area of: "I did x yesterday, didn't finish so I continue on x today and I need to visit the dentist". We are not interested in your status or your agenda. At a standup we are inspecting if we get work done. Nothing about the individual, all about the team. So stop with those questions! One approach you can use is to talk about the work items on the board. Go from almost done in the top right corner, to stuff that can be started in the bottom left. Ask yourselves: what do we need today to get the top story in the done state. Don't make it individual work, but group up to finish the work. Find a way to finish it, then move to the next item on the list until you run out of team members. The work on the top is the most important, so it is also very important to finish it first.

Next on the agenda is the review. A review is a key moment to talk to the stakeholders, inspect the work that has been done, agree on the next goals. You can do a demo if it adds to the experience of the stakeholder. But most important is that you talk about what needs to be done and what you need from each other. Offer insights in work, show metrics, ask what the stakeholder needs and measure. Become predictable. Stakeholders don't need status updates or reports, they want to be involved and this is the moment to do that.

Then comes the retrospective. Most teams draw a boat or a racing car and talk about what is slowing them down and what can speed them up. It's a nice exercise, but it doesn't get to the point of the retrospective. Where the review is about what we do, the retrospective is about how we do it. What is our way of working, are we a team, can we trust each other, what can we do to improve flow and focus. In the retrospective you inspect and adapt on the way you work as a team.

And the last event in Scrum is planning. In this event we don't distribute tasks, we inspect our parameters. What capacity do we have, how does that relate to our speed of delivery and how much work can we take on. Is there any unplanned work we need to take in account? What stuff are we not going to do. Can we finish all the work and be predictable? So we inspect the work load, we adapt our forecast and we publish our sprint log to make it transparent.

We've talked about four Scrum events and how they relate to the pillars. Transparency is needed to build trust, inspection is needed to learn and adaption is needed to improve.

Working in an agile team

How does working in an agile team differ from working in a traditional team? Working in an agile team for the first time can be very challenging and you might rethink the way you work.

The first rule of forming an agile team is multidiscipline. An agile team should be able to do all the work, but not everybody in the team has to be able to perform all tasks. The second rule is a clear product vision and purpose; what needs to be build and maintained and why. And the third rule is: don't meddle. Let the team decide and give room for mistakes. Help the team get better and never punish.

Traditional roles are fading in a team. A programmer is learning to test, an operator is learning to code and a tester is learning to develop. All roles become less important and the knowledge is shared. Local heroes are replaced by team players; who support each other in getting the job done. 

One of the most important behaviors of an agile team is working in pairs. This makes sharing of knowledge possible, facilitates reviewing and saves time. While this sounds counter intuitive, results show that lead time and cycle time for changes drop significantly when people pair up. There is also a lower defect rate, less technical debt and more customer satisfaction.

Over time the roles are becoming responsibilities. A multidisciplinary team consists of engineers who are responsible for a part of the development and maintenance of a product. They are responsible for the piece of knowledge they bring to the team. They are expected to keep developing that knowledge and share it with the team and similar interest groups within the company. It is not really important anymore what exactly the job description of that person is. When they belong to the team, they become engineers that want to build excellent products and services for their customers.

An agile team is all about customer value. They engage in customer relations and find the best way to improve customer value. Work does not come from managers anymore but directly from customers. An agile team is not part of any project, it is part of an organisation that builds stuff for customers. In a big enterprise vision and goals (long term) are shared by leaders and teams use that to focus goals on a quarterly basis and in sprints. They organise the work they receive from customers around these goals.

Because an agile team is responsible for the work they do they are multi disciplinary, because they need to serve the customer best they have self autonomy and because they need to know where to go they need vision. 

How to fail without fear?

Failing in production is bad. There have been many solutions, but non is guaranteeing a 100% success rate. There is always a chance things will go bad when we deploy to production. What can we do to mitigate this risk? Over the years there have been many solutions, albeit most of them are technical ones. Let's have a look at different solution that we have come up over the years and find out how we can minimize risk when we deploy.

Fix in production

The first thing we did when something fails is to fix it as soon as possible, most of the time this was in a production environment. Although we can bring our service back up real quickly, the uncertainty of the deployment makes us feel weary to actually do it more often than necessary or on any regular basis.

DTAP

In order to tackle all the problems we have encountered in a production fix, we introduce different environments to test every possible situation. I have seen up to seven different environments in one situation; just imagine the cost and the result is that a lot of environments are doing absolutely nothing for most of the time. 

Next thing we see here is that not all the environments are exactly the same. There is always a minor difference in those environments that can bite us in the last step to production. I've seen different size clusters, different machine configurations, different firewalls and even different data connections.

The strength of multiple environments that are dedicated to one phase gets nullified by the increase in maintenance, the extended process and the cost.

Cloud

In order to solve the DTAP-trap we can spin up environments on demand to test. This means we don't need idle DTAP stations. We can actually do with one instance of a cloud platform. Just spin up an environment, run the necessary tests and then kill the instance. We get billed by the time we use the system, the system is identical to production and our process is smaller.

Pipeline

In order to facilitate a solid process of testing and deploying it is mandatory to use an automated pipeline. A pipeline can be seen as all the steps that are needed from development to production within one automated process. When we hook this up to the cloud and we make it as fast as possible. We can almost mimic fixes in production this way. And because we can implement safe guards through the whole process we can be almost sure that our stuff works.

Experimentation

In order to fail without fear we need to move to pipelines and cloud solutions (either on or off premise) and we need to feel relaxed with experimentation. Experimentation early in the process gives us the opportunity to learn new methods, processes and technical solutions. The more experience we get, the less chances there are that anything will fail.

Conclusion

In order to be sure we deploy safely and with the lowest chance of failure we need to focus on cloud-like delivery through automated pipelines. 

Fight technical debt

Many software projects suffer from technical debt and there is not enough priority to solve them. With technical debt comes dependency and versioning problems, security issues and a non sustainable application. The longer technical debt is not treated it becomes exponential harder to solve.

When does technical debt occur?

Technical debt arises when the world around the application changes. For instance when dependencies are upgraded, supporting applications are upgraded, interfaces are renewed or new vulnerabilities are found. 

Another reason for technical debt is shortcutting, this means that we bring out a new feature that a client really needs but we take some shortcuts to make it work. The solution is there for the client, but it hasn't been implemented in a proper technical way. In this case we can reap the benefits of the new feature early, but we need to fix the implementation later on in the game. It gives us a short boost and an advantage over our competitors but it needs rework once released.

How does an application suffer from entropy?

Entropy is the aging of systems and the loss of relevance and compatibility of the application during this process. All systems are suffering from entropy, which is solved by updating those systems on a regular basis. When an application isn't updated while the surrounding infrastructure and application are, the application becomes seriously affected by entropy. Therefor refactoring of the application becomes mandatory.

How can we find technical debt?

Finding technical debt can be hard. Luckily there are applications that can help us spotting technical debt. One of those applications is SonarQube. With SonarQube you can find security flaws when new security insights are found. You can scan for CVE and OWASP vulnerabilities. You can also use this tool to spot deprecated dependencies. 

What can we do to fight technical debt?

Reserving time to refactor the application is important. When someone touches code, they should also solve reported problems in that piece of code. Writing unit tests on all the code we touch helps us in refactoring without breaking the correct working of the code.

Having a clear understanding of the dependencies  we use in the code is important. We need to keep ourselves updated on the versions of the dependencies and we need to do refactoring when a dependency is deprecated. 

By regularly building and testing our code we can get a grip of all the vulnerabilities in de the code. Even when we don't touch the code for a while, we need to regularly test for new vulnerabilities and implement fixes when we spot them. Refactoring and unit testing here is important as always.

Last but not least we need to give ourselves time and priority to do all the above on a regular basis. In other words, setup a build that runs automatically at least every week, preferably every day that inspects against entropy, dependencies and vulnerabilities and act on the results as soon as possible.

Conclusion

By inspecting our code on regular intervals and refactor it, we can keep our application safe and up to date. Keep technical debt as low as possible by acting on a changing world. Implement an automated build pipeline to make life easier and write unit tests so you can easily refactor.

Vision, mission and goals

Your team, department or company is just another fish in the pond. You produce stuff that people buy and that's it. When the world changes you might face a serious problem. Many companies have gone broke and ceased to exist. Out of business, for good. How can this happen? Why do some companies survive while others perish? 

Kodak and Fuji are good examples. The first one produced camera rolls, just like the second one. But when the digital camera disrupted the market, Kodak chose to stick to camera rolls, while Fuji focused on film; all kinds of film, layered plastic film. Kodak could not pivot by lack of vision, they where short sighted and only focused on the analog camera roll. Fuji on the other hand understood that it was making film that could be used in analogue cameras, but also understood that there are many other purposes for film. The first one went out of business while the latter one still produces film.

In this example we see that Fuji had a vision, a broad view of the world and its needs over a long period of time. Kodak did not have a vision for a longer term and they gambled and lost.

Another example. Elon Musk is successful because he has a vision that one day, men will be living among the stars. To make that happen he needs to be able to travel to space, so he makes rockets. But he knows that the planet will probably not survive long enough to reach to the stars due to pollution. So in order to achieve his vision he needs to act on that as well, hence he started Tesla.

Vision

Vision is one of the most important aspects of a company. Small startups have success because their vision is still alive and clear. Big companies that produce a lot of different stuff with a lack of vision might survive, but have a big chance to fail at any market disruption because they are not able to pivot based on their vision.

Developing a vision is imperative to give a company focus. A vision is about what a company wants to achieve in four years or even longer. We should ask ourselves: "What is the purpose of the company?", "Why do we exist?" A vision is about one sentence capturing the essence of the company.

Some examples:

Nike - "Bring inspiration and innovation to every athlete in the world"

Unilever -  "Make sustainable living commonplace" 

Netflix - "Helping content creators around the world to find a global audience."

As we can see from these examples, the companies could pivot, change their products and still fulfill their vision. Nike could go into gyms, food and stadiums. Unilever could go into housing and farmlands. And Netflix isn't bound to the internet. 

Mission

A mission is about how you are going to achieve the vision. Typical questions are: "What will we do?", "How will we do it?", "What do I need to do to reach the stars?"

A clear mission is made for a middle long term, about one to four years. It gives meaning to the vision, makes it less abstract. A mission should be revised as soon as the market is disrupted, while probably the vision will remain the same.

A mission for Nike could be: to start a company that creates composites for new soles or they could produce a movie about the origins of the Olympics.

Goals

Goals are the short term anchors on which to focus in order to achieve the said mission. Goals stretch from three months to about a year. They are clear, market focused and descriptive. Goals are something like a plan for the next quarter or year.

Cascading

This system of vision, mission and goals can be cascaded through the company from C-level to teams. A company will have a vision, mission and goals. Departments will have their own vision, mission and goals to cover the goals of the company. The same goes for teams. They will formulate their own vision, mission and goals based on the goals of the department. The further down the chain, the shorter lived the three elements are.

Just imagine a pivot of a company, this means all department and team elements must be flushed and remade in line with the new company goals.

Battleplan

In order to have your teams, departments and the whole company move in the right direction, you should formulate a goal. Mind you that this is a hard exercise and should not be taken lightly. You are going to be stuck with the vision for a long time, so spent enough time on it to get it right. Get the people involved, sketch, write, bin it and start over. Refactor the outcomes, right until it feels right, gives you meaning and energy. Writing a vision can take days or weeks. It is important to get the feeling right.

Once the vision is done, start with the mission. This is also no more then about a sentence, it describes how we are going to achieve the vision. It is important to stick to the how. Again sketch a lot, throw all away, start over and refactor until it is right.

Based on vision and mission the goals need to be created. There might be many goals, the rule of thumb is that there are about as many goals as there are departments, so every department can fulfill one goal. Do not base the goals on departments though, if there is no match, perhaps the department should be reworked and not the goal. Form the company about the vision, mission and goals, not the other way around.

Refactor

The world is constantly changing, we need to adapt to survive. The only way we can do that is to keep our eyes and ears open and pivot when we need. Pivoting is changing the course of the company to match the market demand. Pivot is what Fuji did and Kodak did not. 

In order to pivot, we need to refactor or rework our mission and goals often. We need to inspect them on a regular basis and we should adapt them when we see a disruption. Managing a company, department or team is hard work, these three mentioned elements should be at the core of that work. 

New world leaders

A new world leaders is someone who owns the vision, mission and goals. That leader can explain why a company is working the way it does. Why stuff needs to be done. A new world leader is always explaining why we are doing the things we do, because of the vision the leader wants to achieve. When you can explain the why as a leader, you build a successful company, department or team.