Dates are not evil, but the behavior around them often is detrimental to achieving agility.
In most cases, the misperception around dates is that they motivate teams to focus and deliver. In fact, driving a team to a date has the opposite effect and often leads to sub-par solutions, unhappy users, and unhappy teams.
Ask yourself the following questions about dates before reading this post:
- How often do you hit the dates you set?
- Does quality suffer if you struggle to hit a predetermined date?
- Does blaming ever surround missed dates?
- Is employee performance influenced by missing a date?
- Do you learn more by planning or doing? Which is encouraged by the presence of a predetermined delivery date?
- Is software development complex? Uncertain? Custom? Creative?
- When do you know the least in software development? At the beginning? Is that when the delivery date is set?
- Do dates make change easier or harder?
- Is a sustainable pace supported by dates?
- Do dates support continuous improvement?
- Who often sets the date? Is it the Agile team?
- Do dates motivate your teams?
- Is output or outcome more important? Are dates output focused?
This is an initial blog post in a series on dealing with dates, reducing our use of them and the detrimental behavior around them. Having a strategy for breaking date driven behavior is critical to ensure success with Agile.
This series is not advocating that dates never exist. Every software development effort and every organization, large and small, will experience an occasional date constraint. However, we tend to manufacture dates when we do not have a constraint under the false belief in a date as a motivator. This series will describe how to deal with dates when they exist and eliminate date driven behavior with or without the presence of a date constraint.
This first post illustrates why date driven behavior results in waste and inhibits agility, and why you should, therefore, minimize it. Also outlined are general strategies for breaking the date driven behavior Agile anti-pattern.
The Date Driven Behavior Anti-pattern
As described below, the date driven behavior anti-pattern results in significant waste and works against the Agile mindset.
When you focus on dates and output to the detriment of focusing on value and the iteration necessary to achieve it, you exhibit date driven behavior. Date driven behavior manifests in four key ways:
Symptom 1: Expecting certainty in the midst of uncertainty
We believe if we plan better, design better, and think more thoroughly, we can predict when we will be done. This results in hefty up front planning and design in an attempt to outline all activities and their timeline. This often aims to predict the date to the day and time of delivery.
The date forecast occurs at the beginning of the effort before any work has commenced. This is the time when the least information exists and learning has not started. To make it worse, this date becomes the “baseline.” Any movement away from the baseline becomes a failure indicator.
No credence is given to the fact that software is a complex domain with significant uncertainty. Below are common forms of uncertainty that are present in every software development endeavor:
- Scope: There is uncertainty in what we are building. There are no two software endeavors with the same needs for the same users and stakeholders. Every software development effort is custom and is therefore a creative endeavor requiring frequent and short feedback loops to arrive at the right product.
- Social: Every software effort brings together a unique group of people developing the product, using the product, and funding the product. The social interactions between the individuals is unique and must move through stages of forming, storming, norming, and performing1. Human variability comes into play as well. People get sick, have issues at home, and have bad days. All of this creates an environment that is not predictable early in the project or during its execution.
- Technical: There is one certainty in software development—the changing landscape of technology. Even if a fictitious situation existed where a software effort was solving the exact needs of a prior effort with the exact same team and users, the technology in play would likely differ. New architectures, platforms, and methods of designing and developing the software are constantly evolving. Often, teams find themselves working with a new technology for the first time on a software initiative. On its own, this brings a significant amount of uncertainty into the equation when forecasting a completion date.
- Effort: When a team has not solved a custom set of needs for users before, has not worked with the same team members, users, and stakeholders, and is using new technologies, estimates of the necessary effort carry significant amounts of uncertainty. A team can resolve this knowledge gap only by starting the work and learning the effort behind the work in the unique context at play.
With this significantly complex level of uncertainty, spending time to predict a date up front is obviously a wasteful exercise.
Symptom 2: Using dates to motivate behavior
This symptom is present when you hear statements such as:
- “Without a date, the team will not do anything.”
- “An aggressive date will motivate the team to work efficiently and harder.”
- “Stretch goals will drive further slack out of the system and ensure maximum utilization of available capacity.”
- “Hitting a date will give the team a sense of accomplishment.”
- “Putting an aggressive target date in place will create a buzz of activity and minimize social loafing.”
- “The team is padding their estimates or does not know the effort it takes to complete the work. They need a more aggressive date.”
Putting a due date, or a time box, around an uncertain activity plays out in two typical ways:
- Work overflows the time box – Cutting scope occurs when work becomes too large for the time box. Usually quality is cut by cutting corners, resulting in a subpar solution.
- Work is too small for the time box – Work tends to expand to fit a timebox if it is smaller than the time box, resulting in procrastination or over production to fill up the time. As an example, consider a school paper due at the end of the semester and how a student delays starting it until the last possible moment during the semester. Consider how when given spare time, a software developer might add features to software to anticipate some future request.
Given, these two common scenarios resulting from time boxing, coupled with the uncertainty of software development work, motivation is not an obvious outcome. Rather than motivation, frustration is often the result. Teams become frustrated when inadequate time exists to deliver a quality solution or when their estimates do not matter. Users and stakeholders become frustrated when a team cannot deliver the solution by a promised date, the team delivers a buggy or incomplete solution, or a team spends too much time over producing unneeded items.
Motivation does not come from dates. Rather, it comes through Agile leadership crafting a shared vision, helping the team understand the “why” behind the work, and providing adequate support for removing obstacles the team faces in achieving the vision. Motivation comes through developing a real connection to user needs, delivering something that enriches the workflow of a user, making an impact on the business, and rapidly delivering a simple solution to a problem. Refer to these posts for further reading on serving and supporting the team:
- How to Shift Gears to Become an Agile Leader
- Agile Leader Pattern 5 for Building Awesome Teams: Enable Self-Organization
- Agile Leader Pattern 6 for Building Awesome Teams: Serve the Team
Symptom 3: Setting dates without connection to reality
This symptom should raise significant concern. It is also one of the most common reasons for the existence of a date constraint. Figure A illustrates this symptom.
Teams provide the best source of information about the time it will take to deliver a solution. However, in this symptom, team consultation on the estimate and their historical performance do not come into consideration before promising a date. No thinking about detailed needs or options for solving those needs comes into play before setting the date. This is a recipe for disaster.
This scenario plays out often due to bravado, an opportunistic public relations move, pressure to show competency, or pressure to beat the competition. In fact, given the uncertainty of software development and the destructive behavior resulting from dates, the end result will not inspire bragging rights and a public relations campaign will likely need to launch to save face for the missed date.
Symptom 4: Relying on a plan versus the changing conditions of execution
There is comfort in a plan even when nothing is going as planned. This symptom is present when you find yourself blind to the reality on the ground and holding onto the plan as the truth.
This is often present when yearly performance goals are set with executives. The goals have dates. The executives drive their management direct reports to the dates. The managers drive the teams to the dates. Reality on the ground changes but no adjustment can be made to the goals as they were set in stone at the beginning of the year. Missing the date now has a financial impact to individuals. This is a strong incentive to chase the date; value is not the focus.
However, this symptom can also manifest from the pure existence of a detailed plan. Many find comfort in the detail and the guidance of the plan. After all, recipes are easier to follow. All behaviors are about sticking to the plan regardless of the changing reality.
The analogy here is blindly following a map without adjusting for the reality of the terrain on a hike or staring only at your GPS navigation device and not looking out of your windshield or in your mirrors when driving.
Software development requires a sense, orient, and respond approach to deal with the complex realities on the ground. A plan and a date is useless, but continuous adjustment based on realities on the ground is paramount.
Agile and Lean Violations
Date driven behavior violates all Agile values and principles and results in all of the Lean wastes as described below. This results in a diminished Agile mindset.
Agile Values in Violation:
Date driven behavior violates every Agile value in the Agile Manifesto2 as described below.
Individuals and interactions over process and tools
Setting a date without team input moves away from individuals working together and tends to create an “us” versus “them” mentality between the date setter and the team.
Creating a detailed plan and a change management process for the inevitable variance from it results in heavy process and a tendency to ignore the realities of the individuals and interactions on the ground.
Responding to change over following a plan
When we follow a plan blindly as the “baseline,” change is discouraged. Due to the uncertainty of software development, change is certain and frequent. Plans work against responding to change quickly and favor behavior that sticks to the plan.
Even worse, when reality changes during delivery, additional big planning occurs to incoroporate the change into the rigid baseline.
To top it off, when things do not go as planned, the response to change usually is not sustainable as the project is likely behind schedule.
Customer collaboration over contract negotiation
Dates stand in the way of delivering solutions that meet customer needs.
A date and its resulting plan is usually based on a fixed scope. This fixed scope is a contract. Deadlines are set, scope is defined, and cost is estimated. This contract stands in the way of customer collaboration. All of this slows down or stalls the necessary evolution of the solution. Delivering against the contract takes precedence over building the right product for addressing user workflow needs.
Delivering working software over comprehensive documentation
The detailed plans, requirements documentation, and design documentation created to predict delivery by a specific date only work to delay the delivery of working software. Change documentation is also quite comprehensive and adds no value to the user.
Working software is the only way we can learn and adjust. We learn more by doing than thinking or documenting.
Upfront planning documentation is the illusion of certainty in the midst of uncertainty.
Agile Principles in Violation
Date driven behavior puts direct strain on all of the Agile Principles from the Agile Manifesto1.
Frequent, Valuable Working Software
Our highest priority is to satisfy the customer
through early and continuous delivery
of valuable software.
Deliver working software frequently, from a
couple of weeks to a couple of months, with a
preference to the shorter timescale.
Working software is the primary measure of progress.
Dates focus on output rather than value. They put pressure on us to deliver something even if it is not the right thing to deliver. The presence of a date forces us to over-plan, over-design, and be averse to trying a solution for fear of missing the date if wrong. Dates and plans have no value.
Working software has significant learning value. Only through frequent delivery of working software can we sense if what we have delivered meets user needs, orient to any necessary adjustments, and respond by iterating on the solution until we get it right. Dates work counter to the necessary iteration for satisfying the customer.
Teamwork, Trust, and Collaboration
Business people and developers must work together daily throughout the project.
The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
When a stakeholder sets a date for the team, teamwork, trust, and collaboration erodes. When stakeholders set a date, the team has to deal with the resulting daily chaos of trying to meet the date in the midst of uncertainty. Very little engagement from the business occurs during the delivery until the date inevitably falls into jeopardy. Then, the involvement typically is in a tense, unsatisfying manner. Mistrust is the resulting behavior on both sides.
In most cases, the date setting occurs without even consulting the team. The person may not even know the team. Face-to-face engagement in this process is completely absent.
Motivated, Trusted Teams Working In the Environment They Need
Build projects around motivated individuals.
Give them the environment and support they need,
and trust them to get the job done.
If a team operates in an date driven environment, they do not have the optimal environment needed to get the job done. They don’t have control to complete the work within the time they need. There is no trust in the team to deliver on time without a date. The team has no motivation around a common purpose and vision. All they are given is a date to motivate them—one on which they have had no input.
This frustrates the team and slows them down. Consequently, dates severely limit a team’s potential productivity due to the unsustainable pace that often results.
Maximizing the Amount of Work Not Done
Simplicity—the art of maximizing the amount
of work not done—is essential.
Driving to a date can result in defects due to cutting corners or spending more time planning than delivering working software. Defects add work and complexity.
As mentioned above, if a date exists that provides too much time to deliver a solution, the team has a tendency to overproduce elements the user does not need now.
The additional big upfront planning and design and resulting change management activities precipitated by dates create extra work that adds no value to the solution or the end user.
Sustainable Pace Indefinitely
Agile processes promote sustainable development.
The sponsors, developers, and users should be able
to maintain a constant pace indefinitely.
Amidst the uncertainty of software development, things never go as planned. When a date becomes the priority, the team will cut corners or put in overtime to try to hit the date. Cutting corners leads to downstream problems when defects occur or by taking on technical debt and not paying it back in a timely manner. Overtime hurts a team’s ability to maintain a sustainable pace and reduces their happiness. A team’s happiness is a leading indicator of their future performance. The happiness and sustainable pace of everyone is impacted—stakeholders, developers, and users.
The best architectures, requirements and designs emerge from self-organizing teams.
Continuous attention to technical excellence and good design enhances agility.
When a team cuts corners to meet a date or when no slack is available to experiment, learn, and evolve the solution, agility diminishes. The system becomes fragile to change or cannot support the final solution.
Consider if the team cuts corners by foregoing the creation of automated tests for manual testing. This may be acceptable for a few sprints, but eventually the team will hit a wall trying to manually regression test for every source code change. The solution to this will likely reduce testing cycles to decrease the manual testing overhead. Undetected defects will pile up and breed more defects, and defects will escape due to manual testing error.
Welcome changing requirements, even late in
development. Agile processes harness change for
the customer’s competitive advantage.
At regular intervals, the team reflects on how to become more effective, then tunes and adjust its behavior accordingly.
Most date driven efforts have a method of dealing with changes—a scope change review process that ingests a change request, evaluates the priority, assesses the impact, and decides to include or reject the change. While this time consuming and costly process might allow change, it can hardly be considered “welcoming,” and to the customer, it will not appear to harness change to their advantage.
Additionally, the team has no time to reflect and improve the way they work to become more effective. Improvement measures are considered to be a cost to the timeline rather than an investment for the future in the presence of date driven behavior.
Lean Wastes Observed With Dependencies
To round out the costly nature of dependency coordination, you will experience all of the seven Software Development Wastes3 when coordinating and managing dependencies.
Partially Done Work
Partially done work hides defects and delays feedback.
When corners are cut to meet a date, the work will not be done. The date may be met but the product will likely not be delivered “right” and the “right” product will likely not be delivered. You will not attain valuable, working software until completing all of the work to meet user needs with all discovered defects fixed.
Extra Features / Over-production
When the work is smaller than a time box, many will create extra software that is not needed now to fill the time box. Future-proofing a piece of code to anticipate future requests is tempting and is often performed when more time is present than is needed.
Date driven behavior often results in extra planning, documentation, change control boards, and other processes to avoid missing the date. These are processes that you do not otherwise need to deliver the right solution to meet user needs.
When someone other than the development team sets a date for the team without the team’s involvement, there is a handoff to the team. This drives communication issues, relearning, and delays due to the lack of collaboration and shared understanding.
Delays / Waiting
Things never go as planned—work takes longer than estimated, communication and shared understanding fails, other priorities emerge, or people simply go on vacation or get sick. Deadlines slip. This translates to delays, waiting, and unhappiness.
When the technical excellence is compromised due to cutting corners to meet a date, unplanned work, such as defects, will increase. This unplanned work interrupts your flow and results in costly context switching.
Due to cutting corners or rushing to meet a date, defects are guaranteed. Defects are a waste that adds significant complexity and delays to the effort.
Corrective Patterns for Eliminating Date Driven Behavior :
The case is clear that date driven behavior is wasteful, anti-Agile, anti-Lean, and worse in the long run. Given this, eliminating date driven behavior is a priority.
The additional posts in the series will focus on the following patterns for removing date driven behavior:
- Learning more by doing than planning
- Achieving “done” not dates
- Forecasting delivery dates amidst uncertainty
- Having a value focus
- Dealing with a legitimate date constraint
Delivery dates are not bad but the behavior around them often is detrimental to delivering value. Date driven behavior violates all Agile values and principles and results in all Lean wastes. Given date driven behavior is so dangerous to the Agile mindset, minimizing or removing it as part of your Agile transformation it is paramount to your success.
Stay tuned for future posts in the series on how to break the date driven behavior anti-pattern.
- Developmental Sequence in Small Groups, Psychological Bulletin 63, B W Tuckman (1965)
- Agile Manifesto
- Implementing Lean Software Development – From Concept to Cash, Mary and Tom Poppendieck – Sept 7, 2006