Agile Software Development: How Software Actually Gets Built

Agile software development means building software in small increments that work, rather than building it in stages that only come together at the end. The process changes are the visible part. The engineering changes are what make it possible: small batches, integrating continuously, testing as you go, and refactoring rather than accumulating mess.

Agile software development is guided by the Agile Manifesto (2001), which values working software, collaboration and responding to change, and is practised through frameworks such as Scrum, Kanban and Extreme Programming (XP).

Key Highlights

• Agile is not a process wrapped around unchanged engineering. Teams that adopt the events without the practices get slower, not faster.
• Small batch size is the underlying mechanism. Nearly every agile engineering practice exists to make small batches viable.
• If integrating work is painful, the team will do it less often, which makes it more painful. Continuous integration exists to break that loop.
• Testing at the end of an iteration recreates the phase gate that iterations are meant to remove.
• Refactoring is not cleanup deferred to a quiet week. It is how the codebase stays changeable enough to keep iterating.
• The most reliable sign of genuine agility is how long it takes to safely ship a one line change.

Agile Software Development Basics

Agile Manifesto Values and 12 Principles

The Agile Manifesto was written in February 2001 by 17 software practitioners at Snowbird, Utah. It states four values:

1. Individuals and interactions over processes and tools
2. Working software over comprehensive documentation
3. Customer collaboration over contract negotiation
4. Responding to change over following a plan

The items on the right still matter; the items on the left matter more. The 12 principles behind the Manifesto, in summary:

1. Satisfy the customer through early and continuous delivery of valuable software.
2. Welcome changing requirements, even late in development.
3. Deliver working software frequently, from a couple of weeks to a couple of months.
4. Business people and developers work together daily.
5. Build projects around motivated individuals and trust them.
6. Prefer face-to-face conversation for conveying information.
7. Working software is the primary measure of progress.
8. Maintain a sustainable pace indefinitely.
9. Pay continuous attention to technical excellence and good design.
10. Simplicity, maximising the amount of work not done, is essential.
11. The best designs emerge from self-organising teams.
12. Reflect regularly and adjust behaviour to become more effective.

Read the full original text at agilemanifesto.org.

Agile Frameworks Compared

Agile vs Waterfall

Scrum Roles, Events and Artifacts

Based on the Scrum Guide (2020). The Definition of Done is discussed further in the Automated Testing in Agile Development section below.

Benefits of Agile

• Faster feedback from real users, every few weeks.
• Earlier risk discovery through small batches and continuous integration.
• Usable value delivered before the whole project finishes.
• Easier response to changing priorities.
• Higher team ownership and collaboration.

Challenges of Agile

• Adopting the events without the engineering practices (performative Agile).
• Up-front investment in automated testing and CI pipelines.
• Fixed-scope, fixed-date contracts that conflict with iterative delivery.
• Coordinating many teams at scale.
• Product Owner availability and backlog quality.

How Agile Software Development Works

The visible change is the calendar. Work arrives in Sprints rather than phases, and there are events at defined points.

The change that matters is the batch size. Traditional development moves large batches through sequential stages: months of requirements, then months of building, then a testing phase, then release. Agile development moves small batches through all of those activities repeatedly, so requirements, building, testing and release happen every couple of weeks on a fraction of the scope.

That single shift is what makes everything else necessary. A team releasing every two weeks cannot afford a two week manual regression test, so it needs automation. It cannot afford a three day integration effort, so it needs to integrate constantly. It cannot let the code degrade for a year and then clean it up, because there is no gap in which to do that.

This is why adopting the process without the engineering practices tends to make things worse. The team now has all the coordination overhead of iterating and none of the capability that makes iterating cheap, so each Sprint carries the same integration and testing costs the old process amortised across a year.

If you are working through the framework side of this, our free CSM practice test covers it quickly.

Small Batches: Delivering in Increments

Worth understanding properly, because it explains most of the rest.

A large batch of work has three problems. Feedback arrives late, so mistakes compound before anyone notices. Risk concentrates, so a single integration at the end is where everything goes wrong at once. And nothing is usable until all of it is finished, so a project that is ninety percent complete delivers nothing.

Small batches invert all three. Feedback arrives within days. Risk is spread across many small integrations, each cheap to diagnose. And each batch produces something usable, so stopping early still leaves value behind.

The practical implication for a team is that splitting work is not an administrative chore to fit items into a Sprint. It is the core discipline. A team that consistently produces items finishable in a few days will outperform a team producing items that take most of a Sprint, even if the total volume is identical, because the smaller items surface problems earlier and carry less risk.

The common objection is that some things genuinely cannot be split. That is true occasionally and far less often than teams assume. Most large items contain a smaller valuable core plus refinements that could follow later, and finding that core is a skill that improves with practice.

Continuous Integration in Agile

The practice with the clearest cause and effect, and the one teams most often defer.

The problem it solves is straightforward. When several people work on the same codebase separately, their changes diverge. The longer they stay separate, the more they diverge, and the harder the eventual merge becomes. Teams that find merging painful respond by merging less often, which makes the next merge worse. That loop is self reinforcing and it ends with integration weeks.

Continuous integration breaks it by making the merge so frequent that divergence never accumulates. Everyone integrates to a shared trunk regularly, ideally more than once a day, and an automated build verifies each integration.

Three things make it work.

The build must be fast. A build taking forty minutes will be bypassed under pressure. Under ten minutes and people wait for it.

A broken build must be the priority. If the shared build stays red for a day, the safety net is gone and everyone is integrating onto broken foundations.

Everyone must actually integrate. One developer holding a branch for two weeks recreates the problem for everyone else.

None of this is a tooling question. Teams with excellent tooling and long lived branches get none of the benefit, and teams with basic tooling and genuine discipline get most of it.

Automated Testing in Agile Development

The practice that most distinguishes teams that sustain a Sprint cadence from teams that struggle with it.

If testing happens at the end of a Sprint, the team has recreated the phase gate that iterations remove. Anything found there either rushes through without proper fixing or carries over. The last two days become a scramble, and the pattern repeats every Sprint.

The alternative is testing continuously, which requires automation for anything repeated. Manual regression testing simply does not fit inside a two week cycle once a product reaches any size, and teams without automation eventually cannot maintain quality at that cadence regardless of how hard they work.

The Definition of Done is where this gets enforced. If it requires tests, items without tests are not finished, and the standard holds regardless of schedule pressure. Where the accountability sits is covered in our guide to who is responsible for quality in a Scrum team, and the short answer is the Developers, collectively.

One point worth making plainly. Automated testing is an investment with a delayed return, and the first months of building a suite feel slower. Teams under pressure frequently defer it for that reason and then never reach a point where it feels affordable. The cost of not having it is paid continuously and invisibly, in slower changes and more defects, which is why it rarely gets attributed to the decision that caused it.

Refactoring in Agile Development

Widely misunderstood as tidying up, which is why it gets deferred to a week that never arrives.

Refactoring means improving the structure of code without changing what it does. Its purpose is keeping the codebase changeable. A system that is hard to change makes every future iteration slower, and a team iterating on a degrading codebase eventually spends more time working around the structure than building anything.

The important part is that it is continuous rather than scheduled. Small improvements made alongside feature work, constantly, keep the code in reasonable shape. A dedicated refactoring Sprint is a symptom of having deferred it too long, and it is usually the least valuable Sprint a team runs because it delivers nothing to users while consuming a full cycle.

Practical approaches are covered in our guide to code refactoring techniques in agile. The relationship to technical debt is direct: refactoring is how debt gets repaid incrementally rather than allowed to compound.

The conversation this creates with a Product Owner is a recurring source of friction, since refactoring produces no visible feature. The framing that works is consequence based. Changes in this area take three times longer than elsewhere, and most of our defects come from here. Stated that way it competes on the same terms as everything else, rather than sounding like engineering preference.

Working Software as the Measure of Progress

One of the more consequential ideas in the Agile Manifesto, and the one most often reduced to a slogan.

Working software as the primary measure of progress means a feature that is coded but not tested, or tested but not deployable, counts for nothing. Not partially. Nothing.

That sounds harsh and it is the only measure that does not lie. Percentage complete is an estimate of an estimate. Tasks closed measures activity. Working software either runs or it does not, and no amount of optimism changes the answer.

The practical consequence is that a team should be able to demonstrate something real at the end of every Sprint. Not a slide describing what was built, not a walkthrough of code, something a stakeholder could use. Teams that cannot do this usually have items too large to finish, or a Definition of Done that permits partially finished work.

It also reframes what documentation is for. The Manifesto values working software over comprehensive documentation, which is not an argument against documentation. It is an argument against documentation as evidence of progress. A detailed specification is not a working system, and a project measuring itself by documents produced can be entirely on schedule and have nothing that runs.

Performative Agile: Ceremony Without Engineering

The failure mode has a name now, and it is worth recognising because it is the most common outcome of a partial adoption.

Performative agile is what happens when the events continue on schedule but nothing they produce changes anything. Planning happens. The Daily Scrum runs. Reviews occur. Delivery outcomes stay flat. The events have stopped being decision points and become procedural overhead, and the team is doing them because the framework says to rather than because they produce a decision that would not otherwise have been reached.

The diagnostic is straightforward. For each event, ask what decision it produced that would not have happened anyway. If Sprint Planning produces a commitment nobody believes, if the Retrospective produces a list nobody actions, if the Review produces no change to the backlog, the events are running and the mechanism is not.

What makes this specifically an engineering problem is the direction of causation. Teams frequently respond by fixing the meetings, tightening facilitation, shortening the standup, trying a new Retrospective format. That rarely helps, because the events are not the constraint. A team that cannot integrate more than once a fortnight, or cannot test without a manual regression pass, will produce flat outcomes regardless of how well its meetings run.

There is a measurable version of this. Teams that stop running Retrospectives consistently report declining velocity and rising technical debt within three to four Sprints, which is a short enough window that the cause and effect are visible if anyone is looking. The Retrospective is the mechanism by which engineering practice actually improves, and removing it removes the only regular point at which the team examines how it builds rather than what it builds.

Measuring Agility With DORA Metrics

If you want one measure of whether a team is genuinely working this way, use this: how long does it take to safely ship a one line change?

This is not an informal heuristic. It is close to what DORA, the research programme that has studied software delivery performance for over a decade, calls change lead time: the time from a change being committed to version control to being deployed in production. It is one of five metrics DORA now tracks, alongside deployment frequency, change fail rate, failed deployment recovery time and deployment rework rate. Worth noting that the set was four for years and is now five, so articles referring to the four key metrics are describing an earlier version.

A team that can do it in under an hour has automated testing, continuous integration, a deployment pipeline and a codebase they trust. Everything described above is in place, and it shows in the answer.

A team that needs a week has manual testing, a release process involving several people, and enough fragility that nobody wants to touch anything without extensive checking. Whatever their process looks like, they are not iterating in any meaningful sense.

The value of this question is that it cannot be answered aspirationally. Sprint length, story points and burndown charts can all look healthy in a team that takes three weeks to ship a trivial fix. The lead time for a small change reflects actual capability rather than intended process.

The reason to prefer it over the metrics teams usually watch is that it resists manipulation. Velocity can be inflated by estimating more generously. Tickets closed can be raised by splitting tickets. Change lead time is measured by the pipeline, and the only way to improve it is to genuinely improve the pipeline.

It is also the most useful thing for a Scrum Master to be curious about, because improving it improves everything else. Recognising that the constraint is engineering capability rather than process discipline, and raising it as an impediment rather than coaching the team to work harder inside a broken system, is a substantial part of the role. That diagnostic instinct is covered in CSM Certification Training.

A caution on using it as a target. Change lead time is a diagnostic, and the moment it becomes something a team is measured on by management, the incentive shifts to reporting a good number rather than improving the underlying capability. It works as a question the team asks itself, and stops working as a figure reported upward.

Why Agile Fails: Common Mistakes

Adopting the events, keeping the engineering. The most common failure by a wide margin. Sprints and Retrospectives layered on top of monthly integration and manual testing produce overhead without benefit, and the team reasonably concludes agile does not work.

Treating automated testing as optional. It is the enabling constraint. Without it, quality and cadence cannot both hold, and quality is what gives way.

Long lived branches. Recreates big bang integration inside a framework designed to eliminate it.

Deferring refactoring to a dedicated Sprint. Guarantees the debt compounds and produces a cycle with nothing to show for it.

Items too large to finish. Half done work marked complete, carryover as routine, and a Sprint Review with nothing usable.

Measuring activity instead of output. Points completed, tickets closed and hours logged all move independently of whether anything works.

Agile and AI

AI coding assistants and agents let teams produce code faster, which makes Agile's engineering disciplines more important, not less. When code is cheap to generate, small batches, automated tests, code review and continuous integration are what keep quality and change lead time healthy. Teams are also using AI to draft user stories and acceptance criteria, summarise Sprint Reviews and spot delivery risks, but decisions about value and priority still belong to the Product Owner and team.

Frequently Asked Questions

1. What is agile software development?

Building software in small working increments rather than sequential phases, supported by engineering practices that make frequent delivery viable: small batches, continuous integration, automated testing and continuous refactoring.

2. Is agile software development the same as Scrum?

No. Scrum is a framework for organising the work. Agile software development describes the engineering approach underneath it. A team can run Scrum correctly and still build software in a way that makes iterating expensive, which is covered further in our comparison of agile and Scrum.

3. Do you need automated testing to be agile?

For anything beyond a small codebase, in practice yes. Manual regression testing does not fit inside a short iteration once a product reaches any size.

4. How small should work items be?

Small enough to finish comfortably within the Sprint, and ideally within a few days. Items consuming most of a Sprint carry more risk and surface problems later.

5. Does agile mean no documentation?

No. It means documentation is not the measure of progress. Write what is genuinely useful and stop treating documents as evidence that work is advancing.

6. How often should a team integrate?

At least daily, ideally more. The longer changes stay separate, the more expensive merging becomes, and the cost grows faster than most teams expect.

7. Can agile work for non-software projects?

The principles transfer, and many of the engineering practices here do not, since they assume a codebase. Adaptations exist and they are genuinely different rather than a straightforward port.

8. Why did agile not work for our team?

The most common reason is adopting the process without the engineering practices. A second is an organisation that expects fixed scope and a fixed date, which removes the flexibility iterating depends on.

9. What is performative agile?

Running the events on schedule while nothing they produce changes anything. Planning happens, the standup runs, the Retrospective generates a list nobody actions, and delivery outcomes stay flat. It is usually a symptom of an engineering constraint rather than a facilitation problem.

10. How do we know if our engineering practices are holding us back?

Measure how long it takes to safely ship a one line change. If the answer is days or weeks rather than hours, the constraint is the pipeline rather than the process, and improving the meetings will not move it. Supporting the team to make that case is part of what CSM Certification Training prepares a Scrum Master to do.

11. What are the 4 values of the Agile Manifesto?

Individuals and interactions over processes and tools; working software over comprehensive documentation; customer collaboration over contract negotiation; and responding to change over following a plan.

12. What are the main Agile methodologies?

The most common are Scrum, Kanban, Extreme Programming (XP) and Lean, with SAFe, LeSS and Nexus used to scale Agile across many teams.

13. What is the difference between Agile and Waterfall?

Agile delivers working software in small increments with continuous feedback; Waterfall completes each phase (requirements, design, build, test) in sequence and releases once at the end.

14. What are the 5 Cs of Agile?

Communication, collaboration, commitment, customer focus and continuous improvement.

15. Does AI change Agile software development?

AI speeds up coding and documentation, but it raises the importance of automated testing, small batches and continuous integration, because more code is produced faster and must still be verified.

Closing Thoughts

The gap between teams that get value from agile and teams that do not is rarely about how well they run the events. It is about whether the engineering underneath can support working in small increments.

Small batches are the foundation. Continuous integration keeps them cheap to combine. Automated testing keeps them safe to release. Refactoring keeps the code changeable enough for the next batch. Remove any one and the others become progressively harder, which is why partial adoption so often produces worse outcomes than not starting.

The one line change question is worth returning to periodically. It is uncomfortable, difficult to answer optimistically, and points directly at whatever is actually constraining the team.

If you are the person responsible for spotting that constraint and raising it, CSM Certification Training covers the framework and the diagnostic skills that separate a process problem from an engineering one. Request the curriculum to see the agenda and upcoming dates, or start with the free CSM practice test to check your foundation first.