Scalability and Agile Development

How can you scale a module that you are building in an agile method? Should you turn to Agile Development to be able to scale better your services?
Agile development has many of the components that are critical to be able to scale when you need.
I added to Version One diagram about Agile Benefits my version of the Scalability chart:

Agile Development gives you the ability to develop you services in shorter cycles of development toward the right scale target.
To be able to scale in an agile way to need to practice the basics of agile development; Tests, Refactoring and Continuous Integration and Deployment. I will explain why.

Tests

The heart of every agile development is automatic testing practice. It can be called Test Driven Development, Behavior Driven Development or Design by Contract. It can have Sophisticated system tests or a simple client testing for your services using Selenium or WebDriver. It should give you the feeling that you are protected from changes that might break your services. The tests are working against your API definition. The API can be on a class level to allow you to use it from other classes, or it can be for your whole service with Authentication and REST calls. The more tests you have, the more agile you can be.
These tests are used to check that you are delivering the needed functionality, and can be used to check that you are delivering the needed non-functionality (Scalability, High-Availability...). These tests are the trigger to identify when and what to scale, and should later be used to check that the new code did the trick.

When you are protected by tests, you can change your implementation and check that you didn't kill the needed functionality, and if you did break something, you discover it very early.
The Agile way of development your tests and services is as follows:

• Write Functional Tests
• Write Simple Implementation to satisfy the tests
• Add (or extend) Functional Tests
• Refactor your implementation to satisfy the new tests as well
• Add (or extend) Non-Functional Tests
• Re-Architect your implementation to satisfy the non-functional tests as well

Refactoring

if you are already practicing refactoring ('a series of small behavior preserving transformations. Each transformation (called a 'refactoring') does little, but a sequence of transformations can produce a significant restructuring'), you in the right mind set for breaking out your smelly modules.
Refactoring thinking is very similar to the thinking that is needed for the right Scaling decisions. Most of the changes that you are doing are not giving any functional benefit. They are just reducing the bad smells of your code. The same reason works for Scalability smells, which are usually on a different order than code smells, as they are system smells. I discussed the Scalability smells in "When to Scale".
There are early patterns for Scalability in the refactoring patterns. You can use for example, Extract Package pattern to start your journey to "divide and conquer" you system. But there is a need to a new set of patterns to allow a more significant changes in a system, beyond simple Java only and Single System scope that are covered by the refactoring patterns.
I think that it is worth to learn from one of the recent refactoring patterns, split loop,  that was added by Martin Fowler:

"You often see loops that are doing two different things at once, because they can do that with one pass through a loop. Indeed most programmers would feel very uncomfortable with this refactoring as it forces you to execute the loop twice - which is double the work.
But like so many optimizations, doing two different things in one loop is less clear than doing them separately. It also causes problems for further refactoring as it introduces temps that get in the way of further refactorings. So while refactoring, don't be afraid to get rid of the loop. When you optimize, if the loop is slow that will show up and it would be right to slam the loops back together at that point. You may be surprised at how often the loop isn't a bottleneck, or how the later refactorings open up another, more powerful, optimization."

I believe that this is the same reasoning for systems that are doing two different things at once. Trying to optimize the services, by merging them to a single machine/DB/WAR file..., is sometimes the wrong way. Splitting in many cases can open up another, more powerful, optimization.

DevOps

The concept of DevOps or Continuous Deployment is an advanced concepts in the agile development world. After we put on the shoulder of the developers the task of writing tests ("Am I a Tester?", "I don't have time to write my code AND the tests..."), we are now adding the task of deployment and operation.
These are certainly not easy concepts to implement, especially if you are not starting with a blank page. You can read for example an interesting description of introducing Continuous Deployment in Outbrain. Even for star developers like the ones in Outbrain, it was a real challenge, how can we expect for "the rest of us" to be able to do it.
Here as well, the ability to split your system into independent services, that each one of them is able to have its own release cycle, makes it is much easier to apply DevOps techniques. It will help you to:

• Code in small Batches - if the system is too big even refactoring causes changes in too many places in your code, forcing you to deploy (build, test, copy...) too many changes each time
• Keep the service MVP (minimal Viable Product) properties - it is easier to play (A/B testing) with a smaller service, and keep it focused
• Maintain Functional Test Coverage - Independent service is easier to test independently
• Maintain Monitoring and Alerts - faster problem detection (and prevention)
• Roll Forward Fixes - if the service is small, you can fix issues with actual enhancements instead of rolling back to the last working version, as most operation guys do

The Scalability Paradox

Instead of having a small team of expensive developers and large teams of less expensive testers and system operators, we are reducing the size of the less expensive teams and increasing (at least the load on) the more expensive developer team. This strange economic math is true in the world of scalability.