Deliver 30% more PRs to production by having developers own their own testing
Is testing "slow" in your team? Do tickets pile up in the "ready to test" column every sprint?
If you think that the testers on your team need to fix this you're wrong!
In a cross-functional product development team balancing the cadence and handover between development and testing is often a pain point. Read on as to why this is team problem and it's best resolved by getting developers to thoroughly test their own work.
Broken Kanban
A modern cross-functional team is expected to maintain a steady velocity of getting work out to production. i.e. the team as a whole is responsible for getting work out to production, not the last "function" before production.
Unfortunately many teams still operate with a fixed role mindset and a strict process of Design -> Dev -> Test -> Deploy. The last function before production is often QA and they get work piled on them to test and get out ASAP.
In particular if you're running Kanban then your team should only be working on a volume of work that matches the part of delivery that takes the team the most effort.
If one of your team's "columns" is filling up then you need to stop adding more stuff in to the pipeline! Developers are usually the folks starting new tickets so they should stop and instead help clear the test backlog.
It needs to be instinctual in the team to stop working and help get current work out to production. If this isn't happening in your team you might need to semi-formalise and evangelize Developer Testing.
Developer testing?
Developer testing doesn't mean filling in forms and clicking buttons (although that might be the thing that's needed to unblock your team's test queue sometimes!).
You need to go a bit deeper to figure out why it really takes so long for another team member to prove that your code solution works as expected and doesn't breaking anything else.
Developer testing is a mindset change. Think of it like you're trying to get your code through a quality assurance function faster. There are a few general ways to do this.
You can make help the function itself faster, you can reduce how many times the same code needs to go through the function or you can make the work item take less effort to pass through.
Consider these common causes of testing pain
- Regression testing the existing functionality is manual
- There are more devs producing more change than testers to verify it
- Devs don't test their own stuff properly so there are issues in PRs
- Handover is a jira ticket number in Slack
- The scope of change is massive
- The tester gets all changes at once with 3 days to go in the sprint 👎
These are all things that can and should be the responsibility of the developer. You can work on improving these for your current piece of work rather than starting a new item.
The levels of dev testing in a team
These levels describe some states of your developer testing. You might recognise your team in one of these! I believe all teams should operate at level 3 and this is what I try to get any team I work on to do.
Level 0
In teams where developers don't do much of their own testing they will handover broken software to testers. A developer will "complete" a task and set it to "In Test" or similar and the developer quickly grabs the next task.
There's no automated testing here.
The test team members have to perform enormous amounts of regression testing for ALL code changes. Developers keep pushing work to some test queue and an undesirable testing backlog quickly forms. The test team is stressed because they are forced to be gatekeepers.
When the test team need to ask the devs questions the dev needs to pull context from a few days ago because they've moved on to something else already.
The test team can only be reactive at this level because they're so busy testing and retesting the same stuff. They have no time to do proactive quality assurance work or future planning for product complexity (things like investigating and selecting an integration test framework).
Level 1
At this level the developer is writing unit tests for their code and the quality is better. The engineer and tester might have a good discussion about a change so there is some handover of knowledge between the two roles in person.
The test team might have had just enough slack so they were able to add an integration test framework to the project.
The test team is still busy manually testing all the work. They're finding all the issues that the developers miss and the team is having to redo work all the time.
The test team don't really have time to add many integration tests so the initiative isn't seen as a successful and the team get disheartened.
There is still lots of manual work, knowledge handover and context switching back and forth between tester and developer causing inefficiency.
Level 2
This team has unit tests, integration tests to cover the main scenarios or any scenarios that take a long time to set up and they discuss changes before during and after working on them.
The developers are spending time adding integration tests that prove their work solves the problem and doesn't cause unintended side-effects. This effort to write integration tests seems to slow down developer throughput initially. In fact it halves how many tickets the developers do per week.
Quickly the test team notices there are far fewer issues in the code changes they receive. They can regression test the main app functionality in 2 minutes where it used to take 30 minutes to an hour. Developers are discussing the tests to implement with the QA expert early so there is adequate coverage.
The test team have time to review what's coming in the product pipeline and suggest concerns or ideas to the team before the coding even begins. They can work on introducing more quality automation and 10x'ing their contribution to the product rather than just doing the same regression testing everyday.
Level 3
Now the team has a full suite of integration tests for the product. There are so many that the tests are split into suites. The smoke test suite is run on every deploy in CI so you don't have people manually checking the QA or production environment anymore. There is no context switching across days for developers and testers now. Everyone gets feedback on their work in minutes.
There is no test backlog requiring a combined release train. The developers know they can release whenever their code is ready, so they make smaller changes! A developer can arrive in the morning, make a change, get the change and their tests reviewed in the afternoon and deploy it.
Guess what? There is no QA column on your Jira board anymore. Testing is just a part of development like writing code and doing code reviews. The team can confidently put changes in to production safely and calmly multiple times per day.
This is at least where every team should be! This should be the baseline for your quality and delivery practices in software product engineering.
Evangelizing testing
Now that you know where you're at you can start evangelizing testing in your team. Every team is different but there are some things I've found useful in the past. Developers can be given some lessons on what QA is.
All of the team must agree on what is risky for your product. Developers and test analysts often have different ideas of what is high risk. You should have a discussion about all the risk factors that affect code in your system and make sure that there is general understanding and empathy across the team for what each team member considers risky.
Have a test analyst in the team run some sessions on how to test for common issues in your application. Have everyone on the team work through some of the items in the following two links so they are all aware of some of the tools and techniques available for testing software.
Have the developers review the test heuristics cheat sheet here: http://testobsessed.com/wp-content/uploads/2011/04/testheuristicscheatsheetv1.pdf
You can run a session with the team about Exploratory Testing based on this: https://docs.microsoft.com/en-us/previous-versions/visualstudio/visual-studio-2013/jj620911(v=vs.120)?redirectedfrom=MSDN#bkmk_tours
Specific changes you can make to the ways you work
First thing is to measure where you're at. The number of defects that make it to test stage currently or defects in production are good things to measure before making changes. Your measure doesn't have to be perfect because the improvements from shifting to testing earlier in the process will be significant.
Choose a unit test or integration test framework (depending on what level you're at). Don't over think this, anything is better than nothing. If developers are going to be writing most of the tests then a code first framework is probably better than a GUI test framework.
Communicate what's happening to all the stakeholders. When teams are changing the way they work and are learning to work cross functionally it can be hard to convince everyone else in an organisation to follow that pattern.
This means that stakeholders on the fringes of the team might still expect waterfall type progress and might be frustrated that developers are spending so much time writing tests. This will fade over time with the improved team throughput but it needs to be managed carefully during the transition.
This change needs to go both ways - if devs are getting comfortable testing then testers need to get comfortable with code. Even with developers writing tests they may only test the happy path or will sometimes make bad assumptions. A tester needs to code-review the code and tests that a developer writes.
You will have a much better experience if you can have one change per deploy. This is easier for web based applications and harder for apps on mobile devices that need to go through approval. You CI system should make deploying to production automatic.
You should decouple deploying code and releasing code. The business side of the organisation should be able to control releasing new features and that shouldn't dictate how you engineer the feature. You can use a feature toggle framework to accomplish this.
Focus on integration tests instead of unit tests if you're just starting out with testing and there is no existing coverage. Integration tests will provide more value per test. Make sure you add tests every time a defect gets to production. The point here is to reduce regressions from occurring.
Conclusion
If testing is a bottleneck in your team your first reaction might be something like - "We need more testers to handle all the code we're writing".
Instead consider that your test function is swamped just testing regressions in the code. They have to do this because they regularly find regressions in your code! It's a viscous cycle. Next problem is that testing regressions in your system is probably manual, and therefore slow.
Developers have the ability to fix a significant amount of these issues at the root by proving early that their own code works.
Modern testing frameworks make proving that your code works relatively easy to implement. This proof can also act as excellent living documentation for yourself and others later.
If you're a developer take ownership and test your own code fully with automated tests before handing it to anyone else!