Stop Creating a Repo For Your QA Automated Tests
QA teams can be responsible for a variety of automated test types, including UI, API, performance, and security tests. Many times, I see these automated tests in a code repository that is separate from the application code repository.
Why is this a bad idea?##
1. Encourages waterfall behavior
In this model, developers merge their changes before QA merges theirs. Even if the developer's code is perfect, QA might need additional support for things such as element IDs or locators. By the time QA realizes that, the developer has moved onto their next assignment and are sometimes not excited to context switch and address a task they feel is already complete.
Even worse, unexpected DOM changes may cause existing tests to fail. What happens when our automated tests have not yet "caught up" and run against these latest dev changes? The tests fail, and the tests are labeled as “brittle”.
2. Reduces collaboration between Dev and QA
Separating the repos can also result in Dev and QA working independently in separate silos. Ideally, manual and automated testing would happen in parallel. When working together, QA can quickly identify issues and communicate that info to the developer, shifting the testing left.
In addition, Devs can provide technical guidance related to building automation tests. This collaboration will result in better code coverage, with tests pushed down the test pyramid.
3. The CI pipeline needs to coordinate between 2 repos. But how?
Hopefully the automated tests are running in a CI pipeline to validate developer changes are good. When developers merge their changes, the pipeline will run these automated tests.
But the QA repo has not been updated when the developer merges their changes. How do we manage that disconnect? If the 2 repos are not in sync, then we can't.
So what do we do?
The Alternative##
Instead of splitting the code and tests into two repos, we need to consolidate into a single repo.
With this strategy will come collaboration between Dev and QA. The work is not considered complete until both the code and tests are done.
In addition, Devs know what they are testing with unit tests and that insight may help reduce the number of duplicate tests at higher levels.
With the automated tests in sync with the code, automated tests can be run locally before merging to the repo, greatly reducing the chances of breaking the build.
Once the merge happens, the CI process will not need to worry about two repos that are not in sync. We now have a single repo with code and tests. We have run the tests locally and can feel confident the merge will not break the build.
Wrap-up
In conclusion, even though I have referred to “Dev” and “QA” a lot in this post, these terms make my eye twitch a bit. Ideally an agile team is not Dev or QA, but a single team. Team member may have different specialties, but they all work together to create quality software. A single repo for the code and automated tests can help encourage that mindset.