 |
| By Cosmocatalano (Own work) [CC0], via Wikimedia Commons |
Wikipedia does a pretty decent job of describing the technical details and constraints that define the
project management triangle (see the image to the left) but while a decent textbook definition helps explain what this means, working in software and experiencing it firsthand makes it a bit more potent. In my current project we're pushing a significant amount of scope very quickly. We're also doing so with a staff augmented by a third party contracting company, but don't have a large enough team fully engaged to deliver both features and quality - It's getting lots of features out fast, and comparatively cheap, but the quality has some obvious hurdles for us to get past. I'd like to focus on our strategy for addressing them here. We have some catch-up work to do, and since we're starting to focus on it I think it's a good thing for me to write about.
Technical Debt
Application Logs
In order to ensure that our application performs well, we need to rely on application logging statements. As of right now the logging framework we have in place can do a pretty good job of capturing system events and error messages, but we have to ensure that all of our end points and database access properly make use of the logging in the first place. Once we have the log messages, we're collecting them all using an instance of LogStash, and have them on a dashboard rotating above our heads.
Static Code Analysis
We have a SonarQube instance that runs static code analysis over our source code, and also aggregates some of the other quality metrics that we collect. It shows us the most critical areas of technical debt that we need to address (and believe me, there are several) and also shows the coverage and completion rate of our tests. Really, it's my goal right now to make SonarQube our central hub for capturing all of our testing metrics, because I think it's easier to remember to look in one place than to have to do so in many.
Feature Quality
The Testing Pyramid
I'm not going to spend a significant amount of time explaining what the testing pyramid is,
this article probably does a way better job anyway. Suffice it to say that we're stuck with an ice cream cone, and summer is heating up quickly, it's going to melt.
Failing Tests
Right now, even though we do have some test coverage generated by a suite of integration tests, one of the biggest things we need to address and eliminate is the team's willingness to check in broken tests. We have some tests failing, and some tests that just won't even run due to errors in the test code. Our analysis tool doesn't have "100%" as the success rate of our tests. Without having assurance that all of our tests succeed, there's really no reason to be writing test cases in the first place.
Application Performance
Operational Monitoring
We're using a commercial tool to help us monitor application state, specifically it helps us watch things like response times and apdex score. In order to ensure that all of our other forms of testing are in fact improving our application quality, this tool will really be the go-to for showing how things are going in the wild. Without a tool like this one we'd really lose the "Ops" part of "DevOps" in our culture, because this shows us where we need to focus our attention the most critically. All of the other developer testing we're doing helps us to greatly improve the chances that our operational runtime will be successful. But when it isn't, we need the right tool in place to show us where to look first.
Performance, Load, and Stress Testing
Application performance testing is one of the most severe gaps in the software development process right now where we are. With all of the other quality checks in place that we have, one place that we just recently focused our attention is overall user satisfaction with response time. We recently (as in, last week) have a jMeter library set up that runs automatically and collects statistics about response times for our application. We'll keep using jMeter to push our application as best we can, to weed out bottlenecks and eliminate them as quickly as possible.
