Should I Change Penetration Testing Companies Each Year?

We often get asked whether it is a good idea to change penetration testing companies each year. Obviously we don’t want our clients to leave us and we pride ourselves on building a long term relationship with them, but we will always offer advice that is in line with their best interests. As with anything, there are pros and cons when you change penetration testing companies, and in this blog we will lay out some of those, along with our suggested approach.

Pros When You Change Penetration Testing Companies

The most often cited reason for rotating penetration testing firms is to get “a new set of eyes” on the test. A different engineer will sometimes find different things or things that were missed by the previous tester, due to having different backgrounds or different specialties. This is not unusual for penetration tests, and speaks to the artistic side of testing and the manual processes required. It’s also why you want to make sure that whoever is doing the testing, they are trained and experienced in order to provide thorough results. Additionally, different companies may have slightly different methodologies (although you should make sure any methodology is based on an industry standard). This means that different companies may focus more on one aspect of testing than another company.

Cons When You Change Penetration Testing Companies

On the flip side, there are a few major drawbacks that come when you change penetration testing companies. First, you lose any statistical trends that can help show the progress your organization made year-over-year. For example, in our reports, we rate each assessment on a scale of 1 to 5 overall. 1 represents significant issues and 5 represents best practice. So if we performed a series of tests for your organization, our report would show the risk trend over time. This helps to show the progress you’ve made in your environment from a security perspective.

Another major drawback is that all penetration testing firms are not created equal. There is a lot of trust involved in having someone assess your security and actively exploit vulnerabilities during a penetration test. Not only that, but we often hear from clients about issues they have had with other firms. These range from “we basically got a vulnerability scan” to “I had lots of issues with their responsiveness and overall communication.” So from a quality and trust perspective, once you find a firm that you like, it may not be worth the frustration of finding another equivalent organization.

The third and final drawback can be the loss in customization and situational awareness that comes with changing companies. When a penetration testing company partners with you, they get to know you and your environment over time. This helps them to customize the documentation, prioritize findings more in alignment with your organization’s business, and helps provide more relevant remediation recommendations. For example, we have some customers who have service level agreements that require them to meet up-time requirements. For those companies, we know we need to stress any findings that relate to availability. For others, we know they need to keep certain client data strictly confidential, so we focus the report on those issues. Additionally, we get to know your team and what your strategic priorities are. If you are trying to justify rolling out multi-factor authentication, we can stress that in the report and presentations. We will always note it as a finding because it is a legitimate security concern, but if we know it is your priority and you are trying to cultivate buy-in, we can help make sure it is highlighted.

Our Approach

So how do we handle some of these considerations and risks at Triaxiom when we’re testing clients multiple years in a row? First of all we make sure that you have a fresh set of eyes to make sure nothing was missed. Our approach is to rotate who performs each test for your organization. If Engineer 1 did the test in 2018, Engineer 2 will do the test in 2019. However, to make sure you still get all those benefits we discussed of using the same company, we will have the engineer from the previous year perform Quality Assurance for the project. This way, you are getting a fresh set of eyes on your environment and a slightly different skill set, but you keep the advantages of sticking with one firm.