CTF vs Real Penetration Testing

In today’s blog, we’ll discuss the differences between a CTF vs real, professional penetration testing, and the mindset required for each. We’re primarily aiming this article at aspiring and junior penetration testers, by highlighting some of the things to think about when transitioning from a CTF-style environment to that of a professional penetration testing firm.

Capture the Flag

For those of you that are unfamiliar with the term, CTF stands for Capture the Flag and is essentially a hacking competition. These CTF competitions can come in various forms. For example, one CTF competition may consist of a ‘red team’ (attackers) vs a ‘blue team’ (defenders) where the red team are attempting to compromise the blue team’s network. Alternatively, it could be a competition where various competitors or teams compete against each other in a race to capture the most ‘flags’ from various machines on a target network. These flags will often come in the form of a text file or snippet hidden somewhere on the target machine. Therefore, to capture the flags a competitor has to achieve various levels of compromise or access to a target, presumably leveraging security vulnerabilities in the process.

CTFs are a great way to learn about hacking techniques and the various tools that are used. For many aspiring penetration testers, CTFs are a great way to gain real-world, hands-on experience. However, while the benefits of CTFs for aspiring penetration testers cannot be overstated, the transition from CTF competitor to a professional penetration tester is not as seamless as some often assume. There are various adjustments that need to be made.

CTF vs Real Penetration Tests

For many people looking to become penetration testers, their only hands-on experience may come in the form of CTFs where the main goal is to compromise the system and capture the flag. And these skills will most certainly come in handy. But a professional penetration test is conducted differently. The primary goal of a penetration test is to find as many vulnerabilities as possible, in order to help the client understand their level of risk and take the relevant remediation steps. Therefore, many vulnerabilities that are unlikely to lead to direct system compromise in a CTF environment will often be overlooked or even ignored. However, overlooking said vulnerabilities during a professional penetration test could lead to catastrophic consequences. For example, if a Denial-of-Service vulnerability were identified during a CTF, it would likely be ignored, as it is not going to help the competitor achieve their primary goal of capturing the flag. However, in the real world, for a large company with an SLA agreement that consists of 99.999% up-time, for example, even the thought of being the victim of a DoS attack would be enough to keep a CISO up at night. Thus, a vulnerability that would have likely been ignored in a CTF environment, would be a big deal in a real penetration test, and could not be ignored.

Although the above was just one example, this should help to illustrate the difference in objective of a CTF and a professional penetration test. Junior penetration testers will need to adjust from the ‘get root’ mentality to that of trying to find as many vulnerabilities as possible, even as trivial as some might seem, so that the organization in question can make sure they have all their bases covered when it comes to securing their environment.

Another adjustment that will need to be made is restraint and the ability to think methodically. Many exploits that are commonly used by penetration testers and/or black hat hackers have the potential to crash systems if not used carefully and with due diligence. This may not be a problem in a CTF environment where target machines will likely have the ability to revert to their original state in the event of a system crash. However, this could lead to disaster in a corporate network. Therefore, every exploit should be carefully examined and tested before being used during a penetration test.

Finally, we will discuss the need for good documentation. Even the most technically gifted hacker will not make a good penetration tester if they are unable to present their findings to their client. It is no good being able to take full control of a company’s network if you are unable to convey how you did it, and what the company needs to do for remediation. Some of the most sought-after penetration testing certifications, such as the OSCP and eCPPT for example, both stress the importance of good documentation and we would highly recommend aspiring penetration testers get into the habit of taking good notes and honing their documentation and presentation skills.

In conclusion, we hope this short blog will be helpful to junior and aspiring penetration testers, and it is by no means meant to bash or denigrate CTFs and the experience they provide. In fact, here at Triaxiom many of our engineers regularly take part in various CTFs to test and sharpen our skills in a fun learning environment. As previously stated, the benefits of participating in CTFs cannot be overstated and we would encourage all infosec professionals, both red and blue teamers, to take part in CTFs whenever the opportunity arises. However, there are a few adjustments to be aware of when transitioning to a professional penetration testing role, some of which we have highlighted.  

For those of you that want to learn more about a career in penetration testing, check out our blog here.