Monday, June 23, 2008

Penetration testing as an art

| Armando Romeo |

I found Chris Eng post about the correct definition of Penetration testing quite interesting. Whether you consider it an art or a science is not just a play on words or a way to make you feel Einstein or Michelangelo.

It's a way to have it carried out in the correct way.

Into Penetration testing, the approach ( modus operandi ) is most of the time much more important than the tools or checklists you use.

While someone thinks, aided by the so many checklist-type security being trendy recently, that penetration testing is a science, there's a lot of other experts who look at it as an art.

I believe both are wrong. Security is known to be one of the most difficult problems an engineer has to solve. Above all you stand from the good-guys side.

Only people in this room will understand what I am now going to say. It is this: Security is perhaps the most difficult intellectual profession on the planet

This is what Dan Geer's said in a keynote at SOURCE Boston.

Actually it's this difficulty that makes someone think that security and specifically penetration testing is an art and not a science.

It's not, IMHO, that a problem has too many variables and factors to take into consideration in order to be solved that it takes some talented mind to actually do it. It's all a matter of skills in this case.

If you could master all the possible aspects, details, behaviours and even the most hidden tricks of the system you're pentesting probably it would merely be a science since it would engage your experience/skills and not your talent as a genius mind.

It would just require your past knowledge and not your intuition.

There's a sharp distinction between the intellect and the experience.

The problem is that you can never know all the details, behaviours and features of your target because every target has its level of uniqueness.

And this is where the innate ability of intuition comes in handy, or better comes in *necessary* to master the most difficult intellectual profession on the planet.

It's just here that the science leaves the scene to the art.

A talented penetration tester wouldn't only mark the check lists but would find new unlinested methods to achieve his goal of breakinf/securing the system he's on.

Logic flaws comes to my mind here. But how could I forget about social engineering. In the end, Mitnick is probably the most renowned hacker ever just because of his innate talent. I don't remember of him being celebrated for elite attacks learned into some ethical hacker certification course.
(That you have to master anyway if you want to enter the field).

Both social engineers and all those flaws related to logic cannot be solved through a scientific approach to the problem. No tools baby and no checklist.

That's why I believe that, yes, penetration testing, must have an agreed
and standardized scientific approach, based on checklist, best practices etc.
But for each checks in that list, for each of those steps in the scientific/deterministic process of penetration testing there must be space for further investigation left for talent.

And Talent is not something you achieve with CISSP. You're right dre.
Day by day hands-on practice is the only training for your talent.

If I needed a penetration tester I would ask him how many years of experience he had in the trenches rather than how many certs.
And possibly if he had a past as a hacker.
Real hackers have some talent, and hands on practice by definition.

I don't believe into born-white-hat security professionals.
No matter how CISSP they are are.
Do you?

Free Security Magazines