Intrusion detection and prevention systems are something of a way of life to my buddy Arron Finnon(aka @finux), he's a fairly regular speaker at conferences on evasion techniques, technical misgivings and the general mis-use of intrusion detection/prevention systems. After much discussion with him at @Athcon earlier this year we were agreed that something was missing from the community. IDS / IPS have become something of a dark art in network security, with the overhead of managing endless tuning profiles, architectural issues, false positives, false negatives and claims from over zealous vendors that are very rarely reality in deployment. At @AthC0n I floated an idea with Arron that me and the other makeitcompliant bloggers have been discussing for a while which was the need for security-vendor testing criteria that could be repeatable, automated and consistent across products so that different vendors can be evaluated in a neutral manner, rather than a paid-for lab certification. This lead to a conversation on the sheer volume of IDS technology in the market, in no small part thanks to :
PCI DSS - 11.4 Use intrusion-detection systems, and/or intrusion-prevention systems to monitor all traffic at the perimeter of the cardholder data environment as well as at critical points inside of the cardholder data environment, and alert personnel to suspected compromises.
Keep all intrusion-detection and prevention engines, baselines, and signatures up-to-date.
IDS functionality is becoming prolific now, integrated into firewalls, as host software, stand alone infrastructure, appliances the list is endless. However in my experience I'd seen common issues in deployment. These were not compliance failings, the PCI standard is very flexible with IDS specifics because it is such a broad technology deployable in many ways. These issues were because of the breadth, confusion seems to be the norm in just where to begin with an IDS/IPS deployment. Having seen IDS regularly deployed in scenarios where its inspecting 1% of the throughput because the other 99% is encrypted, or its been deployed post-breach and then tuned to a potentially compromised environment - I am always somewhat sceptical of the real value of an IDS. There are numerous evasion techniques readily available in metasploit already, and as the subject of one of Arron's talks, there are even techniques that were originally designed for evasion, that due to issues with metasploit were run as the norm and IDS' tuned to them meaning the original exploit technique actually goes un-noticed by the IDS!
So, over in the Athens heat a conversation started along the lines of, "wouldn't it be nice if we had an OWASP like framework for intrusion management"..... This lead to the concept that would become OSNIF, something we hope will give consistent guidance on the use, testing and deployment of IDS/IPS.
When we met up again at DeepIntel, we stumbled across Richard Perlotto from Shadowserver.org and were mid way through a conversation about how to sensibly go about setting up a IDS/IPS testing methodology that could be done consistently without just depending on the metasploit tools. After a short "mmmm" Richard said, we might be able to help with that. Shadowserver does lots of AV testing and scoring against malware in the wild and has a whole stash of pcap resources that would be beneficial to run against an IDS / IPS in a similar way..... We will definitely be talking to them in future about how they can help and what we can do with some of their data.
Arron's managed to herd the cats caught up in the initial discussion, some goals have been set and I'm quite pleased to be one of the volunteers on this project.
The initial objectives of OSNIF are as follows -
· Develop an OSNIF Top 5.
· Developing a “Risk Assessment” guide for deploying detection systems.
· Developing a “Best Practices” deployment guideline.
· Developing an Open Source IDS/IPS testing methodology.
· Operate as an Independent legal organisation to maintain, and manage community data.
The OSNIF framework could well be the start of some common-sense open and collaborative thinking in this space. I hope so. Head over to osnif.org to get connected with the various mailing lists etc.