NIST SP 800-115: technical security testing and assessment, explained

NIST Special Publication 800-115, Technical Guide to Information Security Testing and Assessment, is a September 2008 NIST guide — authored by Karen Scarfone, Murugiah Souppaya, Amanda Cody and Angela Orebaugh — written "to provide guidelines for organizations on planning and conducting technical information security testing and assessments, analyzing findings, and developing mitigation strategies." It replaced the earlier SP 800-42, Guideline on Network Security Testing, and remains current on NIST's publication list as of this guide's research date, with no superseding revision published. Although developed under NIST's FISMA responsibilities for federal agencies, NIST's own text states it "may be used by nongovernmental organizations on a voluntary basis and is not subject to copyright."

Three types of assessment method, not just "testing"

SP 800-115 opens by separating three distinct assessment methods that are often conflated under the single word "testing": Testing ("exercising one or more assessment objects under specified conditions to compare actual and expected behaviors"), Examination ("checking, inspecting, reviewing, observing, studying, or analyzing one or more assessment objects to facilitate understanding, achieve clarification, or obtain evidence"), and Interviewing ("conducting discussions with individuals or groups ... to facilitate understanding, achieve clarification, or identify the location of evidence"). The document's technical focus is on the first two, organized into named techniques.

The three categories of technique

Section 2.2 groups the document's technical techniques into three categories, each carrying a different level of risk to the systems being assessed:

Category	What it does	Risk level	Example techniques
Review	Passive examination of systems, policies, and configuration	Minimal — "generally conducted manually"	Documentation, log, ruleset and system configuration review; network sniffing; file integrity checking
Target Identification and Analysis	Finds systems, ports, services, and potential vulnerabilities, usually with automated tools	Moderate	Network discovery; network port/service identification; vulnerability scanning; wireless scanning; application security examination
Target Vulnerability Validation	Proves a vulnerability is real by exploiting it	Highest — "the greatest amount of risk in assessments"	Password cracking; penetration testing; social engineering; application security testing

NIST is explicit that no single technique is sufficient on its own: "Since no one technique can provide a complete picture of the security of a system or network, organizations should combine appropriate techniques to ensure robust security assessments." A typical penetration test, for instance, relies on target identification and analysis techniques (port scanning, vulnerability scanning) to find candidates before the attack phase attempts to exploit them.

Vulnerability scanning vs. penetration testing

The document draws a sharp, often-blurred line between these two Target Identification/Analysis and Target Vulnerability Validation techniques. Vulnerability scanning "identifies hosts and host attributes (e.g., operating systems, applications, open ports), but it also attempts to identify vulnerabilities rather than relying on human interpretation of the scanning results" by comparing a host's fingerprint against a signature database — NIST flags that this approach "can have a high false positive error rate" and only reliably finds "surface vulnerabilities," weaknesses that exist in isolation rather than in combination with others.

Penetration testing goes a step further: "While vulnerability scanners check only for the possible existence of a vulnerability, the attack phase of a penetration test exploits the vulnerability to confirm its existence." Most penetration tests specifically hunt for "combinations of vulnerabilities on one or more systems that can be used to gain more access than could be achieved through a single vulnerability" — exactly the kind of chained, interacting weakness a scanner's isolated signature matching tends to miss.

The four-phase penetration testing methodology

Section 5.2.1 lays out what it calls an example of how penetration testing can be divided into phases — NIST notes directly that "there are many acceptable ways of grouping the actions involved" — using four phases that run partly in parallel:

1. Planning. Rules are set, management approval finalized, goals agreed. "No actual testing occurs in this phase."
2. Discovery. Two parts: information gathering/scanning (host and service enumeration, OS fingerprinting, open-source reconnaissance), then vulnerability analysis — comparing findings against vulnerability databases such as NIST's own National Vulnerability Database (NVD).
3. Attack. The exploitation step, with a loopback: per Figure 5-2, a successful attack moves through Gaining Access → Escalating Privileges → System Browsing → Installing Additional Tools, and any new foothold can trigger another round of Discovery against newly reachable systems.
4. Reporting. Runs concurrently with the other three phases rather than only at the end — the assessment plan is itself a reporting artifact from the Planning phase, and periodic status reports continue through Discovery and Attack before a final report closes the engagement.

The Attack phase's most common vulnerability categories, per the document, include misconfigurations, kernel flaws, buffer overflows, insufficient input validation (e.g., SQL injection), symbolic-link abuse, file descriptor attacks, race conditions, and incorrect file/directory permissions.

Testing viewpoints: external/internal and overt/covert

SP 800-115 separates two independent axes of "viewpoint" that shape how a test is run. External vs. internal: external testing is "conducted from outside the organization's security perimeter," typically starting with public reconnaissance before enumerating what's reachable through the firewall; internal testing places assessors "on the internal network" with some baseline access, focused on privilege escalation and system-level hardening. NIST recommends running external testing first if both are planned, "as it keeps [assessors] from acquiring insider information ... that would not be available to an adversary."

Overt vs. covert: overt ("white hat") testing happens "with the knowledge and consent of the organization's IT staff," which lets staff limit impact and learn from watching the test. Covert ("black hat") testing happens "without the knowledge of the organization's IT staff but with the full knowledge and permission of upper management" — often mediated by a trusted third party so the target's own staff doesn't inadvertently confirm a drill is underway before it's complete. NIST is candid about the trade-off: covert testing's purpose "is to examine the damage or impact an adversary can cause — it does not focus on identifying vulnerabilities," and it is typically "time-consuming and costly" because the team must operate "under the radar," whereas overt testing "is less expensive, carries less risk ... and is more frequently used."

Planning: the assessment plan or Rules of Engagement

Section 6 treats planning as the foundation of a safe assessment. Every engagement should have an assessment plan or Rules of Engagement (ROE, with a sample template in Appendix B) answering five questions the document poses directly: What is the scope? Who is authorized to conduct it? What are the logistics? How should sensitive data be handled? What happens if an incident occurs? The plan should name explicitly excluded systems, the authorized testing IP ranges, data handling and retention requirements, and an incident-handling protocol — NIST's worked example: if assessors discover a real intruder mid-test, should testing stop, and by whose authority can it resume?

Where this sits next to other security-testing references

SP 800-115 is a technique-and-methodology reference, not a compliance checklist or a risk-scoring framework. For prioritizing which vulnerabilities matter once they're found, pair it with CVSS (see our CVSS v4 vs v3.1 guide) or CISA's KEV/EPSS signals (see our KEV + EPSS guide). For OT-specific testing considerations, SP 800-115 repeatedly cross-references NIST SP 800-82 (see our NIST SP 800-82 OT security guide) for compensating controls where active scanning risks disrupting control systems.

A workable starting point

1. Pick the right category for the objective — a quick compliance check needs review and target-identification techniques; proving real-world exploitability needs target vulnerability validation, with its correspondingly higher risk and cost.
2. Write the ROE before any tool runs, including an explicit exclude list and an incident-handling protocol, so a discovered real intrusion during testing doesn't get mistaken for test traffic or vice versa.
3. Don't treat a vulnerability scan as a penetration test. A clean scan result means no known signature matched — it says nothing about whether chained, lower-severity issues could still be combined into a real compromise.
4. Decide overt vs. covert deliberately. Covert testing measures detection and response readiness; overt testing measures the technical security posture itself. They answer different questions and cost different amounts.

SoftWSP's device and firmware security work can help scope a technical assessment — from documentation review through validated penetration testing — against connected products and their supporting infrastructure: see device & firmware security services.

This guide summarises NIST SP 800-115; it does not constitute legal or compliance advice. The document is from 2008 and has not been revised since — confirm current tool and technique recommendations (e.g. specific scanner names in its appendices) against more recent sources before relying on them operationally.