Benchmark Basics

Formal computer security guidance helps enterprises of all sizes to implement effective security actions. These security actions are typically specified via benchmarks.

A computer security BENCHMARK is a document that specifies settings and option selections that minimize the security risks associated with a computer hardware or software system. Written in Extensible Configuration Checklist Description Format (XCCDF) language, it can be read, translated, and presented by checking tools. However, it does not contain the information needed to evaluate compliance. An AUTOMATED BENCHMARK document is required to perform the compliance check. Automated benchmarks specify the same settings and option selections as standard benchmarks, but in addition to being written in XCCDF they use one or more standardized checking languages to contain the information needed to evaluate compliance. MITRE advocates the use of automated benchmarks because of the compliance checks.

Benchmark Components

Benchmarks can be written for installation and configuration issues, deployment options, account and password policies, settings for security features, product key features, etc.


Each benchmark contains a "Rule," which is the unit of a benchmark containing the recommendation, rationale, and how to of the specified action. In the case of an automated benchmark, a Rule consists of the recommendation, rationale, how-to, and the compliance check.

The subcomponents of the Rule are defined as follows:

  • Recommendation — explains what security-relevant action to take
  • Rationale — explains why the action should be taken
  • How-To — details how to carry out the action
  • Compliance Check — a Boolean check used to determine compliance with a desired state. Ideally, the compliance check is expressed in a standardized format such as OVAL to ensure the guidance is easily consumed by a broad audience.

Example of a Rule:

Recommendation: Change the default encryption key during the installation of the Tivoli Identity Manager
Rationale: The encryption key is used to encrypt TIM passwords and other sensitive data. The default encryption key is [sunshine]. If the default encryption key is not changed a malicious user could potentially use the default key to gain access to TIM passwords and sensitive data. Changing the default encryption key will help protect Tivoli passwords and sensitive data.
  1. Log in to Central Administration.
  2. Navigate to Application Management » SharePoint Web Application Management.
  3. Select Create or extend Web application.
  4. Select Create a new Web application if creating a new application, or Extend an existing Web application if extending an existing application.
    1. If extending an existing Web application, select the appropriate Web application.
  5. Navigate to Security Configuration » Use Secure Sockets Layer (SSL).
  6. Select the option [Yes].
  7. Enter other options with values appropriate to the deployment.
  8. Select OK.
Compliance Check: oval:org.mitre.oval:def:5275



Rules are categorized as they are developed and help analysts to understand the level of effort required to add compliance checks to a benchmark, provide a rough estimate of how much of a guide can be covered with compliance checks, and helps indicate which checking system to use. Rules fall into three categories:

  • Check Category — applied to a Rule that indicates a tool can run a test and determine a true or false result indicating that the system is in compliance or not
  • Question Category — applied to a Rule that indicates a question must be asked of a user to determine compliance
  • Report Category — applied to a Rule that indicates a report is needed for further analysis to determine compliance



Once the Rule is created, the guidance must be mapped to existing configuration controls. Common control standards include NIST Special Publication 800-53: Recommended Security Controls for Federal Information Systems and Common Configuration Enumeration (CCE™). Control stands are used because they refer to widely-recognized regulatory frameworks, cite precise configuration controls, facilitate clear communication, and provide pointers to further information.

800-53 Example:

ID: AC-9
CONTROL: The information system notifies the user, upon successful logon, of the date and time of the last logon, and the number of unsuccessful logon attempts since the last successful logon.

CCE Example:

ID: CCE-2891-0
DESCRIPTION: The "Disable CTRL+ALT+Delete Requirement for Logon" policy should be set correctly.
PARAMETER: Enabled / Disabled
  1. HKEY_LOCAL_MACHINE\Software\Microsoft\


  2. defined by Local or Group Policy


Compliance Test (Automated Benchmarks only)

Compliance tests are Boolean checks used to determine compliance with the desired state defined by the Rule. There are currently two standards for compliance tests that ensure the guidance can be easily consumed by a broad audience, Open Checklist Interactive Language (OCIL) for gathering structured information from end users in order to accurately perform the compliance testing, and Open Vulnerability and Assessment Language (OVAL®) for system configuration checking and reporting the results.

Example of an OVAL compliance test: oval:org.mitre.oval:def:5275.


Benchmark Development Process

There are five phases in the benchmark development process.

  • Phase 1 — Writing Good Guidance
  • Phase 2 — Augmenting Guidance
  • Phase 3 — Automating Assessment
  • Phase 4 — Benchmark Structuring and Tailoring
  • Phase 5 — Managing Compliance (An ongoing process)

For instruction on the five phases, take our Free Online Course. Additional information on the five phases is available in the course downloads.


Additional Information

For additional information please see About Benchmark Development, How to Write a Good Benchmark, and Benchmark Development Resources.


Page Last Updated: April 15, 2011