How to Build and Run Compliance Assessments

Overview:

Why a pass rate is only useful if you trust how it was measured

A compliance dashboard that shows a single percentage is easy to produce and easy to misread. The number means nothing without knowing what was actually tested, how each check decided pass or fail, and what happens to a check that could not be evaluated at all. A 16% pass rate and a 96% pass rate are not more or less trustworthy than each other unless the rules behind them are inspectable.

The operational problem is usually not the framework, it is the rule underneath it. A CIS or GDPR benchmark is just a named bundle of individual checks, and each check is only as good as its audit logic and its remediation script. Frameworks built entirely from vague or untested rules produce dashboards that look authoritative and mean very little.

The Compliance module treats this as three separate problems solved in order: author and test individual rules, bundle tested rules into a framework mapped to a real standard, then deploy that framework to real endpoints and read back a per-rule, per-endpoint result rather than a single blended score.

This guide walks the full workflow end to end: authoring a compliance rule with audit and remediation logic, using the built-in AI command generator to draft that logic, live-testing a rule before it ships, building a framework from your own rules or the built-in benchmark library, deploying that framework to endpoints, and reading the resulting evaluation, including what an INVALID result means and how it differs from a genuine failure.

• Measure what actually matters: rules are written against your own environment and standards, not assumed from a generic score.
• Prove compliance per framework: CIS, GDPR, HIPAA, PCI DSS, ISO, RBI and SOC 2 mappings are built in, or define your own.
• Catch drift automatically: audit logic re-evaluates on schedule, so a control that silently reverted is caught before an auditor finds it.
• Remediate, not just report: every rule can carry a remediation script, so a failed check can be corrected automatically instead of only logged.
• Trust the number: a pass rate is broken down rule by rule, endpoint by endpoint, so nobody has to take the top-line percentage on faith.

Prerequisites

Confirm all four before authoring your first rule. Skipping straight to a framework built on untested rules is the most common cause of a dashboard full of INVALID results.

1. Administrator access: your account holds a role permitted to manage the Compliance module. Read-only operators can view assessment results but cannot create rules, frameworks, or deployments.
2. Agent installed and reporting: the endpoint agent is installed, registered, and reporting healthy on every target device. A rule cannot evaluate on a device the agent cannot reach.
3. Target OS identified: rules and frameworks are written and bound per operating system (Windows Server editions, Windows 10/11, Ubuntu, RHEL/CentOS). Know which OS versions you're actually assessing before building rules for one you don't run.
4. A defined standard, or none: know whether this assessment needs to map to a named regulatory or benchmark standard (CIS, GDPR, HIPAA, PCI DSS, ISO, RBI, SOC 2) or is a custom internal check. This decides whether Step 5 uses the built-in library or a custom framework.

Architecture: how a rule becomes a compliance result

A rule is the smallest unit of truth in the system. A framework is a named collection of rules mapped to a standard. A deployment binds a framework to endpoints and triggers evaluation. The result is reported per rule, per endpoint, never collapsed into a single number until you choose to view it that way.

Step 1: Open the Compliance module

Get oriented before creating anything, so you know where rules, frameworks, and deployments each live.

Instructions

1. From the main console, go to Endpoints → Compliance.
2. Note the three sections in the left sidebar: Compliance Rule, Compliance Framework, Compliance Deployment.
3. Open Compliance Rule to see the existing rule library: each row shows Name, Bindings (which OS/group it applies to), Type, and Tags.

Path: Console / Endpoints / Compliance / Compliance Rule

Result: You're in the Compliance module with visibility into the existing rule library before authoring anything new.

Step 2: Create a compliance rule

Author a single check: what it audits for, and what it does to remediate a failure.

Instructions

1. Go to Compliance Rule → Create Compliance Rule.
2. Enter a Rule Name that states the check plainly, e.g. Ensure windows firewall is enabled, and toggle Status on.
3. Set Execution Type to Command.
4. Set Bindings to the target OS/group, e.g. W2016MS.
5. Set Rule Type to Security and Impact to Medium, High, or Critical depending on the risk of the check failing.
6. Add Tags to make the rule searchable later.
7. Write the Description in plain language: what the rule checks and why.
8. Fill in Audit (what to check) and Remediation (what to do if the check fails) as separate statements, e.g. Audit: Verify the firewall status on all network profiles; Remediation: Enable Windows Defender Firewall on all profiles, if it is disabled.

Path: Console / Compliance / Compliance Rule / Create Compliance Rule

Console: rule details

Result: A draft rule exists with its audit and remediation intent stated in plain language. The actual executable logic is added next, in Rule Execution Type.

Step 3: Generate the rule's command using AI (optional)

Use the built-in AI command generator to draft the actual audit/remediation script instead of writing it from scratch.

Instructions

1. Within the rule's Rule Execution Type section, set the type to Command, then select Generate Command Using AI.
2. Select the target Platform (e.g. Windows).
3. In Command Description, describe the script you need in plain language, e.g. enable Windows Defender Firewall on all profiles if disabled.
4. Review the generated Result script before accepting it. Treat AI-generated command output the same as any other unreviewed script: read it fully before it runs on a production endpoint.
5. Under Execution Rules, confirm the generated Command Type (e.g. cmd), and note the Editable flag: change the script if needed before saving.

Path: Console / Compliance / Compliance Rule / Create Compliance Rule / Generate Command Using AI

Result: A first-draft script is populated into the rule's execution rule, editable before it's attached to any endpoint.

Step 4: Live-test the rule before publishing

Confirm the rule's logic actually behaves as intended on a real endpoint before it becomes part of a framework.

Instructions

1. Under Execution Rules, confirm the rule (e.g. RULE1) has its command and command type set.
2. Optionally select Add Output Condition to define what output counts as a pass versus a fail, or Add Rule to chain additional execution logic.
3. Scroll to Live test.
4. Select a Test Endpoint and run the rule against it to preview the actual result before the rule is saved into general use.
5. Confirm the output matches what you expect for both a compliant and a non-compliant device, if you can test both states.

Path: Console / Compliance / Compliance Rule / Create Compliance Rule / Live test

Result: You've confirmed the rule's real output on a live endpoint rather than trusting the script's logic on paper alone.

Step 5: Create a compliance framework

Bundle individual rules into a named collection mapped to the standard you need to report against.

Instructions

1. Go to Compliance Framework → Create Compliance Framework.
2. Enter a Bundle Name, e.g. Windows firewall security baseline, and optionally upload an Icon File.
3. Set Compliance Type to the standard this framework maps to: CIS, HIPAA, PCI, FISMA, ISO, RBI, or SOC-2.
4. Write a Description stating what the framework covers and its intended effect, e.g. Firewall is enabled on all Windows devices. It audits the firewall status and automatically enables it if it is found to be disabled.
5. Under Compliance Rules, search the rule library and move the rules you want into this framework. Selected rules move from the left (unselected) list into the right (selected) list, showing a running item count.

Path: Console / Compliance / Compliance Framework / Create Compliance Framework

Console: framework definition

Result: A framework exists bundling one or more tested rules, tagged to the standard it satisfies.

Step 6: Use the built-in benchmark library

Skip authoring your own rules entirely for common standards by using the pre-built framework library.

Instructions

1. Go to Compliance Framework and browse the existing list rather than creating a new one.
2. Filter or search by OS and standard. Built-in frameworks cover, among others: CIS benchmarks for Windows 10/11, Windows Server 2012 through 2025, and Ubuntu 14.04 through newer LTS releases, plus GDPR technical control mappings per OS.
3. Open a framework to review its description and confirm it matches the OS and standard you need before deploying it as-is.

Path: Console / Compliance / Compliance Framework

Console: sample built-in frameworks

Result: A ready-to-deploy framework mapped to a recognized standard, with no custom rule-authoring required.

Step 7: Create a compliance deployment

Bind a framework to real endpoints and trigger evaluation.

Instructions

1. Go to Compliance Deployment → Create Compliance Deployment.
2. Enter a Deployment Name describing the assessment, e.g. GDPR win 11.
3. Set Scope Endpoints to Specific Endpoints (or broader scope) and select the target device(s).
4. Under What to deploy, set Select Compliance Framework to the framework from Step 5 or 6, e.g. GDPR Technical Controls for Windows 11 Enterprise.
5. Set Deployment Policy to an existing delivery policy, e.g. OOB Instant deployment policy (type: Instant).
6. Optionally set Notify to for completion alerts.
7. Select Publish (or Save as Draft).

Path: Console / Compliance / Compliance Deployment / Create Compliance Deployment

Console: deployment definition

Result: A task is created and the deployment appears in Compliance Deployment with Stage initially showing Initiated.

Step 8: Monitor the deployment task

Track the deployment from initiation through completion before reading results.

Instructions

1. Go to Compliance Deployment to see all deployments with their Stage (Initiated, Completed), Progress, and Policy.
2. Open the specific deployment task, e.g. ADR-127: GDPR win 11, to see its framework name and description.
3. Review the per-endpoint summary: Evaluation Status, Tested, Passed, Failed, Invalid counts.
4. Wait for Evaluation Status to move from in-progress to a completed state (e.g. SUCCESS) before treating the results as final.

Path: Console / Compliance / Compliance Deployment

Console: task summary

Result: The deployment reports a completed evaluation with rule counts broken down by outcome, ready for detailed review.

Step 9: Review the evaluation results

Drill into rule-by-rule results rather than reporting only the top-line pass rate.

Instructions

1. Open Evaluated Rules for the completed deployment.
2. Read the summary: pass rate percentage, and counts of Passed, Failed, Invalid.
3. Review the rule list, each showing its name (often mapped to an article or control number, e.g. Art.32: SMB signing required) and its Status.
4. Distinguish Failed (the audit ran and the check did not pass) from Invalid (the audit could not produce a determinate result, e.g. an unsupported command, a missing dependency, or a binding mismatch).
5. Use Search by rule name, expected or actual output and the status filter to isolate a subset for investigation.

Path: Console / Compliance / Compliance Deployment / <Task> / Evaluated Rules

Result: A rule-level breakdown showing exactly what passed, what failed, and what couldn't be determined, the actual evidence behind the summary percentage.

Step 10: Remediate and re-evaluate

Close the loop: fix what the rules found, then confirm the fix with another evaluation rather than assuming it worked.

Instructions

1. For each Failed rule, review its remediation script (defined back in Step 2) and confirm it's appropriate to run automatically, or run it manually if the rule has no attached remediation.
2. For each Invalid rule, go back to Compliance Rule, open the rule, and re-test it live against the same endpoint (Step 4) to isolate whether the script, the binding, or the endpoint is the cause.
3. Once corrected, re-publish the deployment or create a new deployment against the same scope and framework to re-evaluate.
4. Compare the new pass rate against the previous run. A rising pass rate driven by falling Invalid counts confirms the rules themselves are now trustworthy; a rising pass rate driven by falling Failed counts confirms the endpoints are actually more compliant.

Path: Console / Compliance / Compliance Deployment / Create Compliance Deployment (re-run)

Result: A second evaluation that reflects genuine remediation rather than a single unexamined snapshot.

Best practices for compliance assessments

Troubleshooting: common issues and resolutions

Frequently asked questions