maester-test-expert
GitHub用于创建、编辑、验证和文档化 Maester 安全测试,涵盖 Microsoft 365 租户评估、属性配置及调试。
触发场景
安装
npx skills add maester365/maester --skill maester-test-expert -g -y
SKILL.md
Frontmatter
{
"name": "maester-test-expert",
"description": "Write, validate, and document Maester security checks for Microsoft 365 tenants. Use when asked to create, edit, review, or debug a Maester check (a native test: Test.<ID>.ps1 with a [MaesterTest] attribute plus Test.<ID>.md), its attributes, or a user's custom test, including converting Pester-format tests. Covers the [MaesterTest] attribute, choosing Product, Feature, Area, ResourceType, Severity, UserImpact, ImplementationCost, Mitre and Control (CIS, CISA and NIST SP 800-53 mappings) with the attribute rubric and the taxonomy, Service\/License gates, test parameters, families, Graph API data retrieval, Add-MtTestResultDetail formatting, the MT, CIS, CISA, EIDSCA and ORCA suites, remediation guidance, Entra ID, Exchange, SharePoint, Teams, Defender, Conditional Access, and the validation checklist for new checks."
}
Maester Test Expert
Create, edit, validate, and document Maester security checks -- the unit of work that assesses a Microsoft 365 tenant's security posture.
From Maester 3.0 every check is a native test: two files, Test.<ID>.ps1 and Test.<ID>.md. There are no
*.Tests.ps1 wrappers, no separate helper function in powershell/public/, no maester-config.json row and no
FunctionsToExport entry. Maester's own engine discovers the test from its attribute, checks that the services and
licences it declares are available, runs it, catches errors and writes the result row.
The user-facing guide with every rule is website/docs/writing-tests/index.mdx. This skill is the contributor's
condensed version; if they disagree, the code and that page win. How to choose the value of each classification,
priority and ATT&CK attribute is in website/docs/writing-tests/attribute-rubric.md (the rubric); this skill
links to it and does not repeat it.
Agent Selection
Use maester-test-expert as the primary agent for all normal workflows.
Specialized planner and issue-manager agents are optional workflow components and should generally stay in the background to avoid user confusion.
When to Use This Skill
- Creating a new Maester security check
- Editing or refactoring an existing check
- Adding or correcting a check's attributes: product, feature, area, resource type, severity, user impact, implementation cost, ATT&CK techniques, service or licence declarations
- Writing or updating a check's
.md(description, remediation, result template) - Reviewing a check for correctness, attributes that fit the rubric, or documentation completeness
- Debugging a failing, erroring or skipped check
- Writing or converting a user's custom test (
custom/folder)
Do NOT use this skill for:
- Editing EIDSCA tests by hand --
tests/eidsca/Test.EIDSCA.*are generated bybuild/eidsca/Update-EidscaTests.ps1from templates inbuild/eidsca/templates/. Change the generator or its data. - Editing ORCA tests by hand --
tests/orca/Test.ORCA.*(header "Generated by ./build/orca/Update-OrcaTests.ps1") andpowershell/internal/generated/orca/check-ORCA*.ps1are generated. Change the generator. - Editing
website/docs/tests/-- test pages are generated from the test files bywebsite/scripts/generate-test-docs.mjs. - Unit tests of the module itself (
powershell/tests/).
Prerequisites
- PowerShell 7.4 or later.
- A source checkout:
Import-Module ./powershell/Maester.psd1loads the module and defines every built-in test from./tests. No build and no .NET SDK are needed unless you changesrc/Maester.Engine. - A working
Connect-Maestersession to run a check against a tenant. - Review the Maester contribution guidelines before submitting changes.
Anatomy of a Check
| File | Location | Purpose |
|---|---|---|
Test.<ID>.ps1 |
tests/<suite>/[<service>/] |
One function with a [MaesterTest(...)] attribute. Reads the tenant, decides, calls Add-MtTestResultDetail, returns $true/$false. |
Test.<ID>.md |
Same folder, same base name | Description, #### Remediation action, optional #### Related links, then <!--- Results ---> and %TestResult%. Shown in the report. |
suite.json |
Each suite folder (already exists) | The suite's ID, name, Source and help URL template. Do not edit for a single check. |
frameworks/<key>.json |
Beside the suite.json of the CIS and CISA suites |
The catalogue of the benchmark: its name and version, and one entry per requirement with its level, profiles or criticality. It does not list tests. A CIS or CISA check names its requirement in Control; add the requirement here in the same pull request if it is new, or the build fails. |
Suite folders: tests/maester/<service>/ (MT.xxxx; services such as entra, exchange, teams, defender,
azure, intune, and xspm for exposure management, which has its own suite.json), tests/cisa/<service>/
(CISA.MS.*), tests/cis/ (CIS.M365.*), tests/ad/<area>/ (AD-*), and the generated tests/eidsca/ and
tests/orca/. Every folder name is lowercase kebab-case; see the repository layout in
the contributing guide.
The function keeps a descriptive name (Test-MtCaMfaForRiskySignIn). Function names must be unique across
powershell/internal, powershell/public and every test file, because built-in tests share the module scope.
Code shared by several checks goes in powershell/internal/checks/<suite>/<service>/ as an internal helper;
reusable data access for a service goes in powershell/internal/services/<service>/.
The Test File
Template
Scaffold it, then edit:
New-MtTest -Id MT.1234 -Title 'At least one Conditional Access policy blocks device code flow.' `
-Product entra -Feature conditional-access -Area authentication -ResourceType Tenant `
-Severity High -UserImpact Low -ImplementationCost Low -Mitre T1078.004, T1528, T1566 `
-Service Graph -Path ./tests/maester/entra
Only -Id and -Title are required, so you can scaffold first and set the other values once the .md is
written (see Choosing the attribute values). New-MtTest does not write
Author; add it.
function Test-MtCaBlockDeviceCodeFlow {
<#
.SYNOPSIS
Checks if at least one enabled Conditional Access policy blocks the device code flow.
.DESCRIPTION
{Detailed description of the control and what the check evaluates.}
#>
[MaesterTest(
Id = 'MT.1234',
Title = 'At least one Conditional Access policy blocks device code flow.',
Product = 'entra',
Feature = 'conditional-access',
Area = 'authentication',
ResourceType = 'Tenant',
Severity = 'High',
UserImpact = 'Low',
ImplementationCost = 'Low',
Mitre = ('T1078.004', 'T1528', 'T1566'),
Control = ('cisa-scuba:MS.AAD.3.1', 'nist-800-53-r5:IA-2'),
Service = 'Graph',
License = 'AAD_PREMIUM',
Author = 'your-github-handle'
)]
[CmdletBinding()]
[OutputType([bool])]
param()
$policies = Get-MtConditionalAccessPolicy | Where-Object { $_.state -eq 'enabled' }
$blocking = @($policies | Where-Object {
$_.conditions.authenticationFlows.transferMethods -match 'deviceCodeFlow' -and
$_.grantControls.builtInControls -contains 'block'
})
if ($blocking.Count -gt 0) {
$result = "Well done. These policies block the device code flow:`n`n%TestResult%"
Add-MtTestResultDetail -Result $result -GraphObjects $blocking -GraphObjectType ConditionalAccess
return $true
}
Add-MtTestResultDetail -Result 'No enabled Conditional Access policy blocks the device code flow.'
return $false
}
Note what is not there: no Test-MtConnection guard, no Get-MtLicenseInformation check, no outer
try/catch, no -Description, no -Severity, no -TestTitle, and no Tag.
File rules (enforced by Read-MtNativeTest; violations are InvalidMetadata)
- File name is exactly
Test.<Id>.ps1; the.mdsits beside it. - Exactly one function with
[MaesterTest(...)]. Other functions (private helpers, a family's instance source) may live in the same file. - Top level contains only function definitions: no statements, no script
param()block. - The test function has
[CmdletBinding()].
The [MaesterTest] attribute
Arguments are named and constant: quoted strings, $true/$false, a parenthesised list ('a', 'b'), a single
string for a list property, or a bare flag (LongRunning). No variables, expressions, @() or hashtables. The
schema lives in powershell/assets/MaesterTestSchema.psd1. How to choose the values from Product to Mitre is
in Choosing the attribute values.
| Property | Type | Values | Required for built-ins | Notes |
|---|---|---|---|---|
Id |
string | ^(?=.*[0-9.\-])[A-Za-z][A-Za-z0-9]*([.\-][A-Za-z0-9_]+)*$, max 64 |
Yes | Never change an existing ID. |
Title |
string | One line | Yes | The only title. |
Product |
string | A product key from the taxonomy: entra, exchange, defender, ... |
Yes | Write the key, not an alias such as Entra ID. Groups the console progress and the run summary. |
RelatedProduct |
string[] | Product keys | No | Never the same key as Product. -Product matches it too. |
Feature |
string | A feature key of the test's product: conditional-access, ... |
Yes | |
Area |
string | An area key: authentication, privileged-access, ... |
Yes | |
ResourceType |
string | A resource type key: Tenant, User, ConditionalAccessPolicy, ... |
Yes | |
Severity |
string | Critical, High, Medium, Low, Info |
Yes | A TestSettings row in a user's config overrides it. |
UserImpact |
string | Low, Medium, High |
Yes, unless Severity is Info |
Leave it out of an Info check. |
ImplementationCost |
string | Low, Medium, High |
Yes, unless Severity is Info |
Leave it out of an Info check. |
Mitre |
string[] | ATT&CK technique IDs (T1110, T1110.003), or the single value None |
Yes | At most five. Never a technique beside one of its own sub-techniques. |
Control |
string[] | <framework-key>:<requirement-id> |
CIS and CISA checks | The requirements the check implements or supports. A CIS check sets Control = 'cis-m365:5.1.5.1', a CISA check Control = 'cisa-scuba:MS.AAD.3.1'; any other check may name the same requirements. The first one that a framework file knows becomes the Benchmark of the result. For a shipped framework the requirement must exist in its file. |
Service |
string[] | Graph, Azure, ExchangeOnline, SecurityCompliance, SharePointOnline, Teams, Dataverse, AzureDevOps, GitHub, ActiveDirectory, or None |
Yes | All listed must be connected. Registry: powershell/assets/MaesterServiceRegistry.psd1. |
License |
string[] | Tokens from powershell/assets/MaesterLicenseTable.psd1; A&B = both |
No | Tenant needs any one element. |
Permission |
string[] | One permission per element, no commas | No | The permissions the check needs, for example Policy.Read.All. Recorded; not enforced in 3.0. |
TenantType |
string[] | Workforce, External |
No | Default Workforce. Recorded; enforced only when a run turns it on. |
Cloud |
string[] | Commercial, GCC, GCCHigh, DoD, China, Bleu, Delos, GovSG |
No | Default all. Recorded; enforced only when a run turns it on. |
Platform |
string[] | Windows, Linux, MacOS |
No | Declare Windows for Windows-only commands (lint checks common ones). |
Lifecycle |
string | Preview, Stable, Deprecated |
No | Default Stable. Use Preview for a check that depends on preview APIs or is still being validated; it is not run by default. A Deprecated check still runs. |
ReplacedBy |
string | The ID of an existing test | No | Only with Lifecycle = 'Deprecated'. |
LongRunning |
bool | No | Slow in large tenants. Not run by default. | |
InstanceSource |
string | Function in the same file | No | Makes the check a family (below). |
Exclusive |
bool | No | Must not run concurrently (for example it writes to the tenant). | |
Tag |
string[] | No commas | No | Built-in checks do not use it. See Tags. |
Author |
string[] | GitHub handles | Yes | The initial author(s). |
Contributor |
string[] | GitHub handles | No | Add yourself when you substantially change a check. |
HelpUrl |
string | https://... |
No | Defaults to https://maester.dev/docs/tests/<Id>. |
Category and Preview were removed in 3.0. A test that still has one is InvalidMetadata, with the replacement
in the message: Product and Feature instead of Category, Lifecycle = 'Preview' instead of Preview.
Reserved names (rejected today): OptionalService, TimeoutSeconds.
Licence tokens: AAD_PREMIUM (Entra ID P1 or better), AAD_PREMIUM_P2, Entra_Identity_Governance,
AAD_WRKLDID_P1, AAD_WRKLDID_P2, ATP_ENTERPRISE (MDO P1 or P2), THREAT_INTELLIGENCE (MDO P2),
EOP_ENTERPRISE, EXCHANGE_S_STANDARD, EXCHANGE_DLP, M365_ADVANCED_AUDITING, DEFENDER_XDR,
LOCKBOX_ENTERPRISE, INTUNE_A. Built-in checks use only these; add a token to the table if you need a new one.
Return values and outcomes
| The check does | Result | ReasonCode |
|---|---|---|
return $true / return $false |
Passed / Failed | |
Add-MtTestResultDetail -Investigate |
Investigate (whatever it returns) | |
Add-MtTestResultDetail -SkippedBecause <code> |
Skipped; the call ends the test | TestSkipped, or NotApplicable |
throws, or -SkippedBecause Error |
Error | TestError |
returns $null without skipping |
Skipped | NoResult |
| returns a non-boolean or several values | Error | InvalidReturn |
- Return exactly one boolean. Uncaptured output (
$list.Add($x), a stray expression) becomes a second return value and anInvalidReturnerror; send it to$nullorOut-Null. - The engine owns the
try: any terminating or statement-terminating error ends the check asErrorwith the message. Atry/catchis fine for a specific condition (for example a 403 reported as-SkippedBecause NotAuthorized), not as a blanket wrapper. A skip raised inside your owntrystill skips. - Use
-SkippedBecause NotApplicablewhen the tenant state makes the check meaningless (for example no policies of that kind exist). Never return$nullto mean "not applicable". - Never use
NotConnected*orNotLicensed*skip codes in a built-in check; declareService/License. - Checks that need delegated permissions skip with
-SkippedBecause NotSupportedAppPermissionwhen(Get-MgContext).AuthType -ne 'Delegated'.
Parameters
Tunable values are function parameters, never hard-coded and never read from wrapper arguments.
param(
# Longest validity period, in days, that a certificate may be issued for.
[ValidateRange(1, 3650)]
[int] $MaximumValidityDays = 365,
# Members of these groups are exempt from this check.
[MaesterParameter(Kind = 'Entra.Group')]
[string[]] $ExcludedGroups
)
- Types:
int,bool,switch,string,string[]. Constant defaults. Nothing mandatory: the check must run with no config. - Every parameter has a one-line
#comment directly above it (only the nearest comment line is read), or.PARAMETERhelp. The unit tests require a description. - Thresholds get
ValidateRange; choices getValidateSet. [MaesterParameter(Kind = ...)]for IDs:Entra.User,Entra.Group,Entra.ServicePrincipal,Entra.Application,Entra.ConditionalAccessPolicy,Entra.NamedLocation,Entra.Domain,Entra.DirectoryRole,Azure.Subscription,Azure.ResourceGroup,Azure.Resource(powershell/assets/MaesterParameterKinds.psd1).Instanceand names starting withMtare engine-owned.- Users set values in
maester-config.jsonunderTestSettings[].Parameters, or withInvoke-MtTest -Parameter.
Families (one check, several results)
When a check produces one result per item, declare InstanceSource = '<function in the same file>'. The engine
calls that function at run time (never at discovery) and runs the check once per returned object, passed as
$Instance (declare param($Instance) with a description comment).
Each instance object has Id (required, unique, ^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$; the row ID becomes
<Id>.<suffix>) and optional Title, Severity, Tag, Data. See tests/maester/drift/Test.MT.1060.ps1 and
tests/maester/entra/Test.MT.1024.ps1.
Settings
Read tenant-wide settings with Get-MtSetting -Name <name> (for example EmergencyAccessAccounts). New settings
Maester itself reads go in powershell/assets/MaesterSettings.psd1 with a type, default and description.
Data Retrieval
Always prefer Invoke-MtGraphRequest over Invoke-MgGraphRequest or Get-Mg* cmdlets. It provides:
- Built-in caching -- reduces API calls when multiple checks query the same endpoint.
- Automatic pagination -- no need for
-All. - ConsistencyLevel header -- included by default for read-only calls.
- Batching -- pass an array of IDs via
-UniqueId. - Named parameters --
Select,Filter,QueryParameters.
$policies = Invoke-MtGraphRequest -RelativeUri 'policies/authorizationPolicy' -ApiVersion v1.0
$users = Invoke-MtGraphRequest -RelativeUri 'users' -Filter "userType eq 'Member'" -Select id, displayName
$users = Invoke-MtGraphRequest -RelativeUri 'users' -UniqueId $userIds -Select id, displayName, onPremisesSyncEnabled
$policySplat = @{
ApiVersion = 'beta'
RelativeUri = 'policies/roleManagementPolicyAssignments'
Filter = "scopeId eq '/' and scopeType eq 'DirectoryRole' and roleDefinitionId eq '$($role.id)'"
QueryParameters = @{ expand = 'policy(expand=rules)' }
}
$policy = Invoke-MtGraphRequest @policySplat
Other helpers: Get-MtConditionalAccessPolicy, Get-MtUser, Get-MtRole, Get-MtRoleMember, Get-MtGroupMember,
Get-MtExo, Get-MtSpo, Invoke-MtAzureRequest, Invoke-MtAzureResourceGraphRequest. Caches are reached only
through these helpers; never assign $script:, $global: or $__MtSession state in a check (lint).
New Graph permission scopes go in Get-MtGraphScope.
Result Formatting
# Result text; the .md supplies the description
Add-MtTestResultDetail -Result $resultMarkdown
# Objects rendered as a table with admin portal deep links; -Result must contain %TestResult%
Add-MtTestResultDetail -Result "Found issues:`n`n%TestResult%" -GraphObjects $failing -GraphObjectType ConditionalAccess
# Needs a human to review
Add-MtTestResultDetail -Result $result -Investigate
# Custom skip
Add-MtTestResultDetail -SkippedBecause Custom -SkippedCustomReason 'All alerts have been suppressed.'
-GraphObjectType: Users, Groups, Devices, ConditionalAccess, AuthenticationMethod,
AuthorizationPolicy, ConsentPolicy, Domains, IdentityProtection, UserRole.
Output messages: a pass message starts with "Well done." and describes the positive outcome; a fail message says what is wrong and what to change; tables have header and separator rows; link to the admin portal where possible.
The Markdown File (Test.<ID>.md)
No front matter. Template:
{One-line summary of what the check verifies.}
{Rationale: explain WHY this configuration matters for security.}
#### Remediation action
1. In **Entra ID**, navigate to **{Section}** > **[{Page Name}]({direct admin portal URL})**.
2. {Step-by-step instructions to fix the issue.}
3. Click **Save**.
#### Related links
* [{Admin portal link description}]({URL})
* [{Benchmark or baseline reference}]({URL})
<!--- Results --->
%TestResult%
- Exactly one
<!--- Results --->followed by%TestResult%; everything above it is the description. - Do not pass
-DescriptiontoAdd-MtTestResultDetail, so the.mdis used. - The
.mdis also shown for rows that did not run (not selected, service not connected), so it must make sense on its own. - The website page
website/docs/tests/<suite>/<ID>.mdis generated from this file and the attribute.
Choosing the Attribute Values
A built-in check says what it is about, how urgent a failure is and which attack techniques it addresses. Each of those values is chosen with the attribute rubric. Read the rubric's section for an attribute before you set, change or review its value, and do not take the values of a neighbouring check without checking them against it.
| Attribute | The question it answers |
|---|---|
Product |
In which product is the fix made? |
RelatedProduct |
Under which other products would an admin expect to find the check? Optional. |
Feature |
Which part of the product, a settings page or an object type, is the check about? |
Area |
Which security theme is it, whatever the product? |
ResourceType |
What does a failing result point at: the tenant, or a kind of object it lists? |
Severity |
How much risk does the organization carry while the check fails? |
UserImpact |
How much does the recommended fix disrupt end users? |
ImplementationCost |
How much administrator work does the recommended fix take? |
Mitre |
Which MITRE ATT&CK techniques does the checked control prevent or detect? |
Control |
Which requirements of CIS, CISA SCuBA or NIST SP 800-53 does a passing result support? |
The rubric rates a check by the fix its own .md recommends, so write the .md before you choose the values.
The values of the generated EIDSCA and ORCA checks are in build/eidsca/eidsca-test-metadata.json and
build/orca/orca-test-metadata.json: change a value there and regenerate.
Framework requirements and NIST controls (Control)
Fill in Control on every new check. Do not leave it out because it is optional: decide, and leave it empty
only when nothing applies. It is the only way a check is mapped to a framework. Work through these in order:
- The requirement the check is written for. A CIS check lists
cis-m365:<id>orcis-github:<id>and a CISA check listscisa-scuba:<id>, first. If the requirement is not in the suite's framework file yet, add it there with its facts (level and profiles, or criticality) in the same pull request. - A CIS or CISA requirement that asks for the same setting. A Maester check that verifies what
cisa-scuba:MS.AAD.3.1asks for lists it. Look the requirement up intests/cis/frameworks/*.jsonandtests/cisa/frameworks/cisa-scuba.json, or in the lookup of the test builder.Get-MtTest -Suite CISA | Format-Table Id, Title, Controlshows what the existing checks map to. - NIST SP 800-53 Rev. 5 controls, written
nist-800-53-r5:<control>, chosen with the NIST part of the rubric's Control section from the pinned listpowershell/assets/MaesterNistControls.json. At most three, best fit first, and only when a passing result is direct evidence for the control. Skip this step when step 1 or 2 gave acisa-scubareference: CISA publishes the NIST mapping of its policies and the result gets it from the framework file.
The first value that names a requirement in a framework file becomes the Benchmark of the result, so the order
matters. An inventory count, or a health, licensing or tidiness check, usually lists nothing.
Where the keys come from
Product,Feature,AreaandResourceTypetake keys frompowershell/assets/MaesterTaxonomy.psd1: the products with their features, the areas and the resource types. A feature key belongs to one product.Productfollows the folder: a check intests/maester/entraortests/cisa/entraisentra. The folder map is inpowershell/tests/general/NativeTestLint.Tests.ps1; a new service folder needs an entry there.Mitretakes technique IDs from the pinned ATT&CK release,powershell/assets/MaesterAttackTechniques.json. An ID that the release retired or does not know fails the unit tests. Tactics are looked up from the ID and never typed.Controltakes requirement IDs from the framework files (tests/<suite>/frameworks/<key>.json) and NIST control IDs frompowershell/assets/MaesterNistControls.json. The unit tests fail on an ID that is in neither.Get-MtTest -Product entra -Feature conditional-access | Format-Table Id, Title, Severity, Area, ResourceTypelists the existing checks of a feature, to compare your values with.- A new key is added to the taxonomy in the same pull request as the first test that uses it. Prefer an existing key. Do not add a key ahead of its test: the unit tests fail on a product, feature, area or resource type that no test uses. A new area also needs a row in the rubric's area table, which the unit tests compare with the taxonomy.
Tags
Tag holds free labels for the owner of a test folder. It is not how Maester classifies a check.
- Built-in checks carry no tag for product, feature, area, suite, benchmark, licence, severity or their own ID. Each of those has an attribute or is derived, and the unit tests fail a built-in check that tags one. Remove such a tag when you edit an older check.
- Do not put
PrevieworLongRunninginTag; useLifecycle = 'Preview'and theLongRunningflag. - Users select checks with
-Product,-Feature,-Area,-Severityand-SuiteonInvoke-MaesterandGet-MtTest. The tags of Maester 2.x, such asCAorCIS, still select the checks that carried them, with a warning, throughpowershell/assets/Maester.LegacyTags.json. That file is generated and never edited, and a check added after 3.0 is not in it, so do not add a tag to make a new check selectable. - A user's custom test may tag freely.
How the values are checked
The unit tests in powershell/tests/general/NativeTestAttributes.Tests.ps1 check what is mechanical. The
attribute-review check on the pull request has a reviewer model judge each new or changed value against the
rubric. The person or agent who reviews the pull request applies the same rubric. See
How the rubric is enforced.
When the attribute review fails, change the value, or explain in the pull request why the declared value fits the rubric. When you review a check yourself, follow "Reviewing a test's attributes" in AGENTS.md; it also shows how to run the attribute review before you push.
Step-by-Step Workflow: Creating a New Check
- Reserve an ID. Check issue #697 and comment to reserve
the next
MT.xxxx(the maester-issue-manager agent can do this).Get-MtTest MT.12*shows which IDs exist. - Scaffold with
New-MtTest -Id <ID> -Title '...' -Service <service> -Product <product key> -Path ./tests/<suite>/<service>. - Fill in the attribute:
License,Author, flags as needed. - Write the function: read with
Invoke-MtGraphRequestor the helpers, decide,Add-MtTestResultDetail, return a boolean. Add parameters for tunable values. - Write the guide, the
.md, in full: a one-line summary, why the setting matters, numbered remediation steps with a direct link to the admin portal page, related links (Microsoft Learn, and the benchmark when the check maps to one), and the results footer. Replace every placeholder thatNew-MtTestwrote; a check with scaffold text in its guide is not finished. Verify product facts and links with Microsoft Learn. - Choose the attribute values with the rubric:
Feature,Area,ResourceType,Severity,UserImpact,ImplementationCost,Mitre,Control, andRelatedProductwhen one applies. See Choosing the attribute values and Framework requirements and NIST controls. The test builder shows the allowed values and the rubric for each attribute and writes the attribute; use it to check a value you are unsure of. - Validate offline:
Import-Module ./powershell/Maester.psd1 -Force; Get-MtTest <ID>must showIsValid = Trueand noErrors. Name the ID and leave out-Path:Get-MtTest -Path <file>applies the rules of a custom test, so it does not report a missing attribute or a key that is not in the taxonomy. - Run against a tenant:
Connect-Maester; Invoke-MtTest -Path <file>(orInvoke-MtTest -Id <ID>). Try-Parameter @{...}for each parameter. Check the row'sResult,ReasonCode,ResultDetail.TestResult. - Run the unit tests:
./powershell/tests/pester.ps1.powershell/tests/general/NativeTestLint.Tests.ps1andpowershell/tests/general/NativeTestAttributes.Tests.ps1check every built-in test.
Converting a User's Pester-format Test
Users' 2.x custom tests (*.Tests.ps1) still run when Pester 5.7.1+ is installed. To convert:
Convert-MtTest -Path ./custom -WhatIf
Convert-MtTest -Path ./custom | Format-Table Id, Status, Notes
It writes Test.<ID>.ps1 + .md per It, turns a Test-MtConnection guard into Service, removes a blanket
-SkippedBecause Error catch, and rewrites a trailing | Should -Be $true to return. The Describe name
becomes Feature, so the row's Block stays what it was. The Pester tags are kept in Tag, except that a
Preview tag becomes Lifecycle = 'Preview' and a LongRunning tag the LongRunning flag. Rows marked
NeedsReview have TODO comments; tests with names built at run time are Skipped and must be rewritten as
families. Once converted, delete the Pester file: the native test runs and the Pester test with the same ID no
longer does.
A user's custom native test lives in their custom/ folder, uses their own ID prefix (not MT., CISA., CIS.,
EIDSCA., ORCA., AD-, AZDO., MT1060., or MP., which is reserved for Maester Packs), needs a .md, and needs only Id and Title in its attribute.
It may set Product, Feature, Area and ResourceType to values of its own (a value that matches the taxonomy
is stored as the key), may tag freely and may use Control; the rubric applies to it only if its owner wants it
to. It can call only Maester's exported commands; to reuse a built-in check it calls Invoke-MtTest -Id <ID>.
Validation Checklist
Test logic
- Returns exactly one
$true/$false; no stray pipeline output. - No
Test-MtConnectionguard, noGet-MtLicenseInformationcheck;ServiceandLicensedeclared instead. - No blanket
try { } catch { Add-MtTestResultDetail -SkippedBecause Error ... }. -
-SkippedBecause NotApplicable(not$null) when the check does not apply. - Data retrieval uses
Invoke-MtGraphRequestor Maester helpers. - No hard-coded GUIDs, tenant IDs or names; tunables are described parameters.
- No writes to
$script:,$global:,$env:or$__MtSession. -
Platform = 'Windows'if it uses Windows-only commands.
Attribute and files
-
Test.<Id>.ps1andTest.<Id>.mdin the right suite folder; ID unique and not reused. -
Id,Title,Product,Feature,Area,ResourceType,Severity,Mitre,Service,Authorset; values constant. -
UserImpactandImplementationCostset, or both left out whenSeverityisInfo. -
Product,Feature,AreaandResourceTypeare keys frompowershell/assets/MaesterTaxonomy.psd1; a new key is added there in this pull request. - Each of those values, and
RelatedProductif set, checked against its section of the attribute rubric, with the reason visible in the description or the remediation text. - No
CategoryorPreview; noTagfor product, feature, area, suite, benchmark, licence, severity or the ID.Lifecycle = 'Preview'and theLongRunningflag where they apply. - A CIS or CISA check sets
Controlto its requirement (cis-m365:<id>,cis-github:<id>orcisa-scuba:<id>), first, and the requirement is in the suite's framework file. -
Controlwas considered for every check: a CIS or CISA requirement that asks for the same setting, and NIST SP 800-53 controls frompowershell/assets/MaesterNistControls.json(none when acisa-scubareference already brings them). Left empty only when nothing applies. -
[CmdletBinding()]and[OutputType([bool])]; comment-based help inside the function. -
Get-MtTest <ID>(without-Path) reports no errors.
Documentation
-
.mdhas summary, rationale,#### Remediation actionwith portal links, and ends with<!--- Results --->+%TestResult%. - No scaffold placeholder is left in the
.md("Describe what this test checks", "Explain how to fix"), and its links open the page they name. - Pass message starts with "Well done."; fail message is actionable.
Code style
- One True Brace Style; Pascal Case; full cmdlet names (no aliases); 4-space indentation.
- UTF-8 with BOM for
.ps1files.
Pre-pull request
-
./powershell/tests/pester.ps1passes (PSScriptAnalyzer included). - Ran against a tenant with
Invoke-MtTest. - Commit messages follow Conventional Commits.
Common Mistakes
| Mistake | Why it matters | Fix |
|---|---|---|
Adding a Test-MtConnection guard or licence check |
Duplicates the engine's gate; lint fails | Declare Service / License |
Wrapping the body in try/catch that reports -SkippedBecause Error |
Lint fails; the engine already does it | Remove it; catch only specific conditions |
Returning $null for "not applicable" |
Reported as Skipped/NoResult |
Add-MtTestResultDetail -SkippedBecause NotApplicable |
Uncaptured output ($list.Add(...)) |
Error/InvalidReturn |
$null = $list.Add(...) |
Variables or @() in the attribute |
InvalidMetadata |
Constants and ('a', 'b') lists only |
Category or Preview in the attribute |
InvalidMetadata; both were removed in 3.0 |
Product and Feature; Lifecycle = 'Preview' |
A Tag such as 'CA', 'Entra' or 'Maester' on a built-in check |
Unit tests fail; tags are not the taxonomy | Use the attribute that says it; see Tags |
A product alias ('Entra ID') or a key that is not in the taxonomy |
Unit tests fail | Use the key from powershell/assets/MaesterTaxonomy.psd1, or add the key in the same pull request |
Copying Severity, Area or Mitre from a similar check |
The value may not fit this check, and the attribute review fails a value that does not fit | Choose each value with the rubric |
Validating a built-in check with Get-MtTest -Path only |
It applies the rules of a custom test and reports no missing attribute | Get-MtTest <ID> |
File name does not match Id |
InvalidMetadata |
Rename to Test.<Id>.ps1 |
Missing [CmdletBinding()] |
InvalidMetadata |
Add it before param() |
| Hard-coded thresholds or object IDs | Users cannot tune the check | Make them described parameters |
Multi-line # comment above a parameter |
Only the last line is read as its description | One-line comment or .PARAMETER |
Invoke-MgGraphRequest / Get-Mg* |
Bypasses caching | Invoke-MtGraphRequest |
Using -Severity or -TestTitle on Add-MtTestResultDetail |
Ignored for single-result checks | Set them in the attribute |
Editing generated EIDSCA/ORCA files or website/docs/tests/ |
Overwritten | Change the generator or the test files |
Adding the function to FunctionsToExport |
Check functions are internal in 3.0 | Don't |
Missing %TestResult% in -Result with -GraphObjects |
The object table is not inserted | Include the placeholder |
Reference Examples
tests/maester/entra/Test.MT.1012.ps1+.md-- simple Graph check with a licence gate.tests/maester/entra/Test.MT.1198.ps1-- check with aValidateRangeparameter,LongRunning.tests/maester/drift/Test.MT.1060.ps1-- family withInstanceSourceand$Instance,Service = 'None'.tests/cisa/entra/Test.CISA.MS.AAD.3.5.ps1-- CISA check.tests/cisa/exchange/Test.CISA.MS.EXO.14.1.ps1-- check with aRelatedProduct.tests/maester/azure-devops/Test.AZDO.1045.ps1--Infocheck withMitre = 'None'andLifecycle = 'Preview'.tests/eidsca/Test.EIDSCA.AF01.ps1-- generated thin check comparing a tenant value.tests/orca/Test.ORCA.100.ps1-- generated ORCA check.website/docs/writing-tests/index.mdx-- the full native test guide.website/docs/writing-tests/attribute-rubric.md-- how to choose each attribute value.powershell/assets/MaesterTaxonomy.psd1-- the product, feature, area and resource type keys.website/docs/configuration/applicability.md-- gate order and reason codes.
Troubleshooting
| Symptom | Likely cause | Resolution |
|---|---|---|
Error / InvalidMetadata |
Attribute or file rule broken | Get-MtTest <ID> lists each problem with its line (Get-MtTest -Path <file> for a custom test) |
| Attribute review fails on the pull request | A value does not fit the rubric | Change it to the expected value, or explain in the pull request why the declared value fits |
Error / InvalidReturn |
Non-boolean or multiple outputs | Capture stray output; return one boolean |
Error / InvalidConfiguration |
Bad parameter value in config or -Parameter |
Check Get-MtTest <ID> parameters; fix the value |
Error / TestError |
Exception in the body (often a permission) | Read ReasonDetail / ErrorRecord |
Skipped / ServiceNotConnected |
Declared service not connected | Connect-Maester -Service <service> |
Skipped / LicenseNotFound |
Tenant lacks every License element |
Expected; check the token |
Skipped / NoResult |
Returned $null |
Return a boolean or skip explicitly |
NotRun / LongRunning or Preview |
Excluded by default | -IncludeLongRunning / -IncludePreview, or Invoke-MtTest -Id |
NotRun / OptInServiceNotConnected |
Active Directory check without AD connection | Connect-Maester -Service ActiveDirectory |
| Report shows no description | .md missing, misnamed, or -Description passed |
Name it Test.<ID>.md; drop -Description |
%TestResult% appears literally |
Placeholder missing from -Result with -GraphObjects |
Add it |
| 403 from Graph | Missing scope | Add it to Get-MtGraphScope |
版本历史
-
bd9ab9d
当前 2026-10-20 00:56
新增关于 Control 框架映射、Maester Packs 保留前缀警告及实施裁决的详细文档说明。
- 6bf1d98 2026-08-20 14:03


