Using the CRS Sandbox
Info
The CRS Sandbox is a shared public resource. Please review the CRS Sandbox Security Policy before use.
Introducing the CRS Sandbox
We have set up a public CRS Sandbox which you can use to send attacks at the CRS. You can choose between various WAF engines and CRS versions. The sandbox parses audit logs and returns our detections in an easy and useful format.
The sandbox is useful for:
- integrators and administrators: you can test out our response in case of an urgent security event, such as the Log4j vulnerability;
- exploit developers/researchers: if you have devised a payload, you can test beforehand if it will be blocked by the CRS and by which versions;
- CRS developers/rule writers: you can quickly check if the CRS catches a (variant of an) exploit without the hassle of setting up your own CRS instance.
Quick reference
Header: x-backend
| Value | Engine |
|---|---|
apache (default) | Apache 2 + ModSecurity 2.9 |
nginx | Nginx + ModSecurity 3 |
coraza | Coraza WAF on Caddy |
Header: x-crs-version
| Value | Description |
|---|---|
v4-lts (default) | Newest release of the CRS 4 long-term support line (currently 4.25.x) |
v3-lts | Newest release of the CRS 3.3 line (currently 3.3.x); apache and nginx only |
latest | Newest CRS release (currently 4.30.x) |
The values are aliases, not version numbers. The X-CRS-Version response header tells you the exact tag that handled your request.
Header: x-crs-paranoia-level (blocking and detection paranoia level)
| Value | Description |
|---|---|
1 (default) | Least strict |
2 | Moderate |
3 | Strict |
4 | Most strict |
Header: x-crs-detection-paranoia-level (detection paranoia level only, 1 to 4; must not be lower than the paranoia level)
Header: x-crs-mode
| Value | Description |
|---|---|
| (omitted) | Blocking mode |
detection | Detection only (SecRuleEngine DetectionOnly): rules match, nothing is blocked |
Header: x-crs-inbound-anomaly-score-threshold (default: 5)
Header: x-crs-outbound-anomaly-score-threshold (default: 4)
Header: x-format-output
| Value | Description |
|---|---|
| (omitted) | Full audit log as JSON (browsers get an HTML report instead) |
txt-matched-rules | Human-readable matched rules, one per line |
txt-matched-rules-extended | Extended text with explanation |
json-matched-rules | JSON list of {id, message, paranoia_level} |
json-report | JSON report: engine, CRS tag, HTTP status, blocked, anomaly scores and the matched rules with their matched data |
csv-matched-rules | CSV formatted |
html-matched-rules | HTML table |
Apache with text output and the CRS 4 LTS line:
Nginx with JSON output and CRS 3.3:
Coraza with extended text output and the newest CRS release:
Basic usage
The sandbox is located at https://sandbox.coreruleset.org/.
An easy way to use the sandbox is to send requests to it with curl, although you can use any HTTPS client. The sandbox answers CORS preflights, so a web page can call it from the browser as well.
The sandbox has many options, which you can change by adding HTTP headers to your request. One is very important so we will explain it first; this is the X-Format-Output: txt-matched-rules header. If you add this header to your request, the sandbox will parse the WAF’s output, and return to you the matched CRS rule IDs with descriptions, and the score for your request.
Example
In this example, we sent ?file=/etc/passwd as a GET payload. The CRS should catch the string /etc/passwd which is on our blocklist. Try out the command in a terminal now if you like!
You can send anything you want at the sandbox, for instance, you can send HTTP headers, POST data, use various HTTP methods, et cetera. Behind every WAF sits a small HTTP mirror (go-httpbin) that accepts any path and method, so /get, /post?x=1 or /anything/you/like all work. The response body is always the WAF’s verdict in the format you asked for, not the mirror’s answer.
If no attack is detected, the sandbox returns an empty result in the requested format (e.g., an empty JSON array [] for json-matched-rules, or an empty response for txt-matched-rules).
The response status code is the one the WAF produced: a 403 means the request was blocked, a 200 means it went through. In blocking mode a blocked request always lists rule 949110 (inbound anomaly score exceeded).
The sandbox also adds a few headers to the response. X-Unique-ID contains a unique value that you can use to refer to your request when communicating with us. x-backend names the service that answered and X-CRS-Version the exact CRS tag it runs. With curl -i you can see the returned headers.
Example showing the response headers
Discovering what is available
GET https://sandbox.coreruleset.org/backends returns the engines, the CRS version aliases each one offers, the output formats and the control headers as JSON. It is the authoritative list; the tables on this page are a snapshot.
Default options
It’s useful to know that you can tweak the sandbox in various ways. If you don’t send any X- headers, the sandbox will use the following defaults.
- The default backend is Apache 2 with ModSecurity 2.9.
- The default CRS version is
v4-lts, the newest release of the CRS 4 long-term support line. Uselatestfor the newest release. - The default Paranoia Level is 1, which is the least strict setting.
- By default, the response is the full audit log from the WAF, which is verbose and includes unnecessary information, hence why
X-Format-Output: txt-matched-rulesis useful. A browser (a client whoseAcceptheader liststext/html) gets an HTML report instead.
Changing options
Let’s say you want to try your payload on different WAF engines or CRS versions, or like the output in a different format for automated usage. You can do this by adding the following HTTP headers to your request:
x-crs-version: will pick another CRS version. Available values are the aliasesv4-lts(default),v3-ltsandlatest; see the quick reference above for what each one currently resolves to. Version numbers such as4.25.0are not accepted.x-crs-paranoia-level: will run CRS in a given paranoia level. Available values are1(default),2,3,4.x-crs-detection-paranoia-level: sets the detection paranoia level on its own, so rules from higher levels are reported without affecting the blocking decision. Available values are1to4; it must not be lower thanx-crs-paranoia-level.x-crs-mode: can be changed to return the http status code from the backend WAF. Default value is blocking (On), and can be changed usingdetection(will set engine toDetectionOnly). Values are case insensitive.x-crs-inbound-anomaly-score-threshold: defines the inbound anomaly score threshold. Valid values are any integer > 0, with5being the CRS default. ⚠️ Anything different than a positive integer will be taken as 0, so it will be ignored. This only makes sense ifblockingmode is enabled (the default now).x-crs-outbound-anomaly-score-threshold: defines the outbound anomaly score threshold. Valid values are any integer > 0, with4being the CRS default. ⚠️ Anything different than a positive integer will be taken as 0, so it will be ignored. This only makes sense ifblockingmode is enabled (the default now).x-backendallows you to select the specific backend web server:apache(default) will send the request to Apache 2 + ModSecurity 2.9.nginxwill send the request to Nginx + ModSecurity 3.corazawill send the request to Coraza WAF on Caddy.
x-format-outputformats the response to your use-case (human or automation). Available values are:- omitted/default: the WAF’s audit log is returned unmodified as JSON (HTML report for browsers)
txt-matched-rules: human-readable list of CRS rule matches, one rule per linetxt-matched-rules-extended: same but with explanation for easy inclusion in publicationsjson-matched-rules: JSON list of CRS rule matches withid,messageandparanoia_level; this format is stable and will not grow new fieldsjson-report: JSON report with the engine, the CRS version alias and tag, the HTTP status, whether the request wasblocked, the anomaly scores and the matched rules including the matched data and severity; new fields are added herecsv-matched-rules: CSV formattedhtml-matched-rules: HTML page with a styled table of matched rules
Invalid x-format-output values default to json-matched-rules. An unsupported x-backend / x-crs-version combination (for instance coraza with v3-lts) is answered with a 400 and a JSON body listing the supported combinations.
The header names are case-insensitive.
Tip
If you work with JSON output (either unmodified or matched rules), jq is a useful tool to work with the output, for example you can add | jq . to get a pretty-printed JSON, or use jq to filter and modify the output.
Advanced examples
Let’s say you want to send a payload to the CRS 3.3 line and choose Nginx + ModSecurity 3 as a backend, because this is what you are interested in. You want to get the output in JSON because you want to process the results with a script. (For now, we use jq to pretty-print it.)
The command would look like:
If you want to know more than the rule ids, json-report adds the verdict and the data that matched:
You can also test the same payload across different WAF engines to compare detection behavior:
Let’s say you are working on a vulnerability publication and want to add a paragraph to explain how CRS protects (or doesn’t!) against your exploit. Then the txt-matched-rules-extended can be a useful format for you.
Capacity
- Please do not send more than 10 requests per second.
- We will try to scale in response to demand.
Data and privacy
All requests sent to the sandbox are logged and processed by the sandbox infrastructure. Do not send real personal data, credentials, API keys, or any sensitive information in your requests. By using the sandbox, you acknowledge that submitted request content may be retained and analyzed by the OWASP CRS Team for the purpose of improving CRS rules and sandbox infrastructure. See the CRS Sandbox Security Policy for full details.
Architecture
The sandbox consists of various parts. An NLB (Network Load Balancer) sits in front of the frontend and distributes incoming traffic. The frontend that receives the requests runs on OpenResty (Nginx with Lua). It handles the incoming request, selects the backend running CRS based on the request headers, proxies the request to the backend, and waits for the response. Then it parses the WAF audit log and sends the matched rules back in the format chosen by the user.
graph LR Client --> NLB[NLB] NLB --> OpenResty[OpenResty<br>Lua routing] OpenResty --> WAF[WAF Backend] WAF --> Mirror[Mirror Backend<br>go-httpbin] WAF --> CRS[ModSecurity /<br>Coraza CRS]
There is a backend service for every engine and version. The current backends are:
| Backend | Engine | Services |
|---|---|---|
| Apache + ModSecurity 2.9 | apache | apache-v4-lts, apache-v3-lts, apache-latest |
| Nginx + ModSecurity 3 | nginx | nginx-v4-lts, nginx-v3-lts, nginx-latest |
| Coraza WAF on Caddy | coraza | coraza-v4-lts, coraza-latest |
The backends write their JSON audit logs to a shared file system. OpenResty reads the per-transaction audit log file to extract matched rules and format the response. The logs are also copied to an S3 bucket and indexed in Elasticsearch for monitoring.
Known issues
In some cases, the sandbox will not properly handle and finish your request.
- Malformed HTTP requests: The frontend, OpenResty, is itself a HTTP server which performs parsing of the incoming request. The backend servers running CRS are regular webservers such as Apache and Nginx. Either one of these may reject a malformed HTTP request with an error 400 before it is even processed by CRS. This happens for instance when you try to send an Apache 2.4.50 attack that depended on a URL encoding violation. When the frontend is the one rejecting the request, the body is a JSON error whose
sourceissandbox-edge; a rejection by the backend web server comes back as that server’s own error page without anX-Unique-ID. Either way the request was not scanned by CRS. - ReDoS: If your request leads to a ReDoS and makes the backend spend too much time to process a regular expression, this leads to a timeout from the backend server. The frontend will cancel the request with an error 502. If you have to wait a long time and then receive an error 502, there was likely a ReDoS situation.
- Browsers rewrite URLs: when you call the sandbox from a web page, the browser resolves
..path segments, drops#fragmentsand percent-encodes most special characters before the request is sent, so the WAF may see something different from what you typed. For raw bytes usecurl(--path-as-iskeeps..segments).
Questions and suggestions
If you have any issues with the CRS sandbox, please open a GitHub issue at https://github.com/coreruleset/coreruleset/issues and we will help you as soon as possible. Include the X-Unique-ID of the request if you have it.
If you have suggestions for extra functionality, a GitHub issue is appreciated.