Web Application Firewall and CRS

2020-02-10 (152) by Ton Kersten, tagged as sysadm security

During my stay at CfgMgmtCamp I attended the presentation of Franziska Bühler (@bufrasch) titled “Web Application Firewall - Friend of your DevOps pipeline?”. She talked about Web Application Firewalls (WAF) and the Core Rule Set (CRS) for owasp

Being into security and stuff like that myself, I decided I wanted to try to get the web application with ModSecurity up and running in my own test environment.

My test environment consists of a CentOS8 machine with NGINX and it turned out to be a little trickier than I thought.

The ModSecurity modules are standard available for the Apache webserver, so I could have used that. But I like a good challenge, so CentOS8 and NGINX it is.

Searching the web I found a couple of resources that helped me along, not completely, but it got me going.

Enable the PowerTools repository

As all configuration takes place on CentOS8, all development tools need to be made available, so the PowerTools repository needs to be enabled.

dnf config-manager --set-enabled PowerTools

And a lot of development tools need to be present

dnf -y install      \
    autoconf        \
    automake        \
    GeoIP-devel     \
    bison           \
    bison-devel     \
    curl            \
    curl-devel      \
    doxygen         \
    flex            \
    gcc             \
    gcc-c++         \
    git             \
    libcurl-devel   \
    libxml2-devel   \
    lmdb-devel      \
    lua-devel       \
    openssl-devel   \
    ssdeep-devel    \
    yajl            \
    yajl-devel      \
    zlib-devel

Get all sources

Now that all software is in place, setup an environment to work in. Be aware, as this is a testing envіronment no sudo is used, as everyting is done as root.

cd
mkdir owasp
cd owasp

and download all sources. GeoIP2 s needed, because GeoIP will soon be deprecated, thus download it from the link given and install with NGINX

wget https://nginx.org/download/nginx-1.17.8.tar.gz
wget https://github.com/SpiderLabs/ModSecurity/releases/download/v3.0.4/modsecurity-v3.0.4.tar.gz
wget https://github.com/SpiderLabs/owasp-modsecurity-crs/archive/v3.2.0.tar.gz -O CRS_v3.2.0.tar.gz
git clone --recursive https://github.com/maxmind/libmaxminddb
git clone             https://github.com/SpiderLabs/ModSecurity
git clone             https://github.com/SpiderLabs/ModSecurity-nginx
git clone             https://github.com/leev/ngx_http_geoip2_module

MaxMin library

The ModSecurity library and the ModSecurity module both need libmaxmin, so that is first

cd libmaxminddb
./bootstrap
./configure
make
make check
make install
cd ..

Configure and install modsecurity library

And after the libmaxmin the modsecurity library

tar -xvf modsecurity-v3.0.4.tar.gz
cd modsecurity-v3.0.4
./configure --with-lmdb --with-maxmind=/usr/local
make
make install
cd ..

Configure and install modsecurity module

Now that the modsecurity library is available, create the module

cd ModSecurity
sh build.sh
git submodule init
git submodule update
./configure --with-lmdb --with-maxmind=/usr/local
make
make install
cd ..

The tricky bit

For NGINX to accept the modules and load them, they have to be compiled with exactly the same configure options as the installed NGXINX. These can be determined with the nginx -V command.

But somehow I could not get this working. I tried all options I could find, but I kept running into binairy incompatibility errors. So I decided to compile NGINX from scratch as well. This has, of course the disadvantage that NGINX cannot be upgraded through the packagemanager anymore, but was already rendered impossible with the strict match between the modules and the nginx binary. But I wanted to make sure the self-build nginx stuff would not interfere with the rest of the system, so I place everything in /usr/local/nginx. As a starting point I took the configuration option the installed NGINX gave me and ended up with:

tar -xvf nginx-1.17.8.tar.gz
cd nginx-1.17.8
./configure                                                     \
    --prefix=/usr/local/nginx                                   \
    --sbin-path=/usr/local/nginx/sbin/nginx                     \
    --modules-path=/usr/local/nginx/modules                     \
    --conf-path=/usr/local/nginx/etc/nginx.conf                 \
    --error-log-path=/var/log/nginx/error.log                   \
    --http-log-path=/var/log/nginx/access.log                   \
    --pid-path=/var/run/nginx.pid                               \
    --lock-path=/var/run/nginx.lock                             \
    --http-client-body-temp-path=/var/cache/nginx/client_temp   \
    --http-proxy-temp-path=/var/cache/nginx/proxy_temp          \
    --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp      \
    --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp          \
    --http-scgi-temp-path=/var/cache/nginx/scgi_temp            \
    --user=nginx                                                \
    --group=nginx                                               \
    --with-file-aio                                             \
    --with-threads                                              \
    --with-http_addition_module                                 \
    --with-http_auth_request_module                             \
    --with-http_dav_module                                      \
    --with-http_flv_module                                      \
    --with-http_gunzip_module                                   \
    --with-http_gzip_static_module                              \
    --with-http_mp4_module                                      \
    --with-http_random_index_module                             \
    --with-http_realip_module                                   \
    --with-http_secure_link_module                              \
    --with-http_slice_module                                    \
    --with-http_ssl_module                                      \
    --with-http_stub_status_module                              \
    --with-http_sub_module                                      \
    --with-mail                                                 \
    --with-mail_ssl_module                                      \
    --with-stream                                               \
    --with-stream_ssl_module                                    \
    --with-compat                                               \
    --with-cc-opt='-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -DNGX_HTTP_HEADERS' \
    --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--as-needed'i \
    --add-dynamic-module=../ModSecurity-nginx                   \
    --add-dynamic-module=../ngx_http_geoip2_module
make
make install
cd ..
useradd -m -c'nginx' nginx
mkdir -p /var/cache/nginx/client_temp
chown nginx:nginx /var/cache/nginx/client_temp

Configure ModSecurity

Once nginx is compiled and installed, on to modsec The creator of ModSec (SpiderLabs) has a default configuration available for download, so let’s start with that.

mkdir -p /usr/local/nginx/etc/modsec
wget
https://raw.githubusercontent.com/SpiderLabs/ModSecurity/v3/master/modsecurity.conf-recommended \
    -O /usr/local/nginx/etc/modsec/modsecurity.conf
cp -p /root/owasp/modsecurity-v3.0.4/unicode.mapping /usr/local/nginx/etc/modsec/unicode.mapping

sed -i 's/^SecRuleEngine.*/SecRuleEngine On/' /usr/local/nginx/etc/modsec/modsecurity.conf
cat <<- '@EOF' > /usr/local/nginx/etc/modsec/main.conf
    Include "/usr/local/nginx/etc/modsec/modsecurity.conf"

    # Basic test rule
    SecRule ARGS:blogtest "@contains test" "id:1111,deny,status:403"
    SecRule REQUEST_URI "@beginsWith /admin"
    "phase:2,t:lowercase,id:2222,deny,msg:'block admin'"
@EOF

And configure nginx to use the ModSec module.

worker_processes  1;
load_module modules/ngx_http_modsecurity_module.so;
load_module modules/ngx_http_geoip2_module.so;
load_module modules/ngx_stream_geoip2_module.so;
events {
    worker_connections  1024;
}
http {
    include            mime.types;
    default_type       application/octet-stream;
    sendfile           on;
    keepalive_timeout  65;
    server {
        listen         80;
        server_name    localhost;
        modsecurity    on;
        modsecurity_rules_file /usr/local/nginx/etc/modsec/main.conf;
        location / {
            root   html;
            index  index.html index.htm;
        }
        error_page  500 502 503 504 /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

To verify it’s not all completely broken, run /usr/local/nginx/sbin/nginx -t and ensure everything is in working order. Check if ModЅecurity is working with:

curl http://localhost/adminaccess
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.17.8</center>
</body>
</html>

And the /var/log/nginx/error.log file shows:

2020/02/07 16:04:08 [error] 17871#17871: *3 [client 127.0.0.1]
ModSecurity: Access denied with code 403 (phase 2). Matched "Operator
`BeginsWith' with parameter `/admin' against variable `REQUEST_URI'
(Value: `/adminaccess' ) [file "/usr/local/nginx/etc/modsec/main.conf"]
[line "7"] [id "2222"] [rev ""] [msg "block admin"] [data ""] [severity
"0"] [ver ""] [maturity "0"] [accuracy "0"] [hostname "127.0.0.1"] [uri
"/adminaccess"] [unique_id "158108784890.091254"] [ref
"o0,6v4,12t:lowercase"], client: 127.0.0.1, server: localhost, request:
"GET /adminaccess HTTP/1.1", host: "localhost"

Almost there

At this point ModSecurity is running with NGINX, so all that’s needed is the Core Rule Set. Ans that is a breeze once you get this far.

cd /usr/local/nginx/etc
tar -xvf ~/owasp/CRS_v3.2.0.tar.gz
ln -s owasp-modsecurity-crs-3.2.0 owasp-crs
cp -p /usr/local/nginx/etc/owasp-crs/crs-setup.conf.example /usr/local/nginx/etc/owasp-crs/crs-setup.conf

And add these lines to: /usr/local/nginx/etc/modsec/main.conf

Include "/usr/local/nginx/etc/owasp-crs/crs-setup.conf"
Include "/usr/local/nginx/etc/owasp-crs/rules/*.conf"

Also make sure that the file /usr/local/nginx/etc/owasp-crs/crs-setup.conf contains the lines

SecDefaultAction "phase:1,log,auditlog,deny,status:403"
SecDefaultAction "phase:2,log,auditlog,deny,status:403"

If a normal curl is issued, e.g. curl http://localhost/trololo_singer.html this does not trigger any security rules and a plain 404 error is diplayed:

<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.17.8</center>
</body>
</html>

But, if a curl command is requesting a protected file, like the .htaccess file, the Core Rule Set is triggered and issues an access denied error.

<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.17.8</center>
</body>
</html>

And the /var/log/nginx/error.log file shows:

2020/02/07 16:17:28 [error] 2724#2724: *8 [client 127.0.0.1]
ModSecurity: Access denied with code 403 (phase 2). Matched "Operator
`PmFromFile' with parameter `restricted-files.data' against variable
`REQUEST_FILENAME' (Value: `/.htaccess' ) [file
"/usr/local/nginx/etc/owasp-crs/rules/REQUEST-930-APPLICATION-ATTACK-LFI.conf"]
[line "104"] [id "930130"] [rev ""] [msg "Restricted File Access
Attempt"] [data "Matched Data: .htaccess found within REQUEST_FILENAME:
/.htaccess"] [severity "2"] [ver "OWASP_CRS/3.2.0"] [maturity "0"]
[accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag
"platform-multi"] [tag "attack-lfi"] [tag "OWASP_CRS"] [tag
"OWASP_CRS/WEB_ATTACK/FILE_INJECTION"] [tag "WASCTC/WASC-33"] [tag
"OWASP_TOP_10/A4"] [tag "PCI/6.5.4"] [hostname "127.0.0.1"] [uri
"/.htaccess"] [unique_id "15813292242.837003"] [ref
"o1,9v4,10t:utf8toUnicode,t:urlDecodeUni,t:normalizePathWin,t:lowercase"],
client: 127.0.0.1, server: localhost, request: "GET /.htaccess
HTTP/1.1", host: "localhost"

Which surely is a message that the request was blocked by the Core Rule Set.

Ton Kersten