Brief Notes on WASD Test and Exercise
-------------------------------------

Many tools exist at the time of writing that didn't twenty years before when
WASD was first being developed.  Some are on-line, "free" site health checks
and penetration testing.  Others are tools that can (often) be used from your
platform of choice, many of which are free and open-source (FOSS).  We are
spoiled for choice.

In WASD's earlier years tools such as Apache Bench, WASD Bench, along with
batched cURL and wget requests were used to exercise and, in some limited
fashion, fuzz the server (providing invalid, unexpected, or random request
data) in an effort to discover flaws in server code and execution.  These
home-built tools included capabilities to "fuzz" server connections, by
having the client breaking network connection at various stages during request
processing.

With the increased use of TLS/SSL and the advent of HTTP/2 the above toolsets'
usefulness began to wane.  In recent years, WASD's exercising, testing and load
testing has employed two main tools (from many others in the marketplace).

  1.  OWASD Zed Attack Proxy (ZAP)
      https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project

  2.  h2load
      https://nghttp2.org/documentation/h2load-howto.html
      https://nghttp2.org

Using OWASP ZAP to to test and exercise WASD HTTP/1.n
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  "Zed Attack Proxy (ZAP) is a free, open-source penetration testing tool
   being maintained under the umbrella of the Open Web Application Security
   Project (OWASP). ZAP is designed specifically for testing web applications
   and is both flexible and extensible. 

   ZAP provides functionality for a range of skill levels from developers,
   to testers new to security testing, to security testing specialists.
   ZAP has versions for each major OS and Docker, so you are not tied to
   a single OS.  Additional functionality is freely available from a variety
   of add-ons in the ZAP Marketplace, accessible from within the ZAP client."

   https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project

ZAP is used to exercise the in-development WASD, in particular the following
aspects (not in any particular order).

  o  Traffic Loading ...
     server behaviour under load; continuing to process correctly while not
     exhibiting bottlenecks in performance, or worse, failing with soft
     (internal assertion checking) or hard (e.g. ACCVIO) bugchecks
  o  Graded Alerts ...
     reports and counts of known attack vectors or general recommendations
     after spidering or penetration scans
  o  Directory Traversal ...
     aims to access files and directories that are stored outside the server
     root, web root or web application folders
  o  Data Injection ...
     covers a variety of attacks where request parameters are used to execute
     (CLI) commands, SQL queries, interpreted script code (e.g. JavaScript,
     PHP), or platform-executable binary code
  o  Buffer Overflow ...
     overwriting of memory fragments of the process, which should never be
     modified intentionally or unintentionally (see Request Fuzzing)
  o  Request Fuzzing ...
     where malformed or spurious data is automatically generated and injected
     into the processing in an effort to induce unexpected behaviour or failure
  o  Cross Site Scripting ...
     where a malicious web element such as JavaScript, HTML, or other
     browser-side code is injected into otherwise benign and trusted web
     content from a non-same-origin, third-party source 

It should be noted that these are provided "out-of-the-box", is a subset of
that out-of-the-box functionality of particular interest in WASD development,
and utilise only a tiny percentage of ZAP total capabilities.

ZAP default behaviour is to crawl the site and then fuzz the discovered URIs.

Using h2load to test and exercise WASD HTTP/2
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
h2load is benchmarking tool for HTTP/2 and HTTP/1.1.  It supports SSL/TLS and
clear text for all supported protocols.  The nghttp/h2load package is available
for many platforms.  I use it at the macOS bash command-line.

WASD testing under stress
~~~~~~~~~~~~~~~~~~~~~~~~~
Server stress is not only represented by sheer load but also suboptimal network
and client behaviour, as well as malicious actors.  The behavioural and
malicious aspects of stress are adequately handled by the ZAP application. 
However, server behaviour under network misbehaviour and normal client actions
(e.g. disconnection at any stage during request processing) does not seem to be
handled by either ZAP or h2load.

So to test these conditions the WASD server has an in-built "network
misbehavor" capability.  See NetTestBreak() and NetTestSupervisor() in the
NET.C module.  When built using the WATCH_MOD=1 parameter, the presence of the
logical name WASD_NET_TEST_BREAK activates code that breaks network connections
once a minute.  The integers in the logical name value represents the
proportion of current connections to be broken.  If 5 then evey fifth
connection, if 3 every third, and if 0 or 1 then every network connection.  The
succession of integers represents the succession of minutes, so in the example
the first two minutes sever every fifth, the third minute every fourth, the
fourth minute every third, and so on.  When the integers are exhausted the
sequence restarts.

  $ DEFINE /SYSTEM WASD_NET_TEST_BREAK "5 5 4 3 2 2 1"

The h2load utility is not a crawler, it must be supplied with a URL or URLs to
load against.  Using the bash shell the following command will provide a
continuous load.  The ./wasd_h2load_urls if a plain text file containing the
URLs to be used.

  $ while sleep 1; do h2load -n100000 -c10 -t6 -m8 -i./wasd_h2load_urls; done