NOTE: SOME FUNCTIONALITY EMPLOYS JAVASCRIPT

WASD Features and Facilities

For version 12.2 release of WASD VMS Web Services.

Published January 2024

Document generated using wasDOC version 2.0.0

Abstract

This document describes the more significant features and facilities available with the WASD Web Services package.

For installation and update details see WASD Web Services - Installation

For detailed configuration information see WASD Web Services - Configuration

For information on CGI, CGIplus, ISAPI, OSU, etc., scripting, see WASD Web Services - Scripting

And for a description of WASD Web document, SSI and directory listing behaviours and options, WASD Web Services - Environment

Online Search

   

WASD VMS Web Services – Copyright © 1996-2024 Mark G. Daniel

Apache License, Version 2.0
License

Licensed under the Apache License, Version 2.0 (the "License");

you may not use this software except in compliance with the License. You may obtain a copy of the License at

https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Mark.Daniel@wasd.vsm.com.au
A pox on the houses of all spamers. Make that two poxes.

All copyright and trademarks within this document belong to their rightful owners. See 15. Attribution and Acknowledgement.

This is a static (file), single document.
Alternative multi-part static and dynamic documents.
Links followed by ⤤ open in a new page.

Table of Content

1.…………………Introduction
1.1…………………Troubleshooting?
2.…………………Package Overview
2.1…………………Server Behaviour
2.2…………………VMS Versions
2.3…………………TCP/IP Packages
2.4…………………International Features
3.…………………Authentication and Authorization
3.1…………………Rule Interpretation
3.2…………………Authentication Policy
3.3…………………Permissions, Path and User
3.4…………………Authorization Configuration File
3.5…………………Authentication Sources
3.6…………………Realm, Full-Access, Read-Only
3.7…………………Virtual Servers
3.8…………………Authorization Configuration Examples
3.8.1…………………KISS
3.9…………………Authorization Cache
3.10…………………SYSUAF-Authenticated Users
3.10.1…………………ACME
3.10.2…………………Logon Type
3.10.3…………………Rights Identifiers
3.10.4…………………WASD "Hard-Wired" Identifiers
3.10.5…………………VMS Account Proxying
3.10.6…………………Nil-Access VMS Accounts
3.10.7…………………SYSUAF and SSL
3.10.8…………………SYSUAF Security Profile
3.10.9…………………SYSUAF Profile For Full Site Access
3.11…………………Token Authentication
3.12…………………Skeleton-Key Authentication
3.13…………………Controlling Server Write Access
3.14…………………Securing All Requests
3.15…………………User Password Modification
3.16…………………Cancelling Authorization
4.…………………Transport Layer Security
4.1…………………Let's Encrypt
4.2…………………TLS/SSL Functionality Sources
4.3…………………WASD SSL Quick-Start
4.4…………………OPENSSL.EXE Application
4.5…………………SSL Configuration
4.5.1…………………WASD_CONFIG_SERVICE
4.5.2…………………TLS/SSL Versions
4.5.3…………………SSL Ciphers
4.5.4…………………(Open)SSL Options
4.5.5…………………Forward Secrecy
4.5.6…………………Session Resumption
4.5.7…………………Strict Transport Security
4.5.8…………………SSL Server Certificate
4.5.9…………………SSL Private Key
4.5.10…………………SSL Virtual Services
4.5.11…………………SSL Access Control
4.5.12…………………Authorization Using X.509 Certification
4.5.13…………………X.509 Certificate Renegotiation
4.5.14…………………Features
4.5.15…………………Subject Alternative Name and Other Extensions
4.5.16…………………X509 Configuration
4.5.17…………………Certificate Authority Verification File
4.5.18…………………X.509 Authorization CGI Variables
4.6…………………Certificate Management
4.6.1…………………Server Certificate
4.6.2…………………Certificate Signing Request
4.7…………………SSL CGI Variables
4.8…………………SSL Service Evaluation
4.9…………………SSL References
5.…………………HTTP/2
5.1…………………WASD HTTP/2
5.2…………………HTTP/2 and Performance
5.3…………………HTTP/2 Configuration
5.3.1…………………Global Configuration
5.3.2…………………Service Configuration
5.3.3…………………HTTP/2 Set Rules
5.4…………………HTTP/2 Detection
5.5…………………HTTP/2 References
6.…………………WebDAV
6.1…………………HTTP Methods Supported
6.1.1…………………COPY Restrictions
6.1.2…………………DELETE Restrictions
6.1.3…………………MOVE Restrictions
6.1.4…………………If: Restrictions
6.2…………………WebDAV Configuration
6.2.1…………………WebDAV Set Rules
6.2.2…………………File Naming
6.2.3…………………File-system Access
6.2.4…………………File-system Authorisation
6.2.5…………………Concurrent Authorisation
6.2.6…………………Real-World Example
6.3…………………WebDAV Metadata
6.4…………………WebDAV Locking
6.5…………………Some Wrinkles
6.5.1…………………OS X Finder
6.5.2…………………Gnome/gvfs/Nautilus
6.5.3…………………Dreamweaver
6.6…………………Microsoft Miscellanea
6.6.1…………………Mapping
6.6.2…………………FrontPage Extensions
6.6.3…………………Avoiding Microsoft Property Clutter
6.6.4…………………OPTIONS header "MS-Author-Via: DAV"
6.6.5…………………Repairing broken XP Web Folders
6.6.6…………………Adding a port number to the webfolder-address
6.6.7…………………Adding a number-sign ("#") to the webfolder-address
6.6.8…………………Force Windows XP to use Basic Authentication
6.6.9…………………Microsoft XP Explorer BASIC Authentication
6.6.10…………………Microsoft Windows 7 BASIC Authentication
6.6.11…………………Error 0x800700DF: The file size exceeds the limit allowed and cannot be saved
6.7…………………References
7.…………………Proxy Services
7.1…………………HTTP Proxy Serving
7.1.1…………………Enabling A Proxy Service
7.1.2…………………Proxy Affinity
7.1.3…………………Proxy Bind
7.1.4…………………Proxy Chaining
7.1.5…………………Controlling Proxy Serving
7.2…………………Proxy Cache
7.3…………………CONNECT Serving
7.3.1…………………Enabling CONNECT Serving
7.3.2…………………Controlling CONNECT Serving
7.4…………………SOCKS Version 5
7.5…………………FTP Proxy Serving
7.5.1…………………FTP Query String Keywords
7.5.2…………………"login" Keyword
7.6…………………Gatewaying Using Proxy
7.6.1…………………Reverse Proxy
7.6.2…………………Proxy Rework
7.6.3…………………One-Shot Proxy
7.6.4…………………DNS Wildcard Proxy
7.6.5…………………Originating SSL
7.7…………………Tunneling Using Proxy
7.7.1…………………[ServiceProxyTunnel] CONNECT
7.7.2…………………[ServiceProxyTunnel] RAW
7.7.3…………………[ServiceProxyTunnel] FIREWALL
7.7.4…………………Encrypted Tunnel
7.7.5…………………Encrypted Tunnel With Authentication
7.7.6…………………Shared SSH Tunnel
7.7.7…………………Complex Private Tunneling
7.7.8…………………Tunnelling Source
7.8…………………Browser Proxy Configuration
7.8.1…………………Manual
7.8.2…………………Automatic
8.…………………Instances and Environments
8.1…………………Server Instances
8.1.1…………………VMS Clustering Comparison
8.1.2…………………Considerations
8.1.3…………………Configuration
8.1.4…………………Status
8.2…………………Server Environments
9.…………………Server Administration
9.1…………………Access Before Configuration
9.2…………………Access Configuration
9.3…………………Server Instances
9.4…………………HTTPd Server Reports
9.5…………………HTTPd Server Revise
9.6…………………HTTPd Server Action
9.7…………………HTTPd Command Line
9.7.1…………………Accounting
9.7.2…………………Access Control
9.7.3…………………Alignment Faults
9.7.4…………………Authentication
9.7.5…………………Cache
9.7.6…………………Configuration Check
9.7.7…………………DCL/Scripting Processes
9.7.8…………………DECnet Scripting Connections
9.7.9…………………Hhelppp!
9.7.10…………………HTTP/2 Connection
9.7.11…………………Instances
9.7.12…………………Instance Status
9.7.13…………………Logging
9.7.14…………………Mapping
9.7.15…………………Network Connection
9.7.16…………………Shutdown and Restart
9.7.17…………………Secure Sockets Layer
9.7.18…………………Throttle
9.7.19…………………WebSocket
10.…………………WATCH Facility
10.1…………………Server Instances
10.2…………………Event Categories
10.3…………………Request Filtering
10.4…………………Report Format
10.5…………………WASD_WATCH_ONE_SHOT
10.6…………………Usage Suggestions
10.7…………………Command-Line Use
11.…………………Server Performance
11.1…………………Simple File Request Turn-Around
11.2…………………Scripting
12.…………………HTTPd Web Update
13.…………………Utilities and Facilities
13.1…………………Echo Facility
13.2…………………Hiss Facility
13.3…………………Stream Facility
13.4…………………Where Facility
13.5…………………Xray Facility
13.6…………………CALogs
13.7…………………CSPreport[er]
13.8…………………HTAdmin
13.9…………………HTTPd Monitor
13.10…………………MD5digest
13.11…………………QDLogStats
13.12…………………SECHAN Utility
13.13…………………StreamLF Utility
13.14…………………WAStee Utility
13.15…………………WOTSUP Utility
14.…………………Index
15.…………………Attribution and Acknowledgement


1.Introduction

1.1Troubleshooting?

With the installation, update and detailed configuration of the WASD Web Services package provided in WASD Web Services - Install and Config why have an introduction in this subsequent document? After getting the basics up and running (often the first thing we want to do) it's time to stop and consider the tool and what we're trying to accomplish with it. So this section provides an overview of the package's design philosophy, history and significant features and capabilities by topic.

The document assumes a basic understanding of Web technologies and uses terms without explaining them (e.g. HTTP, HTML, URL, CGI, SSI, etc.) The reader is refered to documents specifically on these topics.

Objectives

WASD Web Services originated from a 1993 decision by Wide Area Surveillance Division (WASD) management (then High Frequency Radar Division, HFRD) to make as much information as possible, both administrative and research, available online to a burgeoning personal desktop workstation and PC environment (to use the current term … an intranet) using the then emerging Web technologies.

It then became the objective of this author to make all of our systems' VMS-related resources available via HTTP and HTML, regardless of the underlying data or storage format. An examination of the WASD package will show that this objective is substantially achieved.

Reasons For Yet Another Web Package

Reasons for developing (remember; back in 1994!) a local HTTP server were few but compelling:

1.1Troubleshooting?

When initially installing or configuring WASD, and sometimes later where something breaks spectacularly, it is most useful to be able to gain insight into what the server is up to.

The go-to tool is  WATCH  (yes, all capitals, and for no other reason than it makes it stand out).

WATCH is described in detail in 10. WATCH Facility of this document.

For most circumstances WATCH can be made available for troubleshooting even if the configuration is significantly broken. This is done by using a skeleton-key to authorise special access into the server.

The skeleton-key is described in detail in 3.12 Skeleton-Key Authentication, also in this document.

TL;DR

Enable at the command-line with the username anything beginning with an underscore and at least 8 characters, same for the password length.

$ HTTPD /DO=AUTH=SKELKEY=_username:password

Then using a browser access any available service, entering the above username (including underscore) and password when prompted.

https://the.host.name:port /httpd/-/admin/report/WATCH

The service administration facilities (of which WATCH is one) are also available and useful.

https://the.host.name:port /httpd/-/admin/

2.Package Overview

2.1Server Behaviour
2.2VMS Versions
2.3TCP/IP Packages
2.4International Features

The most fundamental component of the WASD VMS Web Services environment is the HTTP server (HyperText Transport Protocol Daemon, or HTTPd). WASD has a single-process, multi-threaded, asynchronous I/O design.

The following bullet-points summarise the features and facilities, many of which are described in significant detail in following chapters.

General
Scripting
Access Control
Administration

2.1Server Behaviour

The technical aspects of server design and behaviour are described in WASD_ROOT:[SRC.HTTPD]READMORE.TXT

2.2VMS Versions

The WASD server is supported on any VMS version from V7.0 upwards, on Alpha, Itanium and x86-64 architectures. The current version (as of 2021), V8.4 Alpha and Itanium, as is commonly the case on VMS platforms, required nothing more than relinking. Obviously no guarantees can be made for yet-to-be-released versions but at a worst-case these should only require the same.

The WASD distribution and package organisation fully supports mixed-architecture clusters (Alpha, Itanium and/or x86-64 in the one cluster) as one integrated installation.

2.3TCP/IP Packages

The WASD server uses the TCP/IP Services (UCX) BG $QIO interface. The following packages support this interface and may be used.

To deploy IPv6 services this package must support IPv6.

2.4International Features

WASD provides a number of features that assist in the support of non-English and multi-language sites. These "international" features only apply to the server, not necessarily to any scripts!


3.Authentication and Authorization

3.1Rule Interpretation
3.2Authentication Policy
3.3Permissions, Path and User
3.4Authorization Configuration File
3.5Authentication Sources
3.6Realm, Full-Access, Read-Only
3.7Virtual Servers
3.8Authorization Configuration Examples
3.8.1KISS
3.9Authorization Cache
3.10SYSUAF-Authenticated Users
3.10.1ACME
3.10.2Logon Type
3.10.3Rights Identifiers
3.10.4WASD "Hard-Wired" Identifiers
3.10.5VMS Account Proxying
3.10.6Nil-Access VMS Accounts
3.10.7SYSUAF and SSL
3.10.8SYSUAF Security Profile
3.10.9SYSUAF Profile For Full Site Access
3.11Token Authentication
3.12Skeleton-Key Authentication
3.13Controlling Server Write Access
3.14Securing All Requests
3.15User Password Modification
3.16Cancelling Authorization

Authentication is the verification of a user's identity, usually through username/password credentials. Authorization is allowing a certain action to be applied to a particular path based on authentication of the originator.

Generally, authorization is a two step process. First authentication, using a username/password database. Second authorization, determining what the username is allowed to do for this transaction.

Basic authorization was discussed in Authorization Configuration (Basics) of WASD Configuration. This section discusses all the aspects of WASD authentication and authorization.

Overview

By default, the logical name WASD_CONFIG_AUTH locates a common authorization rule file. Simple editing of the file and reloading into the running server changes the processing rules.

Server authorization is performed using a configuration file, authentication source, and optional full-access and read-only authorization grouping sources, and is based on per-path directives. There is no user-configured authorization necessary, or possible! In the configuration file paths are associated with the authentication and authorization environments, and so become subject to the HTTPd authorization mechanism. Reiterating … WASD HTTPd authorization administration involves those two aspects, setting authorization against paths and administering the authentication and authorization sources.

Authorization is applied to the request path (i.e. the path in the URL used by the client). Sometimes it is possible to access the same resource using different paths. Where this can occur care must be exercised to authorize all possible paths.

Where a request will result in script activation, authorization is performed on both script and path components. First script access is checked for any authorization, then the path component is independently authorized. Either may result in an authorization challenge/failure. This behaviour can be disabled using a path SETting rule, see SET Rule of WASD Configuration.

The authentication source name is refered to as the realm, and refers to a collection of usernames and passwords. It can be the system's SYSUAF database.

The authorization source is refered to as the group, and commonly refers to a collection of usernames and associated permissions.

3.1Rule Interpretation

The configuration file rules are scanned from first towards last, until a matching rule is encountered (or end-of-file). Generally a rule has a trailing wildcard to indicate that all sub-paths are subject to the same authorization requirements.

String Matching

Rule matching is string pattern matching, comparing the request specified path, and optionally other components of the request when using configuration conditionals Conditional Configuration of WASD Configuration, to a series of patterns, until one of the patterns matches, at which stage the authorization characteristics are applied to the request and authentication processing is undertaken. If a matching pattern (rule) is not found the path is considered not to be subject to authorization. Both wildcard and regular expression based pattern matching is available String Matching of WASD Configuration.

3.2Authentication Policy

A policy regarding when and how authorization can be used may be established on a per-server basis. This can restrict authentication challenges to "https:" (SSL) requests (4. Transport Layer Security), thereby ensuring that the authorization environment is not compromised by use in non-encrypted transactions. Two server qualifiers provide this.

Note also that individual paths may be restricted to SSL requests using either the mapping conditional rule configuration or the authorization configuration files. See Conditional Mapping of WASD Configuration.

In addition, the following configuration parameters have a direct role in an established authorization policy.

Authentication Failures

Details of authentication failures are logged to the server process log.

Failures may also be directed to the OPCOM facility OPCOM Logging of WASD Configuration.

3.3Permissions, Path and User

Both paths and usernames have permissions associated with them. A path may be specified as read-only, read and write, write-only (yes, I'm sure someone will want this!), or none (permission to do nothing). A username may be specified as read capable, read and write capable, or only write capable. For each transaction these two are combined to determine the maximum level of access allowed. The allowed action is the logical AND of the path and username permissions.

The permissions may be described using the HTTP method names, or using the more concise abbreviations R, W, and R+W.

HTTP Methods
Path/User DELETE GET HEAD POST PROPFIND PUT WebDAV
READ or R no yes yes no yes no no
WRITE or W yes no no yes no yes yes
R+W yes yes yes yes yes yes yes
NONE no no no no no no no
DELETE yes yes no no no no no
GET no yes no no no no no
HEAD no no yes no no no no
POST no no no yes no no no
PROPFIND no no no no yes no no
PUT no yes no no no yes no
Other WebDAV no no no no no no yes

3.4Authorization Configuration File

Requiring a particular path to be authorized in the HTTP transaction is accomplished by applying authorization requirements against that path in a configuration file. This is an activity distinct from setting up and maintaining any authentication/authorization databases required for the environment.

By default, the system-table logical name WASD_CONFIG_AUTH locates a common authorization configuration file, unless an individual rule file is specified using a job-table logical name. Simple editing of the file changes the configuration. Comment lines may be included by prefixing them with the hash "#" character, and lines continued by placing the backslash character "\" as the last character on a line.

The [IncludeFile] is a directive common to all WASD configuration, allowing a separate file to be included as a part of the current configuration. (see Include File Directive of WASD Configuration.

Configuration directives begin either with a "[realm]", "[realm;group]" or "[realm;group-r+w;group-r]" specification, with the forward-slash of a path specification, or with a "[AuthProxy]" or "[AuthProxyFile]" introducing a proxy mapping. Following the path specification are HTTP method keywords controlling group and world permissions to the path, and any access-restricting request scheme ("https:") and/or host address(es) and/or username(s).

The same path cannot be specified against two different realms for the same virtual service. The reason lies in the HTTP authentication schema, which allows for only one realm in an authentication dialog. How would the server decide which realm to use in the authentication challenge? Of course, different parts of a given tree may have different authorizations, however any tree ending in an asterisk results in the entire sub-tree being controlled by the specified authorization environment, unless a separate specification exists for some inferior portion of the tree.

There is a thirty-one character limit on authentication source names.

Reserved Names

The following realm names are reserved and have special functionality.

Reserved Username

The following username is reserved.

Access Restriction Keywords

If a host name, protocol identifier or username is included in the path configuration directive it acts to further limit access to matching clients (path and username permissions still apply). If more than one are included a request must match each. If multiple host names and/or usernames are included the client must match at least one of each. Host and username strings may contains the asterisk wildcard, matching one or more consecutive characters. This is most useful when restricting access to all hosts within a given domain, etc. In addition a VMS security profile may be associated with the request.

For example
/web/secret/* *.three.stooges,~Moe,~Larry,~Curly,read
restricts read access to Curly, Larry and Moe accessing from within the three.stooges network, while
/web/secret/* https:,*.three.stooges,~Moe,~Larry,~Curly,read
applies the further restriction of access via "https:" (SSL) only.

These examples show the use of a network mask to restrict based on the source network of the client. The first, four octets supplied as a mask. The second a VLSM used to specify the length of the network component of the address.

/web/secret/* https:,#131.185.250.128/255.255.255.192,~Moe,~Larry,~Curly,read /web/secret/* https:,#131.185.250.128/26,~Moe,~Larry,~Curly,read

These examples both specify a 6 bit subnet. With the above examples the host 131.185.250.250 would be accepted, but 131.185.250.50 would be rejected.

Note that it more efficient to place protocol and host restrictions at the front of a list.

3.5Authentication Sources

Authentication credentials may be validated against one of several sources, each with different characteristics.

Multiple Source Types

A realm directive may contain one or more different types of authorization information source, with the following restrictions.

Realm Description

It is possible to supply text describing the authentication realm to the browser user that differs from the actual source name. This may be used to disguise the actual source or to provide a more informative description than the source name conveys.

Prefixing the actual realm source name with a double-quote delimited string (of up to 31 characters) and an equate symbol will result in the string being sent to a browser as the realm description during an authentication challenge. Here are some examples.

["the local host"=VMS] ["Social Club"=SOCIAL_CLUB_RW=id] ["Finance Staff"=FINANCE=list] ["Just Another Database"=DBACCESS=hta]
Note

The Digest authentication scheme uses the realm description at both server and browser in the encrypted password challenge and response. When passwords are stored in an HTA file this realm synonym cannot be changed without causing these passwords to be rendered invalid.

3.6Realm, Full-Access, Read-Only

WASD authorization offers a number of combinations of access control. This is a summary. Please note that when refering to the level-of-access a particular username may be allowed (read-only or full, read-write access), that it is always moderated by the level-of-access provided with a path configured within that realm. See 3.3 Permissions, Path and User.

3.7Virtual Servers

As described in Virtual Services of WASD Configuration, virtual service syntax may be used with authorization mapping to selectively apply rules to one specific service. This example provides the essentials of using this syntax. Note that service-specific and service-common rules may be mixed in any order allowing common authorization environments to be shared.

# authorization rules example for virtual servers [[alpha.example.com:443]] # ALPHA SSL is the only service permitting VMS (SYSUAF) authentication [LOCAL=vms] /web/* https:,r+w ; r /httpd/-/admin/* ~daniel,https:,r+w [[beta.example.com:80]] # BETA has its own HTA database [BETA_USER=hta] /web/* r+w ; r [[gamma.example.com:80]] # GAMMA likewise [GAMMA_DEVELOPER=id;PROJECT-A=list] /web/project/a/* r+w ; r [GAMMA_DEVELOPER=id;PROJECT-B=list] /web/project/b/* r+w ; r [[*]] # allow anyone from the local subnet to upload to here [WORLD] /web/unload/* 131.185.200.*,r+w

The online Server Administration facility path authorization report (9.4 HTTPd Server Reports) provides a selector allowing the viewing and checking of rules showing all services or only one particular virtual server, making it simpler to see exactly what any particular service is authorizing against.

3.8Authorization Configuration Examples

Mixed case is used in the configuration examples (and should be in configuration files) to assist in readability. Rule interpretation however is completely case-insensitive.

  1. In the following example the authentication realm is "WASD", a synonym for SYSUAF authentication, and the permissions group "SOCIALCLUB", a simple list of usernames. The directive allows those authenticated from the WASD realm and in the SOCIALCLUB group full access (read and write), and the world read-only.
    [WASD=vms;SOCIALCLUB=list] /web/socialclub/* r+w ; read
  2. This example illustrates restricting access according internet address. Both the group and world restriction is identical, but the group address is being specified numerically, while the world access is being specified alphabetically (just for the purposes of illustration). This access check is done doing simple wildcard comparison, and makes numerical specifications potentially more efficient because they are usually shorter. The second line restricts that path's write access even further, to one username, "BLOGGS".
    [WASD=vms;SOCIALCLUB=list] /web/socialclub/* 131.185.45.*,get,post; *.example.com,get /web/socialclub/accounts/* 131.185.45.*,~BLOGGS,get,post; *.example.com,get
  3. Three sources for authorization are specified in the following example. As the authentication source is VMS (by rights identifier), the full-access group and read-only group can also be determined by possessing the specified identifiers. The first path can only be written to by those holding the full-access identifier (librarian), the second path can only be read by both. The world has no access to these paths.
    [DEVELOPER=id;PROJECT_A_LIBRARIAN=id;PROJECT_A_USER=id] /web/projects/a/* r+w /web/projects/* r
  4. This example is the same as the one above, except in this case everyone else (that can authenticate against the resource) gets read-only access to the projects.
    [DEVELOPER=id;PROJECT_A_LIBRARIAN=id;*] /web/projects/a/* r+w /web/projects/* r
  5. In the following example the authentication realm and group are a single HTA database, "ADMIN". The first directive allows those in the ADMIN group to read and write, and the world to read ("get,post;get"). The second line restricts write and even read access to ADMIN group, no world access at all ("get,post").
    [ADMIN=hta] /web/everyone/* get,post;get /web/select/few/* get,post
  6. With this example usernames are used to control access to the specified paths. These usernames are authenticated from the COMPANY database. The world has read access in both cases. Note the realm description, "The Company".
    ["The Company"=COMPANY=hta] /web/docs/* ~Howard,~George,~Fred,r+w ; r /web/accounts/* ~George,r+w ; r
  7. The following example shows a path specifying the local system's SYSUAF being used to authenticate any usernames. Whenever using SYSUAF authentication it is strongly recommended to limit the potential hosts that can authenticate in this way by always using a host-limiting access restriction list. The world gets read access.
    [VMS] /web/local/area/* 131.185.250.*,r+w ; r
  8. To restrict server administration to browsers executing on the server system itself and the SYSUAF-authenticated username DANIEL use a restriction list similar to the following. It also shows the use of SYSUAF-authentication being hidden by using a realm description.
    ["not the VMS SYSUAF"=VMS] /httpd/-/admin/* #localhost,~daniel,r+w
  9. This example uses the RFC1413 identification protocol as the authentication source and a host group to control full access to paths in the realm.
    ["Ident Protocol"=RFC1413;131.185.250.0/24] /web/local/* r+w
  10. The following example illustrates providing a read and writable area (GET, POST and PUTable) to hosts in the local network without username authentication (careful!).
    [WORLD] /web/scratch/* *.local.hosts.only,r+w

3.8.1KISS

WASD authorization allows for very simple authorization environments and provides the scope for quite complex ones. The path authentication scheme allows for multiple, individually-maintained authentication and authorization databases that can then be administered by autonomous managers, applying to widely diverse paths, all under the ultimate control of the overall Web administrator.

Fortunately great complexity is not generally necessary.

Most sites would be expected to require only an elementary setup allowing a few selected Web information managers the ability to write to selected paths. This can best be provided with the one authentication database containing read and write permissions against each user, with and access-restriction list against individual paths.

For example. Consider a site with three departments, each of which wishes to have three representatives capable of administering the departmental Web information. Authentication is via the SYSUAF. Web administrators hold an approriate VMS rights identifier, "WEBADMIN". Department groupings are provided by three simple lists of names, including the Web administrators (whose rights identifier would not be applied if access control is via a simple list), a fourth lists those with read-only access into the Finance area. The four grouping files would look like:

# Department 1 # Department 2 WEB1 WEB1 WEB2 WEB2 JOHN RINGO PAUL CURLY GEORGE LARRY # Department 3 # Finance (read access) WEB1 PAUL WEB2 GEORGE MOE JOHN SHEMP RINGO MAC

The authorization configuration file then contains:

####################################################################### # allow web masters (!) to use the server administration facility # to revise web configuration files # world has no access (read or write) # access is only allowed from a browser in the same subnet as the HTTPd ["Hypo Thetical Corp."=HYPOTHETICAL=vms;WEBADMIN=id] /httpd/-/admin/* #150.15.30.*,r+w /wasd_root/local/* #150.15.30.*,r+w # allows Department 1 representatives to maintain their web # this may only be done from within the company subnet # world has read access ["Hypo Thetical Corp."=HYPOTHETICAL=vms;DEPARTMENT1=list] /web/dept/general/* 150.15.30.*,r+w ; r # and so on for the rest of the departments ["Hypo Thetical Corp."=HYPOTHETICAL=vms;DEPARTMENT2=list;FINANCE=list] # no world read access into finance, only those in the FINANCE list /web/dept/finance/* 150.15.30.*,r+w ["Hypo Thetical Corp."=HYPOTHETICAL=vms;DEPARTMENT3=list] /web/dept/inventory/* 150.15.30.*,r+w ; r /web/dept/production/* 150.15.30.*,r+w ; r # (the next uses line continuation just for illustration) /web/dept/marketing/* 150.15.30.*,\ r+w ;\ read # we need an area for general POSTing (just for illustration :-) [WORLD] /web/world/* r+w #######################################################################

3.9Authorization Cache

Access to authentication sources, SYSUAF, simple lists and HTA databases, are relatively expensive operations. To reduce the impact of this activity on request latency and general server performance, authentication and realm-associated permissions for each authenticated username are stored in a cache. This means that only the initial request needs to be checked from appropriate databases, subsequent ones are resolved more quickly and efficiently from cache.

Such cached entries have a finite lifetime associated with them. This ensures that authorization information associated with that user is regularly refreshed. This period, in minutes, is set using the [AuthCacheMinutes] configuration parameter. Zero disables caching with a consequent impact on performance.

Implication

Where-ever a cache is employed there arises the problem of keeping the contents current. The simple lifetime on entries in the authentication cache means they will only be checked for currency whenever it expires. Changes may have occured to the databases in the meantime.

Generally there is are other considerations when adding user access. Previously the user attempt failed (and was evaluated each time), now the user is allowed access and the result is cached.

When removing or modifying access for a user the cached contents must be taken into account. The user will continue to experience the previous level of access until the cache lifetime expires on the entry. When making such changes it is recommended to explicitly purge the authentication cache either from the command line using /DO=AUTH=PURGE (9.7 HTTPd Command Line) or via the Server Administration facility (9. Server Administration). Of course the other solution is just to disable caching, which is a less than optimal solution.

3.10SYSUAF-Authenticated Users

The ability to authenticate using the system's SYSUAF is controlled by the server /SYSUAF[=keyword] qualifier. By default it is disabled.

WARNING!

SYSUAF authentication is not recommended except in the most secure of LAN environments or when SSL is employed.
HTTP credentials (username and password) are transmitted as encoded plain-text making them vulnerable to evesdropping.

By default accounts with SYSPRV authorized are always rejected to discourage the use of potentially significant usernames (e.g. SYSTEM). This behaviour can be changed through the use of specific identifiers, see 3.10.3 Rights Identifiers immediately below. Accounts that are disusered, have passwords that have expired or that are captive or restricted are always rejected. Accounts that have access day/time restricting access will have those restrictions honoured (see 3.10.3 Rights Identifiers for a workaround for this).

Also see 3.10.6 Nil-Access VMS Accounts.

3.10.1ACME

By default the Authentication and Credential Management Extension (ACME) is used to authenticate SYSUAF requests on Alpha and Itanium running VMS V7.3 or later (3.5 Authentication Sources). The advantage of ACME is with the processing of the (rather complex) authentication requirements by a vendor-supplied implementation. It also allows SYSUAF password change to be made subject to the full site policy (password history, dictionary checking, etc.) which WASD does not implement.

Should ACME be unavailable

for whatever reason (x86-64 EAK for example) then define the logical name WASD_NO_ACME to force reversion to SYSUAF authentication.

3.10.2Logon Type

By default SYSUAF authentication uses the NETWORK access restriction from the account SYSUAF record. Alternatives LOCAL, DIALUP and REMOTE may be specified using global configuration directive

# WASD_CONFIG_GLOBAL [AuthSYSUAFlogonType] REMOTE
and/or authorization rule parameter 'param="logon=REMOTE"'
["VMS Credentials"=WASD_VMS_RW=ID] /secured/* r+w,https,param="logon=REMOTE"
(which takes precedence).

3.10.3Rights Identifiers

Whether or not any particular username is allowed to authenticate via the SYSUAF may be controlled by that account holding or not holding a particular VMS rights identifier. When a username has been authenticated via the SYSUAF, rights identifiers associated with that account may be used to control the level-of-access within that realm.

Use of identifiers for these purposes are enabled using the /SYSUAF=ID server startup qualifier.

The first three reserved identifier names are optional. A warning will be reported during startup if these are not found. The fourth must exist if SYSUAF proxy mappings are used in a /SYSUAF=ID environment.

Identifiers may be managed using the following commands. If unsure of the security implications of this action consult the relevant VMS system management security documentation.

$ SET DEFAULT SYS$SYSTEM $ MCR AUTHORIZE UAF> ADD /IDENTIFIER WASD_HTTPS_ONLY UAF> ADD /IDENTIFIER PROJECT_USER UAF> ADD /IDENTIFIER PROJECT_DEVELOPER UAF> ADD /IDENTIFIER PROJECT_LIBRARIAN

They can then be provided to desired accounts using commands similar to the following:

UAF> GRANT /IDENTIFIER PROJECT_USER <account>
and removed using:
UAF> REVOKE /IDENTIFIER PROJECT_USER <account>

Be aware that, as with all successful authentications, and due to the WASD internal authentication cache, changing database contents does not immediately affect access. Any change in the RIGHTSLIST won't be reflected until the cache entry expires or it is explicitly flushed ().

3.10.4WASD "Hard-Wired" Identifiers

Deprecated and Discouraged

As this has been deprecated for some years now the documentation for this functionality has been removed.

3.10.5VMS Account Proxying

Any authentication realm can have its usernames mapped into VMS usernames and the VMS username used as if it had been authenticated from the SYSUAF. This is a form of proxy access.

CAUTION

This is an extremely powerful mechanism and as a consequence requires enabling on the command-line at server startup using the /SYSUAF=PROXY qualifier and keyword. If identifiers are used to control SYSUAF authentication (i.e. /SYSUAF=ID) then any account mapped by proxy access must hold the WASD_PROXY_ACCESS identifier described in 3.10.3 Rights Identifiers (and server startup would be something like "/SYSUAF=(ID,PROXY)").

When a proxy mapping occurs request user authorization detail reflects the SYSUAF username characteristics, not the actual original authentication source. This includes username, user details (i.e. becomes that derived from the owner field in the SYSUAF), constraints on the username access (e.g. SSL only), and user capabilities including any profile if enabled. Authorization source detail remains unchanged, reflecting the realm, realm description and group of the original source. For CGI scripting an additional variable, WWW_AUTH_REMOTE_USER, provides the original remote username.

For each realm, and even for each path, a different collection of mappings can be applied. Proxy entries are strings containing no white space. There are three basic variations, each with an optional host or network mask component.

The "SYSUAF" is the VMS username being mapped to. The remote is the remote username (CGI variable WWW_REMOTE_USER). The first variation maps a matching remote username (and optional host/network) onto the specific SYSUAF username. The second maps all remote usernames (and optional host/network) to the one SYSUAF username (useful as a final mapping). The third maps all remote usernames (optionally on the remote host/network) into the same SYSUAF username (again useful as a final mapping if there is a one-to-one equivalence between the systems).

Proxy mappings are processed sequentially from first to last until a matching rule is encountered. If none is found authorization is denied. Match-all and default mappings can be specified.

[RFC1413] [AuthProxy] bloggs@131.185.250.1=fred [AuthProxy] doe@131.185.250.*=john system=- *@131.185.252.0/24=* [AuthProxy] *=GUEST

In this example the username bloggs on system 131.185.250.1 can access as if the request had been authenticated via the SYSUAF using the username and password of FRED, although of course no SYSUAF username or password needs to be supplied. The same applies to the second mapping, doe on the remote system to JOHN on the VMS system. The third mapping disallows a system account ever being mapped to the VMS equivalent. The fourth, wildcard mapping, maps all accounts on all systems in 131.185.250.0 8 bit subnet to the same VMS username on the server system. The fifth mapping provides a default username for all other remote usernames (and used like this would terminate further mapping).

Note that multiple, space-separated proxy entries may be placed on a single line. In this case they are processed from left to right and first to last.

["Just an Example"=EXAMPLE=list] [AuthProxy] bloggs@131.185.250.1=fred doe@131.185.250.1=doe system=- \ *@131.185.252.0/24=* *=GUEST

Proxy mapping rules should be placed after a realm specification and before any authorization path rules in that realm. In this way the mappings will apply to all rules in that realm. It is possible to change the mappings between rules. Just insert the new mappings before the (first) rule they apply to. This cancels any previous mappings and starts a new set. This is an example.

["A Bunch of Users"=USERS=hta] [AuthProxy] bloggs@131.185.250.1=fred doe@131.185.250.1=john /fred/and/johns/path/* r+w [AuthProxy] *=GUEST /other/path/* read

An alternative to in-line proxy mapping is to provide the mappings in one or more independent files. In-line and in-file mappings may be combined.

["Another Bunch of Users"=MORE_USERS=hta] [AuthProxy] SYSTEM=- [AuthProxyFile] WASD_ROOT:[LOCAL]PROXY.CONF /path/for/proxy* r+w

To cancel all mappings for following rules use an [AuthProxy] (with no following mapping detail). Previous mappings are always cancelled with the start of a new realm specification. Where proxy mapping is not enabled at the command line or a proxy file cannot be loaded at startup a proxy entry is inserted preventing all access to the path.

REMEMBER – proxy processing can be observed using the WATCH facility.

3.10.6Nil-Access VMS Accounts

It is possible, and may be quite effective for some environments, to have a SYSUAF account or accounts strictly for HTTP authorization, with no actual interactive or other access allowed to the VMS system itself. This would relax the caution on the use of SYSUAF authentication outside of SSL transactions. An obvious use would be for the HTTP server administrator. Additional accounts could be provided for other authorization requirements, all without compromising the system's security.

In setting up such an environment it is vital to ensure the HTTPd server is started using the /SYSUAF=ID qualifier (3.2 Authentication Policy). This will require all SYSUAF-authenticated accounts to possess a specific VMS resource identifier, accounts that do not possess the identifier cannot be used for HTTP authentication. In addition the identifier WASD_NIL_ACCESS will need to be held (3.10.3 Rights Identifiers), allowing the account to authenticate despite being restricted by REMOTE and NETWORK time restrictions.

To provide such an account select a group number that is currently unused for any other purpose. Create the desired account using whatever local utility is used then activate VMS AUTHORIZE and effectively disable access to that account from all sources and grant the appropriate access identifier (see 3.10.3 Rights Identifiers above).

$ SET DEFAULT SYS$SYSTEM $ MCR AUTHORIZE UAF> MODIFY <account> /NOINTERACTIVE /NONETWORK /NOBATCH /FLAG=DISMAIL UAF> GRANT /IDENTIFIER WASD_NIL_ACCESS <account> UAF> GRANT /IDENTIFIER WASD_VMS_RW <account>

3.10.7SYSUAF and SSL

When SSL is in use (4. Transport Layer Security) the username/password authentication information is inherently secured via the encrypted communications of SSL. To enforce access to be via SSL add the following to the WASD_CONFIG_MAP configuration file:

/whatever/path/you/like/* "403 Access denied." ![sc:https]
or alternatively the following to the WASD_CONFIG_AUTH configuration file:
[REALM] /whatever/path/you/like/* https:

Note that this mechanism is applied after any path and method assessment made by the server's authentication schema.

The qualifier /SYSUAF=SSL provides a powerful mechanism for protecting SYSUAF authentication, restricting SYSUAF authenticated transactions to the SSL environment. The combination /SYSUAF=(SSL,ID) is particularly effective.

Also see 3.2 Authentication Policy.

3.10.8SYSUAF Security Profile

It is possible to control access to files and directories based on the VMS security profile of a SYSUAF-authenticated remote user. This functionality is implemented using VMS security system services involving SYSUAF and RIGHTSLIST information. The feature must be explicitly allowed using the server /PROFILE qualifier. By default it is disabled.

Note

Use caution when deploying the /PROFILE qualifier. It was really designed with a very specific environment in mind, that of an Intranet where the sole purpose was to provide VMS users access to their normal VMS resources via a Web interface.

When a SYSUAF-authenticated user (i.e. the VMS realm) is first authenticated a VMS security-profile is created and stored in the authentication cache (3.9 Authorization Cache). A cached profile is an efficient method of implementing this as it obviously removes the need of creating a user profile each time a resource is assessed. If this profile exists in the cache it is attached to each request authenticated for that user. As it is cached for a period, any change to a user's security profile in the SYSUAF or RIGHTSLIST won't be reflected in the cached profile until the cache entry expires or it is explicitly flushed (9.6 HTTPd Server Action).

When a request has this security profile all accesses to files and directories are assessed against it. When a file or directory access is requested the security-profile is employed by a VMS security system service to assess the access. If allowed, it is provided via the SYSTEM file protection field. Hence it is possible to be eligible for access via the OWNER field but not actually be able to access it because of SYSTEM field protections! If not allowed, a "no privilege" error is generated.

Once enabled using /PROFILE it can be applied to all SYSUAF authenticated paths, but must be enabled on a per-path basis, using the WASD_CONFIG_AUTH profile keyword (‘Access Restriction Keywords’ in 3.4 Authorization Configuration File)

# WASD_CONFIG_AUTH [VMS;VMS] /wasd_root/local/* profile,https:,r+w
or the WASD_CONFIG_MAP SET profile and noprofile mapping rules (see SET Rule of WASD Configuration).
# WASD_CONFIG_MAP set /wasd_root/local/* profile set * noprofile

Of course, this functionality only provides access for the server, IT DOES NOT PROPAGATE TO ANY SCRIPT ACCESS. If scripts must have a similar ability they should implement their own scheme (which is not too difficult, see WASD_ROOT:[SRC.MISC]CHKACC.C) based on the CGI variable WWW_AUTH_REALM which would be "VMS" indicating SYSUAF-authentication, and the authenticated name in WWW_REMOTE_USER.

Performance Impact

If the /PROFILE qualifier has enabled SYSUAF-authenticated security profiles, whenever a file or directory is assessed for access an explicit VMS security system service call is made. This call builds a security profile of the object being assessed, compares the cached user security profile and returns an indication whether access is permitted or forbidden. This is addition to any such assessments made by the file system as it is accessed.

This extra security assessment is not done for non-SYSUAF-authenticated accesses within the same server.

For file access this extra overhead is negligible but becomes more significant with directory listings ("Index of") where each file in the directory is independently assessed for access.

3.10.9SYSUAF Profile For Full Site Access

Much of a site's package directory tree is inaccessible to the server account. One use of the SYSUAF profile functionality is to allow authenticated accesss to all files in that tree. This can accomplished by creating a specific mapping for this purpose, subjecting that to SYSUAF authentication with /PROFILE behaviour enabled (3.10.8 SYSUAF Security Profile), and limiting the access to a SYSTEM group account. As all files in the WASD package are owned by SYSTEM the security profile used allows access to all files.

The following example shows a path with a leading dollar (to differentiate it from general access) being mapped into the package tree. The "set * noprofile" limits the application of this to the /$WASD_ROOT/ path (with the inline "profile").

# WASD_CONFIG_MAP set * noprofile . . . pass /wasd_root/* /wasd_root/* pass /$WASD_ROOT/* /wasd_root/* profile

This path is then subjected to SYSUAF authentication with access limited to an SSL request from a specific IP address (the site administrator's) and the SYSTEM account.

# WASD_CONFIG_AUTH [["/$WASD_ROOT/ Access"=WASD_TREE_ACCESS=id]] /$WASD_ROOT/* https,10.1.1.2,~system,read

3.11Token Authentication

This is a niche authorisation environment for addressing niche requirements.

A token is an HTTP cookie delivered representation of authentication established in another context. Originally devised to allow controlled access to very large datasets without the overhead of SSL in the transmission but with access credentials supplied in the privacy of an SSL connection.

A common scenario is where the client starts off attempting to access a resource in non-SSL space which is controlled by token authentication. In the first instance the authenticator detects there is no access token present and redirects the client (browser) to the SSL equivalent of that space, where credentials can be supplied encrypted. In this example scenario the SSL area is controlled by WASD SYSUAF authentication (can be SSL client certificate, etc.) and the username/password is prompted for. When correctly entered this generates a token. The token is stored (with corresponding detail) as a record in a server-internal database and then returned to the browser as a set-cookie value.

With the token data stored the browser is transparently redirected back to the non-SSL space where the actual access is to be undertaken, this time the browser presenting the cookie containing the token. The authenticator examines the token, looking it up in the database. If found, has originated from the same IP address, represents the same authentication realm, and has not expired, it then allows the non-SSL space access to proceed, and in this example scenario the dataset transfer is initiated (in unencrypted clear-text). If the token is not found in the database or has expired, then the process is repeated with a redirect back into SSL space. If the realms differ a 403 forbidden response is issued (see configuration below).

The token itself is a significant sequence of pseudo-random characters, is short-lived (configurable as anything from a few seconds to a few tens of seconds, or more), and as a consequence is frequently regenerated. The token is just that, containing no actual credential data at all. It might be possible to snoop but as it contains nothing of value in itself, expires relatively quickly, and has an originating IP address check, the fairly remote risk of playback is just that.

The authenticator does all the work, implicitly redirecting the user from non-SSL space to SSL space for the original authentication, and then back again with the token used for access in the non-SSL space. With the expiry of a token it undertakes that cycle again, redirecting back to the SSL-space where the browser-cached credentials will be supplied automatically allowing the fresh token to be issued, and then redirected back into non-SSL space for access. To emphasise - all this is transparent to the user.

As a consequence of this model the resource being controlled can ONLY be accessed from non-SSL space using the controlled path. To access the same resource from SSL space a distinct path to the resource must be provided.

Configuration

As token authorisation relies on the client agent having HTTP cookies enabled (globally or specifically for the site) it is useful to have this tested for and/or advised about, on some related but other area of the site. There are simple techniques using JavaScript for detecting the availability of cookie processing. Search the Web for a suitable solution.

The automatic authorisation and redirection occurs using a combination of two distinguishable authorisation rules, one for supplying the credentials, the other for using the token for authorisation. In this example (and commonly) the resources are at "/location/" and the configuration accepts user-supplied credentials in SSL space and uses the token in non-SSL space. The asterisk just indicates that in the absence of any other parameter this authorisation rule has a complementary token rule where access will actually occur.

# WASD_CONFIG_AUTH if (ssl:) ["VMS credentials"=WASD_VMS_RW=id+"TOKEN=*"] /location/* r+w else [WASD_VMS_RW=TOKEN] /location/* r+w endif

And in this example, the same arrangement but with non-standard ports (specified using an integer with a leading colon).

# WASD_CONFIG_AUTH if (ssl:) ["VMS credentials"=WASD_VMS_RW=id+"TOKEN=:7080"] /location/* r+w else [WASD_VMS_RW=TOKEN+"TOKEN=:7443"] /location/* r+w endif

To prevent potential thrashing, where multiple, distinct realms within a single request are authorised using tokens, corresponding multiple token (cookie) names must be used. It is expected that this would be an uncommon but not impossible scenario. The "thrashing" would be a result of authorisation associated with a single, particular token name. Where a realm differs from a previous token generated another is required. The token authorisation scheme forces the use of distinct token names by 403-forbidding change of realm using the one token. Use explicitly specified, independent token (cookie) names, or an integer preceded by an ampersand (which appends the integer to the base token name), ensuring the complementary rules are using the same name/integer.

# WASD_CONFIG_AUTH if (ssl:) ["VMS credentials"=WASD_VMS_RW=id+"TOKEN=&42"] /location/* r+w else [WASD_VMS_RW=TOKEN+"TOKEN=&42"] /location/* r+w endif

For the final example, the token is contained in the non-default cookie named "Wasd_example" and the authentication performed using an X509 client certificate (which can only be supplied via SSL).

# WASD_CONFIG_AUTH if (ssl:) [X509+"TOKEN=WaSd_example"] /location/* r+w else [X509=TOKEN+"TOKEN=WaSd_example"] /location/* r+w endif

Some additional detail is available from the AUTHTOKEN.C code module.

3.12Skeleton-Key Authentication

Provides a username and password that is authenticated from data placed into the global common (i.e. in memory) by the site administrator. The username and password expire (become non-effective) after a period, one hour by default or an interval specified when the username and password are registered.

It is a method for allowing ad hoc authenticated access to the server, primarily intended for non-configured access to the online Server Administration facilities (9.1 Access Before Configuration) but is available for other purposes where a permanent username and password in an authentication database is not necessary. A skeleton-key authenticated request is subject to all other authorization processing (i.e. access restrictions, etc.), and can be controlled using the likes of '~_*', etc.

The site administrator uses the command line directive

$ HTTPD /DO=AUTH=SKELKEY=username:password[:period]
to set the username/password, and optionally the period in minutes. This authentication credential can be cancelled at any time using
$ HTTPD /DO=AUTH=SKELKEY=0

The username must begin with an underscore (to reduce the chances of clashing with a legitimate username) and have a minimum of 6 other characters. The password is delimited by a colon and must be at least 8 characters. The optional period in minutes can be from 1 to 10080 (one week). If not supplied it defaults to 60 (one hour). After the period expires the skeleton key is no longer accepted until reset.

Note

Choose username and password strings that are less-than-obvious and a period that's sufficient to the task!
After all, it's your site that you might compromise!

The authentication process (with skeleton-key) is performed using these basic steps.

  1. Is a skeleton-key set? If not continue on with the normal authentication process.
  2. If set then check the request username leading character for an underscore. If not then continue on with normal authentication.
  3. If it begins with an underscore then match the request and skeleton-key usernames. If they do not match then continue with normal authentication.
  4. If the usernames match then compare the request and skeleton-key passwords. If matched then it's authenticated. If not it becomes an authentication failure.

Note that the authenticator resumes looking for a username from a configured authentication source unless the request and skeleton-key usernames match. After that the passwords either match allowing access or do not match resulting in an authentication failure.

Examples
$ HTTPD /DO=AUTH=SKELKEY=_FRED2ACC:USE82PA55 $ HTTPD /DO=AUTH=SKELKEY=_ANDY2WERP:EGGO4TEE:10

3.13Controlling Server Write Access

The server account should have no direct write access to into any directory structure. Files in these areas should be owned by SYSTEM ([1,4]). Write access for the server into VMS directories (using the POST or PUT HTTP methods) should be controlled using VMS ACLs. This is in addition to the path authorization of the server itself of course! The recommendation to have no ownership of files and provide an ACE on required directories prevents inadvertant mapping/authorization of a path resulting in the ability to write somewhere not intended.

Two different ACEs implement two grades of access.

  1. If the ACE grants CONTROL access to the server account then only VMS-authenticated usernames with security profiles can potentially write to the directory. Only potentially, because a further check is made to assess whether that VMS account in particular has write access.

    This example shows a suitable ACE that applies only to the original directory:

    $ SET SECURITY directory.DIR - /ACL=(IDENT=HTTP$SERVER,ACCESS=READ+WRITE+EXECUTE+DELETE+CONTROL)
    This example shows setting an ACE that will propagate to created files and importantly, subdirectories:
    $ SET SECURITY directory.DIR - /ACL=((IDENT=HTTP$SERVER,OPTIONS=DEFAULT,ACCESS=READ+WRITE+EXECUTE+DELETE+CONTROL), - (IDENT=HTTP$SERVER,ACCESS=READ+WRITE+EXECUTE+DELETE+CONTROL))
  2. If the ACE grants WRITE access then the directory can be written into by any authenticated username for the authorized path.

    This example shows a suitable ACE that applies only to the original directory:

    $ SET SECURITY directory.DIR - /ACL=(IDENT=HTTP$SERVER,ACCESS=READ+WRITE+EXECUTE+DELETE)
    This example shows setting an ACE that will propagate to created files and importantly, subdirectories:
    $ SET SECURITY directory.DIR - /ACL=((IDENT=HTTP$SERVER,OPTIONS=DEFAULT,ACCESS=READ+WRITE+EXECUTE+DELETE), - (IDENT=HTTP$SERVER,ACCESS=READ+WRITE+EXECUTE+DELETE))

To assist with the setting of the required ACEs an example, general-purpose DCL procedure is provided, WASD_ROOT:[EXAMPLE]AUTHACE.COM).

3.14Securing All Requests

Some sites may be sensitive enough about Web resources that the possibility of providing inadvertant access to some area or another is of major concern. WASD provides a facility that will automatically deny access to any path that does not appear in the authorization configuration file. This does mean that all paths requiring access must have authorization rules associated with them, but if something is missed some resource does not unexpectedly become visible.

At server startup the /AUTHORIZE=ALL qualifier enables this facility.

For paths that require authentication and authorization the standard realms and rules apply. To indicate that a particular path should be allowed access, but that no authorization applies the "NONE" realm may be used. The following example provides some indication of how it should be used.

# allow the librarian to update this area, world to read it [VMS;LIBRARIAN=id] /web/library/* r+w ; read # indicate there is no authorization to be applied [NONE] # allow access to general web areas /web/* read # allow access to the WASD_ROOT tree /wasd_root/* read

There is also a per-path equivalent of the /AUTHORIZE=ALL functionality, described in SET Rule of WASD Configuration). This allows a path tree to be require authorization be enabled against it.

# avoid an absence of authorization allowing unintentional access set /web/sensitive/* auth=all

3.15User Password Modification

The server provides for users to be able to change their own HTA passwords (and SYSUAF if required). This functionality, though desirable from the administrator's viewpoint, is not mandatory if the administrator is content to field any password changes, forgotten passwords, etc. Keep in mind that passwords, though not visible during entry, are passed to the server using clear-text form fields (which is why SSL is recommended).

Password modification is enabled by including a mapping rule to the internal change script. For example:

pass /httpd/-/change/* /httpd/-/change/*

Any database to be enabled for password modification must have a writable authorization path associated with it. For example:

[GROUP=id;GROUP=id] /httpd/-/change/group/* r+w [ANOTHER_GROUP=id;ANOTHER_GROUP=id] /httpd/-/change/another_group/* r+w
Note

What looks like redundancy in specifying an identical realm and group authorization is what allows multiple, independant identifiers to be individually controlled for password change (i.e. one group of identifier holders allowed to change the password, another not).

Use some form of cautionary wrapper if providing this functionality over something other than an Intranet or SSL connection:

<H2>Change Your Authentication</H2> <blockquote> Change the password used to identify yourself to the REALM Web environment for some actions. Note that this <u>not</u> an operating system password, nor has it anything to do with it. Due to the inherent weaknesses of using non-encrypted password transmissions on networks <font color="#ff0000"><u>DO NOT</U> use a password you have in use anywhere else, especially an operating system password!</font> You need your current password to make the change. If you have forgotten what it is contact <a href="/web/webadmin.html">WebAdmin</a>, preferably via e-mail, for the change to be made on your behalf. </blockquote> <ul> <li><a href="/httpd/-/change/REALM/">REALM</a> realm. </ul>
Password Expiry

When using SYSUAF authentication it is possible for a password to pre-expired, or when a password lifetime is set for a password to expire and require respecification. By default an expired password cannot be used for access. This may be overridden using the following global configuration directive.

[AuthSYSUAFacceptExpPwd] enabled

Expired passwords may be specially processed by specifying a URL with WASD_CONFIG_GLOBAL [AuthSysUafPwdExpURL] configuration directive Alphabetic Listings of WASD Configuration).

The WASD_CONFIG_MAP set auth=sysuaf=pwdexpurl=<string> rule allows the same URL to be specified on a per-path basis. When this is set a request requiring SYSUAF authentication that specifies a username with an expired password is redirected to the specified URL. This should directly or via an explanatory (wrapper) page redirect to the password change path described above. The password change dialog will have a small note indicating the password has expired and allows it to be changed.

The following WASD_CONFIG_GLOBAL directive

# WASD_CONFIG_GLOBAL [AuthSysUafPwdExpURL] https:///httpd/-/change/ # WASD_CONFIG_AUTH [WASD_VMS_ID=id;WASD_VMS_RW=id] /httpd/-/change/* r+w
would allow expired passwords to be changed.

It is also possible to redirect an expired password to a site-specific page for input and change. This allows some customization of the language and content of the expired password change dialog. An example document is provided at WASD_ROOT:[EXAMPLE]EXPIRED.SHTML (what it looks like) ready for relocation and customisation. Due to the complexities of passing realm information and then submitting that information to the server-internal change facility some dynamic processing is required via an SSI document.

This example assumes the site-specific document has been located at WEB:[000000]EXPIRED.SHTML and is accessed using SSL.

# WASD_CONFIG_GLOBAL [AuthSysUafPwdExpURL] https:///web/expired.shtml?httpd=ignore&realm=vms # WASD_CONFIG_AUTH [WASD_VMS_ID=id;WASD_VMS_RW=id] /httpd/-/change/vms/* r+w /web/expired.shtml r+w

3.16Cancelling Authorization

The reason authorization information is not required to be reentered on subsequent accesses to controlled paths is cached information the browser maintains. It is sometimes desirable to be able to access the same path using different authentication credentials, and correspondingly it would be useful if a browser had a purge authorization cache button, but this is commonly not the case. To provide this functionality the server must be used to "trick" the browser into cancelling the authorization information for a particular path.

This is achieved by adding a specific query string to the path requiring cancellation. The server detects this and returns an authorization failure status (401) regardless of the contents of request "Authorization:" field. This results in the browser flushing that path from the authorization cache, effectively requiring new authorization information the next time that path is accessed.

There are two variations on this mechanism.

  1. The basic procedure is as follows:
  2. A little more functional, if using a revalidation period via [AuthRevalidateUserMinutes] or 'SET auth=revalidate=' (perhaps set to something like 23:59:00, or one day), when the logout query string is supplied the server resets the entry forcing any future access to require revalidation. A successful logout message is then generated, circumventing the need for the username/password dialog described above.

    Also when using logout with a revalidation period a redirection URL may be appended to the logout query string. It then redirects to the supplied URL. It is important that the redirection is returned to the browser and not handled internally by WASD. Normal WASD redirection functionality applies.

    ?httpd=logout&goto=/// ?httpd=logout&goto=///help/logout.html ?httpd=logout&goto=http://the.host.name/

    These examples redirect to

    respectively.
    Authentication Cache

    User revalidation relies on an entry being maintained in the authentication cache. Each time the entry is flushed, for whatever reason (cache congestion, command-line purge, server restart, etc.), the user will be prompted for credentials. It may be necessary to increase the size of the cache by adjusting [AuthCacheEntriesMax].

4.Transport Layer Security

4.1Let's Encrypt
4.2TLS/SSL Functionality Sources
4.3WASD SSL Quick-Start
4.4OPENSSL.EXE Application
4.5SSL Configuration
4.5.1WASD_CONFIG_SERVICE
4.5.2TLS/SSL Versions
4.5.3SSL Ciphers
4.5.4(Open)SSL Options
4.5.5Forward Secrecy
4.5.6Session Resumption
4.5.7Strict Transport Security
4.5.8SSL Server Certificate
4.5.9SSL Private Key
4.5.10SSL Virtual Services
4.5.11SSL Access Control
4.5.12Authorization Using X.509 Certification
4.5.13X.509 Certificate Renegotiation
4.5.14Features
4.5.15Subject Alternative Name and Other Extensions
4.5.16X509 Configuration
4.5.17Certificate Authority Verification File
4.5.18X.509 Authorization CGI Variables
4.6Certificate Management
4.6.1Server Certificate
4.6.2Certificate Signing Request
4.7SSL CGI Variables
4.8SSL Service Evaluation
4.9SSL References

Transport Layer Security (TLS), and its predecessor Secure Sockets Layer (SSL), are cryptographic protocols designed to provide communication privacy over a network, in the case of HTTP between the browser (client) and the server. It also authenticates server and optionally client identity. TLS/SSL operates by establishing an encrypted communication path between the two applications, "wrapping" the entire application protocol inside the secure link, providing complete privacy for the entire transaction. In this way security-related data such as user identification and password, as well as sensitive transaction information can be protected from unauthorized access while in transit. This section is not a tutorial on TLS/SSL. It contains only information relating to WASD's use of it. See 4.9 SSL References for further information on TLS/SSL technology.

TLS and SSL

The terms are used interchangably in this document to represent cryptographic communication technology. They are similar but with important differences. TLS is the more modern and considered the more secure. The term SSL is still in common usage though and retained here even if WASD (and OpenSSL) now only implements TLS. When OpenSSL(.org) considers changing its name WASD will toss out the term SSL

WASD implements SSL using a freely available software toolkit supported by the OpenSSL Project.

OpenSSL licensing allows unrestricted commercial and non-commercial use. This toolkit is in use regardless of whether the WASD OpenSSL package, VSI SSL for OpenVMS product, or other stand-alone OpenSSL environment is installed. It is always preferable to move to the latest support release of OpenSSL as known bugs in previous versions are progressively addressed (ignoring the issue of new bugs being introduced ;-)

TLS functionality is not supplied with the basic WASD package

In part this is due to the relative bulk of this component, in further part that the updates to each are not necessarily coincident, and also considers potential patent issues and export restrictions on some cryptography technology in some jurisdictions.
Cryptography Software

Be aware that export/import and/or use of cryptography software, or even just providing cryptography hooks, is illegal in some parts of the world. When you re-distribute this package or even email patches/suggestions to the author or other people, please PAY CLOSE ATTENTION TO ANY APPLICABLE EXPORT/IMPORT LAWS. The author of this package is not liable for any violations you make here.

4.1Let's Encrypt

Have (or want) a TLS/SSL secured site?

Using self-signed or commercial server certificate(s)?

Let's Encrypt makes it possible to obtain and maintain browser-trusted certificates, simply, automatically and at no cost.

See WASD Certificate Management Environment (wuCME) on the WASD download page at https://wasd.vsm.com.au/wasd/

4.2TLS/SSL Functionality Sources

Secure Sockets Layer functionality is easily integrated into WASD and is available from one (or more) of the following sources. See for the basics of installing WASD SSL and for configuration of various aspects.

  1. The VSI SSL111 for OpenVMS product

    This is provided from the directory SYS$COMMON:[SSL111] containing static and shared libraries, executables, etc. Similarly, if this product is installed and started the WASD installation and update procedures should detect it.

  2. The VSI SSL3 for OpenVMS product

    This is provided from the directory SYS$COMMON:[SSL3] containing static and shared libraries, executables and templates for certificate management, etc. If this product is installed and started the WASD installation and update procedures should detect it and provide the option of compiling and/or linking WASD against its shareable or static libraries.

  3. WASD OpenSSL Package

    As described in VSI OpenSSL SSL111-V0101-1S and SSL3-V0300-7 it is now possible to install VSI OpenSSL releases on pre-V8.4 VMS. This is the recommended approach to providing and maintaining OpenSSL for WASD.

4.3WASD SSL Quick-Start

SSL functionality can be installed with a new package, or with an update, or it can be added to an existing non-SSL enabled site. The following steps give a quick outline for support of SSL.

  1. If using the VSI SSL111 product or an already installed OpenSSL toolkit go directly to step 2. To install the WASD OpenSSL package the ZIP archive needs to be restored.
  2. It is then necessary to build the (server and Open)SSL executables.
  3. Once linked the UPDATE.COM procedure will prompt for permission to execute the demonstration/check procedure.

    It is also possible to check the SSL package at any other time using the server demonstration procedure. It is necessary to specify that it is to use the SSL executable. Follow the displayed instructions.

    $ @WASD_ROOT:[INSTALL]DEMO.COM SSL
  4. Modification of server startup procedures should not be necessary. If an SSL image is detected during startup it will be used in preference to the standard image.
  5. Modify the WASD_CONFIG_SERVICE configuration file to specify an SSL service. For example the following adds a generic SSL service on port 443.
    [[https://*:443]]
  6. Shutdown the server completely, then restart.
    $ HTTPD /DO=EXIT $ @WASD_ROOT:[STARTUP]STARTUP
  7. To check the functionality (on default ports) access the server via
  8. Once the server has been proved functional with the example certificate it is recommended that a server-specific certificate be deployed. As self-signed certificates are increasingly resisted or rejected by browsers it is recommended to install a Certificate Authority signed certificate, or using the WASD wuCME application obtain a free-and-at-no-cost one from 4.1 Let's Encrypt.

4.4OPENSSL.EXE Application

The OPENSSL.EXE application is a command line tool for using the various cryptography functions of OpenSSL's crypto library from the shell. It is described being used several times in this section of the documentation. Refer to the OpenSSL Man page for descriptions of the various commands and their syntax.

It is commonly used as a foreign verb on VMS systems and assigned during SYLOGIN.COM or LOGIN.COM and depends on the distribution and version in use. For example:

A simple addition to SYLOGIN.COM or LOGIN.COM for WASD-specific OpenSSL kits to assign the OPENSSL verb is:

$ @WASD_ROOT:[EXAMPLE]WASDVERBS.COM SSL

4.5SSL Configuration

The example server startup procedure already contains support for the SSL executable. If this has been used as the basis for startup then an SSL executable will be started automatically, rather than the standard executable. The SSL executable supports both standard HTTP services (ports) and HTTPS services (ports). These must be configured using the [service] parameter. SSL services are distinguished by specifying "https:" in the parameter. The default port for an SSL service is 443.

WASD can configure services using the WASD_CONFIG_GLOBAL [SSL..] directives, the per-service WASD_CONFIG_SERVICE [ServiceSSL..] directives, or the /SSL= qualifier. Configuration precedence is WASD_CONFIG_SERVICE, /SSL= and finally WASD_CONFIG_GLOBAL.

4.5.1WASD_CONFIG_SERVICE

SSL service configuration using the WASD_CONFIG_SERVICE configuration is slightly simpler, with a specific configuration directive for each aspect. (see Service Configuration of WASD Configuration). This example illustrates configuring the same services as used in the previous section.

[[http://alpha.example.com:80]] [[https://alpha.example.com:443]] [ServiceSSLversion] TLSvALL [ServiceSSLcert] WASD_ROOT:[local]alpha.pem [[https://beta.example.com:443]] [ServiceSSLversion] SSLv3 [ServiceSSLcert] WASD_ROOT:[local]beta.pem

4.5.2TLS/SSL Versions

SSL Versions

As WASD uses the OpenSSL package in one distribution or another it largely supports all of the capability of that underlying package. The obsolete SSLv2, and the deprecated SSLv3 are no longer accepted by default. WASD default comprise the TLS family of protocols, at the time of writing, TLSv1, TLSv1.1, TLSv1.2 and TLSv1.3.

Some older clients employing SSLv3 may fail. Symptoms are dropped connection establishment and WATCH [x]SSL variously showing "SSL routines SSLn_GET_RECORD wrong version number", "SSL routines SSLn_GET_CLIENT_HELLO unknown protocol", possibly others. It is generally considered SSL best-practice not to have SSLv3 enabled but if required may be supported by configuring WASD_CONFIG_GLOBAL [SSLversion] with "SSLv3,TLSvALL", the per-service WASD_CONFIG_SERVICE equivalent, or using the /SSL=(SSLv3,TLSvALL) command line parameter during server startup.

TLS Version 1.3

TLSv1.3 perhaps should have been designated TLSv2.0 and not be considered as an incremental improvement over earlier versions of TLS but a significant upgrade!

TLSv1.3 can be tested for as demonstrated at ‘test TLS Version 1.3’ in 4.8 SSL Service Evaluation.

4.5.3SSL Ciphers

Ciphers are the algorithms, designed and implemented on mathematical computations, that render the readable plaintext into unreadable ciphertext. Ciphers tend to be available in suites (or families) where variants, usually based on key size and therefore resistence to decryption without a known key, that browsers and otheragents negotiate on and accept when setting up a secure (encrypted) network transports with servers.

Cipher selection is important to the overall security of the supported environment as well as the range of clients and servers that can establish communication due to shared cipher suites. Including only more recent (and technically secure) ciphers can preclude older clients from establishing secure connection, and including older (and perhaps more susceptible to modern attack) ciphers increases site vunerability. Some environments, for example HTTP/2, are quite prescriptive regarding the secure connection, to the point of blacklisting protocol versions and cipher suites no longer considered secure enough.

Fortunately a number of sites provide cipher guidelines based on requirements. The Mozilla Developer Network provides these amongst other useful information on security and server side TLS.

https://wiki.mozilla.org/Security/Server_Side_TLS

WASD has a default (built-in) functional cipher list that is general in application and relevant to when it was compiled. This in particular and site cipher lists in general, should be reviewed from time to time as opinions and requirements do change.

Many agents (browsers) require the elliptic curve ciphers provided by Forward Secrecy elements (4.5.5 Forward Secrecy) to negotiate later TLS versions.

SSL Options
TLS/SSL Options
OpenSSL Options

4.5.4(Open)SSL Options

The OpenSSL package provides for various options to be flagged against an TLS/SSL service. WASD sets the (OpenSSL) default options and then allows these to be overwitten/set/reset using hexadecimal values representing bit patterns. OpenSSL defaults are suitable for most sites.

The SSL options directives in global and per-service configuration, and the OPTIONS= keyword for the /SSL= qualifier, accept

Alternatively, the following OpenSSL option mnemonics can be used with a leading "+" to enable, or "-" to disable

4.5.5Forward Secrecy

Forward secrecy, sometimes known as perfect forward secrecy (PFS), is a property of key-agreement protocols ensuring that a session key derived from a set of long-term keys cannot be compromised if one of the long-term keys is compromised in the future.

http://en.wikipedia.org/wiki/Forward_secrecy

OpenSSL supports forward secrecy using Diffie-Hellman key exchange with elliptic curve cryptography and this relies on generating emphemeral keys based on unique, safe prime numbers. These are expensive to generate and so this is done infrequently, often during software build or installation. In the case of WASD, to maximise flexibility, these numbers are stored in external PEM-format files, by default located in the WASD_ROOT:[LOCAL] directory. These files are only briefly accessed during server startup SSL initialisation and the content later used during network connection SSL negotiation to generate the required ephemeral keys.

PFS requires a small number of elements working in concert

The detail is described in these references

Generate directly at the command-line using the OpenSSL dhparam utility, located in the WASD_ROOT:[LOCAL] directory, the names use the format DH_PARAM_number-of-bits.PEM, as in these examples;

$ set default wasd_root:[local] $ openssl dhparam -out dh_param_512.pem 512 $ openssl dhparam -out dh_param_1024.pem 1024 $ openssl dhparam -out dh_param_2048.pem 2048
Note

Key generation can take some considerable time!

4.5.6Session Resumption

When a TLS/SSL connection is initiated an expensive handshake (in terms of time and compute) is required to establish the cryptographic and other elements of the connection. Mitigation of this expense is undertaken by allowing the resumption of a previous session (abbreviating the handshake exchanges) using connection state stored either at the server or at the client.

With Session Tickets being the more modern, flexible and efficient solution to session resumption (and being available cluster-wide) it is recommended that WASD sites disable Session ID caching.

The default maximum period for session reuse is five minutes. This may be set globally using the [SSLsessionLifetime] directive or on a per-service basis using [ServiceSSLsessionLifetime].

To some extent, the relatively long-lived connections and lower concurrency with HTTP/2 means the importance of session resumption in improving request latency and connection overhead is reduced.

4.5.7Strict Transport Security

HTTP Strict Transport Security (HSTS) is a security policy mechanism which helps protect sites against protocol downgrade attack and cookie hijacking. It allows web servers to declare that browsers and other complying agents should only interact using secure (TLS) HTTP connections and never via clear-text HTTP. HSTS is an IETF standard specified in RFC 6797.

When global configuration directive [SSLstrictTransSec] is non-zero, or per-service configuration directive [ServiceSSLstrictTransSec] is non-zero, or a path is SET response=sts=<value>, TLS/SSL HTTP responses include a "Strict-Transport-Security: max-age=seconds" header field. Conforming agents note this period and refuse to communicate with the site via clear-text HTTP for the period represented by the integer number of seconds specified.

4.5.8SSL Server Certificate

The server certificate is used by the browser to authenticate the server against the server certificate Certificate Authority (CA), in making a secure connection, and in establishing a trust relationship between the browser and server. By default this is located using the WASD_CONFIG_GLOBAL [SSLcert] or WASD_CONFIG_SERVICE [ServiceSSLcert] configuration directive, the WASD_CONFIG_SSL_CERT logical name, or using the /SSL= command-line qualifier, however if required. Each SSL service can have an individual certificate configured as in the example above.

4.5.9SSL Private Key

The private key is used to validate and enable the server certificate. A private key is enabled using a secret, a password. It is common practice to embed this (encrypted) password within the private key data. This private key can be appended to the server certificate file, or it can be supplied separately. If provided separately it can be located using the WASD_CONFIG_GLOBAL [SSLkey] or WASD_CONFIG_SERVICE [ServiceSSLkey] configuration directive, tor using the WASD_CONFIG_SSL_KEY logical. When the password is embedded in the private key information it becomes vulnerable to being stolen as an enabled key. For this reason it is possible to provide the password separately and manually.

If the password key is not found with the key during startup the server will request that it be entered at the command-line. This request is made via the HTTPDMON "STATUS:" line (see OPCOM Logging of WASD Configuration), and if any OPCOM category is enabled via an operator message. If the private key password is not available with the key it is recommended that OPCOM be configured, enabled and monitored at all times.

When a private key password is requested by the server it is supplied using the /DO=SSL=KEY=PASSWORD directive (9.7 HTTPd Command Line). This must be used at the command line on the same system as the server is executing. The server then prompts for the password.

Enter private key password []:
The password is not echoed. When entered the password is securely supplied to the server and startup progresses. An incorrect password will be reprompted for twice (i.e. up to three attempts are allowed) before the startup continues with the particular service not configured and unavailable. Entering a password consisting of all spaces will cause the server to abort the full startup and exit from the system.

4.5.10SSL Virtual Services

Multiple virtual SSL services (https:) sharing the same or individual certificates (and other characteristics) can essentially be configured against any host name (unique IP address or host name alias) and/or port in the same way as standard services (http:).

WASD SSL implements Server Name Indication (SNI), an extension to the TLS protocol that indicates what hostname the client is attempting to connect to at the start of the handshaking process. This allows a server to present multiple certificates on the same IP address and port number and hence allows multiple secure (HTTPS) websites (or any other Service over TLS) to be served off the same IP address without requiring all those sites to use the same certificate.

When the client presents an SNI server name during SSL connection establishment, WASD searches the list of services it is offering for an SSL service (the first hit) operating with a name matching the SNI server name. If matched, the SSL context (certificate, etc.) of that service is used to establish the connection. If not matched, the service the TCP/IP connection originally arrived at is used.

4.5.11SSL Access Control

When authorization is in place (3. Authentication and Authorization) access to username/password controlled data/functionality benefits enormously from the privacy of an authorization environment inherently secured via the encrypted communications of SSL. In addition there is the possibility of authentication via client X.509 certification (4.5.12 Authorization Using X.509 Certification). SSL may be used as part of the site's access control policy, as whole-of-site, see 3.2 Authentication Policy, or on a per-path basis (see Request Processing Configuration of WASD Configuration).

4.5.12Authorization Using X.509 Certification

The server access control functionality (authentication and authorization) allows the use of public key infrastructure (PKI) X.509 v3 client certificates for establishing identity and based on that apply authorization constraints. See 3. Authentication and Authorization for general information on WASD authorization and 3.4 Authorization Configuration File for configuring a X509 realm. 4. Transport Layer Security provides introductory references on public-key cryptography and PKI.

A client certificate is stored by the browser. During an SSL transaction the server can request that such a certificate be provided. For the initial instance of such a request the browser activates a dialog requesting the user select one of any certificates it has installed. If selected it is transmitted securely to the server which will usually (though optionally not) authenticate its Certificate Authority to establish its integrity. If accepted it can then be used as an authenticated identity. This obviates the use of username/password dialogs.

Important

Neither username/password nor certificate-based authentication addresses security issues related to access to individual machines and stored certificates, or to password confidentiality. Public-key cryptography only verifies that a private key used to sign some data corresponds to the public key in a certificate. It is a user responsibility to protect a machine's physical security and to keep private-key passwords secret.

The initial negotiation and verification of a client certificate is a relatively resource intensive process. Once established however, OpenSSL sessions are usually either stored in a cache or stored encrypted withing the client, reducing subsequent request overheads significantly. Each session has a specified expiry period after which the client is forced to negotiate a new session. This period is adjustable using the "[LT:integer]" and "[TO:integer]" directives described below.

4.5.13X.509 Certificate Renegotiation

An X.509 client certificate is requested at either TLS/SSL connection establishment (WASD_CONFIG_GLOBAL [SSLverifyPeer], WASD_CONFIG_SERVICE [ServiceSSLverifyPeer]) or once the request has been made and assessed against authorisation rules. If an X509 realm controls access to the resources then the TLS/SSL connection is queried for an X.509 client certificate to authenticate the client and authorise the access.

This is performed via a TLS/SSL renegotiation and for this the connection must have been cleared of request data. In the case of a HEAD, GET, OPTIONS, etc. request, this already has implicitly occurred by there being no request body. For POST, PROPFIND, PUT, etc. requests, the client most likely already will be transmitting the request body. This (application data) must be absorbed before the client certificate renegotiation can be performed.

In avoiding disruption to the current request, any request body must be buffered (in full, based on the content length specified in the header) before issuing the renegotiation. This consumes memory and potentially large quantities. The default maximum buffer space is 1MB. The maximum request body size and hence maximum memory accomodated can be configured using the per-service WASD_CONFIG_SERVICE [ServiceSSLverifyDataMax] directive, or the global WASD_CONFIG_GLOBAL configuration directive [SSLverifyDataMax].

Where a request with a body exceeds the maximum allowed buffer space the authorisation fails. This can be observed using WATCH. Where very large files are being sent the only solution is to first authenticate with a request without a body (e.g. using OPTIONS) then using the persistent connection and associated X.509 authentication perform the PUT or POST.

4.5.14Features

WASD provides a range of capabilities when using X.509 client certificates.

4.5.15Subject Alternative Name and Other Extensions

The basic syntax for this field is the full extension name, and the short-hand equivalent.

[X509] /VMS/* r+w,param="[ru:X509v3_subject_Alternative_Name]" /VMS/* r+w,param="[ru:X509v3_SAN]"

The Subject Alternative Name (SAN) extension (in common with many others) may contain multiple data elements, each with a leading name, a colon, and a (if multi line) carriage-control terminated value. WASD parses these into unqiue fields using keywords fixed in function SesolaCertKeyword() and the site configurable logical name WASD_X509_EXTENSION_KEYWORDS value. To select one of these fields, for example the common (Microsoft) user principal name (UPN), append the required field name to the extension name as shown in the following example (includes "shorthand" equivalents, along with the underscore and equate variants). Note that the identifying name match is not case sensitive.

[X509] /VMS/* r+w,param="[ru:X509V3_Subject_Alternative_Name_UserPrincipalName]" /VMS/* r+w,param="[ru:X509V3_Subject_Alternative_Name=UserPrincipalName]" /VMS/* r+w,param="[ru:X509v3_SAN_UPN]" /VMS/* r+w,param="[ru:X509v3_SAN=UPN]" /VMS/* r+w,param="[ru:X509V3_Subject_Alternative_Name_rfc822Name]" /VMS/* r+w,param="[ru:X509V3_Subject_Alternative_Name=rfc822Name]" /VMS/* r+w,param="[ru:X509v3_SAN_822]" /VMS/* r+w,param="[ru:X509v3_SAN=822]"

Object Identifiers (OIDs) may be used for either record and field name (if an unknown otherName) by prefixing with "OID_". For example, the SAN may be alternatively selected, and the (Microsoft) UPN, as in the following examples.

/VMS/* r+w,param="[ru:OID_2_5_29_17]" /VMS/* r+w,param="[ru:OID_2_5_29_17_UPN]" /VMS/* r+w,param="[ru:OID_2_5_29_17=UPN]" /VMS/* r+w,param="[ru:X509v3_SAN_OID_1_3_6_1_20_2_3]" /VMS/* r+w,param="[ru:X509v3_SAN_OID=1_3_6_1_20_2_3]"
Extension Visibility

X509 certificate extensions are in general visible from WATCH and accessible via CGI variables (when enabled using SET SSLCGI=apache_mod_ssl_extens and SSLCGI=apache_mod_ssl_client path mappings). The identifying names derived from X509 extensions are built of the alphanumerics in the element names. Non-alphanumerics (e.g. spaces) have underscores substituted. Multiple underscores are compressed into singles. Where elements have identical names the first multiple has TWO underscores and the digit two appended, the second mutiple, two underscores and three appended, etc.

4.5.16X509 Configuration

Of course, the WASD OpenSSL component must be installed and in use to apply client X.509 certificate authorization. There is general server setup, then per-service and per-resource configuration.

General Setup

Client certificate authorization has reasonable defaults. If some aspect requires site refinement the WASD_CONFIG_GLOBAL [SSL..] directives (see WASD Configuration) or command-line /SSL= qualifier parameters can provide per-server defaults.

The location of the CA verification file can also be determined using the logical name WASD_CONFIG_SSL_CAFILE. The order of precedence for using these specifications is

  1. per-service configuration using WASD_CONFIG_SERVICE or WASD_CONFIG_GLOBAL
  2. per-server using /SSL=CAFILE=filename
  3. per-server using WASD_CONFIG_SSL_CAFILE
By Service

The WASD_CONFIG_SERVICE directive is provided for per-service CA file specification, if necessary allowing different services to accept a different mix of CAs.

[[https://the.example.com:443]] [ServiceSSLVerifyPeer] enabled [ServiceSSLVerifyPeerCAfile] WASD_ROOT:[LOCAL]CA_THE_HOST_NAME.TXT
By Resource

Client certificate authorization is probably most usefully applied on a per-resource (per-request-path) basis using WASD_CONFIG_AUTH configuration file rules. Of course, per-resource control also applies to services that always require a client certificate (the only difference is the certificate has already been negotiated for during the initial connection handshake). The reserved realm name "X509" activates client certificate authentication when a rule belonging to that realm is triggered. The following example shows such a rule providing read access to those possessing any verified certificate.

[X509] /path/requiring/cert/* r

Optional directives may be supplied to the X.509 authenticator controlling what mode the certificate is accepted in, as well a further access-restriction rules on specifically which certificates may or may not be accepted for authorization. Such directives are passed via the "param=" mechanism. The following real-life example shows a script path requiring a mandatory certificate, but not necessarily having the CA verified. This would allow a certificate display service to be established, the "[to:EXPIRED]" directive forcing the client to explicitly select a certificate with each access.

[X509] /cgi-bin/client_cert_details r,param="[vf:OPTIONAL][to:EXPIRED]"

A number of such directives are available controlling some aspects of the certificate negotiation and verification. The "[LT:integer]" directive causes a verified certificate selection to continue to be valid for the specified period as long as requests continue during that period (lifetime is reset with each access).

Optional "param=" passed conditionals may also be used to provide additional filtering on which certificates may or may not be used against the particular path. This is based on pattern matching against client certificate components.

These functions can be used in a similar fashion to mapping rule conditionals (see Conditional Configuration of WASD Configuration). This includes the logical ORing, ANDing and negating of conditionals. Asterisk wildcards match any zero or more characters, percent characters any single character. Matching is case-insensitive.

Note that the "IS:" and "SU:" conditionals each have a specific-record and an entire-field mode. If the conditional string begins with a slash then it is considered to be a match against a specified record contents within the field. If it begins with a wildcard then it is matched against the entire field contents. Certificate DN records recognised by WASD,

The following (fairly contrived) examples provide an illustration of the basics of X509 conditionals. When matching against Issuer and Subject DNs some knowlege of their contents and structure is required (see 4. Transport Layer Security for some basic resources).

[X509] # only give "VeriSign"ed ones access /controlled/path1/* r+w,param="[IS:/O=VeriSign\ Inc.]" # only give non-"VeriSign"ed ones access /controlled/path2/* r+w,param="[!IS:/O=VeriSign\ Inc.]" # only allow 128 bit keys using RC4-MD5 access /controlled/path3/* r+w,param="[KS:128][CI:RC4-MD5]" # only give a "Thawte"-signed client based in Australia # with the following email address access /controlled/path4/* r+w,param="\ [IS:*/O=Thawte\ Consulting\ cc/*]\ [SU:*/C=AU/*/Email=mark.daniel@wasd.vsm.com.au*]" # use the subject DN common-name record as the remote-user name # furthermore, restrict the CA's allowed to be used this way /VMS/* r+w,param="[RU:/CN=][IS:/O=WASD\ CA\ Cert]"

Of course, access control via group membership is also available. The effective username for the list is the 32 digit fingerprint of the client certificate (shown as REMOTE_USER IN the first example of 4.5.18 X.509 Authorization CGI Variables), or the Subject DN record as specified using the [RU:/record=] directive. This may be entered into simple lists as part of a group of which membership then controls access to the resource. The following examples show the contents of simple list files containing the X.509 fingerprints, derived remote-user names, and the required WASD_CONFIG_AUTH realm entries.

# FINGERPRINTS.$HTL # (a file of X.509 fingerprints for access to "/path/requiring/cert/") 106C8342890A1703AAA517317B145BF7 mark.daniel@wasd.vsm.com.au 6ADA07108C20338ADDC3613D6D8B159D just.another@where.ever.com # CERT_CN.$HTL # (a file of X.509 remote-user names derived using [RU:/CN=] Mark_Daniel mark.daniel@wasd.vsm.com.au Just_Another just.another@where.ever.com [X509;FINGERPRINTS=list] /path/requiring/cert/* r+w [X509;CERT_CN=list] /path/requiring/cn/* r+w

In a similar fashion the effective username can be placed in an access restriction list. The following configuration would only allow the user of the certificate access to the specified resources. Other verified certificate holders would be denied access.

[X509] /httpd/-/admin/* ~106C8342890A1703AAA517317B145BF7,r+w /wasd_root/local/* ~106C8342890A1703AAA517317B145BF7,r+w /other/path/* ~Mark_Daniel,r+w,param="[ru:/cn=]" /yet/another/path/* ~Just_Another,r+w,param="[ru:/cn=]"

4.5.17Certificate Authority Verification File

For the CA certificate component of the client certificate to be verified as being what it claims to be (and thus establishing the integrity of the client certificate) a list of such certificates must be provided for comparison purposes. For WASD this list is contained in a single, plain-text file variously specified using either the WASD_CONFIG_SSL_CAFILE logical or per-service "[ServiceSSLclientCAfile]" directives, or the global [SSLverifyPeerCAFile] directive.

Copies of CA certificates are available for such purposes. The PEM copies (base-64 encoded versions of the binary certificate) can be placed into this file using any desired text editor. Comments may be inserted by prefixing with the "#" character. For WASD this would be best stored in the WASD_ROOT:[LOCAL] directory, or site equivalent.

An example of how such a file appears is provided below (bulk of the file has been 8< snipped 8< for bevity).

## ## Bundle of CA Root Certificates ## ## Certificate data from Mozilla as of: Wed Jan 18 04:12:05 2017 GMT ## ## This is a bundle of X.509 certificates of public Certificate Authorities ## (CA). These were automatically extracted from Mozilla's root certificates ## file (certdata.txt). This file can be found in the mozilla source tree: ## https://hg.mozilla.org/releases/mozilla-release/raw-file/default/security/nss/lib/ckfw/builtins/certdata.txt ## ## It contains the certificates in PEM format and therefore ## can be directly used with curl / libcurl / php_curl, or with ## an Apache+mod_ssl webserver for SSL client authentication. ## Just configure this file as the SSLCACertificateFile. ## ## Conversion done with mk-ca-bundle.pl version 1.27. ## SHA256: dffa79e6aa993f558e82884abf7bb54bf440ab66ee91d82a27a627f6f2a4ace4 ## GlobalSign Root CA ================== -----BEGIN CERTIFICATE----- MIIDdTCCAl2gAwIBAgILBAAAAAABFUtaw5QwDQYJKoZIhvcNAQEFBQAwVzELMAkGA1UEBhMCQkUx GTAXBgNVBAoTEEdsb2JhbFNpZ24gbnYtc2ExEDAOBgNVBAsTB1Jvb3QgQ0ExGzAZBgNVBAMTEkds b2JhbFNpZ24gUm9vdCBDQTAeFw05ODA5MDExMjAwMDBaFw0yODAxMjgxMjAwMDBaMFcxCzAJBgNV BAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNhMRAwDgYDVQQLEwdSb290IENBMRswGQYD VQQDExJHbG9iYWxTaWduIFJvb3QgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDa DuaZjc6j40+Kfvvxi4Mla+pIH/EqsLmVEQS98GPR4mdmzxzdzxtIK+6NiY6arymAZavpxy0Sy6sc THAHoT0KMM0VjU/43dSMUBUc71DuxC73/OlS8pF94G3VNTCOXkNz8kHp1Wrjsok6Vjk4bwY8iGlb Kk3Fp1S4bInMm/k8yuX9ifUSPJJ4ltbcdG6TRGHRjcdGsnUOhugZitVtbNV4FpWi6cgKOOvyJBNP c1STE4U6G7weNLWLBYy5d4ux2x8gkasJU26Qzns3dLlwR5EiUWMWea6xrkEmCMgZK9FGqkjWZCrX gzT/LCrBbBlDSgeF59N89iFo7+ryUp9/k5DPAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNV HRMBAf8EBTADAQH/MB0GA1UdDgQWBBRge2YaRQ2XyolQL30EzTSo//z9SzANBgkqhkiG9w0BAQUF AAOCAQEA1nPnfE920I2/7LqivjTFKDK1fPxsnCwrvQmeU79rXqoRSLblCKOzyj1hTdNGCbM+w6Dj Y1Ub8rrvrTnhQ7k4o+YviiY776BQVvnGCv04zcQLcFGUl5gE38NflNUVyRRBnMRddWQVDf9VMOyG j/8N7yy5Y0b2qvzfvGn9LhJIZJrglfCm7ymPAbEVtQwdpf5pLGkkeB6zpxxxYu7KyJesF12KwvhH hm4qxFYxldBniYUr+WymXUadDKqC5JlR3XC321Y9YeRq4VzW9v493kHMB65jUr9TU/Qr6cf9tveC X4XSQRjbgbMEHMUfpIBvFSDJ3gyICh3WZlXi/EjJKSZp4A== -----END CERTIFICATE----- 8< snip 8<

The WASD OpenSSL package provides an example CA verification file. The exact date and source can be found in the opening commentary of the file itself. The contents of this file easily can be pared down to the minimum certificates required for any given site.

The bundle may be refreshed at any time using any reliable source. The cURL project provides such a resource suitable for its own use, Apache mod_ssl and WASD. This is sourced from the root certificates used by the Mozilla Foundation for its Firefox product (and others). Mozilla uses a non-PEM format source which must be converted before use by WASD. The cURL site provides this already converted for use with its own utility and made available as a general resource.

Download the bundle using a command-line tool as in this example

curl -o ca-bundle_crt.txt https://curl.se/ca/cacert.pem
or as a save-as dialogue click from your favourite browser and then a transfer onto the VMS system.

4.5.18X.509 Authorization CGI Variables

CGI variables specific to client certificate authorization are always generated for use by scripts and SSI documents. These along with the general WASD authorization variables are shown in the example below. Note, that due to length of particular items some in this example are displayed wrapped.

WWW_AUTH_ACCESS == "READ+WRITE" WWW_AUTH_GROUP == "" WWW_AUTH_REALM == "X509" WWW_AUTH_REALM_DESCRIPTION == "X509 Client Certs" WWW_AUTH_TYPE == "X509" WWW_AUTH_USER == "Mark Daniel, mark.daniel@wasd.vsm.com.au" WWW_AUTH_X509_CIPHER == "RC4-MD5" WWW_AUTH_X509_FINGERPRINT == "10:6C:83:42:89:0A:17:03:AA:A5:17:31:7B:14:5B:F7" WWW_AUTH_X509_ISSUER == "/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=www.verisign.com/repository/RPA Incorp. By Ref.,LIAB.LTD(c)98/CN=VeriSign Class 1 CA Individual Subscriber-Persona Not Validated" WWW_AUTH_X509_KEYSIZE == "128" WWW_AUTH_X509_SUBJECT == "/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=www.verisign.com/repository/RPA Incorp. by Ref.,LIAB.LTD(c)98/OU=Persona Not Validated/OU=Digital ID Class 1 - Netscape /CN=Mark Daniel/Email=mark.daniel@wasd.vsm.com.au" WWW_REMOTE_USER == "106C8342890A1703AAA517317B145BF7"

Other CGI variables optionally may be enabled using WASD_CONFIG_MAP mapping rules. See 4.5.18 X.509 Authorization CGI Variables. Specific client certificate variables providing the details of such certificates are available with SSLCGI=apache_mod_ssl. These are of course in addition to the more general apache_mod_ssl variables described in the above section. Note that where some ASN.1 records are duplicated (as in SSL_CLIENT_S_DN) some variables will contain newline characters (0x10) between those elements (e.g. SSL_CLIENT_S_DN_OU). The line breaks in this example do not necesarily reflect those characters.

WWW_SSL_CIPHER == "TLS_AES_256_GCM_SHA384" WWW_SSL_CIPHER_ALGKEYSIZE == "256" WWW_SSL_CIPHER_USEKEYSIZE == "256" WWW_SSL_PROTOCOL == "TLSv1.3" WWW_SSL_SERVER_A_KEY == "rsaEncryption" WWW_SSL_SERVER_A_SIG == "sha256WithRSAEncryption" WWW_SSL_SERVER_E_AUTHORITY_INFORMATION_ACCESS == "OCSP - URI:http://ocsp.int-x3.letsencrypt.org.CA Issuers 8< snip 8< WWW_SSL_SERVER_E_AUTHORITY_INFORMATION_ACCESS_URI == "http://ocsp.int-x3.letsencrypt.org" WWW_SSL_SERVER_E_AUTHORITY_INFORMATION_ACCESS_URI__2 == "http://cert.int-x3.letsencrypt.org/" WWW_SSL_SERVER_E_CT_PRECERTIFICATE_SCTS == "Signed Certificate Timestamp:. Version : v1 (0x0). Log ID : 8< snip 8< WWW_SSL_SERVER_E_X509V3_AUTHORITY_KEY_IDENTIFIER == "keyid:A8:4A:6A:63:04:7D:DD:BA:E6:D1:39:B7:A6:45:65:EF:F3:A8:EC:A1." WWW_SSL_SERVER_E_X509V3_AUTHORITY_KEY_IDENTIFIER_KEYID == "A8:4A:6A:63:04:7D:DD:BA:E6:D1:39:B7:A6:45:65:EF:F3:A8:EC:A1" WWW_SSL_SERVER_E_X509V3_BASIC_CONSTRAINTS == "CA:FALSE" WWW_SSL_SERVER_E_X509V3_BASIC_CONSTRAINTS_CA == "FALSE" WWW_SSL_SERVER_E_X509V3_CERTIFICATE_POLICIES == "Policy: 2.23.140.1.2.1.Policy: 1.3.6.1.4.1.44947.1.1.1. 8< snip 8< WWW_SSL_SERVER_E_X509V3_CERTIFICATE_POLICIES_CPS == " http://cps.letsencrypt.org" WWW_SSL_SERVER_E_X509V3_CERTIFICATE_POLICIES_POLICY == " 2.23.140.1.2.1" WWW_SSL_SERVER_E_X509V3_CERTIFICATE_POLICIES_POLICY__2 == " 1.3.6.1.4.1.44947.1.1.1" WWW_SSL_SERVER_E_X509V3_EXTENDED_KEY_USAGE == "TLS Web Server Authentication, TLS Web Client Authentication" WWW_SSL_SERVER_E_X509V3_KEY_USAGE == "Digital Signature, Key Encipherment" WWW_SSL_SERVER_E_X509V3_SAN == "dNSName:the.host.name..dNSName:the.host.name" WWW_SSL_SERVER_E_X509V3_SUBJECT_ALTERNATIVE_NAME == "dNSName:the.host.name..dNSName:the.host.name" WWW_SSL_SERVER_E_X509V3_SUBJECT_KEY_IDENTIFIER == "4E:6A:0B:56:F0:EF:1B:1E:71:E1:33:53:A0:39:32:D3:0C:D6:3C:0C" WWW_SSL_SERVER_I_DN == "/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3" WWW_SSL_SERVER_I_DN_C == "US" WWW_SSL_SERVER_I_DN_CN == "Let's Encrypt Authority X3" WWW_SSL_SERVER_I_DN_O == "Let's Encrypt" WWW_SSL_SERVER_M_SERIAL == "03AC67E421D5E26AA843A14F50343FEB1F84" WWW_SSL_SERVER_M_VERSION == "3" WWW_SSL_SERVER_S_DN == "/CN=the.host.name" WWW_SSL_SERVER_S_DN_CN == "the.host.name" WWW_SSL_SERVER_V_END == "Jul 17 13:50:24 2020 GMT" WWW_SSL_SERVER_V_START == "Apr 18 13:50:24 2020 GMT" WWW_SSL_SESSION_ID == "533d71a813a1ee8c5c68ae30c4cd05ac3b673ee9b04ac04567cad18418730dfe" WWW_SSL_TLS_ALPN == "h2" WWW_SSL_TLS_SNI == "the.host.name" WWW_SSL_VERSION_INTERFACE == "HTTPd-WASD/11.5.0 OpenVMS/AXP SSL" WWW_SSL_VERSION_LIBRARY == "OpenSSL 1.1.1c 28 May 2019"

4.6Certificate Management

This is not a tutorial on X.509 certificates and their management. Refer to the listed references, 4. Transport Layer Security, for further information on this aspect. It does provide some basic guidelines.

Certificates identify something or someone, associating a public cryptographic key with the identity of the certificate holder. It includes a distinguished name, identification and signature of the certificate authority (CA, the issuer and guarantor of the certificate), and the period for which the certificate is valid, possibly with other, additional information.

The three types of certificates of interest here should not be confused.

The various OpenSSL tools are available for management of all of these certificate types in each of the three SSL environments.

4.6.1Server Certificate

The server uses a certificate to establish its identity during the initial phase of the SSL protocol exchange. Each server should have a unique certificate. An example certificate is provided with the WASD OpenSSL package. If this is not available (for instance when using the VSI SSL111 product) then the server will fallback to an internal, default certificate that allows SSL functionality even when no external certification is available. If a "live" SSL site is required a unique certificate issued by a third-party Certificate Authority is desirable.

Let's Encrypt

Self-signing certificates as described below has a number of shortcomings for general web server certification. Fortunately Let's Encrypt makes it possible automatically to obtain and maintain a browser-trusted certificate, simply, and at no cost. This is accomplished by running a certificate management agent on the web server. The WASD Certificate Management Environment (wuCME) may be used to perform this function on VMS.

See wuCME on the WASD download page at https://wasd.vsm.com.au/wasd/


Self-Signed Certificates

A less satisfactory alternative to obtaining one of these certificates is provided by the WASD support DCL procedures, which are quick hacks to ease the production of certificates on an ad hoc basis. In all cases it is preferable to directly use the utilities provided with OpenSSL, but the documentation tends to be rather sparse.

The VSI SSL111$COM:SSL111$CERT_TOOL.COM described above can create self-signed certificates.

Also note that the WASD server dynamically generates a self-signed certificate for TLS services that otherwise do not have a configured server certificate. This is largely for testing a server immediately after installation (e.g. using @WASD_ROOT:[INSTALL]DEMO SSL at the command-line). This certificate suffers all the short-comings of self-signed certificates with modern browsers (post-2019) but is better than no certificate all all. Interestingly, Incognito/[In]Private instances of a browser are often more relaxed about accepting certificates with recognised security deficiencies (e.g. unknown Certificate Authority signing). At least at the time of writing.

Loading Authority Certificates

The first requirement may be a tailored "Certificate Authority" certificate. As the Certificate Authority is non-authoritative (not trying to be too oxymoronic, i.e. not a well-known CA) these certificates have little value except to allow SSL transactions to be established with trusting clients. More commonly "Server Certificates" for specific host names are required.

CA certificates can be loaded into browsers to allow sites using that CA to be accessed by that browser without further dialog. Browsers commonly invoke a server certificate load dialog when encountering a site using a valid but unknown server certificate.

A manual load is accomplished by requesting the certificate in a format appropriate to the particular browser. This triggers a browser dialog with the user to confirm or refuse the loading of that certificate into the browser Certificate Authority database.

To facilitate loading CA certificates into a browser ensure the following entries are contained in the HTTP$CONFIG configuration file:

[AddIcon] /httpd/-/binary.gif [BIN] application/x-x509-ca-cert [AddType] .CRT application/x-x509-ca-cert - DER certifcate (MSIE) .PEM application/x-x509-ca-cert - Privacy Enhanced Mail certificate

Then just provide a link to the required certificate file(s), and click.

Changing Server Certificates

If a site's server (or CA certificate) is changed and the server restarted any executing browsers will probably complain (Netscape Navigator reports an I/O error). In this case open the browser's certificate database and delete any relevant, permanently stored certificate entry, then close and restart the browser. The next access should initiate the server certificate dialog, or the CA certificate may be explicitly reloaded.

4.6.2Certificate Signing Request

Recognised Certificate Authorities (CAs) such as Thawte and VeriSign publish lists of requirements for obtaining a server certificate. These often include such documents required to prove organisational name and the right to use the domain name being requested. Check the particular vendor for the exact requirements.

In addition, a document containing the site's private key is required. This is known as the Certificate Signing Request (CSR) and must be generated digitally at the originating site.

Using the VSI SSL111 for OpenVMS product "SSL Certificate Tool" described in 4.6 Certificate Management a CSR can easily be generated using its menu-driven interface. The alternative is using a command-line interface tool.

The following instructions provide the basics for generating a CSR at the command-line in the WASD and generally the any OpenSSL environment (including the VSI SSL111 for OpenVMS product).

  1. Change to a secure directory. The following is a suggestion.
    $ SET DEFAULT WASD_ROOT:[LOCAL]
  2. Assign a foreign verb for the OPENSSL application. The location may vary a little depending on which OpenSSL package you have installed. See 4.4 OPENSSL.EXE Application.
  3. Specify a source of lots of "random" data (can be any big file for the purposes of this exercise).
    $ RANDFILE = "WASD_EXE:HTTPD_SSL.EXE"
  4. Find the template configuration file. You will need to specify this location in a step described below. Should be something like the following.
    WASD_ROOT:[SRC.OPENSSL-version.WASD]TEMPLATE.CNF
  5. Generate your private key (RANDFILE data is used by this). The output from this looks something like what's shown. Notice the pass phrase prompts. This is your private key, don't forget it!
    $ OPENSSL GENRSA -DES3 -OUT SERVER.KEY 1024 Generating RSA private key, 1024 bit long modulus .....++++++ ......++++++ e is 65537 (0x10001) Enter PEM pass phrase: Verifying password - Enter PEM pass phrase:
  6. Generate the Certificate Signing Request using syntax similar to the following (this is where you are required to specify the location of the configuration template). Note that there are quite a few fields - GET THEM RIGHT! They need to be unique and local - they're your distinguishing name (DN). "Common Name" is the host you want the certificate for. It can be a fully qualifier host name (e.g. "klaatu.local.net"), or a local wildcard (e.g. "*.local.net") for which you may pay more.
    $ OPENSSL REQ -NEW -KEY SERVER.KEY -OUT SERVER.CSR -CONFIG - WASD_ROOT:[SRC.OPENSSL-0_9_6B.WASD]TEMPLATE.CNF Using configuration from template.cnf Enter PEM pass phrase: You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [AU]:AU State or Province Name (full name) [Some-State]:South Australia Locality Name (eg, city) []:Adelaide Organization Name (eg, company) [Internet Widgits Pty Ltd]:Example Organizational Unit Name (eg, section) []:WASD Common Name (eg, YOUR name) []:klaatu.local.net Email Address []:Mark.Daniel@wasd.vsm.com.au Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []:
  7. That's it! You should have two files in your default directory.
    SERVER.CSR;1 2 14-MAR-2002 04:38:26.15 SERVER.KEY;1 2 14-MAR-2002 04:31:38.76

    Keep the SERVER.KEY file secure. You'll need it when you receive the certificate back from the CA.

    The SERVER.CSR is what you send to the CA (usually by mail or Web form). It looks something like the following

    $ TYPE SERVER.CSR -----BEGIN CERTIFICATE REQUEST----- MIIBPTCB6AIBADCBhDELMAkGA1UEBhMCWkExFTATBgNVBAgTDFdlc3Rlcm4gQ2Fw ZTESMBAGA1UEBxMJQ2FwZSBUb3duMRQwEgYDVQQKEwtPcHBvcnR1bml0aTEYMBYG A1UECxMPT25saW5lIFNlcnZpY2VzMRowGAYDVQQDExF3d3cuZm9yd2FyZC5jby56 YTBaMA0GCSqGSIb3DQEBAQUAA0kAMEYCQQDT5oxxeBWu5WLHD/G4BJ+PobiC9d7S 6pDvAjuyC+dPAnL0d91tXdm2j190D1kgDoSp5ZyGSgwJh2V7diuuPlHDAgEDoAAw DQYJKoZIhvcNAQEEBQADQQBf8ZHIu4H8ik2vZQngXh8v+iGnAXD1AvUjuDPCWzFu pReiq7UR8Z0wiJBeaqiuvTDnTFMz6oCq6htdH7/tvKhh -----END CERTIFICATE REQUEST-----

    You can see the details of this file using

    $ OPENSSL RSA -NOOUT -TEXT -IN SERVER.CSR
After Receiving The Certificate

Once the signed certificate has been issued by the Certificate Authority it can be placed directly into the server configuration directory, usually WASD_ROOT:[LOCAL], and configured for use from there. Using the certificate direct from the CA requires that the private key password be given to the server each time (4.5.9 SSL Private Key). It is possible to embed the password into the certificate key so that this is not required.

Remember to keep original files secure, only work on copies!

  1. Assign a foreign verb for the OPENSSL application. The location may vary a little depending on which OpenSSL package you have installed.
    $ OPENSSL == "$WASD_ROOT:[SRC.OPENSSL-version.AXP.EXE.APPS]OPENSSL.EXE"

    When using the VSI SSL111 product or other OpenSSL toolkit the verb may already be available.

    $ SHOW SYMBOL OPENSSL OPENSSL == "$ SSL111$EXE:OPENSSL"
  2. Go to wherever you want to do the work.
    $ SET DEFAULT WASD_ROOT:[LOCAL]
  3. You may require these additional steps (based on user experience):
  4. Using the original key file embed your password into a copy. When prompted "Enter PEM pass phrase:" enter the password.
    $ OPENSSL rsa -in SERVER.KEY -out WORK.PEM
  5. Append this password-embedded key file to your certificate file.
    $ COPY CERTIFICATE.PEM,WORK.PEM CERTIFICATE.PEM;0
  6. Delete the temporary file.
    $ DELETE WORK.PEM;*

4.7SSL CGI Variables

CGI variables specific to SSL transactions optionally may be enabled using WASD_CONFIG_MAP mapping rules. (See Request Processing Configuration of WASD Configuration). The may be done on a specific per-path or general CGI basis. In the following examples, due to length of particular items, some in this example are displayed wrapped. Also, where some ASN.1 records are duplicated (as in SSL_CLIENT_S_DN), some variables will contain newline characters (0x10) between those elements (e.g. SSL_CLIENT_S_DN_OU). The line breaks in the examples do not necesarily reflect those characters.

set /path/* SSLCGI=apache_mod_ssl

WWW_SSL_CIPHER == "TLS_AES_256_GCM_SHA384" WWW_SSL_CIPHER_ALGKEYSIZE == "256" WWW_SSL_CIPHER_USEKEYSIZE == "256" WWW_SSL_PROTOCOL == "TLSv1.3" WWW_SSL_SERVER_A_KEY == "rsaEncryption" WWW_SSL_SERVER_A_SIG == "sha256WithRSAEncryption" WWW_SSL_SERVER_E_AUTHORITY_INFORMATION_ACCESS == "OCSP - URI:http://ocsp.int-x3.letsencrypt.org.CA Issuers 8< snip 8< WWW_SSL_SERVER_E_AUTHORITY_INFORMATION_ACCESS_URI == "http://ocsp.int-x3.letsencrypt.org" WWW_SSL_SERVER_E_AUTHORITY_INFORMATION_ACCESS_URI__2 == "http://cert.int-x3.letsencrypt.org/" WWW_SSL_SERVER_E_CT_PRECERTIFICATE_SCTS == "Signed Certificate Timestamp:. Version : v1 (0x0). Log ID : 8< snip 8< WWW_SSL_SERVER_E_X509V3_AUTHORITY_KEY_IDENTIFIER == "keyid:A8:4A:6A:63:04:7D:DD:BA:E6:D1:39:B7:A6:45:65:EF:F3:A8:EC:A1." WWW_SSL_SERVER_E_X509V3_AUTHORITY_KEY_IDENTIFIER_KEYID == "A8:4A:6A:63:04:7D:DD:BA:E6:D1:39:B7:A6:45:65:EF:F3:A8:EC:A1" WWW_SSL_SERVER_E_X509V3_BASIC_CONSTRAINTS == "CA:FALSE" WWW_SSL_SERVER_E_X509V3_BASIC_CONSTRAINTS_CA == "FALSE" WWW_SSL_SERVER_E_X509V3_CERTIFICATE_POLICIES == "Policy: 2.23.140.1.2.1.Policy: 1.3.6.1.4.1.44947.1.1.1. 8< snip 8< WWW_SSL_SERVER_E_X509V3_CERTIFICATE_POLICIES_CPS == " http://cps.letsencrypt.org" WWW_SSL_SERVER_E_X509V3_CERTIFICATE_POLICIES_POLICY == " 2.23.140.1.2.1" WWW_SSL_SERVER_E_X509V3_CERTIFICATE_POLICIES_POLICY__2 == " 1.3.6.1.4.1.44947.1.1.1" WWW_SSL_SERVER_E_X509V3_EXTENDED_KEY_USAGE == "TLS Web Server Authentication, TLS Web Client Authentication" WWW_SSL_SERVER_E_X509V3_KEY_USAGE == "Digital Signature, Key Encipherment" WWW_SSL_SERVER_E_X509V3_SAN == "dNSName:the.host.name..dNSName:the.host.name" WWW_SSL_SERVER_E_X509V3_SUBJECT_ALTERNATIVE_NAME == "dNSName:the.host.name..dNSName:the.host.name" WWW_SSL_SERVER_E_X509V3_SUBJECT_KEY_IDENTIFIER == "4E:6A:0B:56:F0:EF:1B:1E:71:E1:33:53:A0:39:32:D3:0C:D6:3C:0C" WWW_SSL_SERVER_I_DN == "/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3" WWW_SSL_SERVER_I_DN_C == "US" WWW_SSL_SERVER_I_DN_CN == "Let's Encrypt Authority X3" WWW_SSL_SERVER_I_DN_O == "Let's Encrypt" WWW_SSL_SERVER_M_SERIAL == "03AC67E421D5E26AA843A14F50343FEB1F84" WWW_SSL_SERVER_M_VERSION == "3" WWW_SSL_SERVER_S_DN == "/CN=the.host.name" WWW_SSL_SERVER_S_DN_CN == "the.host.name" WWW_SSL_SERVER_V_END == "Jul 17 13:50:24 2020 GMT" WWW_SSL_SERVER_V_START == "Apr 18 13:50:24 2020 GMT" WWW_SSL_SESSION_ID == "533d71a813a1ee8c5c68ae30c4cd05ac3b673ee9b04ac04567cad18418730dfe" WWW_SSL_TLS_ALPN == "h2" WWW_SSL_TLS_SNI == "the.host.name" WWW_SSL_VERSION_INTERFACE == "HTTPd-WASD/11.5.0 OpenVMS/AXP SSL" WWW_SSL_VERSION_LIBRARY == "OpenSSL 1.1.1c 28 May 2019"

The Apache mod_ssl client certificate details described in 4.5.18 X.509 Authorization CGI Variables above are not shown in the above example but would be included if the request was X.509 authenticated.

X509 certificate extensions are in general visible from WATCH and accessible via CGI variables when enabled using SET SSLCGI=apache_mod_ssl_extens and SSLCGI=apache_mod_ssl_client path mappings.

4.8SSL Service Evaluation

This section is just the barest introduction to a significant topic.

Qualys SSL Lab

"How well do you know SSL? If you want to learn more about the technology that protects the Internet, you've come to the right place."

https://www.ssllabs.com/

Not necessarily an endorsement by WASD but a useful resource in itself.

Provides a free and unencumbered, comprehensive SSL Server test service

https://www.ssllabs.com/ssltest/

reporting on certificate status, protocol version, cipher suites, handshakes with various simulated clients, and protocol details including known vulnerabilities. It also summarises the report with a colour-coded rating.

At Home

So to speak.

The OPENSSL command-line application (4.4 OPENSSL.EXE Application) provides a configurable client for checking and testing various aspects of server configuration and behaviour. The basic operation represented by the command-line

$ openssl s_client -host host name or address> -port 443
provides a comprehensive report including certificates and certificate chain, the protocol version and cipher negotiated, along with more esoteric elements of TLS/SSL. Some data have been 8< snipped 8< for brevity in the following example.
$ openssl s_client -host klaatu.private -port 443 WARNING: can't open config file: SSLROOT:[000000]openssl.cnf CONNECTED(00000003) depth=0 C = AU, ST = SA, L = Adelaide, O = WASD Server Cert, OU 8< snip 8< verify error:num=20:unable to get local issuer certificate verify return:1 depth=0 C = AU, ST = SA, L = Adelaide, O = WASD Server Cert, OU 8< snip 8< verify error:num=27:certificate not trusted verify return:1 depth=0 C = AU, ST = SA, L = Adelaide, O = WASD Server Cert, OU 8< snip 8< verify error:num=21:unable to verify the first certificate verify return:1 --- Certificate chain 0 s:/C=AU/ST=SA/L=Adelaide/O=WASD Server Cert/OU=OpenSSL 1.0.1 8< snip 8< i:/C=AU/ST=SA/L=Adelaide/O=WASD CA Cert/OU=OpenSSL 1.0.1j Te 8< snip 8< --- Server certificate -----BEGIN CERTIFICATE----- MIIFsjCCBJqgAwIBAgIBBDANBgkqhkiG9w0BAQQFADCBtjELMAkGA1UEBhMCQVUx 8< snip 8< pErvrfr69iDbJbhO+mRmIkZIXHc5CFV/M1zzLD5240ixxu/d6nAUBhGba0W4Kste x1SgLJ0BqFTjegxuHRXkK5lOlY11Hw== -----END CERTIFICATE----- subject=/C=AU/ST=SA/L=Adelaide/O=WASD Server Cert/OU=OpenSSL 1. 8< snip 8< issuer=/C=AU/ST=SA/L=Adelaide/O=WASD CA Cert/OU=OpenSSL 1.0.1j 8< snip 8< --- No client certificate CA names sent --- SSL handshake has read 1791 bytes and written 625 bytes --- New, TLSv1/SSLv3, Cipher is AES256-GCM-SHA384 Server public key is 2048 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE SSL-Session: Protocol : TLSv1.2 Cipher : AES256-GCM-SHA384 Session-ID: 61FEC1629DA3E675AA124223CDB9CB5AB7701D872E85E15 8< snip 8< Session-ID-ctx: Master-Key: F4260DFE9A7370B3EA85D22D89DB8A7925C655159C3C509 8< snip 8< Key-Arg : None PSK identity: None PSK identity hint: None SRP username: None TLS session ticket lifetime hint: 300 (seconds) TLS session ticket: 0000 - 63 d6 2a 84 19 fe f6 9a-13 60 e1 8a 65 dd f9 fc c.*......`..e... 8< snip 8< 00a0 - 9a 2d 29 9b 8e aa ab 69-11 0d 45 ed 63 48 f5 4f .-)....i..E.cH.O Start Time: 1415828121 Timeout : 300 (sec) Verify return code: 21 (unable to verify the first certificate) --- 8< snip 8<

A "bad select 38" is a VMS (C-RTL) limitation of earlier versions of OpenSSL and is not present in later versions or on other platforms, and the default use of -s_client will prompt for an HTTP request line, send that to the server, and report the response.

Checking whether a specific protocol version is enabled on a site:

$ openssl s_client -ssl2 -host host name or address> -port 443 $ openssl s_client -ssl3 -host host name or address> -port 443 $ openssl s_client -tls1 -host host name or address> -port 443 $ openssl s_client -tls1_1 -host host name or address> -port 443 $ openssl s_client -tls1_2 -host host name or address> -port 443 $ openssl s_client -tls1_3 -host host name or address> -port 443

The following example shows a server test where the protocol version is NOT supported.

$ openssl s_client -ssl3 -host klaatu.private -port 443 8< snip 8< SSL handshake has read 7 bytes and written 0 bytes --- New, (NONE), Cipher is (NONE) Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE SSL-Session:style="background-color:yellow"> Protocol : SSLv3 Cipher : 0000 8< snip 8<
TLS Version 1.3
test TLS Version 1.3

Server TLSv1.3 response may be checked using an OPENSSL.EXE v1.1.1 or later.

$ OPENSSL version OpenSSL 1.1.1 11 Sep 2018 $ OPENSSL s_client --host wasd.xxxxxxxxxx.xxx --port 443 CONNECTED(00000003) depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3 verify error:num=20:unable to get local issuer certificate --- Certificate chain 0 s:CN = wasd.xxxxxxxxx.xxx i:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3 1 s:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3 i:O = Digital Signature Trust Co., CN = DST Root CA X3 --- Server certificate -----BEGIN CERTIFICATE----- MIIHJDCCBgygAwIBAgISA8gmjxQDyTgXeAfy7ehpvXeBMA0GCSqGSIb3DQEBCwUA 8< snip 8< rL2n3YpsP2xuCwV6ZT+etAl1IrtmXuC9tnG2QRVtVJn7wyUacUTz3XuKagS9w6Bo be0oPuGGnT0= -----END CERTIFICATE----- subject=CN = wasd.xxxxxxxxx.xxx issuer=C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3 --- No client certificate CA names sent Peer signing digest: SHA256 Peer signature type: RSA-PSS Server Temp Key: X25519, 253 bits --- SSL handshake has read 3827 bytes and written 393 bytes Verification error: unable to get local issuer certificate --- New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384 Server public key is 4096 bit Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE No ALPN negotiated Early data was not sent Verify return code: 20 (unable to get local issuer certificate) --- --- Post-Handshake New Session Ticket arrived: SSL-Session: Protocol : TLSv1.3 Cipher : TLS_AES_256_GCM_SHA384 Session-ID: 0074FBDFD12EF693B0419611204FF9EC6BFA3C006A2A7D312A9435CF7D79FE3A Session-ID-ctx: Resumption PSK: 3176C237B08F4E83B7AC32CBC79C8B79CC8FBA20837419682C4A97998898ECDE13F5254E0820C977AEC0B63C9B4B21C8 PSK identity: None PSK identity hint: None SRP username: None TLS session ticket lifetime hint: 5400 (seconds) TLS session ticket: 0000 - a7 99 08 ba aa 75 1d 53-68 c4 66 fb 5e 43 5e b2 .....u.Sh.f.^C^. 8< snip 8< 00d0 - 5d a5 3c 10 5e 4c 41 4b-bb 15 c9 5c 08 fe e1 1f ].<.^LAK...\.... Start Time: 1537620807 Timeout : 7200 (sec) Verify return code: 20 (unable to get local issuer certificate) Extended master secret: no Max Early Data: 0 --- read R BLOCK --- Post-Handshake New Session Ticket arrived: SSL-Session: Protocol : TLSv1.3 Cipher : TLS_AES_256_GCM_SHA384 Session-ID: 8DB922A11FD02889CED45C4D125C5A55B5F76B42B49826EF39CA265988FA4FA9 Session-ID-ctx: Resumption PSK: 60F73CE06DDDA5737B607A20DF7E13D85CBFFD695DB98B53B9AF09A0DABE6B34A0F50F86E2578845F1E0EA799B014B42 PSK identity: None PSK identity hint: None SRP username: None TLS session ticket lifetime hint: 5400 (seconds) TLS session ticket: 0000 - a7 99 08 ba aa 75 1d 53-68 c4 66 fb 5e 43 5e b2 .....u.Sh.f.^C^. 8< snip 8< 00d0 - 92 32 8d 2c 9c 22 54 b1-6e 24 9a c3 de 1a de a2 .2.,."T.n$...... Start Time: 1537620807 Timeout : 7200 (sec) Verify return code: 20 (unable to get local issuer certificate) Extended master secret: no Max Early Data: 0 --- read R BLOCK read:errno=0

4.9SSL References

The following provide a starting-point for investigating SSL and OpenSSL further (verified available at time of publication).


5.HTTP/2

5.1WASD HTTP/2
5.2HTTP/2 and Performance
5.3HTTP/2 Configuration
5.3.1Global Configuration
5.3.2Service Configuration
5.3.3HTTP/2 Set Rules
5.4HTTP/2 Detection
5.5HTTP/2 References

HTTP/2 is the most recent standard (RFC 9113, 2022) for implementing how HTTP is represented by, and transported between, client and server. It is not a ground-up rewrite of the established standard, HTTP/1.1 (RFC 2616, 1999). Those elements and semantics remain substantially the same. Instead HTTP/2 modifies how the data is encapsulated (framed) and transferred between agents, abstracting the complexity of this within the new protocol layer, leaving the application level largely insulated from change. As a result all existing HTTP/1.1 web-based environments should be able to continue without modification.

The focus of the protocol is on performance, in particular end-user perceived page rendering and web application responsiveness. With the original web use case being a relatively simple, single resource request-response, and early markup involving text with a few illustrative images, the single network connection, back-to-back request-response paradigm was simple to implement and worked well enough. In short time this moved to multiple network connections, each loading elements in parallel as the complexity and density of the individual elements on the pages increased, and to the introduction of HTTP/1.1 pipelining (back-to-back requests over a single connection) in an attempt to avoid request-response-request latency. Modern web documents and applications tend to have dozens of fine-grained elements that dynamically load resources based on the content of the page and/or user interaction. The single, then multiple network connections, each with its round-trip TCP connection establishment overhead and request-response blocking of resources, did not scale effectively. HTTP/2 replaces it with a single TCP connection on which multiple resources concurrently can be requested, pushed, and transferred. A more rigorous and effective implementation of the pipeline concept.

While multiplexing communication over a single network connection is a core performance technology there are other contributing elements. The framing layer uses binary tokens and parameters. The plain-text request and response headers of HTTP/1.n are replaced with tokenised, encoded and dynamically cached equivalents, commonly providing compression in excess of eighty percent. The relationship and priority of resources can be established allowing inferior resources to be delivered after or dependent on superior ones. The HTTP/2 server can send multiple responses to a single request. Known as server push it can be used to pre-load the browser (cache) with resources it has not encountered yet.

HTTP/2 has the potential to place additional load on the client and server in comparison to HTTP/1.n. One particular consideration for WASD sites is the stream concurrency setting of the HTTP/2 connection. The server specifies to the client the maximum number of concurent request-response (and server push) streams it will accept. RFC 9113 contains, "This limit is directional: it applies to the number of streams that the sender permits the receiver to create. Initially, there is no limit to this value. It is recommended that this value be no smaller than 100, so as to not unnecessarily limit parallelism." This translates to a hypothetical ten browsers connected to the site each with up to one hundred concurrent streams, or potentially one thousand active requests! Time to check those server configuration and SYSGEN parameters…

Note that HTTP/1.1 has recently been revisited with RFC 7230 family of specifications (2014) providing some clarifications and refinements on the original.

5.1WASD HTTP/2

WASD HTTP/2 implements all of the essential requirements of RFC 9113 (naturally enough). This includes the framing protocol, datagram (message) and stream management, header compression (RFC 7541), connection settings and flow control, along with HTTP/2 connection establishment and termination (TLS ALPN and HTTP upgrade). It does not ((perhaps) currently) provide server-push or stream prioritisation and dependency.

Prior to the introduction of HTTP/2, WASD's fundamental abstraction was the request, with each request interfacing directly with the network stack. With an HTTP/2 protocol connection somewhat supplanting the role of a Transmission Control Protocol (TCP) connection in HTTP/1.n, a new level of communication abstraction was required between the request processing and the network processing. It should be noted that HTTP/2 itself is transported on TCP.

Another new layer of abstraction required interfacing each protocol's request/response header formats with the underlying server processing (avoiding excessive duplication of code). HTTP/1.n has a plain-text, carriage-control separated format, while HTTP/2 has a binary, compressed, lookup-table oriented format (RFC 7541). The layer was implemented using a key-value dictionary.

The accomodations for handling both HTTP/2 and HTTP/1.1, along with related and ancilliary design and code changes, have not measurably impacted overall WASD performance, although as noted below there is a server process CPU impost associated with HTTP/2.

It's fair to say…

Reimplementing the complexities and subtleties of TCP — and adding a few of its own — up in the application layer has made HTTP/2 a significantly more complicated and less transparent protocol of HTTP/1.1 and while solving some minor annoyances with that has sacrificed the usefulness and elegance of a once readable byte-stream. Certainly added layers and associated processing to WASD, breaking the original I/O event driven design for possibly minor performance improvements.
HTTP/2 and WATCH

WATCH reports have the network item: [x]HTTP/2. This provides a detailed overview of the underlying framing and connection management exchanges between client and server. WATCH reports are available to HTTP/2 connected clients with one consideration. Due to multiplexed requests over the single network connection, WATCHing the [x]HTTP/2 item of another request in the same browser (using the same HTTP/2 connection - and there can be multiple from a single browser) is not possible (or at least more code than it's worth). The HTTP/2 activity of the WATCHing generates more report items which generate … a descent into reporting oblivion.

WASD detects when a request is initiated on the same HTTP/2 connection as an [x]HTTP/2 WATCHing client and if this sort of reporting cascade is possible (any networking group item) advises

|Time_______|Module__|Line|Item|Category__|Event...| |22:00:55.22 WATCH 1823 0004 CONNECT HTTP/2 with 192.168.1.2,62446 on https://klaatu.private,443 (0.0.0.0)| |22:00:55.22 WATCH 1454 0004 CONNECT HTTP/2 rabbit hole|
Such a request is not reported on further.

Workarounds?

5.2HTTP/2 and Performance

With HTTP/2 not modifying the fundamentals of HTTP/1.1 semantics the commonly touted payoff for all the additional complexity (in implementation) is performance. While this is often stated in terms of page rendering speeds or web application responsiveness there is another significant measure of performance - efficiency. HTTP/2 much more efficiently utilises each network (TCP) connection, as well as reducing the (time and processing) overhead of setting-up and tearing-down of each of these required for parallelism under HTTP/1.1.

Is it all worth it?  As might be expected – that depends.

There are a number of sufficiently good analyses of both the factors that affect HTTP/2 performance and the actual performance relative to HTTP/1.1. See the references section and search the Web. This section contains some observations made during WASD HTTP/2 development. All of these seem to correspond with others' observations, as well as what might reasonably be expected considering the strategies employed by the protocol.

YMMV!

After some months (and now years) accessing WASD HTTP/2 over various LANs and WANs the developer, FWIW, can't shake the perception that it seems generally more responsive in the real world. Yet interestingly …
Performance Assessment

As described in Server and Site Testing in WASD Configuration the OWASP ZAP application is integral to WASD test and exercise. It can generate an intense stream of traffic via cleartext (port 80) or TLS (port 443).

┌───────────┐                    ┌────────────┐
│           │──HTTP/1.1 clear──│            │
│ OWASP ZAP │                    │    WASD    │
│           │───HTTP/1.1 TLS───│            │
└───────────┘                    └────────────┘

Using the nghttpx proxy utility (see reference below) it is also used to exercise WASD's HTTP/2.

┌───────────┐                    ┌────────────┐                ┌────────────┐ 
│           │──HTTP/1.1 clear──│            │                │            │
│ OWASP ZAP │                    │  nghttpx   │──HTTP/2 TLS──│    WASD    │
│           │───HTTP/1.1 TLS───│            │                │            │
└───────────┘                    └────────────┘                └────────────┘

On the development bench (modest) X*6 system formal performance assessment is an encouraging ~50% of the average duration of HTTP/1.1 (see image immediately below).

HTTP Report

WASD keeps track of HTTP family statistics.

After 3.8 million requests via OWASP ZAP using the above configuration over a number of spider-generated scans, one third of which were HTTP/2, one third over TLS HTTP/1.1, and another third cleartext HTTP/1.1, the following image suggests requests using HTTP/2 take approximately 50% of HTTP/1.1.

Other Assessment

The simplest tool for getting a feel for, and elementary measurement of HTTP/2 may be found in the WASD_ROOT:[EXERCISE] directory. The document DOTTY.HTML and its companion files provide a page that loads a selectable number of resources (images) in a consistent and reproducible manner. This DOTTY.HTML can be accessed via unencrypted HTTP (http://), encrypted HTTP (https://) and services configured to provide HTTP/2 or HTTP/1.1. Using these combinations with the selectable volume of resources, elementary comparisons may be made in target environments.

The Server Admin, HTTP Report (9. Server Administration) contains comparative duration and bytes-per-second minimum/maximum/average for total server HTTP/2 and HTTP/1.n requests. These cannot simply be taken at face value without some consideration of the respective load profile but under controlled conditions can provide useful metrics.

Other development and load/performance tools were employed from a Linux platform. For someone educated in computing during the (19)70s, the availability of VM technology for such purposes is just brilliant!   But you know, we were happy in those days, though we were poor.

Indispensible were

Many thanks to the developer(s) of this package.

5.3HTTP/2 Configuration

While effectively transparent to the end-user, HTTP/2 has some aspects that need to be carefully considered by the server administrator.

5.3.1Global Configuration

HTTP/2 and its features are globally enabled and configured using directives contained in the WASD_CONFIG_GLOBAL configuration file.

HTTP/2 Global Configuration
Directive Description Default
[Http2Protocol] enabled or disabled on a whole-of-server basis disabled
[Http2FrameSizeMax] maximum frame size in octets (bytes) the server is prepared to receive 16384
[Http2HeaderListMax] maximum number of octets (bytes) permitted in a received header once uncompressed 65535
[Http2HeaderTableMax] maximum number of bytes permitted in the server-end header cache 4096
[Http2PingSeconds] number of seconds between connection RTT pings 300
[Http2StreamsMax] maximum number of concurrent streams (requests) the server permits on the connection 32
[Http2InitWindowSize] initial window size (number of octets in transit) for flow-control purposes 6291456

These largely reflect settings and defaults from RFC 9113 6.5.1

5.3.2Service Configuration

Using the WASD_CONFIG_SERVICE directive [ServiceHttp2Protocol] HTTP/2 may be disabled on a per-service basis. The default is enabled if HTTP/2 is enabled globally.

5.3.3HTTP/2 Set Rules

WASD request processing rules may be used on a per-path basis to modify (some) global configuration settings and provide other WevDAV configuation. See Request Processing Configuration of WASD Configuration).

Rule Description
HTTP2=PROTOCOL=1.1 send a "HTTP_1_1_REQUIRED" error causing the client to use HTTP/1.1 (RFC 9113 section 7)
HTTP2=SEND=GOAWAY send a "GOAWAY" frame to the client resulting in it dropping the HTTP/2 connection
HTTP2=SEND=PING send a "PING" frame to the client calculating the Round Trip Time (RTT) of the connection
HTTP2=SEND=RESET send a "RST_STREAM" frame to the client causing it to drop the HTTP/2 stream (request in progress)
HTTP2=STREAMS=MAX=integer set the maximum concurrent streams on a per-path basis
HTTP2=WRITE=low|normal|high When request data is written it is queued at the specified priority, where high priority are written before normal (default) and low priority, and normal priority before low. This is only for associated stream (request) and is not a connection or whole-of-server prioritisation.

Use path SETings to prioritise some resources (e.g. CSS and JavaScript) over others (e.g. images) and potentially improve page rendering speed. Where multiple concurrent requests are being serviced on the one HTTP/2 connection this will deliver the higher priority content before others.

# WASD_CONFIG_MAP SET **.css http2=write=high SET **.js http2=write=high

5.4HTTP/2 Detection

A request using HTTP/2 may be detected during processing with the http2: conditional.

if (http2:) do this endif

See Conditional Configuration of WASD Configuration).

A script may detect HTTP/2 using the REQUEST_PROTOCOL CGI variable with the value "HTTP/2". Other protocol versions are similarly represented.

A Server-Side Includes (SSI) document can use variations on the following construct (and similar to the script suggestion immediately above) to detect and process the request protocol.

<!--#if var={request_protocol} eqs="HTTP/2" --> HTTP/2 <!--#else--> HTTP/1.n <!--#endif-->
This is demonstrated in the example SSI document:

WASD_ROOT:[EXERCISE]SHTML.SHTML

At the time of writing there is no browser-supported mechanism for a dynamic document (i.e. JavaScript) determining the underlying HTTP protocol used to access a resource. To access this information the server must be used. The suggested method, and the one employed by the DOTTY.HTML tool described above, is to provide one JavaScript source for HTTP/2 and another for everything else.

The document would contain

<script type="text/javascript" src="/example-path/http.js"></script>
and the server configuration
# WASD_CONFIG_MAP if (http2:) map /example-path/http.js /example-path/http2.js else map /example-path/http.js /example-path/http1.js endif
where each contains a minimum variable setting or similar flag detectable by the document.

5.5HTTP/2 References

The following provide a starting-point for investigating HTTP/2 (verified available at time of publication).


6.WebDAV

6.1HTTP Methods Supported
6.1.1COPY Restrictions
6.1.2DELETE Restrictions
6.1.3MOVE Restrictions
6.1.4If: Restrictions
6.2WebDAV Configuration
6.2.1WebDAV Set Rules
6.2.2File Naming
6.2.3File-system Access
6.2.4File-system Authorisation
6.2.5Concurrent Authorisation
6.2.6Real-World Example
6.3WebDAV Metadata
6.4WebDAV Locking
6.5Some Wrinkles
6.5.1OS X Finder
6.5.2Gnome/gvfs/Nautilus
6.5.3Dreamweaver
6.6Microsoft Miscellanea
6.6.1Mapping
6.6.2FrontPage Extensions
6.6.3Avoiding Microsoft Property Clutter
6.6.4OPTIONS header "MS-Author-Via: DAV"
6.6.5Repairing broken XP Web Folders
6.6.6Adding a port number to the webfolder-address
6.6.7Adding a number-sign ("#") to the webfolder-address
6.6.8Force Windows XP to use Basic Authentication
6.6.9Microsoft XP Explorer BASIC Authentication
6.6.10Microsoft Windows 7 BASIC Authentication
6.6.11Error 0x800700DF: The file size exceeds the limit allowed and cannot be saved
6.7References

Web-based Distributed Authoring and (not) Versioning for the WASD package.

Effective WASD WebDAV file-space (without significant naming constraints) relies on being hosted on ODS-5 volumes. Behaviour hosting file-space on ODS-2 volumes is untested (though possible provided file naming is constrained to ODS-2 conventions).

WASD WebDAV methods and request headers, etc., are also propagated to the scripting environment and so functionality may be implemented using CGI, CGIplus or RTE based applications.

WASD proxy-serving supports WebDAV methods, header fields, etc.

Generally WebDAV clients are applications other than browsers and so response bodies with human-readable error explanations are unnecessary and consume bandwidth to no good purpose, and so not provided.

File-systems are notoriously latent components relative to the rest of the system (more so with VMS). Any operation to collections (directories) are not going to be atomic and for large collections requiring many sub-operations the potential for the process to be interrupted or otherwise disturbed are enormous. File-systems are not databases amenable to extensive ACID operations.

In addition each file under WebDAV management has the potential for an associated but independent metadata file. This of course means for every DAV-specific resource file activity there is at least a file-system action to check for a metadata file and for some actions such as COPY the potential for an associated but entirely independent file operation.

Of course WebDAV was not intended or designed as a general file-system protocol but one for distributed management of somewhat restricted collections of Web-related resources and so in context probably works well enough.

See sections below on file-system operation method restrictions.

Caution

If using WebDAV in any serious fashion the likes of
$ HTTPD/DO=RESTART=NOW

during server WebDav file-system modifications is a recipe for inconsistency and/or corruption!


6.1HTTP Methods Supported

A list of WebDAV methods, what WASD does with them, and any limitations or restrictions. Some of these are familiar HTTP/1.n methods and other are RFC 4981 specific. Some of the HTTP/1.n methods are overloaded with additional or variant behaviours when used in a WebDAV context. Issues of atomicity with the manipulation of file-system trees containing numbers of individual files makes strict RFC 4918 compliance difficult. See "…Restrictions" below.

Method Description
COPY** Reproduces both single resources (files) and collections (directory trees). Will overwrite files (if specified by the request) but will respond 209 (Conflict) if it would overwrite a tree.
DELETE** deletes files and directory trees
GET just the vanilla HTTP/1.1 behaviour
HEAD ditto
LOCK** see WEBDAV LOCKING below
MKCOL** create a directory
MOVE** Moves (rename or copy) a file or a directory tree. Will 'overwrite' files (if specified by the request) but will respond 209 (Conflict) if it would overwrite a tree.
OPTIONS If WebDAV is enabled and available for the path this reports the WebDAV extension methods
PROPFIND** Retrieves the requested file characteristics, DAV lock status and 'dead' properties for individual files, a directory and its child files, or a directory tree.
PROPPATCH** set and remove 'dead' meta-data properties
PUT Against a WebDAV resource behaves a little differently to historical WASD implementation of PUT.
UNLOCK** see WebDAV locking below
**WebDAV RFC 4918 method

WASD Statistics Reports gather WebDAV related data. Where a method can be used both for vanilla HTTP/1.n and WebDAV purposes it is counted in WebDAV statistics if the request header contains some other indication of a WebDAV activity.

6.1.1COPY Restrictions

Does not comply with the overwrite:T directive for collections (does so for files). Will not preemptively delete the existing tree. It returns a 209 (Conflict) response instead.

COPY does not maintain collection consistent URL namespace if a member resource cannot be moved as required by RFC 4918. It should maintain the source subtree completely uncopied. Instead it is best-effort and continues copying resources until exhausted. This is consistent with file-system behaviour. The RFC 4918 requirement, while not impossible, is fraught with issues inside a file-system.

6.1.2DELETE Restrictions

Deletion of collections is particularly fraught with issues for a file-system. In userland it is almost impossible to predetermine if an individual file in a directory tree is going to resist deletion (due to locking, protections, etc) and in kernel land it's probably no easier. It leaves the undeleted tree hierachy (resource ancestors) intact. This is RFC 4918 compliant however!

So, in the case of WASD WebDAV it's just best-effort and if something down the tree won't disappear, it just reports the failure in the 207 response and carries merrily on through the tree regardless. This IS acceptable WebDAV server behaviour!

6.1.3MOVE Restrictions

Does not comply with the overwrite:T directive for collections (does so for files). Will not currently pre-emptively delete the existing tree. It returns a 209 (Conflict) response instead.

MOVE first attempts to rename the file or directory. This is reasonably efficient, especially for directory trees but obviously only suitable for a target on the same disk volume. If a rename failure is due to a different device it falls back to using a COPY then DELETE in two separate phases. Needless-to-say this is hardly atomic and can lead to inconsistencies between source and target.

MOVE does not maintain collection consistent URL namespace if a member resource cannot be moved as required by RFC 4918. It should maintain the source subtree unmoved. Instead it is best-effort and continues moving resources until exhausted. This is consistent with file-system behaviour. The RFC 4918 requirement, while not impossible, is fraught with issues inside a file-system.

6.1.4If: Restrictions

The conditional "If:" request header field does not have full RFC 4918 support. It implements lock token and etag token processing with parenthetical OR and NOT processing. For unsupported features WATCH reports that the header was not understood and always returns an abort status. WebDAV "If:" processing is an extrodinarily complex kludge for on-the-fly decision making by the server and much of what I have read indicates most clients only ever use extremely simple conditions anyway.

6.2WebDAV Configuration

WebDAV and its features are globally enabled and configured using directives contained in the WASD_CONFIG_GLOBAL configuration file.

Directive Description
[PutMaxKBytes] maximum size of a file (PUT and POST)
[WebDAV] This directive enables and disables WebDAV.
[WebDAVlocking] Enables and disables WebDAV locking.
[WebDAVlockTimeoutDefault] see ‘Locking Timeout’ in 6.4 WebDAV Locking
[WebDAVlockTimeoutMax] see ‘Locking Timeout’ in 6.4 WebDAV Locking
[WebDAVlockCollectionDepth] See ‘Locking Depth’ in 6.4 WebDAV Locking
[WebDAVmetaDir] see 6.3 WebDAV Metadata
[WebDAVquota] Enables and disables RFC 4331 functionality (disk quota reporting).

In addition these and other configurations are provided on a per-path basis using mapping rules.

6.2.1WebDAV Set Rules

WASD request processing rules (see Request Processing Configuration of WASD Configuration) may be used on a per-path basis to modify (some) global configuration settings and provide other WevDAV configuation.

Rule Description
ODS=NAME=8BIT|UTF8|DEFAULT When a file is PUT using WebDAV (or upload), for non-7bit ASCII file names use native ODS-5 8bit syntax (default) or UTF-8 encoded character sequences (see 6.2.2 File Naming)
PUT=MAX=<integer> | * Maximum number of kilobytes file size, if "*" then effectively unlimited (per-path equivalent of the global directive [PutMaxKBytes]).
WEBDAV=[NO]HIDDEN list (default) or hide U*x hidden files (i.e. those with names beginning with period)
WEBDAV=[NO]LOCK allow/apply WebDAV locking to this path
WEBDAV=[NO]PROFILE WebDAV access according to SYSUAF profile
WEBDAV=[NO]PROP allow/apply WebDAV 'dead' property(ies) to this path
WEBDAV=[NO]PUT=LOCK a resource must be locked before a PUT is allowed
WEBDAV=[NO]READ WebDAV methods allowed read this tree
WEBDAV=[NO]SERVER WebDAV access as server account (best effort)
WEBDAV=[NO]WINPROP when NOWINPROP windows properties are ignored and emulated
WEBDAV=[NO]WRITE WebDAV methods allowed write to this path (implied read)
WEBDAV=LOCK=TIMEOUT=DEFAULT= hh:mm:ss
WEBDAV=LOCK=TIMEOUT=MAX= hh:mm:ss
WEBDAV=META=DIR= per-path equivalent of global [WevbDAVmetaDir] (see 6.3 WebDAV Metadata)

An essential function of the path setting rules is for specifying which paths in server Web-space are allowed to be accessed using the WebDAV protocol and what sort of access (read, write, etc.) that path is allowed.

6.2.2File Naming

By default files that are PUT via WebDAV (or upload) support the ISO Latin-1 character set. ASCII and non-7-bit file names use the native ODS-5 syntax. Where character sets other than ISO Latin-1, or where compatibility with other WebDAV implementations is desired (e.g. Apache), a path can be set to allow file names supplied using UTF-8 sequences.

For example, the English language word "naïve", having a diaeresis mark over the "i" character (indicating it is pronounced separately from the preceding vowel) is commonly respresented using the 8 bit character 0xEF, or as the two byte UTF-8 sequence 0xC3AF. This word if used as the file name with a type (extension) of ".TXT" by default would have the sequence of 8-bit characters

0x6E 0x61 0xEF 0x76 0x65 0x2e 0x54 0x58 0x54
and if the path had been set ods=name=utf8 the sequence would be
0x6E 0x61 0xC3 0xAF 0x76 0x65 0x2E 0x54 0x58 0x54

"Index of" (directory) listings will honour a path set ods=name=utf8 and make the listing character set UTF-8 resulting in a browser correctly rendering the name (WebDAV listings are by definition UTF-8).

File Name Ambiguity

While files and directories created via WebDAV will have a consistent naming schema applied those created by applications or manual operation on the VMS system can result in files that are not accessible with WebDAV.

For example the file name

This^_is^_an^_EXAMPLE^.txt.;1
would be presented to the client as
This is an EXAMPLE.txt
which when provided in a URL as
This%20is%20an%20EXAMPLE.txt
and translated from that URL into the file specification
This^_is^_an^_EXAMPLE.txt;1
of course will not be able to be accessed.

In addition, the two files

This^_is^_an^_EXAMPLE.txt;1 This^_is^_an^_EXAMPLE^.txt.;1
are distinct in the file-system, independently parsed from the directory structure, would be presented to the client as consecutive entries having the same name, with only the accessible file name actually available.
This is an EXAMPLE.txt This is an EXAMPLE.txt

To avoid this situation a potentially ambiguous file name containing an escaped period and no type (extension) is ignored by directory listings and WebDAV property lists. When an ambiguous file name is detected it is reported in WATCH reports.

Avoid "Interesting" File Names

While most of these are corner-cases it is best to try and avoid interesting file names that can challenge the rather convoluted VMS file-system environment. Inaccessible file names cannot of course be deleted or renamed via WebDAV and may result in directory (folder) deletion problems. These situations generally require manual intervention.

6.2.3File-system Access

Is controlled using the mapping rules:

Rule Description
WEBDAV=PROFILE access using request SYSUAF-authenticated security profile
WEBDAV=WRITE unconditional permission to read/write
WEBDAV=READ unconditional permission to read
WEBDAV=SERVER access using server account permissions

All access by WebDAV operations must have at least one set against the path. If access is permitted by one of the above settings SYSPRV is enabled to allow that access using the server account. Therefore files and directories should have a SYSTEM:READ+WRITE+EXECUTE+DELETE protection or equivalent ACL permissions, or the access may fail totally or in some part of a supposedly atomic action.

These file-system access settings are applied in the order listed above. That is, if a path successively has one or more of the above settings applied during rule processing, when it comes to applying those access controls, SYSUAF profile is applied, then if no profile SETing access to read/write, then to read-only, then access via the server account.

In addition WebDAV access requires an authorisation rule against each path.

6.2.4File-system Authorisation

All access by WebDAV operations must have one set against the path.

All WebDAV access is a combination of WASD_CONFIG_MAP path setting and WASD_CONFIG_AUTH authorisation permissions. The least permissive of the two overrides the more. The combination of an authorisation rule and a path mapping rule mitigates the chance of opening unintended access into the file-system.

These is the test-bench environment used during development:

# WASD_CONFIG_MAP pass /dweb/* /dweb/* ods=5 webdav=write webdav=nowinprop # WASD_CONFIG_AUTH ["KLAATU"=WASD_VMS_RW=id] /dweb/* r+w

Note that WebDAV read/write access is a combination of the mapping and the authorisation rule (mapping WEBDAV=READ overrides authorisation read+write). Expect complications with Microsoft environments.

For test-benching you could avoid authorisation issues completely with:

# WASD_CONFIG_AUTH [world] /dweb/* r+w

6.2.5Concurrent Authorisation

A common requirement is to provide concurrent general access and authorised WebDAV acccess to the same Web-space. This is accomplished by using two paths mapped into the same file-system space, the general access (non-authorised) path, and a WebDAV (authorised) path. The WebDAV client uses the authorised path and can then apply WebDAV methods to maintain the resources.

# WASD_CONFIG_MAP pass /web/* /web/* ods=5 pass /davweb/* /web/* ods=5 webdav=profile webdav=nowinprop # WASD_CONFIG_AUTH ["KLAATU"=WASD_VMS_RW=id] /davweb/* r+w

6.2.6Real-World Example

The following configuration is taken from a site using WebDAV to allow users to manage their Web presence. The user mapping is a fairly standard configuration for VMS accounts (see Mapping User Directories (tilde character ("~")) of WASD Configuration). User Web areas are in the [.WWW] subdirectory of the account home area.

# WASD_CONFIG_MAP # general and WebDAV access (order is important) user /~*/dav/* /*/www/* webdav=profile notepad=webdav user /~*/dav /*/www webdav=profile notepad=webdav if (pass:-1 && notepad:webdav) pass /~*/dav/* /d1/*/www/* if (pass:-1 && notepad:webdav) pass /~*/dav/* /d2/*/www/* user /~*/* /*/www/* dir=access if (pass:-1) pass /~*/* /d1/*/www/* if (pass:-1) pass /~*/* /d2/*/www/*

The four WebDAV access rules are located before the three general user access rules. The WebDAV rules are more specific. The first USER rule maps subdirectories - and the parent if a trailing slash is included. The second USER rule maps the parent directory for user agents that do not include trailing slash on their directory specifications (most it seems).

The second pair of rules reverse-maps the VMS file-system specifications represented by the result (right side) of the PASS rule into the path represented by the template (left side) of the PASS rule. Mapping from file-specifications to paths is necessary because of the way the PROPFIND method searches the file-system and then reports its results to the client as URLs.

The use of the notepad rule with a string of "webdav" (the actual string is not significant as long as it is unique within the rules) is used to conditionally process the reverse-mapping rules. They will be applied only to the requests originally mapped by the USER rules. The pass:-1 ensures the rules are only applied during reverse-mapping, not during request mapping.

The fifth rules maps general Web access to the user area. Remember, web access is to a user home subdirectory [.WWW].

The sixth and seventh rules reverse-map the VMS file-system specifications for the general USER rules for similar reasons to those described above. Why two? The user directories occur across two disk volumes and so each must be reverse-mapped.

# WASD_CONFIG_AUTH ["VMS username/password"=WASD_VMS_RW=id] /~*/dav/* read+write,profile,https: /~*/dav read+write,profile,https:

As noted above, WASD WebDAV requires both mapping and authorization rules (even for "world" - or non-authenticated - access).

In this case authorisation is only required for WebDAV access. There are two rules. The first authorises subdirectories and parent directories for agents that supply a trailing slash. The second for agents that do not provide a trailing slash.

Why use …

… two rules for each location? Why

user /~*/dav/* /*/www/* user /~*/dav /*/www
rather than
user /~*/dav* /*/www*
which would accomplish a similar result?

For finer control. The first only matches requests with a path of "/~user/dav/subdir/" and "/~user/dav", whereas the latter matches "/~user/dav/subdir/" and "/~user/dav" and "/~user/david/" and "/~user/davros", etc.

6.3WebDAV Metadata

Metadata is data (information) about data. WebDAV uses the concept of a resource property. There are "live" properties and "dead" properties. Essentially the live properties are the dynamic characteristics of a file-system object represented by creation and modification date-times, object size, etc. WebDAV dead properties are those supplied by WebDAV clients as XML entities and stored associated with the particular WebDAV object, in WASD's case the file-system object (file or directory). WASD also uses the file metadata to store resource lock data (see 6.4 WebDAV Locking).

Metadata Files

WASD manages resource metadata using a separate file associated by name with the data file. This is done for reasons of programmatic simplicity and for the convenience of any command-line owner or sysadmin of the resources. No specialised tools are required. This metadata file can be stored in one of three locations.

  1. By default, WASD uses a metadata file in the same directory and the same name with "__wasdav" appended to the extension (type). All non-WebDAV WASD functionality ignores "*.*__wasdav;" files (e.g. directory listing, file GET). Of course other applications (e.g. directory listing) do not.
    $ DIRECTORY/SIZE/DATE 01234*.* Directory WEB:[DAVweb] 01234^.56789.TXT;1 0.50KB 8-JUN-2009 23:07:19.26 01234^.56789.txt__wasdav;1 1KB 19-JUN-2009 03:20:34.50 0123456789.TXT;1 0.50KB 8-JUN-2009 23:06:59.16 0123456789.txt__wasdav;1 1KB 19-JUN-2009 03:19:14.67
  2. An alternate but still local location, is in the WASD_CONFIG_GLOBAL [WebDAVmetadir] globally specified, or per-path SET /path webdav=meta=dir directives. If specified as a subdirectory the metadata file is stored in a subdirectory of the data file directory using the same name with "__wasdav" appended to the extension (type). This is owned by the owner of the parent directory. The metadata directory does not appear in WASD WebDAV or file system listings. Choose something unique as the name cannot be used elsewhere in WebDAV space.

    For example, with the global directive

    # WASD_CONFIG_GLOBAL [WebDAVmetaDir] [.^.dav]
    specifying a subdirectory with a name containing a leading period (i.e. a U*x hidden file), the data files
    Directory WEB:[DAVweb] 01234^.56789.TXT;1 0.50KB 8-JUN-2009 23:07:19.26 0123456789.TXT;1 0.50KB 8-JUN-2009 23:06:59.16
    would have the associated metadata files
    Directory WEB:[DAVweb.^.dav] 01234^.56789.txt__wasdav;1 1KB 19-JUN-2009 03:20:34.50 0123456789.txt__wasdav;1 1KB 19-JUN-2009 03:20:24.77
  3. The final alternative uses the same directives as above but specifies a full directory path. In this case WebDAV metadata is stored completely separately from the data. This can be anywhere in available file-space. The web server account requires full access to this directory, with the simplest method of ensuring this to give ownership to the directory. This global location is only suitable for ODS-5 volumes. Sixteen hexadecimal named subdirectories are used to partition metadata files with file names generated using data file full name escaped using extended parse syntax. Using this approach a sysadmin can easily locate specific metadata files if required.

    For example, with the global directive

    # WASD_CONFIG_GLOBAL [WebDAVmetaDir] DKA0:[WASDAVMETA]
    the data files
    Directory WEB:[DAVweb] 01234^.56789.TXT;1 0.50KB 8-JUN-2009 23:07:19.26 0123456789.TXT;1 0.50KB 8-JUN-2009 23:06:59.16
    would have the associated metadata files
    Directory DKA0:[WASDAVMETA.06] web^:^[davweb^]01234^.56789.txt__wasdav;1 1KB 19-JUN-2009 03:21:34.40 web^:^[davweb^]0123456789.txt__wasdav;1 1KB 19-JUN-2009 03:21:14.67
Directory Metadata

The metadata file associated with a directory is stored in the same metadata location as files contained by that directory (not in the metadata location associated with the parent directory that contains the directory file). This metadata file is named ".DIR__wasdav" (i.e. no name, just an extension), with the following example illustrating how this would appear in each of the three metadata locations, for a subdirectory named "New Folder".

WEB:[DAVweb.New^_Folder].DIR__wasdav;1 WEB:[DAVweb.New^_Folder.^.dav].DIR__wasdav;1 DKA0:[WASDAVMETA.06]web^:^[davweb^.new^_folder^].dir__wasdav;1
Metadata XML

All metadata is stored using XML. Multiple XML data can be contained in a single metadata file. Each can be individually manipulated by a WebDAV client. The property elements are stored as-supplied by the client. It is presumed that their XML well-formedness is guaranteed by the original request XML parsing. Metadata files have content similar to the following:

$ TYPE 0123456789.txt__wasdav;1 <?xml version="1.0" encoding="UTF-8"?> <WASDAV:data xmlns:WASDAV="WASD.VMS.WebDAV" updated="2009-06-18T17:49:14Z 19-JUN-2009 03:19:14"> <WASDAV:lock token="opaquelocktoken:4D462D61B0E0427F19B425EBEEF2CFF6" depth="0" type="write" scope="exclusive" timeout="Second-86400" expires="2009-06-20T22:49:14Z 21-JUN-2009 08:19:14"> <WASDAV:owner><NS:href xmlns:NS="DAV:">MGD</NS:href></WASDAV:owner> </WASDAV:lock> <WASDAV:prop> <NS:one xmlns:NS="two">three</NS:one> </WASDAV:prop> <WASDAV:prop> <NS:four xmlns:NS="five">six</NS:four> </WASDAV:prop> <WASDAV:prop> <NS:seven xmlns:NS="eight">nine</NS:seven> </WASDAV:prop> </WASDAV:data>

This metadata example contains four properties; an exclusive write lock owned by "MGD" and three set by a client in three different (contrived) namespaces.

Metadata should not be edited manually ...

… unless you really, really know what you're doing. WASD deletes meta-data files it does not understand or otherwise considers damaged (with some resultant loss of information). Of course you can, for example to remove a lock on a resource, but you run the (small) risk of a "lost-update" and other complications. And, again of course, full metadata can be deleted at the command-line.
Microsoft Metadata

An example of such property meta-data generated by a Microsoft Windows (not Internet) Explorer client (example wrapped for presentation):

<?xml version="1.0" encoding="UTF-8"?> <WASDAV:data xmlns:WASDAV="WASD.VMS.WebDAV" updated="2007-07-23T01:39:11Z"> <WASDAV:prop> <NS:Win32CreationTime xmlns:NS="urn:schemas-microsoft-com:"> Tue, 26 Jun 2007 02:00:48 GMT</NS:Win32CreationTime> </WASDAV:prop> <WASDAV:prop> <NS:Win32LastAccessTime xmlns:NS="urn:schemas-microsoft-com:"> Mon, 23 Jul 2007 01:52:32 GMT</NS:Win32LastAccessTime> </WASDAV:prop> <WASDAV:prop> <NS:Win32LastModifiedTime xmlns:NS="urn:schemas-microsoft-com:"> Mon, 23 Jul 2007 01:52:32 GMT</NS:Win32LastModifiedTime> </WASDAV:prop> <WASDAV:prop> <NS:Win32FileAttributes xmlns:NS="urn:schemas-microsoft-com:"> 00000020</NS:Win32FileAttributes> </WASDAV:prop> </WASDAV:data>

Every file written or modified by Windows Explorer generates this sort of metadata which is then stored in an associated metadata file and read each time the data file is accessed. Some might consider this unnecessary clutter in most circumstances (I do). WASD allows this metadata to be suppressed and equivalent data generated (fudged) from file live properties when accessed - often sufficient for purpose. To suppress the actual processing of Windows Explorer metadata set a path using the WEBDAV=NOWINPROP in WASD_CONFIG_MAP.

set /webdav/* webdav=NOwinprop

6.4WebDAV Locking

For efficiency and functionality considerations WebDAV locking may be enabled and disabled (default) as global functionality using the WASD_CONFIG_GLOBAL [WebDAVlocking] directive. Additionally the WEBVDAV=[NO]LOCKING path SETing can configure this on a per-path basis.

Write Access Only

In common with RFC 4918 WASD WebDAV locking controls only write access. Both exclusive and shared locks are provided. Locking applies to the DELETE, LOCK, MKCOL, MOVE, PROPPATCH, PUT, and UNLOCK methods.

Locking Depth

WASD WebDAV locking checks parent collections to a configurable depth. WASD_CONFIG_GLOBAL directive [WebDAVlockCollectionDepth] where the default (0 or 1) checks only WebDAV locking on files, 2 WebDAV locking on the parent directory, 3 on the grandparent, 4 the great-grandparent, etc. Of course each level can add significant latency (and expense) to some operations.

Lock Depth 0

Real world experience has suggested locking depth should be maintained at the default 0 (or 1), allowing the client explicitly to manage and negotiate hierarchies of locking if required. WebDAV clients (probably correctly) assume a minimally compliant and relatively unsophisticated WebDAV server.

For more information on locking operation and implementation details see the DAVLOCK.C module and for meta-data in general the DAVMETA.C module.

Locking Timeout

When a client locks a resource it can specify the period for the lock. In the absence of such a specification WASD will apply the [WebDAVlockTimeoutDefault] value (by default 0-01:00:00 - one hour). WASD also applies the [WebDAVlockTimeoutMax] maximum lock period (by default 7-00:00:00 - one week). When the maximum period expires the lock is no longer valid.

VMS DLM Locking

WASD uses VMS locking to queue and arbitrate access to WebDAV resources and meta-files.

Two lock modes are employed; 'exclusive', when changes are to be made to the resource or its meta-data, and 'concurrent read', when resource and/or meta-data are only to be read. Concurrent read locks are compatible, but an exclusive queued against a resource currently being read waits, as does a read against a current exclusive.

WASD takes out its own VMS DLM locks on resources (files and directories) before beginning any WebDAV operation, and these prevent conflict with other WASD WebDAV operations on the same system or cluster, but RMS does not use these nor does WASD use RMS locks (except when actually acessing the file-system of course), and so there is potential for interactions between the two domains (in common with general file-system actvities). WASD WebDAV deliberately does not try to block file-system actions from other processing (except where RMS locks/blocks). Its own DLM locking is purely for internal purposes.

6.5Some Wrinkles

Some application/environment-specific considerations when using WASD WebDAV. Please report any you encounter for future inclusion in this section. Also see 6.6 Microsoft Miscellanea immediately below.

6.5.1OS X Finder

OS X Finder requires [WebDAVlocking] enabled for read/write access, otherwise access will be read-only.

6.5.2Gnome/gvfs/Nautilus


As at publication, Gnome/gvfs/Nautilus has quite a number of behavioural problems with associated Bugzilla items. Don't expect it to behave well! This has been my experience.

6.5.3Dreamweaver

Dreamwever 8 (at least, the only version I have access to) insists on using a URI with a trailing "/./" occasionally (I'm guessing to specify the "current" directory - cf. "/../", or "parent" syntax). Just absorb this internally using an appropriate mapping internal redirect.

redirect /webdav/**/./ /webdav/*/

6.6Microsoft Miscellanea

A cornucopia of of minor and major considerations!

much of the following is pre- Windows 10/11

and relates to Windows 7, Windows XP and possibly earlier.
Windows 10/11 WebDAV in general is much better behaved and reliable.
The following information continues to be included for historical reference only.

Microsoft approach WebDAV in their own inimitable fashion. Hence Microsoft agents, considering their ubiquity, including their mini-redirector are specifically looked for and functionality modified to accomodate them.

The following is a list topics/issues that were encountered/investigated during WASD WebDAV development. They may or may not be applicable to your site.

Some general references:

DOS/Windows command-line network configuration:

C:\> NET USE Z: http://the.host.name/folder/ C:\> NET USE Z: /DELETE

6.6.1Mapping

Microsoft agents (at least) seem to request the server OPTIONS of the server root regardless of any path provided with the NET USE or other network drive mapping employed. To selectively map such a request into a path that has WebDAV enabled on it (and will therefore respond with the DAV-related options) use a conditional redirect rule. For example

if (webdav:) if (request-method:OPTIONS) redirect / /dav-path/ endif
or if only required for MS agents then something more specific
if (webdav:MSagent) if (request-method:OPTIONS) redirect / /dav-path/ endif

Subsequent rules will probably be required to map typeless directory requests to the actual directory required.

redirect /dav-path /dav-path/ pass /dav-path/* /dav_root/* webdav=read

6.6.2FrontPage Extensions

Requests containing paths /_vti_inf.html and /_vti_bin/* are related to FrontPage protocol discovery probing. They can be adequately handled using a mapping rule lsuch as the following:

pass /_vti_* "404 Not an MS platform!"

6.6.3Avoiding Microsoft Property Clutter

See ‘Microsoft Metadata’ in 6.3 WebDAV Metadata.

6.6.4OPTIONS header "MS-Author-Via: DAV"

If the server's response does not contain an MS-Author-Via header, the OLE DB Provider for Internet Publishing loads the WEC and WebDAV protocol drivers one at a time (WEC first, WebDAV second) and asks them, "Do you know how to handle this URL?", specifying the exact URL passed in by the client. The first protocol which responds "yes" is selected. If neither protocol driver responds "yes" then the method which triggered the automatic driver selection (usually IBindResource::Bind) fails with an OLE DB Provider for Internet Publishing specific error code IPP_E_SERVERTYPE_NOT_SUPPORTED.

6.6.5Repairing broken XP Web Folders

Some Windows XP machines have a broken Web Folders installation. Microsoft includes a Web Folders repair utility built in to Windows to correct the problem. Use the following steps to fix the problem:

  1. Click on the "Start" menu in the lower left corner, and select "Run..."
  2. Type in "webfldrs.msi" and click the "OK" button.
  3. Click on the "Select reinstall mode" button.
  4. Select *ALL* of the checkboxes *except* for the second one ("Reinstall only if file is missing").
  5. Click on the "OK" button.
  6. Click on the "Reinstall" button.
  7. After the reinstallation is complete, reboot the computer.

6.6.6Adding a port number to the webfolder-address

Attach the port-number (80 by default) to the http-address you enter into the field of the "My Network Places"-assistant. As you can see in the following image and the linked screenshot, this will force Windows XP to use the "Microsoft Data Access Internet Publishing Provider DAV 1.1" mechanism instead of "Microsoft-WebDAV-MiniRedir/5.1.2600".

6.6.7Adding a number-sign ("#") to the webfolder-address

It is also possible to add the number sign # to the http-address you enter into the field of the "My Network Places"-assistant. As you can see in the following image and the linked screenshot, this will also force Windows XP to use the "Microsoft Data Access Internet Publishing Provider DAV 1.1" mechanism instead of "Microsoft-WebDAV-MiniRedir/5.1.2600".

http://the.host.name/folder#

6.6.8Force Windows XP to use Basic Authentication

There is a third way to get this working from the client-site. As described in the Microsoft Knowledge Base, Article ID: 841215, Windows XP disables "Basic Auth" in his "Microsoft-WebDAV-MiniRedir/5.1.2600"-mechanism by default for security reasons. See description below.

6.6.9Microsoft XP Explorer BASIC Authentication

You can enable BasicAuth by adding the following registry key and setting it to a non-zero value:

HKEY_LOCAL_MACHINE\SYSTEM \CurrentControlSet\Services\WebClient\Parameters\UseBasicAuth (DWORD)

If you delete the registry key or set it to 0, the behavior reverts to the default, or disabling the use of BasicAuth.

Disabling Basic Authentication over a clear channel:

Because the DAVRdr is part of the remote file-system stack, a computer is open to attack whenever an attempt is made to remotely access files. Although the threat to other applications that use the Internet APIs is less severe than it is for the DAVRdr, a similar attack is possible whenever an application (or the user) attempts to access a URL. For this reason, WinInet is exposing the mechanism by which the DAVRdr disables BasicAuth to other users of the Internet APIs.

With Windows XP Service Pack 2, there are two ways to block the use of Basic Authentication over clear (or unencrypted) channels:

Create the following registry key and set it to a non-zero value.

HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion \InternetSettings\DisableBasicOverClearChannel (DWORD)

This prevents WININET from attempting to use BasicAuth unless the channel is secured (HTTPS or SSL).

The application can disable the use of BasicAuth for its connections by setting the AUTH_FLAG_DISABLE_BASIC_CLEARCHANNEL flag (0x4) in the value supplied in the call to InternetSetOption using INTERNET_OPTION_AUTH_FLAGS.

AND THEN RESTART WINDOWS ***

6.6.10Microsoft Windows 7 BASIC Authentication

You can enable BasicAuth by setting the following registry key to the value 3 and restarting the WebClient service:

HKEY_LOCAL_MACHINE\SYSTEM \CurrentControlSet\Services\WebClient\Parameters\BasicAuthLevel (DWORD)

6.6.11Error 0x800700DF: The file size exceeds the limit allowed and cannot be saved

"In my case I try to copy file over WEBDAV to WEB Client connection e.g. I have mapped drive to web site. file is about 70MB I can copy small files from the same WEBDav folder."

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WebClient\Parameters
  1. Right click on the FileSizeLimitInBytes and click Modify
  2. Click on Decimal
  3. In the Value data box, type 4294967295, and then click OK. Note this sets the maximum you can download from the Webdav to 4 gig at one time, I havent figured out how to make it unlimited so if you want to download more you need to split it up.

6.7References

These are the resources used during WASD WebDAV development.

Client Tools

All these have been used during WASD WebDAV development.


7.Proxy Services

7.1HTTP Proxy Serving
7.1.1Enabling A Proxy Service
7.1.2Proxy Affinity
7.1.3Proxy Bind
7.1.4Proxy Chaining
7.1.5Controlling Proxy Serving
7.2Proxy Cache
7.3CONNECT Serving
7.3.1Enabling CONNECT Serving
7.3.2Controlling CONNECT Serving
7.4SOCKS Version 5
7.5FTP Proxy Serving
7.5.1FTP Query String Keywords
7.5.2"login" Keyword
7.6Gatewaying Using Proxy
7.6.1Reverse Proxy
7.6.2Proxy Rework
7.6.3One-Shot Proxy
7.6.4DNS Wildcard Proxy
7.6.5Originating SSL
7.7Tunneling Using Proxy
7.7.1[ServiceProxyTunnel] CONNECT
7.7.2[ServiceProxyTunnel] RAW
7.7.3[ServiceProxyTunnel] FIREWALL
7.7.4Encrypted Tunnel
7.7.5Encrypted Tunnel With Authentication
7.7.6Shared SSH Tunnel
7.7.7Complex Private Tunneling
7.7.8Tunnelling Source
7.8Browser Proxy Configuration
7.8.1Manual
7.8.2Automatic

A proxy server acts as an intermediary between Web clients and Web servers. It listens for requests from the clients and forwards these to remote servers. The proxy server then receives the responses from the servers and returns them to the clients. Why go to this trouble? There are several reasons, the most common being:

Proxy Serving Quick-Start

No additional software needs to be installed to provide proxy serving.

Proxy servering is essentially configured using a combination of configuration directives in WASD_CONFIG_GLOBAL and WASD_CONFIG_SERVICE to enable proxy serving both globally and then for allow a specific service to make outgoing connections, along with mapping directives in WASD_CONFIG_MAP to control and direct those outgoing connections.

The following steps provide a brief outline of proxy configuration.

  1. Enable proxy serving and specify which particular services are to be proxies (7.1.1 Enabling A Proxy Service and Service Configuration of WASD Configuration)
  2. If providing SSL tunneling (proxy of Secure Sockets Layer transactions) add/modify a service for that (7.3 CONNECT Serving).
  3. Add WASD_CONFIG_MAP mapping rules for controlling this/these services (7.1.5 Controlling Proxy Serving, 7.3.2 Controlling CONNECT Serving, and 7.5 FTP Proxy Serving).
  4. Restart server (HTTPD/DO=RESTART).
Proxy Error Messages
Error Messages

When proxy processing is enabled and WASD_CONFIG_GLOBAL directive [ReportBasicOnly] is disabled it is necessary to make adjustments to the contents of the WASD_CONFIG_MSG message configuration file [status] item beginning "Additional Information". Each of the "/httpd/-/statusnxx.html" links

<a href="/httpd/-/status1xx.html">1<i>xx</i></a> <a href="/httpd/-/status2xx.html">2<i>xx</i></a> <a href="/httpd/-/status3xx.html">3<i>xx</i></a> <a href="/httpd/-/status4xx.html">4<i>xx</i></a> <a href="/httpd/-/status5xx.html">5<i>xx</i></a> <a href="/httpd/-/statushelp.html">help</a>
should be changed to include a local host component
<a href="http://local.host.name/httpd/-/status1xx.html">1<i>xx</i></a> <a href="http://local.host.name/httpd/-/status2xx.html">2<i>xx</i></a> <a href="http://local.host.name/httpd/-/status3xx.html">3<i>xx</i></a> <a href="http://local.host.name/httpd/-/status4xx.html">4<i>xx</i></a> <a href="http://local.host.name/httpd/-/status5xx.html">5<i>xx</i></a> <a href="http://local.host.name/httpd/-/statushelp.html">help</a>

If this is not provided the links and any error report will be interpreted by the browser as relative to the server the proxy was attempting to request from and the error explanation will not be accessible.

7.1HTTP Proxy Serving

WASD provides a proxy service for the HTTP scheme (prototcol).

Proxy serving generally relies on DNS resolution of the requested host name. DNS lookup can introduce significant latency to transactions. To help ameliorate this WASD incorporates a host name cache. To ensure cache consistency the contents are regularly flushed, after which host names must use DNS lookup again, refreshing the information in the cache. The period of this cache purge is contolled with the [ProxyHostCachePurgeHours] configuration parameter.

When a request is made by a proxy server is is common for it to add a line to the request header stating that it is a forwarded request and the agent doing the forwarding. With WASD proxying this line would look something like this:

Forwarded: by http://host.name.domain (HTTPd-WASD/8.4.0 OpenVMS/IA64 SSL)
It is enabled using the [ProxyForwarded] configuration parameter.

An additional, and perhaps more widely used facility, is the Squid extension field to the proxied request header supplying the originating client host name or IP address.

X-Forwarded-For: client.host.name
It is enabled using the [ProxyXForwardedFor] configuration parameter.

7.1.1Enabling A Proxy Service

Proxy serving is enabled on a global basis using the WASD_CONFIG_GLOBAL file [ProxyServing] configuration parameter. After that each virtual service must have proxy functionality enabled as a per-service configuration.

WASD can configure services using the WASD_CONFIG_GLOBAL [service] directive, the WASD_CONFIG_SERVICE configuration file, or even the /SERVICE= qualifier.

WASD_CONFIG_SERVICE

Using directives listed in Service Configuration of WASD Configuration) this example illustrates configuring a non-proxy server (the disabled is the default and essentially redudant) and a proxy service.

[[http://alpha.example.com:80]] [ServiceProxy] disabled [[http://alpha.example.com:8080]] [ServiceProxy] enabled

7.1.2Proxy Affinity

High performance/highly available proxy server configurations require more than one instance configured and running. Whether this is done by running multiple instances on the same host or one instance on multiple hosts, it leads to situations where successive requests will be processed by different instances. As those instances don't share a common name to IP address cache, they will eventually use different IP addresses when trying to connect to an origin server running on multiple hosts.

This may result in the following, user visible, issues:

For these reasons, the proxy server will make every effort to relay successive requests from a given client to the same origin host as long as this one is available (built-in failover capability will ultimately trigger the choice of a new host). This is known as client to origin affinity or proxy affinity capability.

Proxy to origin server affinity is enabled using the following service configuration directive.

[[http://alpha.example.com:8080]] [ServiceProxy] enabled [ServiceProxyAffinity] enabled
Uses HTTP Cookies

Obviously the use of cookies must be enabled in the browser or this facility will not operate for that client. After the first successful connection to an origin host, the proxy server will send a cookie indicating the IP address used to the client browser. Upon subsequent requests, this cookie will be used to select the same host. The cookie is named WasdProxyAffinity_origin.host.name and the value simply the IP address in dotted decimal. This cookie is not propagated beyond the proxy service but may be WATCHed by checking the Proxy Processing item.

7.1.3Proxy Bind

It is possible to make the outgoing request appear to originate from a particular source address. The Network Interface must be able to bind to the specified IP address (i.e. it cannot be an arbitrary address).

[[http://alpha.example.com:8080]] [ServiceProxy] enabled [ServiceProxyBind] 131.185.250.1

The same behaviour may be accomplished with an WASD_CONFIG_MAP mapping rule.

SET http://*.example.com proxy=bind=131.185.250.1

7.1.4Proxy Chaining

Some sites may already be firewalled and have corporate proxy servers providing Internet access. It is quite possible to use WASD proxying in this environment, where the WASD server makes the proxied requests via the next proxy server in the hierarchy. This is known as proxy chaining.

[[http://alpha.example.com:8080]] [ServiceProxy] enabled [ServiceProxyChain] next.proxy.host

Chaining may also be controlled on a virtual service or path basis using an WASD_CONFIG_MAP mapping rule.

SET http://*.com proxy=chain=next.proxy.host:8080
Chain Authorization

If the upstream proxy server requires authorization this may be supplied using a per-service directive

[[http://alpha.example.com:8080]] [ServiceProxy] enabled [ServiceProxyChain] next.proxy.host [ServiceProxyChainCred] basic:username>:<password>
or via mapping rule
SET http://*.com proxy=chain=next.proxy.host:8080 \ proxy=chain=cred=basic:<username>:<password>

The basic: keyword allows WASD to appropriately encode the credentials. Basic authentication is the only scheme currently supported.

7.1.5Controlling Proxy Serving

Requests at a service enabled for proxy processing are directed to proxy processing using a fundamental rule which terminates rule processing and initiates the outgoing connection.

pass * http://
This rule and variant equivalents for FTP and CONNECT processing, and in combination with other rules to purpose, are seen in the examples in this section on proxy.

Controlling both access-to and access-via proxy serving is possible.

Proxy Password

Access to the proxy service can be directly controlled through the use of WASD authorization. Proxy authorization is distinct from general access authorization. It uses specific proxy authorization fields provided by HTTP, and by this allows a proxied transaction to also supply transaction authorization for the remote server. In the WASD_CONFIG_SERVICE configuration file.

[[http://alpha.example.com:8080]] [ServiceProxy] enabled [ServiceProxyAuth] proxy

In addition to the service being specified as requiring authorization it is also necessary to configure the source of the authentication. This is done using the WASD_CONFIG_AUTH configuration file. The following example shows all requests for the proxy virtual service must be authorized (GET and well as POST, etc.), although it is possible to restrict access to only read (GET), preventing data being sent out via the server.

[[alpha.example.com:8080]] ["Proxy Access"=PROXY_ACCESS=id] http://* read+write
Chain Password

An up-stream, chained proxy server (7.1.4 Proxy Chaining) may be permitted to receive proxy authentication from the client via a WASD proxy server using the CHAIN keyword. Unconfigured, WASD does not propagate HTTP proxy authorization fields. Only one proxy server in a chain can be authenticated against.

[[http://alpha.example.com:8080]] [ServiceProxy] enabled [ServiceProxyAuth] chain
Local Password

It is also possible to control proxy access via local authorization, although this is less flexible by removing the ability to then pass authorization information to the remote service. In other repects it is set up in the same way as proxy authorization, but enabled using the LOCAL keyword.

[[http://alpha.example.com:8080]] [ServiceProxy] enabled [ServiceProxyAuth] local
Access Filtering

Extensive control of how, by whom and what a proxy service is used for may be exercised using WASD general and conditional mapping Request Processing Configuration of WASD Configuration) and Conditional Mapping of WASD Configuration) possibly in the context of a virtual service specification for the particular connect service host and port (see Virtual Servers of WASD Configuration). The following examples provide a small indication of how mapping could be used in a proxy service context.

  1. It is possible, though more often not practical, to regulate which hosts are connected to via the proxy service. For example, the following rule forbids accessing any site with the string "hacker" in it (for the proxy service "alpha…:8080".
    [[alpha.example.com:8080]] pass http://*hacker*/* "403 Proxy access to this host is forbidden." pass http://*
  2. Or as in the following example, only allow access to specific sites.
    [[alpha.example.com:8080]] pass http://*.org/* pass http://*.digital.com/* pass http://* "403 Proxy access to this host is forbidden."
  3. It is also possible to restrict access via the proxy service to selected hosts on the internal subnet. Here only a range of literal addresses plus a single host in another subnet are allowed access to the service.
    [[alpha.example.com:8080]] pass http://* "403 Restricted access." ![ho:131.185.250.* ho:131.185.200.10] pass http://*
  4. In the following example POSTing to a particular proxied servers is not allowed (why I can't imagine, but hey, this is an example!)
    [[alpha.example.com:8080]] pass http://subscribe.sexy.com/* "403 POSTing not allowed." [me:POST] pass http://*
  5. It is possible to redirect proxied requests to other sites.
    [[alpha.example.com:8080]] redirect http://www.sexy.com/* http://www.disney.com/ pass http://*
  6. A proxy service is just a specialized capability of a general HTTP service. Therefore it is quite in order for the one service to respond to standard HTTP requests as well as proxy-format HTTP requests. To enforce the use of a particular service as proxy-only, add a final rule to a virtual service's mapping restricting non-proxy requests.
    [[alpha.example.com:8080]] pass http://* pass /* "403 This is a proxy-only service."
  7. This example provides the essentials when supporting reverse proxying. Note that mappings may become quite complex when supporting access to resources across multiple internal systems (e.g. access to directory icons).
    [[main.corporate.server.com:80]] pass /sales/* http://sales.corporate.server.com/* pass /shipping/* http://shipping.corporate.server.com/* pass /support/* http://support.corporate.server.com/* pass * "403 Nothing to access here!"
Note

To expedite proxy mapping is it recommended to have a final rule for the proxy virtual service that explicitly passes the request. This would most commonly be a permissive pass as in example 1, could quite easily be an restrictive pass as in example 2, or a combination as in example 6.
Request Modification

Using path mapping rules (see Request Processing Configuration of WASD Configuration). it is possible to remove or specifically set selected proxied request headers. Many headers are critical to server processing but some are informational or otherwise amenable to change. This can be undertaken using the SET mapping rule proxy=header=<parameter>.

For example, to have a proxy service suppress the "Referer:" request header:

# WASD_CONFIG_MAP set * proxy=header=referer

To modify the "Referer:" request header to a fixed URL:

set * proxy=header=referer=https://whatever/

To modify the "User-Agent:" request header to a specific string:

set * "proxy=header=user-agent=None of your business!"

7.2Proxy Cache

Proxy Cache is OBSOLETE

Caching involves using the local file-system for storage of responses that can be reused when a request for the same URL is made.

As of WASD v12.0 Caching is OBSOLETE

With the overwhelming Internet move to encrypted everything, the usefulness of a proxy server local cache for cleartext responses is marginal at best. Related configuration directives are reported obsolete and ignored.


7.3CONNECT Serving

The connect service provides firewall proxying for any connection-oriented TCP/IP access. Essentially it provides the ability to tunnel any other protocol via a Web proxy server. In the context of Web services it is most commonly used to provide firewall-transparent access for Secure Sockets Layer (SSL) transactions. It is a special case of the more general tunneling provided by WASD, see 7.7 Tunneling Using Proxy.

7.3.1Enabling CONNECT Serving

As with proxy serving in general, CONNECT serving may enabled on a per-service basis using the WASD_CONFIG_GLOBAL [service] directive, the WASD_CONFIG_SERVICE configuration file, or even the /SERVICE= qualifier.

The actual services providing the CONNECT access (i.e. the host and port) are specified on a per-service basis. This means it is possible to have CONNECT and non-CONNECT services deployed on the one server, as part of a general proxy service or standalone. CONNECT proxying is enabled by appending the connect keyword to the particular service specification. The following example shows a non-proxy and proxy services, with and without additional connect processing enabled.

[[http://alpha.example.com:80]] [[http://alpha.example.com:8080]] [ServiceProxy] enabled [[http://alpha.example.com:8081]] [ServiceProxyTunnel] connect [[http://alpha.example.com:8082]] [ServiceProxy] enabled [ServiceProxyTunnel] connect

7.3.2Controlling CONNECT Serving

The connect service poses a significant security dilemma when in use in a firewalled environment. Once a CONNECT service connection has been accepted and established it essentially acts as a relay to whatever data is passed through it. Therefore any transaction whatsoever can occur via the connect service, which in many environments may be considered undesirable.

In the context of the Web and the use of the connect service for proxying SSL transactions it may be well considered to restrict possible connections to the well-known SSL port, 443. This may be done using conditional directives, as in the following example:

[[alpha.example.com:8080]] if (request-method:CONNECT) pass *:443 pass * "403 CONNECT only allowed to port 443." endif
All of the comments on the use of general and conditional mapping made in 7.1.5 Controlling Proxy Serving can also be applied to the connect service.

7.4SOCKS Version 5

SOCKS is an Internet protocol that exchanges network packets between a client and server through a proxy server. SOCKS5 optionally provides authentication so only authorized users may access a server. Practically, a SOCKS server proxies TCP connections to an arbitrary IP address.

WASD SOCKS5 supports only CONNECT TCP/IP and not BIND or UDP-associate.

Enabling SOCKS5 Proxy

A SOCK5 proxy connection must be mapped using the socks5:// pseudo scheme. The following rule allows connection to any host name or address.

[[alpha.example.com:8080]] pass socks5://*
To selectively allow SOCKS5 access then map to a specific host name or address, and optional port.
[[alpha.example.com:8080]] pass socks5://the.host.name pass socks5://134.142.71.8 pass socks5://137.146.74.10:22

7.5FTP Proxy Serving

WASD provides a proxy service for the FTP scheme (prototcol). This provides the facility to list directories on the remote FTP server, download and upload files.

The (probable) file system of the FTP server host is determined by examining the results of an FTP PWD command. If it returns a current working directory specification containing a "/" then it assumes it to be Unix(-like), if ":[" then VMS, if a "\" then DOS. (Some DOS-based FTP servers respond with a Unix-like "/" so a second level of file-system determination is undertaken with the first entry of the actual listing.) Anything else is unknown and reported as such. WASD (for the obvious reason) is particularly careful to perform well with FTP servers responding with VMS file specifications.

Note that the content-type of the transfer is determined by the way the proxy server interprets the FTP request path's "file" extension. This may or may not correspond with what the remote system might consider the file type to be. The default content-type for unknown file types is "application/octet-stream" (binary). When using the alt query string parameters then for any file in a listing the icon provides an alternate content-type. If the file link provides a text document then the icon will provide a binary file. If the link returns a binary file then the icon will return a file with a plain-text content-type.

In addition to content-type the FTP mode in which the file transfer occurs can be determined by either of two conditions. It the content-type is "text/.." then the transfer mode will be ASCII (i.e. record carriage-control adjusted between systems). If not text then the file is transfered in Image mode (i.e. a binary, opaque octet-stream). For any given content-type this default behaviour may be adjusted using the [AddType] directive (see Alphabetic Listing of WASD Configuration) or the "#!+" MIME.TYPES directive (see MIME.TYPES of WASD Configuration).

Rules required in WASD_CONFIG_MAP for mapping FTP proxy. This is preferably made against the virtual service providing the FTP proxy. The service explicitly must make the icon path used available or it must be available to the proxy service in some other part of the mappings. Also the general requirement for error message URLs applies to FTP proxying (‘Proxy Error Messages’ in 7. Proxy Services).

[[proxy.host.name:8080] pass http://* http://* pass ftp://* ftp://* pass /*/-/* /wasd_root/runtime/*/*

7.5.1FTP Query String Keywords

Keywords added to an FTP request query string allow the basic FTP action to be somewhat tailored. These case-insensitive keywords can be in the form of a query keys or query form fields and values. This allows considerable flexibility in how they are supplied, allowing easy use from a browser URL field or for inclusion as form fields.

Keyword Description
alt Adds alternate access (complementary content-type at the icon) for directory listings.
ascii Force the file transfer type to be done as ASCII (i.e. with carriage-control conversion between systems with different representations).
content Explicitly specify the content type for the returned file (e.g. "content:text/plain", or "content=image/gif").
dos When generating a directory listing force the interpretation to be DOS.
email Explicitly specify the anonymous access email address (e.g. "email:daniel@wasd.vsm.com.au" or "email=daniel@wasd.vsm.com.au").
image Force the file transfer type to be done as an opaque binary stream of octets.
list Displays the actual directory plain-text listing returned by the remote FTP server. Can be used for problem analysis.
login Results in the server prompting for a username and password pair that are then used as the login credentials on the remote FTP server.
octet Force the content-type of the file returned to be specified as "application/octet-stream".
text Force the content-type of the file returned to be specified as "text/plain".
unix When generating a directory listing force the interpretation to be Unix.
upload Causes the server to return a simple file transfer form allowing the upload of a file from the local system to the remote FTP server.
vms When generating a directory listing force the interpretation to be VMS.

7.5.2"login" Keyword

The usual mechanism for supplying the username and password for access to a non-anonymous proxied FTP server area is to place it as part of the request line (i.e. "ftp://username:password@the.host.name/path/"). This has the obvious disadvantage that it's there for all and sundry to see.

The "login" query string is provided to work around the more obvious of these issues, having the authentication credentials as part of the request URL. When this string is placed in the request query string the FTP proxy requests the browser to prompt for authentication (i.e. returns a 401 status). When request header authentication data is present it uses this as the remote FTP server username and password. Hence the remote username and password never need to appear in plain-text on screen or in server logs.

7.6Gatewaying Using Proxy

WASD is fully capable of mapping non-proxy into proxy requests, with various limitations on effectiveness considering the nature of what is being performed.

Gatewaying between request schemes (protocols)

and also gatewaying between IP versions

All can be useful for various reasons. One example might be where a script is required to obtain a resource from a secure server via SSL. The script can either be made SSL-aware, sometimes a not insignificant undertaking, or it can use standard HTTP to the proxy and have that access the required server via SSL. Another example might be accessing an internal HTTP resource from an external browser securely, with SSL being used from the browser to the proxy server, which the accesses the internal HTTP resource on its behalf.

Request Redirect

The basic mechanism allowing this gatewaying is "internal" redirection. The redirect mapping rule (see REDIRECT Rule of WASD Configuration) either returns the new URL to the originating client (requiring it to reinitiate the request) or begins reprocessing the request internally (transparently to the client). It is this latter function that is obviously used for gatewaying.

7.6.1Reverse Proxy

The use of WASD proxy serving as a firewall component assumes two configured network interfaces on the system, one of which is connected to the internal network, the other to the external network. (Firewalling could also be accomplished using a single network interface with router blocking external access to all but the server system.) Outgoing (internal to external) proxying is the most common configuration, however a proxy server can also be used to provide controlled external access to selected internal resources. This is sometimes known as reverse proxy and is a specific example of WASD's general non-proxy to proxy request redirection capability (7.6 Gatewaying Using Proxy).

In this configuration the proxy server is contacted by an external browser with a standard HTTP request. Proxy server rules map this request onto a proxy-request format result. For example:

redirect /sales/* /http://sales.server.com/*?

Note that the trailing question-mark is required to propagate any query string (see REDIRECT Rule of WASD Configuration).

The server recognises the result format and performs a proxy request to a system on the internal network. Note that the mappings required could become quite complex, but it is possible. See example 7 in 7.1.5 Controlling Proxy Serving.

Redirection Location Field

If a reverse proxied server returns a redirection response (302) containing a "Location: url" field with the host component the same reverse-proxied-to server it can be rewritten to instead contain the proxy server host. If these do not match the rewrite does not occur. Using the redirection example above, the SET mapping rule proxy=reverse=location specifies the path that will be prefixed to the path component in the location field URL. Usually this would be the same path used to map the reverse proxy redirect (in this example "/sales/"), though could be any string (presumably detected and processed by some other part of the mapping).

set /sales/* proxy=reverse=location=/sales/ redirect /sales/* /http://sales.server.com/*?
This could be simplified a little by using a postfix SET rule along with the original redirect.
redirect /sales/* /http://sales.server.com/*? proxy=reverse=location=/sales/

If the proxy=reverse=location=<string> ends in an asterisk the entire 302 location field URL is appended (rather than just the path) resulting in something along the lines of

Location: http://proxy.server.com/sales/http://sales.server.com/path/
which once redirected by the client can be subsequently tested for and some action made by the proxy server according to the content (just a bell or whistle ;-).
Authorization Verification

WASD can authorize reverse proxy requests locally (perhaps from the SYSUAF) and rewrite that username into the proxied requests "Authorization: …" field. The proxied-to server can then verify that the request originated from the proxy server and extract and use that username as authenticated.

This functionality is described in the WASD_ROOT:[SRC.HTTPD]PROXYVERIFY.C module.

7.6.2Proxy Rework

The proxy rework facility will modify a target string to a replacement string in the request header (e.g. Host:), the response header (e.g. set-cookie:), and in the response body. Rework will be applied to HTML and CSS responses.

These are simple string matches.

Proxy rework must be enabled for a service by setting a maximum size for the HTML response body to be reworked, in kB.

# WASD_CONFIG_SERVICE [[*.1924]] [ServiceReworkMax] 128
Specific paths must then be SET in WASD_CONFIG_MAP to have proxy requests reworked.
# WASD_CONFIG_MAP [[*:1924]] set * proxy=rework=192.168.1.3=192.168.1.2
Note

Proxy rework likely needs a lot more work!

Also consider the ‘proxyMUNGE Utility’ in 7.6.2 Proxy Rework below.

proxyMUNGE Utility

This utility (CGIplus script) can be used to rewrite HTTP response "Location:" fields, "Set-Cookie:" path and domain components and URLs in HTML and CSS content.

This functionality is described in the prologue to the code WASD_ROOT:[SRC.UTILS]PROXYMUNGE.C

Note

The proxyMUNGE Utility handles all response rewriting and so when employing it to perform reverse-proxy processing it is unnecessary to use the proxy=reverse=location=<string> mapping rule described in ‘Redirection Location Field’ in 7.6.1 Reverse Proxy.

7.6.3One-Shot Proxy

This looks a little like reverse proxy, providing access to a non-local resource via a standard (non-proxy) request. The difference allows the client to determine which remote resource is accessed. This works quite effectively for non-HTML resources (e.g. image, binary files, etc.) but non-self-referential links in HTML documents will generally be inaccessible to the client. This can provide provide scripts access to protocols they do not support, as with HTTP to FTP, HTTP to HTTP-over-SSL, etc.

Mappings appropriate to the protocols to be support must be made against the proxy service. Of course mapping rules may also be used to control whom or to what is connected.

[[the.proxy.service:port]] # support "one-shot" non-proxy to proxy redirect redirect /http://* http://* redirect /https://* https://* redirect /ftp://* ftp://* # OK to process these (already, or now) proxy format requests pass http://* http://* pass https://* https://* pass ftp://* ftp://*

The client may the provide the desired URL as the path of the request to the proxy service. Notice that the scheme provided in the desired URL can be any supported by the service and its mappings.

http://the.proxy.service:port/http://the.remote.host/path http://the.proxy.service:port/https://the.remote.host/path http://the.proxy.service:port/ftp://the.remote.host/pub/

7.6.4DNS Wildcard Proxy

This relies on being able to manipulate host record in the DNS or local name resolution database. If a "*.the.proxy.host" DNS (CNAME) record is resolved it allows any host name ending in ".the.proxy.host" to be resolved to the corresponding IP address. Similarly (at least the Compaq TCP/IP Services) the local host database allows an alias like "another.host.name.proxy.host.name" for the proxy host name. Both of these would allow a browser to access "another.host.name.proxy.host.name" with it resolved to the proxy service. The request "Host:" field would contain "another.host.name.proxy.host.name".

Using this approach a fully functioning proxy may be implemented for the browser without actually configuring it for proxy access, where returned HTML documents contain links that are always correct with reference to the host used to request them. This allows the client an ad hoc proxy for selected requests. For a wildcard (CNAME) record the browser user may enter any host name prepended to the proxy service host name and port and have the request proxied to that host name. Entering the following URL into the browser location field

http://the.host.name.the.proxy.service:8080/path
would result in a standard HTTP proxy request for "/path" being made to "the.host.name:80". With the URL
https://the.host.name.the.proxy.service:8443/path
an SSL proxy request. Note that normally the well-known port would be used to connect to (80 for http: and 443 for https:). If the final, period-separated component of the wildcard host name is all digits it is interpreted as a specific port to connect to. The example
http://the.host.name.8001.the.proxy.service:8080/path
would connect to "the.host.name:8001", and
https://the.host.name.8443.the.proxy.service:8443/path
to "the.host.name:8443".
Note

It has been observed that some browsers insist that an all-digit host name element is a port number despite it being prefixed by a period not a colon. These browsers then attempt to contact the host/port directly. This obviously precludes using an all-digit element to indicate a target port number with these browsers.

This wildcard DNS entry approach is a more fully functional analogue to common proxy behaviour but is slightly less flexible in providing gatewaying between protocols and does require more care in configuration. It also relies on the contents of the request "Host:" field to provide mapping information (which generally is not a problem with modern browsers). The mappings must be performed in two parts, the first to handle the wildcard DNS entry, the second is the fairly standard rule(s) providing access for proxy processing.

[[the.proxy.service:port1]] if (host:*.the.proxy.service:port1) redirect * /http://* else pass http://* http://* endif

The obvious difference between this and one-shot proxy is the desired host name is provided as part of the URL host, not part of the request path. This allows the browser to correctly resolve HTML links etc. It is less flexible because a different proxy service needs to be provided for each protocol mapping. Therefore, to allow HTTP to HTTP-over-SSL proxy gatewaying another service and mapping would be required.

[[the.proxy.service:port2]] if (host:*.the.proxy.service:port2) redirect * /https://* else pass https://* https://* endif

7.6.5Originating SSL

This proxy function allows standard HTTP clients to connect to Secure Sockets Layer (4. Transport Layer Security) services. This is very different to the CONNECT service (7.3 CONNECT Serving), allowing scripts and standard character-cell browsers supporting only HTTP to access secure services.

Standard username/password authentication is supported (as are all other standard HTTP request/response interactions). The use of X.509 client certificates (4.5.12 Authorization Using X.509 Certification) to establish outgoing identity is not currently supported.

Enabling SSL

Unlike HTTP and FTP proxy it requires the service to be specifically configured using the [ServiceClientSSL] directive.

There are a number of Secure Sockets Layer related service parameters that should also be considered (see Service Configuration of WASD Configuration). Although most have workable defaults unless [ServiceProxyClientSSLverifyCA] and [ServiceProxyClientSSLverifyCAfile] are specifically set the outgoing connection will be established without any checking of the remote server's certificate. This means the host's secure service could be considered unworthy of trust as the credentials have not been established.

[[http://alpha.example.com:8080]] [ServiceProxy] enabled [ServiceClientSSL] enabled

7.7Tunneling Using Proxy

WASD supports the CONNECT method which effectively allows tunneling of raw octets through the proxy server. This facility is most commonly used to allow secure SSL connections to be established with hosts on the 'other side' of the proxy server. This basic mechanism is also used by WASD to provide an extended range of tunneling services. The term raw is used here to indicate an 8 bit, bidirectional, asynchronous exchange of octets between two entities, as a protocol family, not necessarily as an application (but can be so). Global proxy serving must be enabled (7.1.1 Enabling A Proxy Service) and then each service must be configured and mapped according to the desired mode of tunneling. Disabling or setting timeouts appropriately on the mapped service is important if connections are not to be disrupted by general server timeouts on output and non-progress (quiescent connections).

7.7.1[ServiceProxyTunnel] CONNECT

A service with this configuration is used as a target for CONNECT proxying (usually SSL through a firewall). The client expects an HTTP success (200) response once the remote connection is established, and HTTP error response if there is a problem, and once established just relays RAW octets through the proxy server (classic CONNECT behaviour).

# WASD_CONFIG_SERVICE [[http://*:8080]] [ServiceProxy] enabled [ServiceProxyTunnel] connect
# WASD_CONFIG_MAP [[*:8080]] if (request-method:connect) pass *:443 *:443 pass * "403 CONNECT only allowed to port 443." endif

This configuration enables CONNECT processing and limits any connect to SSL tunneling (i.e. port 443 on the remote system).

7.7.2[ServiceProxyTunnel] RAW

This allows any raw octet client (e.g. telnet) to connect to the port and by mapping be tunnelled to another host and port to connect to its service (e.g. a telnet service). The usual HTTP responses associated with CONNECT processing are not provided.

# WASD_CONFIG_SERVICE [[http://*:10023]] [ServiceProxy] enabled [ServiceProxyTunnel] raw
# WASD_CONFIG_MAP [[*:10023]] if (request-method:connect) pass *:0 raw://another.host:23 timeout=none,none,none endif pass "403"

Telnet is used in the example above but the principle equally applies to any protocol that uses a raw 8 bit, bidirectional, asynchronous exchange of octets. Another example might be an SMTP service (port 25).

SSL to RAW

Using a tunnel it is possible to put a TLS/SSL (https://) front-end service to an otherwise plaintext-only service (http://).

# WASD_CONFIG_SERVICE [[https://tls-host:443]] [ServiceNonSSLRedirect] https://tls.host:443 [ServiceProxy] enabled [ServiceProxyTunnel] raw
# WASD_CONFIG_MAP [[*:443]] if (request-method:connect) pass *:0 raw://non-tls.host:80 endif pass "403"
Chaining RAW

It is possible to have a raw tunnel establish itself through a proxy chain (7.1.4 Proxy Chaining) by transparently generating an intermediate CONNECT request to the up-stream proxy server. Note that not all CONNECT proxy will allow connection to just any specified port. For security reasons it it is quite common to restrict CONNECT to port 443.

# WASD_CONFIG_SERVICE [[http://*:10025]] [ServiceProxy] enabled [ServiceProxyTunnel] raw
# WASD_CONFIG_MAP [[*:10025]] if (request-method:connect) pass *:0 raw://another.host:25 proxy=chain=proxy.host:8080 endif pass "403"

Any error in connecting to the chained proxy, making the request, connecting to the destination, etc. (i.e. any error at all) is not reported. The network connection is just dropped. Use WATCH to establish the cause if necessary.

7.7.3[ServiceProxyTunnel] FIREWALL

With this configuration a service expects that the first line of text from the client contains a host name (or IP address) and optional port (e.g. "the.host.name" or "the.host.name:23"). This allows a variable destination to be mapped. The usual HTTP responses associated with CONNECT processing are not provided.

# WASD_CONFIG_SERVICE [[http://*:10023]] [ServiceProxy] enabled [ServiceProxyTunnel] FIREWALL
# WASD_CONFIG_MAP [[*:10023]] if (request-method:connect) pass *:* raw://*:23 timeout=none,none,none pass * raw://*:23 timeout=none,none,none endif pass "403"

The pass rules force the supplied domain name (and optional port) to be mapped to the telnet port (23). Of course the mapping rules could allow the supplied port to be mapped into the destination if desired.

Chaining FIREWALL

As with [ServiceProxyTunnel] RAW it is possible to chain FIREWALL services to an up-stream proxy server. See ‘Chaining RAW’ in 7.7.2 [ServiceProxyTunnel] RAW.

7.7.4Encrypted Tunnel

Up to this point the tunnels have merely been through the proxy server. It is possible to establish and maintain ENCRYPTED TUNNELS between WASD servers. SSL is used for this purpose. This is slightly more complex as both ends of the tunnel need to be configured.

                 ┌────────────┐               ┌────────────┐
──unencrypted──│ WASD proxy │──ENCRYPTED──│ WASD proxy │──unencrypted──
                 └────────────┘               └────────────┘

This arrangement may be used for any stream-oriented, network protocol between two WASD systems. As it uses standard CONNECT requests (over SSL) it MAY also be possible to be configured between WASD and non-WASD servers.

The following example is going to maintain an encrypted tunnel between WASD servers running on systems KLAATU and GORT. It is designed to allow a user on KLAATU to connect to a specified port using a telnet client, and have a telnet session created on GORT, tunnelled between the two systems via an SSL encrypted connection.

Source of tunnel:

# KLAATU WASD_CONFIG_SERVICE [[http://*:10023]] [ServiceProxy] enabled [ServiceClientSSL] ENABLED [ServiceProxyTunnel] RAW
# KLAATU WASD_CONFIG_MAP [[*:10023]] # if the client is on the local subnet if (remote-addr:192.168.0.0/24 && request-method:connect) pass *:0 https://gort.domain:10443 timeout=none,none,none endif pass "403"

Destination of tunnel:

# GORT WASD_CONFIG_SERVICE [[https://*:10443]] [ServiceProxy] enabled [ServiceProxyTunnel] CONNECT
# GORT WASD_CONFIG_MAP [[*:10443]] # limit the connection to a specific host if (remote-addr:192.168.0.10 && request-method:connect) pass *:0 raw://gort.domain:23 timeout=none,none,none endif pass "403"

When a client connects to the service provided by port 10023 on system KLAATU the connection is immediately processed using a pseudo CONNECT request header. The service on this port is a proxy allowed to initiate SSL connections (client SSL). This service is mapped to system GORT port 10443, an SSL service that allows the CONNECT method (tunneling). KLAATU's proxy initiates an SSL connection with GORT. When established and the CONNECT request from KLAATU is received, it is mapped via a raw tunnel (8 bit, etc.) to its own system port 23 (the telnet service). Telnet is in use at both ends while encrypted by SSL inbetween! Note the use of network addresses and general fail rules used to control access to this service, as well as the disabling of timers that might otherwise shutdown the tunnel.

7.7.5Encrypted Tunnel With Authentication

This arrangement is essentially a variation on example 4. It provides a cryptographic authentication of the originator (source) of the tunnel.

Source of tunnel:

# KLAATU WASD_CONFIG_SERVICE [[http://*:10023]] [ServiceProxy] enabled [ServiceClientSSL] enabled [ServiceProxyTunnel] RAW [ServiceClientSSLcert] WASD_ROOT:[LOCAL]HTTPD.PEM
# KLAATU WASD_CONFIG_MAP [[*:10023]] # if the client is on the local subnet if (remote-addr:192.168.0.0/24 && request-method:connect) pass *:0 https://gort.domain:10443 timeout=none,none,none endif pass "403"

Destination of tunnel:

# GORT WASD_CONFIG_SERVICE [[https://*:10443]] [ServiceProxy] enabled [ServiceProxyTunnel] CONNECT [ServiceProxyAuth] PROXY
# GORT WASD_CONFIG_MAP [[*:10443]] # we'll be relying on X509 authentication if (request-method:connect) pass *:0 raw://gort.domain:23 timeout=none,none,none endif pass "403"
# GORT WASD_CONFIG_AUTH [[*:10443]] [X509] * r+w,param="[VF:OPTIONAL]",~4EAB3CBC735F8C7977EBB41D45737E37

This works by configuring the destination service to insist on proxy authorization. The authorization realm is X509 which causes the destination to demand a certificate from the source (4.5.12 Authorization Using X.509 Certification). The fingerprint of this certificate is checked against the authorization rule before the connection is a allowed to procede.

7.7.6Shared SSH Tunnel

The objective of this raw tunnel variant (see 7.7.2 [ServiceProxyTunnel] RAW) is to allow tunneling of Secure Shell (SSH) via a client site proxy server CONNECT which is usually confined to port 443. Of course most Web servers are configured to provide SSL HTTP on port 443. Sharing of HTTP and SSH on the same port is a little problematic and involves some protocol detection. The following explanation of how it is implemented is so that the reader can understand the requirement for the "timeout quirk".

On configured services; WASD peeks at the incoming TCP byte stream to see if it's SSH protocol. If it is, the socket is associated with a proxy raw tunneling service and proxy tunneling initiated to a mapped SSH server. However (just to make it interesting) some SSH clients do not initiate their own exchange until after the SSH server, and so peeking only works for a subset of clients. Of course this is a Catch-22 of sorts! To provide for these clients; if an input timeout should occur (an SSH client waiting) WASD sets up the tunnel anyway and begins the proxy. The proxied SSH server should then initiate the protocol and the client respond. The directive [ServiceShareSSH] configured to be non-zero both enables this facility for a service and sets the input timeout period (which perhaps should be shorter than the default 30 seconds because such clients will wait that long for any SSH server response).

This approach seems to work well-enough in practice, although users need to be aware that some clients will pause (for the duration of the timeout period – the "timeout quirk") during initial connection setup.

# WASD_CONFIG_SERVICE [[https://*:443] [ServiceShareSSH] 10 [[http://*:10022]] [ServiceProxy] enabled [ServiceProxyTunnel] raw
# WASD_CONFIG_MAP [[*:443] if (request-method:ssh) pass * raw://ssh.server.host:22 \ service=the.proxy.host:10022 \ timeout=none,none,none endif [[*:10022]] pass "403"

This example shows an SSL service, the desired SSH service (which can be local or remote) and the internal proxy service that will provide the connection.

7.7.7Complex Private Tunneling

When creating raw tunnels between WASD servers, and possibly in other circumstances, it is often useful to be able to signal tunnel purpose to the remote end. In this way a single destination port can support multiple tunneling purposes simply through mapping rules. An originating end can inject an HTTP request line, or full request, into the established tunnel connection, which can then be processed by the usual WASD request mapping, and from that alternate services provided based on the intent signalled by the originating end.

This somewhat complex but instructive example illustrates the potential utility and versatility of WASD tunneling. It involves an originating WASD server, a destination (service providing) WASD server, and just to make it interesting an intermediate chained HTTP proxy server (not WASD). The idea is to provide access to various application services not necessarily supported by intermediate HTTP proxies and/or gateways. Four services will be supported by the example; SSH, NNTP IMAP and SMTP.

             inside                       firewall                     outside

         ┌────────────┐               ┌─────────────┐               ┌────────────┐
──raw──│ WASD proxy │──ENCRYPTED──│ other proxy │──ENCRYPTED──│ WASD proxy │──raw──
         └────────────┘               └─────────────┘               └────────────┘

       wasd.internal.net             proxy.internal.net            wasd.external.net
                                     proxy.external.net

 SSH───8022──┐                            ╎    ╎                             ┌────22───SSH
SMTP───8025──┼────────────────────────────┤╌╌╌╌┤─────────────────────────────┼────25───SMTP
NNTP───8119──┤                            ╎╌╌╌╌╎                             ├───119───NNTP
IMAP───8143──┘                            ╎    ╎                             └───143───IMAP
Internal Services

These are the services assigned on the WASD server on the inside of the proxy/gateway. Note that there is one per application to be tunneled. For simplicity each service port number has been selected to parallel the well-known application port number. Note that proxy is enabled on each (allowing them to initiate outgoing connections) and each has SSL enabled (further allowing them to initiate encrypted connections).

# client SSH [[http://*:8022]] [ServiceProxy] enabled [ServiceProxyTunnel] RAW [ServiceClientSSL] enabled # client SMTP [[http://*:8025]] [ServiceProxy] enabled [ServiceProxyTunnel] RAW [ServiceClientSSL] enabled # client IMAP [[http://*:8143]] [ServiceProxy] enabled [ServiceProxyTunnel] RAW [ServiceClientSSL] enabled # client NNTP [[http://*:8119]] [ServiceProxy] enabled [ServiceProxyTunnel] RAW [ServiceClientSSL] enabled

Each client application (i.e. IMAP, SSH) must be configured to connect to its corresponding service port (e.g. IMAP to 8143, SMTP to 8025).

Internal Mapping

These mappings are made on the WASD server on the inside of the proxy/gateway. The rules essentially initiate an outgoing encrypted (SSL) connection to the host wasd.external.net supporting the external WASD proxy server. Each is also configured not to connect directly but to request the chained proxy server proxy.internal.net to establish the connection on their behalf.

!##### SSH ##### [[*:8022]] pass * https://wasd.external.net:443 notimeout \ proxy=tunnel=request="CONNECT wasd-ssh" \ proxy=chain=proxy.internal.net:8080 !##### SMTP ##### [[*:8025]] pass * https://wasd.external.net:443 \ proxy=tunnel=request="CONNECT external-smtp" \ proxy=chain=proxy.internal.net:8080 !##### NNTP ##### [[*:8119]] pass * https://wasd.external.net:443 \ proxy=tunnel=request="CONNECT external-nntp" \ proxy=chain=proxy.internal.net:8080 !##### IMAP ##### [[*:8143]] pass * https://wasd.external.net:443 \ proxy=tunnel=request="CONNECT external-imap" \ proxy=chain=proxy.internal.net:8080

If the up-stream proxy server successfully connects to wasd.external.net port 443 the proxy server allows the byte-stream to be asynchonously and bidirectionally exchanged with the internal WASD server outgoing connection. This internal WASD server has initiated an SSL connection and the external server port 443 expects SSL so they can now both negotiate an SSL-encrypted channel essentially directly with each other.

External Services

The external WASD service configuration is very simple, a single SSL port.

# general SSL service [[https://wasd.external.net:443]] # outgoing proxy/tunnel service [[http://wasd.external.net:1234]] [ServiceProxy] enabled [ServiceProxyTunnel] raw [ServiceClientSSL] ENABLED

Connections to the 443 port are expected to undertake an SSL negotiation to establish an encrypted channel. This includes incoming tunnel connections. The service on port 1234 is required to support the connections outgoing from the external WASD server to the application server ports.

External Mapping

These mappings are all applied to requests at port 443 on the external WASD server wasd.external.net. Each rule checks three request characterstics. First, the request method, "CONNECT". Second, the request URI, varies according to the request. These are the request data injected by the internal WASD server wasd.internal.net using the set=proxy=tunnel=request= mapping rule on the outgoing connection. Third, the originating host (proxy.external.net) address adds an extra filter on from where this facility may be used. The respective pass of the matching rule then initiates an outgoing connection to the respective application server's well-known port. A timeout is applied to limit connection times.

!# SSH tunneling [[*:443]] if (request-method:CONNECT && \ request-uri:"wasd-ssh" && \ remote-addr:205.3.*) \ pass * raw://wasd.external.net:22 service=*:1234 timeout=noprogress=00:00:50 !# SMTP tunneling [[*:443]] if (request-method:CONNECT && \ request-uri:"external-smtp" && \ remote-addr:205.3.*) \ pass * raw://smtp.isp.net:25 service=*:1234 timeout=noprogress=00:00:50 !# NNTP tunneling [[*:443]] if (request-method:CONNECT && \ request-uri:"external-nntp" && \ remote-addr:205.3.*) \ pass * raw://news.isp.net:119 service=*:1234 timeout=noprogress=00:00:* !# IMAP tunneling [[*:443]] if (request-method:CONNECT && \ request-uri:"external-imap" && \ remote-addr:205.3.*) \ pass * raw://imap.isp.net:143 service=*:1234 timeout=noprogress=00:00:50 !# disable general 1234 service usage [[*:1234]] pass * 403 "Internal use only!"
Example In Action

Now let's look at an actual example usage. Consider the internal user's IMAP application, say Thunderbird, is configured to use an IMAP server at host wasd.internal.net port 8143. The internal user activates Thunderbird which then intiates an TCP/IP connection to the configured IMAP server expecting to commence the IMAP application protocol.

This connection arrives at wasd.internal.net port 8143 which has a WASD raw tunnel service listening. The connection is accepted and request processing commences. Mapping rules applied to port 8143 initiate an SSL connection to host wasd.external.net which is not directly accessable because of the firewall and must be connected to using the HTTP proxy server proxy.internal.net as an intermediary. This is specified in the same mapping rule. The mapping rule also injects an HTTP request header providing request characteristics that can be identified and acted upon by the external server.

The internal WASD server initiates a connection to the proxy server proxy.internal.net acting as part of the firewall. As it is endeavouring to initiate an SSL connection with the external wasd.external.net host this proxy connection uses a CONNECT request specifying wasd.external.net port 443. The proxy server establishes a connection with the host wasd.external.net at port 443. Once the connection is established it becomes an asynchronous, bidirectional channel between wasd.internal.net and wasd.external.net with the proxy server as a conduit.

The service connection just established is expecting an SSL negotiation in an attempt to establish an encrypted channel. When this negotiation concludes successfully the communications between wasd.internal.net and wasd.external.net become opaque to all external listeners including proxy.internal.net.

The encrypted connection now established, the request begins to be processed by the WASD server at wasd.external.net. A number of mapping rules apply to port 443. Each rule compares the injected request method and URI until, in this case, the external-imap rule matches. This rule specifies that a raw connection be established with the host imap.isp.net at port 143 using the proxy-capable port 1234 service. A timeout limits the duration this connection can be held unused.

The IMAP application server at imap.isp.external port 143 accepts the connection at begins to communicate using the IMAP protocol.

There is now a raw (8 bit, asynchronous, bidirectional) connection from the Thunderbird client to wasd.internal.net, (encrypted) through to proxy.internal.net, (encrypted) through to wasd.external.net, and raw to the IMAP server at imap.isp.net. This raw connection will be used for communication between Thunderbird and the IMAP server using the IMAP application protocol.

7.7.8Tunnelling Source

When a tunnel is established into a system the source of that connection (IP host-name/address and port) becomes obscured. By setting the path to the destination port proxy=forwarded=for (host name) or proxy=forwarded=address (IP address) the external client can be obtained using data contained in the logical name WASD_TUNNEL.

Consider tunneling external port 22345 to internal port 22 - Secure Shell.

# WASD_CONFIG_SERVICE [[http://*:22345]] [ServiceProxy] enabled [ServiceProxyTunnel] RAW # WASD_CONFIG_MAP [[*:22345]] pass * raw://localhost:22 notimeout

To Secure Shell the source host and port would be localhost and some random port. It can be useful for the login procedure or other service to have the actual client host name (or IP address). Adding the path setting.

# WASD_CONFIG_MAP [[*:22345]] pass * raw://localhost:22 notimeout proxy=forwarded=address
will result in connection data becoming available in the multivalued logical name WASD_TUNNEL. Index 0 contains internal data, and then the rest (1..127) contain one tunneled connection's details each, in the format
internal-host:port>=external-host:port>=client-host:port>
For example
localhost:46851=www.external.net:22345=mydotcom.org:49201

Obtaining the SSH source port, say from TT_ACCPORNAM data, the original client host and port can be searched for with some trivial DCL code. Adapt to suit local requirements.

$ if P1 .eqs. "" then P1 = f$element(1,":",f$getdvi("TT:","TT_ACCPORNAM")) $ value = "" $ local = "" $ service = "" $ client = "" $ index = 1 $ index_loop: $ value = f$trnlnm("WASD_TUNNEL","WASD_TABLE",index) $ if value .eqs. "" then goto end_index_loop $ local = f$element(0,"=",value) $ addr = f$element(0,":",local) $ port = f$element(1,":",local) $ if port .eqs. P1 $ then $ service = f$element(1,"=",value) $ client = f$element(2,"=",value) $ goto end_index_loop $ endif $ index = index + 1 $ goto index_loop $ end_index_loop: $ if f$trnlnm("TT_CLIENT","LNM$PROCESS") .nes. "" - then deassign /process TT_CLIENT $ if client .nes. "" then define /process TT_CLIENT "''client'"

The tunnel data remains current for at least one minute and may become unavailable at any time after that.

Note

The source data only reflects the client that connects to that system's services and so cannot be used across multiple, back-to-back tunnels.

7.8Browser Proxy Configuration

The browser needs to be configured to access URLs via the proxy server. This is done using two basic approaches, manual and automatic.

7.8.1Manual

Most browsers allow the configuration for access via a proxy server. This commonly consists of an entry for each of the common Web protocol schemes ("http:", "ftp:", "gopher:", etc.). Supply the configured WASD proxy service host name and port for the HTTP scheme. This is currently the only one available. This would be similar to the following example:

http: www.example.com 8080

To exclude local hosts, and other servers that do not require proxy access, there is usually a field that allows a list of hosts and/or domain names for which the browser should not use proxy access. This might be something like:

www.example.com,example.com,example.com

7.8.2Automatic

A proxy auto-config (PAC) file defines how web browsers and other user agents can automatically choose the appropriate proxy server (access method) for fetching a given URL.

https://en.wikipedia.org/wiki/Proxy_auto-config

The following is a very simple proxy configuration JavaScript function. This specifies that all URL host names that aren't full qualified, or that are in the "example.com" domain will be connected to directly, with all other being accessed via the specified proxy server.

function FindProxyForURL(url,host) { if (isPlainHostName(host) || dnsDomainIs(host, ".example.com")) return "DIRECT"; else return "PROXY www.example.com:8080; DIRECT"; }

This JavaScript is contained in a file with a specific, associated MIME file type, "application/x-ns-proxy-autoconfig". For WASD it is recommended the file be placed in WASD_ROOT:[LOCAL] and have a file extension of .PAC (which follows Netscape naming convention).

The following WASD_CONFIG_GLOBAL directive would map the file extension to the required MIME type:

[AddType] .PAC application/x-ns-proxy-autoconfig - proxy autoconfig

This file is commonly made the default document available from the proxy service. The following example shows the HTTP$MAP rules required to do this:

[www.example.com:8080] pass http://* http://* pass / /wasd_root/local/proxy.pac pass *

All that remains is to provide the browser with the location from which load this automatic proxy configuration file. In the case of the above set-up this would be:

http://www.example.com:8080/

A template for a proxy auto-configuration file may be found at WASD_ROOT:[EXAMPLE]PROXY_AUTOCONFIG.TXT


8.Instances and Environments

8.1Server Instances
8.1.1VMS Clustering Comparison
8.1.2Considerations
8.1.3Configuration
8.1.4Status
8.2Server Environments

WASD instances and environments are two distinct mechanisms for supporting multiple WASD server processes on a single system.

Server instances are multiple, cooperating server processes providing the same set of configured resources.

Server environments are multiple, independent server processes providing differently configured resources.

8.1Server Instances

The term instance is used by WASD to describe an autonomous server process. WASD will support multiple server processes running on a single system, alone or in combination with multiple server processes running across a cluster. This is not the same as supporting multiple virtual servers (see Virtual Services of WASD Configuration). When multiple instances are configured on a single system they cooperate to distribute the request load between themselves and share certain essential resources such as accounting and authorization information.

WARNING

Versions earlier than Compaq TCP/IP Services v5.3 and some TCPware v5.n (at least) have a problem with socket listen queuing that can cause services to "hang" (should this happen just disable instances and restart the server). Ensure you have the requisite version/ECO/patch installed before activating multiple instances on production systems!

8.1.1VMS Clustering Comparison

The approach WASD has used in providing multiple instance serving may be compared in many ways to VMS clustering.

A cluster is often described as a loosely-coupled, distributed operating environment where autonomous processors can join, process and leave (even fail) independently, participating in a single management domain and communicating with one another for the purposes of resource sharing and high availability.

Similarly WASD instances run in autonomous, detached processes (across one or more systems in a cluster) using a common configuration and management interface, aware of the presence and activity of other instances (via the Distributed Lock Manager and shared memory), sharing processing load and providing rolling restart and automatic "fail-through" as required.

Load Sharing

On a multi-CPU system there are performance advantages to having processing available for scheduling on each. WASD employs AST (I/O) based processing and was not originally designed to support VMS kernel threading. Benchmarking has shown this to be quite fast and efficient even when compared to a kernel-threaded server (OSU) across 2 CPUs. The advantage of multiple CPUs for a single multi-threaded server also diminishes where a site frequently activates scripts for processing. These of course (potentially) require a CPU each for processing. Where a system has many CPUs (and to a lesser extent with only two and few script activations) WASD's single-process, AST-driven design would scale more poorly. Running multiple WASD instances addresses this.

Of course load sharing is not the only advantage to multiple instances …

Restart

When multiple WASD instances are executing on a node and a restart is initiated only one process shuts down at a time. Others remain available for requests until the one restarting is again fully ready to process them itself, at which point the next commences restart. This has been termed a rolling restart. Such behaviour allows server reconfiguration on a busy site without even a small loss of availability.

Fail-Through

When multiple instances are executing on a node and one of these exits for some reason (resource exhaustion, bugcheck, etc.) the other(s) will continue to process requests. Of course requests in-progress by the particular instance at the time of instance failure are disconnected (this contrasts with the rolling restart behaviour described above). If the former process has actually exited (in contrast to just the image) a new server process will automatically be created after a few seconds.

The term fail-through is used rather than failover because one server does not commence processing as another ceases. All servers are constantly active with those remaining immediately and automatically taking all requests in the absence any one (or more) of them.

8.1.2Considerations

Of course "there is no such thing as a free lunch" and supporting multiple instances is no exception to this rule. To coordinate activity between and access to shared resources, multiple instances use low-level mutexes and the VMS Distributed Lock Manager (DLM). This does add some system overhead and a little latency to request processing, however as the benchmarks indicate increases in overall request throughput on a multi-CPU system easily offset these costs. On single CPU systems the advantages of rolling restart and fail-through need to be assessed against the small cost on a per-site basis. It is to be expected many low activity sites will not require multiple instances to be active at all.

When managing multiple instances on a single node it is important to consider each process will receive a request in round-robin distribution and that this needs to be considered when debugging scripts, using the Server Administration page and the likes of WATCH, etc. (see 8.1 Server Instances).

8.1.3Configuration

If not explicitly configured only one instance is created. The configuration directive [InstanceMax] allows multiple instances to be specified Global Configuration of WASD Configuration). When this is set to an integer that many instances are created and maintained. If set to "CPU" then one instance per system CPU is created. If set to "CPU-integer" then one instance for all but one CPU is created, etc. The current limit on instances is eight, although this is somewhat arbitrary. As with all requests, Server Administration page access is automatically shared between instances. There are occasions when consistent access to a single instance is desirable. This is provided via an admin service (see Service Configuration of WASD Configuration).

When executing, the server process name appends the instance number to the "WASD". Associated scripting processes are named accordingly. This example shows such a system:

Pid Process Name State Pri I/O CPU Page flts Pages 21600801 SWAPPER HIB 16 0 0 00:06:53.65 0 0 21600807 CLUSTER_SERVER HIB 12 1879 0 00:01:14.51 91 112 21600808 CONFIGURE HIB 10 30 0 00:00:01.46 47 23 … 21600816 ACME_SERVER HIB 10 71525 0 00:01:28.08 508 713 M 21600818 SMISERVER HIB 9 11197 0 00:00:02.29 158 231 21600819 TP_SERVER HIB 9 1337711 0 00:05:55.78 80 105 … 216421F1 WASD1:80 HIB 5 5365731 0 00:23:12.86 37182 7912 2164523F WASD2:80 HIB 5 5347938 0 00:23:31.41 38983 7831 2162BA5D WASD_WOTSUP HIB 3 2111 0 00:00:00.47 735 518 2164ABCF WASD1:80-651 LEF 6 57884 0 00:00:16.71 3562 3417 2164CBDB WASD2:80-612 LEF 4 19249 0 00:00:04.16 3153 3116 21631BDC WASD2:80-613 LEF 5 18663 0 00:00:07.19 3745 3636 2164BBE6 WASD1:80-658 LEF 5 3009 0 00:00:00.94 2359 2263 …

8.1.4Status

The instance management infrastructure distributes basic status data to all instances on the node and/or cluster. The intent is to provide an easily comprehended snapshot of multi-instance/multi-node WASD processing status. The data comprises:

The data are constrained to these items due to the need to accomodate it within a 64 byte lock value block for cluster purposes. Single node environments do not utilise the DLM, each instance updating its table entry directly.

Each node has a table with an entry for every other instance in that WASD environment. Instance data are updated once every minute so any instance with data older than one minute is no longer behaving correctly. This could be due to some internal error, or that the instance no longer exists (e.g. been stopped, exited or otherwise no longer executing). An entry for an instance that no longer exists is retained indefinitely, or until a /DO=STATUS=PURGE is performed removing all such expired entries, or a /DO=STATUS=RESET removing all entries (and allowing those currently executing to repopulate the instance data over the next minute.

These status data are accessible via command-line and in-browser reports, intended for larger WASD installations, primarily those operating across multiple nodes in a cluster. With the data being stored in a common, another of those other nodes can provide a per-cluster history even if one or more nodes become completely non-operational.

This is an example report on a 132 column terminal display. Due to screen width constraints the date/time omits the year field of the date.

$ httpd/do=status Instance Ago Up Ago Count Exit Ago Status Version /Min /Hour ~~~~~~~~~~~~~~~~ ~~~~ ~~~~~~~~~~~~~~~ ~~~~ ~~~~~ ~~~~~~~~~~~~~~~ ~~~~ ~~~~~~~~~~ ~~~~~~~ ~~~~ ~~~~~ 1 KLAATU::WASD:80 41s 18-DEC 23:27:57 54m 21 18-DEC 23:27:57 54m %X00000001 11.2.0 2 17 KLAATU::WASD1:80---1d-17-DEC-02:49:21---1d-----5-17-DEC-02:50:03---1d-%X00000001-11.2.0----3-----15 KLAATU::WASD2:80---1d-17-DEC-02:49:25---1d-----5-17-DEC-02:50:07---1d-%X00000001-11.2.0----0-----10 KLAATU::WASD3:80---1d-17-DEC-02:49:29---1d-----6-17-DEC-02:50:11---1d-%X00000001-11.2.0----0------3 as at 19-DEC-2017 00:22:41

This provides an example CLI report showing a single node, where a single instance has been started, changed to a three instance configuration, restarted so that the three instances have begun processing. The configuration has been returned a single instance and then the existing three instances restarted the previous day, resulting in the original single instance returning to processing. That instance was last (re)started some 54 minutes ago (a normal exit status showing) and its status was last updated some 41 seconds ago. Note that the three instances showing white-space struck-through with hyphens are stale, having last been updated 1 day ago. Entries older than three minutes are displayed in this format to differentiate them from current entries.

The same report on an 80 column terminal. Note that the overt date/time has been omitted, leaving only the period ago the event happened.

$ httpd/do=status Instance Ago Up Count Exit Status Version /Min /Hour ~~~~~~~~~~~~~~~~ ~~~~ ~~~~ ~~~~~ ~~~~ ~~~~~~~~~~ ~~~~~~~ ~~~~ ~~~~~ 1 KLAATU::WASD:80 5s 58m 21 58m %X00000001 11.2.0 1 18 KLAATU::WASD1:80---1d---1d-----5---1d-%X00000001-11.2.0----3-----15 KLAATU::WASD2:80---1d---1d-----5---1d-%X00000001-11.2.0----0-----10 KLAATU::WASD3:80---1d---1d-----6---1d-%X00000001-11.2.0----0------3 as at 19-DEC-2017 00:25:05

Where multiple instances exist, or have existed, and the terminal page size is greater than 24 lines, HTTPMON displays an equivalent of the 80 column report at the bottom of the display.

Similarly, the Server Admin report (9. Server Administration) shows an HTML equivalent of the 80 column report immediately below the control and time panels.

Using Instance Status

8.2Server Environments

WASD server environments allow multiple, distinctly configured environments to execute on a single system. Generally, WASD's unlimited virtual servers and multiple account scripting eliminates the need for multiple execution environments to kludge these requirements. However there may be circumstances that make this desirable; regression and forward-compatibility testing comes to mind.

See Server Environments in WASD Installation for deltained information on maintaining multiple installations of WASD.


9.Server Administration

9.1Access Before Configuration
9.2Access Configuration
9.3Server Instances
9.4HTTPd Server Reports
9.5HTTPd Server Revise
9.6HTTPd Server Action
9.7HTTPd Command Line
9.7.1Accounting
9.7.2Access Control
9.7.3Alignment Faults
9.7.4Authentication
9.7.5Cache
9.7.6Configuration Check
9.7.7DCL/Scripting Processes
9.7.8DECnet Scripting Connections
9.7.9Hhelppp!
9.7.10HTTP/2 Connection
9.7.11Instances
9.7.12Instance Status
9.7.13Logging
9.7.14Mapping
9.7.15Network Connection
9.7.16Shutdown and Restart
9.7.17Secure Sockets Layer
9.7.18Throttle
9.7.19WebSocket

The online Server Administration facility provides a rich collection of functionality, including server control, reports and configuration. Some of these are intended as general administration tools while others provide more detailed information intended for server debugging and development purposes.

The administration interface also provides some basic server statistics in the lower right panel;  local date/time, internet (UTC) equivalent, client host, connection protocol, and request RTT (the essential network overhead between client and server), up-times for system, server process, server executable, CPU consumed by it, along with current connection and requests-in-progress statistics. Alerts (in red) also can appear in this panel.

The value of the WATCH facility 10. WATCH Facility as a general configuration and problem-solving tool cannot be overstated.

All server configuration files, with the exception of the authentication databases, are plain text and may be modified with any prefered editor. However the majority of these can also be administered online through a browser. In addition the update facility allows some administration of file system portions of the Web. See 12. HTTPd Web Update.

Access to many portions of the package is constrained by file protections and directory listing access files. See for a method for circumventing these restrictions.

9.1Access Before Configuration

It is often a significant advantage for the inexperienced administrator on a new and largely unconfigured installation to be able to gain access to the facilities offered by Server Administration, particularly the WATCH facility (10. WATCH Facility). This can be done quite simply by using the authentication skeleton-key (3.12 Skeleton-Key Authentication). This allows the site administrator to register a username and password from the command-line that can be used to gain access to the server. In addition, the server ensures that requesting an otherwise non-authorized Server Administration facility generates a challenge which invokes a username/password dialog at the browser allowing the user to enter the previously registered username and password and gain access.

Method

9.2Access Configuration

Once established the site should make the Server Administration facility a configured facility of the site. The value of its facilities cannot be overstated.

It is also recommended that for production sites the path to these reports be controlled via authentication and authorization, using both host and username restrictions, similar to the following:

[WHATEVER-REALM] /httpd/-/admin/* host.ip.addr,~WebMaster,~WhoEverElse,r+w

If a full authorization environment is not required but administration via browser is still desired restrict access to browsers executing on the server system itself, using an appropriate SYSUAF-authenticated username. Provision of a VMS account for server administration only is quite feasable, see 3.10.6 Nil-Access VMS Accounts.

[VMS] /httpd/-/admin/* #localhost,~username,r+w

If SSL is in use (4. Transport Layer Security) then username/password privacy is inherently secured via the encrypted communications. To restrict server administration functions to this secure environment add the following to the WASD_CONFIG_MAP configuration file:

/httpd/-/admin/* "403 Access denied." ![sc:https]

When using the revise capability of the Server Administration facility it is necessary to comply with all the requirements for Web update of files. This is discussed in general terms in 12. HTTPd Web Update. Revision of server configuration files requires path permissions allowing write access for the username(s) doing the administration, as well as the required ACL on the target directory (in the following example WASD_ROOT:[LOCAL]).

[VMS] /httpd/-/admin/* #localhost,~username,r+w /wasd_root/local/* #localhost,~username,r+w

It is possible to allow general access to the Server Administration facility and reports while restricting the ability to initiate server actions such as a restart! Using the WORLD realm against the path is necessary, for the obvious security reason, the server administration module will not allow itself to be used without an authenticated username, provided as a pseudo-authenticated "WORLD".

[VMS] /httpd/-/admin/control/* #localhost,~username,r+w [WORLD] /httpd/-/admin/* r

When GZIP compression is configured for the server (see GZIP Encoding of WASD Configuration) it is not by default applied to Server Admin reports or other pages. It can be applied, selectively if desired, using mapping rules. For instance, to apply it to all requests not from the local intranet a rule similar to the following can be added before the Server Admin path mapping itself.

if (!remote-addr:192.168.0.0/8) set /httpd/-/admin/* response=GZIP=all pass /httpd/-/admin/* /httpd/-/admin/*

GZIP content-encoding can never be applied to WATCH reports.

9.3Server Instances

With a single instance (see 8.1 Server Instances) access to Server Administration reports, etc. is always serviced by the one server process. If multiple instances are configured then in common with all requests administration requests will be serviced by any one of the associated processes depending on the momentary state of the round-robin distribution.

There are many circumstances where it is preferable to access only the one server. This can be accomplished for two differing objectives.

  1. To facilitate access to a specific instance's Server Administration page, including instance-specific reports etc. This is provided through the use of an administration service port (see Administration Services of WASD Configuration) available from the Server Administration page.
  2. The Server Administration page (‘Control Section’ in 9.6 HTTPd Server Action) and the command-line 9.7.11 Instances) provides the capability to explicitly set the number of instances supported, overriding any configuration directive. After explicitly setting this, using either means, the server must be restarted. The explicit startup setting remains in effect until it is changed to "max" allowing the WASD_CONFIG_GLOBAL configuration directive [InstanceMax] to once again determine the number of instances required.

The latter approach is particularly useful when performing detailed WATCH activities (10. WATCH Facility).

When multiple per-node instances are executing the Server Administration pages and reports all include an indication of which process serviced the request. When accessing no instance in particular the process name is presented in parentheses after the page title

HTTPd www.example.com:80 Server Administration (HTTPd:80)
When a particular instance's administration service port is being used the process name is separated from the page title by a hyphen
HTTPd www.example.com:80 Server Administration - HTTPd:80

Multi-instance status (see 8.1.4 Status) snapshots are available via HTTPDMON, the Server Admin main page and can be reported from the command line using

$ HTTPD /DO=STATUS

9.4HTTPd Server Reports

The server provides a number of internally generated reports. Some of these are of general interest. Others are more for evaluating WASD behaviour and performance for development purposes. Appropriate reports have a refresh selector allowing the report to be updated at the selected period. The following list is in the approximate order in which they occur top-to-bottom, left-to-right in the menu layout.

It is possible to use this facility standalone, without configuring authorisation (9.1 Access Before Configuration).

9.5HTTPd Server Revise

The server provides a comprehensive configuration revision facility.

Enabling Server Access

Many of the server activites listed above require server account write access to the directory in which the configuration files are stored. Where an autononmous scripting account is in use this poses minimal threat to server configuration integrity.

  1. Specifically map the /wasd_root/local/ path and mark it as access always requiring authorization (ensure this is one on the first mappings in the file and certainly before any other /wasd_root/ ones).
    # WASD_CONFIG_MAP pass /wasd_root/local/* auth=all
  2. Add appropriate authorization rules (example from Authorization Configuration (Basics) of WASD Configuration).
    # WASD_CONFIG_AUTH ["Web Admin"=WASD_WEBADMIN=id] /httpd/-/admin/* r+w /wasd_root/local/* r+w
  3. Update access to the directory can be applied using the SECHAN utility (13.12 SECHAN Utility).
    $ SECHAN /WRITE WASD_ROOT:[000000]LOCAL.DIR $ SECHAN /WRITE WASD_ROOT:[LOCAL]
  4. Load the new mapping and authorization rules.
    $ HTTPD /DO=MAP $ HTTPD /DO=AUTH=LOAD
Alternative Using /PROFILE

If a site is using SYSUAF authentication and security profiles enabled using the /PROFILE startup qualifier (13.12 SECHAN Utility) then a more restrictive set up is possible, retaining the default no-access to the [LOCAL] directory. This relies on the administering account(s) having read and write access to the [LOCAL] directory. It is then not necessary to grant that to the server account. It is possible to limit the application of VMS user profiles. This is an example.

# WASD_CONFIG_MAP set /wasd_root/local/* profile auth=all set * noprofile

To use this approach perform steps 1, 2 and 4 from above, substituting the following for step 3.

$ SECHAN /PACKAGE WASD_ROOT:[000000]LOCAL.DIR $ SECHAN /PACKAGE WASD_ROOT:[LOCAL] $ SECHAN /CONTROL WASD_ROOT:[000000]LOCAL.DIR

9.6HTTPd Server Action

The server allows certain run-time actions to be initiated. Many of these functions can also be initiated from the command line, see 9.7 HTTPd Command Line.

When multiple servers are executing on a single node or within a cluster a JavaScript-driven checkbox appears in the bottom left of the administration menu. Checking that box applies any subsequently selected action to all servers!

Control Section
Configuration Action Section

9.7HTTPd Command Line

A foreign command for the HTTPD control functionality will need to be assigned in the adminstration users' LOGIN.COM, for example:

$ HTTPD == "$WASD_EXE:HTTPD"
or (perhaps more likely)
$ HTTPD == "$WASD_EXE:HTTPD_SSL"

Some control of the executing server is available from the DCL command line on the system on which it is executing. This functionality, via the /DO= qualifier, is available to the privileged user.

These directives are communicated from the command-line (and Server Administration page analogue - ‘Control Section’ in 9.6 HTTPd Server Action) to the per-node or per-cluster servers using the Distributed Lock Manager. On pre-VMS V8.2 the command buffer is limited to 15 bytes. From VMS V8.2 the buffer space available is 63 bytes. In a cluster all systems must support the larger buffer before WASD enables it. The smaller buffer space limits some of the directives that take free-form parameters (e.g. /DO=DCL=PURGE=USER=DANIEL).

Multi-Server/Cluster-Wide

If multiple servers are executing on a host or cluster it is possible to control all of them by adding the /CLUSTER or /ALL qualifiers. Of course, these commands are available from batch jobs as well as interactively. In a clustered WASD environment the same functionality is available via checkboxes from the online Server Administration facility.

Need it to be jogged?

Can't quite remember what it can (and by implication can't) do?

$ HTTPD /DO=HELP
Server Log Annotation

Significant server events (e.g. restart, exit, mapping rule change) can often benefit (post-mortem :-) from an annotation in the server process log, especially in a production environment. The command-line /NOTE="<string>" can be used to insert the supplied string as an ad hoc annotation, or in conjunction with a /DO=".." CLI command.

$ HTTPD /NOTE="just a note test!" $ HTTPD /DO=RESTART /NOTE="adding services ""download."" and ""mail."""

The server process log annotation appear as follows.

%HTTPD-I-NOTE, 10-DEC-2017 22:32:30, just a note test! %HTTPD-I-NOTE, 10-DEC-2017 22:33:05, adding services "download." and "mail."

Note may also be inserted from the Server Admin main page by using the [/DO=] button and field and prefixing the string with /NOTE= (string delimitting quotation marks are not required). Using the Server Admin page annotation and commands cannot be combined.

9.7.1Accounting

Server counters may be zeroed. These counters are those visible from the statistics Server Admininstration item and when using the HTTPDMON utility.

$ HTTPD /DO=ZERO

The HTTPDMON utility displays a status line during startup or server exit on error. For example:

KLAATU:: 1 HTTPDMON v2.6.0 AXP Friday, 21-SEP-2018 21:40:54 Process: WASD:80 PID: 00001F9B User: HTTP$SERVER Version: 11.3.0 Up: 6 18:21:20.96 CPU: 0 00:07:25.54 Startup: 55 Exit: %X00000001 8< snip 8< Rx: 1,365,809 (0 err) Tx: 26,965,420 (0 err) (477kB/s) STATUS: %HTTPD-I-STARTUP, 21-SEP-2018 21:40:52, WASD:80

On occasion this can status message become constantly displayed (e.g. command-line misoperation) with

$ HTTPD /DO=ZERO=STATUS
restoring normal request information.

9.7.2Access Control

Server access via the WASD_CONFIG_ACCEPT and WASD_CONFIG_REJECT configuration files can reloaded and the IP reject list purged (cleared) using the following commands.

$ HTTPD /DO=ACCEPT $ HTTPD /DO=REJECT $ HTTPD /DO=REJECT=PURGE

9.7.3Alignment Faults

Does not apply to x86-64.

Alignment faults can be a significant performance issue and considerable effort has been invested in completely eliminating them. This was done using an internal reporting tool (primarily intended for the WASD developer) available from the Server Admin interface. Defining the logical name WASD_ALIGN_MAP to be a linker map of the build provides additional information.

$ HTTPD /DO=ALIGN=START $ HTTPD /DO=ALIGN=STOP $ HTTPD /DO=ALIGN=ZERO $ HTTPD /DO=ALIGN=FAULT=1

9.7.4Authentication

See 3. Authentication and Authorization.

The authorization rule file (WASD_CONFIG_AUTH) may be reloaded using either of these variants.

$ HTTPD /DO=AUTH $ HTTPD /DO=AUTH=LOAD

The authentication cache may be purged, resulting in re-authentication for all subsequent authorization-controlled accesses. This may be useful when disabling authorization or if a user has been locked-out due to too many invalid password attempts (3.9 Authorization Cache).

$ HTTPD /DO=AUTH=PURGE

A "skeleton-key" username and password may be entered, amongst things allowing access to the Server Administration facility (9. Server Administration).

$ HTTPD /DO=AUTH=SKELKEY=_<username>:<password>[:<period>]

9.7.5Cache

Server cache control may also be exercised from the Server Administration page (9. Server Administration). The file cache (see Cache Configuration of WASD Configuration) may be enabled, disabled and have the contents purged (declared invalid and reloaded) using

$ HTTPD /DO=CACHE=ON $ HTTPD /DO=CACHE=OFF $ HTTPD /DO=CACHE=PURGE

9.7.6Configuration Check

Changes to configuration files can be validated at the command-line before reload or restart. This detects and reports any syntactical and fatal configuration errors but of course cannot check the intent of the rules.

$ HTTPD /DO=AUTH=CHECK $ HTTPD /DO=CONFIG=CHECK $ HTTPD /DO=GLOBAL=CHECK $ HTTPD /DO=MAP=CHECK $ HTTPD /DO=MSG=CHECK $ HTTPD /DO=SERVICE=CHECK

The config check sequentially processes each of the authorization, global, mapping, message and service configuration files.

If additional server startup qualifiers are required to enable specific configuration features then these must also be provided when checking. For example:

$ HTTPD /DO=AUTH=CHECK /SYSUAF /PROFILE

9.7.7DCL/Scripting Processes

These commands can be useful for flushing any currently executing CGIplus applications from the server, enabling a new version to be loaded with the next access. See "Scripting Environment" document.

All scripting processes, busy with a request or not, can be deleted (this may cause the client to lose data).

$ HTTPD /DO=DCL=DELETE

A gentler alternative is to delete idle processes and mark busy ones for deletion when completed processing.

$ HTTPD /DO=DCL=PURGE

A more selective DELETE and PURGE is possible, where user name, script name, or script file name is supplied and only matching tasks have the specified action peformed.

$ HTTPD /DO=DCL=PURGE=USER=username $ HTTPD /DO=DCL=PURGE=SCRIPT=script-path $ HTTPD /DO=DCL=PURGE=FILE=script-file-name

When using the proctor facility (Script Proctor in WASD Scripting) revised rules in WASD_CONFIG_GLOBAL may be applied to the running server (proctored scripting processes created and deleted), or merely loaded into the server ruleset (requiring subsequent DCL=PURGE or DCL=DELETE to activate).

$ HTTPD /DO=DCL=PROCTOR=APPLY $ HTTPD /DO=DCL=PROCTOR=LOAD

9.7.8DECnet Scripting Connections

All DECnet connections, busy with a request or not, can be disconnected (this may cause the client to lose data).

$ HTTPD /DO=DECNET=DISCONNECT

Purging is a better alternative, disconnecting idle tasks and marking busy ones for disconnection when complete.

$ HTTPD /DO=DECNET=PURGE

9.7.9Hhelppp!

$ HTTPD /DO=HELP o ALIGN= START, STOP, ZERO with [<buf-size>,<items>,<mask>] o AUTH reload authorization file o AUTH=CHECK elementary check of authorization file … o ZERO zero all accounting o ZERO=NOTICED zero the 'errors noticed' accounting o ZERO=PROXY zero proxy accounting $

9.7.10HTTP/2 Connection

Disconnect idle HTTP/2 connections.

$ HTTPD /DO=HTTP2=PURGE

All HTTP/2 connections can be disconnected (this may cause clients to lose data), or a specific connection number.

$ HTTPD /DO=HTTP2=PURGE=ALL $ HTTPD /DO=HTTP2=PURGE=number

9.7.11Instances

The number of server instances (see 8.1 Server Instances) may be set from the command line. This overrides any configuration file directive and applies at the next startup. Any configuration directive value may be used from the command line.

$ HTTPD /DO=INSTANCE=MAX $ HTTPD /DO=INSTANCE=CPU $ HTTPD /DO=INSTANCE=integer

Note that the server must be restarted for this to take effect, that this can be applied to the current node only or to all servers within a cluster, and that it remains in effect until explicitly changed to "MAX" allowing the WASD_CONFIG_GLOBAL configuration directive [InstanceMax] to once again determine the number of instances required. The same functionality is available from the Server Administration page (9.6 HTTPd Server Action).

There are also directives to assist with WATCH activities (8.1 Server Instances).

$ HTTPD /DO=INSTANCE=PASSIVE $ HTTPD /DO=INSTANCE=ACTIVE

9.7.12Instance Status

Multi-instance (see 8.1 Server Instances) status (see 8.1.4 Status) can be reported from the command line using

$ HTTPD /DO=STATUS

In addition, stale entries in the status table may be purged using

$ HTTPD /DO=STATUS=PURGE
and the table completely emptied then repopulated over the next minute using
$ HTTPD /DO=STATUS=RESET

9.7.13Logging

Server logging control may also be exercised from the server administration menu (9. Server Administration).

Open the access log file(s).

$ HTTPD /DO=LOG=OPEN

Close the access log file(s).

$ HTTPD /DO=LOG=CLOSE

Close then reopen the access log file(s).

$ HTTPD /DO=LOG=REOPEN

Unwritten log records may be flushed to the file(s).

$ HTTPD /DO=LOG=FLUSH

9.7.14Mapping

See Request Processing Configuration of WASD Configuration.

The mapping rule file (WASD_CONFIG_MAP) may be reloaded using either of these variants.

$ HTTPD /DO=MAP $ HTTPD /DO=MAP=LOAD

9.7.15Network Connection

Current network connections can be listed at the CLI.

$ HTTPD /DO=NET=LIST

This can display in an 80 character terminal depending on column widths (e.g. service and client names) but in some circumstances will require 132 characters to use effectively. The CLI command requests the running server to generate a report and return that via the $BRKTHRU service.

Note that with HTTP/1.n there is a one-to-one relationship between requests in progress and a network connection, displayed as a single integer, e.g. 1651. With HTTP/2 there can be a many to one, where listed "connections" being processed (i.e. requests in progress) are virtual connections being transported by an independent actual connection, and displayed as 1639->1632, where ->1632 is the actual connection.

Connect Service / Request Client Time Duration ---------- ------------------- ---------------- -------- -------- 1651 https:wasd.lan:4443 router.lan,56314 08:05:02 6.636s [persistent:4] 1639->1632 https:wasd.lan:443 router.lan,56310 08:00:52 4.147s GET /httpd/-/admin/report/WATCH?rqp=1&rsp=1&con=1&err=1&htp=i&cl... 1626->1606 https:wasd.lan:443 router.lan,56267 07:59:57 00:10:45 GET /cgi-bin/smonitor?classes=&MODES=2&PROCESSES=3&SYSTEM=1&inte... ->1632 https:wasd.lan:443 router.lan,56310 08:00:36 19.88s current:1 peak:1 count:5 ->1606 https:wasd.lan:443 router.lan,53267 07:25:41 00:35:14 current:1 peak:4 count:13 1 HTTP/1.n, 2 via HTTP/2, 2 HTTP/2, 17-SEP-2021 07:58:17

Disconnect idle (persistent HTTP/1.n and HTTP/2) connections.

$ HTTPD /DO=NET=PURGE

All network connections can be disconnected (this may cause clients to lose data), selectively idle HTTP/1.n or HTTP/2 connections, a specific connection number and those matching the specified URI.

$ HTTPD /DO=NET=PURGE=ALL $ HTTPD /DO=NET=PURGE=HTTP1 $ HTTPD /DO=NET=PURGE=HTTP2 $ HTTPD /DO=NET=PURGE=number $ HTTPD /DO=NET=PURGE=URI=pattern

Additionally, network connection acceptance can be suspended (leaving in-progress requests to complete), suspended and in-progress disconnected, and resumed.

$ HTTPD /DO=NET=SUSPEND $ HTTPD /DO=NET=SUSPEND=NOW $ HTTPD /DO=NET=RESUME

9.7.16Shutdown and Restart

Server shutdown may also be exercised from the Server Administration page (9. Server Administration).

The server may be shut down, without loss of existing client requests. Connection acceptance is stopped and any existing requests continue to be processed until conclusion.

$ HTTPD /DO=EXIT

The server may be immediately and unconditionally shut down.

$ HTTPD /DO=EXIT=NOW

The server may be restarted, without loss of existing client requests. Connection acceptance is stopped and any existing requests continue to be processed until conclusion. This effectively causes the server to exit normally and the DCL wrapper procedure to restart it.

$ HTTPD /DO=RESTART

The now variant restarts the server immediately regardless of existing connections.

$ HTTPD /DO=RESTART=NOW

The when– quiet variant restarts the server whenever request processing drops to zero for more than one second. It allows (perhaps non-urgent) changes to be put into effect through restart when everything has gone "quiet" and no demands are being placed on the server.

$ HTTPD /DO=RESTART=QUIET

Significant server events such as these are prime candidates for server log annotation!

$ HTTPD /DO=RESTART=NOW /NOTE="Restarting the server just so I can note it :-)"

9.7.17Secure Sockets Layer

If the optional SSL component is installed and configured these directives become effective.

If X.509 authentication is enabled the Certificate Authority (CA) verification list can be reloaded.

$ HTTPD /DO=SSL=CA=LOAD

Server certificates, after being updated, may be reloaded into the running services (i.e. without restart). This is a synonym for /DO=SERVICE=LOAD.

$ HTTPD /DO=SSL=CERT=LOAD

If a private key password is not included with the encode key it is requested by the server during startup. The following example shows the directive and the resulting prompt. When entered the password is not echoed.

$ HTTPD /DO=SSL=KEY=PASSWORD Enter private key password []:

9.7.18Throttle

Unconditionally release all queued requests for immediate processing.

$ HTTPD /DO=THROTTLE=RELEASE

Unconditionally terminate all requests queued waiting for processing. Clients receive a 503 "server too busy" response.

$ HTTPD /DO=THROTTLE=TERMINATE

For VMS V8.2 and later, a more selective RELEASE and TERMINATE is possible. A user name or script name can be supplied and only matching requests have the specified action peformed.

$ HTTPD /DO=THROTTLE=TERMINATE=REMOTE=pattern $ HTTPD /DO=THROTTLE=TERMINATE=SCRIPT=pattern

9.7.19WebSocket

Unconditionally disconnects all WebSocket applications.

$ HTTPD /DO=WEBSOCKET=DISCONNECT

For VMS V8.2 and later, more selective disconnects are possible. Disconnects WebSocket applications with connection number, with matching script names, and with matching scripting account usernames, respectively.

$ HTTPD /DO=WEBSOCKET=DISCONNECT=number $ HTTPD /DO=WEBSOCKET=DISCONNECT=SCRIPT=pattern $ HTTPD /DO=WEBSOCKET=DISCONNECT=USER=pattern

10.WATCH Facility

10.1Server Instances
10.2Event Categories
10.3Request Filtering
10.4Report Format
10.5WASD_WATCH_ONE_SHOT
10.6Usage Suggestions
10.7Command-Line Use

The WATCH facility is a powerful adjunct in server administration. From the Server Administration facility (9. Server Administration) it provides an online, real-time, in-browser-window view of request processing in the running server. The ability to observe live request processing on an ad hoc basis, without changing server configuration or shutting-down/restarting the server process, makes this facility a great configuration and problem resolution tool. It allows (amongst other uses)

A single client per server process can access the WATCH facility at any one time. It can be used in one of two modes.

Options immediately below the duration selector allows the WATCH output to concurrently be included in the server process log. This allows a permanent record (at least as permanent as server logs) to be simply produced.

10.1Server Instances

With a single instance (see 8.1 Server Instances) access to WATCH is always through the one server process. If multiple instances are configured WATCH requests, in common with all others, will be serviced by any one of the associated processes depending on the momentary state of the round-robin distribution.

This is often an issue for request WATCHing. The simplest scenario involves two instances. When the WATCH report is activated it will be serviced by the first process, when the request wishing to be WATCHed is accessed it (in the absence of any other server activity) will be serviced by the other process and will not be reported by WATCH on the first.

The solution is to suspend the round-robin request processing for the period of the WATCH activity. This does not shut any instance down but instead makes all but the supervisor instance quiescent. (Technically, it dequeues all the listening I/Os from non-supervisor instance server sockets, making the TCP/IP network driver send all connection requests to the one instance left with listening I/Os.) It is just a matter of making the non-supervisor instances active again when the WATCH activity is concluded.

This may be done from the command-line using

$ HTTPD /DO=INSTANCE=PASSIVE $ HTTPD /DO=INSTANCE=ACTIVE
or using the Server Administration facility (9. Server Administration) where there are [Active] and [Passive] buttons available when multiple instances are in use. Neither transition disrupts any requests being established or in-progress.

10.2Event Categories

An event is considered any significant point for which the server code has a reporting call provided. These have been selected to provide maximum information with minimum clutter and impact on server performance. Obvious examples are connection acceptance and closure, request path resolution, error report generation, network reads and writes, etc. Events are collected together into groupings to allow clearly defined areas of interest to be selected for reporting.

The report menu provides for the inclusion of any combination of the following categories.

Request
Response
General
Network
Other
Proxy
Code Modules

If the server has been compiled using the WATCH_MOD=1 macro a set of module WATCHing statements is included. These provide far more detailed processing information than available with the generic WATCH, are intended primarily for debugging the server during development and testing. This is considered a specialized tool, with the quantity and level of detail produced most likely proving counter-productive in addressing general site configuration issues. The module items are shown below the usual WATCH items.

10.3Request Filtering

By default all requests to all services are WATCHed. Fine control may be exercised over exactly which requests are reported, allowing only a selected portion of all requests being processed to be concentrated on, even on a live and busy server. This is done by filtering requests according the following criteria.

In addition there are in and out selectors against each of the filters which include or exclude the particular request based on it matching the filter.

These filters are controlled using fully-specified, wildcarded strings or using regular expression patterns (see Request processing Configuration of WASD Configuration). In common with all WASD processing, filter matching is case-insensitive. Of course, due to the point of application of a particular filter during request processing, some information may or may not be displayed. When a request is into or out of the report because of a matching filter a FILTER informational item is reported.

Examples
  1. This first example shows various strings and patterns that could be applied to the client filter.
    alpha.example.com *.example.com 131.185.250.202 131.185.250.* ^10.68.250.*|10.68.251.*
  2. This example various filters applied to the service (virtual server).
    beta.example.com:8000 beta.example.com:* http://* https:* *:80
  3. The request filter contains the entire HTTP request header. This includes multiple, newline-delimited fields. Filtering can be simple or quite complex. These examples filter all POST requests (either in or out of the report depending on the respective selector), and all POSTs to the specified script respectively.
    POST * POST /cgi-bin/example*

    These are the equivalent regular expressions but also will stop comparing at the end of the initial request line. The second, in this case, will also only filter against HTTP/1.1 version requests (note the final period matching the <CR> of the <CR><LF> carriage control).

    ^^POST .*$ ^^POST */cgi-bin/example *HTTP/1\.1.$

    This example uses a regular expression to constrain the match to a single header field (line, or newline-delimited string), matching all requests where the user agent reports using the "Gecko" browser component (Mozilla, Firefox, etc.)

    ^^User-agent:.*Gecko.*$
  4. The path and track filter. The path contains a proxied origin server request and so can be used to filter proxy requests to specific sites.
    /wasd_root/src/* /cgi-bin/* /web/*/cyrillic/* $ORoKJAOef8sAAAkuACc http://proxied.host.name/*
  5. The authentication filters, realm and user, can be used to select requests for a particular authenticated user, all authenticated requests or all non-authenticated requests, amongst other application. The realm field allows the authenticated user to be further narrowed as necessary. All of the following examples show only the user field with the default in selector set.

    Authenticated requests for user DANIEL.

    DANIEL

    All authenticated requests.

    %*

10.4Report Format

The following example illustrates the format of the WATCH report. It begins with multi-line heading. The first two record the date, time and official server name, with underline. The third provides the WASD server version. The fourth provides some TCP/IP agent information. Lines following can show OpenSSL version (if deployed), system information, server startup command-line, and then current server process quotas. The last three lines of the header provide a list of the categories being recorded, the filters in use, and the last, column headings described as follows:

Note that some items also include a block of data. The request header category does this, providing the blank-line terminated text comprising the HTTP header. Rule mapping also provides a block of information representing each rule as it is interpreted. Generally WATCH-generated information can be distinguished from other data by the uniform format and delimiting vertical bars. Initiative and imagination is sometimes required to interpret the free-form data but a basic understanding of HTTP serving and a little consideration is generally all that is required to deduce the essentials of any report.

01-NOV-2021 23:24:40 WATCH REPORT x86vms.lan:80 ------------------------------------------------- HTTPD_SSL 12.0.0 31-OCT-2021 07:38:27.62 DKA100:[WASD_ROOT.][X86_64]HTTPD_SSL.EXE (28-OCT-2021 02:51:54.41) HP TCPIP$IPC_SHR X6.0-12 (31-AUG-2021 20:01:12.49) OpenSSL 1.1.1k 25 Mar 2021 (Tue Mar 30 04:14:48 2021 UTC) [SYS0.SYSCOMMON.SSL111.INCLUDE]*.H SYS$COMMON:[SYSLIB]SSL111$LIBSSL_SHR32.EXE $ CC (V8.4-2L1/70430528) /DECC /STAND=RELAXED_ANSI /PREFIX=ALL /NAMES=UPPER /OPTIMIZE /NODEBUG /WARNING=(NOINFORM,DISABLE=(PREOPTW)) /FLOAT=IEEE /IEEE=DENORM /DEFINE=(WASD_VMS_V7,SESOLA,WATCH_CAT=1,WATCH_MOD=0,WASD_ACME=1,WASD_GETSPI=1) innotek GmbH VirtualBox with 2 CPUs and 3584MB running VMS V9.1-A (ODS-5 enabled, VMS NAML, VMS FIB, ODS-DIRECT enabled, ZLIB X00018292 (%RMS-E-FNF, file not found), REGEX enabled, lksb$b_valblk[64]) $ HTTPD /PRIORITY=4 /SYSUAF=(ID,SSL,PROXY)/PERSONA=RELAXED/PROFILE AST:1978/2000 BIO:1984/2000 BYT:4026752/4999424 DIO:977/1000 ENQ:462/500 FIL:293/300 PGFL:345472/512000 PRC:0/100 TQ:98/100 DCL Scripting: detached, as HTTP$NOBODY, PERSONA enabled Process: WASD:80 OTHER DKA100:[wasd_root.][startup]startup_server.com;1 DKA100:[wasd_root.][log_server]X86VMS_20211101015323.LOG;1 Instances: X86VMS::WASD:80 Watching: connect, request, req-header, response, error (539) via HTTP/2 Filter: NONE |Time_______|Module__|Line|Item__|Category__|Event...| |23:24:52.89 HTTP2REQ 0308 023002 CONNECT HTTP/2 begin 23 with gort.lan,53801| |++++++++++++++++++++++++++++++++++++++++++++ |23:24:52.89 HTTP2REQ 0324 023002 REQ-HEADER HEADER 371 bytes| GET /httpd/-/admin/ HTTP/1.1 accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 accept-encoding: br, gzip, deflate user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.2 Safari/605.1.15 accept-language: en-au authorization: ******************************* host: x86vms.gets-it.net |23:24:52.89 REQUEST 3703 023002 REQ-HEADER DATA| ENTRY 001 [012] $ {12}request_line={28}GET /httpd/-/admin/ HTTP/1.1 ENTRY 002 [014] > {6}accept={63}text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 ENTRY 003 [018] > {15}accept-encoding={17}br, gzip, deflate ENTRY 004 .001. > {10}user-agent={119}Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.2 Safari/605.1.15 ENTRY 005 [007] > {15}accept-language={5}en-au ENTRY 006 [031] > {13}authorization={30}****************************** ENTRY 007 [024] > {4}host={18}x86vms.gets-it.net |23:24:52.89 SERVICE 1747 023002 CONNECT VIRTUAL x86vms.gets-it.net:443| |23:24:52.89 REQUEST 4413 023002 REQUEST GET /httpd/-/admin/| |23:24:52.89 ADMIN 0265 023002 RESPONSE ADMIN /httpd/-/admin/| |23:24:52.89 REQUEST 1435 023002 REQUEST STATUS 200 (OK) rx:106 tx:19536 bytes 10.000ms 1,964,219 B/s| |-------------------------------------------- |23:24:52.89 HTTP2REQ 1165 023002 CONNECT HTTP/2 end 23 with gort.lan,53801| |23:24:53.40 HTTP2REQ 0308 025002 CONNECT HTTP/2 begin 25 with gort.lan,53801| |++++++++++++++++++++++++++++++++++++++++++++ |23:24:53.40 HTTP2REQ 0324 025002 REQ-HEADER HEADER 310 bytes| GET /rtt?ping HTTP/1.1 accept: */* accept-encoding: br, gzip, deflate user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.2 Safari/605.1.15 accept-language: en-au referer: https://x86vms.gets-it.net/httpd/-/admin/ host: x86vms.gets-it.net |23:24:53.40 REQUEST 3703 025002 REQ-HEADER DATA| ENTRY 001 [012] $ {12}request_line={22}GET /rtt?ping HTTP/1.1 ENTRY 002 [014] > {6}accept={3}*/* ENTRY 003 [018] > {15}accept-encoding={17}br, gzip, deflate ENTRY 004 .001. > {10}user-agent={119}Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.2 Safari/605.1.15 ENTRY 005 [007] > {15}accept-language={5}en-au ENTRY 006 [013] > {7}referer={41}https://x86vms.gets-it.net/httpd/-/admin/ ENTRY 007 [024] > {4}host={18}x86vms.gets-it.net |23:24:53.40 ADMIN 4414 025002 CONNECT RTT PING!| |23:24:53.40 REQUEST 1435 025002 REQUEST STATUS 204 (No Content) rx:60 tx:369 bytes 0.0s 0 B/s| |-------------------------------------------- |23:24:53.40 HTTP2REQ 1165 025002 CONNECT HTTP/2 end 25 with gort.lan,53801|

10.5WASD_WATCH_ONE_SHOT

When defined logical name WASD_WATCH_ONE_SHOT specifies the items shown in a one-shot WATCH. Without the logical name all items are displayed. For example

$ DEFINE /SYSTEM WASD_WATCH_ONE_SHOT 67109395
displays items connect, request, response, error, http/2 instead of all.

The integer representing these items can be found in the WATCH report header. See "|Watching: …" in examples above.

10.6Usage Suggestions

The following provides a brief explanation on the way WATCH operates and any usage implications.

A single client may be connected to the WATCH facility at any given time. When connecting the client is sent an HTTP response header and the WATCH report heading lines. The request then remains connected until the WATCH duration expires or the client overtly aborts the connection. During this period the browser behaves as if receiving a sometimes very slow, sometimes stalled, plain-text document. As the server processes WATCHable events the text generated is sent to the WATCH-connected client.

If the connection is aborted by the user some browsers will consider document retrieval to be incomplete and attempt to reconnect to the service if an attempt is made to print or save the resulting document. As the printing of WATCH information is often quite valuable during problem resolution this behaviour can result in loss of information and generally be quite annoying. Appropriate use of the duration selector when requesting a report can work around this, as at expiry the server disconnects, browsers generally interpreting this as legitimate end-of-document (when no content-length has been specified).

During report processing some browsers may not immediately update the on-screen information to reflect received data without some application activity. If scroll-bars are present on the document window manipulating either the horizonal or vertical slider will often accomplish this. Failing that minimizing then restoring the application will usually result in the most recent information being visible.

Browser reload/refresh may be used to restart the report. A browser will quite commonly attempt to remain at the current position in the document, which with a WATCH report's sustained but largely indeterminate data stream may take some time to reach. It is suggested the user ensure that any vertical scroll-bar is at the beginning of the current report, then refresh the report.

Selecting a large number of categories, those that generate copious output for a single event (e.g. response body) or collecting for extended periods can all result in the receipt of massive reports. Some browsers do not cope well with documents megabytes in size.

Note

WATCH reports are written using non-blocking I/O into an internal buffer. This buffer is written when filled, or flushed at a one second interval. Slight latency may be experienced with sporadic WATCH report items.

When supplying WATCH output as part of a problem report please ZIP the file and include it an an e-mail attachment. Mailers often mangle the report format making it difficult to interpret.

10.7Command-Line Use

Although intended primarily as a tool for online use WATCH can be deployed at server startup with a command-line qualifier and provide report output to the server process log. This is slightly more cumbersome than the Web interface but may still be useful in some circumstances. Full control over event categories and filters is possible.

The following examples illustrate the command-line WATCH specification.

/NOWATCH /WATCH=NOSTARTUP,ITEMS=(REQUEST,RESPONSE,MAPPING) /WATCH="ITEMS=(REQUEST,RESPONSE,ERROR),*,*,/cgi-bin/*" /WATCH=LIST

11.Server Performance

11.1Simple File Request Turn-Around
11.2Scripting
These Are v12.1 X86 Results

The server has a single-process, multi-threaded, asynchronous I/O design. On a single-processor system this is the most efficient approach. On a multi-processor system it is limited by the single process context (with scripts executing within their own context). For I/O constrained processing (the most common in general Web environments) the AST-driven approach is quite efficient.

The test-bench system was

Dell Optiplex 9020 4 core i7 3.4Ghz 16GB Win 10 Pro 22H2
innotek GmbH VirtualBox with 2 CPU and 7574MB running VMS V9.2
Approximate System VUPs Rating : 284.4 ( min: 281.4 max: 286.6 )

... not the fastest kid on the block

but as with previous benchmarks, these are intended as relative not absolute comparisons. Importantly, it has recent releases of system software, courtesy of VSI's ISV support programme. For performance purposes, this allows comparison with recent releases of CSWS (VMS Apache).


This performance data (WASD v12.1) has been collected very differently to those over a decade ago (WASD v10.0). Earlier benchmarking tools were WASD-in-house, ApacheBench (AB) and WASDbench (WB), executing on the same system as the server, eliminating network traffic on-the-wire.

Benchmark Setup

VMS V9.2 with VSI TCP/IP Services for OpenVMS x86_64 Version X6.0

The data have been collected using the h2load utility (https://nghttp2.org/documentation/h2load.1.html) from the HTTP/2 C Library (https://nghttp2.org). This utility can be used to configurably load HTTP, HTTPS and HTTP/2 servers. Note that the number of client threads (-t) is explicitly set to the connection concurrency (-c) to maximise h2load processing. The h2load utility maintains connection persistence between HTTP/1.1 requests so the network connection setup is generally only a factor for the first of many.

The h2load utility is running on a 6 core Intel Core i7 3.2 GHz 16GB Mac Mini, across a 1Gbps LAN to the Dell.

Shell scripts for h2load benchmarking runs are included in the WASD_ROOT:[EXERCISE]PERF*. directory.

Clear text HTTP (port 80) data is collected to measure internal server processing without the CPU-intensive overhead of encryption. Encrypted HTTP (port 443) data provides more real-world scenarios (especially now clear-text is largely deprecated). Both WASD and Apache were using OpenSSL 3.0 and negotiated TLS v1.3.

Output from h2load benchmarking runs are included in the WASD_ROOT:[EXERCISE]PERF*1210*.TXT directory and is summarised below.

These results are indicative only!

Every endeavour has been made to ensure the comparison is as equitable as possible. Both servers run on the same machine in the same relatively quiescent environment, access logging and name lookup enabled. Each test run was interleaved between each server to try and distribute any environment variations. Those runs that are very high throughput use a larger number of requests to improve sample period validity. Both servers were configured pretty-much "out-of-the-box", minimal changes (generally just enough to get the test environment going). Multiple data collections have yielded essentially equivalent relative results.

For the test-bench WASD v12.1 is present on ports 80 and 443.

Apache Comparison

The Apache comparison used the latest VSI X86VMS CSWS V2.4-54A kit (based on Apache 2.4.54 and OpenSSL 3.0).

Apache is present on ports 7780 and 7443.

11.1Simple File Request Turn-Around

A series of tests using batches of accesses. The first test returned an empty file measuring response and file access time, without any actual transfer. The second requested a file of 64k characters, testing performance with a more realistic load. All were done using one and ten concurrent requests.

HTTP/1.1 clear
Concurrency 1
Requests/Second Data Rate MBps
Response WASD Apache WASD Apache
0k 1409 104 0.465 0.026
64k 124 78 7.59 4.8
Concurrency 10
Requests/Second Data Rate MBps
Response WASD Apache WASD Apache
0k 205 12.5 0.674 0.032
64k 29 8 17.9 5.0
HTTP/1.1 encrypted
Concurrency 1
Requests/Second Data Rate MBps
Response WASD Apache WASD Apache
0k 869 91 0.319 0.023
64k 116 59 7.14 3.6
Concurrency 10
Requests/Second Data Rate MBps
Response WASD Apache WASD Apache
0k 606 135 0.223 0.034
64k 97 79 5.95 4.83
HTTP/2 (encrypted)

(VMS Apache currently does not support HTTP/2)

Concurrency 1
Requests/Second Data Rate MBps
Response WASD Apache WASD Apache
0k 496 - 0.068 -
64k 107 - 6.53 -
Concurrency 10
Requests/Second Data Rate MBps
Response WASD Apache WASD Apache
0k 437 - 0.061 -
64k 90 - 5.52 -

Data file (extraneous output snipped):

File Transfer Rate

Requests for a large binary file (5.69MB - 11673 blocks) indicate a potential transfer rate of multiple tens of Mbytes per second.

Data Rate - MBytes/Second

(VMS Apache currently does not support HTTP/2)

Concurrent WASD Apache
HTTP/1.1
(clear)
1 33.71 20.54
10 35.51 32.61
HTTP/1.1
(encrypted)
1 11.03 9.16
10 11.82 12.25
HTTP/2
(encrypted)
1 11.93 -
10 12.00 -

Data file (extraneous output snipped):

File Record Format

The WASD server can handle STREAM, STREAM_LF, STREAM_CR, FIXED and UNDEFINED record formats very much more efficiently than VARIABLE or VFC files. With STREAM, FIXED and UNDEFINED files the assumption is that HTTP carriage-control is within the file itself (i.e. at least the newline (LF), all that is required required by browsers), and does not require additional processing. With VARIABLE record files the carriage-control is implied and therefore each record requires additional processing by the server to supply it. Even with variable record files having multiple records buffered by the HTTPd before writing them collectively to the network improving efficiency, stream and binary file reads are by Virtual Block and are written to the network immediately making the transfer of these very efficient indeed!

11.2Scripting

A simple performance evaluation shows the relative merits of WASD scripting and Apache in CGI and persistent environments, using WASD_ROOT:[SRC.CGIPLUS]CGIPLUSTEST.C which executes in standard CGI, CGIplus and Apache loadable module environments. CGIplus and Apache modules are somewhat analagous. A series of accesses were made. The first test returned only the HTTP header, evaluating raw request turn-around time. The second test requested a body of 64k characters, again testing performance with a more realistic load.

Concurrency 1 - Requests/Second
Response WASD CGI WASD CGIplus Apache CGI Apache module
0kB 19 143 7 77
64kB 12 18 6 45
Concurrency 10 - Requests/Second
Response WASD CGI WASD CGIplus Apache CGI Apache module
0kB 28 337 10 51
64kB 22 65 9 73

Data file (extraneous output snipped):

Persistent Scripting

CGI scripting is notoriously slow (as above), hence the effort expended by designers in creating persistent scripting environments - those where the scripting engine (and perhaps other state) is maintained between requests. Both WASD and Apache implement these as integrated features, the former as CGIplus/RTE, and in the latter as loadable modules.

The CGIplus and Apache module data from the above CGIPLUSTEST.EXE table show the benefits of having scripts persist, reducing activation latency, thereby increasing throughput, and potentially retaining state, including the scripts themselves in local caches. Both WASD and VMS Apache use their respective persistence technologies to provide common scripting environments, including Perl, PHP and Python.

The WASD CGIplus/RTE technology used to implement its persistent scripting environments are available for general use and based on CGI principles offer a ready adaptation of well-known principles. Most site-specific scripts can also be built using the libraries, code fragments, and example scripts provided with the WASD package, and obtain similar efficiencies and low latencies. See WASD Scripting Environment document.


12.HTTPd Web Update

The Update facility allows Web documents and file environments to be administered from a standard browser. This capability is available to Web administrator and user alike. Availability and capability depends on the authorization environment within the server.

It should be stressed that this is not designed as a full hypertext administration or authoring tool, and for document preparation relies on the editing capabilities of the <TEXTAREA> widget of the user's browser. It does however, allow ad-hoc changes to be made to documents fairly easily, as well as allowing documents to be deleted, and directories to be created and deleted.

Consult the Current UPDate documentation for usage detail.

Here is an example of the interface (access may be denied).

Update Access Permission

If SSL is in use (see 4. Transport Layer Security) then username/password privacy of the authorization environment is inherently secured via the encrypted communications. To restrict web update functionality to this secure environment add the following to the WASD_CONFIG_MAP configuration file:

/upd/* "403 Access denied." ![sc:https]

Of course, the user must have write (POST/PUT) access to the document or area on the server (i.e. the path) and the server account have file system permission to write into the parent directory.

The server will report "Insufficient privilege or object protection violation ... /path/document" if it does not have file system permission to write into a directory.

Also see 3.13 Controlling Server Write Access for information on write access control for the server account.


13.Utilities and Facilities

13.1Echo Facility
13.2Hiss Facility
13.3Stream Facility
13.4Where Facility
13.5Xray Facility
13.6CALogs
13.7CSPreport[er]
13.8HTAdmin
13.9HTTPd Monitor
13.10MD5digest
13.11QDLogStats
13.12SECHAN Utility
13.13StreamLF Utility
13.14WAStee Utility
13.15WOTSUP Utility

Foreign commands for external utilities (and the HTTPD control functionality) will need to be assigned from the adminstration users' LOGIN.COM either explicitly or by calling the WASD_ROOT:[EXAMPLE]WASDVERBS.COM procedure.

$ AB == "$WASD_EXE:AB" $ HTTPD == "$WASD_EXE:HTTPD" $ HTTPDMON == "$WASD_EXE:HTTPDMON" $ MD5DIGEST == "$WASD_EXE:MD5DIGEST" $ QDLOGSTATS == "$WASD_EXE:QDLOGSTATS" $ SECHAN == "$WASD_EXE:SECHAN" $ STREAMLF == "@WASD_EXE:STREAMLF" $ WB == "$WASD_EXE:WB"

13.1Echo Facility

Ever had to go to extraordinary lengths to find out exactly what your browser is sending to the server? The server provides a request echo facility. This merely returns the complete request as a plain-text document. This can be used for for checking the request header lines being provided by the browser, and can be valuable in the diagnosis of POSTed forms, etc.

This facility must be enabled through a mapping rule entry.

script /echo/* /echo/*

It may then be used with any request merely by inserting "/echo" at the start of the path, as in the following example.

http://www.example.com/echo/wasd_root/

13.2Hiss Facility

The hiss facility provides a response stream made up of random alpha-numeric characters (a sort of alpha-numeric white-noise). No response header is generated and the stream will continue (by default) up to one megabyte of output, or until the client closes the connection. This maximum may be controlled my appending an integer representing the number of kilobytes maximum to the mapping. This facility must be enabled through a mapping rule entry and may then be used for specific requests.

map /**.dll* /hiss/64/*.dll* map /**/system32/* /hiss/64/*/system32/* map /**default.ida* /hiss/64/*default.ida* script /hiss/* /hiss/*

Usage details are described in Security Considerations of WASD Configuration).

13.3Stream Facility

The stream facility provides a quantified or unlimited response stream of printable or binary octets. It is intended as a light-weight data source delivering content at the maximum throughput capable by the server and platform. This can be used as a test source or for end-to-end metrics. This facility must be enabled through a mapping rule.

script /stream/* /stream/*

It may then be used to generate streams of data with various characteristics and sizes by including parameters in the URL.

13.4Where Facility

Need to locate where VMS has the HTTPd files? This simple facility maps the supplied path then parses it to obtain a resulting VMS file specification. This does not demonstrate whether the path actually exists!

This facility must be enabled through a mapping rule entry.

script /where/* /where/*

It may then be used with any request merely by inserting "/where" at the start of the path, as in the following example.

http://www.example.com/where/wasd_root/

13.5Xray Facility

The Xray facility returns a request's complete response, both header and body, as a plain text document. Being able to see the internals of the response header as well as the contents of the body rendered in plain text can often be valuable when developing scripts, etc.

This facility must be enabled through a mapping rule entry.

script /Xray/* /Xray/*

It may then be used with any request merely by inserting "/xray" at the start of the path, as in the following example.

http://www.example.com/xray/wasd_root/

13.6CALogs

The Consolidate Access LOGS utility (pronounced similar to the breakfast cereal brand :-) merges multiple HTTP server common and combined format access logs into a single log file with records in time-order. Due to the granularity of HTTP server entry timestamps (one second) the records are sorted to the one second but not within the one second.

It uses RMS and the VMS sort-merge routines to provide the basic consolidation functionality. An RMS search uses the supplied wildcard log file specification. Matching files are opened and each record read. The date/time field is parsed and a binary timestamp generated. Records with formats or date/time fields that do not make sense to the utility are discarded. When all files have been processed the sort-merge is performed using the timestamp as the key. The sorted records are then written to the specified output file.

$ calogs <log-file-spec> [<output-file-name>] [<qualifiers>]

Parameters and Qualifiers
Parameter Description
/HELP basic usage information
/NOPROXY discard proxy service records
/NOWASD discard WASD server status/timestamp entries
/OUTPUT= alternate method of specifying merged file name
/PROXY discard non-proxy service records
/QUIET no messages apart from errors
/VERBOSE per-file progress messages
/VERSION display the utility version and copyright message
Usage Examples
$ CALOGS == "$WASD_EXE:CALOGS" $ CALOGS WASD_LOGS:*200205*.LOG 2002_MAY.LOG $ CALOGS /VERBOSE WASD_LOGS: $ CALOGS /NOWASD WASD_LOGS:*200206*.LOG_* /OUTPUT=2002_JUNE.LOG $ CALOGS /PROXY /NOWASD WASD_LOGS:*2002*.LOG 2002_PROXY.LOG

13.7CSPreport[er]

Content Security Policy (CSP) is an added layer of security that helps to detect and mitigate certain types of attacks, including Cross Site Scripting (XSS) and data injection attacks.

https://en.wikipedia.org/wiki/Content_Security_Policy
https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP

WASD provides CSP support using mapping rules. See Content Security Policy (CSP) of WASD Configuration.

When POSTed to, this utility appends a timestamp and CSP report JSON to the file specified by the CSPREPORT_FILE logical name. This file must be located somewhere the scripting account has read+write access to. When accessed using a GET the utility accesses the stored CSP reports and returns a formatted HTML report listing each. GET requests (reporting) must be subject to authentication and authorisation.

For further infomation check the descriptive prologue in the WASD_ROOT:[SRC.UTILS]CSPREPORT.C source code.

13.8HTAdmin

The HTAdmin utility assists in with the command-line maintenance of $HTA authorization databases. See Authorization Configuration (Basics) of WASD Configuration and 3. Authentication and Authorization.

htadmin <database> [<username>] [<qualifiers>]

Parameters and Qualifiers
Parameter Description
/ADD add a new record
/CONFIRM confirm deletion of database
/CONTACT="<string>" contact information for record
/CREATE create a new database
/CSV[=TAB|char] comma-separated listing (optional character)
/DATABASE= database name (or as command-line parameter)
/DELETE delete a database or username record from a database
/DISABLED username record is disabled (cannot be used)
/EMAIL="<string>" email address for record
/ENABLED username record is enabled (can be used)
/FULL listing showing full details
/GENERATE generate a six character password
/HELP basic usage information
/[NO]HTTPS synonym for /SSL
/LIST listing (brief by default, see /FULL and /CSV)
/MODIFY synonym for /UPDATE
/NAME="<string>" full name for username record
/OUTPUT= alternate output for database listing
/PASSWORD[=<string>] username record password (prompts if not supplied)
/PIN generate four-digit "PIN number" for password
/[NO]READ username can/can't read
/SORT[=<parameters>] sort the records into a new/another database
/[NO]SSL user can only authenticate via SSL ("https:")
/[NO]WRITE username can/can't write
/UPDATE update an existing username record
/USER=<string> username
/VERSION display version of HTADMIN
Usage Examples
Sort Details

The /SORT qualifier sorts the current database records according to the /SORT= parameters. It can be used with the /LIST qualifier to produce ordered reports or will output the records into another authentication file. By default it sorts ascending by username. Qualifier parameters allow a sort by DATE or COUNT. Each of these allows the further specification of which date or count; ACCESS, CHANGE or FAILURE.

13.9HTTPd Monitor

The HTTP server may be monitored in real-time using the HTTPDMON utility.

This utility continuously displays a screen of information comprising four or five of the following sections:

  1. System Information
    The nodename, instance number(s), monitor version and current date/time.
  2. Process Information
    HTTPd process information includes its up-time, CPU-time consumed (excluding any subprocesses), I/O counts, and memory utilization. The "Servers:" item shows how many servers are currently running on the node/cluster. Changes in this count are indicated by the second, parenthesized number.
  3. General Server Counters
    The server counters keep track of the total connections received, accepted, rejected, etc., totals for each request type (file transfer, directory listing, image mapping, etc.).
    ** The request count of 3.8M is a real value, as are the others, with the screenshot taken during x86-64 (V9.1-A) testing using OWASP ZAP.
  4. Proxy Serving Counters
    The server counters keep track of proxy serving connections, network and cache traffic, cache status, etc.
  5. Latest Request
    This section provides the response status code, and some transaction statistics, the service being accessed, originating host and HTTP request. Note that long request strings may be truncated (indicated by a bolded ellipsis).
  6. Status Message
    If the server is in an exceptional condition, for example exited after a fatal error, starting up, etc., a textual message may be displayed in place of the the request information. This may be used to initiate remedial actions, etc.

The "/HELP" qualifier provides a brief usage summary.

The server counter values are carried over when a server (re)starts (provided the system has stayed up). To reset the counters use the online Server Administration facility (9. Server Administration).

If [DNSlookup] is disabled for the HTTP server the HTTPDMON utility attempts to resolve the literal address into a host name. This may be disabled using the /NORESOLVE qualifier.

13.10MD5digest

From RFC1321 …

" The [MD5] algorithm takes as input a message of arbitrary length and produces as output a 128-bit "fingerprint" or "message digest" of the input. It is conjectured that it is computationally infeasible to produce two messages having the same message digest, or to produce any message having a given prespecified target message digest. "

The MD5DIGEST utility is primarily provided with WASD for verifying kits as unchanged from the originals released. With the proliferation of mirror sites and other distribution resources it has become good practice to ensure kits remain unchanged from release, to distribution, to installation site (changes due to to data corruption or malicious intent - as remote a possibility as that may seem). Of course it may also be used for any other purpose where the MD5 hash is useful.

For verifying the contents of a WASD release connect to the original WASD distribution site, refer to the download page, and make a comparison between the release MD5 hash found against the list of all archive hashes and the MD5 hash of your archive. That can be done as follows

$ MD5DIGEST == "$WASD_EXE:MD5DIGEST" $ MD5DIGEST device:[dir]archive.ZIP
The result will look similar to
MD5 (kits:[000000]htroot710.zip;1) = 404bbdfe0f847c597b034feef2d13d2d

Of course, if you have not yet installed your first WASD distribution using the MD5DIGEST utility that is part of it is not feasable. The original site can provide kits and pre-built executables for this purpose.

13.11QDLogStats

Quick-and-Dirty LOG STATisticS is a utility to extract very elementary statistics from Web server common/combined format log files. It is intended for those moments when we think "I wonder how many times that new archive has been downloaded?", "How much data was transfered during November?", "How often is such-and-such a client using the authenticated so-and-so service?", "How much has the mail service been used?" … and want the results in a matter of seconds (or at least a few tens of seconds ;-) It is available at the command-line and as a CGI script.

For QDLOGSTATS to be available as a CGI script it must have authorization enabled against it (to prevent potential ad hoc browsing of a site's logs). The following provides some indication of this configuration, although of course it requires tailoring for any given site.

[VMS] /cgi-bin/qdlogstats ~webadmin,131.185.250.*,r+w ;

It could then be accessed using

http://the.host.name/cgi-bin/qdlogstats

The initial access provides a form allowing the various filters and other behaviours to be selected. The CGI form basically parallels the command-line behaviour described below.

Filters

A number of filters allow subsets of the log contents to be selected. These filters support the same string matching expressions as the server (see String Matching of WASD Configuration).

A knowlege of the format and contents of the common and combined log formats will assist in deciding which and to what purpose filters should be used. Record filtering is done in the same order as is finally displayed, so method would be processed before user-agent for instance. Normally a record match terminates on the first non-matched filter (to expedite processing). To compare and report each filter for every record apply the /ALL qualifier. To view records as they are processed use the /VIEW qualifier. This by default displays all matched records, but the optional =ALL or =NOMATCH parameters will display all records, or all those but the matches.

QDLOGSTATS log-file-spec [pattern qualifiers] [other qualifiers]

Parameters and Qualifiers
Parameter Description
/ALL compare and report on all supplied filters
/AUTHUSER= pattern (any authenticated username)
/BEFORE= log files before this VMS date/time
/CLIENT= pattern (client host name or IP address)
/DATETIME= pattern ("11/Jun/1999:14:08:49 +0930")
/DECODE[=keyword] URL-decode PATH, QUERY, REFERER before match
/METHOD= pattern (HTTP "GET", "POST", etc.)
/OUTPUT= file specification
/PATH= pattern (URL path component only)
/PROGRESS show progress during processing; a "+" for each file started, a "." for each 1000 records processed
/QUERY= pattern (URL query component only)
/REFERER= pattern (HTTP "Referer:" field, COMBINED only)
/REMOTEID= pattern (RFC819 file)
/RESPONSE= pattern (HTTP response code)
/SINCE= log files after this VMS date/time
/SIZE[=keyword] response size (in bytes) MIN=integer MAX=integer
/USERAGENT= pattern (HTTP "User-Agent:" field, COMBINED only)
/VIEW[=type] display matching log records (ALL, NOMATCH, MATCH)
Usage Examples

13.12SECHAN Utility

The SECHAN utility (pronounced "session") is used by [INSTALL]SECURE.COM and associated procedures to make file system security settings. It is also available for direct use by the site administrator. See Security Considerations of WASD Configuration).

13.13StreamLF Utility

This simple procedure used the FDL facility to convert files to STREAM_LF format. The WASD HTTPd server access STREAM_LF files in block/IO-mode, far more efficiently that the record-mode required by variable-record format files.

NOTE: The server can also be configured to automatically convert any VARIABLE record format files it encounters to STREAM_LF.

13.14WAStee Utility

WAStee is a utility to generate time-stamped log files containing intervals of a long-lived WASD server process, and/or to consolidate all process log files generated during the defined period. It is the tee in a PIPE sequence.

This utility is UNSUITABLE for sites using multiple instances and/or environments on a node. Only the first of multiple server processes will have the log teed.

For further information check the descriptive prologue in the WASD_ROOT:[SRC.UTILS]WASTEE.C source code.

13.15WOTSUP Utility

The "WASD Over-The-Shoulder Uptime Picket" is designed to monitor WASD in a production environment for the purpose of alerting operations staff to conditions which might cause that production to be adversely impacted.

Alert triggers include:

Alert reports can be delivered via any combination of:

The utility runs in a detached process and monitors the server environment by periodically polling various server data at a default interval is 15 seconds. As the utility requires access to global memory accounting a per-system WOTSUP is required for each node to be monitored.

The following (somewhat contrived) example illustrates the format and content of a WOTSUP report delivered via OPCOM. Reports delivered via other mechanisms have the same content and similar format.

%%%%%%%%%% WOTSUP 24-OCT-2006 13:32:56.44 %%%%%%%%%%% Message from user SYSTEM on KLAATU Over-The-Shoulder (WASD_WOTSUP) reports: 1. server PID 001C0950 exit %X00000001 (%SYSTEM-S-NORMAL) 2. server STARTUP (10) 3. server PIDs are 0018C14F (HTTPd:80), 001C0950 (HTTPe:80) 4. pagfilcnt:395432 pgflquota:500000 79% <= 80%

For further information check the descriptive prologue in the WASD_ROOT:[SRC.UTILS]WOTSUP.C source code.


14.Index

A‘Abstract’ in WASD Features and Facilities
 9.1 Access Before Configuration
 9.2 Access Configuration
 9.7.2 Access Control
 ‘Access Control’ in 2. Package Overview
 ‘Access Filtering’ in 7.1.5 Controlling Proxy Serving
 ‘Access Restriction Keywords’ in 3.4 Authorization Configuration File
 9.7.1 Accounting
 3.10.1 ACME
 6.6.7 Adding a number-sign ("#") to the webfolder-address
 6.6.6 Adding a port number to the webfolder-address
 ‘Administration’ in 2. Package Overview
 ‘After Receiving The Certificate’ in 4.6.2 Certificate Signing Request
 9.7.3 Alignment Faults
 ‘Alternative Using /PROFILE’ in 9.5 HTTPd Server Revise
 ‘Apache Comparison’ in 11. Server Performance
 ‘Apache License, Version 2.0’ in WASD Features and Facilities
 ‘As of WASD v12.0 Caching is OBSOLETE’ in 7.2 Proxy Cache
 ‘At Home’ in 4.8 SSL Service Evaluation
 15. Attribution and Acknowledgement
 9.7.4 Authentication
 3. Authentication and Authorization
 ‘Authentication Cache’ in 3.16 Cancelling Authorization
 ‘Authentication Cache and Revalidation’ in 3.2 Authentication Policy
 ‘Authentication Failures’ in 3.2 Authentication Policy
 3.2 Authentication Policy
 3.5 Authentication Sources
 3.9 Authorization Cache
 3.8 Authorization Configuration Examples
 3.4 Authorization Configuration File
 4.5.12 Authorization Using X.509 Certification
 ‘Authorization Verification’ in 7.6.1 Reverse Proxy
 7.8.2 Automatic
 ‘Avoid "Interesting" File Names’ in 6.2.2 File Naming
 6.6.3 Avoiding Microsoft Property Clutter
B‘Benchmark Setup’ in 11. Server Performance
 ‘Bjöern Höehrmann’ in 15. Attribution and Acknowledgement
 7.8 Browser Proxy Configuration
 ‘By Resource’ in 4.5.16 X509 Configuration
 ‘By Service’ in 4.5.16 X509 Configuration
C9.7.5 Cache
 13.6 CALogs
 3.16 Cancelling Authorization
 ‘Caution’ in 6. WebDAV
 ‘CAUTION’ in 3.10.5 VMS Account Proxying
 4.5.17 Certificate Authority Verification File
 4.6 Certificate Management
 4.6.2 Certificate Signing Request
 ‘Chain Authorization’ in 7.1.4 Proxy Chaining
 ‘Chain Password’ in 7.1.5 Controlling Proxy Serving
 ‘Chaining FIREWALL’ in 7.7.3 [ServiceProxyTunnel] FIREWALL
 ‘Chaining RAW’ in 7.7.2 [ServiceProxyTunnel] RAW
 ‘Changing Server Certificates’ in 4.6.1 Server Certificate
 ‘Clark Cooper, et.al.’ in 15. Attribution and Acknowledgement
 ‘Client Tools’ in 6.7 References
 ‘Code Modules’ in 10.2 Event Categories
 10.7 Command-Line Use
 7.7.7 Complex Private Tunneling
 ‘Concurrency 1’ in 11.1 Simple File Request Turn-Around
 ‘Concurrency 1’ in 11.1 Simple File Request Turn-Around
 ‘Concurrency 1’ in 11.1 Simple File Request Turn-Around
 ‘Concurrency 1 - Requests/Second’ in 11.2 Scripting
 ‘Concurrency 10’ in 11.1 Simple File Request Turn-Around
 ‘Concurrency 10’ in 11.1 Simple File Request Turn-Around
 ‘Concurrency 10’ in 11.1 Simple File Request Turn-Around
 ‘Concurrency 10 - Requests/Second’ in 11.2 Scripting
 6.2.5 Concurrent Authorisation
 ‘Configuration’ in 3.11 Token Authentication
 8.1.3 Configuration
 ‘Configuration Action Section’ in 9.6 HTTPd Server Action
 9.7.6 Configuration Check
 7.3 CONNECT Serving
 8.1.2 Considerations
 ‘Control Section’ in 9.6 HTTPd Server Action
 7.3.2 Controlling CONNECT Serving
 7.1.5 Controlling Proxy Serving
 3.13 Controlling Server Write Access
 6.1.1 COPY Restrictions
 ‘Cryptography Software’ in 4. Transport Layer Security
 13.7 CSPreport[er]
D‘Data Rate - MBytes/Second’ in 11.1 Simple File Request Turn-Around
 9.7.7 DCL/Scripting Processes
 9.7.8 DECnet Scripting Connections
 6.1.2 DELETE Restrictions
 ‘Deprecated and Discouraged’ in 3.10.4 WASD "Hard-Wired" Identifiers
 ‘Directory Metadata’ in 6.3 WebDAV Metadata
 7.6.4 DNS Wildcard Proxy
 6.5.3 Dreamweaver
E13.1 Echo Facility
 7.1.1 Enabling A Proxy Service
 7.3.1 Enabling CONNECT Serving
 ‘Enabling Server Access’ in 9.5 HTTPd Server Revise
 ‘Enabling SOCKS5 Proxy’ in 7.4 SOCKS Version 5
 ‘Enabling SSL’ in 7.6.5 Originating SSL
 7.7.4 Encrypted Tunnel
 7.7.5 Encrypted Tunnel With Authentication
 6.6.11 Error 0x800700DF: The file size exceeds the limit allowed and cannot be saved
 ‘Error Messages’ in 7. Proxy Services
 10.2 Event Categories
 ‘Example In Action’ in 7.7.7 Complex Private Tunneling
 ‘Examples’ in 3.12 Skeleton-Key Authentication
 ‘Examples’ in 10.3 Request Filtering
 ‘Extension Visibility’ in 4.5.15 Subject Alternative Name and Other Extensions
 ‘External Mapping’ in 7.7.7 Complex Private Tunneling
 ‘External Services’ in 7.7.7 Complex Private Tunneling
F‘Fail-Through’ in 8.1.1 VMS Clustering Comparison
 4.5.14 Features
 ‘File Name Ambiguity’ in 6.2.2 File Naming
 6.2.2 File Naming
 ‘File Record Format’ in 11.1 Simple File Request Turn-Around
 ‘File Transfer Rate’ in 11.1 Simple File Request Turn-Around
 6.2.3 File-system Access
 6.2.4 File-system Authorisation
 ‘Filters’ in 13.11 QDLogStats
 6.6.8 Force Windows XP to use Basic Authentication
 4.5.5 Forward Secrecy
 ‘Free Software Foundation’ in 15. Attribution and Acknowledgement
 6.6.2 FrontPage Extensions
 7.5 FTP Proxy Serving
 7.5.1 FTP Query String Keywords
G7.6 Gatewaying Using Proxy
 ‘General’ in 2. Package Overview
 ‘General’ in 10.2 Event Categories
 ‘General Setup’ in 4.5.16 X509 Configuration
 5.3.1 Global Configuration
 6.5.2 Gnome/gvfs/Nautilus
H9.7.9 Hhelppp!
 13.2 Hiss Facility
 13.8 HTAdmin
 ‘HTTP Methods’ in 3.3 Permissions, Path and User
 6.1 HTTP Methods Supported
 7.1 HTTP Proxy Serving
 ‘HTTP Report’ in 5.2 HTTP/2 and Performance
 ‘HTTP/1.1 clear’ in 11.1 Simple File Request Turn-Around
 ‘HTTP/1.1 encrypted’ in 11.1 Simple File Request Turn-Around
 5. HTTP/2
 ‘HTTP/2 (encrypted)’ in 11.1 Simple File Request Turn-Around
 5.2 HTTP/2 and Performance
 ‘HTTP/2 and WATCH’ in 5.1 WASD HTTP/2
 5.3 HTTP/2 Configuration
 9.7.10 HTTP/2 Connection
 5.4 HTTP/2 Detection
 ‘HTTP/2 Global Configuration’ in 5.3.1 Global Configuration
 5.5 HTTP/2 References
 5.3.3 HTTP/2 Set Rules
 9.7 HTTPd Command Line
 13.9 HTTPd Monitor
 9.6 HTTPd Server Action
 9.4 HTTPd Server Reports
 9.5 HTTPd Server Revise
 12. HTTPd Web Update
I6.1.4 If: Restrictions
 ‘Implication’ in 3.9 Authorization Cache
 ‘Important’ in 4.5.12 Authorization Using X.509 Certification
 14. Index
 9.7.12 Instance Status
 9.7.11 Instances
 8. Instances and Environments
 ‘Internal Mapping’ in 7.7.7 Complex Private Tunneling
 ‘Internal Services’ in 7.7.7 Complex Private Tunneling
 2.4 International Features
 1. Introduction
 ‘Is it all worth it?  As might be expected – that depends.’ in 5.2 HTTP/2 and Performance
 ‘It's fair to say…’ in 5.1 WASD HTTP/2
K3.8.1 KISS
L7.5.2 "login" Keyword
 4.1 Let's Encrypt
 ‘Let's Encrypt’ in 4.6.1 Server Certificate
 ‘License’ in WASD Features and Facilities
 ‘Licensed under the Apache License, Version 2.0’ in 15. Attribution and Acknowledgement
 ‘Load Sharing’ in 8.1.1 VMS Clustering Comparison
 ‘Loading Authority Certificates’ in 4.6.1 Server Certificate
 ‘Local Password’ in 7.1.5 Controlling Proxy Serving
 ‘Lock Depth 0’ in 6.4 WebDAV Locking
 ‘Locking Depth’ in 6.4 WebDAV Locking
 ‘Locking Timeout’ in 6.4 WebDAV Locking
 9.7.13 Logging
 3.10.2 Logon Type
M7.8.1 Manual
 9.7.14 Mapping
 6.6.1 Mapping
 13.10 MD5digest
 ‘Metadata Files’ in 6.3 WebDAV Metadata
 ‘Metadata should not be edited manually ...’ in 6.3 WebDAV Metadata
 ‘Metadata XML’ in 6.3 WebDAV Metadata
 ‘Method’ in 9.1 Access Before Configuration
 ‘Microsoft Metadata’ in 6.3 WebDAV Metadata
 6.6 Microsoft Miscellanea
 6.6.10 Microsoft Windows 7 BASIC Authentication
 6.6.9 Microsoft XP Explorer BASIC Authentication
 6.1.3 MOVE Restrictions
 ‘much of the following is pre- Windows 10/11’ in 6.6 Microsoft Miscellanea
 ‘Multi-Server/Cluster-Wide’ in 9.7 HTTPd Command Line
 ‘Multiple Source Types’ in 3.5 Authentication Sources
N‘Need it to be jogged?’ in 9.7 HTTPd Command Line
 ‘Network’ in 10.2 Event Categories
 9.7.15 Network Connection
 3.10.6 Nil-Access VMS Accounts
 ‘None of the following licensing appears incompatible with the Apache License’ in 15. Attribution and Acknowledgement
 ‘not really an endorsement but’ in 4.6 Certificate Management
 ‘Note’ in 4.5.5 Forward Secrecy
 ‘Note’ in 7.1.5 Controlling Proxy Serving
 ‘Note’ in 3.5 Authentication Sources
 ‘Note’ in 7.6.2 Proxy Rework
 ‘Note’ in 7.6.4 DNS Wildcard Proxy
 ‘Note’ in 7.7.8 Tunnelling Source
 ‘Note’ in 3.15 User Password Modification
 ‘Note’ in 3.12 Skeleton-Key Authentication
 ‘Note’ in 3.10.8 SYSUAF Security Profile
 ‘Note’ in 10.6 Usage Suggestions
 ‘Note’ in 7.6.2 Proxy Rework
O‘Objectives’ in 1. Introduction
 ‘Ohio State University’ in 15. Attribution and Acknowledgement
 7.6.3 One-Shot Proxy
 ‘Online Search’ in WASD Features and Facilities
 ‘OpenSSL Options’ in 4.5.3 SSL Ciphers
 ‘OpenSSL Project’ in 15. Attribution and Acknowledgement
 4.4 OPENSSL.EXE Application
 6.6.4 OPTIONS header "MS-Author-Via: DAV"
 7.6.5 Originating SSL
 6.5.1 OS X Finder
 ‘Other’ in 10.2 Event Categories
 ‘Other Assessment’ in 5.2 HTTP/2 and Performance
 ‘Overview’ in 3. Authentication and Authorization
P2. Package Overview
 ‘Parameters and Qualifiers’ in 13.11 QDLogStats
 ‘Parameters and Qualifiers’ in 13.8 HTAdmin
 ‘Parameters and Qualifiers’ in 13.6 CALogs
 ‘Password Expiry’ in 3.15 User Password Modification
 ‘Paul E. Jones’ in 15. Attribution and Acknowledgement
 ‘Performance Assessment’ in 5.2 HTTP/2 and Performance
 ‘Performance Impact’ in 3.10.8 SYSUAF Security Profile
 3.3 Permissions, Path and User
 ‘Persistent Scripting’ in 11.2 Scripting
 ‘Proxy’ in 10.2 Event Categories
 7.1.2 Proxy Affinity
 7.1.3 Proxy Bind
 7.2 Proxy Cache
 ‘Proxy Cache is OBSOLETE’ in 7.2 Proxy Cache
 7.1.4 Proxy Chaining
 ‘Proxy Error Messages’ in 7. Proxy Services
 ‘Proxy Password’ in 7.1.5 Controlling Proxy Serving
 7.6.2 Proxy Rework
 7. Proxy Services
 ‘Proxy Serving Quick-Start’ in 7. Proxy Services
 ‘proxyMUNGE Utility’ in 7.6.2 Proxy Rework
Q13.11 QDLogStats
 ‘Qualys SSL Lab’ in 4.8 SSL Service Evaluation
R6.2.6 Real-World Example
 ‘Realm Description’ in 3.5 Authentication Sources
 3.6 Realm, Full-Access, Read-Only
 ‘Reasons For Yet Another Web Package’ in 1. Introduction
 ‘Redirection Location Field’ in 7.6.1 Reverse Proxy
 6.7 References
 6.6.5 Repairing broken XP Web Folders
 10.4 Report Format
 ‘Request’ in 10.2 Event Categories
 10.3 Request Filtering
 ‘Request Modification’ in 7.1.5 Controlling Proxy Serving
 ‘Request Redirect’ in 7.6 Gatewaying Using Proxy
 ‘Reserved Names’ in 3.4 Authorization Configuration File
 ‘Reserved Username’ in 3.4 Authorization Configuration File
 ‘Response’ in 10.2 Event Categories
 ‘Restart’ in 8.1.1 VMS Clustering Comparison
 7.6.1 Reverse Proxy
 3.10.3 Rights Identifiers
 ‘RSA Data Security’ in 15. Attribution and Acknowledgement
 3.1 Rule Interpretation
S‘Scripting’ in 2. Package Overview
 11.2 Scripting
 13.12 SECHAN Utility
 9.7.17 Secure Sockets Layer
 3.14 Securing All Requests
 ‘Self-Signed Certificates’ in 4.6.1 Server Certificate
 9. Server Administration
 2.1 Server Behaviour
 4.6.1 Server Certificate
 ‘Server CLI /SYSPLUS’ in 9.4 HTTPd Server Reports
 8.2 Server Environments
 10.1 Server Instances
 9.3 Server Instances
 8.1 Server Instances
 ‘Server Log Annotation’ in 9.7 HTTPd Command Line
 11. Server Performance
 5.3.2 Service Configuration
 4.5.6 Session Resumption
 ‘set /path/* SSLCGI=apache_mod_ssl’ in 4.7 SSL CGI Variables
 7.7.6 Shared SSH Tunnel
 ‘Should ACME be unavailable’ in 3.10.1 ACME
 9.7.16 Shutdown and Restart
 11.1 Simple File Request Turn-Around
 3.12 Skeleton-Key Authentication
 7.4 SOCKS Version 5
 6.5 Some Wrinkles
 ‘Sort Details’ in 13.8 HTAdmin
 4.5.11 SSL Access Control
 4.7 SSL CGI Variables
 4.5.3 SSL Ciphers
 4.5 SSL Configuration
 ‘SSL Options’ in 4.5.3 SSL Ciphers
 4.5.9 SSL Private Key
 4.9 SSL References
 4.5.8 SSL Server Certificate
 4.8 SSL Service Evaluation
 ‘SSL to RAW’ in 7.7.2 [ServiceProxyTunnel] RAW
 ‘SSL Versions’ in 4.5.2 TLS/SSL Versions
 4.5.10 SSL Virtual Services
 8.1.4 Status
 13.3 Stream Facility
 13.13 StreamLF Utility
 4.5.7 Strict Transport Security
 ‘String Matching’ in 3.1 Rule Interpretation
 ‘Stuart Langridge’ in 15. Attribution and Acknowledgement
 4.5.15 Subject Alternative Name and Other Extensions
 ‘System Report PLUS’ in 9.4 HTTPd Server Reports
 3.10.7 SYSUAF and SSL
 3.10.9 SYSUAF Profile For Full Site Access
 3.10.8 SYSUAF Security Profile
 3.10 SYSUAF-Authenticated Users
T‘Table of Content’ in WASD Features and Facilities
 ‘Tatsuhiro Tsujikawa’ in 15. Attribution and Acknowledgement
 2.3 TCP/IP Packages
 ‘test TLS Version 1.3’ in 4.8 SSL Service Evaluation
 ‘These Are v12.1 X86 Results’ in 11. Server Performance
 ‘These results are indicative only!’ in 11. Server Performance
 9.7.18 Throttle
 ‘TLS and SSL’ in 4. Transport Layer Security
 ‘TLS functionality is not supplied with the basic WASD package’ in 4. Transport Layer Security
 ‘TLS Version 1.3’ in 4.8 SSL Service Evaluation
 ‘TLS Version 1.3’ in 4.5.2 TLS/SSL Versions
 4.2 TLS/SSL Functionality Sources
 ‘TLS/SSL Options’ in 4.5.3 SSL Ciphers
 4.5.2 TLS/SSL Versions
 3.11 Token Authentication
 4. Transport Layer Security
 1.1 Troubleshooting?
 7.7 Tunneling Using Proxy
 7.7.8 Tunnelling Source
U‘Update Access Permission’ in 12. HTTPd Web Update
 ‘Usage Examples’ in 13.6 CALogs
 ‘Usage Examples’ in 13.8 HTAdmin
 ‘Usage Examples’ in 13.11 QDLogStats
 10.6 Usage Suggestions
 3.15 User Password Modification
 ‘Uses HTTP Cookies’ in 7.1.2 Proxy Affinity
 ‘Using Instance Status’ in 8.1.4 Status
 13. Utilities and Facilities
V3.7 Virtual Servers
 3.10.5 VMS Account Proxying
 8.1.1 VMS Clustering Comparison
 ‘VMS DLM Locking’ in 6.4 WebDAV Locking
 2.2 VMS Versions
W‘WARNING’ in 8.1 Server Instances
 ‘WARNING!’ in 3.10 SYSUAF-Authenticated Users
 3.10.4 WASD "Hard-Wired" Identifiers
 ‘WASD Features and Facilities’ in WASD Features and Facilities
 5.1 WASD HTTP/2
 4.3 WASD SSL Quick-Start
 ‘WASD VMS Web Services – Copyright © 1996-2024 Mark G. Daniel’ in 15. Attribution and Acknowledgement
 4.5.1 WASD_CONFIG_SERVICE
 ‘WASD_CONFIG_SERVICE’ in 7.1.1 Enabling A Proxy Service
 10.5 WASD_WATCH_ONE_SHOT
 13.14 WAStee Utility
 10. WATCH Facility
 6. WebDAV
 6.2 WebDAV Configuration
 6.4 WebDAV Locking
 6.3 WebDAV Metadata
 6.2.1 WebDAV Set Rules
 9.7.19 WebSocket
 13.4 Where Facility
 ‘Why use …’ in 6.2.6 Real-World Example
 13.15 WOTSUP Utility
 ‘Write Access Only’ in 6.4 WebDAV Locking
X4.5.18 X.509 Authorization CGI Variables
 4.5.13 X.509 Certificate Renegotiation
 4.5.16 X509 Configuration
 13.5 Xray Facility
Y‘YMMV!’ in 5.2 HTTP/2 and Performance

15.Attribution and Acknowledgement

WASD VMS Web Services – Copyright © 1996-2024 Mark G. Daniel
Licensed under the Apache License, Version 2.0

You may not use this software except in compliance with the License. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
None of the following licensing appears incompatible with the Apache License
Clark Cooper, et.al.

This package uses the Expat XML parsing toolkit.

Copyright (c) 1998, 1999, 2000 Thai Open Source Software Center Ltd and Clark Cooper Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006 Expat maintainers. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
Bjöern Höehrmann

This package uses essential algorithm and code from Flexible and Economical UTF-8 Decoder.

Copyright (c) 2008-2009 Bjöern Höehrmann (<bjoern@hoehrmann.de>) Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
Free Software Foundation

This package contains software made available by the Free Software Foundation under the GNU General Public License.

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2, or (at your option) any later version.
Ohio State University

This package contains software provided with the OSU (DECthreads) HTTP server package, authored by David Jones:

Copyright 1994,1997 The Ohio State University. The Ohio State University will not assert copyright with respect to reproduction, distribution, performance and/or modification of this program by any person or entity that ensures that all copies made, controlled or distributed by or for him or it bear appropriate acknowlegement of the developers of this program.
OpenSSL Project

This product can include software developed by the OpenSSL Project for use in the OpenSSL Toolkit (https://www.openssl.org/).

Redistribution and use in source and binary forms, with or without modification, are permitted ...
Paul E. Jones

This package uses SHA-1 hash code.

Copyright (C) 1998, 2009 Paul E. Jones <paulej@packetizer.com> Freeware Public License (FPL) This software is licensed as "freeware." Permission to distribute this software in source and binary forms, including incorporation into other products, is hereby granted without a fee.
RSA Data Security

This software contains code derived in part from RSA Data Security, Inc:

permission granted to make and use derivative works provided that such works are identified as "derived from the RSA Data Security, Inc. MD5 Message-Digest Algorithm" in all material mentioning or referencing the derived work.
Stuart Langridge

SortTable version 2
Stuart Langridge, http://www.kryogenix.org/code/browser/sorttable/

Thanks to many, many people for contributions and suggestions. Licenced as X11: http://www.kryogenix.org/code/browser/licence.html This basically means: do what you want with it.
Tatsuhiro Tsujikawa

nghttp2 - HTTP/2 C Library
Tatsuhiro Tsujikawa, https://github.com/tatsuhiro-t

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

VSI OpenVMS, VSI TCP/IP Services for OpenVMS, VSI C
are registered trademarks of VMS Software Inc.

OpenVMS, HP TCP/IP Services for OpenVMS, HP C, Alpha, Itanium and VAX
are registered trademarks of Hewlett Packard Enterprise

MultiNet and TCPware are registered trademarks of Process Software Corporation

WASD Features and Facilities