/*****************************************************************************/
/*
                                Cache.c

This module implements both a file data and revision time cache, and a general
non-file (e.g. script) output cache.  With file content caching the file
revision time is stored (along with the directory and file IDs for efficiency)
allowing the entry validity to expire and to be periodically revalidated
against the on-disk version.  If the on-disk version has changed the cache is
purged and the content reloaded, otherwise the expiry period just recommences. 
For non-file content the revalidation is not possible so when the validity of
the cache entry expires the entry is purged from the cache forcing it to be
reloaded through full request processing. 

The cache has limits imposed on it.  A maximum number of files that can be
cached at any one time, and a maximum amount of memory that can be allocated
in total by the cache.  Each of these two operates to limit the other.  In
addition the maximum size of a file that can be loaded into the cache can be
specified.

Fine control on cache behaviour is exercised using mapping rules.
See the MAPURL.C module for a list of those applicable.


FILE CONTENT
------------
File content cache data is loaded by the file module while concurrently
transfering that data to the request client, using buffer space supplied by
cache module, space that can then be retained for reuse as cache.  Hence the
cache load adds no significant overhead to the actual reading and initial
transfer of the file.  An entirely accurate estimate of the quantity of cache
memory required for file content can be derived from the file header.  Cache
loads that exceed the configuration or rule mapping maxima do not proceed. 


NON-FILE CONTENT
----------------
Non-file content is a little different from file content in that the size of
the cache memory required is generally not known when the cache load begins
(content-length is often not supplied with script output for example).  When
this is not known the cache module just allocates an ambit quantity of memory,
determined by configuration or rule mapping indicated maxima.  When the load
successfully completes the actual memory required is compared to that
originally allocated and reduced as necessary.  If non-file output exceeds the
original ambit memory allocation (and hence configuration or rule maxima) the
data is just discarded at end-of-request.

Non-file content entries are sourced in two ways.  First, from the CGI module. 
This can discriminate and load either (or both) CGI-compliant responses and
Non-Parse-Header (NPH) responses.  The former allows the server to generate the
response header, the latter contains the response header as the leading output
from the script.  Cached CGI responses generate a new header with each
subsequent use, while cached NPH response reuse the original header.  The
second source of cached response is direct from network-to-client output via
the NET module.  Like scripts, network loaded content can contain only the body
of the response, or the full header and body (viz. NPH).  Some care should be
excercised with the caching of responses using the NET source.

By default only GET requests, without query strings, generating responses with
a success HTTP status (200) will be cached.  Other response status content will
be discarded.  Requests containing query strings may be cached by using the
'cache=query' mapping rule.  This sort of caching should be done carefully and
selectively.  The multitude of differing query string usually accepted by such
resources would soon fill the cache with generally non-repeated data and thus
render the cache completely ineffective.


BYTE RANGES
-----------
See comments in FILE.C module.


TERMINOLOGY
-----------
"hit" refers to a request path being found in cache. If the data is still
valid the request can be supplied from cache.

"load"ing the cache refers to reading the contents of a file into cache
memory.

"valid" means that the file from which the cached data was originally read has
not had it's revision date changed (the implication being is the file contents
have not changed.


WHY IMPLEMENT CACHING?
----------------------
Caching, be definition, attempts to improve performance by keeping data in
storage that is faster to access than it's usual location.  The performance
improvement can be assessed in three basic ways.  Reduction in latency when
accessing the data, of processing involved, and in impact on the usual storage
location.

This cache is provided to address all three.  Where networks are particularly
responsive a reduction in request latency can often be noticable.  Where
servers are particularly busy or where disk systems particularly loaded a
reduction in the need to access the file system can significantly improve
performance.  My suggestion is though, that for most VMS sites high levels of
hits are not a great concern, and for these caching can easily be left
disabled.
            
CACHE SUITABILITY CONSIDERATIONS
--------------------------------
A cache is not always of benefit!  It's cost *may* outweigh it's return.

Any cache's efficiencies can only occur where subsets of data are consistently
being demanded. Although these subsets may change slowly over time a consistent
and rapidly changing aggregate of requests lose the benefit of more readily
accessable data to the overhead of cache management due to the constant and
continuous flushing and reloading of cache data. This server's cache is no
different, it will only improve performance if the site experiences some
consistency in the files requested. For sites that have only a small
percentage of files being repeatedly requested it is probably better that the
cache be disabled. The other major consideration is available system memory.
On a system where memory demand is high there is little value in having cache
memory sitting in page space, trading disk I/O and latency for paging I/O and
latency. On memory-challenged systems cache is probably best disabled.

With "loads not hit", the count represents the cumulative number of files
loaded but never subsequently hit. If this percentage is high it means most
files loaded are never hit, indicating the site's request profile is possibly
unsuitable for caching.

The item "hits" respresents the cumulative, total number of hits against the
cumulative, total number of loads. The percentage here can range from zero to
many thousands of percent :^) with less than 100% indicating poor cache
performance, from 200% upwards better and good performance. The items "1-9",
"10-99" and "100+" show the count and percentage of total hits that occured
when a given entry had experienced hits within that range (e.g. if an entry
has had 8 previous hits, the ninth increments the "1-9" item whereas the tenth
and eleventh increments the "10-99" item, etc.)

Other considerations also apply when assessing the benefit of having a cache.
For example, a high number and percentage of hits can be generated while the
percentage of "loads not hit" could be in the also be very high. The
explanation for this would be one or two frequently requested files being hit
while most others are loaded, never hit, and flushed as other files request
cache space. In situations such as this it is difficult to judge whether cache
processing is improving performance or just adding overhead.

Again, my suggestion is, that for most VMS sites, high levels of access are not
a great concern, and for these caching can easily be left disabled.
            

DESCRIPTION
-----------
An MD5 digest (16 byte hash) is used to uniquely identify cached files.  The
hash is generated either from a mapped path or from the file name before
calling any of the cache search, load or completion functions.  This MD5 hash
is then stored along with the file details and contents so that identical
paths/files can be matched during subsequent searches.  The MD5 algorithm
"guarantees" a unique identifier for any resource name, and the 16 byte size
makes matching using just 4 longword comparisons very efficient.

If using a path it MUST be the mapped path, not the client-supplied, request
path.  With conditional mapping, identical request paths may be mapped to
completely different virtual paths.

Space for a file's data is dynamically allocated and reallocated if necessary
as cache entries are reused.  It is allocated in user-specifiable chunks.  It
is expected this mechanism provides some efficiencies when reusing cache
entries.

A simple hash table is used to try and initially hit an entry.  A collision
list allows rapid subsequent searching.  The hash table is a fixed 4096 entries
with the hash value generated by directly using a fixed three bytes of the
MD5 hash for a range from 0..4095.

Cache entries are also maintained in a global linked list with the most recent
and most frequently hit entries towards the head of the list. The linked-list
organisation allows a simple implementation of a least-recently-used (LRU)
algorithm for selecting an entry when a new request demands an entry and space
for cache loading. The linked list is naturally ordered from most recently and
most frequently accessed at the head, to the least recently and least
frequently accessed at the tail. Hence an infrequently accessed entry is
selected from the tail end of the list, it's data invalidated and given to the
new request for cache load. Invalidated data cache entries are also
immediately placed at the tail of the list for reuse/reloading.

When a new entry is initially loaded it is placed at the top of the list. Hits
on other entries result in a check being made against the number of hits of
head entry in the list. If the entry being hit has a higher hit count it is
placed at the head of the list, pushing the previously head entry "down". If
not then it is again checked against the entry immediately before it in the
list. If higher then the two are swapped. This results in the most recently
loaded entries and the more frequently hit being nearest and migrating towards
the start of the search.

To help prevent the cache thrashing with floods of requests for not currently
loaded files, any entry that has a suitably high number of hits over the recent
past (suitably high ... how many is that, and recent past ... how long is
that?) are not reused until no hits have occured within that period.  Hopefully
this prevents lots of unnecessary loads of one-offs at the expense of genuinely
frequently accessed files.

To prevent multiple loads of the same path/file, for instance if a subsequent
request demands the same file as a previous request is still currently loading,
any subsequent request will merely transfer the file, not concurrently load it
into the cache.


CACHE CONTENT VALIDATION
------------------------
The cache will automatically revalidate the data after a specified number of
seconds.  With file content this is done by comparing the original file
revision time to the current revision time.  If different the file contents
have changed and the cache contents declared invalid.  If found invalid the
file transfer then continues outside of the cache with the new contents being
concurrently reloaded into the cache. With non-file content the entry is just
purged and the full request processing is used to reload the content. Cache
validation is also always performed if the request uses "Pragma: no-cache"
(i.e. as with the Netscape Navigator reload function).  Hence there is no need
for any explicit flushing of the cache under normal operation.  If a document
does not immediately reflect and changes made to it (i.e. validation time has
not been reached) validation (and consequent reload) can be "forced" with a
browser reload.  There is a discretional "guard" period that can be set which
prevent forced reloading of cached data within the specified number of seconds
since last loaded or revalidated.  This period can be used to prevent a cache
entry or entries from constantly being reloaded through pragma directives
(Mozilla for instance has a user-option cache setting which causes the request
header always to contain a 'no-cache' indication causing reload).  The entire
cache may be purged of cached data either from the server administration menu
or using command line server control.


PERMANENT ENTRIES
-----------------
Permanent entries are indicated by a path mapping a SET rule.  Permanent
entries are designed to allow certain classes of (mainly file) entry, those
that are relatively static and/or frequently being used by requests, that once
loaded remain in the cache, never validated, never able to be reclaimed or
otherwise removed from the cache during routine server activity.  These
entries, along with the volatile ones, can of course be purged from the cache
either via the CLI /DO=CACHE=PURGE command or using Server Administration menu. 
Permanent entries do not use any of the data memory ([CacheTotalKBytesMax]) set
aside for the volatile cache.  They use memory from their own specific VM pool. 
Unlike for volatile entries there is no upper limit (apart from system and
server process virtual memory) on the memory that can be allocated for
permanent entries (be careful!).  Entry slots ([CacheEntriesMax]) are shared
between permanent and volatile entries (and can be a constraint on both).


GZIP COMPRESSED CONTENT
-----------------------
If ZLIB is being used by the server (see GZIP.C) the cache will generate a
buffer of GZIP compressed content from the cached binary content.  This is then
is returned to clients that can accept GZIP encoding saving the processing
expense of generating the GZIP compression dynamically with each request.  When
the cache load is complete the content-type, etc., is tested for suitability
for GZIP compression.  It is performed as part of the cache post-processing
rather than just collecting the original request's GZIP compressed network
stream (for instance) for three basic reasons.  First; the point at which the
deflate is finalized is different to when the cache load is finalized.  Second;
not all initial cache load requests may accept GZIP encoding.  This approach
ensures GZIP content is available for any subsequent access.  Third; it's
slightly more expensive in terms or processing (the possible double GZIPing on
the initial cache load) but was fairly straight-forward and self-contained when
introducing to the code.


VERSION HISTORY
---------------
23-JAN-2020  MGD  bugfix; CacheLoadEnd() free rqCache.ContentPtr on fail
                    (seemed to be induced only by HTTP/2 requests - hmmmm)
05-FEB-2019  MGD  CacheLoadBegin() CacheLoadEnd() accept 203 as 200
01-JAN-2018  MGD  refactor some code paths (sigh! *so* tightly coupled with
                     FILE.C and REQUEST.C modules)
14-JAN-2016  MGD  CacheLoadResponse() in line with HTTP/2 changes
                  bugfix; CacheLoadResponse() initialise storage
04-MAR-2011  MGD  CacheSearch() implement request cache control
01-MAR-2011  MGD  CacheLoadResponse() checks response header for
                  "Cache-Control:" directives and adjusts accordingly
27-FEB-2011  MGD  CacheLoadEnd() buffer all content-type data (per JPP)
                  (previous behaviour truncated at ';' or white-space)
05-OCT-2009  MGD  use OutputFileBufferSize to maximise transfer
19-AUG-2009  MGD  bugfix; CacheAcpInfoAst() byte-range limit negative offset
09-JUN-2007  MGD  use STR_DSC
28-FEB-2007  MGD  bugfix; CacheNext() don't adjust GZIP content for CGI header
04-JUL-2006  MGD  use PercentOf32() for more accurate percentages
11-AUG-2005  MGD  CacheEntryReclaimCount report item
26-MAR-2005  MGD  provide caching and supply of GZIP deflated content
28-JUL-2004  MGD  support use of entity tag in file responses
15-MAY-2004  MGD  bugfix; content pointer needs to be NULLed
                  before first call to CacheNext()
24-APR-2004  MGD  extend cache hit accounting to 100-999 and 1000 plus
09-JUL-2003  MGD  rework for non-file caching requirements,
                  support byte-range requests on cached *files*
16-JUN-2003  MGD  bugfix; FileSetCharset() moved from FILE.C module
24-MAY-2003  MGD  permanent cache entries,
                  path specified maximum file size
11-MAR-2002  MGD  bugfix; ensure only one request revalidates a cache entry at
                  a time (multiple could cause eventual channel exhaustion)
22-NOV-2001  MGD  ensure there are reasonable cache minima
29-SEP-2001  MGD  instance support
04-AUG-2001  MGD  use MD5 hash to identify cache entries,
                  modify hash table as fixed 4096 entrie,
                  support module WATCHing
27-MAY-2000  MGD  BUGFIX; CacheEntryNotValid() linked list :^{
                  bugfix; CacheLoadBegin() #else before memcpy()
09-APR-2000  MGD  simplified cache search
04-MAR-2000  MGD  use FaolToNet(), et.al.
28-DEC-1999  MGD  support ODS-2 and ODS-5 using ODS module,
                  add a number of other WATCH points
26-SEP-1999  MGD  minor changes in line with RequestExecute(),
                  CacheReport() now only optionally reports cached files,
                  scavenge failure should not result in a sanity check exit
20-JAN-1999  MGD  report format refinenment
19-SEP-1998  MGD  improve granularity of cache operation,
                  add check for existing entry to CacheLoadBegin()
14-MAY-1998  MGD  request-specified content-type ("httpd=content&type=")
18-MAR-1998  MGD  use file's VBN and first free byte to check size changes
                  (allows variable record files to be cached more efficiently)
24-FEB-1998  MGD  if CacheAcpInfo() reports a problem
                  then let the file module handle/report it,
                  add file size check to entry validation (allow for extend)
10-JAN-1998  MGD  added a (much overdue) hash collision list,
                  fixed problem with cache purge (it mostly didn't :^)
22-NOV-1997  MGD  sigh, bugfix; need to proactively free memory at capacity
05-OCT-1997  MGD  initial development for v4.5
*/
/*****************************************************************************/

#ifdef WASD_VMS_V7
#undef _VMS__V6__SOURCE
#define _VMS__V6__SOURCE
#undef __VMS_VER
#define __VMS_VER 70000000
#undef __CRTL_VER
#define __CRTL_VER 70000000
#endif

/* standard C header files */
#include <ctype.h>
#include <errno.h>
#include <stdio.h>
#include <string.h>

/* VMS related header files */
#include <atrdef.h>
#include <descrip.h>
#include <fibdef.h>
#include <iodef.h>
#include <libdef.h>
#include <ssdef.h>
#include <stsdef.h>

/* application header files */
#include "wasd.h"
#include "gzip.h"
#include "md5.h"

#define WASD_MODULE "CACHE"

/******************/
/* global storage */
/******************/

BOOL  CacheEnabled,
      CacheHashTableInitialised;

/* this is run-time storage */
int  CacheChunkInBytes,
     CacheCurrentlyLoading,
     CacheCurrentlyLoadingInUse,
     CacheEntriesMax,
     CacheEntryCount,
     CacheEntryKBytesMax,
     CacheFrequentHits,
     CacheFrequentSeconds,
     CacheGuardSeconds,
     CacheHashTableMask,
     CacheMemoryInUse,
     CachePermEntryCount,
     CachePermMemoryInUse,
     CacheTotalKBytesMax,
     CacheValidateSeconds;

/* these counters may be zeroed when accounting is zeroed */
int  CacheGzipDeflateCount,
     CacheHashTableCollsnCount,
     CacheHashTableCount,
     CacheHashTableHitCount,
     CacheHashTableMissCount,
     CacheHashTableCollsnHitCount,
     CacheHashTableCollsnMissCount,
     CacheHitCount,
     CacheHits0,
     CacheHits10,
     CacheHits100,
     CacheHits1000,
     CacheHits1000plus,
     CacheListHitCount,
     CacheLoadCount,
     CacheNotHitCount,
     CacheNoHitsCount,
     CacheReclaimCount,
     CacheSearchCount;

uint64  CacheGzipDeflateBytesIn64,
        CacheGzipDeflateBytesOut64;

LIST_HEAD  CacheList;

/* the hash table index is the first byte of the MD5 hash */
#define CACHE_HASH_TABLE_ENTRIES 4096 /* 12 bits, 16384 bytes, 32 page(lets) */
struct GenericCacheEntry  *CacheHashTable [CACHE_HASH_TABLE_ENTRIES];

/********************/
/* external storage */
/********************/

extern BOOL  CliCacheEnabled,
             CliCacheDisabled,
             GzipResponse;

extern int  HttpdTickSecond,
            OdsExtended,
            OutputFileBufferSize;

extern short  HttpdTime7[];

extern char  ErrorSanityCheck[],
             HttpProtocol[],
             SoftwareID[];

extern ACCOUNTING_STRUCT  *AccountingPtr;
extern CONFIG_STRUCT  Config;
extern MSG_STRUCT  Msgs;
extern HTTPD_PROCESS  HttpdProcess;
extern WATCH_STRUCT  Watch;

/*****************************************************************************/
/*
Initialize cache run-time parameters at startup (even though caching might be
initially disabled).  Also allocate the hash table when appropriate.
*/ 

void CacheInit (BOOL Startup)

{
   int  HashValue;

   /*********/
   /* begin */
   /*********/

   if (WATCH_MODULE(WATCH_MOD_CACHE))
      WatchThis (WATCHALL, WATCH_MOD_CACHE, "CacheInit()");

   if (Startup)
   {
      if (!CliCacheDisabled && (CliCacheEnabled || Config.cfCache.Enabled))
         CacheEnabled = true;
      else
         CacheEnabled = false;

      CacheChunkInBytes = CACHE_CHUNK_SIZE;
      CacheEntriesMax = Config.cfCache.EntriesMax;
      if (CacheEntriesMax < 256) CacheEntriesMax = 256;
      CacheEntryKBytesMax = Config.cfCache.FileKBytesMax;
      if (CacheEntryKBytesMax <= 0) CacheEntryKBytesMax = 65;
      CacheFrequentHits = Config.cfCache.FrequentHits;
      CacheFrequentSeconds = Config.cfCache.FrequentSeconds;
      CacheGuardSeconds = Config.cfCache.GuardSeconds;
      if (!CacheGuardSeconds) CacheGuardSeconds = 15;
      CacheTotalKBytesMax = Config.cfCache.TotalKBytesMax;
      if (CacheTotalKBytesMax < 1024) CacheTotalKBytesMax = 1024;
      CacheValidateSeconds = Config.cfCache.ValidateSeconds;
      if (!CacheValidateSeconds) CacheValidateSeconds = 300;

      CacheEntryCount = CacheMemoryInUse =
         CachePermEntryCount = CachePermMemoryInUse = 0;

      CacheZeroCounters ();
   }

   if (CacheEntriesMax && CacheEnabled)
   {
      /* 100% overhead for ambit memory allocations, fragmentation, etc. */
      VmCacheInit (CacheTotalKBytesMax * 2);

      /* the permanent cache space will be created on the first perm entry */
      /** VmPermCacheInit (CacheTotalKBytesMax); **/

      for (HashValue = 0; HashValue < CACHE_HASH_TABLE_ENTRIES; HashValue++)
         CacheHashTable[HashValue] = NULL;

      CacheHashTableInitialised = true;
   }
}

/*****************************************************************************/
/*
Just zero the cache-associated counters :^)
*/ 

void CacheZeroCounters ()

{
   /*********/
   /* begin */
   /*********/

   if (WATCH_MODULE(WATCH_MOD_CACHE))
      WatchThis (WATCHALL, WATCH_MOD_CACHE, "CacheZeroCounter()");

   CacheGzipDeflateCount =
      CacheGzipDeflateBytesIn64 = CacheGzipDeflateBytesOut64 =
      CacheHitCount = CacheHashTableHitCount =
      CacheHashTableCollsnCount = CacheHashTableCollsnHitCount =
      CacheHashTableCollsnMissCount = CacheHashTableCount =
      CacheHashTableMissCount =
      CacheNoHitsCount = CacheHits0 = CacheHits10 =
      CacheHits100 = CacheHits1000 =
      CacheHits1000plus = CacheLoadCount = 
      CacheReclaimCount = CacheSearchCount = 0;
}

/*****************************************************************************/
/*
Look through the cache list for a resource hash (MD5) that matches.  If one is
found then call CacheBegin() and return success, otherwise return error status
to indicate the search was unsuccessful.

NOTE: as with CacheLoadBegin(), the path passed to this function MUST be the
mapped path, not the request path. With conditional mapping identical request
paths may be mapped to completely different virtual paths.
*/ 

int CacheSearch (REQUEST_STRUCT *rqptr)

{
   int  status,
        EntryCount,
        HashValue;
   char  *cptr, *sptr;
   FILE_CENTRY  *captr;
   LIST_ENTRY  *leptr;
   MD5_HASH  *md5ptr;

   /*********/
   /* begin */
   /*********/

   if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE,
                 "CacheSearch() file:!&B", rqptr->FileTaskPtr);

   if (!CacheEnabled) return (SS$_NOSUCHFILE);

   if (!CacheHashTableInitialised) CacheInit (false);

   /* if the request specified that cache not be used */
   if (rqptr->rqHeader.CacheControlNoCache ||
       rqptr->rqHeader.CacheControlNoStore ||
       rqptr->rqHeader.PragmaNoCache)
   {
      if (WATCHING (rqptr, WATCH_RESPONSE))
         WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "CACHE request no-cache");
      return (SS$_NOSUCHFILE); 
   }

   /* if not interested in anything with a query string */
   if (!rqptr->rqPathSet.CacheQuery &&
       rqptr->rqHeader.QueryStringLength)
   {
      if (WATCHING (rqptr, WATCH_RESPONSE))
         WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "CACHE request query-string");
      return (SS$_NOSUCHFILE); 
   }

   /* any request containing a cookie that is not from a static file */
   if (rqptr->rqHeader.CookiePtr &&
       !rqptr->FileTaskPtr &&
       !rqptr->rqPathSet.CacheCookie)
   {
      if (WATCHING (rqptr, WATCH_RESPONSE))
         WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "CACHE request cookie");
      return (SS$_NOSUCHFILE); 
   }

   /* if not a GET request */
   if (rqptr->rqHeader.Method != HTTP_METHOD_GET) return (SS$_NOSUCHFILE);

   /* if a SET mapping rule has specified the path should not be cached */
   if (rqptr->rqPathSet.NoCache) return (SS$_NOSUCHFILE);

   /* server has specifically set this request not to be cached */
   if (rqptr->rqCache.DoNotCache) return (SS$_NOSUCHFILE); 

   /* must be using this for a cache load (e.g. during directory listing) */
   if (rqptr->rqCache.EntryPtr) return (SS$_NOSUCHFILE); 

   if (rqptr->FileTaskPtr)
      md5ptr = &rqptr->FileTaskPtr->Md5Hash;
   else
      md5ptr = &rqptr->Md5HashPath;

   if (WATCHING (rqptr, WATCH_RESPONSE))
      WatchThis (WATCHITM(rqptr), WATCH_RESPONSE,
                 "CACHE search !&?file\rpath\r !16&H",
                 rqptr->FileTaskPtr, md5ptr);

   /* check the hash table */
   CacheHashTableCount++;
   /* 12 bit hash value, 0..4095 from a fixed 3 bytes of the MD5 hash */
   HashValue = md5ptr->HashLong[0] & 0xfff;
   if (!(captr = CacheHashTable[HashValue]))
   {
      /*************/
      /* not found */
      /*************/

      if (rqptr->rqHeader.CacheControlOnlyIfCached)
      {
         rqptr->rqResponse.HttpStatus = 504;
         ErrorVmsStatus (rqptr, SS$_TIMEOUT, FI_LI);
         return (SS$_TIMEOUT);
      }

      /* nope, no pointer against that hash value */
      CacheHashTableMissCount++;
      return (SS$_NOSUCHFILE);
   }

   EntryCount = 0;
   for ( /*set above*/ ; captr; captr = captr->HashCollisionNextPtr)
   {
      if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
         WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE,
"file:!&B valid:!&B revalidating:!&B !AZ !16&H !&?match\rno-match\r",
                    captr->FromFile,
                    captr->EntryValid, captr->EntryRevalidating,
                    captr->FileOds.ExpFileName,
                    &captr->Md5Hash,
                    MATCH16 (&captr->Md5Hash, md5ptr));

      /* note the first collision entry */
      if (++EntryCount == 2) CacheHashTableCollsnCount++;

      if (!MATCH16 (&captr->Md5Hash, md5ptr)) continue;

      /********/
      /* hit! */
      /********/

      /* first entry is always the hash table, after that the collision list */
      if (EntryCount == 1)
         CacheHashTableHitCount++;
      else
         CacheHashTableCollsnHitCount++;

      if (!captr->EntryValid ||
          captr->DataLoading ||
          captr->EntryRevalidating)
      {
         /**************/
         /* not usable */
         /**************/

         rqptr->rqCache.NotUsable = true;
         return (SS$_NOSUCHFILE);
      }

      /**********/
      /* usable */
      /**********/

      if (rqptr->rqHeader.CacheControlMaxAge &&
          HttpdTickSecond - captr->LoadSeconds >
          rqptr->rqHeader.CacheControlMaxAge)
      {
         /***********/
         /* too old */
         /***********/

         if (WATCHING (rqptr, WATCH_RESPONSE))
            WatchThis (WATCHITM(rqptr), WATCH_RESPONSE,
                       "CACHE request max-age !UL",
                       rqptr->rqHeader.CacheControlMaxAge);

         return (SS$_NOSUCHFILE);
      }

      if (!ResponseEntityMatch (rqptr, captr->EntityTag))
      {
         /********************/
         /* failed condition */
         /********************/

         /* ResponseEntityMatch() should generate its own HTTP response */
         return (SS$_ABORT);
      }


      /************/
      /* letsdoit */
      /************/

      rqptr->rqCache.EntryPtr = captr;
      status = CacheBegin (rqptr);
      if (VMSnok (status)) rqptr->rqCache.EntryPtr = NULL;
      return (status);
   }

   /***********/
   /* not hit */
   /***********/

   CacheHashTableCollsnMissCount++;
   return (SS$_NOSUCHFILE);
}

/*****************************************************************************/
/*
Use this cached entry as the contents for the client.  If the entry needs to
be validated then generate an asynchronous ACP QIO to get the required file
details, other wise call the AST processing function directly.

NOTE: as with CacheLoadBegin(), the path passed to this function MUST be the
mapped path, not the request path.  With conditional mapping identical request
paths may be mapped to completely different virtual paths.
*/ 
 
int CacheBegin (REQUEST_STRUCT *rqptr)

{
   BOOL  RevalidateEntry;
   FILE_CENTRY  *captr;

   /*********/
   /* begin */
   /*********/

   if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE, "CacheBegin() !AZ",
                 rqptr->rqCache.EntryPtr->FileOds.ExpFileName);

   captr = rqptr->rqCache.EntryPtr;

   if (captr->EntryPermanent)
      RevalidateEntry = false;
   else
   if (rqptr->NotFromCache &&
       captr->GuardTickSecond < HttpdTickSecond)
      RevalidateEntry = true;
   else
   if (captr->ExpiresAfterPeriod == CACHE_EXPIRES_DAY)
      if (captr->ExpiresAfterTime != HttpdTime7[2])
         RevalidateEntry = true;
      else
         RevalidateEntry = false;
   else
   if (captr->ExpiresAfterPeriod == CACHE_EXPIRES_HOUR)
      if (captr->ExpiresAfterTime != HttpdTime7[3])
         RevalidateEntry = true;
      else
         RevalidateEntry = false;
   else
   if (captr->ExpiresAfterPeriod == CACHE_EXPIRES_MINUTE)
      if (captr->ExpiresAfterTime != HttpdTime7[4])
         RevalidateEntry = true;
      else
         RevalidateEntry = false;
   else
   if (captr->ExpiresTickSecond < HttpdTickSecond)
      RevalidateEntry = true;
   else
      RevalidateEntry = false;

   if (!RevalidateEntry)
   {
      /* status of non-zero used to determine whether the ACPQIO was used */
      captr->FileOds.FileQio.IOsb.Status = 0;
      CacheAcpInfoAst (rqptr);
      return (SS$_NORMAL);
   }

   if (captr->FromFile)
   {
      if (WATCHING (rqptr, WATCH_RESPONSE))
         WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "CACHE revalidate !AZ",
                    captr->FileOds.ExpFileName);

      captr->EntryRevalidating = true;
      captr->InUseCount++;
      captr->ValidatedCount++;
      OdsFileAcpInfo (&captr->FileOds, &CacheAcpInfoAst, rqptr); 
      return (SS$_NORMAL);
   }

   /* non-file entries that are invalid are purged and re-cached */
   if (WATCHING (rqptr, WATCH_RESPONSE))
      WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "CACHE stale !AZ",
                 captr->FileOds.ExpFileName);

   CacheRemoveEntry (captr, false);
   /* move it to the end of the cache list ready for reuse */
   ListMoveTail (&CacheList, captr);

   return (SS$_NOSUCHFILE);
}

/*****************************************************************************/
/*
Called either explicitly or as an AST from an ACP QIO in CacheBegin(). Check if
modified, if not then just reply with a 302 header.  If contents should be
transfered to the client then begin. If there is a problem at all then declare
the AST to continue processing the request and let it worry about it, otherwise
begin providing the file data from the cache entry.
*/ 
 
void CacheAcpInfoAst (REQUEST_STRUCT *rqptr)

{
   BOOL  RangeValid;
   int  idx, status,
        ContentLength;
   unsigned short  Length;
   char  *cptr, *sptr, *zptr;
   RANGE_BYTE  *rbptr;
   REQUEST_AST AstFunction;
   FILE_CENTRY  *captr, *tcaptr;
   FILE_CONTENT  *fcptr;
   FILE_QIO  *fqptr;
   LIST_ENTRY  *leptr;

   /*********/
   /* begin */
   /*********/

   if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE,
                 "CacheAcpInfoAst() !&F !&S !%D !%D", &CacheAcpInfoAst,
                 rqptr->rqCache.EntryPtr->FileOds.FileQio.IOsb.Status,
                 &rqptr->rqCache.EntryPtr->RdtTime64,
                 &rqptr->rqCache.EntryPtr->FileOds.FileQio.RdtTime64);

   captr = rqptr->rqCache.EntryPtr;
   fqptr = &captr->FileOds.FileQio;

   /* proactively remove the association between the entry and the request */
   rqptr->rqCache.EntryPtr = NULL;

   /* status of non-zero used to determine whether the ACPQIO was used */
   if (fqptr->IOsb.Status)
   {
      /*********************/
      /* revalidating file */
      /*********************/

      /* finished getting the revalidate data */
      captr->EntryRevalidating = false;
      captr->InUseCount--;

      /* first deassign the channel allocated by OdsFileAcpInfo() */
      sys$dassgn (fqptr->AcpChannel);

      if (VMSnok (fqptr->IOsb.Status))
      {
         /***********************/
         /* file access problem */
         /***********************/

         /* entry no longer valid */
         CacheRemoveEntry (captr, false);
         /* move it to the end of the cache list ready for reuse */
         ListMoveTail (&CacheList, captr);

         CacheEnd (rqptr);
         return;
      }

      if (rqptr->rqResponse.HttpVersion == HTTP_VERSION_1_1)
         if (captr->EntityTag[0])
            strzcpy (rqptr->rqResponse.EntityTag, captr->EntityTag,
                     sizeof(rqptr->rqResponse.EntityTag));

      fqptr->EndOfFileVbn = ((fqptr->RecAttr.fat$l_efblk & 0xffff) << 16) |
                            ((fqptr->RecAttr.fat$l_efblk & 0xffff0000) >> 16);
      fqptr->FirstFreeByte = fqptr->RecAttr.fat$w_ffbyte;

      if (captr->RdtTime64 != fqptr->RdtTime64 ||
          captr->EndOfFileVbn != fqptr->EndOfFileVbn ||
          captr->FirstFreeByte != fqptr->FirstFreeByte)
      {
         /************************/
         /* data no longer valid */
         /************************/

         if (WATCHING (rqptr, WATCH_RESPONSE))
            WatchThis (WATCHITM(rqptr), WATCH_RESPONSE,
                       "CACHE stale !AZ !%D (!%D / !%D) (!UL,!UL / !UL,!UL)",
                       captr->FileOds.ExpFileName, &captr->CdtTime64,
                       &captr->RdtTime64, &fqptr->RdtTime64,
                       captr->EndOfFileVbn, captr->FirstFreeByte,
                       fqptr->EndOfFileVbn, fqptr->FirstFreeByte);

         /* entry no longer valid */
         CacheRemoveEntry (captr, false);
         /* move it to the end of the cache list ready for reuse */
         ListMoveTail (&CacheList, captr);

         CacheEnd (rqptr);
         return;
      }

      /********************/
      /* data still valid */ 
      /********************/

      /* note the time the cached data was last validated */
      captr->ValidateTime64 = rqptr->rqTime.BeginTime64;

      if (captr->ExpiresAfterPeriod == CACHE_EXPIRES_DAY)
         captr->ExpiresAfterTime = HttpdTime7[2];
      else
      if (captr->ExpiresAfterPeriod == CACHE_EXPIRES_HOUR)
         captr->ExpiresAfterTime = HttpdTime7[3];
      else
      if (captr->ExpiresAfterPeriod == CACHE_EXPIRES_MINUTE)
         captr->ExpiresAfterTime = HttpdTime7[4];
      else
      if (captr->ExpiresAfterPeriod == CACHE_EXPIRES_NONE)
         captr->ExpiresTickSecond = 0xffffffff;
      else
      if (captr->ExpiresAfterPeriod)
         captr->ExpiresTickSecond = HttpdTickSecond + captr->ExpiresAfterPeriod;
      else
         captr->ExpiresTickSecond = HttpdTickSecond + CacheValidateSeconds;

      if (captr->GuardSeconds)
         captr->GuardTickSecond = HttpdTickSecond + captr->GuardSeconds;
      else
         captr->GuardTickSecond = HttpdTickSecond + CacheGuardSeconds;
   }

   if (WATCHING (rqptr, WATCH_RESPONSE))
      WatchThis (WATCHITM(rqptr), WATCH_RESPONSE,
                 "CACHE hit !&?permanent\rvolatile\r !AZ",
                 captr->EntryPermanent, captr->FileOds.ExpFileName);

   /**************************************/
   /* request to be fulfilled from cache */
   /**************************************/

   /* cancel any no-such-file callback, we've obviously got it! */
   if (rqptr->FileTaskPtr) rqptr->FileTaskPtr->NoSuchFileFunction = NULL;

   captr->HitTime64 = rqptr->rqTime.BeginTime64;
   captr->FrequentTickSecond = HttpdTickSecond + CacheFrequentSeconds;

   CacheHitCount++;
   if (!captr->HitCount) CacheHits0--;
   captr->HitCount++;
   if (captr->HitCount < 10)
      CacheHits10++;
   else
   if (captr->HitCount < 100)
      CacheHits100++;
   else
   if (captr->HitCount < 1000)
      CacheHits1000++;
   else
      CacheHits1000plus++;

   rqptr->AccountingDone =
      InstanceGblSecIncrLong (&AccountingPtr->CacheHitCount);

   /* if this entry has more hits move it to the head of the cache list */
   leptr = CacheList.HeadPtr;
   tcaptr = (FILE_CENTRY*)leptr;
   if (captr->HitCount > tcaptr->HitCount)
      ListMoveHead (&CacheList, captr);
   else
   {
      /* if this entry has more hits than the one "above" it swap them */
      leptr = captr;
      if (leptr->PrevPtr)
      {
         leptr = leptr->PrevPtr;
         tcaptr = (FILE_CENTRY*)leptr;
         if (captr->HitCount > tcaptr->HitCount)
         {
            ListRemove (&CacheList, captr);
            ListAddBefore (&CacheList, leptr, captr, LIST_ENTRY_TYPE_CACHE);
         }
      }
   }

   if (captr->ContentHandlerFunction)
   {
      /***********************************/
      /* this file has a content handler */
      /***********************************/

      if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
         WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE, "!&X",
                    captr->ContentHandlerFunction);

      /* reassociated the cache entry with this request */
      rqptr->rqCache.EntryPtr = captr;

      /* allow one extra character for a(n implicit) terminating null */
      rqptr->FileContentPtr = fcptr = (FILE_CONTENT*)
         VmGetHeap (rqptr, sizeof(FILE_CONTENT) + captr->ContentLength+1);
      fcptr->ContentSize = fcptr->ContentSizeMax = captr->ContentLength+1;

      /* buffer space immediately follows the structured storage */
      fcptr->ContentPtr = (char*)fcptr + sizeof(FILE_CONTENT);
      memcpy (fcptr->ContentPtr,
              captr->ContentPtr,
              fcptr->ContentLength = captr->ContentLength);

      /* populate the file contents structure with some file data */
      zptr = (sptr = fcptr->FileName) + sizeof(fcptr->FileName);
      for (cptr = captr->FileOds.ExpFileName;
           *cptr && sptr < zptr;
           *sptr++ = *cptr++);
      if (sptr >= zptr)
      {
         ErrorGeneralOverflow (rqptr, FI_LI);
         CacheEnd (rqptr);
         return;
      }
      *sptr = '\0';
      fcptr->FileNameLength = sptr - fcptr->FileName;

      fcptr->CdtTime64 = captr->CdtTime64;
      fcptr->RdtTime64 = captr->RdtTime64;

      fcptr->UicGroup = captr->UicGroup;
      fcptr->UicMember = captr->UicMember;
      fcptr->Protection = captr->Protection;

      fcptr->ContentHandlerFunction = captr->ContentHandlerFunction;
      CacheEnd (rqptr);
      return;
   }

   if (!rqptr->rqResponse.HeaderGenerated)
   {
      /**************************/
      /* full response required */
      /**************************/

      if (captr->FromFile)
      {
         /********/
         /* file */
         /********/

         if (rqptr->rqHeader.RangeBytePtr &&
             rqptr->rqHeader.RangeBytePtr->Total)
         {
            /**************/
            /* byte-range */
            /**************/

            RangeValid = true;
            rbptr = rqptr->rqHeader.RangeBytePtr;
            for (idx = 0; idx < rbptr->Total; idx++)
            {
               if (!rbptr->Last[idx])
               {
                  /* last byte not specified, set at EOF */
                  rbptr->Last[idx] = captr->ContentLength - 1;
               }
               else
               if (rbptr->Last[idx] < 0)
               {
                  /* first byte a negative offset from end, last byte at EOF */
                  rbptr->First[idx] = captr->ContentLength + rbptr->Last[idx];
                  if (rbptr->First[idx] < 0) rbptr->First[idx] = 0;
                  rbptr->Last[idx] = captr->ContentLength - 1;
               }
               else
               if (rbptr->Last[idx] >= captr->ContentLength)
               {
                  /* if the last byte is ambit make it at the EOF */
                  rbptr->Last[idx] = captr->ContentLength - 1;
               }
               /* if the range still does not make sense then back out now */
               if (rbptr->Last[idx] < rbptr->First[idx])
               {
                  RangeValid = false;
                  rbptr->Length = 0;
               }
               else
                  rbptr->Length = rbptr->Last[idx] - rbptr->First[idx] + 1;

               if (WATCHING (rqptr, WATCH_RESPONSE))
                  WatchThis (WATCHITM(rqptr), WATCH_RESPONSE,
                             "RANGE !UL !UL-!UL !UL byte!%s!&? INVALID\r\r",
                             idx+1, rbptr->First[idx], rbptr->Last[idx],
                             rbptr->Length, !rbptr->Length);
            }
            if (RangeValid) rbptr->Count = idx;
         }

         if (rqptr->rqPathSet.CharsetPtr)
         {
            cptr = captr->FileOds.ResFileName;
            if (!*cptr) cptr = captr->FileOds.ExpFileName;
            rqptr->rqPathSet.CharsetPtr = FileSetCharset (rqptr, cptr);
         }

         if (rqptr->rqResponse.HttpVersion == HTTP_VERSION_1_1)
            if (captr->EntityTag[0])
               strzcpy (rqptr->rqResponse.EntityTag, captr->EntityTag,
                        sizeof(rqptr->rqResponse.EntityTag));

         /* in this case /status/ is used as a boolean */
         status = FileResponseHeader (rqptr,
                                      captr->ContentType,
                                      captr->ContentLength,
                                      &captr->RdtTime64);
         if (!status)
         {
            captr->HitNotModifiedCount++;
            CacheEnd (rqptr);
            return;
         }
      }
      else
      {
         /************/
         /* non-file */
         /************/

         if (captr->ContentType[0])
         {
            /* if any retained CGI header fields add these to the header */
            if (captr->CgiHeaderLength)
            {
               cptr = captr->ContentPtr;
               ContentLength = captr->ContentLength -
                               captr->CgiHeaderLength - 1;
            }
            else
            {
               cptr = NULL;
               ContentLength = captr->ContentLength;
            }
            ResponseHeader (rqptr, 200, captr->ContentType,
                            ContentLength, &captr->RdtTime64, cptr);
         }
         else
         {
            /* when no associated content-type it's an NPH script */
            rqptr->rqResponse.HttpStatus = 200;
         }
      }

      /* quit here if the HTTP method was HEAD */
      if (rqptr->rqHeader.Method == HTTP_METHOD_HEAD)
      {
         CacheEnd (rqptr);
         return;
      }
   }

   /************/
   /* transfer */
   /************/

   /* 'CacheInUse' keeps track of whether the entry is in use or not */
   captr->InUseCount++;

   /* initialize this for start-of-transfer detection in CacheNext() */
   rqptr->rqCache.ContentPtr = NULL;

   /* reassociated the cache entry with this request */
   rqptr->rqCache.EntryPtr = captr;

   /* network writes are checked for success, fudge the first one! */
   rqptr->NetIoPtr->WriteStatus = SS$_NORMAL;

   /* begin the transfer */
   if (STR_DSC_LEN(&rqptr->NetWriteBufferDsc))
   {
      /* after ensuring the current contents are output */
      NetWriteFullFlush (rqptr, &CacheNext);
   }
   else
      CacheNext (rqptr);
}

/*****************************************************************************/
/*
Write the next (or first) block of data from the cache buffer to the client.
*/ 

void CacheNext (REQUEST_STRUCT *rqptr)

{
   int  status,
        DataLength;
   unsigned short  Length;
   unsigned char  *DataPtr;
   FILE_CENTRY  *captr;
   RANGE_BYTE  *rbptr;
   REQUEST_AST AstFunction;

   /*********/
   /* begin */
   /*********/

   if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE,
                 "CacheNext() !&F !UL !&S", &CacheNext,
                 rqptr->rqCache.EntryPtr->InUseCount,
                 rqptr->NetIoPtr->WriteStatus);

   if (rqptr->RequestState >= REQUEST_STATE_ABORT)
   {
      CacheEnd (rqptr);
      return;
   }

   if (VMSnok (rqptr->NetIoPtr->WriteStatus))
   {
      CacheEnd (rqptr);
      return;
   }

   captr = rqptr->rqCache.EntryPtr;

   if (!rqptr->rqCache.ContentPtr)
   {
      /* first read, initialize content pointer and length */
      if (rqptr->rqHeader.RangeBytePtr &&
          rqptr->rqHeader.RangeBytePtr->Count)
      {
         /* returning a byte range within the file (partial content) */
         rbptr = rqptr->rqHeader.RangeBytePtr;
         rbptr->Length = rbptr->Last[rbptr->Index] -
                         rbptr->First[rbptr->Index] + 1;
         if (rbptr->Count > 1)
         {
            /* returning 'multipart/byteranges' range content */
            char Buffer [256];
            FaoToBuffer (Buffer, sizeof(Buffer), &Length,
"!AZ--!AZ\r\n\
Content-Type: !AZ\r\n\
Range: bytes !UL-!UL/!UL\r\n\
\r\n",
                         rbptr->Index ? "\r\n" : "",
                         rqptr->rqResponse.MultipartBoundaryPtr,
                         captr->ContentType,
                         rbptr->First[rbptr->Index],
                         rbptr->Last[rbptr->Index],
                         captr->ContentLength);
            /* synchronous network write (just for the convenience of it!) */
            NetWrite (rqptr, NULL, Buffer, Length);
         }
         rqptr->rqCache.ContentPtr = captr->ContentPtr +
                                     rbptr->First[rbptr->Index];
         rqptr->rqCache.ContentLength = rbptr->Length;
      }
      else
      if (captr->GzipContentPtr &&
          rqptr->rqResponse.ContentEncodeAsGzip &&
          /* if the header has been sent this is something like a readme! */
          !rqptr->rqResponse.HeaderSent)
      {
         /* GZIP content is available and we're going to provide one */
         rqptr->rqCache.ContentPtr = captr->GzipContentPtr;
         rqptr->rqCache.ContentLength = captr->GzipContentLength;
         rqptr->rqResponse.ContentEncodeAsGzip = false;
         rqptr->rqResponse.ContentIsEncodedGzip = true;

         if (WATCHING (rqptr, WATCH_RESPONSE))
            WatchThis (WATCHITM(rqptr), WATCH_RESPONSE,
                       "CACHE (gzip deflated) !UL->!UL bytes, !UL%",
                       captr->ContentLength, captr->GzipContentLength,
                       PercentOf32(captr->GzipContentLength,
                                   captr->ContentLength));
      }
      else
      {
         /* use the unencoded content */
         rqptr->rqCache.ContentPtr = captr->ContentPtr;
         rqptr->rqCache.ContentLength = captr->ContentLength;

         if (captr->CgiHeaderLength)
         {
            /* adjust body of response for any retained CGI header fields */
            rqptr->rqCache.ContentPtr += captr->CgiHeaderLength + 1;
            rqptr->rqCache.ContentLength -= captr->CgiHeaderLength + 1;
         }
      }
   }

   if (rqptr->rqCache.ContentLength > OutputFileBufferSize)
   {
      DataPtr = rqptr->rqCache.ContentPtr;
      DataLength = OutputFileBufferSize;
      rqptr->rqCache.ContentPtr += DataLength;
      rqptr->rqCache.ContentLength -= DataLength;

      if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
      {
         WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE, "CACHE !UL/!UL",
                    DataLength, rqptr->rqCache.ContentLength);
         WatchDataDump (DataPtr, DataLength);
      }

      NetWrite (rqptr, &CacheNext, DataPtr, DataLength);
      return;
   }
   else
   {
      DataPtr = rqptr->rqCache.ContentPtr;
      DataLength = rqptr->rqCache.ContentLength;
      rqptr->rqCache.ContentLength = 0;

      if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
      {
         WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE, "CACHE !UL/!UL",
                    DataLength, rqptr->rqCache.ContentLength);
         WatchDataDump (DataPtr, DataLength);
      }

      NetWrite (rqptr, &CacheEnd, DataPtr, DataLength);
      return;
   }
}

/*****************************************************************************/
/*
End of transfer to client using cached contents.
*/

void CacheEnd (REQUEST_STRUCT *rqptr)

{
   char  *cptr, *sptr, *zptr;
   FILE_CENTRY  *captr;

   /*********/
   /* begin */
   /*********/

   if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE,
                 "CacheEnd() !&F", &CacheEnd);

   if (!(captr = rqptr->rqCache.EntryPtr))
   {
      /* entry is no longer valid */
      if (rqptr->FileTaskPtr)
         FileParse (rqptr);
      else
         RequestExecutePostCache1 (rqptr);
      return;
   }

   if (rqptr->rqHeader.RangeBytePtr &&
       rqptr->rqHeader.RangeBytePtr->Count)
   {
      /* transfering byte-range(s) */
      rqptr->rqHeader.RangeBytePtr->Index++;
      if (rqptr->rqHeader.RangeBytePtr->Index <
          rqptr->rqHeader.RangeBytePtr->Count)
      {
         /* multiple byte ranges, restart with next range */
         rqptr->rqCache.ContentPtr = NULL;
         rqptr->rqCache.ContentLength = 0;
         SysDclAst (&CacheNext, rqptr);
         return;
      }
      if (rqptr->rqHeader.RangeBytePtr->Count > 1)
      {
         /* end of multiple byte ranges, provide final boundary */
         char Buffer [64];
         zptr = (sptr = Buffer) + sizeof(Buffer)-1;
         for (cptr = "\r\n--"; *cptr && sptr < zptr; *sptr++ = *cptr++);
         for (cptr = rqptr->rqResponse.MultipartBoundaryPtr;
              *cptr && sptr < zptr;
              *sptr++ = *cptr++);
         for (cptr = "--\r\n"; *cptr && sptr < zptr; *sptr++ = *cptr++);
         *sptr = '\0';
         /* synchronous network write (for the convenience of it!) */
         NetWrite (rqptr, NULL, Buffer, sptr-Buffer);
      }
   }

   /* this cache entry is no longer associated with this request */
   rqptr->rqCache.EntryPtr = NULL;

   /* this cache entry is no longer in use (if now zero) */
   if (captr->InUseCount) captr->InUseCount--;

   if (captr->Purge && !captr->InUseCount)
   {
      if (captr->PurgeCompletely)
         CacheRemoveEntry (captr, true);
      else
      {
         CacheRemoveEntry (captr, false);
         /* move it to the end of the cache list ready for reuse */
         ListMoveTail (&CacheList, captr);
      }
   }

   rqptr->BytesTxGzipPercent = PercentOf32 (captr->GzipContentLength,
                                            captr->ContentLength);

   if (rqptr->FileTaskPtr)
      FileEnd (rqptr);
   else
      RequestEnd (rqptr);
} 

/*****************************************************************************/
/*
Check that we're interested in caching this particular data and that the
requested size is allowed to be cached.  Find/create a cache structure ready to
contain the data buffered.  This also blocks other concurrent loads of the same
resource.  It is entirely possible, given that the maximum number of cache
entries has been reached and all are currently in use (either for data transfer
or being loaded) - though this if fairly unlikely, that there will be no cache
entry available for this request to use and the load will fail.  If available
allocate an appropriate chunk of either volatile or permanent cache memory and
use that to load the to-be-cached data.
*/ 

BOOL CacheLoadBegin
(
REQUEST_STRUCT *rqptr,
int SizeInBytes,
char *ContentTypePtr
)
{
   int  secs, MaxKBytes;

   /*********/
   /* begin */
   /*********/

   if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE,
                 "CacheLoadBegin() !UL bytes !&Z !&Z http:!UL in-use:!&B",
                 SizeInBytes, ContentTypePtr, rqptr->rqHeader.QueryStringPtr,
                 rqptr->rqResponse.HttpStatus, rqptr->rqCache.ContentPtr);

   rqptr->rqCache.LoadCheck = true;

   /* could be disabled between search and load beginning */
   if (!CacheEnabled) return (false);

   /* only interested in HTTP success status */
   if (rqptr->rqResponse.HttpStatus &&
       rqptr->rqResponse.HttpStatus != 200 &&
       rqptr->rqResponse.HttpStatus != 203) return (false);

   /* if this cache structure is already in use (e.g. directory listing) */
   if (rqptr->rqCache.EntryPtr) return (false);

   /* if not interested in query strings */
   if (!rqptr->rqPathSet.CacheQuery &&
       rqptr->rqHeader.QueryStringLength) return (false);

   /* if a SET mapping rule has specified the path should not be cached */
   if (rqptr->rqPathSet.NoCache) return (false);

   /* if not a GET request */
   if (rqptr->rqHeader.Method != HTTP_METHOD_GET) return (false);

   /* server has specifically set this request not to be cached */
   if (rqptr->rqCache.DoNotCache) return (false); 

   MaxKBytes = 0;
   if (rqptr->rqCgi.ScriptControlCacheMaxKBytes)
   {
      /* start off with any script specified value */
      MaxKBytes = rqptr->rqCgi.ScriptControlCacheMaxKBytes;
      if (rqptr->rqPathSet.CacheMaxKBytes)
      {
         /* any path setting should limit anything script supplied */
         if (MaxKBytes > rqptr->rqPathSet.CacheMaxKBytes)
            MaxKBytes = rqptr->rqPathSet.CacheMaxKBytes;
      }
      else
      {
         /* configuration setting should limit anything script supplied */
         if (MaxKBytes > CacheEntryKBytesMax) MaxKBytes = CacheEntryKBytesMax;
      }
   }
   else 
   if (rqptr->rqPathSet.CacheMaxKBytes)
   {
      /* path specified maximum overrides configuration maximum */
      MaxKBytes = rqptr->rqPathSet.CacheMaxKBytes;
   }
   /* fall back to using the configuration setting */
   if (!MaxKBytes) MaxKBytes = CacheEntryKBytesMax;

   /* zero indicates exact size is unknown so start with an ambit maximum */
   if (SizeInBytes <= 0) SizeInBytes = MaxKBytes << 10;

   /* if it's larger than the maximum allowed */
   if ((SizeInBytes >> 10) > MaxKBytes)
   {
      if (WATCHING (rqptr, WATCH_RESPONSE))
         WatchThis (WATCHITM(rqptr), WATCH_RESPONSE,
                    "CACHE load fail, too large (!ULkB>!ULkB)",
                    SizeInBytes >> 10, MaxKBytes);
      return (false);
   }

   if (!CacheLoadResponse (rqptr)) return (false);

   /* if we can't get one then just forget it! */
   if (!CacheAllocateEntry (rqptr)) return (false);

   /* calculate the cache chunk */
   if (SizeInBytes % CacheChunkInBytes) SizeInBytes += CacheChunkInBytes;
   SizeInBytes = SizeInBytes / CacheChunkInBytes;
   SizeInBytes *= CacheChunkInBytes;

   if (rqptr->rqPathSet.CachePermanent)
      rqptr->rqCache.ContentPtr = VmGetPermCache (SizeInBytes);
   else
      rqptr->rqCache.ContentPtr = VmGetCache (SizeInBytes);

   rqptr->rqCache.CurrentPtr = rqptr->rqCache.ContentPtr;
   rqptr->rqCache.ContentRemaining = SizeInBytes;
   rqptr->rqCache.ContentBufferSize = SizeInBytes;
   rqptr->rqCache.ContentLength = rqptr->rqCache.RecordBlockLength = 0;

   rqptr->rqCache.ContentTypePtr = ContentTypePtr;
   rqptr->rqCache.Loading = true;
   rqptr->rqCache.LoadStatus = 0;

   CacheCurrentlyLoadingInUse += SizeInBytes;
   CacheCurrentlyLoading++;

   return (true);
}

/*****************************************************************************/
/*
This function checks any cache-control in that and if it should not be cached
returns false.  It also checks for an entity-tag header and stores it locally
to the request, and for the presence of a cookie (responses with cookies or
vary headers are never cached).  The rqCache.Response.. storage conveys
information back to the cache load to determine some characterstics.  Return
true to allow caching (the default), and false to prohibit caching.  This
function is very much a collection of pragmatics.
*/

BOOL CacheLoadResponse (REQUEST_STRUCT *rqptr)

{
   BOOL  ok2cache,
         CookieHit = false,
         VaryCookie = false;
   int  len,
        CacheFor = CACHE_EXPIRES_NONE,
        VaryCount = 0;
   char  *cptr, *sptr, *zptr;
   DICT_ENTRY_STRUCT  *denptr;

   /*********/
   /* begin */
   /*********/

   if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE, "CacheLoadResponse()");

   if (denptr = DictLookup (rqptr->rqDictPtr, DICT_TYPE_RESPONSE,
                            "cache-control", 13))
   {
      cptr = DICT_GET_VALUE(denptr);

      if (WATCHING (rqptr, WATCH_RESPONSE))
         WatchThis (WATCHITM(rqptr), WATCH_RESPONSE,
                    "CACHE cache-control: !AZ", cptr);

      while (*cptr)
      {
         if (strsame (cptr, "max-age=", len=8))
            CacheFor = rqptr->rqCache.ResponseCacheControl = atoi(cptr+8);
         else
         if (strsame (cptr, "must-revalidate", len=15))
            CacheFor = 0;
         else
         if (strsame (cptr, "no-cache", len=8))
            CacheFor = 0;
         else
         if (strsame (cptr, "no-store", len=8))
            CacheFor = 0;
         else
         if (strsame (cptr, "private", len=7))
            CacheFor = 0;
         else
         if (strsame (cptr, "proxy-revalidate", len=16))
            CacheFor = 0;
         else
         if (strsame (cptr, "public", len=6))
            CacheFor = CACHE_EXPIRES_NONE;
         else
         if (strsame (cptr, "s-maxage=", len=9))
            CacheFor = rqptr->rqCache.ResponseCacheControl = atoi(cptr+9);
         else
            len = 0;

         while (*cptr && !ISLWS(*cptr) && *cptr != ',') cptr++;
         while (*cptr && (ISLWS(*cptr) || *cptr == ',')) cptr++;
      }
   }

   if (denptr = DictLookup (rqptr->rqDictPtr, DICT_TYPE_RESPONSE, "etag", 4))
   {
      cptr = DICT_GET_VALUE(denptr);

      if (WATCHING (rqptr, WATCH_RESPONSE))
         WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "CACHE etag: !AZ", cptr);

      zptr = (sptr = rqptr->rqCache.ResponseEntityTag) +
             sizeof(rqptr->rqCache.ResponseEntityTag)-1;
      if (*cptr == '\"') cptr++;
      while (*cptr && !ISLWS(*cptr) && *cptr != '\"' && sptr < zptr)
         *sptr++ = *cptr++;
      if (sptr >= zptr)
      {
         ErrorNoticed (rqptr, SS$_RESULTOVF, NULL, FI_LI);
         sptr = rqptr->rqCache.ResponseEntityTag;
      }
      *sptr = '\0';
   }

   if (denptr = DictLookup (rqptr->rqDictPtr, DICT_TYPE_RESPONSE, "pragma", 6))
   {
      cptr = DICT_GET_VALUE(denptr);

      if (WATCHING (rqptr, WATCH_RESPONSE))
         WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "CACHE pragma: !AZ", cptr);

      while (*cptr && ISLWS(*cptr)) cptr++;
      if (strsame (cptr, "no-cache", 8)) CacheFor = 0;
   }

   if (denptr = DictLookup (rqptr->rqDictPtr, DICT_TYPE_RESPONSE,
                            "set-cookie", 10))
   {
      if (WATCHING (rqptr, WATCH_RESPONSE))
         WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "CACHE set-cookie:");

      CookieHit = true;
   }

   if (denptr = DictLookup (rqptr->rqDictPtr, DICT_TYPE_RESPONSE, "vary", 4))
   {
      cptr = DICT_GET_VALUE(denptr);

      if (WATCHING (rqptr, WATCH_RESPONSE))
         WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "CACHE vary: !AZ", cptr);

      while (*cptr && ISLWS(*cptr)); cptr++;

      /* server generated response based on specific request characteristics */
      VaryCount = 1;
      VaryCookie = false;
      while (*cptr)
      {
         if (*cptr == ',')
            VaryCount++;
         else
         if (to_upper(*cptr) == 'C' && strsame (cptr, "Cookie", 6))
            VaryCookie = true;
         cptr++;
      }
      /* if the only variation is cookie and not one in this response */
      if (VaryCount == 1 && VaryCookie && !CookieHit) VaryCount = 0;
   }

   /* return true if (all other things being equal) it's OK to cache */
   ok2cache = !CookieHit && !VaryCount && CacheFor != 0;

   if (WATCHING (rqptr, WATCH_RESPONSE))
   {
      if (!ok2cache)
         WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "CACHE no");
      else
      if (CacheFor != CACHE_EXPIRES_NONE)
         WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "CACHE for !UL", CacheFor);
   }

   return (ok2cache);
} 

/*****************************************************************************/
/*
Check the success of the load.  This is indicated using an end-of-file status. 
If the load failed or finding a cache entry failed just discard the loaded data
and return the memory to the cache pool.  If the content-type, etc., lends
itself to being GZIP encoded then go ahead and process the cached binary
content into a buffer of GZIP compressed content.
*/ 

void CacheLoadEnd (REQUEST_STRUCT *rqptr)

{
   int  status,
        ReclaimEntryBytes,
        ReclaimEntryCount,
        SizeInBytes;
   char  *cptr, *sptr, *zptr;
   FILE_CENTRY  *captr, *lcaptr;
   FILE_TASK  *ftkptr;
   LIST_ENTRY  *leptr;

   /*********/
   /* begin */
   /*********/

   if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
      if (rqptr->FileTaskPtr)
         WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE,
                    "CacheLoadEnd() file:!&B !UL !&S",
                    rqptr->FileTaskPtr, rqptr->rqResponse.HttpStatus,
                    rqptr->rqCache.LoadStatus);
      else
         WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE,
                    "CacheLoadEnd() file:FALSE !UL !&S",
                    rqptr->rqResponse.HttpStatus, rqptr->rqCache.LoadStatus);

   /* better check, caching could be enabled between ASTs since search */
   if (!CacheHashTableInitialised) CacheInit (false);

   if (!rqptr->rqCache.Loading)
      ErrorExitVmsStatus (SS$_BUGCHECK, ErrorSanityCheck, FI_LI);

   captr = rqptr->rqCache.EntryPtr;
   /* entry is no longer associated with this request */
   rqptr->rqCache.EntryPtr = NULL;

   captr->InUseCount--;

   if (captr->FromFile = rqptr->rqCache.LoadFromFile) cptr = "FILE";
   if (captr->FromNet = rqptr->rqCache.LoadFromNet) cptr = "NET";
   if (captr->FromScript = rqptr->rqCache.LoadFromCgi) cptr = "CGI";

   rqptr->rqCache.Loading =
      rqptr->rqCache.LoadFromCgi =
      rqptr->rqCache.LoadFromFile =
      rqptr->rqCache.LoadFromNet = false;

   CacheCurrentlyLoading--;
   CacheCurrentlyLoadingInUse -= rqptr->rqCache.ContentBufferSize;

   /* not interested in anything but successful responses */
   if ((rqptr->rqResponse.HttpStatus &&
        rqptr->rqResponse.HttpStatus != 200 &&
        rqptr->rqResponse.HttpStatus != 203) ||
       /* an early HTTP success status but an error during processing */
       rqptr->rqResponse.ErrorReportPtr)
      rqptr->rqCache.LoadStatus = SS$_CANCEL;

   /* end-of-file is used to indicate successful cache data load */
   if (rqptr->rqCache.LoadStatus != RMS$_EOF &&
       rqptr->rqCache.LoadStatus != SS$_ENDOFFILE)
   {
      /*********************/
      /* load unsuccessful */
      /*********************/

      if (WATCHING (rqptr, WATCH_RESPONSE))
         WatchThis (WATCHITM(rqptr), WATCH_RESPONSE,
                    "CACHE load from !AZ fail, data %!&M",
                    cptr, rqptr->rqCache.LoadStatus);

      CacheRemoveEntry (captr, false);
      /* move it to the end of the cache list ready for reuse */
      ListMoveTail (&CacheList, captr);

      if (rqptr->rqCache.ContentPtr)
      {
         if (rqptr->rqPathSet.CachePermanent)
            VmFreePermCache (rqptr->rqCache.ContentPtr, FI_LI);
         else
            VmFreeCache (rqptr->rqCache.ContentPtr, FI_LI);
         rqptr->rqCache.ContentPtr = NULL;
      }
      return;
   }

   if (WATCHING (rqptr, WATCH_RESPONSE))
      WatchThis (WATCHITM(rqptr), WATCH_RESPONSE,
                 "CACHE load from !AZ complete, !UL bytes", 
                 cptr, rqptr->rqCache.ContentLength);

   /**********************/
   /* check memory usage */
   /**********************/

   /* calculate the required cache chunk */
   SizeInBytes = rqptr->rqCache.ContentLength;
   if (SizeInBytes % CacheChunkInBytes) SizeInBytes += CacheChunkInBytes;
   SizeInBytes = SizeInBytes / CacheChunkInBytes;
   SizeInBytes *= CacheChunkInBytes;
   if (!SizeInBytes) SizeInBytes = CacheChunkInBytes;

   if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE, "!UL < !UL !&B",
                 SizeInBytes, rqptr->rqCache.ContentBufferSize,
                 SizeInBytes < rqptr->rqCache.ContentBufferSize);

   if (SizeInBytes < rqptr->rqCache.ContentBufferSize)
   {
      /* actually required cache chunk is smaller than original ambit chunk */
      cptr = rqptr->rqCache.ContentPtr;
      if (rqptr->rqPathSet.CachePermanent)
      {
         sptr = VmGetPermCache (SizeInBytes);
         memcpy (sptr, cptr, rqptr->rqCache.ContentLength);
         VmFreePermCache (cptr, FI_LI);
      }
      else
      {
         sptr = VmGetCache (SizeInBytes);
         memcpy (sptr, cptr, rqptr->rqCache.ContentLength);
         VmFreeCache (cptr, FI_LI);
      }
      rqptr->rqCache.ContentPtr = sptr;
      rqptr->rqCache.ContentBufferSize = SizeInBytes;
   }

   /************************/
   /* populate cache entry */
   /************************/

   InstanceGblSecIncrLong (&AccountingPtr->CacheLoadCount);
   CacheLoadCount++;
   CacheHits0++;

   /* move entry to the head of the cache list */
   ListMoveHead (&CacheList, captr);

   if (rqptr->rqPathSet.CachePermanent)
   {
      /* permanent cache entry */
      captr->EntryPermanent = true;
      CachePermEntryCount++;
      CacheEntryCount--;
      CachePermMemoryInUse += rqptr->rqCache.ContentBufferSize;
   }
   else
      CacheMemoryInUse += rqptr->rqCache.ContentBufferSize;

   captr->ContentPtr = rqptr->rqCache.ContentPtr;
   captr->EntrySize = rqptr->rqCache.ContentBufferSize;
   captr->ContentLength = rqptr->rqCache.ContentLength;
   rqptr->rqCache.ContentPtr = NULL;

   captr->HitCount = 0;
   captr->DataLoading = false;
   captr->EntryValid = true;

   /* quadword time file was loaded/validated, created, last modified */
   captr->LoadTime64 = rqptr->rqTime.BeginTime64;
   captr->ValidateTime64 = rqptr->rqTime.BeginTime64;

   /* some file details (if applicable) */
   if (ftkptr = rqptr->FileTaskPtr)
   {
      captr->EndOfFileVbn = ftkptr->FileOds.FileQio.EndOfFileVbn;
      captr->FirstFreeByte = ftkptr->FileOds.FileQio.FirstFreeByte;
      captr->UicGroup = (ftkptr->FileOds.FileQio.AtrUic & 0x0fff0000) >> 16;
      captr->UicMember = (ftkptr->FileOds.FileQio.AtrUic & 0x0000ffff);
      captr->Protection = ftkptr->FileOds.FileQio.AtrFpro;
      captr->CdtTime64 = ftkptr->FileOds.FileQio.CdtTime64;
      captr->RdtTime64 = ftkptr->FileOds.FileQio.RdtTime64;
      strzcpy (captr->EntityTag, ftkptr->EntityTag, sizeof(captr->EntityTag));
      /* copy the entire on-disk structure from file to cache */
      OdsCopyStructure (&captr->FileOds, &ftkptr->FileOds);
      /* the cache entry will reuse the original content handler (if any) */
      captr->ContentHandlerFunction = ftkptr->ContentHandlerFunction;
      /* drop through to buffer the content-type */
      cptr = ftkptr->ContentTypePtr;
   }
   else
   {
      captr->FirstFreeByte = captr->EndOfFileVbn =
         captr->UicGroup = captr->UicMember = captr->Protection = 0;
      memset (&captr->FileOds, 0, sizeof(captr->FileOds));
      captr->CdtTime64 = rqptr->rqTime.BeginTime64;
      captr->RdtTime64 = rqptr->rqTime.BeginTime64;
      strzcpy (captr->EntityTag, rqptr->rqCache.ResponseEntityTag,
               sizeof(captr->EntityTag));
      captr->ContentHandlerFunction = NULL;
      /* add the path purely for cache report purposes */
      zptr = (sptr = captr->FileOds.ExpFileName) +
             sizeof(captr->FileOds.ExpFileName)-1;
      for (cptr = rqptr->ServicePtr->ServerHostPort;
           *cptr && sptr < zptr;
           *sptr++ = *cptr++);
      for (cptr = rqptr->rqHeader.RequestUriPtr;
           *cptr && *cptr != '?' && sptr < zptr;
           *sptr++ = *cptr++);
      if (rqptr->rqPathSet.CacheQuery)
      {
         /* those cached regardless include any request query string */
         while (*cptr && sptr < zptr) *sptr++ = *cptr++;
      }
      *sptr = '\0';
      /* drop through to buffer the content-type */
      cptr = rqptr->rqCache.ContentTypePtr;
   }

   /* note if the cached content contains any CGI header fields */
   if (captr->FromScript && rqptr->rqCgi.HeaderLength)
      captr->CgiHeaderLength = rqptr->rqCgi.HeaderLength;

   /* buffer the content type */
   if (!cptr) cptr = "";
   zptr = (sptr = captr->ContentType) + sizeof(captr->ContentType)-1;
   while (*cptr && sptr < zptr) *sptr++ = *cptr++;
   if (sptr >= zptr) ErrorNoticed (rqptr, SS$_RESULTOVF, NULL, FI_LI);
   *sptr = '\0';

   if (GzipResponse)
   {
      /*************************************/
      /* generated GZIP compressed content */
      /*************************************/

      if (captr->ContentLength)
         if (GzipShouldDeflate (rqptr, captr->ContentType,
                                       captr->ContentLength))
            GzipDeflateCache (rqptr, captr);
   }

   /********************/
   /* set entry expiry */
   /********************/

   /* for max-age, etc. */
   captr->LoadSeconds = HttpdTickSecond;

   captr->ExpiresAfterPeriod = rqptr->rqPathSet.CacheExpiresAfter;

   if (captr->ExpiresAfterPeriod == CACHE_EXPIRES_DAY)
      captr->ExpiresAfterTime = HttpdTime7[2];
   else
   if (captr->ExpiresAfterPeriod == CACHE_EXPIRES_HOUR)
      captr->ExpiresAfterTime = HttpdTime7[3];
   else
   if (captr->ExpiresAfterPeriod == CACHE_EXPIRES_MINUTE)
      captr->ExpiresAfterTime = HttpdTime7[4];
   else
   if (captr->ExpiresAfterPeriod == CACHE_EXPIRES_NONE)
      captr->ExpiresTickSecond = 0xffffffff;
   else
   if (captr->ExpiresAfterPeriod)
      captr->ExpiresTickSecond = HttpdTickSecond + captr->ExpiresAfterPeriod;
   else
      captr->ExpiresTickSecond = HttpdTickSecond + CacheValidateSeconds;

   captr->GuardSeconds = rqptr->rqPathSet.CacheGuardSeconds;
   if (captr->GuardSeconds)
      captr->GuardTickSecond = HttpdTickSecond + captr->GuardSeconds;
   else
      captr->GuardTickSecond = HttpdTickSecond + CacheGuardSeconds;

   /* if we're using too much cache memory */
   if ((CacheMemoryInUse >> 10) > CacheTotalKBytesMax)
   {
      /******************/
      /* reclaim memory */
      /******************/

      CacheReclaimCount++;
      ReclaimEntryCount = ReclaimEntryBytes = 0;

      /* mark the current one just so *it* won't be reclaimed */
      captr->InUseCount++;

      /* process the cache entry list from least to most recent */
      for (leptr = CacheList.TailPtr; leptr; leptr = leptr->PrevPtr)
      {
         /* remember, use a separate pointer or we get very confused :^) */
         lcaptr = (FILE_CENTRY*)leptr;

         if (WATCHPNT(rqptr) &&
             WATCH_MODULE(WATCH_MOD_CACHE) &&
             WATCH_MODULE(WATCH_MOD__DETAIL))
            WatchThis (WATCHALL, WATCH_MOD_CACHE,
                       "!&Z !UL !UL !&B !&B !UL",
                       captr->FileOds.ExpFileName,
                       captr->EntrySize, captr->ContentLength,
                       captr->EntryValid, captr->EntryRevalidating,
                       captr->InUseCount);

         /* if it's permanent or in use in some way then just continue */
         if (lcaptr->EntryPermanent || lcaptr->InUseCount) continue;

         ReclaimEntryCount++;
         ReclaimEntryBytes += lcaptr->EntrySize;

         /* entry no longer valid */
         CacheRemoveEntry (lcaptr, false);
         lcaptr->EntryReclaimed = true;
 
         /* if we've reclaimed enough memory */
         if ((CacheMemoryInUse >> 10) <= CacheTotalKBytesMax) break;
      }

      /* remove the reclaim prophylactic */
      captr->InUseCount--;

      if (WATCHING (rqptr, WATCH_RESPONSE))
         WatchThis (WATCHITM(rqptr), WATCH_RESPONSE,
                    "CACHE load, reclaim !UL entries, !UL kBytes",
                    ReclaimEntryCount, ReclaimEntryBytes >> 10);
   }
}

/*****************************************************************************/
/*
Copy the referenced data into the cache pre-allocated buffer (if there's still
space available).
*/ 

int CacheLoadData
(
REQUEST_STRUCT *rqptr,
char *DataPtr,
int DataLength
)
{
   /*********/
   /* begin */
   /*********/

   if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE,
"CacheLoadData() file:!&B cgi:!&B net:!&B !UL+!UL=!UL !UL-!UL=!UL",
                 rqptr->rqCache.LoadFromFile,
                 rqptr->rqCache.LoadFromCgi,
                 rqptr->rqCache.LoadFromNet,
                 rqptr->rqCache.ContentLength, DataLength,
                 rqptr->rqCache.ContentLength + DataLength,
                 rqptr->rqCache.ContentBufferSize,
                 rqptr->rqCache.ContentLength + DataLength,
                 rqptr->rqCache.ContentBufferSize -
                    rqptr->rqCache.ContentLength - DataLength);

   if (DataLength <= rqptr->rqCache.ContentRemaining)
   {
      memcpy (rqptr->rqCache.CurrentPtr, DataPtr, DataLength);
      rqptr->rqCache.CurrentPtr += DataLength;
      rqptr->rqCache.ContentLength += DataLength;
      rqptr->rqCache.ContentRemaining -= DataLength;
      return (SS$_NORMAL);
   }

   return (rqptr->rqCache.LoadStatus = SS$_BUFFEROVF_ERROR);
}

/*****************************************************************************/
/*
Allocate a cache entry ready for data loading.  Returns true if entry
available, false if not.  Sets 'rqCache.CacheEntry' to point to the allocated
entry.
*/ 

BOOL CacheAllocateEntry (REQUEST_STRUCT *rqptr)

{
   int  HashValue;
   FILE_CENTRY  *captr, *lcaptr;
   LIST_ENTRY  *leptr;
   MD5_HASH  *md5ptr;

   /*********/
   /* begin */
   /*********/

   if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE, "CacheAllocateEntry() !16&H",
                 rqptr->FileTaskPtr ? &rqptr->FileTaskPtr->Md5Hash :
                                      &rqptr->Md5HashPath);

   if (rqptr->FileTaskPtr)
      md5ptr = &rqptr->FileTaskPtr->Md5Hash;
   else
      md5ptr = &rqptr->Md5HashPath;

   /**********************************/
   /* check it doesn't already exist */
   /**********************************/

   /* 12 bit hash value, 0..4095 from a fixed 3 bytes of the MD5 hash */
   HashValue = md5ptr->HashLong[0] & 0xfff;
   for (captr = CacheHashTable[HashValue];
        captr;
        captr = captr->HashCollisionNextPtr)
   {
      if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
         WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE,
"file:!&B valid:!&B revalidating:!&B !AZ !16&H !&?match\rno-match\r",
                    captr->FromFile, captr->EntryValid,
                    captr->EntryRevalidating, captr->FileOds.ExpFileName,
                    &captr->Md5Hash,
                    MATCH16 (&captr->Md5Hash, md5ptr));

      if (!MATCH16 (&captr->Md5Hash, md5ptr)) continue;

      if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
         WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE, "entry exists");

      return (false);
   }

   /**********************/
   /* find a cache entry */
   /**********************/

   captr = (FILE_CENTRY*)CacheList.TailPtr;
   if (captr && !captr->EntryValid && !captr->InUseCount)
   {
      /****************************/
      /* reuse invalid tail entry */
      /****************************/

      if (WATCHING (rqptr, WATCH_RESPONSE))
         WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "CACHE load, reuse entry");
   }
   else
   if (CacheEntryCount + CachePermEntryCount >= CacheEntriesMax)
   {
      /***********************/
      /* reuse a cache entry */
      /***********************/

      /* process the cache entry list from least to most recent */
      for (leptr = CacheList.TailPtr; leptr; leptr = leptr->PrevPtr)
      {
         captr = (FILE_CENTRY*)leptr;

         if (WATCHPNT(rqptr) &&
             WATCH_MODULE(WATCH_MOD_CACHE) && WATCH_MODULE(WATCH_MOD__DETAIL))
            WatchThis (WATCHALL, WATCH_MOD_CACHE,
                       "!&Z !UL !UL !&B !&B !UL",
                       captr->FileOds.ExpFileName,
                       captr->EntrySize, captr->ContentLength,
                       captr->EntryValid, captr->EntryRevalidating,
                       captr->InUseCount);

         /* if it's permanent or in use in some way then just continue */
         if (captr->EntryPermanent || captr->InUseCount) continue;

         /* if it can be considered frequently hit */
         if (captr->EntryValid &&
             CacheFrequentHits &&
             captr->HitCount > CacheFrequentHits &&
             captr->FrequentTickSecond > HttpdTickSecond) continue;

         /* entry no longer valid */
         CacheRemoveEntry (captr, false);

         if (WATCHING (rqptr, WATCH_RESPONSE))
            WatchThis (WATCHITM(rqptr), WATCH_RESPONSE,
                       "CACHE load, reuse entry");
         break;
      }

      /* if we got to the end of the list */
      if (!leptr)
      {
         /**********************/
         /* all entries in use */
         /**********************/

         if (WATCHING (rqptr, WATCH_RESPONSE))
            WatchThis (WATCHITM(rqptr), WATCH_RESPONSE,
                       "CACHE load, all entries in use");

         return (false);
      }
   }
   else
   {
      /*******************/
      /* new cache entry */
      /*******************/

      captr = VmGet (sizeof(FILE_CENTRY));
      CacheEntryCount++;

      /* add it to the list */
      ListAddTail (&CacheList, captr, LIST_ENTRY_TYPE_CACHE);

      if (WATCHING (rqptr, WATCH_RESPONSE))
         WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "CACHE load, new entry");
   }

   /**************/
   /* init entry */
   /**************/

   captr->EntryReclaimed =
      captr->EntryRevalidating =
      captr->EntryValid =
      captr->FromScript =
      captr->FromNet =
      captr->Purge =
      captr->PurgeCompletely = false;
   captr->ContentPtr = NULL;
   captr->CgiHeaderLength = captr->EntrySize = 0;
   if (rqptr->FileTaskPtr)
      captr->FromFile = true;
   else
      captr->FromFile = false;
   captr->DataLoading = true;
   captr->InUseCount++;

   /*************************/
   /* add to the hash table */
   /*************************/

   memcpy (&captr->Md5Hash, md5ptr, sizeof(MD5_HASH));
   /* 12 bit hash value, 0..4095 from a fixed 3 bytes of the MD5 hash */
   HashValue = md5ptr->HashLong[0] & 0xfff;
   if (!CacheHashTable[HashValue])
   {
      /* set hash table index */
      CacheHashTable[HashValue] = captr;
      captr->HashCollisionPrevPtr = captr->HashCollisionNextPtr = NULL;
   }
   else
   {
      /* add to head of hash-collision list */
      lcaptr = CacheHashTable[HashValue];
      lcaptr->HashCollisionPrevPtr = captr;
      captr->HashCollisionPrevPtr = NULL;
      captr->HashCollisionNextPtr = lcaptr;
      CacheHashTable[HashValue] = captr;
   }

   rqptr->rqCache.EntryPtr = captr;
   return (true);
}

/*****************************************************************************/
/*
Purge a cache entry, either just the data, or completely from the cache list.
*/

void CacheRemoveEntry
(
FILE_CENTRY *captr,
BOOL Completely
)
{
   int  HashValue;

   /*********/
   /* begin */
   /*********/

   if (WATCH_MODULE(WATCH_MOD_CACHE))
      WatchThis (WATCHALL, WATCH_MOD_CACHE,
                 "CacheRemoveEntry() !&B", Completely);

   if (captr->InUseCount)
   {
      if (Completely)
         captr->Purge = captr->PurgeCompletely = true;
      else
         captr->Purge = true;
      return;
   }

   /* note the number of entries loaded but never subsequently hit */
   if (captr->EntryValid && !captr->HitCount) CacheNoHitsCount++;

   if (captr->EntryPermanent)
   {
      /* permanent entry reverts to volatile when purged */
      captr->EntryPermanent = false;
      if (captr->ContentPtr)
      {
         VmFreePermCache (captr->ContentPtr, FI_LI);
         CachePermMemoryInUse -= captr->EntrySize;
         captr->ContentPtr = NULL;
         captr->EntrySize = 0;
      }
      if (captr->GzipContentPtr)
      {
         VmFreePermCache (captr->GzipContentPtr, FI_LI);
         CachePermMemoryInUse -= captr->GzipEntrySize;
         captr->GzipContentPtr = NULL;
         captr->GzipContentLength = captr->GzipEntrySize = 0;
      }
      CacheEntryCount++;
      CachePermEntryCount--;
   }
   else
   {
      if (captr->ContentPtr)
      {
         VmFreeCache (captr->ContentPtr, FI_LI);
         CacheMemoryInUse -= captr->EntrySize;
         captr->ContentPtr = NULL;
         captr->EntrySize = 0;
      }
      if (captr->GzipContentPtr)
      {
         VmFreeCache (captr->GzipContentPtr, FI_LI);
         CacheMemoryInUse -= captr->GzipEntrySize;
         captr->GzipContentPtr = NULL;
         captr->GzipContentLength = captr->GzipEntrySize = 0;
      }
   }

   captr->DataLoading =
      captr->EntryValid =
      captr->Purge =
      captr->PurgeCompletely = false;

   /* 12 bit hash value, 0..4095 from a fixed 3 bytes of the MD5 hash */
   HashValue = captr->Md5Hash.HashLong[0] & 0xfff;
   if ((FILE_CENTRY*)(CacheHashTable[HashValue]) == captr)
   {
      /* must be at the head of any collision list */
      CacheHashTable[HashValue] = captr->HashCollisionNextPtr;
      if (captr->HashCollisionNextPtr)
         captr->HashCollisionNextPtr->HashCollisionPrevPtr =
            captr->HashCollisionPrevPtr;
   }
   else
   {
      /* if somewhere along the collision list */
      if (captr->HashCollisionPrevPtr)
         captr->HashCollisionPrevPtr->HashCollisionNextPtr =
            captr->HashCollisionNextPtr;
      /*
         *** SITE OF ONE OF MY MOST STUPID AND COSTLY PROGRAMMING ERRORS ***
         If this isn't an argument against coding for speed and efficiency
         instead of with tested and common code routines I don't know what is!
      */
      if (captr->HashCollisionNextPtr)
         captr->HashCollisionNextPtr->HashCollisionPrevPtr =
            captr->HashCollisionPrevPtr;
   }
   captr->HashCollisionPrevPtr = captr->HashCollisionNextPtr = NULL;

   if (Completely)
   {
      ListRemove (&CacheList, captr);
      VmFree (captr, FI_LI);
   }
} 

/*****************************************************************************/
/*
Scan through the cache list.  If a cache entry is currently not in use then
free the data memory associated with it.  If purge completely then also remove
the entry from the list and free it's memory.  If the entry is currently in use
then mark it for purge and if necessary for complete removal.
*/

void CachePurge
(
BOOL Completely,
int *PurgeCountPtr,
int *MarkedForPurgeCountPtr
)
{
   int  PurgeCount,
        MarkedForPurgeCount;
   FILE_CENTRY  *captr;
   LIST_ENTRY  *leptr;

   /*********/
   /* begin */
   /*********/

   if (WATCH_MODULE(WATCH_MOD_CACHE))
      WatchThis (WATCHALL, WATCH_MOD_CACHE,
                 "CachePurge() !&B", Completely);

   CacheZeroCounters ();

   PurgeCount = MarkedForPurgeCount = 0;

   /* do it backwards! seeing they are pushed to the tail of the list */
   leptr = CacheList.TailPtr;
   while (leptr)
   {
      captr = (FILE_CENTRY*)leptr;

      /* now, before stuffing around with the entry get the next one */
      leptr = leptr->PrevPtr;

      if (WATCH_MODULE(WATCH_MOD_CACHE) && WATCH_MODULE(WATCH_MOD__DETAIL))
         WatchThis (WATCHALL, WATCH_MOD_CACHE,
                    "!&Z !UL !UL !&B !&B !UL",
                    captr->FileOds.ExpFileName,
                    captr->EntrySize, captr->ContentLength,
                    captr->EntryValid, captr->EntryRevalidating,
                    captr->InUseCount);

      if (captr->InUseCount)
      {
         if (Completely)
            captr->Purge = captr->PurgeCompletely = true;
         else
            captr->Purge = true;
         MarkedForPurgeCount++;
      }
      else
      {
         if (captr->EntrySize)
         {
            CacheRemoveEntry (captr, Completely);
            PurgeCount++;
         }
      }
   }

   if (PurgeCountPtr) *PurgeCountPtr = PurgeCount;
   if (MarkedForPurgeCountPtr)
      *MarkedForPurgeCountPtr = MarkedForPurgeCount;
} 

/*****************************************************************************/
/*
Return a report on cache usage.  This function blocks while executing.
*/ 

void CacheReport
(
REQUEST_STRUCT *rqptr,
BOOL IncludeEntries
)
{
   static char BeginPageFao [] =
"<p><table class=\"ctgry\">\n\
<tr><td>\n\
\
<table class=\"rghtlft\">\n\
<tr><th><u>Configuration</u></th></tr>\n\
<tr><td></td></tr>\n\
<tr><th>Caching:</th>\
<td style=\"text-align:left;\" colspan=\"2\">!AZ</td></tr>\n\
<tr><th>Memory&nbsp;&nbsp;/Permanent:</th>\
<td>!UL</td><td><font size=\"-1\">kB</font></td></tr>\n\
<tr><th>/Volatile:</th>\
<td>!UL</td><td><font size=\"-1\">kB</font></td></tr>\n\
<tr><th>/Max:</th>\
<td>!UL</td><td><font size=\"-1\">kB</font></td></tr>\n\
<tr><th>Entries&nbsp;&nbsp;/Permanent:</th>\
<td>!UL</td></tr>\n\
<tr><th>/Volatile:</th>\
<td>!UL</td></tr>\n\
<tr><th>/Max:</th>\
<td>!UL</td></tr>\n\
<tr><th>/Valid:</th>\
<td>!UL</td></tr>\n\
<tr><th>/In-Use:</th>\
<td>!UL</td></tr>\n\
<tr><th>/Loading:</th>\
<td>!UL</td></tr>\n\
<tr><th>/Reclaimed:</th>\
<td>!UL</td></tr>\n\
<tr><th>Max File Size:</th>\
<td>!UL</td><td><font size=\"-1\">kB</font></td></tr>\n\
<tr><th>Guard:</th>\
<td>!UL</td><td><font size=\"-1\">seconds</font></td></tr>\n\
<tr><th>Validate:</th>\
<td>!UL</td><td><font size=\"-1\">seconds</font></td></tr>\n\
<tr><th>Frequent&nbsp;&nbsp;/Hits:</th>\
<td>!UL</td><td><font size=\"-1\">hits</font></td></tr>\n\
<tr><th>/Within:</th>\
<td>!UL</td><td><font size=\"-1\">seconds</font></td></tr>\n\
</table>\n\
\
</td><td>\n\
<table class=\"rghtlft\">\n\
<tr><th><u>Activity</u></th></tr>\n\
<tr><td></td></tr>\n\
<tr><th>Search:</th><td>!UL</td></tr>\n\
<tr><th>&nbsp;<u>Hash Table</u></th></tr>\n\
<tr><th>Hit:</th>\
<td>!UL</td><td>(!UL%)</td></tr>\n\
<tr><th>Miss:</th>\
<td>!UL</td><td>(!UL%)</td></tr>\n\
<tr><th><u>Collision</u></th></tr>\n\
<tr><th>Total:</th>\
<td>!UL</td><td>(!UL%)</td></tr>\n\
<tr><th>Hit:</th>\
<td>!UL</td><td>(!UL%)</td></tr>\n\
<tr><th>Miss:</th>\
<td>!UL</td><td>(!UL%)</td></tr>\n\
<tr><th><u>Memory</u></th></tr>\n\
<tr><th>Loading:</th>\
<td>!UL</td><td>!UL&nbsp;<font size=\"-1\">MB</font></td>\
<tr><th>Reclaim:</th>\
<td>!UL</td><td>(!UL%)</td></tr>\n\
<tr><th>&nbsp;<u>GZIP&nbsp;Deflate</u></th></tr>\n\
<tr><th>Count:</th><td>!UL</td></tr>\n\
<tr><th>Bytes In:</th><td colspan=\"2\">!&,@UQ</td></tr>\n\
<tr><th>Bytes Out:</th><td colspan=\"2\">!&,@UQ</td></tr>\n\
<tr><th>Ratio:</th><td>!UL%</td></tr>\n\
</table>\n\
\
</td><td>\n\
<table class=\"rghtlft\">\n\
<tr><th><u>Entries</u></th></tr>\n\
<tr><td></td></tr>\n\
<tr><th>Load:</th><td>!UL</td></tr>\n\
<tr><th>Not Hit:</th>\
<td>!UL</td><td>(!UL%)</td><td></td></tr>\n\
<tr><th>&nbsp;&nbsp;Total Hit:</th>\
<td>!UL</td><td>(!UL%)</td></tr>\n\
<tr><th>1-9:</th>\
<td>!UL</td><td>(!UL%)</td></tr>\n\
<tr><th>10-99:</th>\
<td>!UL</td><td>(!UL%)</td></tr>\n\
<tr><th>100-999:</th>\
<td>!UL</td><td>(!UL%)</td></tr>\n\
<tr><th>&gt;1000:</th>\
<td>!UL</td><td>(!UL%)</td></tr>\n\
</table>\n\
\
</td></tr>\n\
</table>\n\
!&@";

   /* the final column just adds a little white-space on the page far right */
   static char  EntriesFao [] =
"<p><table class=\"lftlft\">\n\
<tr>\
<th></th>\
<th class=\"sbttl\">Flags</th>\
<th class=\"sbttl\">Size&nbsp;/&nbsp;Length</th>\
<th class=\"sbttl\">GZ&nbsp;Size&nbsp;/&nbsp;Len&nbsp;/&nbsp;%</th>\
<th class=\"sbttl\"><nobr>In-Use</nobr></th>\
<th class=\"sbttl\">Hash</th>\
<th class=\"sbttl\">Revised</th>\
<th class=\"sbttl\">Validated</th>\
<th class=\"sbttl\">Loaded</th>\
<th colspan=\"2\" class=\"sbttl\">Hit&nbsp;/&nbsp;304</th>\
</tr>\n";

   /* the empty 99% column just forces the rest left with long request URIs */
   static char  CacheFao [] =
"<tr>\
<td><b><a href=\"?entry=!16&H\">!#ZL</a></b>&nbsp;&nbsp;</td>\
<td style=\"text-align:left;\">\
!&?P\rV\r\
!&?F\r\r!&?N\r\r!&?S\r\r\
!&?C\r<strike>C</strike>\r\
!&?T\r<strike>T</strike>\r\
&nbsp;&nbsp;</td>\
<td>!UL&nbsp;/&nbsp;!UL</td>\
<td>!UL&nbsp;/&nbsp;!UL&nbsp;/&nbsp;!UL</td>\
<td>!UL</td>\
<td>!UL&nbsp;/&nbsp;!UL</td>\
<td>!&@</td>\
<td>!&@</td>\
<td>!20%D</nobr></td>\
<td>!20%D</nobr></td>\
<td>!UL&nbsp;/&nbsp;!UL</td>\
</tr>\n\
<tr class=\"hlght\"><td></td>\
<td class=\"talft maxvis\" colspan=\"10\"><tt>!AZ</tt></td>\
</tr>\n";

   static char EmptyCacheFao [] =
"<tr class=\"hlght\"><th>000</th>\
<td colspan=\"10\"><i>empty</i></td><tr>\n";

   static char  EntriesButtonFao [] =
"</table>\n\
<p><a class=\"abttn\" href=\"!AZ\">Entries</a>\n\
!AZ\
</div>\n\
</body>\n\
</html>\n";

   static char  EndPageFao [] =
"</table>\n\
!AZ\
</div>\n\
</body>\n\
</html>\n";

   int  cnt,
        status,
        Count,
        GzipDeflatePercent,
        HashCollisionListLength,
        InUseEntryCount,
        LoadingEntryCount,
        ReclaimedEntryCount,
        ValidEntryCount;
   unsigned long  FaoVector [64];
   unsigned long  *vecptr;
   char  *cptr, *sptr,
         *LastColPtr;
   FILE_CENTRY  *captr;
   LIST_ENTRY  *leptr;

   /*********/
   /* begin */
   /*********/

   if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE, "CacheReport()");

   InUseEntryCount = LoadingEntryCount =
      ReclaimedEntryCount = ValidEntryCount = 0;
   for (leptr = CacheList.HeadPtr; leptr; leptr = leptr->NextPtr)
   {
      captr = (FILE_CENTRY*)leptr;
      if (captr->EntryValid) ValidEntryCount++;
      if (captr->EntryReclaimed) ReclaimedEntryCount++;
      if (captr->InUseCount) InUseEntryCount++;
      if (captr->DataLoading) LoadingEntryCount++;
   }

   AdminPageTitle (rqptr, "Cache Report");

   vecptr = FaoVector;

   if (CacheEnabled)
      *vecptr++ = "[enabled]";
   else
      *vecptr++ = "<font color=\"#ff000\">[disabled]</font>";

   *vecptr++ = CachePermMemoryInUse >> 10;
   *vecptr++ = CacheMemoryInUse >> 10;
   *vecptr++ = CacheTotalKBytesMax;
   *vecptr++ = CachePermEntryCount;
   *vecptr++ = CacheEntryCount;
   *vecptr++ = CacheEntriesMax;
   *vecptr++ = ValidEntryCount;
   *vecptr++ = InUseEntryCount;
   *vecptr++ = LoadingEntryCount;
   *vecptr++ = ReclaimedEntryCount;
   *vecptr++ = CacheEntryKBytesMax;
   *vecptr++ = CacheGuardSeconds;
   *vecptr++ = CacheValidateSeconds;
   *vecptr++ = CacheFrequentHits;
   *vecptr++ = CacheFrequentSeconds;

   *vecptr++ = CacheHashTableCount;
   *vecptr++ = CacheHashTableHitCount;
   if (CacheHashTableCount)
      *vecptr++ = CacheHashTableHitCount * 100 / CacheHashTableCount;
   else
      *vecptr++ = 0;
   *vecptr++ = CacheHashTableMissCount;
   if (CacheHashTableCount)
      *vecptr++ = CacheHashTableMissCount * 100 / CacheHashTableCount;
   else
      *vecptr++ = 0;
   *vecptr++ = CacheHashTableCollsnCount;
   if (CacheHashTableCount)
      *vecptr++ = CacheHashTableCollsnCount * 100 / CacheHashTableCount;
   else
      *vecptr++ = 0;
   *vecptr++ = CacheHashTableCollsnHitCount;
   if (CacheHashTableCount)
      *vecptr++ = CacheHashTableCollsnHitCount * 100 / CacheHashTableCount;
   else
      *vecptr++ = 0;
   *vecptr++ = CacheHashTableCollsnMissCount;
   if (CacheHashTableCount)
      *vecptr++ = CacheHashTableCollsnMissCount * 100 / CacheHashTableCount;
   else
      *vecptr++ = 0;

   *vecptr++ = CacheCurrentlyLoading;
   *vecptr++ = CacheCurrentlyLoadingInUse >> 10;
   *vecptr++ = CacheReclaimCount;
   if (CacheLoadCount)
      *vecptr++ = CacheReclaimCount * 100 / CacheLoadCount;
   else
      *vecptr++ = 0;

   *vecptr++ = CacheGzipDeflateCount;
   *vecptr++ = &CacheGzipDeflateBytesIn64;
   *vecptr++ = &CacheGzipDeflateBytesOut64;
   *vecptr++ = PercentOf64 (&CacheGzipDeflateBytesOut64,
                            &CacheGzipDeflateBytesIn64);

   *vecptr++ = CacheLoadCount;
   *vecptr++ = CacheHits0;
   *vecptr++ = PercentOf32(CacheHits0,CacheLoadCount);
   *vecptr++ = CacheHitCount;
   *vecptr++ = PercentOf32(CacheHitCount,CacheLoadCount);
   *vecptr++ = CacheHits10;
   *vecptr++ = PercentOf32(CacheHits10,CacheLoadCount);
   *vecptr++ = CacheHits100;
   *vecptr++ = PercentOf32(CacheHits100,CacheLoadCount);
   *vecptr++ = CacheHits1000;
   *vecptr++ = PercentOf32(CacheHits1000,CacheLoadCount);
   *vecptr++ = CacheHits1000plus;
   *vecptr++ = PercentOf32(CacheHits1000plus,CacheLoadCount);

   if (CacheEnabled && !IncludeEntries)
   {
      *vecptr++ = EntriesButtonFao;
      *vecptr++ = ADMIN_REPORT_CACHE_ENTRIES;
      *vecptr++ = AdminRefresh (rqptr);
   }
   else
      *vecptr++ = "";

   status = FaolToNet (rqptr, BeginPageFao, &FaoVector);
   if (VMSnok (status)) ErrorNoticed (rqptr, status, NULL, FI_LI);

   if (!CacheEnabled || !IncludeEntries)
   {
      rqptr->rqResponse.PreExpired = PRE_EXPIRE_ADMIN;
      ResponseHeader200 (rqptr, "text/html", &rqptr->NetWriteBufferDsc);

      AdminEnd (rqptr);
      return;
   }

   /*****************/
   /* cache entries */
   /*****************/

   status = FaolToNet (rqptr, EntriesFao, NULL);
   if (VMSnok (status)) ErrorNoticed (rqptr, status, NULL, FI_LI);

   Count = 0;

   if (WATCHPNT(rqptr) &&
       WATCH_MODULE(WATCH_MOD_CACHE) && WATCH_MODULE(WATCH_MOD__DETAIL))
       WatchThis (WATCHALL, WATCH_MOD_CACHE, "!&X !&X",
                  CacheList.HeadPtr, CacheList.TailPtr);

   /* process the cache entry list from most to least recently hit */
   for (leptr = CacheList.HeadPtr; leptr; leptr = leptr->NextPtr)
   {
      captr = (FILE_CENTRY*)leptr;

      if (WATCHPNT(rqptr) &&
          WATCH_MODULE(WATCH_MOD_CACHE) && WATCH_MODULE(WATCH_MOD__DETAIL))
          WatchThis (WATCHALL, WATCH_MOD_CACHE, "!&X<-!&X->!&X !&X",
                     leptr->PrevPtr, leptr, leptr->NextPtr, captr);

      HashCollisionListLength = 0;
      /* count further down the list */
      while (captr->HashCollisionNextPtr)
      {
         captr = captr->HashCollisionNextPtr;
         HashCollisionListLength++;
      }
      captr = (FILE_CENTRY*)leptr;
      /* count further up the list */
      while (captr->HashCollisionPrevPtr)
      {
         captr = captr->HashCollisionPrevPtr;
         HashCollisionListLength++;
      }
      captr = (FILE_CENTRY*)leptr;

      vecptr = FaoVector;

      *vecptr++ = &captr->Md5Hash;
      if (CacheEntriesMax >= 10000)
         *vecptr++ = 5;
      else
      if (CacheEntriesMax >= 1000)
         *vecptr++ = 4;
      else
         *vecptr++ = 3;
      *vecptr++ = ++Count;
      *vecptr++ = captr->EntryPermanent;
      *vecptr++ = captr->FromFile;
      *vecptr++ = captr->FromNet;
      *vecptr++ = captr->FromScript;
      *vecptr++ = captr->CgiHeaderLength;
      *vecptr++ = captr->ContentType[0];
      *vecptr++ = captr->EntrySize;
      *vecptr++ = captr->ContentLength;
      *vecptr++ = captr->GzipEntrySize;
      *vecptr++ = captr->GzipContentLength;
      *vecptr++ = PercentOf32(captr->GzipContentLength,captr->ContentLength);
      *vecptr++ = captr->InUseCount;
      /* 12 bit hash value, 0..4095 from a fixed 3 bytes of the MD5 hash */
      *vecptr++ = captr->Md5Hash.HashLong[0] & 0xfff;
      *vecptr++ = HashCollisionListLength;

      if (captr->FromFile)
      {
         *vecptr++ = "!20%D";
         *vecptr++ = &captr->RdtTime64;
      }
      else
         *vecptr++ = "n/a";

      if (captr->FromFile &&
          captr->EntryValid)
      {
         *vecptr++ = "!20%D&nbsp;!UL";
         *vecptr++ = &captr->ValidateTime64;
         *vecptr++ = captr->ValidatedCount;
      }
      else
      {
         if (captr->EntryReclaimed)
            *vecptr++ = "RECLAIMED";
         else
         if (!captr->EntryValid)
            *vecptr++ = "INVALID";
         else
         if (!captr->FromFile)
            *vecptr++ = "n/a";
         else
            *vecptr++ = "?";
      }

      *vecptr++ = &captr->LoadTime64;
      *vecptr++ = &captr->HitTime64;
      *vecptr++ = captr->HitCount;
      *vecptr++ = captr->HitNotModifiedCount;
      *vecptr++ = captr->FileOds.ExpFileName;

      status = FaolToNet (rqptr, CacheFao, &FaoVector);
      if (VMSnok (status)) ErrorNoticed (rqptr, status, NULL, FI_LI);
   }
   if (!CacheList.HeadPtr)
   {
      status = FaolToNet (rqptr, EmptyCacheFao, NULL);
      if (VMSnok (status)) ErrorNoticed (rqptr, status, NULL, FI_LI);
   }

   vecptr = FaoVector;
   *vecptr++ = AdminRefresh (rqptr);

   status = FaolToNet (rqptr, EndPageFao, &FaoVector);
   if (VMSnok (status)) ErrorNoticed (rqptr, status, NULL, FI_LI);

   rqptr->rqResponse.PreExpired = PRE_EXPIRE_ADMIN;
   ResponseHeader200 (rqptr, "text/html", &rqptr->NetWriteBufferDsc);

   AdminEnd (rqptr);
}

/*****************************************************************************/
/*
Return a report on a single cache entry.
*/ 

void CacheReportEntry
(
REQUEST_STRUCT *rqptr,
char *Md5HashHexString
)
{
   static char BeginPageFao [] =
"<p><table class=\"ctgry\">\n\
<tr><td>\n\
<table class=\"rghtlft\">\n\
<tr><th>Resource:</th>\
<td style=\"text-align:left;\"><nobr>!&;AZ</nobr></td></tr>\n\
<tr><th>Entity&nbsp;Tag:</th>\
<td style=\"text-align:left;\">!AZ</td></tr>\n\
<tr><th>Flags:</th><td style=\"text-align:left;\">\
!&?Permanent\rVolatile\r, \
!&?File\r\r!&?Network\r\r!&?Script\r\r, \
!&?CGI-fields\r<strike>CGI-fields</strike>\r, \
!&?Content-Type\r<strike>Content-Type</strike>\r, \
<tr><th>Size:</th><td style=\"text-align:left;\">!UL byte!%s</td></tr>\n\
<tr><th>Length:</th><td style=\"text-align:left;\">!UL byte!%s</td></tr>\n\
<tr><th>GZIP Size:</th><td style=\"text-align:left;\">!UL byte!%s</td></tr>\n\
<tr><th>GZIP Length:</th>\
<td style=\"text-align:left;\">!UL byte!%s&nbsp;&nbsp;(!UL%)</td></tr>\n\
<tr><th>CGI&nbsp;Fields:</th>\
<td style=\"text-align:left;\">!UL byte!%s</td></tr>\n\
<tr><th>Content-Type:</th><td style=\"text-align:left;\">!&;AZ</td></tr>\n\
<tr><th><nobr>In-Use:</nobr></th><td style=\"text-align:left;\">!UL</td></tr>\n\
<tr><th><nobr>Hash/Idx/Colsn:</nobr></th>\
<td style=\"text-align:left;\">!16&H&nbsp;/&nbsp;!UL&nbsp;/&nbsp;!UL</td></tr>\n\
<tr><th>Revised:</th><td style=\"text-align:left;\">!&@</td></tr>\n\
<tr><th>Validated:</th><td style=\"text-align:left;\">!&@</td></tr>\n\
<tr><th>Loaded:</th><td style=\"text-align:left;\">!20%D</td></tr>\n\
<tr><th>Hit:</th>\
<td style=\"text-align:left;\">!20%D, !UL time!%s, 304 response !UL time!%s</td></tr>\n\
</table>\n\
</td></tr>\n\
</table>\n\
<p><pre>";

   int  idx, status,
        ItemCount,
        HashCollisionListLength;
   unsigned long  FaoVector [64];
   unsigned long  *vecptr;
   char  *cptr;
   MD5_HASH  Md5Hash;
   FILE_CENTRY  *captr;
   LIST_ENTRY  *leptr;

   /*********/
   /* begin */
   /*********/

   if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE,
                 "CacheReportEntry() !&Z", Md5HashHexString);

   /* convert hex string back into binary hash */
   cptr = Md5HashHexString;
   for (idx = 0; idx < sizeof(Md5Hash); idx++)
   {
      if (*cptr >= '0' && *cptr <= '9')
         Md5Hash.HashChar[idx] = (*cptr - '0') << 4;
      else
      if (to_upper(*cptr) >= 'A' && to_upper(*cptr) <= 'F')
         Md5Hash.HashChar[idx] = (*cptr - '7') << 4;
      if (*cptr) cptr++;
      if (*cptr >= '0' && *cptr <= '9')
         Md5Hash.HashChar[idx] |= (*cptr - '0') & 0xf;
      else
      if (to_upper(*cptr) >= 'A' && to_upper(*cptr) <= 'F')
         Md5Hash.HashChar[idx] |= (*cptr - '7') & 0xf;
      if (*cptr) cptr++;
   }

   captr = NULL;
   ItemCount = 0;
   for (leptr = CacheList.HeadPtr; leptr; leptr = leptr->NextPtr)
   {
      captr = (FILE_CENTRY*)leptr;
      if (!MATCH0 (&captr->Md5Hash, &Md5Hash, sizeof(Md5Hash))) continue;
      break;
   }
   if (!leptr)
   {
      rqptr->rqResponse.HttpStatus = 400;
      ErrorGeneral (rqptr, "Entry not found.", FI_LI);
      AdminEnd (rqptr);
      return;
   }

   AdminPageTitle (rqptr, "Cache Entry Report");

   HashCollisionListLength = 0;
   /* count further down the list */
   while (captr->HashCollisionNextPtr)
   {
         captr = captr->HashCollisionNextPtr;
         HashCollisionListLength++;
   }
   captr = (FILE_CENTRY*)leptr;
   /* count further up the list */
   while (captr->HashCollisionPrevPtr)
   {
      captr = captr->HashCollisionPrevPtr;
      HashCollisionListLength++;
   }
   captr = (FILE_CENTRY*)leptr;

   vecptr = FaoVector;

   *vecptr++ = captr->FileOds.ExpFileName;
   *vecptr++ = captr->EntityTag;
   *vecptr++ = captr->EntryPermanent;
   *vecptr++ = captr->FromFile;
   *vecptr++ = captr->FromNet;
   *vecptr++ = captr->FromScript;
   *vecptr++ = captr->CgiHeaderLength;
   *vecptr++ = captr->ContentType[0];
   *vecptr++ = captr->EntrySize;
   *vecptr++ = captr->ContentLength;
   *vecptr++ = captr->GzipEntrySize;
   *vecptr++ = captr->GzipContentLength;
   *vecptr++ = PercentOf32(captr->GzipContentLength,captr->ContentLength);
   *vecptr++ = captr->CgiHeaderLength;
   if (captr->ContentType[0])
      *vecptr++ = captr->ContentType;
   else
      *vecptr++ = "n/a";
   *vecptr++ = captr->InUseCount;
   *vecptr++ = &captr->Md5Hash.HashLong;
   /* 12 bit hash value, 0..4095 from a fixed 3 bytes of the MD5 hash */
   *vecptr++ = captr->Md5Hash.HashLong[0] & 0xfff;
   *vecptr++ = HashCollisionListLength;

   if (captr->FromFile)
   {
      *vecptr++ = "!20%D";
      *vecptr++ = &captr->RdtTime64;
   }
   else
      *vecptr++ = "n/a";

   if (captr->FromFile &&
       captr->EntryValid)
   {
      *vecptr++ = "!20%D, !UL time!%s";
      *vecptr++ = &captr->ValidateTime64;
      *vecptr++ = captr->ValidatedCount;
   }
   else
   {
      if (captr->EntryReclaimed)
         *vecptr++ = "RECLAIMED";
      else
      if (!captr->EntryValid)
         *vecptr++ = "INVALID";
      else
      if (!captr->FromFile)
         *vecptr++ = "n/a";
      else
         *vecptr++ = "?";
   }

   *vecptr++ = &captr->LoadTime64;
   *vecptr++ = &captr->HitTime64;
   *vecptr++ = captr->HitCount;
   *vecptr++ = captr->HitNotModifiedCount;

   status = FaolToNet (rqptr, BeginPageFao, &FaoVector);
   if (VMSnok (status)) ErrorNoticed (rqptr, status, NULL, FI_LI);

   vecptr = FaoVector;

   if (captr->ContentPtr)
   {
      CacheDumpData (rqptr, captr->ContentPtr, captr->ContentLength);
      NetWriteBuffered (rqptr, NULL,
"<hr size=\"1\" noshade width=\"60%\" align=\"left\">", -1);
   }

   if (captr->GzipContentPtr)
   {
      CacheDumpData (rqptr, captr->GzipContentPtr, captr->GzipContentLength);
      NetWriteBuffered (rqptr, NULL,
"<hr size=\"1\" noshade width=\"60%\" align=\"left\">", -1);
   }

   NetWriteBuffered (rqptr, NULL, "</pre>\n</body>\n</html>\n", -1);


   rqptr->rqResponse.PreExpired = PRE_EXPIRE_ADMIN;
   ResponseHeader200 (rqptr, "text/html", &rqptr->NetWriteBufferDsc);

   AdminEnd (rqptr);
}

/*****************************************************************************/
/*
Dump data a la WATCH.
*/ 

CacheDumpData
(
REQUEST_STRUCT *rqptr,
char *DataPtr,
int DataLength
)
{
/* 32 bytes by 128 lines comes out to 4096 bytes, the default buffer-full */
#define MAX_LINES 128
#define BYTES_PER_LINE 32
#define BYTES_PER_GROUP 4
#define GROUPS_PER_LINE (BYTES_PER_LINE / BYTES_PER_GROUP)
#define CHARS_PER_LINE ((BYTES_PER_LINE * 3) + GROUPS_PER_LINE + 1)
#define HTML_ESCAPE_SPACE 1024

   static char  HexDigits [] = "0123456789ABCDEF";
   static char  Woops [] = "ERROR: Buffer overflow!";

   int  ByteCount,
        CurrentDataCount,
        DataCount;
   char  *cptr, *sptr, *zptr,
         *CurrentDataPtr,
         *CurrentDumpPtr;
   char  DumpBuffer [(CHARS_PER_LINE * MAX_LINES)+HTML_ESCAPE_SPACE+1];

   /*********/
   /* begin */
   /*********/

   if (WATCHMOD (rqptr, WATCH_MOD_CACHE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_CACHE,
                 "CacheDumpData() !&A !UL", DataPtr, DataLength);

   if (!DataPtr) return;

   zptr = (sptr = DumpBuffer) + sizeof(DumpBuffer)-1;
   cptr = DataPtr;
   DataCount = DataLength;

   while (DataCount)
   {
      CurrentDumpPtr = sptr;
      CurrentDataPtr = cptr;
      CurrentDataCount = DataCount;

      ByteCount = BYTES_PER_LINE;
      while (ByteCount && DataCount)
      {
         if (sptr < zptr) *sptr++ = HexDigits[*(unsigned char*)cptr >> 4];
         if (sptr < zptr) *sptr++ = HexDigits[*(unsigned char*)cptr & 0xf];
         cptr++;
         DataCount--;
         ByteCount--;
         if (!(ByteCount % BYTES_PER_GROUP) && sptr < zptr)  *sptr++ = ' ';
      }
      while (ByteCount)
      {
         if (sptr < zptr) *sptr++ = ' ';
         if (sptr < zptr) *sptr++ = ' ';
         ByteCount--;
         if (!(ByteCount % BYTES_PER_GROUP) && sptr < zptr) *sptr++ = ' ';
      }

      cptr = CurrentDataPtr;
      DataCount = CurrentDataCount;

      ByteCount = BYTES_PER_LINE;
      while (ByteCount && DataCount)
      {
         if (isalnum(*cptr) || ispunct(*cptr) || *cptr == ' ')
         {
            if (*cptr == '<')
            {
               if (sptr < zptr) *sptr++ = '&';
               if (sptr < zptr) *sptr++ = 'l';
               if (sptr < zptr) *sptr++ = 't';
               if (sptr < zptr) *sptr++ = ';';
               cptr++;
            }
            else
            if (*cptr == '&')
            {
               if (sptr < zptr) *sptr++ = '&';
               if (sptr < zptr) *sptr++ = 'a';
               if (sptr < zptr) *sptr++ = 'm';
               if (sptr < zptr) *sptr++ = 'p';
               if (sptr < zptr) *sptr++ = ';';
               cptr++;
            }
            else
            {
               if (sptr < zptr) *sptr++ = *cptr;
               cptr++;
            }
         }
         else
         {
            if (sptr < zptr) *sptr++ = '.';
            cptr++;
         }
         DataCount--;
         ByteCount--;
      }
      /* ensure there is a right margin using a couple of spaces */
      if (sptr < zptr) *sptr++ = ' ';
      if (sptr < zptr) *sptr++ = ' ';
      if (sptr < zptr) *sptr++ = '\n';
      if (sptr >= zptr)
      {
         NetWriteBuffered (rqptr, NULL, Woops, sizeof(Woops)-1);
         return;
      }

      if (!DataCount || !ByteCount)
      {
         *sptr = '\0';
         NetWriteBuffered (rqptr, NULL, DumpBuffer, sptr - DumpBuffer);
         zptr = (sptr = DumpBuffer) + sizeof(DumpBuffer)-1;
      }
   }
}

/*****************************************************************************/