/*****************************************************************************/ /* File.c This module implements a full multi-threaded, AST-driven, asynchronous file send. The AST-driven nature makes the code a little more difficult to follow, but creates a powerful, event-driven, multi-threaded server. All of the necessary functions implementing this module are designed to be non-blocking. It can operate in one of four modes. 1) File direct to network. ~~~~~~~~~~~~~~~~~~~~~~ A smallish, standard output buffer is allocated if variable length record file, successive records are read into the buffer until it fills, then written to the network as a single block. Records always have a newline character added to each record (variable length record files are invariably text). If a stream, fixed or undefined record format the buffer is filled in a single virtual block read and then immediately written to the network. It uses the same buffer space as, and interworks with, NetWriteBuffered(). If there is already data (text) buffered in this area the file module will, for record-oriented, non-HTML-escaped transfers, continue to fill the area (using its own buffering function), flushing when and if necessary. At end-of-file explicitly flush the buffer only if escaping HTML-forbidden characters, otherwise allow subsequent processing to do it as necessary. For block-mode files the buffer is explicitly flushed before commencing the file transfer. 2) File into file cache buffer, simultaneously to network. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A file cache buffer of sufficient size to contain the whole file is allocated. If reading in record mode this is filled with successive reads (as per above) until a chunk the size of a standard output buffer is filled, at which time it is written to the network as a block. In block mode each group of blocks read is output to the network. 3) File into contents buffer. ~~~~~~~~~~~~~~~~~~~~~~~~~ A file contents buffer of sufficient size to contain the whole file is allocated. When reading in either record or block mode the buffer is just filled with the file reads - NO NETWORK OUTPUT. When complete the request gets control of the contents buffer for subsequent processing. 4) File into file cache as well as into contents buffer. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Both file cache and contents buffers of sufficient size are allocated. Records and blocks fill both buffers during the reads - NO NETWORK OUTPUT. When complete the file is cached and the contents buffer filled. When complete the request gets control of the contents buffer for subsequent processing. (Subsequent access to the file can then be serviced from the file cache.) BYTE RANGES ----------- The "Range:" request header is supported for non variable length record files (i.e. those that can be considered to contain 'binary' content, e.g. stream-LF, stream-CR, stream, fixed, undefined). The cache module also supports byte-ranges for cached file content (though without the restriction - all cache content is 'binary' in nature). The majority of files that byte ranges might be applied to are probably large and having 'binary' content (e.g. restarting ZIP archive transfers, accessing 'linearized' PDF documents, etc.) The $READs of virtual blocks used to access these files allow a relatively simple algorithm to access these ranges and return 206 (partial content) responses. No effort is made to support this for variable record length files. If a byte-range is invalid or cannot be applied to the particular file type it is just ignored and a 200 full transfer is performed instead. MD5 DIGEST ---------- An MD5 digest (16 byte hash) is used to uniquely identify cached files. The hash is generated either from a mapped path or from the file name. See the CACHE.C module for further detail. FILE LANGUAGE VARIANTS ---------------------- If the path is SET to ACCEPT=LANG then this module attempts to find language-specific variants of the file. The format of the file names for these variants is ._ where 'language' is one of the ISO language abbreviations, e.g. "en" for English, "fr" for French, "de" for German, "ru" for Russian, etc. Hence if the basic file name is EXAMPLE.HTML then a specifically English version would be EXAMPLE.HTML_EN, a French version EXAMPLE.HTML_FR, etc. Language variants may be provided for any file type the WASD_CONFIG_GLOBAL directive [AddType] specifies as content-type "text/..". The language variant code behaves as follows. When FileBegin() is called with a path SET to ACCEPT=LANG and a default language is specified (for those files without the language variant abbreviation) this is checked against the request's accepted languages to see if the default would be the request's first choice. If so then there is no need for further accept language processing. If not then a series of functions search using the basic specification for files matching "EXAMPLE.HTML_*". All files matching this wildcard have the '*' portion (e.g. "EN", "FR", "DE", "RU") added to a list of matching variants. When the search is complete a final function compares each of the request's "Accept-Language:" list to each of the searched-for variants. If one matches the contents of that file are returned. If none are matched the original EXAMPLE.HTML would be returned. Example behaviour. A directory contains EXAMPLE.HTML EXAMPLE.HTML_FR EXAMPLE.HTML_DE EXAMPLE.HTML_RU and a request specifies Accept-Language: en,fr,de then the EXAMPLE.HTML_FR file will be returned. For a request specifying Accept-Language: ru then the EXAMPLE.HTML_RU file is returned, and if Accept-Language: en EXAMPLE.HTML returned (without having searched for any other variants). One or other file is always returned, with the default, non-language file always the fallback source of data. If it does not exist and no other language variant is selected the request returns a 404 file-not-found error. IGNORED CONTENT-TYPE -------------------- Some browsers and/or some operating systems and/or some versions of both insist on ignoring the response header specified content-type and instead seem to second-guess (wrongly) on the file name extension. A common example is the content of DCL procedures on Windows and up-until-fairly-recent versions of Internet Explorer. I note that versions 9 and 10 seem more compliant with the the returned header. Notwithstanding, if a '$' and then a second extension is added to the URI this is often sufficient to coerce the browser into accepting (and display) the content with the type associated with that extension. In the case of the DCL procedure /wasd_root/src/build_all.com making the URI into /wasd_root/src/build_all.com$.txt, etc. Any suitable additional file extension that will be accepted can be used. GENERAL COMMENTS ---------------- Uses the ACP-QIO interface detailed in the "OpenVMS I/O User's Reference Manual" to retrieve file record attributes, revision date/time and size. Use QIOs to access and transfer disk blocks (just a bit more efficient and flexible than RMS). RMS structures are used to parse the file name and obtain the DID and FID uses to QIO access the file. Some record types are considered binary content and can be served without 'massaging' the data. These are STREAM (CR,LF), FIXED-512 (or where the record size falls on a block boundary), and UNDEFINED. Those record types that have an internal structure and must be 'massaged' to get the data into the stream format the Web is so fond of have functions for doing just that. These are VARIABLE, VFC and FIXED-non-span-non-512 (where the record size is not on a block boundary and is not allowed to span block boundaries - who the hell uses this stuff anyway?!) VARIABLE and VFC record formats have newline carriage control added to records. When not buffering to cache or contents the module can encapsulate plain-text and escape HTML-forbidden characters. Works in conjunction with the CACHE module. A file read can be simultaneously used to send the data to the client and load a cache buffer. The request structure fields 'CacheContentPtr' and 'CacheContentCount' being used in both block I/O and record mode access to track through the available buffer space, for this located in the cache structure. For non-cache-load reads uses the standard buffer space pointed to by 'OutputBufferPtr' and for record mode access tracked using 'OutputBufferCurrentPtr' and 'OutputBufferRemaining' fields. Implements defacto HTTP/1.0 persistent connections. Only provide "keep-alive" response if we're sending a file that has 'binary content' (e.g. stream, fixed 512) and know its precise length. An accurate "Content-Length:" field is vital for the client's correct reception of the transmitted data. Currently the only other time a "keep-alive" response can be provided is when a "304 not-modified" header is returned (it has a content length of precisely zero!). October 1997: noted that Netscape Navigator 3.n seems to pay no attention to a "Content-Length: 0" with a "Keep-Alive:" connection, it just sits there waiting until the keep-alive time closes the connection, after which it reports "document contains no data". (IE 3.02 seems to behave correctly ;^) I have therefore disabled persistent connections for zero-length files. VERSION HISTORY --------------- 24-OCT-2024 MGD bugfix; FileNextBlocks() StrDscBegin() 01-SEP-2023 MGD bugfix; FileAcpInfoAst() 64 bit file size 29-JUN-2021 MGD bugfix; FileBegin() ERROR_REPORTED() free file task 20-MAR-2021 MGD FILE_VAR_ASIS provides exactly what is on the disk 12-NOV-2020 MGD file ->SizeInBytes64 now 64 bit datum 04-JAN-2019 MGD bugfix; AuthCompleted() 13-AUG-2018 MGD FileVariableRecord() implement SET mapping carriage control 02-MAY-2016 MGD bugfix; FileParseAst() regression with search list file 21-JUN-2014 MGD bugfix; FileParseAst() allow for non-dir .DIR files 21-SEP-2013 MGD FileAcpInfoAst() '$.' file extension kludge 07-SEP-2013 MGD bugfix; ensure "?httpd=content&type=" is URL-decoded 01-JUN-2013 MGD bugfix; non-ODS_EXTENDED platforms must OdsParse() NAM$M_NOCONCEAL before OdsSearchNoConceal() 30-JAN-2012 MGD bugfix; FileAcpInfoAst() SS$_BADPARAM >2GB <4GB (per JPP) 06-NOV-2010 MGD FileInfoAcpAst() if directory listing README ignore NOPRIV 23-MAY-2010 MGD FileNextBlocks() change QIO file size from long to quad to cater for files greater than 4GB (4GB+ is limited to file serving only, no ranges, etc.) 05-OCT-2009 MGD use OutputFileBufferSize to maximise file transfer 19-AUG-2009 MGD bugfix; FileAcpInfoAst() byte-range limit negative offset 11-JUN-2009 MGD "*.*__WASDAV;" reports not found 09-JUN-2007 MGD use STR_DSC FileGenerateEntityTag() 20-OCT-2005 MGD bugfix; FileNextBlocks() ensure VARiable record format files have records read on word (even byte) boundaries 28-MAY-2005 MGD bugfix; if none-match entity and IfModifiedSince() logic 29-NOV-2004 MGD bugfix; FileVariableRecord() memset only if positive 19-SEP-2004 MGD bugfix; even number of bytes on a disk $QIO READVBLK 20-JUL-2004 MGD HTTP/1.1 compliance, HttpIfUnModifiedSince() check, calculate content-length for byte range response header 26-APR-2004 MGD major changes to eliminate RMS from file access (WASD's doing all the content conversion work anyway!) by using ACP/QIOs and massaging record content explicitly (outgrowth of returns from 18-FEB-2004 changes) 18-FEB-2004 MGD read variable record format files using block IO and then explicitly process the those records to produce a stream-LF block of data in their place! (provides in excess of 400% throughput boost!!! :^) bugfix; rare RECTOOBIG on variable record length file where longest record exceeded 'OutputBufferSize' so initialize buffer to maximum of 'OutputBufferSize' or file lrl (use 'rqptr->rqOutput.BufferSize' instead of 'OutputBufferSize') 21-AUG-2003 MGD support byte-range requests on non-VAR files 12-AUG-2003 MGD access to HTA or HTL file now reports not found 09-JUL-2003 MGD rework for new caching requirements 16-JUN-2003 MGD bugfix; FileSetCharset() following initial CacheSearch() moved to CACHE.C module (ACCVIO if entry NULLed) 05-OCT-2002 MGD no sneaky getting directory contents by downloading files! refine VMS security profile usage 03-JUN-2002 MGD bugfix; (well sort of) it would appear that after NO_CONCEAL searching and a sys$open() you must sys$close() *before* the SYNCHCK sys$parse() release of resources otherwise a channel to the device is left assigned!! 27-APR-2002 MGD make SYSPRV enabled ASTs asynchronous 19-MAR-2002 MGD bugfix; OdsParse() for VMS authenticated request 15-MAR-2002 MGD bugfix; FileNextRecordAst() VAR file into contents buffer 18-NOV-2001 MGD FileAcceptLang..() 23-OCT-2001 MGD bugfix; FileNextBlocksAst() 'ContentRemaining' 04-AUG-2001 MGD modifications in line with changes in the handling of file and cache (now MD5 hash based) processing, block I/O complete if _rsz is less than _usz support module WATCHing 10-MAY-2001 MGD bugfix; FileNextRecordAst() buffer flush 05-MAR-2001 MGD bugfix; FileNextBlocks() 32767 to 0xfe00 (65024) 29-DEC-2000 MGD allow a file's contents to be read into a buffer 23-DEC-2000 MGD allow access to an HTL if it is authorized 06-DEC-2000 MGD make a search list DNF appear as a FNF 01-SEP-2000 MGD add optional, local path authorization (for calls from the likes of SSI.C) 09-JUN-2000 MGD search-list processing refined 04-MAR-2000 MGD use NetWriteInit(), et.al. 27-DEC-1999 MGD support ODS-2 and ODS-5 using ODS module 10-OCT-1999 MGD "scrunched" (in fact all) SSI files, prevent streamLFing 17-SEP-1999 MGD bugfix; sys$parse() NAM$M_NOCONCEAL for search lists 23-MAY-1999 MGD do not allow "?httpd=content" requests to be cached 05-FEB-1999 MGD bugfix; FileNextRecord() zero '_usz' 07-NOV-1998 MGD WATCH facility 19-SEP-1998 MGD improve granularity of file open, connect, close, ACP 14-MAY-1998 MGD request-specified content-type ("httpd=content&type=") 19-MAR-1998 MGD buffer VBN and first free byte for use by cache module 19-JAN-1998 MGD new NetWriteBuffered() and structures 07-JAN-1998 MGD groan, bugfix; record processing for files > 4096 bytes completely brain-dead ... sorry 22-NOV-1997 MGD sigh, bugfix; heap corruption by file cache 05-OCT-1997 MGD file cache, keep-alive now not used if the content-length is zero 17-SEP-1997 MGD if block-mode open is locked retry open in record-mode 17-AUG-1997 MGD message database, SYSUAF-authenticated users security-profile, addressed potential problem with FIXed and odd-byte records 08-JUN-1997 MGD if request "Pragma: no-cache" then always return 27-FEB-1997 MGD delete on close for "temporary" files 01-FEB-1997 MGD HTTPd version 4 01-AUG-1996 MGD Variable to stream-LF file conversion "on-the-fly" 12-APR-1996 MGD determine read method (record or binary) from record format; implemented persistent connections ("keep-alive") 01-DEC-1995 MGD HTTPd version 3 27-SEP-1995 MGD added If-Modified-Since: functionality; changed carriage-control on records from to single ('\n' ... newline), to better comply with some browsers (Netscape was spitting on X-bitmap files, for example!) 07-AUG-1995 MGD ConfigcfReport.MetaInfoEnabled to allow physical file specification included as commentary within an HTML file 13-JUL-1995 MGD bugfix; occasionally a record was re-read after flushing the records accumulated in the buffer NOT due to RMS$_RTB 20-DEC-1994 MGD initial development for multi-threaded daemon */ /*****************************************************************************/ #ifdef WASD_VMS_V7 #undef _VMS__V6__SOURCE #define _VMS__V6__SOURCE #undef __VMS_VER #define __VMS_VER 70000000 #undef __CRTL_VER #define __CRTL_VER 70000000 #endif /* standard C header files */ #include #include #include /* VMS related header files */ #include #include #include #include #include #include #include #include /* application header files */ #include "ods.h" #include "wasd.h" #define WASD_MODULE "FILE" /******************/ /* global storage */ /******************/ char ErrorBufferFileMaxBytes [] = "Exceeds reasonable limit on the size of this type of file."; /********************/ /* external storage */ /********************/ #ifdef DBUG extern BOOL Debug; #else #define Debug 0 #endif extern BOOL CacheEnabled, OdsExtended, WebDavEnabled; extern int EfnWait, EfnNoWait, HttpdTickSecond, OutputFileBufferSize; extern ulong SysPrvMask[]; extern char ConfigContentTypeSsi[], ConfigContentTypeUnknown[], ConfigDefaultFileContentType[], ErrorSanityCheck[], HttpProtocol[], SoftwareID[]; extern ACCOUNTING_STRUCT *AccountingPtr; extern CONFIG_STRUCT Config; extern MSG_STRUCT Msgs; extern WATCH_STRUCT Watch; /*****************************************************************************/ /* Initalize the file task structure. NOTE: FILE PATHS ARE ALWAYS AUTHORIZED UNLESS EXPLICITLY TURNED OFF. */ FILE_TASK* FileNewTask (REQUEST_STRUCT *rqptr) { FILE_TASK *tkptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileNewTask()"); if (tkptr = rqptr->FileTaskPtr) ErrorExitVmsStatus (SS$_BUGCHECK, ErrorSanityCheck, FI_LI); rqptr->FileTaskPtr = tkptr = (FILE_TASK*)VmGetHeap (rqptr, sizeof(FILE_TASK)); tkptr->AuthorizePath = true; return (tkptr); } /*****************************************************************************/ /* Authorize the path to FileBegin() 'FileName' before accessing. NOTE: FILE PATHS ARE ALWAYS AUTHORIZED UNLESS EXPLICITLY TURNED OFF. */ void FileSetAuthorizePath ( REQUEST_STRUCT *rqptr, BOOL YesNo ) { FILE_TASK *tkptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileSetAuthorizePath() !&B", YesNo); /* if not previously initialised */ if (!(tkptr = rqptr->FileTaskPtr)) tkptr = FileNewTask (rqptr); tkptr->AuthorizePath = YesNo; } /*****************************************************************************/ /* Read the file into memory pointed to by 'rqptr->FileContentPtr->ContentPtr' and 'rqptr->FileContentPtr->ContentLength' in length. File is null-terminated in case it is text of some sort (this null is not counted in the length). */ void FileSetContentHandler ( REQUEST_STRUCT *rqptr, REQUEST_AST ContentHandlerFunction, int SizeMax ) { FILE_TASK *tkptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileSetContentHandler() !&X !UL", ContentHandlerFunction, SizeMax); /* if not previously initialised */ if (!(tkptr = rqptr->FileTaskPtr)) tkptr = FileNewTask (rqptr); /* setting this non-NULL indicates the file contents should be buffered */ tkptr->ContentHandlerFunction = ContentHandlerFunction; tkptr->FileContentsSizeMax = SizeMax; } /*****************************************************************************/ /* File allowed to be cached? */ void FileSetCacheAllowed ( REQUEST_STRUCT *rqptr, BOOL YesNo ) { FILE_TASK *tkptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileSetCacheAllowed() !&B", YesNo); /* if not previously initialised */ if (!(tkptr = rqptr->FileTaskPtr)) tkptr = FileNewTask (rqptr); tkptr->CacheAllowed = YesNo; } /*****************************************************************************/ /* Escape HTML-forbidden characters (e.g. '<') during file output. */ void FileSetEscapeHtml ( REQUEST_STRUCT *rqptr, BOOL YesNo ) { FILE_TASK *tkptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileSetEscapeHtml() !&B", YesNo); /* if not previously initialised */ if (!(tkptr = rqptr->FileTaskPtr)) tkptr = FileNewTask (rqptr); tkptr->EscapeHtml = YesNo; if (!rqptr->rqCache.DoNotCache) rqptr->rqCache.DoNotCache = YesNo; } /*****************************************************************************/ /* Enclose the file output with 
...
tags. */ void FileSetPreTag ( REQUEST_STRUCT *rqptr, BOOL YesNo ) { FILE_TASK *tkptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileSetPreTag() !&B", YesNo); /* if not previously initialised */ if (!(tkptr = rqptr->FileTaskPtr)) tkptr = FileNewTask (rqptr); tkptr->PreTagFileContents = YesNo; if (!rqptr->rqCache.DoNotCache) rqptr->rqCache.DoNotCache = YesNo; } /*****************************************************************************/ /* Begin to transfer a file. */ FileBegin ( REQUEST_STRUCT *rqptr, REQUEST_AST NextTaskFunction, REQUEST_AST NoSuchFileFunction, MD5_HASH *Md5HashPtr, char *FileName, char *ContentTypePtr ) { int status, FilePathLength; char *cptr, *sptr, *zptr; FILE_TASK *tkptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileBegin() !&F !&B !&A !&A !16&H !&Z !&Z", &FileBegin, ERROR_REPORTED(rqptr), NextTaskFunction, NoSuchFileFunction, Md5HashPtr, FileName, ContentTypePtr); if (ERROR_REPORTED (rqptr)) { /* previous error, cause threaded processing to unravel */ if (rqptr->FileTaskPtr) { VmFreeFromHeap (rqptr, rqptr->FileTaskPtr, FI_LI); rqptr->FileTaskPtr = NULL; } SysDclAst (NextTaskFunction, rqptr); return; } InstanceGblSecIncrLong (&AccountingPtr->DoFileModuleCount); /* if not previously initialised */ if (!(tkptr = rqptr->FileTaskPtr)) tkptr = FileNewTask (rqptr); tkptr->ContentTypePtr = ContentTypePtr; tkptr->NextTaskFunction = NextTaskFunction; tkptr->NoSuchFileFunction = NoSuchFileFunction; if (rqptr->rqPathSet.AcceptLangChar && ((ContentTypePtr && strsame (ContentTypePtr, "text/", 5)) || rqptr->rqContentInfo.TypeText || rqptr->rqPathSet.AcceptLangWildcard)) { /* if a default language specified and default is satisfactory */ if (!rqptr->rqPathSet.AcceptLangPtr || !FileAcceptLangDefault (rqptr)) { /* try and resolve a language-specific document */ if (!FileName) { /* must be a cache search, not interested at this stage */ FileEnd (rqptr); return; } FileAcceptLangBegin (rqptr, FileName); return; } } if (!(cptr = FileName)) cptr = ""; zptr = (sptr = tkptr->FileName) + sizeof(tkptr->FileName); while (*cptr && sptr < zptr) *sptr++ = *cptr++; if (sptr >= zptr) { ErrorGeneralOverflow (rqptr, FI_LI); FileEnd (rqptr); return; } *sptr = '\0'; tkptr->FileNameLength = sptr - tkptr->FileName; if (Md5HashPtr) { /* buffer the supplied resource hash in the task structure */ memcpy (&tkptr->Md5Hash, Md5HashPtr, sizeof(MD5_HASH)); } else { /* generate a hash representing the file name being accessed */ Md5Digest (tkptr->FileName, tkptr->FileNameLength, &tkptr->Md5Hash); } /* no file name supplied indicates only searching the cache */ if (tkptr->AuthorizePath && tkptr->FileNameLength) { /***********************/ /* check authorization */ /***********************/ cptr = MapVmsPath (tkptr->FileName, rqptr); if (!*cptr) { /* MAPURL errors are returned with a leading null (historical ;^) */ ErrorGeneral (rqptr, cptr+1, FI_LI); FileEnd (rqptr); return; } Authorize (rqptr, cptr, -1, NULL, 0, &FileAuthorizationAst); if (VMSnok (rqptr->rqAuth.FinalStatus)) { /* if asynchronous authentication is not underway */ if (rqptr->rqAuth.FinalStatus != AUTH_PENDING) FileEnd (rqptr); return; } } /* not to-be-authorized, or authorized ... just carry on regardless! */ FileAuthorizationAst (rqptr); } /*****************************************************************************/ /* There is a default language set against the path. Compare the request header "Accept-Language:" first language (if any) to the default language. If the same return true, otherwise false. */ BOOL FileAcceptLangDefault (REQUEST_STRUCT *rqptr) { char *cptr, *sptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileAcceptLangDefault()"); if (WATCHING (rqptr, WATCH_RESPONSE)) WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "LANGUAGE default:!AZ accept:!AZ", rqptr->rqPathSet.AcceptLangPtr, rqptr->rqHeader.AcceptLangPtr ? rqptr->rqHeader.AcceptLangPtr : "(none)"); if (!(cptr = rqptr->rqPathSet.AcceptLangPtr)) return (true); if (!(sptr = rqptr->rqHeader.AcceptLangPtr)) return (true); if (!*sptr || *sptr == '*') return (true); while (*cptr && *sptr && *sptr != ',' && *sptr != ';' && to_lower(*cptr) == to_lower(*sptr)) { cptr++; sptr++; } if (!*cptr && (!*sptr || *sptr == ',' || *sptr == ';')) return (true); else return (false); } /*****************************************************************************/ /* This series if functions attempts to resolve a language-specific document based on the file name originally supplied. A seachable file specification is contructed and used to find all possible language variants of the file originally specified. */ FileAcceptLangBegin ( REQUEST_STRUCT *rqptr, char *FileName ) { BOOL LowerCaseHit; int cnt, status; char *cptr, *sptr, *zptr; FILE_TASK *tkptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileAcceptLangBegin() !&Z", FileName); if (WATCHING (rqptr, WATCH_RESPONSE)) WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "LANGUAGE !AZ", FileName); tkptr = rqptr->FileTaskPtr; cptr = FileName; zptr = (sptr = tkptr->FileName) + sizeof(tkptr->FileName); /* copy the file name until the end of directory */ while (*cptr && sptr < zptr) { if (*cptr == ']' && !SAME2(cptr,'][')) break; *sptr++ = *cptr++; } LowerCaseHit = false; /* copy the file name to the type delimiting period */ while (*cptr && sptr < zptr) { #ifdef ODS_EXTENDED if (OdsExtended) { if (*cptr == '^' && SAME2(cptr,'^.')) { *sptr++ = *cptr++; if (sptr < zptr) *sptr++ = *cptr++; continue; } } if (*cptr == '.') break; if (islower(*cptr)) LowerCaseHit = true; *sptr++ = *cptr++; #else /* ODS_EXTENDED */ if (*cptr == '.') break; *sptr++ = *cptr++; #endif /* ODS_EXTENDED */ } #ifdef ODS_EXTENDED /* introduce an escaping '^' for the original type delimiting period */ if (OdsExtended && *cptr && rqptr->rqPathSet.AcceptLangChar == '.' && sptr < zptr) *sptr++ = '^'; #endif /* ODS_EXTENDED */ if (rqptr->rqPathSet.AcceptLangTypeVariant) { /*********************/ /* variant file type */ /*********************/ /* if there is no existing period add one */ if (!*cptr && sptr < zptr) *sptr++ = '.'; /* copy file type to end of string */ while (*cptr && sptr < zptr) { #ifdef ODS_EXTENDED if (islower(*cptr)) LowerCaseHit = true; #endif /* ODS_EXTENDED */ *sptr++ = *cptr++; } /* note the point at which we begin to add wildcarded variants */ tkptr->AcceptLangVariantPtr = sptr; /* append the variant delimiting character and a wildcard */ if (sptr < zptr) *sptr++ = rqptr->rqPathSet.AcceptLangChar; if (sptr < zptr) *sptr++ = '*'; } else { /*********************/ /* variant file name */ /*********************/ /* note the point at which we begin to add wildcarded variants */ tkptr->AcceptLangVariantPtr = sptr; /* insert the variant delimiting character, then wildcard(s :^) */ if (sptr < zptr) *sptr++ = rqptr->rqPathSet.AcceptLangChar; /* Bit of a shonky here. I'm reserving space in the filename for the largest possible ISO language string which (as far as I know) is 5 characters (e.g. "fr-BE", "fr-CA"). This makes reusing the file buffer space in FileAcceptLangSelect() much easier (I'm getting lazier faster than I'm actually getting older). The multiple wildcards used here (seem to) make no difference for RMS. */ for (cnt = FILE_ACCEPT_LANG_VARIANT_MAX; cnt && sptr < zptr; cnt--) *sptr++ = '*'; /* copy file type to end of string */ while (*cptr && sptr < zptr) { #ifdef ODS_EXTENDED if (islower(*cptr)) LowerCaseHit = true; #endif /* ODS_EXTENDED */ *sptr++ = *cptr++; } } if (sptr >= zptr) { ErrorGeneralOverflow (rqptr, FI_LI); FileEnd (rqptr); return; } *sptr = '\0'; tkptr->FileNameLength = sptr - tkptr->FileName; tkptr->AcceptLangLowerCase = LowerCaseHit; OdsStructInit (&tkptr->FileOds, true); OdsParse (&tkptr->FileOds, tkptr->FileName, tkptr->FileNameLength, NULL, 0, 0, &FileAcceptLangParseAst, rqptr); } /*****************************************************************************/ /* AST called from FileAcceptLangBegin() when asynchronous parse completes. Check the status and if OK begin the search. */ FileAcceptLangParseAst (REQUEST_STRUCT *rqptr) { int status; FILE_TASK *tkptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileAcceptLangParseAst() !&F sts:!&S stv:!&S", &FileAcceptLangParseAst, rqptr->FileTaskPtr->FileOds.Fab.fab$l_sts, rqptr->FileTaskPtr->FileOds.Fab.fab$l_stv); tkptr = rqptr->FileTaskPtr; if (VMSnok (status = tkptr->FileOds.Fab.fab$l_sts)) { /*****************/ /* error parsing */ /*****************/ /* ensure only the original file name appears in any error messages */ tkptr->FileName[tkptr->FileNameLength] = '\0'; /* if its a search list treat directory not found as if file not found */ if ((tkptr->FileOds.Nam_fnb & NAM$M_SEARCH_LIST) && status == RMS$_DNF) status = RMS$_FNF; if (tkptr->NoSuchFileFunction) { if (WATCHING (rqptr, WATCH_RESPONSE)) WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "FILE !&S", status); rqptr->HomePageStatus = status; FileEnd (rqptr); return; } rqptr->rqResponse.ErrorTextPtr = MapVmsPath (tkptr->FileName, rqptr); rqptr->rqResponse.ErrorOtherTextPtr = tkptr->FileName; ErrorVmsStatus (rqptr, status, FI_LI); FileEnd (rqptr); return; } OdsSearch (&tkptr->FileOds, &FileAcceptLangSearchAst, rqptr); } /*****************************************************************************/ /* AST called from FileAcceptLangParseAst() and then from FileAcceptLangSearchAst() when asynchronous search completes. Check the status and if OK buffer the resolved language component in a comma-separated list. The continue the search. Once no-more-files status occurs call FileAcceptLangSelect() to assess the language variants (if any). */ FileAcceptLangSearchAst (REQUEST_STRUCT *rqptr) { int status; char *cptr, *sptr, *zptr; FILE_TASK *tkptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileAcceptLangSearchAst() !&F sts:!&S stv:!&S", &FileAcceptLangSearchAst, rqptr->FileTaskPtr->FileOds.Fab.fab$l_sts, rqptr->FileTaskPtr->FileOds.Fab.fab$l_stv); tkptr = rqptr->FileTaskPtr; if (VMSnok (status = tkptr->FileOds.Fab.fab$l_sts)) { if (status == RMS$_FNF || status == RMS$_NMF) { /***************************/ /* end of directory search */ /***************************/ tkptr->FileOds.ParseInUse = false; FileAcceptLangSelect (rqptr); return; } /**********************/ /* sys$search() error */ /**********************/ /* ensure only the original file name appears in any error messages */ tkptr->FileName[tkptr->FileNameLength] = '\0'; rqptr->rqResponse.ErrorTextPtr = MapVmsPath (tkptr->FileName, rqptr); rqptr->rqResponse.ErrorOtherTextPtr = tkptr->FileName; ErrorVmsStatus (rqptr, status, FI_LI); FileEnd (rqptr); return; } /****************/ /* add language */ /****************/ if (!tkptr->AcceptLangTypesSize) { tkptr->AcceptLangTypesPtr = VmGetHeap (rqptr, FILE_ACCEPT_LANG_SIZE); tkptr->AcceptLangTypesSize = FILE_ACCEPT_LANG_SIZE; } zptr = (sptr = tkptr->AcceptLangTypesPtr) + tkptr->AcceptLangTypesSize; sptr += tkptr->AcceptLangTypesLength; if (tkptr->AcceptLangTypesLength && sptr < zptr) *sptr++ = ','; if (rqptr->rqPathSet.AcceptLangTypeVariant) { /* variant file type */ cptr = tkptr->FileOds.NamVersionPtr; while (cptr > tkptr->FileOds.NamTypePtr && *cptr != rqptr->rqPathSet.AcceptLangChar) cptr--; if (*cptr) cptr++; while (cptr < tkptr->FileOds.NamVersionPtr && sptr < zptr) *sptr++ = *cptr++; } else { /* variant file name */ cptr = tkptr->FileOds.NamTypePtr; while (cptr > tkptr->FileOds.NamNamePtr && *cptr != rqptr->rqPathSet.AcceptLangChar) cptr--; if (*cptr) cptr++; while (cptr < tkptr->FileOds.NamTypePtr && sptr < zptr) *sptr++ = *cptr++; } /* check the length of the (potential) variant (may not be) */ if ((sptr - tkptr->AcceptLangTypesPtr) - tkptr->AcceptLangTypesLength <= FILE_ACCEPT_LANG_VARIANT_MAX) { if (sptr >= zptr) { /* more than FILE_ACCEPT_LANG_SIZE / _VARIANT (256/5~=51!!) */ ErrorNoticed (rqptr, SS$_BUGCHECK, ErrorSanityCheck, FI_LI); ErrorGeneral (rqptr, ErrorSanityCheck, FI_LI); FileEnd (rqptr); return; } *sptr = '\0'; tkptr->AcceptLangTypesLength = sptr - tkptr->AcceptLangTypesPtr; } OdsSearch (&tkptr->FileOds, &FileAcceptLangSearchAst, rqptr); } /*****************************************************************************/ /* Explicitly called by FileAcceptLangSearchAst() when no-more-files status occurs to assess the comma-separated list of language components generated (if any). By comparing each "Accept-Language:" entry against each of the file language components select the first to match (if any). If one matches then adjust the original file name to include the language component with the type. If none matches revert to the original file name. Commence file processing. */ FileAcceptLangSelect (REQUEST_STRUCT *rqptr) { BOOL AcceptLangVariant; int status; char *cptr, *sptr, *tptr, *zptr; FILE_TASK *tkptr; /*********/ /* begin */ /*********/ tkptr = rqptr->FileTaskPtr; if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileAcceptLangSelect() !&F !&Z !&Z", &FileAcceptLangSelect, tkptr->AcceptLangTypesPtr, rqptr->rqHeader.AcceptLangPtr); if (WATCHING (rqptr, WATCH_RESPONSE)) WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "LANGUAGE variants:!AZ accept:!AZ", tkptr->AcceptLangTypesPtr ? tkptr->AcceptLangTypesPtr : "(none)", rqptr->rqHeader.AcceptLangPtr ? rqptr->rqHeader.AcceptLangPtr : "(none)"); AcceptLangVariant = false; if (tkptr->AcceptLangTypesLength) { /*********************************/ /* at least one possible variant */ /*********************************/ cptr = rqptr->rqHeader.AcceptLangPtr; while (*cptr) { while (*cptr && ISLWS(*cptr)) cptr++; if (!*cptr) break; sptr = tkptr->AcceptLangTypesPtr; while (*sptr) { tptr = cptr; while (*tptr && *tptr != ',' && *tptr != ';' && *sptr && *sptr != ',' && to_lower(*tptr) == to_lower(*sptr)) { tptr++; sptr++; } if ((!*tptr || *tptr == ',' || *tptr == ';') && (!*sptr || *sptr == ',')) { AcceptLangVariant = true; break; } while (*sptr && *sptr != ',') sptr++; if (*sptr) sptr++; } if (AcceptLangVariant) break; while (*cptr && *cptr != ',') cptr++; if (*cptr) cptr++; } } if (AcceptLangVariant) { /**********************************/ /* add language variant component */ /**********************************/ /* This handles both the post-file-type language variant, where the wildcard has just been appended to the file type, as well as the post-file-name language variant, where the wildcard was inserted between the end of the file name and the type delimiting period (in this case it reserved space using 5 wildcards). */ zptr = tkptr->FileName + sizeof(tkptr->FileName); /* step over any period-escaping character (would be EFS only) */ if (*(sptr = tkptr->AcceptLangVariantPtr) == '^') sptr++; /* step over the variant delimiting character */ sptr++; /* 'cptr' has been left pointing at the matching language */ tptr = cptr; if (tkptr->AcceptLangLowerCase) while (*cptr && *cptr != ',' && *cptr != ';' && sptr < zptr) *sptr++ = to_lower(*cptr++); else while (*cptr && *cptr != ',' && *cptr != ';' && sptr < zptr) *sptr++ = to_upper(*cptr++); if (sptr >= zptr) { ErrorGeneralOverflow (rqptr, FI_LI); FileEnd (rqptr); return; } /* absorb any remaining wildcards */ for (cptr = sptr; *cptr == '*'; cptr++); while (*cptr && sptr < zptr) *sptr++ = *cptr++; if (sptr >= zptr) { ErrorGeneralOverflow (rqptr, FI_LI); FileEnd (rqptr); return; } *sptr = '\0'; tkptr->FileNameLength = sptr - tkptr->FileName; if (rqptr->rqPathSet.AcceptLangTypeVariant) { /* (re)determine the (possibly) new content-type */ while (sptr > tkptr->FileName && *sptr != '.') sptr--; ConfigContentType (&rqptr->rqContentInfo, sptr); tkptr->ContentTypePtr = rqptr->rqContentInfo.ContentTypePtr; } } else { /*******************************/ /* no language variant, revert */ /*******************************/ /* eliminate any period-escaping character (would be EFS only) */ if (*(sptr = cptr = tkptr->AcceptLangVariantPtr) == '^') cptr++; /* eliminate the variant delimiting character and the wildcard(s) */ cptr++; while (*cptr && *cptr == '*') cptr++; while (*cptr) *sptr++ = *cptr++; *sptr = '\0'; tkptr->FileNameLength = sptr - tkptr->FileName; } if (WATCHING (rqptr, WATCH_RESPONSE)) WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "LANGUAGE !AZ", tkptr->FileName); /*************************/ /* begin processing file */ /*************************/ /* generate a hash representing the file name being accessed */ Md5Digest (tkptr->FileName, tkptr->FileNameLength, &tkptr->Md5Hash); if (WATCHING (rqptr, WATCH_RESPONSE)) WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "FILE !16&H !AZ", &tkptr->Md5Hash, tkptr->FileName); /* no file name supplied indicates only searching the cache */ if (tkptr->AuthorizePath && tkptr->FileNameLength) { /***********************/ /* check authorization */ /***********************/ cptr = MapVmsPath (tkptr->FileName, rqptr); if (!*cptr) { /* MAPURL errors are returned with a leading null (historical ;^) */ ErrorGeneral (rqptr, cptr+1, FI_LI); FileEnd (rqptr); return; } Authorize (rqptr, cptr, -1, NULL, 0, &FileAuthorizationAst); if (VMSnok (rqptr->rqAuth.FinalStatus)) { /* if asynchronous authentication is not underway */ if (rqptr->rqAuth.FinalStatus != AUTH_PENDING) FileEnd (rqptr); return; } } /* not to-be-authorized, or authorized ... just carry on regardless! */ FileAuthorizationAst (rqptr); } /*****************************************************************************/ /* This function provides an AST target is Authorize()ation ended up being done asynchronously, otherwise it is just called directly to continue the modules processing. */ FileAuthorizationAst (REQUEST_STRUCT *rqptr) { int status; char *cptr; FILE_TASK *tkptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileAuthorizationAst() !&F !&S", &FileAuthorizationAst, rqptr->rqAuth.FinalStatus); /* may have been delivered asynchronously */ rqptr->rqAuth.AstFunction = NULL; if (VMSnok (rqptr->rqAuth.FinalStatus)) { FileEnd (rqptr); return; } tkptr = rqptr->FileTaskPtr; if (!tkptr->ContentTypePtr) tkptr->ContentTypePtr = ConfigContentTypeUnknown; if (ConfigSameContentType (tkptr->ContentTypePtr, ConfigContentTypeUnknown, -1)) { /* if the content-type is not known then use the default */ if (Config.cfContent.ContentTypeDefaultPtr[0]) tkptr->ContentTypePtr = Config.cfContent.ContentTypeDefaultPtr; else tkptr->ContentTypePtr = ConfigDefaultFileContentType; } if (!rqptr->rqCache.DoNotCache) { status = CacheSearch (rqptr); if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "CacheSearch() !&S", status); /* success status indicates the file is being supplied from cache */ if (VMSok (status)) return; /* no file name supplied indicates only searching the cache */ if (!tkptr->FileNameLength) { FileEnd (rqptr); return; } } FileParse (rqptr); } /*****************************************************************************/ /* Parse the file task's file name. */ FileParse (REQUEST_STRUCT *rqptr) { FILE_TASK *tkptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileParse() !&F", &FileParse); tkptr = rqptr->FileTaskPtr; AuthAccessEnable (rqptr, tkptr->FileName, AUTH_ACCESS_READ); OdsParse (&tkptr->FileOds, tkptr->FileName, tkptr->FileNameLength, ".", 1, 0, &FileParseAst, rqptr); AuthAccessEnable (rqptr, 0, 0); } /*****************************************************************************/ /* AST when FileParse() asynchronous parsecompletes. If status OK set up and queue an ACP QIO to get file size and revision date/time, ASTing to FileAcpInfoAst(). */ FileParseAst (REQUEST_STRUCT *rqptr) { int status; char *cptr; FILE_TASK *tkptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileParseAst() !&F sts:!&S stv:!&S", &FileParseAst, rqptr->FileTaskPtr->FileOds.Fab.fab$l_sts, rqptr->FileTaskPtr->FileOds.Fab.fab$l_stv); #if WATCH_MOD HttpdCheckPriv (FI_LI); #endif /* WATCH_MOD */ tkptr = rqptr->FileTaskPtr; if (VMSok (status = tkptr->FileOds.Fab.fab$l_sts)) { /**************/ /* exclusions */ /**************/ if (MATCH5 (tkptr->FileOds.NamTypePtr, ".DIR;") || MATCH5 (tkptr->FileOds.NamTypePtr, ".dir;")) { /* no sneaky getting directory contents by downloading files! */ status = OdsReallyADir (rqptr, &tkptr->FileOds); if (VMSok (status)) status = SS$_NOPRIV; } else if (rqptr->RemoteUser[0] && strsame (tkptr->FileOds.NamTypePtr, HTL_FILE_TYPE, sizeof(HTL_FILE_TYPE)-1) && *(tkptr->FileOds.NamTypePtr+sizeof(HTL_FILE_TYPE)-1) == ';') { /* access to an HTL is OK *IF* it is authorized!! (for admin) */ tkptr->ContentTypePtr = "text/plain"; } else if ((strsame (tkptr->FileOds.NamTypePtr, HTA_FILE_TYPE, sizeof(HTA_FILE_TYPE)-1) && *(tkptr->FileOds.NamTypePtr+sizeof(HTA_FILE_TYPE)-1) == ';') || (strsame (tkptr->FileOds.NamTypePtr, HTL_FILE_TYPE, sizeof(HTL_FILE_TYPE)-1) && *(tkptr->FileOds.NamTypePtr+sizeof(HTL_FILE_TYPE)-1) == ';')) { /* attempt to retrieve an HTA/HTL authorization file, scotch that! */ status = SS$_NOSUCHFILE; } else if (DavMetaFile (&tkptr->FileOds)) status = SS$_NOSUCHFILE; } /* if a wildcard the ACP function will return the first matching file! */ if (tkptr->FileOds.Nam_fnb & NAM$M_WILDCARD) status = RMS$_WLD; if (status == SS$_ABORT && (MATCH5 (tkptr->FileOds.NamTypePtr, ".DIR;") || MATCH5 (tkptr->FileOds.NamTypePtr, ".dir;"))) status = SS$_NORMAL; if (VMSok (status)) { if (WebDavEnabled) { /* WebDAV metadata (sub)directory? */ if (rqptr->rqPathSet.WebDavRead || rqptr->rqPathSet.WebDavWrite || rqptr->rqPathSet.WebDavProfile || rqptr->rqPathSet.WebDavServer) if (DavMetaDir (rqptr, &tkptr->FileOds)) status = RMS$_DNF; } } if (VMSnok (status)) { /*****************/ /* error parsing */ /*****************/ /* if its a search list treat directory not found as if file not found */ if ((tkptr->FileOds.Nam_fnb & NAM$M_SEARCH_LIST) && status == RMS$_DNF) status = RMS$_FNF; if (tkptr->NoSuchFileFunction) { if (WATCHING (rqptr, WATCH_RESPONSE)) WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "FILE !&S", status); rqptr->HomePageStatus = status; FileEnd (rqptr); return; } rqptr->rqResponse.ErrorTextPtr = MapVmsPath (tkptr->FileName, rqptr); rqptr->rqResponse.ErrorOtherTextPtr = tkptr->FileName; ErrorVmsStatus (rqptr, status, FI_LI); FileEnd (rqptr); return; } if (tkptr->FileOds.Nam_fnb & NAM$M_SEARCH_LIST && !tkptr->SearchListCount++) { /*******************************/ /* search to get actual device */ /*******************************/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "SEARCH-LIST"); #ifndef ODS_EXTENDED /* for no-conceal search to work */ OdsParse (&tkptr->FileOds, tkptr->FileName, tkptr->FileNameLength, ".", 1, NAM$M_NOCONCEAL, NULL, 0); #endif AuthAccessEnable (rqptr, tkptr->FileName, AUTH_ACCESS_READ); OdsSearchNoConceal (&tkptr->FileOds, &FileParseAst, rqptr); AuthAccessEnable (rqptr, 0, 0); return; } /************/ /* ACP info */ /************/ AuthAccessEnable (rqptr, tkptr->FileName, AUTH_ACCESS_READ); OdsFileAcpInfo (&tkptr->FileOds, &FileAcpInfoAst, rqptr); AuthAccessEnable (rqptr, 0, 0); } /****************************************************************************/ /* AST called from FileParseAst() when ACP QIO completes. If status indicates no such file then call any file open error processing function originally supplied, otherwise report the error. If status OK 'open' the file. */ FileAcpInfoAst (REQUEST_STRUCT *rqptr) { static $DESCRIPTOR (FibDsc, ""); BOOL RangeValid; int idx, status, BufferSize; char *cptr, *sptr, *zptr, *rfmptr; FILE_CONTENT *fcptr; FILE_QIO *fqptr; FILE_TASK *tkptr; RANGE_BYTE *rbptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileAcpInfoAst() !&F !&S", &FileAcpInfoAst, rqptr->FileTaskPtr->FileOds.FileQio.IOsb.Status); #if WATCH_MOD HttpdCheckPriv (FI_LI); #endif /* WATCH_MOD */ tkptr = rqptr->FileTaskPtr; fqptr = &tkptr->FileOds.FileQio; if ((status = fqptr->IOsb.Status) == SS$_NOSUCHFILE) status = RMS$_FNF; if (status == RMS$_FNF) { /*************************/ /* file extension kludge */ /*************************/ zptr = (cptr = rqptr->rqHeader.PathInfoPtr) + rqptr->rqHeader.PathInfoLength; while (zptr > cptr && *zptr != '$' && *zptr != '/') zptr--; if (zptr > cptr && *(USHORTPTR)zptr == '$.') { /* hit what looks like the sentinal */ while (zptr > cptr && *zptr != '.' && *zptr != '/') zptr--; if (zptr > cptr && *zptr == '.') { /* a second period (and perhaps file name extension) */ while (*zptr != '$') zptr++; ResponseLocation (rqptr, cptr, zptr-cptr); FileEnd (rqptr); return; } } } if (status == RMS$_FNF && tkptr->NoSuchFileFunction) { if (WATCHING (rqptr, WATCH_RESPONSE)) WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "FILE !&S", status); rqptr->HomePageStatus = status; FileEnd (rqptr); return; } else { /* the last point at which no-such-file could have been reported */ tkptr->NoSuchFileFunction = NULL; /* if it's a directory listing README then ignore NOPRIV */ if (tkptr->NextTaskFunction == &DirHeading && status == SS$_NOPRIV) status = SS$_NORMAL; if (VMSnok (status)) { if (!rqptr->AccountingDone++) InstanceGblSecIncrLong (&AccountingPtr->DoFileCount); rqptr->rqResponse.ErrorTextPtr = MapVmsPath (tkptr->FileName, rqptr); rqptr->rqResponse.ErrorOtherTextPtr = tkptr->FileName; ErrorVmsStatus (rqptr, status, FI_LI); FileEnd (rqptr); return; } } /***************/ /* file exists */ /***************/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "did:!UL,!UL,!UL fid:!UL,!UL,!UL {!UL}!-!#AZ", fqptr->Fib.fib$w_did[0], fqptr->Fib.fib$w_did[1], fqptr->Fib.fib$w_did[2], fqptr->Fib.fib$w_fid[0], fqptr->Fib.fib$w_fid[1], fqptr->Fib.fib$w_fid[2], fqptr->FileNameDsc.dsc$w_length, fqptr->FileNameDsc.dsc$a_pointer); fqptr->AllocatedVbn = ((fqptr->RecAttr.fat$l_hiblk & 0xffff) << 16) | ((fqptr->RecAttr.fat$l_hiblk & 0xffff0000) >> 16); fqptr->EndOfFileVbn = ((fqptr->RecAttr.fat$l_efblk & 0xffff) << 16) | ((fqptr->RecAttr.fat$l_efblk & 0xffff0000) >> 16); fqptr->FirstFreeByte = fqptr->RecAttr.fat$w_ffbyte; if ((uint)fqptr->EndOfFileVbn <= 1) fqptr->SizeInBytes64 = (uint64)fqptr->FirstFreeByte; else { fqptr->SizeInBytes64 = (uint64)fqptr->EndOfFileVbn - 1; fqptr->SizeInBytes64 *= 512; fqptr->SizeInBytes64 += fqptr->FirstFreeByte; } if (WATCHING (rqptr, WATCH_RESPONSE)) { switch (fqptr->RecAttr.fat$b_rtype) { case FAT$C_VARIABLE : rfmptr = "VAR"; break; case FAT$C_VFC : rfmptr = "VFC"; break; case FAT$C_FIXED : rfmptr = "FIX"; break; case FAT$C_STREAM : rfmptr = "STM"; break; case FAT$C_STMLF : rfmptr = "STMLF"; break; case FAT$C_STMCR : rfmptr = "STMCR"; break; case FAT$C_UNDEFINED : rfmptr = "UDF"; break; default : rfmptr = "?"; } WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, #ifdef ODS_EXTENDED "FILE !AZ ODS:!UL rfm:!AZ ebk:!UL ffb:!UL (!AZ!@UQ bytes!AZ) rdt:!%D", tkptr->FileName, rqptr->PathOds, #else "FILE !AZ rfm:!AZ ebk:!UL ffb:!UL (!AZ!@UQ bytes!AZ) rdt:!%D", tkptr->FileName, #endif /* ODS_EXTENDED */ rfmptr, fqptr->EndOfFileVbn, fqptr->FirstFreeByte, fqptr->RecAttr.fat$b_rtype == FAT$C_VARIABLE || fqptr->RecAttr.fat$b_rtype == FAT$C_VFC ? "~" : "", &fqptr->SizeInBytes64, fqptr->SizeInBytes64 > __UINT32_MAX ? " >4GB!" : "", &fqptr->RdtTime64); } if (rqptr->rqPathSet.CharsetPtr) { cptr = tkptr->FileOds.ResFileName; if (!*cptr) cptr = tkptr->FileOds.ExpFileName; rqptr->rqPathSet.CharsetPtr = FileSetCharset (rqptr, cptr); } if (rqptr->rqHeader.RangeBytePtr && rqptr->rqHeader.RangeBytePtr->Total && !rqptr->rqResponse.HeaderGenerated && !tkptr->ContentHandlerFunction && fqptr->RecAttr.fat$b_rtype != FAT$C_VARIABLE && fqptr->RecAttr.fat$b_rtype != FAT$C_VFC) { /******************************/ /* byte-range on non-VAR file */ /******************************/ RangeValid = true; rbptr = rqptr->rqHeader.RangeBytePtr; for (idx = 0; idx < rbptr->Total; idx++) { if (!rbptr->Last[idx]) { /* last byte not specified, set at EOF */ rbptr->Last[idx] = fqptr->SizeInBytes64 - 1; } else if (rbptr->Last[idx] < 0) { /* first byte a negative offset from end, last byte at EOF */ rbptr->First[idx] = fqptr->SizeInBytes64 + rbptr->Last[idx]; if (rbptr->First[idx] < 0) rbptr->First[idx] = 0; rbptr->Last[idx] = fqptr->SizeInBytes64 - 1; } else if (rbptr->Last[idx] >= fqptr->SizeInBytes64) { /* if the last byte is ambit make it at the EOF */ rbptr->Last[idx] = fqptr->SizeInBytes64 - 1; } /* if the range still does not make sense then back out now */ if (rbptr->Last[idx] <= rbptr->First[idx]) { RangeValid = false; rbptr->Length = 0; } else rbptr->Length = rbptr->Last[idx] - rbptr->First[idx] + 1; if (WATCHING (rqptr, WATCH_RESPONSE)) WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "RANGE !UL !@UQ-@UQ !@UQ byte!%s!&? INVALID\r\r", idx+1, &rbptr->First[idx], &rbptr->Last[idx], &rbptr->Length, !rbptr->Length); } if (RangeValid) rbptr->Count = idx; } if (Config.cfMisc.EntityTag) { FileGenerateEntityTag (tkptr->EntityTag, fqptr); if (rqptr->rqResponse.HttpVersion == HTTP_VERSION_1_1) strzcpy (rqptr->rqResponse.EntityTag, tkptr->EntityTag, sizeof(rqptr->rqResponse.EntityTag)); } if (!rqptr->rqResponse.HeaderGenerated && !tkptr->ContentHandlerFunction) { /**************************/ /* full response required */ /**************************/ if (!rqptr->AccountingDone++) InstanceGblSecIncrLong (&AccountingPtr->DoFileCount); /* variable-record length file size cannot be accurately determined */ if (fqptr->RecAttr.fat$b_rtype == FAT$C_VARIABLE || fqptr->RecAttr.fat$b_rtype == FAT$C_VFC) status = FileResponseHeader (rqptr, tkptr->ContentTypePtr, -1, &fqptr->RdtTime64); else status = FileResponseHeader (rqptr, tkptr->ContentTypePtr, fqptr->SizeInBytes64, &fqptr->RdtTime64); /* here status is a boolean */ if (!status) { InstanceGblSecIncrLong (&AccountingPtr->DoFileNotModifiedCount); FileEnd (rqptr); return; } /* quit here if the HTTP method was HEAD */ if (rqptr->rqHeader.Method == HTTP_METHOD_HEAD) { FileEnd (rqptr); return; } } /******************************/ /* stream-LF file conversion? */ /******************************/ if (Config.cfMisc.StreamLfConversionMaxKbytes && (fqptr->RecAttr.fat$b_rtype == FAT$C_VARIABLE || fqptr->RecAttr.fat$b_rtype == FAT$C_VFC) && rqptr->rqPathSet.StmLF && ConfigSameContentType (tkptr->ContentTypePtr, "text/", 5) && !ConfigSameContentType (tkptr->ContentTypePtr, ConfigContentTypeSsi, -1)) { /* divide by two to get the number of kilobytes (1024) in the file */ if (!fqptr->SizeInBytes64 && (fqptr->SizeInBytes64 >> 10) <= Config.cfMisc.StreamLfConversionMaxKbytes) StmLfBegin (MapVmsPath(tkptr->FileName, rqptr), tkptr->FileName, tkptr->FileNameLength, rqptr->PathOdsExtended, rqptr->rqAuth.VmsUserProfileLength); } /********************/ /* begin processing */ /********************/ /* Cache load is not initiated if this is not a stand-alone file request (i.e. not part of some other activity, e.g. a directory read-me file, if it is already being loaded via another request, if a "temporary" file ('DeleteOnClose'), or via requests with a VMS authentication profile attached. */ if (CacheEnabled && tkptr->CacheAllowed && !rqptr->rqPathSet.NoCache && !rqptr->rqPathSet.CacheNoFile && !rqptr->rqCache.LoadCheck && !rqptr->rqCache.NotUsable && !tkptr->EscapeHtml && !rqptr->DeleteOnClose && !rqptr->rqAuth.VmsUserProfileLength && (!rqptr->rqHeader.RangeBytePtr || !rqptr->rqHeader.RangeBytePtr->Count) && fqptr->SizeInBytes64 <= __INT32_MAX) { /* begin caching during file content read */ rqptr->rqCache.LoadFromFile = CacheLoadBegin (rqptr, (int)fqptr->SizeInBytes64, NULL); } if (tkptr->ContentHandlerFunction) { /************************/ /* buffer file contents */ /************************/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "!@UQ !UL", &fqptr->SizeInBytes64, tkptr->FileContentsSizeMax); if (fqptr->SizeInBytes64 > tkptr->FileContentsSizeMax) { rqptr->rqResponse.HttpStatus = 500; ErrorGeneral (rqptr, ErrorBufferFileMaxBytes, FI_LI); FileEnd (rqptr); return; } /* make buffer size a multiple of 512 byte blocks (for block I/O) */ BufferSize = (fqptr->SizeInBytes64 / 512); if (fqptr->SizeInBytes64 % 512) BufferSize++; BufferSize *= 512; /* allocate a buffer */ rqptr->FileContentPtr = fcptr = (FILE_CONTENT*) VmGetHeap (rqptr, sizeof(FILE_CONTENT) + BufferSize); fcptr->ContentSize = BufferSize; /* buffer space immediately follows the structured storage */ fcptr->ContentPtr = (char*)fcptr + sizeof(FILE_CONTENT); /* populate the file contents structure with some file data */ zptr = (sptr = fcptr->FileName) + sizeof(fcptr->FileName); for (cptr = tkptr->FileName; *cptr && sptr < zptr; *sptr++ = *cptr++); if (sptr >= zptr) { ErrorGeneralOverflow (rqptr, FI_LI); FileEnd (rqptr); return; } *sptr = '\0'; fcptr->FileNameLength = sptr - fcptr->FileName; fcptr->CdtTime64 = fqptr->CdtTime64; fcptr->RdtTime64 = fqptr->RdtTime64; fcptr->UicGroup = (fqptr->AtrUic & 0x0fff0000) >> 16; fcptr->UicMember = (fqptr->AtrUic & 0x0000ffff); fcptr->Protection = fqptr->AtrFpro; /* set the content structure handler to the supplied function */ rqptr->FileContentPtr->ContentHandlerFunction = tkptr->ContentHandlerFunction; /* none of these little tricks, just get the raw file! */ tkptr->PreTagFileContents = tkptr->EscapeHtml = false; } else { /* network writes are checked for success, fudge the first one! */ rqptr->NetIoPtr->WriteStatus = SS$_NORMAL; } /*******************/ /* 'open' the file */ /*******************/ /* Fill out the FIB, override any exclusive locking - requires SYSPRV. Access to this file without SYSPRV has already been established via the ACP QIO that effectively performs the protection checks. */ fqptr->Fib.fib$l_acctl = FIB$M_SEQONLY | FIB$M_NOLOCK; fqptr->Fib.fib$w_nmctl = FIB$M_FINDFID; sys$setprv (1, &SysPrvMask, 0, 0); status = sys$qio (EfnNoWait, fqptr->AcpChannel, IO$_ACCESS | IO$M_ACCESS, &fqptr->IOsb, &FileAccessAst, rqptr, &fqptr->FibDsc, 0, 0, 0, 0, 0); sys$setprv (0, &SysPrvMask, 0, 0); if (VMSnok (status)) { /* let the AST routine handle it! */ fqptr->IOsb.Status = status; SysDclAst (FileAccessAst, rqptr); } } /*****************************************************************************/ /* Match the supplied file name (generally needs to be a NOCONCEAL sys$search() result file name as provided by OdsSearchNoConceal() with the 'name=value' pair 'name' string of a "SET /path/* charset=(pattern,charset)" rule. If a successful match return the corresponding 'value' otherwise return NULL. */ char* FileSetCharset ( REQUEST_STRUCT *rqptr, char *FileName ) { static char Charset [64]; int status; char *cptr; char FileSpec [128]; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileSetCharset() !&Z !&Z", rqptr->rqPathSet.CharsetPtr, FileName); if (!(cptr = rqptr->rqPathSet.CharsetPtr)) return (NULL); if (cptr[0] != '(') return (cptr); if (!FileName) return (NULL); for (;;) { status = StringParseNameValue (&cptr, false, FileSpec, sizeof(FileSpec), Charset, sizeof(Charset)); if (VMSnok (status)) return (NULL); if (StringMatch (NULL, FileName, FileSpec)) return (Charset); } } /****************************************************************************/ /* Generate a response header suitable for the file being returned. This may be a 304 (not modified) if appropriate. This function is also used by the CACHE.C module. Return /true/ to continue to send the file, /false/ if not to. */ BOOL FileResponseHeader ( REQUEST_STRUCT *rqptr, char *ContentType, int64 ContentLength, ulong *RdtTime64Ptr ) { int idx, status; int64 RangeContentLength; ushort Length; char *cptr, *sptr, *zptr, *ContentTypePtr; char Buffer [256], TypeBuffer [256]; RANGE_BYTE *rbptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileResponseHeader()"); /****************/ /* entity match */ /****************/ /* if entity precondition fails (generated 412) then just return */ if (!ResponseEntityMatch (rqptr, rqptr->rqResponse.EntityTag)) return (true); if (rqptr->rqTime.IfModifiedSinceTime64 && !rqptr->rqHeader.PragmaNoCache && !rqptr->DeleteOnClose) { /*********************/ /* if modified since */ /*********************/ if (!HttpIfModifiedSince (rqptr, RdtTime64Ptr, ContentLength)) { if (WATCHING (rqptr, WATCH_RESPONSE)) WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "NOT modified"); return (false); } } if (rqptr->rqTime.IfUnModifiedSinceTime64 && !rqptr->DeleteOnClose) { /*************************/ /* if NOT modified since */ /*************************/ if (HttpIfUnModifiedSince (rqptr, RdtTime64Ptr)) { if (WATCHING (rqptr, WATCH_RESPONSE)) WatchThis (WATCHITM(rqptr), WATCH_RESPONSE, "NOT modified"); return (false); } } rbptr = rqptr->rqHeader.RangeBytePtr; if (rqptr->rqHeader.QueryStringLength && to_lower(rqptr->rqHeader.QueryStringPtr[0]) == 'h' && strsame (rqptr->rqHeader.QueryStringPtr, "httpd=content", 13)) { /* request-specified content-type (default to plain-text) */ if (rbptr) rbptr->Count = 0; /* cancel any byte range */ if (strsame (rqptr->rqHeader.QueryStringPtr+13, "&type=", 6)) { zptr = (sptr = TypeBuffer) + sizeof(TypeBuffer)-1; for (cptr = rqptr->rqHeader.QueryStringPtr + 19; *cptr && *cptr != '&' && sptr < zptr; *sptr++ = *cptr++); *sptr = '\0'; StringUrlDecode (ContentTypePtr = TypeBuffer); } else ContentTypePtr = "text/plain"; } else ContentTypePtr = ContentType; if (rbptr && rbptr->Count && rqptr->rqTime.IfRangeBeginTime64 && !rqptr->NotFromCache && !rqptr->DeleteOnClose) { /*********************/ /* if range modified */ /*********************/ /* cancels range data in the request header if file has been modified */ HttpIfRange (rqptr, RdtTime64Ptr); } if (!rbptr || !rbptr->Count) { /*********************/ /* standard response */ /*********************/ ResponseHeader (rqptr, 200, ContentTypePtr, ContentLength, RdtTime64Ptr, NULL); return (true); } /***********************/ /* byte range response */ /***********************/ if (rbptr->Count == 1) { /* single byte-range requested */ FaoToBuffer (Buffer, sizeof(Buffer), NULL, "Content-Range: bytes !@UQ-!@UQ/!@UQ\r\n", &rbptr->First[0], &rbptr->Last[0], &ContentLength); ResponseHeader (rqptr, 206, ContentTypePtr, rbptr->Last[0] - rbptr->First[0] + 1, RdtTime64Ptr, Buffer); return (true); } /* multiple byte-ranges requested */ rqptr->rqResponse.MultipartBoundaryPtr = VmGetHeap (rqptr, 32+1); FaoToBuffer (rqptr->rqResponse.MultipartBoundaryPtr, 32+1, &Length, "!32&H", &rqptr->Md5HashPath); /* calculate the content-length */ RangeContentLength = 0; for (idx = 0; idx < rbptr->Count; idx++) { FaoToBuffer (Buffer, sizeof(Buffer), &Length, "!AZ--!AZ\r\n\ Content-Type: !AZ\r\n\ Range: bytes !@UQ-!@UQ/!@UQ\r\n\ \r\n", idx ? "\r\n" : "", rqptr->rqResponse.MultipartBoundaryPtr, ContentTypePtr, &rbptr->First[idx], &rbptr->Last[idx], &ContentLength); RangeContentLength += Length + rbptr->Last[idx] - rbptr->First[idx] + 1; } RangeContentLength += 32 + 8; FaoToBuffer (Buffer, sizeof(Buffer), NULL, "multipart/byteranges; boundary=!AZ", rqptr->rqResponse.MultipartBoundaryPtr); ResponseHeader (rqptr, 206, Buffer, RangeContentLength, RdtTime64Ptr, NULL); return (true); } /*****************************************************************************/ /* End of file transfer, successful or otherwise. */ FileEnd (REQUEST_STRUCT *rqptr) { int status; FILE_QIO *fqptr; FILE_TASK *tkptr; FILE_CONTENT *fcptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileEnd() !&F !&A", &FileEnd, rqptr->FileTaskPtr); tkptr = rqptr->FileTaskPtr; fqptr = &tkptr->FileOds.FileQio; if (rqptr->DeleteOnClose && (fqptr->AcpChannel || fqptr->QioChannel)) { /* delete-on-close happens VERY infrequently */ sys$setprv (1, &SysPrvMask, 0, 0); status = sys$qiow (EfnWait, fqptr->AcpChannel ? fqptr->AcpChannel : fqptr->QioChannel, IO$_DELETE | IO$M_DELETE, &fqptr->IOsb, 0, 0, &fqptr->FibDsc, &fqptr->FileNameDsc, 0, 0, 0, 0); sys$setprv (0, &SysPrvMask, 0, 0); if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "IO$_DELETE !&S !&S", status, fqptr->IOsb.Status); if (VMSok (status)) status = fqptr->IOsb.Status; if (VMSnok (status)) ErrorNoticed (rqptr, status, NULL, FI_LI); } if (fqptr->AcpChannel) { /* the file has only had it's attributes read */ sys$dassgn (fqptr->AcpChannel); fqptr->AcpChannel = 0; } else if (fqptr->QioChannel) { /* file has been accessed (contents 'open'ed and read) */ status = sys$qiow (EfnWait, fqptr->QioChannel, IO$_DEACCESS, &fqptr->IOsb, 0, 0, &fqptr->FibDsc, 0, 0, 0, 0, 0); if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "IO$_DEACCESS !&S !&S", status, fqptr->IOsb.Status); if (fqptr->IOsb.Status && VMSnok (fqptr->IOsb.Status)) ErrorNoticed (rqptr, fqptr->IOsb.Status, NULL, FI_LI); if (VMSnok (status)) ErrorNoticed (rqptr, status, NULL, FI_LI); sys$dassgn (fqptr->QioChannel); fqptr->QioChannel = 0; } /* if the file was being cached at the same time */ if (rqptr->rqCache.LoadFromFile) { if (rqptr->FileContentPtr) { /* copy from the cache buffer */ fcptr = rqptr->FileContentPtr; if (fcptr->ContentLength <= rqptr->rqCache.ContentLength) { memcpy (fcptr->ContentPtr, rqptr->rqCache.ContentPtr, rqptr->rqCache.ContentLength); fcptr->ContentLength = rqptr->rqCache.ContentLength; /* null terminate, it's usually text! */ fcptr->ContentPtr[fcptr->ContentLength] = '\0'; } } CacheLoadEnd (rqptr); } /* release internal RMS parse structures */ OdsParseRelease (&tkptr->FileOds); #if WATCH_MOD HttpdCheckPriv (FI_LI); #endif /* WATCH_MOD */ if (tkptr->NoSuchFileFunction) { /* file could not have been found, declare the appropriate handler */ SysDclAst (tkptr->NoSuchFileFunction, rqptr); } else if (rqptr->FileContentPtr) { /* next task gets control once the file has been content-handled */ rqptr->FileContentPtr->NextTaskFunction = tkptr->NextTaskFunction; /* file contents loaded, now process using the specified handler */ SysDclAst (rqptr->FileContentPtr->ContentHandlerFunction, rqptr); rqptr->FileContentPtr->ContentHandlerFunction = NULL; } else if (tkptr->PreTagEndFileContents) { /* success, encapsulating a file, add the end tag */ NetWriteBuffered (rqptr, tkptr->NextTaskFunction, "\n", 7); } else { /* success, declare the next task */ SysDclAst (tkptr->NextTaskFunction, rqptr); } rqptr->FileTaskPtr = NULL; VmFreeFromHeap (rqptr, tkptr, FI_LI); } /*****************************************************************************/ /* The file access $QIO (the 'open') has completed. Check status. Ensure any output buffer contents are flushed. */ FileAccessAst (REQUEST_STRUCT *rqptr) { int status; FILE_QIO *fqptr; FILE_TASK *tkptr; /*********/ /* begin */ /*********/ if (WATCHMOD (rqptr, WATCH_MOD_FILE)) WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileAccessAst() !&F !UL !UL !&S", &FileAccessAst, STR_DSC_LEN(&rqptr->NetWriteBufferDsc), rqptr->FileTaskPtr->FileOds.FileQio.IOsb.Count, rqptr->FileTaskPtr->FileOds.FileQio.IOsb.Status); tkptr = rqptr->FileTaskPtr; fqptr = &tkptr->FileOds.FileQio; if (VMSnok (fqptr->IOsb.Status)) { if (rqptr->rqCache.LoadFromFile) rqptr->rqCache.LoadStatus = fqptr->IOsb.Status; rqptr->rqResponse.ErrorTextPtr = MapVmsPath (tkptr->FileName, rqptr); rqptr->rqResponse.ErrorOtherTextPtr = tkptr->FileName; ErrorVmsStatus (rqptr, fqptr->IOsb.Status, FI_LI); FileEnd (rqptr); return; } /* the file access is on the channel originally assigned for ACP-QIO */ fqptr->QioChannel = fqptr->AcpChannel; fqptr->AcpChannel = 0; if (tkptr->PreTagFileContents) { tkptr->PreTagEndFileContents = true; NetWriteBuffered (rqptr, FileNextBlocks, "
", 5);
      return;
   }

   FileNextBlocks (rqptr);
}

/*****************************************************************************/
/*
QIO a read of blocks from the file.  When the read completes call
FileNextBlocksAst() function to post-process the read and/or send the data to
the client.  Don't bother to test any status here, the AST routine will do
that!
*/ 

FileNextBlocks (REQUEST_STRUCT *rqptr)

{
   int  idx, status;
   ushort  Length;
   uint  BufferSize;
   ulong  EmulBlock;
   char  *bptr;
   FILE_CONTENT  *fcptr;
   FILE_QIO  *fqptr;
   FILE_TASK  *tkptr;
   RANGE_BYTE  *rbptr;

   /*********/
   /* begin */
   /*********/

   if (WATCHMOD (rqptr, WATCH_MOD_FILE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE,
                "FileNextBlocks() !&F !&S !UL !&B",
                 FileNextBlocks, rqptr->NetIoPtr->WriteStatus,
                 rqptr->FileTaskPtr->FileOds.FileQio.BlockNumber,
                 rqptr->FileTaskPtr->FileOds.FileQio.EndOfFile);

   if (rqptr->RequestState >= REQUEST_STATE_ABORT)
      rqptr->NetIoPtr->WriteStatus = SS$_ABORT;

   tkptr = rqptr->FileTaskPtr;
   fcptr = rqptr->FileContentPtr;
   fqptr = &tkptr->FileOds.FileQio;

   if (!fcptr && VMSnok (rqptr->NetIoPtr->WriteStatus))
   {
      /* network write has failed (delivered via AST), bail out now */
      FileEnd (rqptr);
      return;
   }

   if (fqptr->EndOfFile)
   {
      /* calculated EOF */
      fqptr->EndOfFile = false;
      fqptr->IOsb.Status = SS$_ENDOFFILE;
      SysDclAst (FileNextBlocksAst, rqptr);
      return;
   }

   if (!fqptr->BlockNumber)
   {
      /**************/
      /* initialize */
      /**************/

      if (STR_DSC_LEN(&rqptr->NetWriteBufferDsc))
      {
         /* need exclusive use, flush the current contents */
         NetWriteFullFlush (rqptr, FileNextBlocks);
         return;
      }

      /* ensure we have an initialised buffer (round number of TCP segments) */
      BufferSize = (rqptr->NetIoPtr->TcpMaxQio / 512) * 512;
      if (WATCHING (rqptr, WATCH_NETWORK))
         WatchThis (WATCHITM(rqptr), WATCH_NETWORK, "TCP mss:!UL QIO size:!UL",
                    rqptr->NetIoPtr->TcpMaxQio, BufferSize);
      StrDscIfNotBegin (rqptr, &rqptr->NetWriteBufferDsc, BufferSize);

      fqptr->BlockNumber = 1;

      if (rqptr->rqHeader.RangeBytePtr &&
          rqptr->rqHeader.RangeBytePtr->Count)
      {
         /* returning a byte range within the file (partial content) */
         rbptr = rqptr->rqHeader.RangeBytePtr;
         idx = rbptr->Index;
         fqptr->BlockNumber += rbptr->First[idx] / 512;
         rbptr->Offset = rbptr->First[idx] % 512;
         rbptr->Length = rbptr->Last[idx] - rbptr->First[idx] + 1;
         rbptr->Remaining = rbptr->Length;

         if (rbptr->Count > 1)
         {
            /* returning 'multipart/byteranges' range content */
            char Buffer [256];
            FaoToBuffer (Buffer, sizeof(Buffer), &Length,
"!AZ--!AZ\r\n\
Content-Type: !AZ\r\n\
Range: bytes !@UQ-!@UQ/!@UQ\r\n\
\r\n",
                         rbptr->Index ? "\r\n" : "",
                         rqptr->rqResponse.MultipartBoundaryPtr,
                         tkptr->ContentTypePtr,
                         &rbptr->First[idx], &rbptr->Last[idx],
                         &fqptr->SizeInBytes64);
            /* synchronous network write (just for the convenience of it!) */
            NetWrite (rqptr, NULL, Buffer, Length);
         }

         if (WATCHMOD (rqptr, WATCH_MOD_FILE))
            WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE,
                       "range !UL-!UL(!UL) vbn:!UL off:!UL rem:!UL",
                       rbptr->First[idx], rbptr->Last[idx],
                       rbptr->Length, fqptr->BlockNumber,
                       rbptr->Offset, rbptr->Remaining);
      }

      if (fcptr)
      {
         /* file content buffer (perhaps concurrently with cache buffer) */
         fcptr->CurrentPtr = fcptr->ContentPtr;
         fcptr->ContentRemaining = fcptr->ContentSize;
         fcptr->ContentLength = 0;
      }
      else
      if (!rqptr->rqCache.LoadFromFile)
      {
         /* initialize output buffer (round number of TCP segments) */
         BufferSize = (rqptr->NetIoPtr->TcpMaxQio / 512) * 512;
         if (WATCHMOD (rqptr, WATCH_MOD_FILE))
            WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "2 !@UQ !UL",
                       &fqptr->SizeInBytes64, BufferSize);
         StrDscBegin (rqptr, &rqptr->NetWriteBufferDsc, BufferSize);
      }

      if (!fqptr->SizeInBytes64)
      {
         /* empty file */
         fqptr->IOsb.Status = SS$_ENDOFFILE;
         SysDclAst (FileNextBlocksAst, rqptr);
         return;
      }
   }
   else
   {
      /********************/
      /* subsequent reads */
      /********************/

      fqptr->BlockNumber += fqptr->BufferSize >> 9;
   }

   /**************/
   /* queue read */
   /**************/

   if (rqptr->rqCache.LoadFromFile)
   {
      /* populating a cache buffer, load it progressively */
      fqptr->BufferPtr = rqptr->rqCache.CurrentPtr;
      if (rqptr->rqCache.ContentRemaining > 0xfe00)
         BufferSize = 0xfe00;
      else
      {
         BufferSize = rqptr->rqCache.ContentRemaining;
         fqptr->EndOfFile = true;
      }
   }
   else
   if (fcptr)
   {
      /* populating a file contents buffer, load it progressively */
      fqptr->BufferPtr = fcptr->CurrentPtr;
      if (fcptr->ContentRemaining > 0xfe00)
         BufferSize = 0xfe00;
      else
      {
         BufferSize = fcptr->ContentRemaining;
         fqptr->EndOfFile = true;
      }
   }
   else
   {
      /* standard output buffer, make it a round number of 512 byte blocks */
      BufferSize = STR_DSC_SIZE(&rqptr->NetWriteBufferDsc) & 0xfe00;
      fqptr->BufferPtr = STR_DSC_PTR(&rqptr->NetWriteBufferDsc);
   }

   /* adjust the buffer size for where we have been told the EOF to be */
   if (((fqptr->BlockNumber-1)<<9) + BufferSize > fqptr->SizeInBytes64)
   {
      BufferSize = fqptr->SizeInBytes64 - ((fqptr->BlockNumber-1)<<9);
      fqptr->EndOfFile = true;
   }
   fqptr->BufferSize = BufferSize;

   if (WATCHMOD (rqptr, WATCH_MOD_FILE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE,
                 "BufferSize:!UL", BufferSize);

   /*
      Documented in the VMS I/O Users Guide section entitled "Disk Function
      Codes" ... "P2--The number of bytes that are to be read from the disk,
      or written from memory to the disk. An even number must be specified if
      the controller is an RK611, RL11, RX211, or UDA50".
      Well, make sure we ask for an even number of bytes!
      Thanks to Dave Holland for demonstrating this bug on SIMH, and to
      Mark Pizzolato from the SIMH mailing list for pointing out the above.
   */
   if (BufferSize & 1)
   {
      BufferSize++;
      fqptr->AdjustBuffer = 1;
   }
   else
      fqptr->AdjustBuffer = 0;

   bptr = fqptr->BufferPtr;
   if (fqptr->RecAttr.fat$b_rtype == FAT$C_VARIABLE ||
       fqptr->RecAttr.fat$b_rtype == FAT$C_VFC)
   {
      /*
         Records in VARiable format files need to be word aligned.
         If this is on an odd boundary then push the buffer forward one
         byte in the storage ('elbow-room' in memory allocation means this
         is not an issue).  FileVariableRecord() detects and undoes this.
         This should only be necessary when when using a cache or content
         buffer as the FileVariableRecord() processing can create odd
         numbers of bytes when turning them into stream content.
      */
      if ((int)bptr & 1) bptr++;
   }

   status = sys$qio (EfnNoWait, fqptr->QioChannel,
                     IO$_READVBLK, &fqptr->IOsb,
                     FileNextBlocksAst, rqptr,
                     bptr, BufferSize, fqptr->BlockNumber,
                     0, 0, 0);
   if (VMSok (status)) return;

   /* let the AST routine handle it! */
   fqptr->IOsb.Status = status;
   SysDclAst (FileNextBlocksAst, rqptr);
}

/*****************************************************************************/
/*
The QIO read of blocks from the file has completed.  Post-process and/or queue
a network write to the client.  When the network  write completes it will call
the function FileNextBlocks() to queue a read  of the next series of blocks.
*/ 

FileNextBlocksAst (REQUEST_STRUCT *rqptr)

{
   int  status, bcnt;
   char  *bptr, *cptr, *sptr, *zptr;
   FILE_CONTENT  *fcptr;
   FILE_TASK  *tkptr;
   FILE_QIO  *fqptr;
   RANGE_BYTE  *rbptr;

   /*********/
   /* begin */
   /*********/

   if (WATCHMOD (rqptr, WATCH_MOD_FILE))
   {
      WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE,
"FileNextBlocksAst() !&F AcpIOsb.Status:!&S \
QioBlockNumber:!UL QioBufferSize:!UL IOsb.Count:!UL end:!&B %!&M",
                 FileNextBlocksAst,
                 rqptr->FileTaskPtr->FileOds.FileQio.IOsb.Status,
                 rqptr->FileTaskPtr->FileOds.FileQio.BlockNumber,
                 rqptr->FileTaskPtr->FileOds.FileQio.BufferSize,
                 rqptr->FileTaskPtr->FileOds.FileQio.IOsb.Count,
                 rqptr->FileTaskPtr->FileOds.FileQio.EndOfFile,
                 rqptr->FileTaskPtr->FileOds.FileQio.IOsb.Status);
   }

   tkptr = rqptr->FileTaskPtr;
   fcptr = rqptr->FileContentPtr;
   fqptr = &tkptr->FileOds.FileQio;

   if (rqptr->RequestState >= REQUEST_STATE_ABORT)
      tkptr->FileOds.FileQio.IOsb.Status = SS$_ABORT;

   tkptr->NextBlocksTickSecond = HttpdTickSecond;

   if (VMSnok (fqptr->IOsb.Status))
   {
      if (fqptr->IOsb.Status == SS$_ENDOFFILE)
      {
         /***************/
         /* end-of-file */
         /***************/

         if (rqptr->rqHeader.RangeBytePtr &&
             rqptr->rqHeader.RangeBytePtr->Count)
         {
            /* transfering byte-range(s) */
            rqptr->rqHeader.RangeBytePtr->Index++;
            if (rqptr->rqHeader.RangeBytePtr->Index <
                rqptr->rqHeader.RangeBytePtr->Count)
            {
               /* multiple byte ranges, restart with next range */
               fqptr->BlockNumber = 0;
               SysDclAst (FileNextBlocks, rqptr);
               return;
            }
            else
            if (rqptr->rqHeader.RangeBytePtr->Count > 1)
            {
               /* end of multiple byte ranges, provide final boundary */
               char Buffer [64];
               zptr = (sptr = Buffer) + sizeof(Buffer)-1;
               for (cptr = "\r\n--"; *cptr && sptr < zptr; *sptr++ = *cptr++);
               for (cptr = rqptr->rqResponse.MultipartBoundaryPtr;
                    *cptr && sptr < zptr;
                    *sptr++ = *cptr++);
               for (cptr = "--\r\n"; *cptr && sptr < zptr; *sptr++ = *cptr++);
               *sptr = '\0';
               /* synchronous network write (for the convenience of it!) */
               NetWrite (rqptr, NULL, Buffer, sptr-Buffer);
            }
         }
         else
         if (rqptr->rqCache.LoadFromFile)
            rqptr->rqCache.LoadStatus = fqptr->IOsb.Status;

         if (!fcptr)
         {
            /* reset the standard output buffer */
            StrDscNoContent (&rqptr->NetWriteBufferDsc);
         }

         FileEnd (rqptr);
         return;
      }

      /**************/
      /* read error */
      /**************/

      if (rqptr->rqCache.LoadFromFile)
         rqptr->rqCache.LoadStatus = fqptr->IOsb.Status;
      rqptr->rqResponse.ErrorTextPtr = MapVmsPath (tkptr->FileName, rqptr);
      rqptr->rqResponse.ErrorOtherTextPtr = tkptr->FileName;
      ErrorVmsStatus (rqptr, fqptr->IOsb.Status, FI_LI);
      FileEnd (rqptr);
      return;
   }

   /******************/
   /* process blocks */
   /******************/

   /* get the count from the I/O status block (adjusted as necessary) */
   fqptr->IOsb.Count -= fqptr->AdjustBuffer;
   fqptr->BufferCount = fqptr->IOsb.Count;

   /* if NOT transferring the content EXACTLY as it is on-disk */
   if (rqptr->rqPathSet.ResponseVarRecord != FILE_VAR_ASIS)
   {
      /* if not 'binary content' in the blocks then massage */
      if (fqptr->RecAttr.fat$b_rtype == FAT$C_VARIABLE ||
          fqptr->RecAttr.fat$b_rtype == FAT$C_VFC)
         FileVariableRecord (rqptr);
      else
      if (fqptr->RecAttr.fat$b_rtype == FAT$C_FIXED &&
          fqptr->RecAttr.fat$b_rattrib & FAT$M_NOSPAN)
         FileFixedRecordNoSpan (rqptr);
   }

   /* ensure leftover space in a reused buffer is zeroed (just to be tidy) */
   if (fqptr->BlockNumber > 1)
      if (fqptr->BufferSize - fqptr->BufferCount > 0)
         memset (fqptr->BufferPtr + fqptr->BufferCount, 0,
                 fqptr->BufferSize - fqptr->BufferCount);

   if (rqptr->FileTaskPtr->EscapeHtml)
   {
      /* queue a network write to the client, AST to FileNextBlocks() */
      STR_DSC_LEN(&rqptr->NetWriteBufferDsc) = fqptr->BufferCount;

      FileWriteBufferEscapeHtml (rqptr, FileNextBlocks);
      return;
   }

   bptr = fqptr->BufferPtr;
   bcnt = fqptr->BufferCount;

   if (rqptr->rqCache.LoadFromFile)
   {
      /* populating a cache buffer, possibly also a file contents buffer */
      rqptr->rqCache.ContentRemaining -= bcnt;
      rqptr->rqCache.ContentLength += bcnt;
      rqptr->rqCache.CurrentPtr += bcnt;
      if (WATCHMOD (rqptr, WATCH_MOD_FILE))
         WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "!UL",
                    rqptr->rqCache.ContentRemaining);
      if (!rqptr->rqCache.ContentRemaining)
         fqptr->EndOfFile = true;
      else
      if (rqptr->rqCache.ContentRemaining < 0)
      {
         /*
            This is not supposed to happen!
            Possibly in between getting the file size and opening and
            reading this far the file has been extended or rewritten.
            Shouldn't happen often enough to be a worry!
         */
         rqptr->rqResponse.ErrorTextPtr = MapVmsPath (tkptr->FileName, rqptr);
         rqptr->rqResponse.ErrorOtherTextPtr = tkptr->FileName;
         ErrorVmsStatus (rqptr, RMS$_RTB, FI_LI);
         FileEnd (rqptr);
         return;
      }
      if (!fcptr)
      {
         /* not populating file content buffer, write cache buffer to client */
         NetWrite (rqptr, FileNextBlocks, bptr, bcnt);
         return;
      }
      /* filling the cache/contents buffer, just get more file data */
      SysDclAst (FileNextBlocks, rqptr);
      return;
   }

   if (fcptr)
   {
      /* populating a file contents buffer, but not a cache buffer */
      fcptr->ContentRemaining -= bcnt;
      fcptr->ContentLength += bcnt;
      fcptr->CurrentPtr += bcnt;
      /* just get more file data */
      SysDclAst (FileNextBlocks, rqptr);
      return;
   }

   if (rqptr->rqHeader.RangeBytePtr &&
       rqptr->rqHeader.RangeBytePtr->Remaining)
   {
      /* returning a byte-range within the file (partial content) */
      rbptr = rqptr->rqHeader.RangeBytePtr;
      if (rbptr->Offset)
      {
         /* first block read, first byte won't necessarily be at the start */
         bptr += rbptr->Offset;
         bcnt -= rbptr->Offset;                   
         rbptr->Offset = 0;
      }
      /* if the range is less than what was read then discard the rest */
      if (bcnt > rbptr->Remaining) bcnt = rbptr->Remaining;
      rbptr->Remaining -= bcnt;
      /* if no range remaining it's an effective EOF for block I/O! */
      if (!rbptr->Remaining) fqptr->EndOfFile = true;

      if (WATCHMOD (rqptr, WATCH_MOD_FILE))
         WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE,
                    "range !UL-!UL(!UL) vbn:!UL off:!UL rem:!UL",
                    rbptr->First[rbptr->Index],
                    rbptr->Last[rbptr->Index],
                    rbptr->Length,
                    fqptr->BlockNumber,
                    rbptr->Offset,
                    rbptr->Remaining);
   }

   /* write raw/massaged data to client */
   if (VMSok (rqptr->NetIoPtr->WriteStatus))
      NetWrite (rqptr, FileNextBlocks, bptr, bcnt);
   else
      SysDclAst (FileNextBlocks, rqptr);
}

/*****************************************************************************/
/*
Create an in situ stream buffer of characters out of disk virtual block
variable-length records where each record is now terminated by

1)  the default - newline character (LF)
    (i.e. turn variable-length into stream-LF on the fly).

2)  with a SET path mapping

    i)  SET /path response=VAR=CRLF    terminate record with 
   ii)  SET /path response=VAR=LF      terminate record with  (as default)
  iii)  SET /path response=VAR=NONE    do not add carriage control
   iv)  SET /path response=var         revert to default

The word (two byte) record size allows sufficient space to support the single
byte  and two byte  carriage-control.

Handles block spanning and non-spanning, LSB and MSB record length word.  In
other words - all variations (hopefully!)
*/ 

FileVariableRecord (REQUEST_STRUCT *rqptr)

{
   BOOL  MsbRcw, NoSpan;
   int  bcnt;
   ushort  VfcSize, MaxRecordSize, rcnt, recnt;
   char  *bptr, *cptr, *sptr, *zptr;
   FILE_QIO  *fqptr;
   FILE_TASK  *tkptr;
                         
   /*********/
   /* begin */
   /*********/

   if (WATCHMOD (rqptr, WATCH_MOD_FILE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileVariableRecord()");

   tkptr = rqptr->FileTaskPtr;
   fqptr = &tkptr->FileOds.FileQio;

   bptr = fqptr->BufferPtr;
   bcnt = fqptr->BufferCount;
   MsbRcw = fqptr->RecAttr.fat$b_rattrib & FAT$M_MSBRCW;
   NoSpan = fqptr->RecAttr.fat$b_rattrib & FAT$M_NOSPAN;
   VfcSize = fqptr->RecAttr.fat$b_vfcsize;
   MaxRecordSize = fqptr->RecAttr.fat$w_maxrec;
   recnt = rcnt = fqptr->RecordCount;

   if (WATCHMOD (rqptr, WATCH_MOD_FILE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE,
"NoSpan:!&B MsbRcw:!&B VfcSize:!UL MaxRecordSize:!UL rcnt:!UL \
RecordSize:!UL aligned:!&B ResponseVarRecord:!UL",
                 NoSpan, MsbRcw, VfcSize, MaxRecordSize,
                 rcnt, fqptr->RecordSize, (int)bptr & 1,
                 rqptr->rqPathSet.ResponseVarRecord);

   if (WATCHING (rqptr, WATCH_RESPONSE))
   {
      if (!tkptr->WatchVarRecordCC)
      {
         tkptr->WatchVarRecordCC = true;
         switch (rqptr->rqPathSet.ResponseVarRecord)
         {
            case FILE_VAR_CRLF : cptr = ""; break;
            case FILE_VAR_LF   : cptr = ""; break;
            case FILE_VAR_NONE : cptr = "NONE"; break;
            default : if (rqptr->rqPathSet.ResponseVarRecord)
                         cptr = "(unknown)";
                      else
                         cptr = "DEFAULT ()";
         }
         WatchThis (WATCHITM(rqptr), WATCH_RESPONSE,
                    "RFM:VAR carriage control !AZ", cptr);
      }
   }

   zptr = (cptr = sptr = bptr) + bcnt;
   /* see note about VAR record alignment in FileNextBlocks() above */
   if ((int)bptr & 1) cptr++;

   while (cptr < zptr)
   {
      if (WATCHMOD (rqptr, WATCH_MOD_FILE))
         if (rcnt || (cptr > sptr && cptr > bptr+1))
            WatchDataFormatted ("!5ZL |!#AZ|\n",
               rcnt, rcnt <= zptr-cptr ? rcnt : zptr-cptr, cptr);

      if (VfcSize && rcnt && rcnt == fqptr->RecordSize)
      {
         /* adjust for any fixed-length control on start of new record */
         cptr += VfcSize;
         rcnt -= VfcSize;
         if (!rcnt)
         {
            /* ensure this last record has carriage control */
            if (rqptr->rqPathSet.ResponseVarRecord == FILE_VAR_CRLF)
            {
               if (!recnt || (recnt >= 2 && *(USHORTPTR)(sptr-2) != '\r\n'))
               {
                  *(USHORTPTR)sptr = '\r\n';
                  sptr += 2;
               }
            }
            else
            if (rqptr->rqPathSet.ResponseVarRecord != FILE_VAR_NONE)
               if (!recnt || (recnt >= 1 && *(sptr-1) != '\n'))
                  *sptr++ = '\n';
         }
      }

      /* copy the contents of this record */
      while (rcnt && cptr < zptr)
      {
         rcnt--;
         *sptr++ = *cptr++;
      }

      /* if no data left in the buffer */
      if (cptr >= zptr)
      {
         /* if still some data to go in this record */
         if (rcnt) break;

         /* ensure this last record has carriage control */
         if (rqptr->rqPathSet.ResponseVarRecord == FILE_VAR_CRLF)
         {
            if (!recnt || (recnt >= 2 && *(USHORTPTR)(sptr-2) != '\r\n'))
            {
               *(USHORTPTR)sptr = '\r\n';
               sptr += 2;
            }
         }
         else
         if (rqptr->rqPathSet.ResponseVarRecord != FILE_VAR_NONE)
            if (!recnt || (recnt >= 1 && *(sptr-1) != '\n'))
               *sptr++ = '\n';

         break;
      }

      /* step to an even byte boundary */
      if ((int)cptr & 1) cptr++;

      /* if no-span and insufficient space for a record left in this block */
      if (NoSpan && 510 - ((int)cptr & 0x1ff) < MaxRecordSize)
      {
         /* step to the start of the next block */
         cptr = ((int)cptr & ~0x1ff) + 512;
      }

      /* if no data left in the buffer */
      if (cptr >= zptr)
      {
         /* ensure this previous record has carriage control */
         if (rqptr->rqPathSet.ResponseVarRecord == FILE_VAR_CRLF)
         {
            if (!recnt || (recnt >= 2 && *(USHORTPTR)(sptr-2) != '\r\n'))
            {
               *(USHORTPTR)sptr = '\r\n';
               sptr += 2;
            }
         }
         else
         if (rqptr->rqPathSet.ResponseVarRecord != FILE_VAR_NONE)
            if (!recnt || (recnt >= 1 && *(sptr-1) != '\n'))
               *sptr++ = '\n';

         break;
      }

      /* get the new record count */
      rcnt = *(USHORTPTR)cptr;

      /* if MSB then swap the bytes */
      if (MsbRcw) rcnt = (rcnt >> 8) | (rcnt << 8);
      cptr += sizeof(ushort);

      /* note the current record count */
      recnt = rcnt;

      /* note the original size of this record */
      fqptr->RecordSize = rcnt;

      /* ensure previous record has carriage control (after count retrieved!) */
      if (sptr > bptr)
      {
         if (rqptr->rqPathSet.ResponseVarRecord == FILE_VAR_CRLF)
         {
            if (!recnt || (recnt >= 2 && *(USHORTPTR)(sptr-2) != '\r\n'))
            {
               *(USHORTPTR)sptr = '\r\n';
               sptr += 2;
            }
         }
         else
         if (rqptr->rqPathSet.ResponseVarRecord != FILE_VAR_NONE)
            if (!recnt || (recnt >= 1 && *(sptr-1) != '\n'))
               *sptr++ = '\n';
      }

      if (!rcnt && !VfcSize)
      {
         if (rqptr->rqPathSet.ResponseVarRecord == FILE_VAR_CRLF)
         {
            if (!recnt || (recnt >= 2 && *(USHORTPTR)(sptr-2) != '\r\n'))
            {
               *(USHORTPTR)sptr = '\r\n';
               sptr += 2;
            }
         }
         else
         if (rqptr->rqPathSet.ResponseVarRecord != FILE_VAR_NONE)
            if (!recnt || (recnt >= 1 && *(sptr-1) != '\n'))
               *sptr++ = '\n';
      }
   }

   /* zero any remainder from the move */
   if (zptr - sptr > 0) memset (sptr, 0, zptr - sptr);

   fqptr->RecordCount = rcnt;
   fqptr->BufferCount = sptr - bptr;

   if (WATCHMOD (rqptr, WATCH_MOD_FILE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE,
                 "rcnt:!UL bcnt:!UL now:!UL", rcnt, bcnt, sptr - bptr);
}

/*****************************************************************************/
/*
Fixed length records where the record is not allowed to span the blocks (who
the hell uses these formats anyway?!)  Fixed length block spanning records are
just handled in the blocks.  No adjustments to carriage control!
*/ 

FileFixedRecordNoSpan (REQUEST_STRUCT *rqptr)

{
   int  bcnt;
   ushort  MaxRecordSize, rcnt;
   char  *bptr, *cptr, *sptr, *zptr;
   FILE_QIO  *fqptr;
   FILE_TASK  *tkptr;
                         
   /*********/
   /* begin */
   /*********/

   if (WATCHMOD (rqptr, WATCH_MOD_FILE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE, "FileFixedRecordNoSpan()");

   tkptr = rqptr->FileTaskPtr;
   fqptr = &tkptr->FileOds.FileQio;

   bptr = fqptr->BufferPtr;
   bcnt = fqptr->BufferCount;
   MaxRecordSize = fqptr->RecAttr.fat$w_maxrec;
   rcnt = fqptr->RecordCount;

   if (WATCHMOD (rqptr, WATCH_MOD_FILE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE,
                 "MaxRecordSize:!UL rcnt:!UL", MaxRecordSize, rcnt);

   zptr = (cptr = sptr = bptr) + bcnt;
   while (cptr < zptr)
   {
      if (WATCHMOD (rqptr, WATCH_MOD_FILE))
         if (rcnt || cptr > sptr)
            WatchDataFormatted ("!5ZL |!#AZ|\n",
               rcnt, rcnt <= zptr-cptr ? rcnt : zptr-cptr, cptr);

      while (rcnt && cptr < zptr)
      {
         rcnt--;
         *sptr++ = *cptr++;
      }
      if (cptr >= zptr) break;
      /* if insufficient space for a record left in this block */
      if (510 - ((int)cptr & 0x1ff) < MaxRecordSize)
      {
         /* step to the start of the next block */
         cptr = ((int)cptr & ~0x1ff) + 512;
         if (cptr >= zptr) break;
      }
      rcnt = fqptr->RecAttr.fat$w_rsize;
   }

   fqptr->RecordCount = rcnt;
   fqptr->BufferCount = sptr - bptr;

   if (WATCHMOD (rqptr, WATCH_MOD_FILE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE,
                 "rcnt:!UL bcnt:!UL now:!UL", rcnt, bcnt, sptr - bptr);
}

/*****************************************************************************/
/*
Send the buffer contents escaping any HTML-forbidden characters.
*/ 

FileWriteBufferEscapeHtml
(
REQUEST_STRUCT *rqptr,
REQUEST_AST AstFunction
)
{
   FILE_TASK  *tkptr;
                         
   /*********/
   /* begin */
   /*********/

   if (WATCHMOD (rqptr, WATCH_MOD_FILE))
      WatchThis (WATCHITM(rqptr), WATCH_MOD_FILE,
                 "FileWriteBufferEscapeHtml() !&A !UL",
                 AstFunction, STR_DSC_LEN(&rqptr->NetWriteBufferDsc));

   tkptr = rqptr->FileTaskPtr;

   /* ensure the HTML descriptor is initialised */
   StrDscIfNotBegin (rqptr, &tkptr->BufferDsc,
                     STR_DSC_SIZE(&rqptr->NetWriteBufferDsc));

   /* swap the file read descriptor and the HTML descriptor */
   StrDscSwap (&rqptr->NetWriteBufferDsc, &tkptr->BufferDsc);

   /* reset the (now) output descriptor */
   StrDscNoContent (&rqptr->NetWriteBufferDsc);

   /* copy as HTML into the output descriptor */
   StrDscBuildHtmlEscape (&rqptr->NetWriteBufferDsc, &tkptr->BufferDsc, NULL);

   /* write it out */
   NetWriteStrDsc (rqptr, AstFunction);
}

/*****************************************************************************/
/*
The opaque entity identifier is generated from the six byte file-id plus the
four byte revision date-time.  In this way if the file is moved or revised the
entity should reflect that.  Assumes the supplied string is large enough to
store the generated entity tag (29 bytes).
*/ 

FileGenerateEntityTag
(
char *EntityString,
FILE_QIO *fqptr
)
{
   /* hex digits: 12 for FID, 16 for RDT */
   static $DESCRIPTOR (EntityTagFaoDsc, "!4XL!4XL!4XL!8XL!8XL\0");
   static $DESCRIPTOR (EntityTagDsc, "");

   ulong  *qptr;
   char  *cptr;
                         
   /*********/
   /* begin */
   /*********/

   if (WATCH_MODULE(WATCH_MOD_FILE))
      WatchThis (WATCHALL, WATCH_MOD_FILE, "FileGenerateEntityTag()");

   EntityTagDsc.dsc$w_length = 29;
   EntityTagDsc.dsc$a_pointer = EntityString;

   qptr = &fqptr->RdtTime64;
   sys$fao (&EntityTagFaoDsc, 0, &EntityTagDsc,
            fqptr->Fib.fib$w_fid[0], fqptr->Fib.fib$w_fid[1],
            fqptr->Fib.fib$w_fid[2], qptr[0], qptr[1]);
   for (cptr = EntityString; *cptr; cptr++) *cptr = to_lower(*cptr);

   if (WATCH_MODULE(WATCH_MOD_FILE))
      WatchThis (WATCHALL, WATCH_MOD_FILE, "!AZ", EntityString);
}

/*****************************************************************************/