/*****************************************************************************/ /* pyRTE.c A WASD Run-Time Envrionment (persistent engine) for Python. The basics of this RTE processes with a CGIplus request cycle (as do ALL RTEs). The Python scripts do not need to be aware they are executing in a persistent engine. CGI variables are available from the os.environ and POSTed content from the stream so all out-of-the-box Python CGI scripts should work without modification. The byte-code used is either loaded from pre-compiled file content (.PYO or .PYC coresident with the .PY) or compiled dynamically from a source file (.PY). It maintains a cache of this byte-code for a number of scripts (determined by compilation constant and logical name value). Benchmarks on the elementary "hello world" test script indicates an increase in throughput of 30% compared to when the cache is disabled. pyRTE also supports Python scripts explicitly implementing CGIplus themselves. All that is required of the script is a call to wasd.cgiplus_begin() to synchonise the receipt of a new request. All other CGIplus functionality is handled transparently by PYRTE.C. CGI variables are present in the os.environ and POSTed content is as usual available form . A minimal example CGIplus script: import wasd while wasd.cgiplus_begin(): print ('Content-type: text/html') print ('') print ('

hello world!

') The cgiplus_begin() method accepts zero, one or two positional parameters. 1) The first is a boolean (True or False) that if true allows a standard CGI script to call this once before returning false at the second call. This eliminates the need for explicit CGI and CGIplus paths. 2) The second parameter is also a boolean (True or False). If true it gets the script code modification time to check for changes. If modified it returns false causing the CGIplus script to exit gracefully (and consequently have the new code run-up by the server). NOTE: This RTE needs to differentiate between operating in CGIplus mode, where the RTE activated scripts that are unaware they are operating in a persistent engine environment, and when an explicitly CGIplus script is using the RTE as a Python engine. Hence two flags in the code; IsCgiPlus which indicates it's not standard CGI, just an RTE, and IsCgiPlusScript which indicates it's been activated as a CGIplus script. VSI PYTHON 3.10 --------------- https://github.com/vmssoftware/cpython/tree/openvms-3.10.0/vms https://forum.vmssoftware.com/viewforum.php?f=32 https://forum.vmssoftware.com/viewtopic.php?f=32&t=8600 https://forum.vmssoftware.com/viewtopic.php?f=32&t=8856 DEBUGGING SCRIPTS ----------------- Always fun :-/ PyRTE v2.0/3.0 provides some execution data via the WATCH [x]Script report. For example (note the SCRIPT items): |13:37:44.55 SERVICE 1749 009001 CONNECT VIRTUAL wasd.vsm.com.au:443| |13:37:44.55 REQUEST 4418 009001 REQUEST GET /py-bin/pyrte_test2.py| |13:37:44.55 CACHE 0600 009001 RESPONSE CACHE search path 23E5BF0E9DBCDFE3B01647C116E79A09| |13:37:44.55 DCL 1611 009001 RESPONSE SCRIPT as HTTP$NOBODY RTE /py-bin/pyrte_test2.py wasd_root:[src.python.scripts]pyrte_test2.py ($cgi_exe:pyrte3.exe)| |13:37:45.24 DCL 8396 009001 SCRIPT PYRTE IA64-3.0.0 Python 3.10.0 (default, Nov 25 2021, 10:52:09) [C]| |13:37:45.25 DCL 8396 009001 SCRIPT RTE caching /py-bin/pyrte_test2.py /WASD_ROOT/src/python/SCRIPTS/pyrte_test2.py| |13:37:45.25 DCL 8396 009001 SCRIPT CACHE new 1/1| |13:37:45.25 DCL 8396 009001 SCRIPT LOAD /py-bin/pyrte_test2.py /WASD_ROOT/src/python/SCRIPTS/pyrte_test2.py| |13:37:45.25 DCL 8396 009001 SCRIPT CODE /WASD_ROOT/src/python/SCRIPTS/pyrte_test2.py| |13:37:45.26 DCL 8396 009001 SCRIPT EVAL /py-bin/pyrte_test2.py| |13:37:47.75 DCL 8396 009001 SCRIPT PYRTE IA64-3.0.0 USAGE:1/1 REAL:00:00:02.51 CPU:2.49 DIO:473 BIO:2437 FAULTS:957 PGFL:1966832/2%| |13:37:47.75 GZIP 0608 009001 RESPONSE DEFLATE 3366->1521 bytes, 45% (261kB)| |13:37:47.75 NETIO 0514 009001 RESPONSE NETWORK %X0000002C (%SYSTEM-F-ABORT, abort) (non-blocking)| |13:37:47.75 REQUEST 1435 009001 REQUEST STATUS 200 (OK) rx:344 tx:4911 bytes 4.205s 1,249 B/s| |00:33:50.17 DCL 7962 265001 SCRIPT EVAL /py-bin/pyrte_test3.py| |00:33:56.82 DCL 7962 265001 SCRIPT ERROR 'logical name PYRTE_CACHE_ENTRY not defined'| It is also possible to add items from within the script Python code using the wasd.WATCH() function. The example /py-bin/pyrte_test1.py script contains the statement wasd.WATCH('and hello [x]Script') |13:48:12.43 DCL 8396 013001 SCRIPT RTE caching /py-bin/pyrte_test1.py /WASD_ROOT/src/python/SCRIPTS/pyrte_test1.py| |13:48:12.43 DCL 8396 013001 SCRIPT CACHE unused 3/3| |13:48:12.43 DCL 8396 013001 SCRIPT LOAD /py-bin/pyrte_test1.py /WASD_ROOT/src/python/SCRIPTS/pyrte_test1.py| |13:48:12.44 DCL 8396 013001 SCRIPT CODE /WASD_ROOT/src/python/SCRIPTS/pyrte_test1.py| |13:48:12.44 DCL 8396 013001 SCRIPT EVAL /py-bin/pyrte_test1.py| |13:48:12.44 DCL 8396 013001 SCRIPT WATCH and hello [x]Script| |13:48:12.44 DCL 8396 013001 SCRIPT PYRTE IA64-3.0.0 USAGE:2/4 REAL:00:00:00.01 CPU:0.00 DIO:3 BIO:37 FAULTS:2 PGFL:1961088/2%| LATENCIES --------- BY DEFAULT, pyRTE takes AN AGGRESSIVE POSITION ON REDUCING LATENCY. This leaves the interpreter intact for each unique script. The standard CGI "hello world" has it's performance pushed from 2.5 requests/second to 80/S (on my PWS500au test-bench), an improvement of some 35x!! A unique main dictionary is created for each request but there is some potential for any 'fiddles' under-the-hood (e.g. the sort of thing cgitb does to exception handling) may cause interactions between request invocations. As the same interpreter is only used for the same script (based on a unique script file-system name) it is suggested that these may be few-and-far-between, and perhaps mostly benign. Anyway, this mode of interpreter reuse can be disabled on a per-RTE basis using the PYRTE_NOREUSE_INTERPRETER logical name, or using the per-script mapping rule 'script=param="PYRTE=/NOREUSE"', or by the Python script using the wasd.reuse_interpreter(False) function sometime during execution. CGIplus scripting also shines in the latency department. With "hello world" running as CGIplus back-to-back latency drops to approximately 6mS!! This translates to something like 150 requests/second rather than 3.5/S. Nice! Surely with Python's automatic garbage collection and a little care persistent CGIplus applications in Python should be eminently doable. WASD PYTHON MODULE ------------------ The RTE implicitly makes available a module that provides additional functionality for CGIplus implementations and other purposes. The module still needs to be imported into a Python application: import wasd Python functions provided by this module: wasd.cgi_callout() Initiate a CGI callout to the server. The required parameter provides the callout request string. If the request string begins with '!' or '#' the callout will return Py_None, otherwise the response string. wasd.cgiplus_begin() This function is used to synchronise an explicitly CGIplus script (see example above). It accepts zero, one or two arguments. wasd.cgiplus_end() Can be used to explicitly to end a CGIplus request. Normally that functionality would be handled implicitly using wasd.cgiplus_begin(). wasd.cgivar() Normally all CGI variables may be accessed via os.environ but this function can be used to obtain the same data (and some variables that don't make it into os.environ (e.g. FORM_..). wasd.read_cgiplusin() Read a record from the CGIPLUSIN stream (caution!) wasd.reuse_thread() wasd.reuse_interpreter() See discussion of 'latencies' above. Takes one arugment; True to reuse the current interpreter next request, or False to use a fresh interpreter. wasd.rte_cache_entry() Returns a string containing details of the particular cache entry as a series of comma-separated-values (CSV) in the same order as the data structure. Multiple calls step through each cache element until None is returned. Note that this function will not succeed without logical name PYRTE_CACHE_ENTRY being defined. wasd.rte_id() Returns a string containing the RTE software identification. wasd.stat_timer() Returns a string containing some LIB$STAT_TIMER() data. wasd.usage_code() Returns an integer representing the number of times the cached byte-code for this particular script has been (re)used. wasd.usage_thread() wasd.usage_interpreter() Returns an integer representing the number of times this (sub)interpreter has been used (if enabled). wasd.usage_rte() Returns an integer representing the number of times this RTE has been used. wasd.watch() Writes a message to WATCH [x]Script (useful for debugging). wasd.write() Writes directly to with a following fflush(). WSGI INTERFACE -------------- Python Web Server Gateway Interface v1.0 as described in PEP 333 http://www.python.org/dev/peps/pep-0333/ The WSGI code has been developed under the sponsorship of SysGroup http://www.sysgroup.fr/ and generously made available to the wider WASD community. Activate the WSGI function as in the following example. def obvious_example(environ,start_response): status = '200 OK' response_headers = [('Content-type','text/plain')] start_response(status, response_headers) return ['Hello world!\n'] import wasd wasd.wsgi_run(obvious_example) PEP 333 specifies that WSGI should not buffer output and pyRTE complies but some tools (like Mercurial) don't buffer themselves and end up sending many very small records. If buffered the throughput boost for Mercurial is about x4. This forced buffering may be enabled on a per-application basis using the mapping rule 'script=param=PYRTE=/WSGI=BUFFER' or by defining the logical name PYRTE_WSGI_FORCE_BUFFERING (in a wrapper procedure). REFERENCES ---------- Python/C API Reference Manual, Release 2.7.18 and 3.8 Extending and Embedding the Python Interpreter, Release 2.7.18 and 3.8 Lots of Googling with many and varied query strings. http://python3porting.com/cextensions.html PYRTE COPYRIGHT --------------- Copyright (C) 2007-2024 Mark G.Daniel Licensed under the Apache License, Version 2.0 (the "License"). Software under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See http://www.apache.org/licenses/LICENSE-2.0 PYTHON COPYRIGHT ---------------- Python is subject to it's own PSF license. http://www.python.org/psf/license.html PYRTE METRICS REPORT -------------------- When using the byte-code caching RTE it is possible to generate a report on each cached entry. This includes the number of times the cached code has been used, the size of the byte-code, a list of the last ten (default compilation) uses with each use's time-stamp, duration, CPU ticks attributed, BIO count, DIO count, working set pages, page faults, when the cached script was last modified, the script file location, and any associated pre-compiled python byte code file. Each of the entry's uses are listed top to bottom from most to least recent. A little care in interpreting the data is required. Note the CPU consumed is accumulated in 10mS ticks and therefore a little coarse. This report is only available if the logical name PYRTE_METRICS is defined and accessable by the RTE. For example $ DEFINE /SYSTEM PYRTE_METRICS 1 Accessing *any* script with a query string beginning "$metrics$" will then display the report. MAPPING RULES ------------- For standard (non-CGIplus) Python scripts ... To make any in-production .PY and associated .PYO or .PYC file in CGI-BIN:[000000] available as an executable Python script: exec /cgi-bin/**.py* (cgi-bin:[000000]pyrte.exe)/cgi-bin/*.py* \ script=syntax=unix script=query=none map=once ods=5 For Python scripts implementing CGIplus themselves ... Requires the configuration directive: [DclRunTime] .PY $CGI-BIN:[000000]PYRTE.EXE And the mapping: exec+ /cgiplus-bin/**.py* /cgi-bin/*.py* script=syntax=unix script=query=none map=once ods=5 Of course this may be adapted to Python scripts in any location. For PYRTE development purposes this rule directly makes available the test Python scripts in the source directory as if they were in /cgi-bin/: exec /cgi-bin/pyrte_**.py* \ (cgi-bin:[000000]pyrte.exe)/ht_root/src/pyrte/pyrte_*.py* \ script=query=none map=once ods=5 exec+ /cgiplus-bin/pyrte_**.py* /ht_root/src/pyrte/pyrte_*.py* script=syntax=unix script=query=none map=once ods=5 SCRIPT PARAMETERS ----------------- These parameters can control aspects of RTE behaviour on a per-path (and therefore per-script) basis using mapping rules. script=param=PYRTE=/HEAD=GET same as logical PYRTE_HEAD_CVT_GET script=param=PYRTE=/REUSE reuse this Python thread script=param=PYRTE=/NOREUSE do not reuse this Python thread script=param=PYRTE=/NOSTREAM do not issue the stream-mode header script=param=PYRTE=/WSGI=BUFFER force WSGI buffering mode LOGICAL NAMES ------------- PYRTE$DBUG enables all if(dbug) output statements; if some unique part of the script debugs just that script if defined to something like 'c' or '/' or '-' debugs all (this script-name check is CASE-SENSITIVE) PYRTE_ARGV multi-valued logical name containing Python CLI switches PYRTE_BASIC_RTE full interpreter cycle request processing (for comparison) PYRTE_HEAD_CVT_GET if a HEAD method convert to a GET method as per Apache mod_wsgi behaviour PYRTE_CACHE_ENTRY enables function RtePyMethRteCache() PYRTE_CACHE_MAX integer size of code cache (zero to compare uncached) PYRTE_METRICS enable metric collection and report via "?$metrics$" query PYRTE_NOREUSE_THREAD do not reuse thread for each script instance PYRTE_NOREUSE_INTERPRETER backward compatibility with PYRTE_NOREUSE_THREAD PYRTE_ONESHOT voluntary exit after a single use (see PYRTE_USAGE_LIMIT) PYRTE_USAGE_LIMIT integer number of requests before voluntary exit PYRTE_WSGI_FORCE_BUFFERING enable buffering with WASD, this breaks the WSGI specification but may give a performance boost. BUILD DETAILS ------------- $ @BUILD_PYRTE BUILD !compile+link $ @BUILD_PYRTE LINK !link-only $ COPY WASD_EXE:PYRTE.EXE CGI_EXE: VERSION HISTORY (update SOFTWAREVN as well!) --------------- 09-APR-2024 MGD v3.1.0 VSI X86VMS PYTHON A3.10-0RELEASE008 20-MAY-2023 MGD v3.0.0, VSI I64VMS PYTHON A3.10-0RELEASE001 in line with WASD, moved under Apache License add WATCH [x]Script watch-points and wasd.WATCH() v2.0.0, Python 2 equivalent (JFP 2.7.18) 10-SEP-2020 MGD v3.0.0, Python 3 (via JFP Python 3.10.0a0) **UNRELEASED** v2.0.0, Python 2 equivalent **UNRELEASED** 22-JUL-2017 MGD v1.1.12, wasd.cgi_callout and RtePyMethCgiCallout() provides callout environment regardless of WSGI bugfix; stderr = freopen(stderr); 20-JUN-2013 MGD v1.1.11, bugfix; ProcessCachingRte() it appears to be necessary to destroy *EnvironDict along with using Py_EndInterpreter() bugfix; usage limit should be greater than equal to 05-NOV-2011 MGD v1.1.10, Python 2.7.2, see changes to includes 27-FEB-2011 MGD v1.1.9, /CLI= to allow a command-line script activation logical PYRTE_METRICS enables "?$metrics$" report 20-JUL-2010 MGD v1.1.8, WASD v10.1 ProctorDetect() 17-MAY-2010 JFP v1.1.7, add missing Py_Finalize() in ProcessCachingRte() PyErr_Occurred is a borrowed reference - remove erroneous calls to Py_DECREF 30-APR-2010 JFP v1.1.6, WsgiSendResponse(), headers were send too early in some cases 29-APR-2010 MGD v1.1.5, PYRTE_HEAD_CVT_GET and script=param=PYRTE=/HEAD=GET with support in SetupOsEnvCgiVar() (per JFP) JFP Fix a bug in WsgiSendResponse, headers were not sent when the body respond is empty 08-DEC-2009 JFP v1.1.4, fix bugs introduced in 1.1.3 07-DEC-2009 JFP v1.1.3, os.environ is not a dictionary on OpenVMS 19-APR-2008 JFP v1.1.2, WSGI buffered output (see description above) 17-JAN-2008 MGD v1.1.1, CgiVar() and CgiVarDclSymbol() ensure an empty SCRIPT_NAME *is* empty (not the WASD-ism "/") 05-JAN-2008 MGD v1.1.0, WSGI support 22-APR-2007 MGD v1.0.0, initial development */ /*****************************************************************************/ #define SOFTWARECR "Copyright (C) 2007-2024 Mark G.Daniel" #if PY_MAJOR_VERSION >= 3 # define SOFTWAREVN "3.1.0" #else # define SOFTWAREVN "2.0.0" #endif #define SOFTWARENM "PYRTE" #ifdef __ALPHA # define SOFTWAREID SOFTWARENM " AXP-" SOFTWAREVN #endif #ifdef __ia64 # define SOFTWAREID SOFTWARENM " IA64-" SOFTWAREVN #endif #ifdef __x86_64 # define SOFTWAREID SOFTWARENM " X86-" SOFTWAREVN #endif #define SOFTWAREAPL \ "Licensed under the Apache License, Version 2.0 (the \"License\").\n\ Software under the License is distributed on an \"AS IS\" BASIS,\n\ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n\ See http://www.apache.org/licenses/LICENSE-2.0\n\ " /************/ /* includes */ /************/ /* Actually must come before the python include to avoid char *crypt (const char *__key, const char *__salt); ......^ %CC-E-NOTEXPECTING, Error parsing parameter list. Found "(" when expecting one of: , , "...", ")". at line number 406 in module UNISTD of text library SYS$COMMON:[SYSLIB]DECC$RTLDEF.TLB;1 with HP C V7.1-011 on OpenVMS IA64 V8.4, and ctime_r (tsecs,buf); ............^ %CC-W-PTRMISMATCH, In this statement, the referenced type of the pointer value "(const int ...)tsecs" is "const int", which is not compatible with "const unsigned long". with HP C V7.1-011 on OpenVMS IA64 V8.4 and HP C V7.3-009 on OpenVMS Alpha V8.3 ... at least! */ #include /* 2.7.18 build kludge */ #ifdef __ia64 /** VSI C V7.4-001 on OpenVMS IA64 V8.4-2L1 **/ #ifndef intmax_t typedef signed long long int intmax_t; typedef unsigned long long int uintmax_t; #endif #ifndef intptr_t //typedef signed long long int intptr_t; //typedef unsigned long long int uintptr_t; #endif #endif /* Python-specific includes (must come first!) */ /** INTERIM **/ #define PY_SSIZE_T_CLEAN #define _Py_FORCE_UTF8_FS_ENCODING #include /* standard C header files */ #include #include #include #include #include #include #include #include #include #include #include #include /* VMS-specific header files */ #include #include #include #include #include #include #include #include #include #include /**********/ /* macros */ /**********/ /* mainly to allow easy use of the __unaligned directive */ #define ULONGPTR __unaligned unsigned long* #define USHORTPTR __unaligned unsigned short* #define INT64PTR __unaligned __int64* #ifndef BUILD_DATETIME # define BUILD_DATETIME "(undefined)" #endif #define DEBUGPYOBJ(sobj,pobj) \ if (dbug) DebugPyObject (sobj, pobj, __LINE__); #define FI_LI __FILE__, __LINE__ /******************/ /* global storage */ /******************/ int dbug, watches, AppOutputCount, CgiPlusState, CodeMetricEnabled, CurrentTimeSecs, HeadCvtGet, IsCgiPlus, IsCgiPlusScript, IsScriptRte, MainArgc, NoStreamMode, PyrteUsageCount, ReuseThread, RunStarted, UsageLimit, WasdHeadersSent, WsgiHeadersSent, WsgiRunApp, WsgiForceBuffering; unsigned long ExitStatus, ProcessPid, RteJpiCpuTim, RteJpiBufIO, RteJpiDirIO, RteJpiPageFlts, RteJpiPpgCnt; unsigned long CurrentBinTime [2], RteDeltaBinTime [2] = { -1, -1 }; char *CgiPlusEofPtr, *CgiPlusEotPtr, *CgiPlusEscPtr; char **MainArgv; char CgiPlusEof [64], CgiPlusEot [64], CgiPlusEsc [64]; char ErrorCgiVarEnv [] = "Error initialising CGI environment.", ErrorBugcheck [] = "BUGCHECK!", ErrorOverflow [] = "Buffer overflow!", ErrorPython [] = "Error initialising Python environment.", ErrorWsgiEnv [] = "Error initialising WSGI environment."; char SoftwareId [] = SOFTWAREID; FILE *CgiPlusInFp; PyObject *pBuiltins, *pError, *pEnvironDict, *pMainDict, *pMainModule, *pOsDict, *pOsEnvironDict, *pOsModule, *pStdout, *pWasdModule, *pWsgiExcInfo, *pWsgiHeaders, *pWsgiStatus; PyInterpreterState *pInterpState; struct AnExitHandler { unsigned long Flink; unsigned long HandlerAddress; unsigned long ArgCount; unsigned long ExitStatusPtr; } ExitHandler; /*********************/ /* global code cache */ /*********************/ /* somewhat arbitrary */ #define CACHE_MAX 32 int CodeCacheCount = 0, CodeCacheMax = CACHE_MAX; /* number of requests instrumented */ #define METRIC_MAX 10 /* size of instrumentation URI buffer */ #define METRIC_PATH_MAX 48 struct CodeMetricStruct { int MetricIndex; unsigned long JpiBufIO[METRIC_MAX+1], JpiCpuTim[METRIC_MAX+1], JpiDirIO[METRIC_MAX+1], JpiPageFlts[METRIC_MAX+1], JpiPpgCnt[METRIC_MAX+1], UsageNumber[METRIC_MAX+1]; unsigned long DeltaBinTime[METRIC_MAX+1][2], EndBinTime[METRIC_MAX+1][2], StartBinTime[METRIC_MAX+1][2]; char RequestPath [METRIC_MAX+1][METRIC_PATH_MAX]; }; struct CodeCacheStruct { int CodeUsageCount, ThreadUsageCount; unsigned long LastUsedSecs, ScriptMtimeSecs; char *FileNamePtr, *FileNameByteCodePtr, *ScriptNamePtr; PyInterpreterState *pInterpState; PyObject *pMainDict2; #if PY_MAJOR_VERSION >= 3 PyObject *pByteCode; #else PyCodeObject *pByteCode; #endif struct CodeMetricStruct *CodeMetricPtr; } CodeCache [CACHE_MAX]; struct CodeCacheStruct *CodeCachePtr; /****************************/ /* python extension methods */ /****************************/ static PyObject* RtePyMethCgiCallout (); PyDoc_STRVAR (rte_cgi_callout__doc__, "Initiate a CGI callout to the server. \ The required parameter provides the callout request string. \ If the request string begins with '!' or '#' the callout will return \ NULL otherwise the callout response string."); static PyObject* RtePyMethCgiPlusBegin (PyObject*, PyObject*); PyDoc_STRVAR (rte_cgiplus_begin__doc__, "For a Python script implementing the CGIplus protocol itself this method \ synchronises against the receipt of a new request from the server. It handles \ all of the stream flushing and sentinal output required of WASD CGIplus."); static PyObject* RtePyMethCgiPlusEnd (PyObject*, PyObject*); PyDoc_STRVAR (rte_cgiplus_end__doc__, "For a Python script implementing the CGIplus protocol itself this method \ provides all of the flushing and sentinal output required at the end of \ request process. This functionality is implicitly provided by the \ cgiplus_begin() method and so generally is not explicitly required."); static PyObject* RtePyMethGetVar (PyObject*, PyObject*); PyDoc_STRVAR (rte_getvar__doc__, "Returns a string containing the value of the CGI variable specified by name \ in the function argument. Generally CGI variables are accessed from \ os.environ in the same manner as other environment variables."); static PyObject* RtePyMethReadCgiPlusIn (); PyDoc_STRVAR (rte_read_cgiplusin__doc__, "Retuns a string containing a record read from the CGIPLUSIN stream."); static PyObject* RtePyMethReuseThread (); static PyObject* RtePyMethReuseInterpreter (); /* backward compatibility */ PyDoc_STRVAR (rte_reuse_thread__doc__, "Enables (True) or disables (False) interpreter reuse for the next request \ received by pyRTE."); static PyObject* RtePyMethRteCacheEntry (PyObject*, PyObject*); PyDoc_STRVAR (rte_cache_entry__doc__, "Returns a string containing details of a pyRTE cache entry as a series of \ comman-separated values (CSV) in the same order as the data structure. \ Multiple calls step through each cache element until None is returned. \ This function will not succeed unless logical name PYRTE_CACHE_ENTRY is \ defined."); static PyObject* RtePyMethRteId (); PyDoc_STRVAR (rte_id__doc__, "Returns a string containing the pyRTE version and build date/time."); static PyObject* RtePyMethStatTimer (); PyDoc_STRVAR (rte_stat_timer__doc__, "Returns a string containing LIB$STAT_TIMER() statistics for the request."); static PyObject* RtePyMethUsageCode (); PyDoc_STRVAR (rte_usage_code__doc__, "Returns an integer value for how many times the cached byte-code \ has been used."); static PyObject* RtePyMethUsageThread (); static PyObject* RtePyMethUsageInterpreter (); PyDoc_STRVAR (rte_usage_thread__doc__, "Returns an integer value for how many times the current Python \ thread has been used."); static PyObject* RtePyMethUsageRte (); PyDoc_STRVAR (rte_usage_rte__doc__, "Returns an integer value for how many times the RTE has been used."); static PyObject* RteWATCH (PyObject*, PyObject*); PyDoc_STRVAR (wsgi_watch__doc__, "Write to WATCH [x]Script (useful for debugging)."); /* WSGI methods */ static PyObject* WsgiRun (PyObject*, PyObject*); PyDoc_STRVAR (wsgi_wsgi_run__doc__, "WSGI implementation - run this application (function)."); static PyObject* WsgiStartResponse (PyObject*, PyObject*); PyDoc_STRVAR (wsgi_start_response__doc__, "WSGI implementation - begin HTTP response processing."); static PyObject* RteStdoutWrite (PyObject*, PyObject*); PyDoc_STRVAR (wsgi_write__doc__, "Write direct to ."); /* Jean-Francois Pieronne's original '_wasd' module functions */ static PyObject* wasd_cgi_init (); static PyObject* wasd_isCgiPlus (); static PyObject* wasd_cgi_eof (); static PyObject* wasd_cgi_info (); static PyObject* wasd_cgi_read (); static struct PyMethodDef RtePyMethods [] = { /* WASD PyRTE methods */ { "cgi_callout", (PyCFunction)RtePyMethCgiCallout, METH_VARARGS, rte_cgi_callout__doc__ }, { "cgiplus_begin", (PyCFunction)RtePyMethCgiPlusBegin, METH_VARARGS, rte_cgiplus_begin__doc__ }, { "cgiplus_end", (PyCFunction)RtePyMethCgiPlusEnd, METH_VARARGS, rte_cgiplus_end__doc__ }, { "getvar", (PyCFunction)RtePyMethGetVar, METH_VARARGS, rte_getvar__doc__ }, { "read_cgiplusin", (PyCFunction)RtePyMethReadCgiPlusIn, METH_VARARGS, rte_read_cgiplusin__doc__ }, { "reuse_thread", (PyCFunction)RtePyMethReuseThread, METH_VARARGS, rte_reuse_thread__doc__ }, /* and backward compatibility */ { "reuse_interpreter", (PyCFunction)RtePyMethReuseThread, METH_VARARGS, rte_reuse_thread__doc__ }, { "rte_cache_entry", (PyCFunction)RtePyMethRteCacheEntry, METH_VARARGS, rte_cache_entry__doc__ }, { "rte_id", (PyCFunction)RtePyMethRteId, METH_VARARGS, rte_id__doc__ }, { "stat_timer", (PyCFunction)RtePyMethStatTimer, METH_NOARGS, rte_stat_timer__doc__ }, { "usage_code", (PyCFunction)RtePyMethUsageCode, METH_NOARGS, rte_usage_code__doc__ }, { "usage_thread", (PyCFunction)RtePyMethUsageThread, METH_NOARGS, rte_usage_thread__doc__ }, /* and backward compatibility */ { "usage_interpreter", (PyCFunction)RtePyMethUsageThread, METH_NOARGS, rte_usage_thread__doc__ }, { "usage_rte", (PyCFunction)RtePyMethUsageRte, METH_NOARGS, rte_usage_rte__doc__ }, { "WATCH", (PyCFunction)RteWATCH, METH_VARARGS, wsgi_watch__doc__ }, { "watch", (PyCFunction)RteWATCH, METH_VARARGS, wsgi_watch__doc__ }, { "write", (PyCFunction)RteStdoutWrite, METH_VARARGS, wsgi_write__doc__ }, /* WSGI methods */ { "wsgi_run", (PyCFunction)WsgiRun, METH_VARARGS, wsgi_wsgi_run__doc__ }, { "start_response", (PyCFunction)WsgiStartResponse, METH_VARARGS, wsgi_start_response__doc__ }, /* JFP's original '_wasd' module functions */ { "cgi_init", (PyCFunction)wasd_cgi_init, METH_NOARGS, "initialise CGIplus environment" }, { "isCgiPlus", (PyCFunction)wasd_isCgiPlus, METH_NOARGS, "is it activated in CGIplus mode?" }, { "cgi_eof", (PyCFunction)wasd_cgi_eof, METH_NOARGS, "concludes the current CGIplus request" }, { "cgi_info", (PyCFunction)wasd_cgi_info, METH_VARARGS, "return a CGI variable" }, { "read", (PyCFunction)wasd_cgi_read, METH_VARARGS, "read the entire POSTed body" }, { NULL, NULL, 0, NULL } }; #if PY_MAJOR_VERSION >= 3 static struct PyModuleDef RtePyModuleDef = { PyModuleDef_HEAD_INIT, "wasd", /* m_name */ "WASD pyRTE module", /* m_doc */ -1, /* m_size */ RtePyMethods /* m_methods */ }; #endif /**************/ /* prototypes */ /**************/ void vms_set_crtl_values(void); void at_Exit (); void AddMetrics (struct CodeCacheStruct*); int ByteCodeUpToDate (struct CodeCacheStruct*); void CgiPlusBegin (); void CgiPlusEnd (); char* CgiVar (char*); void CollectMetrics (struct CodeCacheStruct*, int); char* DclSymbolNames (); int DbugCheck (int); void DebugPyObject (char*, PyObject*, int); char* EmptyScriptName (char*); void EnsureExit (unsigned long*); void EvalByteCode (struct CodeCacheStruct*); void FlushCacheEntry (struct CodeCacheStruct*); int lib$stop (__unknown_params); int LibVmReport (void*); void LoadByteCode (char*, char*, struct CodeCacheStruct*); int MetricsReport (); int ModuleCgiibLoaded (); int ProcessArgcArgv (); void ProcessCLI (); void ProcessCGI (); void ProcessCGIplus (); void ProcessBasicRte (); void ProcessCachingRte (); void ProcessPyRun (); int ProctorDetect (); int ReadFileIntoMemory (char*, char**, int*); int ReportError (int, int, int, char*, ...); void RteScriptParam (); int ServerSoftware (void); void SetArgv (char*); void SetProgramName (char*); int SetupOsEnvCgiVar (int); char* StatTimer (int); void StatTimerCallout (int); char* TrnLnm (char*); char* Trn2Lnm (char*, char*, int); int WasdModuleInit (void); void WatchCallout (char*, ...); void WatchErrorCallout (); int WsgiSendHeaders (); int WsgiSendResponse (PyObject*); int WsgiOsEnvVar (); PyAPI_FUNC(PyObject*) PyMarshal_ReadObjectFromFile(FILE*); unsigned long PyMarshal_ReadLongFromFile(FILE*); /*****************************************************************************/ /* */ main (int argc, char *argv[]) { static struct { unsigned short buf_len; unsigned short item; void *buf_addr; void *ret_len; } JpiItems [] = { { sizeof(ProcessPid), JPI$_PID, &ProcessPid, 0 }, { 0,0,0,0 } }; int status; char *cptr, *sptr; /*********/ /* begin */ /*********/ #if PY_MAJOR_VERSION >= 3 vms_set_crtl_values (); #endif setenv ("PYTHONDONTWRITEBYTECODE", "1", 1); if (!getenv ("TMPDIR")) setenv ("TMPDIR", "/wasd_root/scratch", 1); MainArgc = argc; MainArgv = argv; /* set the global pointer used by some functions */ CodeCachePtr = &CodeCache[0]; if (argc == 2) { if (!strncasecmp(argv[1], "/run=", 5)) { DbugCheck (1); argv[1] += 5; ProcessPyRun (); exit (SS$_NORMAL); } if (!strncasecmp(argv[1], "/cli=", 5)) { DbugCheck (1); argv[1] += 5; ProcessCLI (); exit (SS$_NORMAL); } if (!strcasecmp(argv[1], "/version")) { fprintf (stdout, "%%PYRTE-I-VERSION, %s (%s) Python %s\n%s\n%s", SoftwareId, BUILD_DATETIME, Py_GetVersion(), SOFTWARECR, SOFTWAREAPL); exit (SS$_NORMAL); } } WasdModuleInit (); Py_Initialize(); PyEval_InitThreads (); #if PY_MAJOR_VERSION >= 3 if (argc >= 2) { ProcessArgcArgv (); exit (SS$_NORMAL); } #endif status = sys$getjpiw (0, 0, 0, &JpiItems, 0, 0, 0); if (!(status & 1)) exit (status); /* if it doesn't look like CGI environment then forget it */ if (!TrnLnm ("HTTP$INPUT")) exit (SS$_ABORT); WsgiForceBuffering = (TrnLnm ("PYRTE_WSGI_FORCE_BUFFERING") != NULL); cptr = TrnLnm ("PYRTE_USAGE_LIMIT"); if (cptr && isdigit(*cptr)) UsageLimit = atoi(cptr); if (TrnLnm ("PYRTE_ONESHOT")) UsageLimit = 1; /* binary mode to eliminate carriage-control */ #if PY_MAJOR_VERSION >= 3 /* DECC$DISABLE_TO_VMS_LOGNAME_TRANSLATION */ if (!(stdin = freopen ("/HTTP$INPUT", "r", stdin, "ctx=bin"))) #else if (!(stdin = freopen ("HTTP$INPUT:", "r", stdin, "ctx=bin"))) #endif exit (vaxc$errno); if (!DbugCheck(1)) { #if PY_MAJOR_VERSION >= 3 /* DECC$DISABLE_TO_VMS_LOGNAME_TRANSLATION */ if (!(stdout = freopen ("/SYS$OUTPUT", "w", stdout, "ctx=bin", "ctx=xplct"))) #else if (!(stdout = freopen ("SYS$OUTPUT:", "w", stdout, "ctx=bin", "ctx=xplct"))) #endif exit (vaxc$errno); /* PY_MAJOR_VERSION >= 3 required interleaving output and errors */ fclose (stderr); stderr = stdout; } dbug = 0; /* need differentiate between RTE/CGIplus and CGIplus script */ CgiPlusEofPtr = strdup(TrnLnm ("CGIPLUSEOF")); CgiPlusEotPtr = strdup(TrnLnm ("CGIPLUSEOT")); CgiPlusEscPtr = strdup(TrnLnm ("CGIPLUSESC")); IsCgiPlus = (CgiPlusEofPtr != NULL); /* set up and declare the exit handler */ ExitHandler.HandlerAddress = (unsigned long)&EnsureExit; ExitHandler.ArgCount = 1; ExitHandler.ExitStatusPtr = (unsigned long)&ExitStatus; status = sys$dclexh (&ExitHandler); if (!(status & 1)) exit (status); /* get handle to python stdout file ready for explicit flushing */ pStdout = PySys_GetObject ("stdout"); if (!pStdout) { ReportError (__LINE__, 0, 1, ErrorPython); exit (SS$_NORMAL); } /* borrowed reference */ pMainModule = PyImport_AddModule ("__main__"); if (!pMainModule) { ReportError (__LINE__, 0, 1, ErrorPython); return; } /* borrowed reference */ pMainDict = PyModule_GetDict (pMainModule); if (!pMainDict) { ReportError (__LINE__, 0, 1, ErrorPython); return; } DEBUGPYOBJ ("pMainDict", pMainDict); /* borrowed reference */ pOsModule = PyImport_AddModule ("os"); if (!pOsModule) { ReportError (__LINE__, 0, 1, ErrorPython); return; } /* borrowed reference */ pOsDict = PyModule_GetDict (pOsModule); if (!pOsDict) { ReportError (__LINE__, 0, 1, ErrorPython); return; } DEBUGPYOBJ ("pOsDict", pOsDict); if (IsCgiPlus) { /* get the first request (which is immediately available) */ CgiVar (""); if (CgiVar("WATCH_SCRIPT") != NULL) WatchCallout ("%s Python %s", SoftwareId, Py_GetVersion()); /* only an RTE will have this variable */ if (IsScriptRte = (CgiVar ("SCRIPT_RTE") != NULL)) { /*******/ /* RTE */ /*******/ if (TrnLnm ("PYRTE_BASIC_RTE")) ProcessBasicRte (); else ProcessCachingRte (); } else { /***********/ /* CGIplus */ /***********/ /* supply the command-line script file name */ ProcessCGIplus (); } } else { /*******/ /* CGI */ /*******/ ProcessCGI (); } Py_Finalize (); exit (SS$_NORMAL); } /*****************************************************************************/ /* Declared exit handler. Python is mulithreaded and these often have difficulty exiting cleanly. Try to ensure that on an error exit (e.g. image bugcheck) the image exits and so allows the scripting process to run-down. */ void EnsureExit (unsigned long *ExitStatusPtr) { /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "EnsureExit() %%X%08.08X\n", ExitStatus); if (ExitStatus & 0x00000001) exit (ExitStatus); lib$stop (ExitStatus, 0, 0); } /*****************************************************************************/ /* Process using standard Python arguments at the command-line. For base-line comparison purposes only. */ #if PY_MAJOR_VERSION >= 3 int ProcessArgcArgv () { PyConfig config; PyStatus status; /*********/ /* begin */ /*********/ if (DbugCheck(1)) fprintf (stdout, "ProcessArgcArgv()\n"); PyConfig_InitPythonConfig (&config); config.isolated = 1; status = PyConfig_SetBytesArgv (&config, MainArgc, MainArgv); if (PyStatus_Exception(status)) goto fail; status = Py_InitializeFromConfig (&config); if (PyStatus_Exception (status)) goto fail; PyConfig_Clear (&config); return (Py_RunMain ()); fail: PyConfig_Clear (&config); if (PyStatus_IsExit(status)) return (status.exitcode); Py_ExitStatusException(status); } #endif /*****************************************************************************/ /* Process from the command-line. For base-line comparison purposes only. */ void ProcessPyRun () { int argc, status; char *cptr, *sptr; FILE *fp; /*********/ /* begin */ /*********/ if (DbugCheck(1)) fprintf (stdout, "ProcessPyRun()\n"); #if PY_MAJOR_VERSION >= 3 wchar_t *program = Py_DecodeLocale("PyRTE", NULL); Py_SetProgramName(program); #endif Py_Initialize (); SetArgv (NULL); fp = fopen (MainArgv[1], "r"); if (fp) { if (dbug) fprintf (stdout, "fp:%u\n", fp); PyRun_AnyFileExFlags (fp, MainArgv[1], 0, NULL); fclose (fp); } else { status = vaxc$errno; ReportError (__LINE__, status, 0, "Error opening !AZ", MainArgv[1]); } Py_Finalize (); if (dbug || TrnLnm ("PYRTE_STAT_TIMER")) fprintf (stdout, "%s\n", StatTimer(TRUE)); } /*****************************************************************************/ /* Process from the command-line. For base-line comparison purposes only. */ void ProcessCLI () { int argc; struct CodeCacheStruct *ccptr; /*********/ /* begin */ /*********/ if (DbugCheck(1)) fprintf (stdout, "ProcessCLI()\n"); WasdModuleInit (); Py_Initialize(); PyEval_InitThreads (); SetArgv (MainArgv[1]); ccptr = &CodeCache[0]; ccptr->FileNamePtr = ccptr->ScriptNamePtr = MainArgv[1]; ccptr->pInterpState = PyInterpreterState_New (); LoadByteCode (MainArgv[1], MainArgv[1], ccptr); if (ccptr->pByteCode) EvalByteCode (ccptr); Py_Finalize (); if (dbug || TrnLnm ("PYRTE_STAT_TIMER")) fprintf (stdout, "%s\n", StatTimer(TRUE)); } /*****************************************************************************/ /* Process with a full, standard CGI request cycle. For base-line comparison purposes only. */ void ProcessCGI () { char *fnptr, *snptr; struct CodeCacheStruct *ccptr; /*********/ /* begin */ /*********/ if (DbugCheck(1)) fprintf (stdout, "ProcessCGI()\n"); watches = (CgiVar("WATCH_SCRIPT") != NULL); fnptr = CgiVar ("PATH_TRANSLATED"); snptr = CgiVar ("PATH_INFO"); if (watches) WatchCallout ("CGI %s %s", snptr, fnptr); StatTimer(FALSE); ccptr = &CodeCache[0]; ccptr->pInterpState = PyInterpreterState_New (); LoadByteCode (fnptr, snptr, ccptr); if (ccptr->pByteCode) { EvalByteCode (ccptr); if (ReportError (0, 0, 0, NULL)) FlushCacheEntry (ccptr); } if (watches) StatTimerCallout (1); } /*****************************************************************************/ /* For a Python script that implements CGIplus itself. */ void ProcessCGIplus () { char *fnptr, *snptr; struct CodeCacheStruct *ccptr; /*********/ /* begin */ /*********/ if (DbugCheck(1)) fprintf (stdout, "ProcessCgiPlus()\n"); watches = (CgiVar("WATCH_SCRIPT") != NULL); /* code behaviours are different for an explicitly CGIplus script */ IsCgiPlusScript = 1; PyrteUsageCount++; StatTimer(FALSE); fnptr = CgiVar ("SCRIPT_FILENAME"); snptr = CgiVar ("SCRIPT_NAME"); if (watches) WatchCallout ("CGIplus %s %s", snptr, fnptr); ccptr = &CodeCache[0]; ccptr->pInterpState = PyInterpreterState_New (); LoadByteCode (fnptr, snptr, ccptr); if (ccptr->pByteCode) { EvalByteCode (ccptr); if (ReportError (0, 0, 0, NULL)) FlushCacheEntry (ccptr); } DbugCheck (0); } /*****************************************************************************/ /* Process with a CGIplus request cycle but a full Python interpreter initialization and rundown cycle as well. For base-line comparison purposes only. */ void ProcessBasicRte () { char *fnptr, *snptr; struct CodeCacheStruct *ccptr; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "ProcessBasicRte()\n"); ccptr = &CodeCache[0]; for (;;) { CgiPlusBegin (); if (dbug) fprintf (stdout, "ProcessBasicRte()\n"); FlushCacheEntry (ccptr); PyGILState_Ensure(); watches = (CgiVar("WATCH_SCRIPT") != NULL); fnptr = CgiVar ("SCRIPT_FILENAME"); snptr = CgiVar ("SCRIPT_NAME"); if (watches) WatchCallout ("RTE basic %s %s", snptr, fnptr); SetProgramName (snptr); LoadByteCode (fnptr, snptr, ccptr); if (ccptr->pByteCode) { EvalByteCode (ccptr); if (ReportError (0, 0, 0, NULL)) FlushCacheEntry (ccptr); RtePyMethCgiPlusEnd (NULL, NULL); } if (watches) StatTimerCallout (1); CgiPlusEnd (); if (UsageLimit && PyrteUsageCount >= UsageLimit) break; } } /*****************************************************************************/ /* Process with an RTE (CGIplus) request cycle. Maintain a cache of the (compiled) byte-code for a number of scripts (determined by compilation constant and logical name value). The byte-code used is either loaded from pre-compiled file content (.PYO or .PYC) or compiled dynamically (and then cached) from a source file (.PY). */ void ProcessCachingRte () { int idx, tidx, retval, status; unsigned long mtime; char *cptr, *fnptr, *sptr, *snptr; struct CodeCacheStruct *ccptr; PyObject *pObject, *pResult; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "ProcessCachingRte()\n"); cptr = TrnLnm ("PYRTE_CACHE_MAX"); if (cptr && isdigit(*cptr)) CodeCacheMax = atoi(cptr); if (CodeCacheMax <= 0 || CodeCacheMax > CACHE_MAX) CodeCacheMax = CACHE_MAX; for (;;) { /********/ /* wait */ /********/ CgiPlusBegin (); if (DbugCheck(1)) fprintf (stdout, "ProcessCachingRte()\n"); watches = (CgiVar("WATCH_SCRIPT") != NULL); if (TrnLnm ("PYRTE_METRICS")) CodeMetricEnabled = 1; else CodeMetricEnabled = 0; if (TrnLnm ("PYRTE_NOREUSE_THREAD") || TrnLnm ("PYRTE_NOREUSE_INTERPRETER")) ReuseThread = 0; else ReuseThread = 1; RteScriptParam (); if (!NoStreamMode) { // fprintf (stdout, "Script-Control: X-stream-mode\n"); fflush (stdout); } /***********/ /* request */ /***********/ fnptr = CgiVar ("SCRIPT_FILENAME"); snptr = CgiVar ("SCRIPT_NAME"); if (watches) WatchCallout ("RTE caching %s %s", snptr, fnptr); sptr = CgiVar ("QUERY_STRING"); if (*sptr == '$' && !strncmp (sptr, "$metrics$", 9) && MetricsReport()) continue; /*******************/ /* byte-code cache */ /*******************/ if (dbug) fprintf (stdout, "CodeCacheCount:%d\n", CodeCacheCount); /* look to see if we have the script byte-code cached */ for (idx = 0; idx < CodeCacheCount; idx++) { ccptr = &CodeCache[idx]; if (!ccptr->FileNamePtr) continue; if (dbug) fprintf (stdout, "%d %d |%s|%s|\n", idx, ccptr->ThreadUsageCount, cptr, ccptr->FileNamePtr); if (!strcasecmp (fnptr, ccptr->FileNamePtr)) break; ccptr = NULL; } if (idx < CodeCacheCount) { /***************/ /* found cache */ /***************/ if (watches) WatchCallout ("CACHE hit %d/%d", idx+1, CodeCacheCount); if (!ByteCodeUpToDate (ccptr)) FlushCacheEntry (ccptr); } else if (CodeCacheMax) { /*********************/ /* new/refresh entry */ /*********************/ /* first look to see if there is an unused entry */ for (tidx = 0; tidx < CodeCacheCount; tidx++) if (!CodeCache[tidx].FileNamePtr) break; if (tidx < CodeCacheCount) { ccptr = &CodeCache[idx = tidx]; if (watches) WatchCallout ("CACHE unused %d/%d", idx+1, CodeCacheCount); } else if (tidx < CodeCacheMax) { /* didn't find an unused entry */ CodeCacheCount++; ccptr = &CodeCache[idx = tidx]; if (watches) WatchCallout ("CACHE new %d/%d", idx+1, CodeCacheCount); } else { /* find the least recently used */ for (idx = tidx = 0; idx < CodeCacheMax; idx++) if (CodeCache[idx].LastUsedSecs < CodeCache[tidx].LastUsedSecs) tidx = idx; ccptr = &CodeCache[idx]; FlushCacheEntry (ccptr); if (watches) WatchCallout ("CACHE lru %d/%d", idx+1, CodeCacheCount); } } else { /* code caching is disabled */ if (watches) WatchCallout ("CACHE disabled"); ccptr = &CodeCache[idx = 0]; if (ccptr->FileNamePtr) FlushCacheEntry (ccptr); } if (dbug) fprintf (stdout, "CodeCacheCount:%d idx:%d usage:%d\n", CodeCacheCount, idx+1, ccptr->CodeUsageCount); /* set the global pointer used by some functions */ CodeCachePtr = ccptr; if (!ccptr->FileNamePtr) { /***************/ /* load script */ /***************/ LoadByteCode (fnptr, snptr, ccptr); AddMetrics (ccptr); } CollectMetrics (ccptr, 0); /******************/ /* execute script */ /******************/ EvalByteCode (ccptr); if (watches) StatTimerCallout (ccptr->ThreadUsageCount); /* after the flush to accumulate any outstanding BIO */ CollectMetrics (ccptr, 1); if (ReportError (0, 0, 0, NULL)) FlushCacheEntry (ccptr); /******************/ /* end of request */ /******************/ CgiPlusEnd (); if (UsageLimit && PyrteUsageCount >= UsageLimit) break; } } /*****************************************************************************/ /* Execute the Python script (the loaded bytecode). */ void EvalByteCode (struct CodeCacheStruct *ccptr) { int error = 0; char *argv [2] = { NULL, NULL }; PyObject *pObject, *pResult; PyGILState_STATE state; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "EvalByteCode()\n"); if (watches) WatchCallout ("EVAL %s", ccptr->ScriptNamePtr); state = PyGILState_Ensure(); if (!ccptr->pByteCode) { FlushCacheEntry (ccptr); ccptr->ThreadUsageCount = 0; PyGILState_Release (state); return; } ccptr->CodeUsageCount++; ccptr->ThreadUsageCount++; ccptr->LastUsedSecs = CurrentTimeSecs; /* borrowed reference */ #if PY_MAJOR_VERSION >= 3 pWasdModule = PyModule_Create (&RtePyModuleDef); #else pWasdModule = Py_InitModule ("wasd", RtePyMethods); #endif if (!pWasdModule) { ReportError (__LINE__, 0, 1, ErrorPython); PyGILState_Release (state); return; } if (!SetupOsEnvCgiVar (1)) { PyGILState_Release (state); return; } PyErr_Clear (); #if PY_MAJOR_VERSION >= 3 DEBUGPYOBJ ("->pByteCode", ccptr->pByteCode); #else DEBUGPYOBJ ("->pByteCode", (struct _object*)ccptr->pByteCode) #endif if (!ccptr->pMainDict2) { ccptr->pMainDict2 = PyObject_CallMethod(pMainDict, "copy", "()"); if (!ccptr->pMainDict2) { ReportError (__LINE__, 0, 1, ErrorPython); PyGILState_Release (state); return; } DEBUGPYOBJ ("ccptr->pMainDict2", ccptr->pMainDict2); } SetProgramName (ccptr->ScriptNamePtr); SetArgv (ccptr->ScriptNamePtr); pResult = PyEval_EvalCode (ccptr->pByteCode, ccptr->pMainDict2, ccptr->pMainDict2); DEBUGPYOBJ ("pResult", pResult); Py_XDECREF (pResult); /* new reference */ if (pError = PyErr_Occurred()) { DEBUGPYOBJ ("pError", pError) Py_XDECREF (pError); if (watches) WatchErrorCallout (); error = ReportError (__LINE__, 0, 1, NULL); } /* explicitly flush the buffer */ pResult = PyObject_CallMethod (pStdout, "flush", NULL); DEBUGPYOBJ ("pResult", pResult); Py_XDECREF (pResult); if (!SetupOsEnvCgiVar (0)) { PyGILState_Release (state); return; } PyGILState_Release (state); } /*****************************************************************************/ /* Create the script byte-code by either loading the source code (.PY) and compiling, or loading the pre-compiled byte code (.PYC) or optimised, pre-compiled byte-code (.PYO). Reports it's own errors. Success is indicated by '->pByteCode' containing a pointer to the code object, and failing by it being NULL. */ void LoadByteCode ( char *FileName, char *ScriptName, struct CodeCacheStruct *ccptr ) { int fclen, status, FileNameLength; unsigned long magic, mtime; char *cptr, *fcptr, *sptr, *zptr, *CharPtr; char FileNameByteCode [256]; FILE *fp; stat_t FstatBuffer; struct CodeMetricStruct *cmptr; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "LoadByteCode() |%s|%s|\n", FileName, ScriptName); if (watches) WatchCallout ("LOAD %s %s", ScriptName, FileName); ccptr->LastUsedSecs = ccptr->ScriptMtimeSecs = ccptr->CodeUsageCount = 0; if (ccptr->pByteCode) { /* remove reference to any cached code object */ Py_DECREF (ccptr->pByteCode); ccptr->pByteCode = NULL; } FlushCacheEntry (ccptr); #if PY_MAJOR_VERSION >= 3 /* create a file name for the cache directory */ zptr = (sptr = FileNameByteCode) + sizeof(FileNameByteCode); for (cptr = FileName; *cptr && sptr < zptr; *sptr++ = *cptr++); while (sptr > FileNameByteCode && *sptr != '/') sptr--; if (sptr <= FileNameByteCode) { ReportError (__LINE__, 0, 0, ErrorBugcheck); return; } fcptr = sptr; for (cptr = "/__pycache__"; *cptr && sptr < zptr; *sptr++ = *cptr++); for (cptr = FileName; *cptr; cptr++); while (cptr > FileName && *cptr != '/') cptr--; if (cptr <= FileName) { ReportError (__LINE__, 0, 0, ErrorBugcheck); return; } while (*cptr && sptr < zptr) *sptr++ = *cptr++; FileNameLength = sptr - FileNameByteCode; CharPtr = sptr; /* first; try to open a '.pyo' (optimised byte-code) */ *sptr++ = 'o'; if (sptr > zptr) { ReportError (__LINE__, 0, 0, ErrorOverflow); return; } *sptr = '\0'; fp = NULL; for (;;) { if (dbug) fprintf (stdout, "|%s|\n", FileNameByteCode); /* note; opened in 'binary' mode */ fp = fopen (FileNameByteCode, "rb", "shr=get"); if (fp && watches) WatchCallout ("BYTECODE %s", FileNameByteCode); if (fp) break; /* third; try to open a '.py' (source) */ if (*CharPtr == 'c') break; /* second; try to open a '.pyc' (compiled byte-code) */ *CharPtr = 'c'; } #else /* PY_MAJOR_VERSION >= 3 */ zptr = (sptr = FileNameByteCode) + sizeof(FileNameByteCode); for (cptr = FileName; *cptr && sptr < zptr; *sptr++ = *cptr++); FileNameLength = sptr - FileNameByteCode; CharPtr = sptr; /* first; try to open a '.pyo' (optimised byte-code) */ *sptr++ = 'o'; if (sptr > zptr) { ReportError (__LINE__, 0, 0, ErrorOverflow); return; } *sptr = '\0'; fp = NULL; for (;;) { if (dbug) fprintf (stdout, "|%s|\n", FileNameByteCode); /* note; opened in 'binary' mode */ fp = fopen (FileNameByteCode, "rb", "shr=get"); if (fp && watches) WatchCallout ("BYTECODE %s", FileNameByteCode); if (fp) break; /* fourth; try to open a '.py' (source) */ if (*CharPtr == 'o') break; /* second; try to open a '.pyc' (compiled byte-code) */ *CharPtr = 'c'; } #endif /* PY_MAJOR_VERSION >= 3 */ if (!fp) { /* well, it's certainly not pre-compiled! */ FileNameByteCode[0] = '\0'; if (dbug) fprintf (stdout, "|%s|\n", FileName); /* note; opened in 'record' mode */ fp = fopen (FileName, "r", "ctx=rec", "shr=get"); if (fp && watches) WatchCallout ("CODE %s", FileName); if (!fp) { if (ProctorDetect ()) return; status = vaxc$errno; ReportError (__LINE__, status, 0, "Error opening !AZ", ScriptName); return; } } if (fstat (fileno(fp), &FstatBuffer) < 0) { status = vaxc$errno; fclose (fp); ReportError (__LINE__, status, 0, "Error fstat() !AZ", ScriptName); return; } PyErr_Clear (); if (FileNameByteCode[0]) { /* byte-code file; discard these first two longwords */ magic = PyMarshal_ReadLongFromFile (fp); mtime = PyMarshal_ReadLongFromFile (fp); /* what's left is the byte-code */ #if PY_MAJOR_VERSION >= 3 ccptr->pByteCode = PyMarshal_ReadObjectFromFile (fp); #else ccptr->pByteCode = (PyCodeObject*)PyMarshal_ReadObjectFromFile (fp); #endif fclose (fp); if (!ccptr->pByteCode) { ReportError (__LINE__, 0, 1, NULL); return; } } else { /* source code file */ status = ReadFileIntoMemory (FileName, &fcptr, &fclen); if (!(status & 1)) { ReportError (__LINE__, status, 0, NULL); return; } #if PY_MAJOR_VERSION >= 3 ccptr->pByteCode = Py_CompileString (fcptr, ScriptName, Py_file_input); #else ccptr->pByteCode = (PyCodeObject*)Py_CompileString (fcptr, ScriptName, Py_file_input); #endif free (fcptr); fclose (fp); if (!ccptr->pByteCode) { ReportError (__LINE__, 0, 1, NULL); return; } } /* successful load */ ccptr->ScriptMtimeSecs = FstatBuffer.st_mtime; ccptr->FileNamePtr = calloc (1, FileNameLength+1); if (!ccptr->FileNamePtr) exit (vaxc$errno); strcpy (ccptr->FileNamePtr, FileName); ccptr->ScriptNamePtr = calloc (1, strlen(ScriptName)+1); if (!ccptr->ScriptNamePtr) exit (vaxc$errno); strcpy (ccptr->ScriptNamePtr, ScriptName); if (FileNameByteCode[0]) { ccptr->FileNameByteCodePtr = calloc (1, FileNameLength+2); if (!ccptr->FileNameByteCodePtr) exit (vaxc$errno); strcpy (ccptr->FileNameByteCodePtr, FileNameByteCode); } if (dbug) { fprintf (stdout, "|%s|%s| %u", ccptr->FileNamePtr, ccptr->FileNameByteCodePtr, ccptr->pByteCode); fflush (stdout); #if PY_MAJOR_VERSION >= 3 DEBUGPYOBJ ("->pByteCode", ccptr->pByteCode); #else DEBUGPYOBJ ("->pByteCode", (struct _object*)ccptr->pByteCode) #endif } } /*****************************************************************************/ /* Check if this script (source or byte) code has been modified. */ int ByteCodeUpToDate (struct CodeCacheStruct *ccptr) { int retval; stat_t StatBuffer; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "ByteCodeUpToDate()\n"); if (ccptr->FileNameByteCodePtr) retval = stat (ccptr->FileNameByteCodePtr, &StatBuffer); else retval = stat (ccptr->FileNamePtr, &StatBuffer); if (retval) return (0); /* if the script file is more recent than the cached byte-code */ if (dbug) fprintf (stdout, "%d - %d %d secs\n", StatBuffer.st_mtime, ccptr->ScriptMtimeSecs, StatBuffer.st_mtime - ccptr->ScriptMtimeSecs); if (watches) WatchCallout ("MODIFIED %d secs", StatBuffer.st_mtime - ccptr->ScriptMtimeSecs); if (StatBuffer.st_mtime != ccptr->ScriptMtimeSecs) return (0); return (1); } /*****************************************************************************/ /* Free dynamic components, the thread, and reset the structure. */ void FlushCacheEntry (struct CodeCacheStruct *ccptr) { int retval; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "FlushCacheEntry()\n"); if (!ccptr) return; if (ccptr->FileNamePtr) free (ccptr->FileNamePtr); if (ccptr->FileNameByteCodePtr) free (ccptr->FileNameByteCodePtr); if (ccptr->ScriptNamePtr) free (ccptr->ScriptNamePtr); if (ccptr->CodeMetricPtr) free (ccptr->CodeMetricPtr); ccptr->FileNamePtr = ccptr->FileNameByteCodePtr = ccptr->ScriptNamePtr = NULL; ccptr->CodeMetricPtr = NULL; if (ccptr->pByteCode) { Py_XDECREF (ccptr->pByteCode); ccptr->pByteCode = NULL; } if (ccptr->pMainDict2) { Py_XDECREF (ccptr->pMainDict2); ccptr->pMainDict2 = NULL; } if (ccptr->pInterpState) { /***** (seems to work without all these shenanigans) PyInterpreterState_Clear (ccptr->pInterpState); PyInterpreterState_Delete (ccptr->pInterpState); ccptr->pInterpState = NULL; *****/ } else ccptr->pInterpState = PyInterpreterState_New (); } /*****************************************************************************/ /* Initialise the internal WASD module. Declared before WasdModuleInit() to avoid protyping. */ #if PY_MAJOR_VERSION >= 3 PyObject* WasdModuleInit2 (void) { PyObject* modptr; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "WasdModuleInit2()\n"); modptr = PyModule_Create (&RtePyModuleDef); if (!modptr) return (modptr); PyModule_AddObject (modptr, "__path__", Py_BuildValue("()")); return (modptr); } #else /* PY_MAJOR_VERSION < 3 */ void WasdModuleInit2 (void) { PyObject* modptr; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "WasdModuleInit2()\n"); modptr = Py_InitModule ("wasd", RtePyMethods); if (!modptr) return; PyModule_AddObject (modptr, "__path__", Py_BuildValue("()")); } #endif /* PY_MAJOR_VERSION >= 3 */ /*****************************************************************************/ /* Initialise the internal WASD module. */ int WasdModuleInit (void) { /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "WasdModuleInit()\n"); if (PyImport_AppendInittab ("wasd", &WasdModuleInit2) >= 0) return (1); ReportError (__LINE__, 0, 1, ErrorPython); return (0); } /*****************************************************************************/ /* Set the programme name (doh). */ void SetProgramName (char* name) { wchar_t *wcptr; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "SetProgramName() |%s|\n", name); if (!name) return; #if PY_MAJOR_VERSION >= 3 wcptr = Py_DecodeLocale (name, NULL); Py_SetProgramName (wcptr); PyMem_RawFree (wcptr); #else Py_SetProgramName (name); #endif } /*****************************************************************************/ /* Set the Python argv. */ #define VARRAY_SIZE 32 void SetArgv (char *ScriptName) { int cnt, idx; char *cptr; char *pargv [VARRAY_SIZE]; wchar_t* wargv [VARRAY_SIZE]; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "SetArgv() %d\n", MainArgc); #if PY_MAJOR_VERSION >= 3 if (MainArgc >= VARRAY_SIZE) exit (SS$_BUGCHECK); memset (wargv, 0, sizeof(wargv)); idx = 0; if (ScriptName) wargv[idx++] = Py_DecodeLocale (ScriptName, NULL); for (cnt = 1; cnt < MainArgc && idx < VARRAY_SIZE; cnt++) { if (dbug) fprintf (stdout, "%d |%s|\n", idx, MainArgv[cnt]); if (!MainArgv[cnt]) break; wargv[idx++] = Py_DecodeLocale (MainArgv[cnt], NULL); } for (cnt = 0; (cptr = Trn2Lnm ("PYRTE_ARGV", NULL, cnt)) && idx < VARRAY_SIZE; cnt++) { if (dbug) fprintf (stdout, "%d |%s|\n", idx, cptr); wargv[idx++] = Py_DecodeLocale (cptr, NULL); } PySys_SetArgv (idx, wargv); /* free array of wide strings */ for (idx = 0; idx < VARRAY_SIZE; idx++) { if (!wargv[idx]) continue; if (wargv[idx]) PyMem_RawFree (wargv[idx]); wargv[idx] = NULL; } #else /* PY_MAJOR_VERSION < 3 */ memset (pargv, 0, sizeof(pargv)); idx = 0; if (ScriptName) pargv[idx++] = strdup (ScriptName); for (cnt = 1; cnt < MainArgc && idx < VARRAY_SIZE; cnt++) { if (dbug) fprintf (stdout, "%d |%s|\n", idx, MainArgv[cnt]); if (!MainArgv[cnt]) break; pargv[idx] = strdup (MainArgv[cnt]); } for (cnt = 0; (cptr = Trn2Lnm ("PYRTE_ARGV", NULL, cnt)) && idx < VARRAY_SIZE; cnt++) { if (dbug) fprintf (stdout, "%d |%s|\n", idx, cptr); pargv[idx++] = strdup (cptr); } PySys_SetArgv (idx, pargv); /* free array of strings */ for (idx = 0; idx < VARRAY_SIZE; idx++) { if (!pargv[idx]) continue; if (pargv[idx]) free (pargv[idx]); pargv[idx] = NULL; } #endif /* PY_MAJOR_VERSION >= 3 */ } /*****************************************************************************/ /* Write data to WATCH [x]Script (useful for debugging). */ static PyObject* RteWATCH ( PyObject *self, PyObject *args ) { int DataLength; #if PY_MAJOR_VERSION >= 3 const char *DataPtr; #else char *DataPtr; #endif PyObject *pValue; /*********/ /* begin */ /*********/ /** if (dbug) fprintf (stdout, "RteWATCH()\n"); DEBUGPYOBJ ("self", self) DEBUGPYOBJ ("args", args) **/ if (!CgiPlusEscPtr) { Py_INCREF (Py_None); return (Py_None); } if (!PyArg_ParseTuple (args, "O", &pValue)) return (NULL); if (pValue == Py_None) { AppOutputCount++; fflush (stdout); Py_INCREF (Py_None); return (Py_None); } #if PY_MAJOR_VERSION >= 3 if (PyUnicode_Check (pValue)) #else if (PyString_Check (pValue)) #endif { #if PY_MAJOR_VERSION >= 3 DataPtr = PyUnicode_AsUTF8AndSize (pValue, &DataLength); #else PyString_AsStringAndSize (pValue, &DataPtr, &DataLength); #endif if (dbug) fprintf (stdout, "%d |%s|\n", DataLength, DataPtr); fflush (stdout); fputs (CgiPlusEscPtr, stdout); fflush (stdout); /* the leading '!' indicates we're not going to read the response */ fputs ("!WATCH: WATCH ", stdout); fwrite (DataPtr, DataLength, 1, stdout); fflush (stdout); fputs (CgiPlusEotPtr, stdout); fflush (stdout); Py_INCREF (Py_None); return (Py_None); } PyErr_SetString (PyExc_TypeError, "type must be String or None"); return (NULL); } /*****************************************************************************/ /* Write data to . Write records (as received) if WSGI. Use buffered output, where appropriate, for non-WSGI. */ static PyObject* RteStdoutWrite ( PyObject *self, PyObject *args ) { int isuni, DataLength; char *DataPtr; PyObject *pValue; /*********/ /* begin */ /*********/ /** if (dbug) fprintf (stdout, "RteStdoutWrite()\n"); DEBUGPYOBJ ("self", self) DEBUGPYOBJ ("args", args) **/ if (!PyArg_ParseTuple (args, "O", &pValue)) return (NULL); if (pValue == Py_None) { AppOutputCount++; fflush (stdout); Py_INCREF (Py_None); return (Py_None); } DEBUGPYOBJ ("pValue", pValue) #if PY_MAJOR_VERSION >= 3 if ((isuni = PyUnicode_Check (pValue)) || PyBytes_Check (pValue)) #else if (PyString_Check (pValue)) #endif { #if PY_MAJOR_VERSION >= 3 if (isuni) DataPtr = (char*)PyUnicode_AsUTF8AndSize (pValue, &DataLength); else PyBytes_AsStringAndSize (pValue, &DataPtr, &DataLength); #else PyString_AsStringAndSize (pValue, &DataPtr, &DataLength); #endif if (dbug) fprintf (stdout, "%d |%s|\n", DataLength, DataPtr); /* if the deprecated WSGI write() method is being used */ if (WsgiRunApp && !WsgiHeadersSent) WsgiSendHeaders (); AppOutputCount += DataLength; fwrite (DataPtr, DataLength, 1, stdout); /* WSGI always writes data unbuffered */ if (WsgiRunApp && !WsgiForceBuffering) fflush (stdout); Py_INCREF (Py_None); return (Py_None); } #if PY_MAJOR_VERSION >= 3 PyErr_SetString (PyExc_TypeError, "type must be Unicode, Bytes or None"); #else PyErr_SetString (PyExc_TypeError, "type must be String or None"); #endif return (NULL); } /*****************************************************************************/ /* Populate/depopulate the script's os.environ with CGI variables. Some of the WASD-'standard' CGI variables are excluded. */ int SetupOsEnvCgiVar (int InitEnv) { static char RequestMethodGet[] = "REQUEST_METHOD=GET"; static PyObject *pMainEnvironDict = NULL; int retval; char *cptr, *sptr, *NamePtr; PyObject *pValue, *pKey, *pKeys; Py_ssize_t pKenLen; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "SetupOsEnvCgiVar(%d)\n", InitEnv); if (!HeadCvtGet) HeadCvtGet = (TrnLnm ("PYRTE_HEAD_CVT_GET") != NULL); if (!pMainEnvironDict) { /* get the "original" environment dictionary */ pMainEnvironDict = PyDict_GetItemString (pOsDict, "environ"); if (!pMainEnvironDict) { ReportError (__LINE__, 0, 1, ErrorCgiVarEnv); return (0); } /* borrowed reference */ Py_INCREF (pMainEnvironDict); DEBUGPYOBJ ("pMainEnvironDict", pMainEnvironDict); } if (!InitEnv) { if (pEnvironDict) Py_DECREF (pEnvironDict); pEnvironDict = NULL; return (1); } /* new reference */ DEBUGPYOBJ ("pMainEnvironDict", pMainEnvironDict); pEnvironDict = PyObject_CallMethod(pMainEnvironDict, "copy", "()"); DEBUGPYOBJ ("pEnvironDict", pEnvironDict); while (cptr = CgiVar ("*")) { if (HeadCvtGet && toupper(*cptr) == 'R' && !strcmp (cptr, "REQUEST_METHOD=HEAD")) cptr = RequestMethodGet; for (sptr = cptr; *sptr && *sptr != '='; sptr++); if (!*sptr) continue; *sptr = '\0'; // if (dbug) fprintf (stdout, "|%s|%s|\n", cptr, sptr+1); #if PY_MAJOR_VERSION >= 3 pValue = PyUnicode_FromString (sptr+1); #else /* wsgi complained it was not a string! */ pValue = PyString_FromString (sptr+1); #endif retval = PyMapping_SetItemString (pEnvironDict, cptr, pValue); Py_DECREF (pValue); *sptr = '='; if (retval == -1) { ReportError (__LINE__, 0, 1, ErrorCgiVarEnv); return (0); } } DEBUGPYOBJ ("pEnvironDict", pEnvironDict); retval = PyDict_SetItemString (pOsDict, "environ", pEnvironDict); if (retval == -1) { ReportError (__LINE__, 0, 1, ErrorCgiVarEnv); return (0); } if (dbug) fprintf (stdout, "PyMapping_SetItemString() %d\n", retval); DEBUGPYOBJ ("pOsDict", pOsDict); return (1); } /*****************************************************************************/ /* */ void DebugPyObject ( char *name, PyObject *pObject, int line ) { /*********/ /* begin */ /*********/ if (!dbug) return; fprintf (stdout, "RtePyMethGetVar()\n"); fprintf (stdout, "\n***** %s %d ***** ", name, line); fflush (stdout); if (pObject) { PyObject_Print (pObject, stdout, 0); fputs ("\n", stdout); } else fputs ("(NULL)\n", stdout); fflush (stdout); } /*****************************************************************************/ /* Python method to return the string object value of the specified CGI variable name. An optional second Python function parameter allows a default object to be returned if the CGI variable does not exist. Uses all borrowed references! */ static PyObject* RtePyMethGetVar ( PyObject *self, PyObject *args ) { char *cptr, *sptr; PyObject *pVarDefault = NULL, *pVarName = NULL; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "RtePyMethGetVar()\n"); if (!PyArg_ParseTuple (args, "S|O", &pVarName, &pVarDefault)) return (NULL); #if PY_MAJOR_VERSION >= 3 if (!(cptr = PyBytes_AsString (pVarName))) return (NULL); #else if (!(cptr = PyString_AsString (pVarName))) return (NULL); #endif if (!(sptr = CgiVar (cptr))) { if (pVarDefault) return (pVarDefault); Py_INCREF(Py_None); return (Py_None); } return (Py_BuildValue ("s", sptr)); } /*****************************************************************************/ /* Python method to return the number of requests processed by this cached code. */ static PyObject* RtePyMethUsageCode () { /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "RtePyMethUsageCode()\n"); return (Py_BuildValue ("i", CodeCachePtr->CodeUsageCount)); } /*****************************************************************************/ /* Python method to return the number of requests processed by this cached code. */ static PyObject* RtePyMethUsageThread () { /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "RtePyMethUsageThread()\n"); return (Py_BuildValue ("i", CodeCachePtr->ThreadUsageCount)); } /*****************************************************************************/ /* Python method to return the number of requests processed by the RTE. */ static PyObject* RtePyMethUsageRte () { /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "RtePyMethUsageRte()\n"); return (Py_BuildValue ("i", PyrteUsageCount)); } /*****************************************************************************/ /* Return a string object containing the LIB$STAT_TIMER statistics. */ static PyObject* RtePyMethStatTimer () { /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "RtePyMethStatTimer()\n"); return (Py_BuildValue ("s", StatTimer (TRUE))); } /*****************************************************************************/ /* Wait for next request. */ void CgiPlusBegin () { /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "CgiPlusBegin()\n"); if (CgiPlusState) { ReportError (__LINE__, SS$_BUGCHECK, 0, NULL); exit (SS$_BUGCHECK); } /* first one (zeroeth) is always available */ if (PyrteUsageCount++) CgiVar (""); CgiPlusState++; DbugCheck (1); AppOutputCount = 0; sys$gettim (&CurrentBinTime); CurrentTimeSecs = decc$fix_time (&CurrentBinTime); RteScriptParam (); if (!SetupOsEnvCgiVar (1)) { ReportError (__LINE__, SS$_BUGCHECK, 0, NULL); exit (SS$_BUGCHECK); } } /*****************************************************************************/ /* Python method to wait for the next CGIplus request. If an RTE caching or non-caching RTE execution just return True. If a CGIplus request has been begun but not explicitly ended this function provides the WASD script EOF sentinal. It then blocks waiting for the next request. If not CGIplus (i.e. standard CGI or RTE) the function returns True immediately and if called again False. That is, a one-shot execution. The function then returns to the caller which with RTE waits for the next request. With CGI that's all folks. The function usage accepts zero, one or two parameters. The first is a boolean (integer 0 or 1) that if True allows a standard CGI script to call this once before returning False. The second parameter is also a boolean (integer 0 or 1). If True it gets the script code modification time to check for changes. If modified it returns False. */ static PyObject* RtePyMethCgiPlusBegin ( PyObject *self, PyObject *args ) { static int OneShot; int AllowSingleCGI = 0, CheckCodeMtime = 0; char *cptr, *sptr; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "RtePyMethCgiPlusBegin() %u %u %d %d %d %d %d\n", self, args, CgiPlusState, IsScriptRte, IsCgiPlusScript, PyrteUsageCount, WsgiRunApp); if (WsgiRunApp) { Py_INCREF (Py_None); return (Py_None); } if (!(IsScriptRte || IsCgiPlusScript)) { /* call from Python script inside standard CGI; allow one off */ if (!PyArg_ParseTuple (args, "|i", &AllowSingleCGI)) return (NULL); if (dbug) fprintf (stdout, "AllowSingleCGI:%d\n", AllowSingleCGI); if (dbug) fprintf (stdout, "OneShot: %d %d\n", OneShot, AllowSingleCGI); if (!OneShot && AllowSingleCGI) { OneShot = 1; Py_INCREF(Py_True); return (Py_True); } if (dbug) fprintf (stdout, "OneShot: %d\n", OneShot); OneShot = 0; Py_INCREF(Py_False); return (Py_False); } if (dbug) fprintf (stdout, "IsCgiPlusScript: %d %d\n", IsCgiPlusScript, CgiPlusState); if (IsCgiPlusScript) { if (!CgiPlusState) /* wait for next request */ CgiPlusBegin (); else CgiPlusEnd (); Py_INCREF(Py_True); return (Py_True); } /* while wasd.cgiplus_begin(True): first is True then False */ if (CgiPlusState++ == 1) { /* first True */ Py_INCREF(Py_True); return (Py_True); } else { /* then False */ Py_INCREF(Py_False); return (Py_False); } } /*****************************************************************************/ /* Provide the end-of-request sentinal. */ void CgiPlusEnd () { /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "CgiPlusEnd() %d\n", CgiPlusState); if (!CgiPlusState) { ReportError (__LINE__, SS$_BUGCHECK, 0, NULL); exit (SS$_BUGCHECK); } DbugCheck (0); CgiPlusState = 0; fflush (stdout); fputs (CgiPlusEofPtr, stdout); fflush (stdout); SetupOsEnvCgiVar (0); } /*****************************************************************************/ /* It's possible to use this function to explicity end a CGIplus script's request processing. If it's not used then the cgiplus_begin() function does all of this instead. It accepts zero or one parameters. The parameter is a boolean (integer 0 or 1). If True it gets the script code modification time to check for changes. If modified it return False. */ static PyObject* RtePyMethCgiPlusEnd ( PyObject *self, PyObject *args ) { int CheckCodeMtime = 0; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "RtePyMethCgiPlusEnd() %u %u %d\n", self, args, CgiPlusState); if (WsgiRunApp) { /* not available to WSGI applications */ Py_INCREF (Py_None); return (Py_None); } if (!IsCgiPlusScript || !CgiPlusState) { Py_INCREF(Py_False); return (Py_False); } if (!CgiPlusState) { Py_INCREF(Py_False); return (Py_False); } CgiPlusEnd (); if (!PyArg_ParseTuple (args, "|i", &CheckCodeMtime)) return (NULL); if (CheckCodeMtime) { if (!ByteCodeUpToDate (CodeCachePtr)) { Py_INCREF(Py_False); return (Py_False); } } Py_INCREF(Py_True); return (Py_True); } /*****************************************************************************/ /* Full CGI callout including response as available. */ static PyObject* RtePyMethCgiCallout ( PyObject *self, PyObject *args ) { int retval; char *sptr; char buf [1024]; PyObject* pReturn; if (!PyArg_ParseTuple (args, "s", &sptr)) return (NULL); fflush(stdout); fputs (CgiPlusEscPtr, stdout); fflush(stdout); fputs (sptr, stdout); fflush(stdout); if (*sptr == '!' || *sptr == '#') { Py_INCREF (Py_None); pReturn = Py_None; } else { /* with plain old CGI might not already be open */ if (!CgiPlusInFp) #if PY_MAJOR_VERSION >= 3 /* DECC$DISABLE_TO_VMS_LOGNAME_TRANSLATION */ if (!(CgiPlusInFp = fopen ("/CGIPLUSIN", "r"))) #else if (!(CgiPlusInFp = fopen (TrnLnm("CGIPLUSIN"), "r"))) #endif exit (vaxc$errno); rewind (CgiPlusInFp); retval = read (fileno(CgiPlusInFp), buf, sizeof(buf)); if (retval < 0) { Py_INCREF (Py_None); pReturn = Py_None; } else { /* excise the (CRTL) trailing newline */ if (retval) retval--; pReturn = PyUnicode_FromStringAndSize (buf, retval); } } fputs (CgiPlusEotPtr, stdout); fflush(stdout); return (pReturn); } /*****************************************************************************/ /* Read a record directly from CGIPLUSIN. Optional parameter specifies non-default (255 bytes) size of buffer. */ static PyObject* RtePyMethReadCgiPlusIn ( PyObject *self, PyObject *args ) { int bsize, retval; char *bptr; PyObject* pReturn; bsize = 255; if (!PyArg_ParseTuple(args, "|i", &bsize)) return (NULL); bptr = calloc (1, bsize+32); rewind (CgiPlusInFp); retval = read (fileno(CgiPlusInFp), bptr, bsize); if (retval < 0) { Py_INCREF (Py_None); pReturn = Py_None; } else { /* excise the (CRTL) trailing newline */ if (retval) retval--; pReturn = PyUnicode_FromStringAndSize (bptr, retval); } free (bptr); return (pReturn); } /*****************************************************************************/ /* Set whether the Python thread being used is disposed of at the end of the current request (0 or False) or resued for the next request (1 or True). */ static PyObject* RtePyMethReuseThread ( PyObject *self, PyObject *args ) { /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "RtePyMethReuseThread()\n"); if (!PyArg_ParseTuple (args, "i", &ReuseThread)) return (NULL); Py_INCREF(Py_None); return (Py_None); } /*****************************************************************************/ /* Return a string object containing details of the pyRTE version and build date. */ static PyObject* RtePyMethRteId () { char RteIdBuf [256]; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "RtePyMethRteId()\n"); sprintf (RteIdBuf, "%s, %s", SoftwareId, BUILD_DATETIME); return (Py_BuildValue ("s", RteIdBuf)); } /*****************************************************************************/ /* Return a string object containing details of the particular cache entry as a series of comma-separated-values (CSV) in the same order as the data structure. Note that this function will not succeed without logical name PYRTE_CACHE_ENTRY being defined. */ static PyObject* RtePyMethRteCacheEntry ( PyObject *self, PyObject *args ) { static int idx = 0; char StringBuf [512]; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "RtePyMethRteCacheEntry()\n"); if (!TrnLnm ("PYRTE_CACHE_ENTRY")) { PyErr_SetString (PyExc_RuntimeError, "logical name PYRTE_CACHE_ENTRY not defined"); return (NULL); } if (!PyArg_ParseTuple (args, "|i", &idx)) return (NULL); for (;;) { if (idx < 0 || idx >= CodeCacheCount) { idx = 0; Py_INCREF(Py_None); return (Py_None); } if (CodeCache[idx].FileNamePtr) break; idx++; } sprintf (StringBuf, "%u,%u,%u,%u,%s,%s,%u", CodeCache[idx].CodeUsageCount, CodeCache[idx].ThreadUsageCount, CodeCache[idx].LastUsedSecs, CodeCache[idx].ScriptMtimeSecs, CodeCache[idx].FileNamePtr, CodeCache[idx].FileNameByteCodePtr ? CodeCache[idx].FileNameByteCodePtr : "(null)", (unsigned long)CodeCache[idx].pByteCode); idx++; return (Py_BuildValue ("s", StringBuf)); } /*****************************************************************************/ /* WSGI method to run a WSGI function (application). */ static PyObject* WsgiRun ( PyObject *self, PyObject *args ) { PyObject *pAppFunc = NULL, *pStartResponse, *pResult; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "WsgiRun()\n"); WsgiRunApp = 1; DEBUGPYOBJ ("self", self) DEBUGPYOBJ ("args", args) if (!PyArg_ParseTuple (args, "O", &pAppFunc)) return (NULL); if (PyCallable_Check (pAppFunc)) { WsgiOsEnvVar (); /* new reference */ pStartResponse = PyObject_GetAttrString (pWasdModule, "start_response"); pResult = PyObject_CallFunctionObjArgs (pAppFunc, pEnvironDict, pStartResponse, NULL); if (pResult) { WsgiSendResponse (pResult); Py_DECREF (pResult); } else ReportError (__LINE__, 0, 1, NULL); /* just in case the application staggered somewhere */ if (pWsgiStatus) Py_DECREF (pWsgiStatus); if (pWsgiHeaders) Py_DECREF (pWsgiHeaders); pWsgiHeaders = pWsgiStatus = NULL; WsgiHeadersSent = 0; } else ReportError (__LINE__, 0, 1, NULL); WsgiRunApp = 0; Py_INCREF(Py_None); return (Py_None); } /*****************************************************************************/ /* WSGI start HTTP response function. */ static PyObject* WsgiStartResponse ( PyObject *self, PyObject *args ) { PyObject *pHeaders, *pStatus, *pValue; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "WsgiStartResponse()\n"); if (!WsgiRunApp) { /* not available to non-WSGI applications */ Py_INCREF (Py_None); return (Py_None); } if (!PyArg_ParseTuple (args, "OO|O:start_response", &pStatus, &pHeaders, &pWsgiExcInfo)) return (NULL); if (pWsgiExcInfo && pWsgiExcInfo != Py_None) { if (WsgiHeadersSent) { PyObject *pErrType = NULL, *pErrValue = NULL, *pErrTraceback = NULL; if (!PyArg_ParseTuple (pWsgiExcInfo, "OOO:", &pErrType, &pErrValue, &pErrTraceback)) return (NULL); Py_INCREF (pErrType); Py_INCREF (pErrValue); Py_INCREF (pErrTraceback); PyErr_Restore (pErrType, pErrValue, pErrTraceback); return (NULL); } /* allowed change of response */ if (pWsgiStatus) Py_DECREF (pWsgiStatus); if (pWsgiHeaders) Py_DECREF (pWsgiHeaders); pWsgiStatus = pWsgiHeaders = NULL; } else if (WsgiHeadersSent) { PyErr_SetString (PyExc_RuntimeError, "headers have already been sent"); return (NULL); } if (pWsgiStatus && pWsgiHeaders) { PyErr_SetString (PyExc_RuntimeError, "start_response() already called"); return (NULL); } pWsgiStatus = pStatus; Py_INCREF (pWsgiStatus); pWsgiHeaders = pHeaders; Py_INCREF (pWsgiHeaders); DEBUGPYOBJ ("pWsgiStatus", pWsgiStatus) DEBUGPYOBJ ("pWsgiHeaders", pWsgiHeaders) return (PyObject_GetAttrString (pWasdModule, "write")); } /*****************************************************************************/ /* */ int WsgiSendResponse (PyObject *pResponse) { int cnt, DataLength; #if PY_MAJOR_VERSION >= 3 const char *DataPtr; #else char *DataPtr; #endif PyObject *pItem, *pIter; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "WsgiSendResponse() 0x%08.08x\n", pResponse); if (!WsgiRunApp || !pResponse) return (1); DEBUGPYOBJ ("pResponse", pResponse) #if PY_MAJOR_VERSION >= 3 if (PyUnicode_Check (pResponse)) #else if (PyString_Check (pResponse)) #endif { #if PY_MAJOR_VERSION >= 3 DataPtr = PyUnicode_AsUTF8AndSize (pResponse, &DataLength); #else PyString_AsStringAndSize (pResponse, &DataPtr, &DataLength); #endif if (dbug) fprintf (stdout, "%d |%s|\n", DataLength, DataPtr); AppOutputCount += DataLength; fwrite (DataPtr, DataLength, 1, stdout); return (1); } pIter = PyObject_GetIter (pResponse); if (!pIter) return (1); while (pItem = PyIter_Next(pIter)) { if (!WsgiHeadersSent) WsgiSendHeaders (); #if PY_MAJOR_VERSION >= 3 if (PyUnicode_Check (pItem)) #else if (PyString_Check (pItem)) #endif #if PY_MAJOR_VERSION >= 3 if (DataPtr = PyUnicode_AsUTF8AndSize (pItem, &DataLength)) #else if (!PyString_AsStringAndSize (pItem, &DataPtr, &DataLength)) #endif if (DataPtr && DataLength) { /** if (dbug) fprintf (stdout, "pIter()\n"); **/ AppOutputCount += DataLength; fwrite (DataPtr, DataLength, 1, stdout); if (!WsgiForceBuffering) fflush (stdout); } Py_DECREF (pItem); } Py_DECREF (pIter); /* If the response was empty, we have to send the headers because the preceding loop was not entered */ if (!WsgiHeadersSent) WsgiSendHeaders (); return (1); } /*****************************************************************************/ /* */ int WsgiSendHeaders () { int StatusCode, StatusLength; char *NamePtr, *ValuePtr; #if PY_MAJOR_VERSION >= 3 const char *StatusPtr; #else char *StatusPtr; #endif PyObject *pItem, *pIter; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "WsgiSendHeaders() 0x%08.08x 0x%08.08x\n", pWsgiStatus, pWsgiHeaders); if (!pWsgiHeaders || !pWsgiStatus) return (0); #if PY_MAJOR_VERSION >= 3 if (!PyUnicode_Check (pWsgiStatus)) #else if (!PyString_Check (pWsgiStatus)) #endif { ReportError (__LINE__, 0, 1, NULL); return (0); } WsgiHeadersSent = 1; #if PY_MAJOR_VERSION >= 3 StatusPtr = PyUnicode_AsUTF8AndSize (pWsgiStatus, &StatusLength); #else StatusPtr = PyString_AsString (pWsgiStatus); #endif StatusCode = atoi(StatusPtr); if (!StatusCode || strlen(StatusPtr) < 5) return (0); StatusPtr += 4; AppOutputCount += fprintf (stdout, "Status: %d %s\r\n", StatusCode, StatusPtr); pIter = PyObject_GetIter (pWsgiHeaders); if (!pIter) return (0); while (pItem = PyIter_Next(pIter)) { if (PyArg_ParseTuple (pItem, "ss", &NamePtr, &ValuePtr)) AppOutputCount += fprintf (stdout, "%s: %s\r\n", NamePtr, ValuePtr); Py_DECREF (pItem); } Py_DECREF (pIter); fputs ("\r\n", stdout); AppOutputCount += 2; fflush (stdout); Py_DECREF (pWsgiStatus); Py_DECREF (pWsgiHeaders); pWsgiHeaders = pWsgiStatus = NULL; return (1); } /*****************************************************************************/ /* Populate the script's os.environ with WSGI variables. */ int WsgiOsEnvVar () { int retval; char *cptr, *sptr; PyObject *pSysModule, *pValue; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "WsgiOsEnvVar()\n"); pValue = Py_BuildValue ("(ii)", 1, 0); retval = PyMapping_SetItemString (pEnvironDict, "wsgi.version", pValue); if (dbug) fprintf (stdout, "retval: %d\n", retval); Py_DECREF(pValue); cptr = CgiVar ("REQUEST_SCHEME"); if (!*cptr || !strcmp (cptr, "http:")) pValue = PyUnicode_FromString ("http"); else pValue = PyUnicode_FromString ("https"); PyMapping_SetItemString (pEnvironDict, "wsgi.url_scheme", pValue); Py_DECREF(pValue); pSysModule = PyImport_AddModule ("sys"); if (!pSysModule) { ReportError (__LINE__, 0, 1, ErrorWsgiEnv); return (0); } pValue = PyObject_GetAttrString (pSysModule, "stdin"); if (!pValue) { ReportError (__LINE__, 0, 1, ErrorWsgiEnv); return (0); } PyMapping_SetItemString (pEnvironDict, "wsgi.input", pValue); Py_DECREF(pValue); pValue = PyObject_GetAttrString (pSysModule, "stderr"); if (!pValue) { ReportError (__LINE__, 0, 1, ErrorWsgiEnv); return (0); } PyMapping_SetItemString (pEnvironDict, "wsgi.errors", pValue); Py_DECREF(pValue); PyMapping_SetItemString (pEnvironDict, "wsgi.multithread", Py_False); PyMapping_SetItemString (pEnvironDict, "wsgi.multiprocess", Py_True); PyMapping_SetItemString (pEnvironDict, "wsgi.run_once", Py_False); DEBUGPYOBJ ("pEnvironDict", pEnvironDict) return (1); } /*****************************************************************************/ /* Jean-Francois Pieronne's original '_wasd' module functions (massaged just a little :-) */ static PyObject* wasd_cgi_init () { Py_INCREF(Py_None); return Py_None; } static PyObject* wasd_isCgiPlus ( PyObject *self, PyObject *args ) { #if PY_MAJOR_VERSION >= 3 if (!PyArg_ParseTuple (args, "")) return (NULL); #else if (!PyArg_NoArgs(args)) return (NULL); #endif if (IsCgiPlusScript) #if PY_MAJOR_VERSION >= 3 return (PyObject *)PyLong_FromLong(1L); #else return (PyObject *)PyInt_FromLong(1L); #endif else { Py_INCREF(Py_None); return Py_None; } } static PyObject* wasd_cgi_eof ( PyObject *self, PyObject *args ) { #if PY_MAJOR_VERSION >= 3 if (!PyArg_ParseTuple (args, "")) return (NULL); #else if (!PyArg_NoArgs(args)) return (NULL); #endif if (IsCgiPlusScript) { fflush(stdout); fprintf (stdout, "%s\n", CgiPlusEofPtr); fflush(stdout); } Py_INCREF(Py_None); return (Py_None); } static PyObject* wasd_cgi_info ( PyObject *self, PyObject *args ) { int s; char* buffer; char* name; PyObject* res; if (!PyArg_ParseTuple (args, "s", &name)) return (NULL); buffer = CgiVar(name); if (buffer == NULL) { Py_INCREF(Py_None); res = Py_None; } else #if PY_MAJOR_VERSION >= 3 res = PyBytes_FromString (buffer); #else res = PyString_FromString(buffer); #endif return (res); } static PyObject* wasd_cgi_read ( PyObject *self, PyObject *args ) { int retval, BufferSize = 0; char *cptr, *BodyPtr = NULL; int BodyLength; PyObject* pBody; if (!PyArg_ParseTuple(args, "i", &BufferSize)) return (NULL); cptr = CgiVar("CONTENT_LENGTH"); if (cptr && isdigit(*cptr)) BufferSize = atoi(cptr); BodyPtr = cptr = calloc (1, BufferSize+32); if (!BodyPtr) return (NULL); while (BufferSize) { retval = read (fileno(stdin), BodyPtr, BufferSize); if (retval <= 0) break; cptr += retval; BufferSize -= retval; } BodyLength = cptr - BodyPtr; if (BodyLength == 0) pBody = PyUnicode_FromString(""); else if (BodyLength == strlen (BodyPtr)) pBody = PyUnicode_FromString (BodyPtr); else pBody = PyUnicode_FromStringAndSize (BodyPtr, BodyLength); if (BodyPtr) free (BodyPtr); return (pBody); } /*****************************************************************************/ /* */ void RteScriptParam () { char *cptr; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "RteScriptParam()\n"); cptr = CgiVar ("PYRTE"); if (!cptr) return; NoStreamMode = HeadCvtGet = 0; while (*cptr) { while (*cptr && *cptr != '/') cptr++; if (!*cptr) break; if (!strncasecmp (cptr, "/wsgi=buffer", 12)) WsgiForceBuffering = 1; else if (!strncasecmp (cptr, "/reuse", 6)) ReuseThread = 1; else if (!strncasecmp (cptr, "/noreuse", 8)) ReuseThread = 0; else if (!strncasecmp (cptr, "/nostream", 9)) NoStreamMode = 1; else if (!strncasecmp (cptr, "/head=get", 9)) HeadCvtGet = 1; else if (!strncasecmp (cptr, "/dbug", 5)) dbug = 1; cptr++; } } /*****************************************************************************/ /* */ int DbugCheck (int enable) { char *cptr, *sptr; /*********/ /* begin */ /*********/ if (!enable) return (dbug = 0); if (cptr = TrnLnm ("PYRTE$DBUG")) { dbug = 1; if (!strchr (cptr, '*')) if (sptr = CgiVar ("SCRIPT_NAME")) if (!strstr (sptr, cptr)) dbug = 0; } if (!dbug) return (0); //putenv("PYTHONHOME="); //putenv("PYTHONPATH="); fprintf (stdout, "Script-Control: X-buffer-records\n\ Content-Type: text/plain\n\ \n\ ***** %s (%s) Python %s *****\n", SoftwareId, BUILD_DATETIME, Py_GetVersion()); fflush (stdout); NoStreamMode = 1; } /****************************************************************************/ /* Read the file contents specified by 'FileName' into memory, set the pointer at 'FileTextPtr' to the contents and the file size at 'FileSizePtr'. Returns a VMS status value that should be checked. */ int ReadFileIntoMemory ( char *FileName, char **FileTextPtr, int *FileSizePtr ) { int status, Bytes, BytesRemaining, BufferCount; char *cptr, *sptr, *BufferPtr, *LinePtr; FILE *FilePtr; stat_t StatBuffer; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "ReadFileIntoMemory() |%s|\n", FileName); if (FileTextPtr != NULL) *FileTextPtr = NULL; if (FileSizePtr != NULL) *FileSizePtr = 0; if (stat (FileName, &StatBuffer) < 0) { status = vaxc$errno; if (dbug) fprintf (stdout, "stat() %%X%08.08X\n", status); return (status); } if (StatBuffer.st_fab_rfm == FAB$C_VAR || StatBuffer.st_fab_rfm == FAB$C_VFC) FilePtr = fopen (FileName, "r", "shr=put"); else FilePtr = fopen (FileName, "r", "shr=put", "ctx=bin"); if (FilePtr == NULL) { status = vaxc$errno; if (dbug) fprintf (stdout, "fopen() %%X%08.08X\n", status); return (status); } Bytes = StatBuffer.st_size; if (dbug) fprintf (stdout, "%d bytes\n", Bytes); /* a little margin for error ;^) */ BufferPtr = calloc (1, Bytes+32); if (!BufferPtr) return (vaxc$errno); BufferCount = 0; if (StatBuffer.st_fab_rfm == FAB$C_VAR || StatBuffer.st_fab_rfm == FAB$C_VFC) { BytesRemaining = Bytes; LinePtr = BufferPtr; while (fgets (LinePtr, BytesRemaining, FilePtr) != NULL) { /** if (dbug) fprintf (stdout, "|%s|\n", LinePtr); **/ if (!*LinePtr) break; for (cptr = LinePtr; *cptr; cptr++); BufferCount += cptr - LinePtr; BytesRemaining -= cptr - LinePtr; LinePtr = cptr; } } else { status = fread (BufferPtr, Bytes, 1, FilePtr); if (status == 1) BufferCount = Bytes; } fclose (FilePtr); if (StatBuffer.st_fab_rfm == FAB$C_STMLF) { /* text file, newlines only (check first 512 characters and quit) */ for (cptr = BufferPtr; *cptr && cptr < BufferPtr+512; cptr++) if (*(USHORTPTR)cptr == '\r\n') { sptr = cptr; while (*cptr) { if (*(USHORTPTR)cptr == '\r\n') cptr++; *sptr++ = *cptr++; } *sptr = '\0'; BufferCount = sptr - BufferPtr; break; } } if (dbug) fprintf (stdout, "%d |%s|\n", BufferCount, BufferPtr); if (FileTextPtr != NULL) *FileTextPtr = BufferPtr; if (FileSizePtr != NULL) *FileSizePtr = BufferCount; return (SS$_NORMAL); } /*****************************************************************************/ /* Self-contained functionality. Designed to be called if the script is not found. If found the script needs to instantiate whatever resources it is proctored for and then return an HTTP 204 to the server. If not found then call this function and if this is the first call then check if there is a REQUEST_METHOD. If there is then return false. If not assume WASD is proactively starting the RTE. Then respond with an HTTP 204 and return true. If the calling routine receives a false then it continues processing, if a true then it concludes and waits for the next request. */ int ProctorDetect () { static int DetectCount; char *cptr; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "ProctorDetect()\n"); if (DetectCount++) return (0); /* if this CGI variable does not exist then probably not scripting */ cptr = CgiVar ("SERVER_SOFTWARE"); if (!(cptr && *cptr)) return (0); cptr = CgiVar ("REQUEST_METHOD"); if (cptr && *cptr) return (0); fputs ("Status: 204 RTE Proctor Response\r\n\r\n", stdout); return (1); } /*****************************************************************************/ /* Make it look a bit like the standard WASD error message. */ int ReportError ( int SourceCodeLine, int VmsStatus, int PythonErrorPrint, char *FaoString, ... ) { static int ErrorReported; static FILE *CgiPlusIn; int argcnt, status, CgiLoaded, CgiTbLoaded, HttpStatus; unsigned short ShortLen; char *cptr, *sptr; char Buffer [2048], FaoBuffer [1024], Format [256]; $DESCRIPTOR (BufferDsc, Buffer); $DESCRIPTOR (FaoBufferDsc, FaoBuffer); $DESCRIPTOR (FaoStringDsc, ""); unsigned long *vecptr; unsigned long FaoVector [32]; va_list argptr; PyObject *pObject; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "ReportError() %d %%X%08.08X %d\n", SourceCodeLine, VmsStatus, PythonErrorPrint); if (!SourceCodeLine) { /* return if reported error while resetting */ status = ErrorReported; ErrorReported = 0; return (status); } CgiLoaded = CgiTbLoaded = 0; if (Py_IsInitialized()) { /* assume that if it's loaded it's also enabled */ pObject = PyUnicode_FromString ("cgitb"); if (pObject) { if (pMainDict && PyDict_Contains (pMainDict, pObject) == 1) CgiTbLoaded = 1; Py_DECREF (pObject); } pObject = PyUnicode_FromString ("cgi"); if (pObject) { if (pMainDict && PyDict_Contains (pMainDict, pObject) == 1) CgiLoaded = 1; Py_DECREF (pObject); } } if (dbug) fprintf (stdout, "CgiLoaded:%d CgiTbLoaded:%d\n", CgiLoaded, CgiTbLoaded); ErrorReported = 1; HttpStatus = 0; if (ServerSoftware() >= 110502) { /* for WASD v11.5.2 and later detect if there has been a response */ if (!CgiPlusIn) #if PY_MAJOR_VERSION >= 3 /* DECC$DISABLE_TO_VMS_LOGNAME_TRANSLATION */ /* DECC$MAILBOX_CTX_STM */ if (!(CgiPlusIn = fopen ("/CGIPLUSIN", "r", "rat=cr"))) #else if (!(CgiPlusIn = fopen ("CGIPLUSIN:", "r"))) #endif exit (vaxc$errno); fflush (stdout); fputs (CgiPlusEscPtr, stdout); fflush (stdout); fputs ("HTTP-STATUS:", stdout); fflush (stdout); fgets (Buffer, sizeof(Buffer), CgiPlusIn); fputs (CgiPlusEotPtr, stdout); fflush (stdout); if (atoi(Buffer) == 200) HttpStatus = atoi(Buffer+4); } if (CgiTbLoaded) { if (!HttpStatus) fputs ("Status: 502\nContent-Type: text/html\n\n", stdout); PyErr_Print(); fprintf (stdout, "\n", SourceCodeLine); return (ErrorReported); } va_count (argcnt); vecptr = FaoVector; va_start (argptr, FaoString); for (argcnt -= 1; argcnt; argcnt--) *vecptr++ = (unsigned long)va_arg (argptr, unsigned long); va_end (argptr); if (FaoString) { FaoStringDsc.dsc$a_pointer = FaoString; FaoStringDsc.dsc$w_length = strlen(FaoString); status = sys$faol (&FaoStringDsc, &ShortLen, &BufferDsc, (unsigned long*)&FaoVector); if (dbug) fprintf (stdout, "sys$fao() %%X%08.08X\n", status); if (status & 1) Buffer[ShortLen] = '\0'; else sprintf (Buffer, "SYS$FAO() %%X%08.08X", status); } else Buffer[0] = '\0'; if (AppOutputCount) { fprintf (stdout, "\n\n", SourceCodeLine); if (VmsStatus) fprintf (stdout, "\n", VmsStatus); if (!CgiTbLoaded) fputs ("
\n----------\n", stdout);
      PyErr_Print();
      if (!CgiTbLoaded) fputs ("----------\n
\n", stdout); return (ErrorReported); } if (!HttpStatus) { fputs ("Status: 502\nContent-Type: text/html\n\n", stdout); fprintf (stdout, "\n\ \n\ \n\ \n\ \n\ ERROR 502 Bad Gateway\n\ \n\ \n\ ERROR 502  -  \ External agent did not respond (or not acceptably)\n", SoftwareId, SourceCodeLine); } if (VmsStatus) fprintf (stdout, "\n", VmsStatus); if (Buffer[0]) fprintf (stdout, "

%s", Buffer); if (PythonErrorPrint) { fprintf (stdout, "

");
      PyErr_Print();
      fprintf (stdout, "
\n"); } cptr = CgiVar ("SERVER_SIGNATURE"); fprintf (stdout, "


\n\ %s%s\ \n\ \n", cptr ? cptr : "", cptr ? "\n" : ""); return (ErrorReported); } /*****************************************************************************/ /* Return the value of a CGI variable regardless of whether it is used in a standard CGI environment or a WASD CGIplus (RTE) environment. Also automatically switches WASD V7.2 and later servers into 'struct' mode for significantly improved performance. WASD by default supplies CGI variables prefixed by "WWW_" to differentiate them from any other DCL symbols (or "env"ironment logicals). This is automatically detected if present and ignored. WASD v12.1.1 and later provides the CGI variable (symbol) GATEWAY_SYMBOLS with a comma-separated list of DCL symbol names used for the CGI environment. */ char* CgiVar (char *VarName) { # ifndef CGIVAR_STRUCT_SIZE # define CGIVAR_STRUCT_SIZE 8192 # endif # define SOUS sizeof(unsigned short) static int CalloutDone, DclSymbol, StructLength, WwwLength; static char *NextVarNamePtr; static char NameBuffer [256], StructBuffer [CGIVAR_STRUCT_SIZE], SymbolsBuffer [4096], NameValueBuffer [4096]; static FILE *CgiPlusIn; static $DESCRIPTOR (NameDsc, NameValueBuffer); static $DESCRIPTOR (SymbolsDsc, SymbolsBuffer); static $DESCRIPTOR (NameValueDsc, NameValueBuffer); static $DESCRIPTOR (GatewayInterfaceDsc, "GATEWAY_INTERFACE"); static $DESCRIPTOR (GatewaySymbolsDsc, "GATEWAY_SYMBOLS"); static $DESCRIPTOR (WwwGatewayInterfaceDsc, "WWW_GATEWAY_INTERFACE"); static $DESCRIPTOR (WwwGatewaySymbolsDsc, "WWW_GATEWAY_SYMBOLS"); int status; int Length; unsigned short slen; char *bptr, *cptr, *sptr, *zptr; /*********/ /* begin */ /*********/ if (dbug && VarName && VarName[0] && VarName[0] != '*') fprintf (stdout, "CgiVar() |%s|\n", !VarName ? "NULL" : VarName); if (!DclSymbol) { /*************************/ /* check for DCL symbols */ /*************************/ DclSymbol = 1; status = lib$get_symbol (&GatewayInterfaceDsc, &NameValueDsc, 0, 0); if (!(status & 1)) status = lib$get_symbol (&WwwGatewayInterfaceDsc, &NameValueDsc, 0, 0); if (dbug) fprintf (stdout, "lib$get_symbol %%X%08.08X\n", status); if (status & 1) { /* CGI symbol detected, therefore CGI request */ IsCgiPlus = 0; status = lib$get_symbol (&GatewaySymbolsDsc, &SymbolsDsc, &slen, 0); if (!(status & 1)) status = lib$get_symbol (&WwwGatewaySymbolsDsc, &SymbolsDsc, &slen, 0); if (status & 1) SymbolsBuffer[slen] = '\0'; else SymbolsBuffer[0] = '\0'; if (dbug) fprintf (stdout, "|%s|\n", SymbolsBuffer); if (getenv ("PYRTE$NOSYMBOLS")) SymbolsBuffer[0] = '\0'; } } if (!VarName || !VarName[0]) { /******************/ /* (re)initialize */ /******************/ StructLength = 0; if (SymbolsBuffer[0]) NextVarNamePtr = SymbolsBuffer; else if (IsCgiPlus) NextVarNamePtr = StructBuffer; else NextVarNamePtr = DclSymbolNames(); if (dbug) fprintf (stdout, "NextVarNamePtr |%s|\n", NextVarNamePtr); if (!VarName) return (NULL); } WwwLength = 0; if (!IsCgiPlus) { /************************/ /* standard CGI symbols */ /************************/ if (VarName[0] == '*') { /* return each CGIplus variable in successive calls */ if (!*NextVarNamePtr) { if (SymbolsBuffer[0]) NextVarNamePtr = SymbolsBuffer; else NextVarNamePtr = DclSymbolNames(); if (dbug) fprintf (stdout, "CGI |NULL|\n"); return (NULL); } for (cptr = sptr = NextVarNamePtr; *sptr && *sptr != ','; sptr++); while (*sptr && *sptr != ',') sptr++; if (*sptr) sptr++; NextVarNamePtr = sptr; } else cptr = VarName; zptr = (sptr = NameValueBuffer) + sizeof(NameValueBuffer)-1; while (*cptr && *cptr != ',' && sptr < zptr) *sptr++ = *cptr++; *sptr = '\0'; NameDsc.dsc$w_length = sptr - NameValueBuffer; NameValueDsc.dsc$a_pointer = sptr + 1; NameValueDsc.dsc$w_length = sizeof(NameValueBuffer) - NameDsc.dsc$w_length - 2; status = lib$get_symbol (&NameDsc, &NameValueDsc, &slen, 0); if (!(status & 1)) return (NULL); NameValueBuffer[NameDsc.dsc$w_length] = '='; NameValueDsc.dsc$a_pointer[slen] = '\0'; if (*((unsigned long*)NameValueBuffer) == 'WWW_') WwwLength = 4; return (NameValueBuffer + WwwLength); } /***********/ /* CGIplus */ /***********/ if (VarName && VarName[0]) { /***************************/ /* return a variable value */ /***************************/ if (dbug) fprintf (stdout, "StructLength %d\n", StructLength); if (!StructLength) return (NULL); if (VarName[0] == '*') { /* return each CGIplus variable in successive calls */ if (!(Length = *(unsigned short*)NextVarNamePtr)) { NextVarNamePtr = StructBuffer; if (dbug) fprintf (stdout, "CGIplus |NULL|\n"); return (NULL); } /* by default WASD CGI variable name are prefixed by "WWW_", ignore */ sptr = (NextVarNamePtr += SOUS); NextVarNamePtr += Length; if (*((unsigned long*)sptr) == 'WWW_') WwwLength = 4; if (dbug) fprintf (stdout, "CGIplus |%s|\n", sptr + WwwLength); return (sptr + WwwLength); } /* by default WASD CGI variable name are prefixed by "WWW_", ignore */ if (*((unsigned long*)(StructBuffer+SOUS)) == 'WWW_') WwwLength = 4; /* return a pointer to this CGIplus variable's value */ for (bptr = StructBuffer; Length = *(unsigned short*)bptr; bptr += Length) { sptr = (bptr += SOUS) + WwwLength; for (cptr = VarName; *cptr && *sptr && *sptr != '='; cptr++, sptr++) if (toupper(*cptr) != toupper(*sptr)) break; /* if found return a pointer to the value */ if (dbug) fprintf (stdout, "VarName |%s|%s|\n", VarName, sptr); if (!*cptr && *sptr == '=') { if (dbug) fprintf (stdout, "CGIplus |%s|\n", sptr+1); cptr = malloc (strlen(sptr)); strcpy (cptr, sptr+1); return (cptr); } } /* not found */ if (dbug) fprintf (stdout, "CGIplus |NULL|\n"); return (NULL); } /*************************/ /* get CGIplus variables */ /*************************/ if (dbug) fprintf (stdout, "CgiPlusIn %u\n", CgiPlusIn); /* cannot "sync" in a non-CGIplus environment */ if (!VarName[0] && !IsCgiPlus) return (NULL); /* the CGIPLUSIN stream can be left open */ #if PY_MAJOR_VERSION >= 3 /* DECC$DISABLE_TO_VMS_LOGNAME_TRANSLATION */ /* DECC$MAILBOX_CTX_STM */ if (!CgiPlusIn) if (!(CgiPlusIn = fopen ("/CGIPLUSIN", "r", "rat=cr"))) exit (vaxc$errno); #else if (!CgiPlusIn) if (!(CgiPlusIn = fopen ("CGIPLUSIN:", "r"))) exit (vaxc$errno); #endif /* get the starting record (the essentially discardable one) */ for (;;) { cptr = fgets (StructBuffer, sizeof(StructBuffer), CgiPlusIn); if (!cptr) exit (vaxc$errno); /* if the starting sentinal is detected then break */ if (*(unsigned short*)cptr == '!\0' || *(unsigned short*)cptr == '!\n' || (*(unsigned short*)cptr == '!!' && isdigit(*(cptr+2)))) break; } /* MUST be done after reading the synchronizing starting record */ if (dbug) fprintf (stdout, "Content-Type: text/plain\n\n"); /* detect the CGIplus "force" record-mode environment variable (once) */ if (*(unsigned short*)cptr == '!!') { /********************/ /* CGIplus 'struct' */ /********************/ /* get the size of the binary structure */ StructLength = atoi(cptr+2); if (StructLength <= 0 || StructLength > sizeof(StructBuffer)) exit (SS$_BUGCHECK); if (!fread (StructBuffer, 1, StructLength, CgiPlusIn)) exit (vaxc$errno); } else { /*********************/ /* CGIplus 'records' */ /*********************/ /* reconstructs the original 'struct'ure from the records */ sptr = (bptr = StructBuffer) + sizeof(StructBuffer); while (fgets (bptr+SOUS, sptr-(bptr+SOUS), CgiPlusIn)) { /* first empty record (line) terminates variables */ if (bptr[SOUS] == '\n') break; /* note the location of the length word */ cptr = bptr; for (bptr += SOUS; *bptr && *bptr != '\n'; bptr++); if (*bptr != '\n') exit (SS$_BUGCHECK); *bptr++ = '\0'; if (bptr >= sptr) exit (SS$_BUGCHECK); /* update the length word */ *(unsigned short*)cptr = bptr - (cptr + SOUS); } if (bptr >= sptr) exit (SS$_BUGCHECK); /* terminate with a zero-length entry */ *(unsigned short*)bptr = 0; StructLength = (bptr + SOUS) - StructBuffer; } if (dbug) { fprintf (stdout, "StructLength %d\n", StructLength); for (bptr = StructBuffer; Length = *(unsigned short*)bptr; bptr += Length) fprintf (stdout, "|%s|\n", bptr += SOUS); } if (!CalloutDone) { /* provide the CGI callout to set CGIplus into 'struct' mode */ fflush (stdout); fputs (CgiPlusEscPtr, stdout); fflush (stdout); /* the leading '!' indicates we're not going to read the response */ fputs ("!CGIPLUS: struct", stdout); fflush (stdout); fputs (CgiPlusEotPtr, stdout); fflush (stdout); /* don't need to do this again (the '!!' tells us what mode) */ CalloutDone = 1; } return (NULL); # undef SOUS } /*****************************************************************************/ /* Standard CGI environment. Clunky, but what else can we do with DCL symbols? */ char* DclSymbolNames () { static char SymbolNames [] = /* standard CGI variable names */ "AUTH_ACCESS,AUTH_AGENT,AUTH_GROUP,AUTH_PASSWORD,\ AUTH_REALM,AUTH_REALM_DESCRIPTION,AUTH_REMOTE_USER,\ AUTH_TYPE,AUTH_USER,CONTENT_LENGTH,CONTENT_TYPE,\ DOCUMENT_ROOT,GATEWAY_BG,GATEWAY_EOF,GATEWAY_EOT,\ GATEWAY_ESC,GATEWAY_INTERFACE,GATEWAY_MRS,\ HTTP_ACCEPT,HTTP_ACCEPT_CHARSET,HTTP_ACCEPT_ENCODING,\ HTTP_ACCEPT_LANGUAGE,HTTP_AUTHORIZATION,HTTP_CACHE_CONTROL,\ HTTP_COOKIE,HTTP_FORWARDED,HTTP_HOST,HTTP_IF_NOT_MODIFIED,\ HTTP_PRAGMA,HTTP_REFERER,HTTP_USER_AGENT,HTTP_X_FORWARDED_FOR,\ PATH_INFO,PATH_ODS,PATH_TRANSLATED,QUERY_STRING,\ REMOTE_ADDR,REMOTE_HOST,REMOTE_PORT,REMOTE_USER,\ REQUEST_CHARSET,REQUEST_CONTENT_TYPE,REQUEST_METHOD,\ REQUEST_SCHEME,REQUEST_TIME_GMT,REQUEST_TIME_LOCAL,REQUEST_URI,\ SCRIPT_FILENAME,SCRIPT_NAME,SCRIPT_RTE,SERVER_ADMIN,\ SERVER_ADDR,SERVER_CHARSET,SERVER_GMT,SERVER_NAME,\ SERVER_PROTOCOL,SERVER_PORT,SERVER_SOFTWARE,SERVER_SIGNATURE,\ UNIQUE_ID" /* mod_ssl names */ "HTTPS,SSL_PROTOCOL,SSL_SESSION_ID,SSL_CIPHER,SSL_CIPHER_EXPORT,\ SSL_CIPHER_USEKEYSIZE,SSL_CIPHER_ALGKEYSIZE,SSL_CLIENT_M_VERSION,\ SSL_CLIENT_M_SERIAL,SSL_CLIENT_S_DN,SSL_CLIENT_S_DN_x509,\ SSL_CLIENT_I_DN,SSL_CLIENT_I_DN_x509,SSL_CLIENT_V_START,\ SSL_CLIENT_V_END,SSL_CLIENT_A_SIG,SSL_CLIENT_A_KEY,SSL_CLIENT_CERT,\ SSL_SERVER_M_VERSION,SSL_SERVER_M_SERIAL,SSL_SERVER_S_DN,\ SSL_SERVER_S_DN_x509,SSL_SERVER_I_DN,SSL_SERVER_I_DN_x509,\ SSL_SERVER_V_START,SSL_SERVER_V_END,SSL_SERVER_A_SIG,\ SSL_SERVER_A_KEY,SSL_SERVER_CERT,SSL_VERSION_INTERFACE,\ SSL_TLS_SNI,SSL_VERSION_LIBRARY," /* Purveyor SSL names */ "SECURITY_STATUS,SSL_CIPHER,SSL_CIPHER_KEYSIZE,SSL_CLIENT_CA,\ SSL_CLIENT_DN,SSL_SERVER_CA,SSL_SERVER_DN,SSL_VERSION," /* X509 names */ "AUTH_X509_CIPHER,AUTH_X509_FINGERPRINT,AUTH_X509_ISSUER,\ AUTH_X509_KEYSIZE,AUTH_X509_SUBJECT"; /* end of list */ /*********/ /* begin */ /*********/ return (SymbolNames); } /*****************************************************************************/ /* Suppress the WASD-ism of an empty or 'root' script name (i.e. "/"). */ char* EmptyScriptName (char *sptr) { /*********/ /* begin */ /*********/ // if (dbug) fprintf (stdout, "EmptyScriptName() |%s|\n", sptr); if (*(unsigned long*)sptr != 'SCRI') return (sptr); if (!strncmp (sptr, "SCRIPT_NAME=/\0", 14)) return ("SCRIPT_NAME="); return (sptr); } /*****************************************************************************/ /* Return 1 if the report was enabled, 0 if not enabled. Displays cache entry statistics. */ char* my_ctime_r (unsigned long *tsecs, char *buf) { ctime_r (tsecs,buf); buf[19] = '\0'; return (buf); } int MetricsReport () { #define LIB$K_DELTA_SECONDS_F 30 static unsigned long LibDeltaSecs = LIB$K_DELTA_SECONDS_F; int bcsize, cnt, idx, midx, status; unsigned long ffloat, tsecs; double tfloat; char lubuf [32], mtbuf [32], pyver [32]; struct CodeCacheStruct *ccptr; struct CodeMetricStruct *cmptr; PyObject *pFunc, *pValue, *pPlatMod; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "MetricsReport()\n"); if (!CodeMetricEnabled) return (0); pPlatMod = PyImport_ImportModule ("platform"); if (!pPlatMod) return (0); pFunc = PyObject_GetAttrString (pPlatMod, "python_version"); if (!pFunc) return (0); pValue = PyObject_CallFunction (pFunc, NULL); if (!pValue) return (0); #if PY_MAJOR_VERSION >= 3 strcpy (pyver, PyBytes_AsString (pValue)); #else strcpy (pyver, PyString_AsString (pValue)); #endif Py_DECREF(pPlatMod); Py_DECREF(pFunc); Py_DECREF(pValue); fprintf (stdout, "Status: 200\n\ Content-Type: text/html\n\ \n\ \n\ \n\ \n\ Metrics Report - %s (%s)\n\ \n\ \n\ \n\ \ Metrics Report  -  %s  -  Python %s\ \n\

PID  %08.08X  Usage Count: %u\n\

%d Item(s)\n", SoftwareId, BUILD_DATETIME, SoftwareId, pyver, ProcessPid, PyrteUsageCount, CodeCacheCount); if (CodeCacheCount) { fputs ( "

\n\ \ \n\ \ \ \ \ \ \ \ \ \ \ \ \ \ \n", stdout); /* lib$cvts..() is not available before VMS V8.2 so work around */ status = lib$cvtf_from_internal_time (&LibDeltaSecs, &ffloat, &RteDeltaBinTime); if (!(status & 1)) exit (status); status = cvt$convert_float (&ffloat, CVT$K_VAX_F, &tfloat, CVT$K_IEEE_T, CVT$M_ROUND_TO_NEAREST); if (!(status & 1)) exit (status); fprintf (stdout, "\ \ \ \ \ \ \ \n", tfloat, (float)(RteJpiCpuTim)/100.0, RteJpiBufIO, RteJpiDirIO, RteJpiPageFlts, RteJpiPpgCnt); } else fputs ("

", stdout); for (idx = 0; idx < CodeCacheCount; idx++) { if (!CodeCache[idx].FileNamePtr) continue; /* possible if metric reporting is enabled on an existing RTE */ if (!CodeCache[idx].CodeMetricPtr) continue; ccptr = &CodeCache[idx]; cmptr = ccptr->CodeMetricPtr; status = lib$cvtf_from_internal_time (&LibDeltaSecs, &ffloat, &cmptr->DeltaBinTime[0]); if (!(status & 1)) exit (status); status = cvt$convert_float (&ffloat, CVT$K_VAX_F, &tfloat, CVT$K_IEEE_T, CVT$M_ROUND_TO_NEAREST); if (!(status & 1)) exit (status); tsecs = decc$fix_time(&cmptr->StartBinTime[0]); pValue = PyObject_GetAttrString ((PyObject*)ccptr->pByteCode, "co_code"); if (pValue) bcsize = PyObject_Size (pValue); else bcsize = -1; fprintf (stdout, "

\ \ \ \ \ \ \ \ \ \ \ \ \ \ \n", idx + 1, ccptr->CodeUsageCount, bcsize, tfloat, (float)(cmptr->JpiCpuTim[0])/100.0, cmptr->JpiBufIO[0], cmptr->JpiDirIO[0], cmptr->JpiPageFlts[0], cmptr->JpiPpgCnt[0], my_ctime_r(&ccptr->ScriptMtimeSecs,mtbuf), ccptr->FileNamePtr, ccptr->FileNameByteCodePtr ? ccptr->FileNameByteCodePtr : ""); midx = cmptr->MetricIndex; for (cnt = 1; cnt <= METRIC_MAX; cnt++) { if (cmptr->StartBinTime[midx][0] == 0 && cmptr->StartBinTime[midx][1] == 0) break; status = lib$cvtf_from_internal_time (&LibDeltaSecs, &ffloat, &cmptr->DeltaBinTime[midx]); if (!(status & 1)) exit (status); status = cvt$convert_float (&ffloat, CVT$K_VAX_F, &tfloat, CVT$K_IEEE_T, CVT$M_ROUND_TO_NEAREST); if (!(status & 1)) exit (status); tsecs = decc$fix_time(&cmptr->StartBinTime[midx]); fprintf (stdout, "\ \ \ \ \ \ \ \ \ \n", cmptr->UsageNumber[midx], my_ctime_r(&tsecs,lubuf), tfloat, (float)(cmptr->JpiCpuTim[midx])/100.0, cmptr->JpiBufIO[midx], cmptr->JpiDirIO[midx], cmptr->JpiPageFlts[midx], cmptr->JpiPpgCnt[midx], cmptr->RequestPath[midx], strlen(cmptr->RequestPath[midx]) == METRIC_PATH_MAX-1 ? "..." : ""); /* zero is the 'cummulative' index */ if (midx > 1) midx--; else midx = METRIC_MAX; } /*** if (pValue) { fputs("\n", stdout); fputs("\n", stdout); Py_DECREF(pValue); } else { } ***/ } fputs ("
UsageSizeNumberLast UsedDurationCPUBIODIOFLTPGSModifiedScript FileByte-Code File
RTE cummulative%.4f%.4f%u%u%u%u
%d.%d%dcummulative%.4f%.4f%u%u%u%u%s%s%s
%u%s%.4f%.4f%u%u%u%u%s%s
", stdout); PyObject_Print ((PyObject*)CodeCache[idx].pByteCode, stdout, 0); fputs("
", stdout); PyObject_Print (pValue, stdout, 0); fputs("
\n

", stdout);

/***
   LibVmReport (NULL);
***/

   fputs ("
\n\


\n\ \n\ \n", stdout); fflush (stdout); fputs (CgiPlusEofPtr, stdout); fflush (stdout); return (1); } /*****************************************************************************/ /* Add a metrics structure and initialise required elements. */ void AddMetrics (struct CodeCacheStruct *ccptr) { int midx; struct CodeMetricStruct *cmptr; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "AddMetrics()\n"); if (!CodeMetricEnabled) return; if (ccptr->CodeMetricPtr) return; ccptr->CodeMetricPtr = cmptr = calloc (1, sizeof(struct CodeMetricStruct)); if (!ccptr->CodeMetricPtr) exit (vaxc$errno); /* set these to delta "zero" (actually the smallest delta time) */ for (midx = 0; midx <= METRIC_MAX; midx++) cmptr->DeltaBinTime[midx][0] = cmptr->DeltaBinTime[midx][1] = -1; } /*****************************************************************************/ /* Accumulate the measurements. */ void CollectMetrics ( struct CodeCacheStruct *ccptr, int AtEnd ) { static unsigned long JpiBufIO, JpiCpuTim, JpiDirIO, JpiPageFlts, JpiPpgCnt; static struct { unsigned short buf_len; unsigned short item; void *buf_addr; void *ret_len; } JpiItems [] = { { sizeof(JpiBufIO), JPI$_BUFIO, &JpiBufIO, 0 }, { sizeof(JpiCpuTim), JPI$_CPUTIM, &JpiCpuTim, 0 }, { sizeof(JpiDirIO), JPI$_DIRIO, &JpiDirIO, 0 }, { sizeof(JpiPageFlts), JPI$_PAGEFLTS, &JpiPageFlts, 0 }, { sizeof(JpiPpgCnt), JPI$_PPGCNT, &JpiPpgCnt, 0 }, { 0,0,0,0 } }; int midx, status; char *cptr, *sptr, *zptr; struct CodeMetricStruct *cmptr; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "CollectMetrics()\n"); if (!CodeMetricEnabled) return; /* possible if metric reporting is enabled on an existing RTE */ if (!(cmptr = ccptr->CodeMetricPtr)) return; status = sys$getjpiw (0, 0, 0, &JpiItems, 0, 0, 0); if (!(status & 1)) exit (status); if (AtEnd) { midx = cmptr->MetricIndex; sys$gettim (&cmptr->EndBinTime[midx]); lib$sub_times (&cmptr->EndBinTime[midx], &cmptr->StartBinTime[midx], &cmptr->DeltaBinTime[midx]); /* accumulate the durations */ lib$add_times (&cmptr->DeltaBinTime[midx], &cmptr->DeltaBinTime[0], &cmptr->DeltaBinTime[0]); lib$add_times (&cmptr->DeltaBinTime[midx], &RteDeltaBinTime, &RteDeltaBinTime); cmptr->JpiBufIO[midx] = JpiBufIO - cmptr->JpiBufIO[midx]; cmptr->JpiBufIO[0] += cmptr->JpiBufIO[midx]; RteJpiBufIO += cmptr->JpiBufIO[midx]; cmptr->JpiCpuTim[midx] = JpiCpuTim - cmptr->JpiCpuTim[midx]; cmptr->JpiCpuTim[0] += cmptr->JpiCpuTim[midx]; RteJpiCpuTim += cmptr->JpiCpuTim[midx]; cmptr->JpiDirIO[midx] = JpiDirIO - cmptr->JpiDirIO[midx]; cmptr->JpiDirIO[0] += cmptr->JpiDirIO[midx]; RteJpiDirIO += cmptr->JpiDirIO[midx]; cmptr->JpiPageFlts[midx] = JpiPageFlts - cmptr->JpiPageFlts[midx]; cmptr->JpiPageFlts[0] += cmptr->JpiPageFlts[midx]; RteJpiPageFlts += cmptr->JpiPageFlts[midx]; cmptr->JpiPpgCnt[midx] = JpiPpgCnt; if (cmptr->JpiPpgCnt[0] < JpiPpgCnt) cmptr->JpiPpgCnt[0] = JpiPpgCnt; if (RteJpiPpgCnt < JpiPpgCnt) RteJpiPpgCnt = JpiPpgCnt; } else { if (cmptr->MetricIndex < METRIC_MAX) cmptr->MetricIndex++; else cmptr->MetricIndex = 1; midx = cmptr->MetricIndex; /* includes the byte-cache search, etc., and so is a little more true */ cmptr->StartBinTime[midx][0] = CurrentBinTime[0]; cmptr->StartBinTime[midx][1] = CurrentBinTime[1]; cmptr->JpiBufIO[midx] = JpiBufIO; cmptr->JpiCpuTim[midx] = JpiCpuTim; cmptr->JpiDirIO[midx] = JpiDirIO; cmptr->JpiPageFlts[midx] = JpiPageFlts; cmptr->UsageNumber[midx] = PyrteUsageCount; zptr = (sptr = cmptr->RequestPath[midx]) + sizeof(cmptr->RequestPath[midx])-1; for (cptr = CgiVar ("SCRIPT_NAME"); sptr < zptr; *sptr++ = *cptr++); *sptr = '\0'; } } /*****************************************************************************/ /* Show (debug) the virtual memory zone. (Just not interested in the user-arg parameter.) Casting from (void*) to avoid compiler message %CC-I-PROTOSCOPE. */ int LibVmReport (void *StrDscPtr) { #define LIB$_NOTFOU 1409652 static unsigned long DetailLevel = 1; static unsigned long DetailLevel64 [2] = { 1, 0 }; int status; unsigned long Context, ZoneId; unsigned long Context64 [2], ZoneId64; struct dsc$descriptor *DscPtr; /*********/ /* begin */ /*********/ if (dbug) fprintf (stdout, "LibVmReport()\n"); if (StrDscPtr) { /* action-routine */ DscPtr = (struct dsc$descriptor*)StrDscPtr; fprintf (stdout, "%*.*s\n", DscPtr->dsc$w_length, DscPtr->dsc$w_length, DscPtr->dsc$a_pointer); return (SS$_NORMAL); } Context = 0; while ((status = lib$find_vm_zone (&Context, &ZoneId)) == SS$_NORMAL) { status = lib$show_vm_zone (&ZoneId, &DetailLevel, &LibVmReport, 0); if (!(status & 1)) break; } fflush (stdout); if (status != LIB$_NOTFOU) return (status); Context64[0] = Context64[1] = 0; while ((status = lib$find_vm_zone_64 (&Context64, &ZoneId64)) == SS$_NORMAL) { status = lib$show_vm_zone_64 (&ZoneId64, &DetailLevel64, &LibVmReport, 0); if (!(status & 1)) break; } fflush (stdout); if (status != LIB$_NOTFOU) return (status); return (SS$_NORMAL); } /*****************************************************************************/ /* If false then initialize the LIB$ stats timer. If true return a pointer to a static buffer containings a stats string. */ char* StatTimer (int ShowStat) { static $DESCRIPTOR (FaoDsc, "REAL:!%T CPU:!UL.!2ZL DIO:!UL BIO:!UL FAULTS:!UL PGFL:!UL/!UL%\0"); static char StatString [96]; static $DESCRIPTOR (StatStringDsc, StatString); static unsigned long LibStatTimerReal = 1, LibStatTimerCpu = 2, LibStatTimerBio = 3, LibStatTimerDio = 4, LibStatTimerFaults = 5; static unsigned long JpiPagFilCnt, JpiPgFlQuo; static struct { unsigned short buf_len; unsigned short item; void *buf_addr; void *ret_len; } JpiItems [] = { { sizeof(JpiPagFilCnt), JPI$_PAGFILCNT, &JpiPagFilCnt, 0 }, { sizeof(JpiPgFlQuo), JPI$_PGFLQUOTA, &JpiPgFlQuo, 0 }, { 0,0,0,0 } }; int percent, status; unsigned long CpuBinTime, CountBio, CountDio, CountFaults; unsigned long RealBinTime [2]; /*********/ /* begin */ /*********/ if (!ShowStat) { lib$init_timer (0); return (NULL); } status = sys$getjpiw (0, 0, 0, &JpiItems, 0, 0, 0); if (!(status & 1)) exit (status); percent = 100 - ((JpiPagFilCnt * 100) / JpiPgFlQuo); /* post-processing reset */ lib$stat_timer (&LibStatTimerReal, &RealBinTime, 0); lib$stat_timer (&LibStatTimerCpu, &CpuBinTime, 0); lib$stat_timer (&LibStatTimerBio, &CountBio, 0); lib$stat_timer (&LibStatTimerDio, &CountDio, 0); lib$stat_timer (&LibStatTimerFaults, &CountFaults, 0); sys$fao (&FaoDsc, 0, &StatStringDsc, &RealBinTime, CpuBinTime/100, CpuBinTime%100, CountDio, CountBio, CountFaults, JpiPagFilCnt, percent); return (StatString); } /*****************************************************************************/ /* Using the WATCH [x]Script or PYRTE_DBUG or PYRTE_STAT_TIMER logical names. */ void StatTimerCallout (int count) { fflush (stdout); fputs (CgiPlusEscPtr, stdout); fflush (stdout); /* the leading '!' indicates we're not going to read the response */ fprintf (stdout, "!WATCH: %s USAGE:%d/%d %s\n", SoftwareId, count, PyrteUsageCount, StatTimer(TRUE)); fflush (stdout); fputs (CgiPlusEotPtr, stdout); fflush (stdout); } /*****************************************************************************/ /* Using the WATCH [x]Script formatted debug output. */ void WatchCallout (char *fmt, ...) { va_list args; fflush (stdout); fputs (CgiPlusEscPtr, stdout); fflush (stdout); /* the leading '!' indicates we're not going to read the response */ fputs ("!WATCH: ", stdout); va_start (args, fmt); vfprintf (stdout, fmt, args); va_end(args); fflush (stdout); fputs (CgiPlusEotPtr, stdout); fflush (stdout); } /*****************************************************************************/ /* Return an integer representing the server sooftware version. "HTTPd-WASD/11.5.2" should become 110502. */ int ServerSoftware (void) { static int SoftwareNumber; char *cptr; /* VSI C x86-64 V7.5-009 (GEM 50XBR) on OpenVMS x86_64 V9.2-2 */ /* G2L-E-ASSERT, GEM ENTRY ServerSoftware already defined as GEM VARIABLE */ if (SoftwareNumber) return (SoftwareNumber); if (!(cptr = CgiVar ("SERVER_SOFTWARE"))) return (SoftwareNumber = 0); while (*cptr && !isdigit(*cptr)) cptr++; SoftwareNumber = atoi(cptr) * 10000; while (*cptr && isdigit(*cptr)) cptr++; if (*cptr == '.') cptr++; SoftwareNumber += atoi(cptr) * 100; while (*cptr && isdigit(*cptr)) cptr++; if (*cptr == '.') cptr++; SoftwareNumber += atoi(cptr); return (SoftwareNumber); } /*****************************************************************************/ /* Using the WATCH [x]Script formatted debug output. */ void WatchErrorCallout () { #if PY_MAJOR_VERSION >= 3 const char *type, *value; #else char *type, *value; #endif PyObject *ptype, *pstype, *pvalue, *psvalue, *ptrace; PyErr_Fetch (&ptype, &pvalue, &ptrace); pstype = PyObject_Repr (ptype); psvalue = PyObject_Repr (pvalue); #if PY_MAJOR_VERSION >= 3 type = PyUnicode_AsUTF8 (pstype); value = PyUnicode_AsUTF8 (psvalue); #else type = PyString_AsString (pstype); value = PyString_AsString (psvalue); #endif WatchCallout ("ERROR %s %s", type, value); PyErr_Restore (ptype, pvalue, ptrace); } /*****************************************************************************/ /* Just a wrapper. */ char* TrnLnm (char *lnm) { return (Trn2Lnm (lnm, NULL, 0)); } /*****************************************************************************/ /* Translate a logical name using LNM$FILE_DEV by default or the specified name table. Returns a pointer to the value string, or NULL if the name does not exist. 'IndexValue' should be zero for a 'flat' logical name, or 0..127 for interative translations. Returns pointer to non-reentrant static buffer which must be strdup()ed if retained. */ char* Trn2Lnm ( char *LogName, char *LogTable, int IndexValue ) { static unsigned short ValueLength; static unsigned long LnmAttributes, LnmIndex; static char LogValue [256]; static $DESCRIPTOR (LogNameDsc, ""); static $DESCRIPTOR (LogTableDsc, ""); static struct { short int buf_len; short int item; void *buf_addr; unsigned short *ret_len; } LnmItems [] = { { sizeof(LnmIndex), LNM$_INDEX, &LnmIndex, 0 }, { sizeof(LnmAttributes), LNM$_ATTRIBUTES, &LnmAttributes, 0 }, { sizeof(LogValue), LNM$_STRING, LogValue, &ValueLength }, { 0,0,0,0 } }; int status; /*********/ /* begin */ /*********/ LnmIndex = IndexValue; if (LogTable) { LogTableDsc.dsc$a_pointer = LogTable; LogTableDsc.dsc$w_length = strlen(LogTable); } else { LogTableDsc.dsc$a_pointer = "LNM$FILE_DEV"; LogTableDsc.dsc$w_length = sizeof("LNM$FILE_DEV")-1; } LogNameDsc.dsc$a_pointer = LogName; LogNameDsc.dsc$w_length = strlen(LogName); status = sys$trnlnm (0, &LogTableDsc, &LogNameDsc, 0, &LnmItems); if (!(status & 1) || !(LnmAttributes & LNM$M_EXISTS)) { if (LogValue) LogValue[0] = '\0'; return (NULL); } LogValue[ValueLength] = '\0'; return (LogValue); } /*****************************************************************************/