TL;DR For want of a nail ...
This one's a doozy :-{
Every software developer, perhaps especially the late Edsger Dijkstra꙳꙳,
knows that despite the best of intentions, efforts and tools, every piece of
non-trivial software when released into the wild always contains defects.
꙳꙳ https://www.google.com/search?q=Edsger_Dijkstra
“Program testing can be used to show the presence of bugs,
but never to show their absence!”
“If debugging is the process of removing software bugs,
then programming must be the process of putting them in.”
These defects often remain unexpressed until the software is disturbed in
some fashion; by modification, unexpected data, porting — including when
rehosting to another platform.
This earlier post describes a WASD example involving rehosting
https://wasd.vsm.com.au/info-WASD/2025/0012
This latest post recounts another WASD latent bug; nominated above as a
doozy | ˈduːzi | (also doozie)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
noun (plural doozies) informal, mainly North American English
something outstanding or unique of its kind: it's gonna be a
doozy of a black eye
As related in this post
https://wasd.vsm.com.au/info-WASD/2025/0039
WASD server code recently ended up so mangled it was decided that
reconstruction using a version from six months before would provide the most
effective resolution. A *big* job. Sifting through the wreckage of the
later version, looking for essential bug fixes and requested enhancements to
reapply, while avoiding the problematic changes inflicted during that period.
Though the review did turn out to have quite the silvery lining.
Previously I had opined (complained)
HTTP/2 Pros/Cons
----------------
The author's opinion; HTTP/2 takes all the complexity of TCP and pushes
behaviour up from OSI layers 4 and 5, into layer 7, and then adds some
more (complexity) for good measure. Just my AU$0.05 worth.
https://wasd.vsm.com.au/info-WASD/2023/0099
and WASD's event-driven, data-driven architecture didn't lend itself to a
straight-forward code path for HTTP/2's own I/O queuing requirements.
HTTP/2 takes a full-duplex TCP connection and multiplexes its own full-duplex
communications onto that. Simple enough I suppose. Parse the TCP stream
into multiple HTTP/2 streams. Writing such a stream is relatively trivial.
Reading such a stream less-so. Especially where the software must request a
read and receive that if such data has already been sent, or wait, perhaps
some time, for that data to become available.
From the inception of WASD HTTP/2 it seemed necessary to maintain an
alternative 'state' independent of classic WASD event-driven 'state'. In
particular, and to repeat, the reading of data. A queue for read requests.
And a queue for data having already been read from the wire. Occasionally a
read request immediately can be supplied from the already-read data queue.
Other times the read request might have to wait, with the data needing to
find the corresponding read request after it arrives. Sometimes it never
arrives, so that also must be accommodated.
In repairing the fractured WASD I made an insightful observation (well, truth
be told; chanced upon an insightful observation). Using one-shot WATCH-peek;
see heading "Requests+" in
https://wasd.vsm.com.au/wasd_root/wasdoc/features/#httpdserverr...
it was noticed that with scripts using HTTP/2 there was always an outstanding
read still queued when the request had concluded.
8< snip 8<
┊ Http2Ptr 0x0051A410
8< snip 8<
┊ DclTaskPtr 0x0029D168
8< snip 8<
┊ CgiPlusInIOsb %X00000001 1722
┊ QueuedCgiPlusIn 0
┊ HttpInputChannel 928 (MBA1376:)
┊ HttpInputIOsb %X00000000 0
┊ QueuedHttpInput 1
┊ ClientReadBufferSize 0
┊ ClientReadStripCrLf 0
8< snip 8<
Hmmm. An undelivered read. Hmm-mmm.
Not only with scripts. This was a more general issue.
A simple experiment involving copious ZAP exercising was promising.
The superimposed 'state' management
┊ if (rqptr = s2ptr->RequestPtr)
┊ {
8< snip 8<
┊ if (rqptr->RequestState < REQUEST_STATE_PROCESSING)
┊ Http2RequestEnd5 (rqptr);
┊ else
┊ if (rqptr->RequestState < REQUEST_STATE_ABORT)
┊ RequestAbort (rqptr);
┊ return (s2ptr);
┊ }
replaced by a simple read delivery
┊ if (rqptr = s2ptr->RequestPtr)
┊ {
8< snip 8<
┊ /* explicitly deliver any outstanding read */
┊ if ((io2ptr = rqptr->NetIoPtr)->ReadAstFunction)
┊ {
┊ io2ptr->ReadIOsb.Status = SS$_VCCLOSED;
┊ io2ptr->ReadIOsb.Count = 0;
┊ SysDclAst (NetIoReadAst, io2ptr);
┊ }
┊ return (s2ptr);
┊ }
Further testing across live sites confirmed the real-world efficacy and
stability of the approach.
That the entire 'state' management could be scrapped (much more extensive
than suggested by the fragment in the first code example), basically due to
fixing this single defect, was a huge relief. This *bug* had been skulking
in the code for a decade :-{ and had resulted in an entire superstructure
being constructed to work around it. This single, concluding read delivery
rippled constructively through HTTP/2 behaviour, from request closure, to
script rundown, to WATCH execution, to ... Sigh.
WASD v12.4 is again (fully) event-driven.
This item is one of a collection at
https://wasd.vsm.com.au/other/#occasional
|