2026/02/09

Monday, February 9, 2026 (#40)

Sunday Monday Tuesday posts: prev < this > next

As of right now, we have the following chain of events/calls (which doesn't quite work anymore):

• caEngDbExport::Go():
  • builds the filespec from user input and app-specific rules, then
  • creates a file-node object from it, and
  • passes that node to Engine\schema\caOps\DoExport() which, at least for MyMar Engines, goes to Engine\schema\Ops\MyMar\DoExport().
• cMyMar::DoExport():
  • creates the cMetaPair metadata object (and passes it the file-node object)
  • uses the cMetaPair object to record the start-time for the export
  • creates a Stream object from the file-node object to receive the data and write it to the output-node's file
  • passes the Stream object to $this->DoExportActual().
• cMyMar::DoExportActual() used to do a lot more (a lot of commented-out code here) but currently:
  • get the Engine Client object
  • tells the Engine Client that we need it to be Open()
  • calls ConveyClass::OUpdater($this); - currently ineffectual, but may still be relevant
  • gets an Exporter object from the Engine Client
  • calls Exporter->DoSchema($this) (in this example, Engine\aux\Exporter\cMyMar::DoSchema())
  • tells the Engine Client that we are done, and it may Shut() itself (if no other open-requests remain).
• Exporter->DoSchema($this):
  • gets credentials and schema name, and configures the CmdLine object to do a dump with that information
  • passes the CmdLine object to the Schema's Engine's Server object (in this case, [WFe]Sys\Data\Engine\endpt\Server\MyMar\cMaria) via DoCommand(), which is currently defined in the parent class ([WFe]Sys\Data\Engine\endpt\Server\caMyMar).
• ...Server\caMyMar::DoCommand():
  • assembles the appropriate command from the CmdLine object (contents and hints)
  • passes that to $this->OServer()->DoCommand().
    • OServer() here is [WFe]IO\Aspect\Connx\Stream\Runner\Remote\cSSH2.
• cSSH2::DoCommand():
  • passes the command to ssh2_exec(), which runs it via ssh on the remote
  • turns on stream blocking for the result stream (so it will halt until all of the output is received)
  • creates its own stream (when it should be using the one we created earlier) to capture the results
  • calls stream::PullBytes() exactly once, which works ok when the return results are short, but not so much for GB of exported SQL data

The key problem here is that the Stream object for receiving the data gets lost when we call Exporter->DoSchema($this) (or, technically, when we call $this->DoExportActual(), but that's not really a problem because it's in the same object and probably doesn't even need to be separate anymore) and there's no way to pass it through the cable-chain. We need to use the proper Stream object because it is connected to the output file (where we want to put the data) -- so either we need to pass only the filespec and just forget about trying to get the meta-pair all set up (but that's really the layer at which that stuff should be implemented, so I dislike this idea), or somehow put the Stream object in a place where it can be accessed when receiving the data.

I think it makes sense to add two objects to the CmdLine object: Lecturer and Listener. In this case, we need to set the Listener to transcribe received output to the data-output file. In the case of importing, we will need a Lecturer to transcribe an existing data-file and send it to the Runner.

...and now we have the last-mile problem of getting the Stream from the Schema object (where it is last seen) to the Exporter (where the CmdLine is finally created).

I'm trying out the idea of giving the Exporter a Listener (but not a Lecturer). If (when?) I do an Importer class, then it will need a Lecturer (but not a Listener). This way I can pass along just the stream needed for the task.