NML replacement discussion

[rmu75]

Discussion of issues of NML and CMS layers and possible replacements.

[andypugh]

Maybe we should start with a description of exactly what NML currently does.
My (probably incorrect) understanding is that it was an interface layer allowing the first EMC implementation to be split between two different computers.
What do we lose if we delete it and don't replace it?

[BsAtHome]

You are right... Was already working on a small picture to illustrate what I so far discovered about the function and interconnections.

[rmu75]

CMS/NML is used as communication layer between GUI and TASK, I think most other uses are gone (iocontrol used to have its own NML connection).

HALUI is another component that "listens" on HAL pins and sends NML messages.

CMS is the part that serializes messages into different formats, NML is the layer that transports those messages AFAIU.

GUI and TASK live in different processes so some form if inter process communication is needed. Usually on contemporary installations that happens over shared memory, but it is also possible to talk over TCP, there a method is needed to encapsulate and delimit messages into a byte stream that can be deencapsulated on the other end.

So if we get rid of that serialization layer, comms over TCP will not be possible any more.

Additionally, the CMS infrastructure is also used to funnel commands form the interpreter / emccanon to task, bot messages from emccanon and from the GUI end up in one giant switch statement. If you turn on coolant from GUI or from G-Code, that both is handled in the same code in TASK IIRC.

In theory this architecture would allow to run the interpreter, the GUI and TASK on different machines. Separating TASK and interpreter would probably need some work though.

[rmu75]

NML: it is unmaintained and doesn't run on "modern" platforms like android/ios, IMO it would be a pain to port to those platforms / languages.

There seems to be a java impl here https://github.com/DjangoReinhard/JCNCScreen/tree/master/src/de/schwarzrot/nml

[rmu75]

CMS also is unmaintained, and seems to suffer from a bunch of problems like being not really 64bit clean (until recently at least), having a bunch of potential misuses of buffers / buffer overflows and generally being written for late-80ies C++. There used to be a code generator that took care of all the boilerplate code, but this thing seems to be lost, also there is no IDL file describing the messages currently used.

Therefore changing CMS messages is tedious, there is no language interoperability (except maybe this https://github.com/DjangoReinhard/JCNCScreen/tree/master/src/de/schwarzrot/nml), keeping stuff like this in sync manually is error prone.

Better alternatives exist for network transparent interprocess-communication like google protobuf, google flatbuffers, json, pyson, even XML or more "enterprisy" stuff like ASN.1. Multiple languages can parse those formats (could make for a cleaner interface to other languages, esp. if we get HAL connected to this msging layer) and would enable web-browsers and handheld devices to be used as GUI platforms.

[BsAtHome]

This is as far as I got through the code (not all details described). The only thing I never traced is the g-code interpreter paths.

[BsAtHome]

So if we get rid of that serialization layer, comms over TCP will not be possible any more.

Why would you want the serialization layer exposed over TCP? Or, for that matter, the network? You'd rather want to have an (extended) interface like linuxcncrsh to do actual remote work. It needs to be more advanced, but the alternative of exposing the inner core communication feels like a (very) bad idea . This TCP stuff is actually the part og NML I never understood why you'd want it. It is a security nightmare.

[rmu75]

There is no security whatsoever in NML.

They wanted network transparent communication because it was needed in the 90ies. Run the GUI on windows, realtime stuff on some realtime OS and the rest on a unix box. Something like that. AFAIK NML was a general comms framework out of NIST that they just used in linuxcnc.

I call serialization the process of transforming a C-struct or C++ object into a form that can be transmitted over a byte-stream. That needs to deal with different alignments, endianness, subobjects/structures, strings, possibly pointers/references. So in that sense I don't understand your comment because serialization as I understand it is what exposes the interface to TCP.

Adding something like GSSAPI to tcp shouldn't be that hard. Comms over shared mem segment also has its security problems. Some sort of IPC is needed, wo don't want all those components that talk NML currently to live in one process -- It is probably not that easy to draw a line and decide which commands may be sent by a GUI and which are forbidden.

[andypugh]

FWIW following G-code commands to Motion is something that Github CoPilot is really fairly good at:

I'll trace the path of a G2/G3 arc command from convert_arc() through to the motion module.Now let me search for more specific information about the command path:Now I can provide you with a comprehensive description of how a G2 or G3 arc command flows through the system from the interpreter to the motion module.

G2/G3 Arc Command Flow in LinuxCNC

Here's a detailed description of how a G2 or G3 command gets from Interp::convert_arc() to the motion module:

1. Initial Arc Processing (interp_convert.cc)

When a G2 or G3 command is encountered, Interp::convert_arc() is called with parameters specifying:

• Arc direction (G_2 for clockwise, G_3 for counterclockwise)
• Block pointer containing the NC instruction data
• Settings pointer to machine state

The function performs validation and delegates to one of three sub-functions based on whether cutter radius compensation is active:

• convert_arc2() - when compensation is OFF
• convert_arc_comp1() - first move after compensation is ON
• convert_arc_comp2() - subsequent moves with compensation ON

2. Arc Data Calculation

These functions call helper functions to:

• Extract arc endpoints using find_ends()
• Calculate arc center points and turn counts using arc_data_ijk() or arc_data_r()
• Handle inverse time feed rate calculations via inverse_time_rate_arc()

3. Canon Command Queuing (interp_queue.cc)

Instead of immediately issuing the motion command, the arc is queued via:

enqueue_ARC_FEED(settings, block->line_number,
                 original_turns,  // sweep angle
                 end1, end2, center1, center2,  // arc geometry
                 turn,  // rotation count
                 end3,  // Z-axis endpoint
                 a, b, c, u, v, w);  // rotary axes

This function creates a queued_canon structure storing the arc data and appends it to the motion queue. This queue is maintained for cutter compensation operations that may modify arc endpoints.

4. Dequeuing and Canonical Function Call (interp_queue.cc)

When dequeue_canons() is called (after cutter compensation adjustments are complete), it iterates through queued commands and issues the actual canonical motion function:

ARC_FEED(q.data.arc_feed.line_number, 
         q.data.arc_feed.end1, 
         q.data.arc_feed.end2, 
         q.data.arc_feed.center1,
         q.data.arc_feed.center2, 
         q.data.arc_feed.turn, 
         q.data.arc_feed.end3,
         q.data.arc_feed.a, q.data.arc_feed.b, q.data.arc_feed.c, 
         q.data.arc_feed.u, q.data.arc_feed.v, q.data.arc_feed.w);

5. ARC_FEED Canonical Function (Multiple Implementations)

The ARC_FEED() function has different implementations depending on the compilation context:

a) Standard EMC Implementation (src/emc/task/emccanon.cc):

void ARC_FEED(int line_number, 
              double first_end, double second_end,
              double first_axis, double second_axis, int rotation,
              double axis_end_point, 
              double a, double b, double c,
              double u, double v, double w)

This implementation:

• Creates EMC_TRAJ_CIRCULAR_MOVE message structures
• Performs coordinate system conversions (active plane mapping)
• Calculates 3D arc geometry (endpoints, center, normal vectors)
• Handles plane-specific logic (XY, XZ, YZ planes)
• Calls motion trajectory functions to enqueue the move

b) Python/Gcode Module (src/emc/rs274ngc/gcodemodule.cc):

• Calls back to Python via callback mechanism
• Invokes arc_feed() method with parameters converted for unit consistency

c) SAI/Debugging Implementation (src/emc/sai/saicanon.cc):

• Echo/logging version for debugging and analysis

6. Motion Trajectory Queuing (From emccanon.cc)

The ARC_FEED() function in emccanon.cc:

• Builds a circular move message:

  auto circularMoveMsg = std::make_unique<EMC_TRAJ_CIRCULAR_MOVE>();

• Sets motion type to EMC_MOTION_TYPE_ARC
• Populates:
  • End position (endpoint coordinates)
  • Center position (arc center)
  • Rotation count (for multi-turn arcs)
  • Feed rate and synchronization settings
• Writes the message to the trajectory module via emcmotQueue

7. Motion Module Processing (src/emc/motion/ directory)

The motion command is processed by the trajectory planner which:

• Reads EMC_TRAJ_CIRCULAR_MOVE messages from the shared queue
• Computes arc trajectory with acceleration/deceleration profiles
• Generates real-time motion commands for servo drivers
• Updates actual position feedback

8. Plane-Specific Arc Handling

The active plane determines which axes are involved:

• XY Plane (G17): Circular motion in X-Y with Z linear
• XZ Plane (G18): Circular motion in X-Z with Y linear
• YZ Plane (G19): Circular motion in Y-Z with X linear

Key Source Files Involved

File	Function	Purpose
interp_convert.cc	convert_arc()	Main arc processing
interp_queue.cc	enqueue_ARC_FEED(), dequeue_canons()	Queue management
gcodemodule.cc	ARC_FEED()	Python callback interface
emccanon.cc	ARC_FEED()	Motion message construction
command.c	emcmotCommandHandler()	Motion command processing

The architecture separates interpreter logic (which calculates what motion is needed) from motion execution (which handles acceleration, feedrate override, and hardware control), with a message queue providing loose coupling between the two systems.

And it's actually quite complimentary about the architecture (to my surprise)

You're welcome! I'm glad that detailed breakdown was helpful. The LinuxCNC architecture is quite elegant in how it separates concerns—the interpreter focuses on G-code semantics and path geometry, while the motion layer handles real-time execution with smooth acceleration profiles and feed rate overrides.

If you have any follow-up questions about the arc processing, cutter compensation logic, queue management, or any other part of the interpreter, feel free to ask. Good luck with your work on the LinuxCNC project!

[zz912]

rmu75

It is probably not that easy to draw a line and decide which commands may be sent by a GUI and which are forbidden.

Could you please explain this sentence more?
What do you consider the problem?

LinuxCNC has several wonderful GUIs, based on which one could write requirements for communication with TASK. The current GUI - HAL - TASK communication is not ideal. There have already been attempts to fix it somehow. Sigma1912 tried to create native HAL pins in Gmoccapy, Chris created HALBRIDGE. IMHO I didn't like either of them. I think that replacing NML with anything (if necessary) could be done at the same time as revising the GUI interface. If this post of mine is not related to what is being addressed here, please ignore it. You are already having too much of a professional discussion with me. However, if it makes sense to write requirements for the GUI interface, then a new Issue could be created for that.

[rmu75]

EMC_TRAJ_CIRCULAR_MOVE is an NML message (although it is not communicated through NML command channel, but I think in principle it could be)

[BsAtHome]

But how does the G2/G3 command get to the interpreter in the first place? There must be several paths because you can run a program or enter it in MDI. That qualifies "fairly good" to incomplete at best...

[rmu75]

It is probably not that easy to draw a line and decide which commands may be sent by a GUI and which are forbidden.

Could you please explain this sentence more? What do you consider the problem?

Mapping what functionality is used by the GUIs would probably help. For example I have no idea how the plasma cutting stuff handles things and if those GUIs generate motion commands at the beginning of a cut. I think I saw functionality to continue aborted cuts that do some "magic".