Xenors · Provide an end-to-end engineering guide for integrating edge AI vision inference systems with industrial PLCs using Modbus TCP, covering network topologies, register maps, Python driver code, ladder logic state machines, and fail-safe latency budgets.

Connecting Edge AI Vision to PLCs: Modbus TCP Architecture, Handshakes & Ladder Logic

A field-tested guide to deterministic handshakes, heartbeat watchdogs, register mapping, and fail-safe state machines between AI inference boxes and industrial controllers.

By Ashok Kumar Yadav · November 13, 2026

The IT/OT Disconnect in Industrial Edge AI

Deploying deep learning models like YOLO or custom segmentation networks on industrial edge devices (such as NVIDIA Jetson Orin modules, Siemens IPCs, or Advantech industrial computers) is only half the battle in factory automation. The point of failure in computer vision deployments is rarely model accuracy in isolation; it is almost always the communication interface between the edge inference engine and the programmable logic controller (PLC).

While computer vision engineers think in terms of frames per second, asynchronously scheduled inferencing, and dynamic JSON payloads, automation engineers think in deterministic scan cycles (typically 2 ms to 20 ms), strict register offsets, and fail-safe hardware interlocks. If an edge computer crashes, drops a packet, or takes 150 ms longer than expected because of memory throttling, an unguided conveyor can jam, a sorting diverter can fire late, or an uninspected defective assembly can reach downstream operations.

This implementation guide walks through the complete architecture required to connect an edge AI vision pipeline to a PLC using Modbus TCP—the most accessible and vendor-agnostic industrial protocol. We examine deterministic handshaking, register layout design, bidirectional heartbeat watchdogs, production-grade Python edge code, and corresponding PLC ladder logic state machines.

System Architecture and Physical Topology

In a standard quality inspection or parts sorting line, the edge AI computer operates alongside the PLC on an isolated Operational Technology (OT) machine network. High-speed cameras trigger on physical proximity sensors, the edge AI classifies or segments the part, and the discrete outcome (Pass/Fail, Defect Class ID, X/Y offset coordinates) must reach the PLC before the part hits the physical reject gate.

Architecture Layer	Hardware / Software Components	Role & Responsibility
Sensing Layer	Through-beam photo eye, 24V PNP inductive sensor, GigE/USB3 Vision camera	Detects raw physical part arrival; triggers camera shutter hardware directly or via PLC digital output.
Inference Layer	Industrial PC / Edge Box (NVIDIA Jetson, Intel Core i7 IPC), TensorRT runtime	Acquires frame, runs pre-processing, performs tensor inference, formats verdict, updates Modbus registers.
Communications Layer	Modbus TCP over unmanaged/managed industrial Ethernet switch (Port 502)	Provides structured, cyclic data exchange (holding registers, input coils) between Edge AI and PLC.
Control Layer	PLC (Siemens S7-1200/1500, Allen-Bradley Micro800/CompactLogix, Schneider Modicon)	Tracks conveyor encoder counts, manages inspection handshakes, executes reject actuation, trips safety alarms.

For related fundamentals on how PLCs process digital signals within their internal scan loops, review our reference on PLC basics, inputs, outputs, and ladder logic. If your edge setup feeds into mobile equipment or warehouse safety cameras, compare this setup against factory-floor AI pedestrian detection systems.

The Deterministic 6-Phase Handshake Protocol

The biggest architectural error engineers make is treating the PLC interface as a fire-and-forget message bus. A vision result written to a Modbus register without an explicit handshake guarantees race conditions: the PLC may read stale data from a previous cycle or read half-written registers mid-scan.

To ensure deterministic synchronization, implement an explicit 6-phase interlocking state machine using two status registers or discrete bits:

Part Present / Trigger (PLC to AI): The PLC detects a part via photo-eye or encoder latch and sets the Trigger bit (Coil or Register bit) HIGH.
Acknowledge & Busy (AI to PLC): The Edge AI sees the trigger, latches an internal processing flag, and sets the Busy bit HIGH while clearing its previous Result_Ready bit.
Trigger Clear (PLC to AI): Upon detecting Busy == 1, the PLC resets its Trigger bit back to LOW. This prevents double-triggering on slow conveyor segments.
Inference & Register Update (Edge AI): The vision engine completes neural network inference, writes numeric scores (defect code, score, coordinates) to data registers, and asserts Result_Ready = 1. Simultaneously, it sets Busy = 0.
PLC Read & Process (PLC): The PLC detects Result_Ready == 1, ingests the data registers into local variables, tracks the part in its shift-register/FIFO queue, and sets an Acknowledge bit HIGH.
Reset & Ready (AI to PLC): The Edge AI sees the PLC's Acknowledge, clears Result_Ready = 0, and returns to the IDLE listening state. The PLC subsequently drops Acknowledge = 0.

[CONVEYOR] ---> Photo-Eye Tripped
                    |
[PLC]      ---> Sets Trigger = 1
                    |---------> [EDGE AI] Detects Trigger
                                            | Sets Busy = 1
[PLC]      <--------------------------------|
[PLC]      ---> Clears Trigger = 0
                                [EDGE AI] Runs Deep Learning Inference
                                            | Writes Class, Bounding Box, Pass/Fail
                                            | Sets Result_Ready = 1, Busy = 0
[PLC]      <--------------------------------|
[PLC]      ---> Reads Registers
           ---> Appends result to Diverter FIFO
           ---> Sets Ack = 1
                                [EDGE AI] Detects Ack = 1
                                            | Sets Result_Ready = 0
[PLC]      <--------------------------------|
[PLC]      ---> Clears Ack = 0 (System IDLE)

Designing the Modbus TCP Register Map

Modbus TCP uses 16-bit word addresses. When transmitting multi-byte data (like 32-bit floats for neural network confidence or signed integers for camera coordinates), you must explicitly define endianness (Word swap and Byte swap). Industrial PLCs like Siemens typically default to Big-Endian (Motorola format), while x86/ARM edge systems run Little-Endian (Intel format).

Standard 100-Word Register Table (Edge AI as Modbus Server)

Address	Tag Name	Data Type	Direction	Description
`40001`	`CMD_CONTROL`	UINT16	PLC → AI	Bit 0: Trigger, Bit 1: Ack, Bit 2: Reset Error, Bit 7: Recipe Reload.
`40002`	`PLC_HEARTBEAT`	UINT16	PLC → AI	Free-running rolling counter (0–65535, increments every 100 ms).
`40003`	`RECIPE_ID`	UINT16	PLC → AI	Product SKU or active model weights index (e.g., 101 = Part A, 102 = Part B).
`40010`	`STATUS_FLAGS`	UINT16	AI → PLC	Bit 0: Ready, Bit 1: Busy, Bit 2: Result Ready, Bit 3: General Fault.
`40011`	`AI_HEARTBEAT`	UINT16	AI → PLC	Echo or independent rolling counter from AI process (watchdog).
`40012`	`INSPECTION_RESULT`	UINT16	AI → PLC	0 = Undetermined, 1 = Pass, 2 = Defect / Fail, 3 = Invalid Image/Occluded.
`40013`	`DEFECT_CLASS`	UINT16	AI → PLC	Encoded category (1: Scratch, 2: Crack, 3: Missing Component, 4: Wrong Color).
`40014–40015`	`CONFIDENCE_SCORE`	FLOAT32	AI → PLC	Model confidence (0.0 to 1.0). Requires 2 consecutive 16-bit registers.
`40016`	`INFERENCE_TIME_MS`	UINT16	AI → PLC	Total duration in milliseconds (frame acquisition + pre-process + inference).
`40017`	`ERROR_CODE`	UINT16	AI → PLC	Diagnostic code (0: OK, 10: Camera Disconnect, 20: CUDA OOM, 30: Timeout).

Edge AI Driver: Python Implementation with PyModbus

On the edge device, running inference and serving Modbus registers in a single thread creates unbounded jitter. If an image frame takes 80 ms to process, the Modbus server must not block incoming PLC heartbeat requests during that window. A robust implementation runs an asynchronous Modbus TCP server in a background event loop while an inference worker interacts with shared thread-safe datastores.

The following example uses pymodbus (v3.x) with a decoupled worker pattern. For building custom object detection pipelines to plug into this script, see our guide on YOLO object detection in Python.

import time
import struct
import threading
import logging
from pymodbus.server import StartAsyncTcpServer
from pymodbus.datastore import ModbusSequentialDataBlock, ModbusSlaveContext, ModbusServerContext

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("EdgeVisionBridge")

# Register Offsets (0-indexed in pymodbus)
REG_CMD_CONTROL      = 0   # 40001
REG_PLC_HEARTBEAT    = 1   # 40002
REG_RECIPE_ID        = 2   # 40003
REG_STATUS_FLAGS     = 9   # 40010
REG_AI_HEARTBEAT     = 10  # 40011
REG_INSPECT_RESULT   = 11  # 40012
REG_DEFECT_CLASS     = 12  # 40013
REG_CONFIDENCE_HI    = 13  # 40014
REG_CONFIDENCE_LO    = 14  # 40015
REG_INFERENCE_MS     = 15  # 40016
REG_ERROR_CODE       = 16  # 40017

# Bitmasks for CMD_CONTROL
MASK_TRIGGER         = 0x0001
MASK_ACK             = 0x0002
MASK_RESET_ERR       = 0x0004

# Bitmasks for STATUS_FLAGS
FLAG_READY           = 0x0001
FLAG_BUSY            = 0x0002
FLAG_RESULT_READY    = 0x0004
FLAG_FAULT           = 0x0008

def pack_float32(val):
    """Packs float32 into two 16-bit registers (Big-Endian word order)."""
    raw = struct.pack(">f", float(val))
    hi, lo = struct.unpack(">HH", raw)
    return hi, lo

class EdgeVisionPipeline:
    def __init__(self, datastore):
        self.store = datastore
        self.running = True
        self.heartbeat_counter = 0
        self.last_plc_heartbeat = 0
        self.heartbeat_timeout_limit = 5.0  # seconds
        self.last_plc_change_time = time.time()

    def run_mock_inference(self):
        """Simulate model inference (e.g., ONNX Runtime or TensorRT)."""
        start = time.perf_counter()
        # Simulate 35ms inference processing time
        time.sleep(0.035)
        elapsed_ms = int((time.perf_counter() - start) * 1000)
        
        # Simulated outputs: Pass (1), Defect None (0), Confidence 0.965
        result = 1
        defect_class = 0
        confidence = 0.965
        return result, defect_class, confidence, elapsed_ms

    def loop(self):
        logger.info("Edge AI Vision Engine loop started.")
        # Initialize status as READY
        self.store.setValues(3, REG_STATUS_FLAGS, [FLAG_READY])

        while self.running:
            loop_start = time.time()
            
            # 1. Heartbeat & Watchdog Supervision
            self.heartbeat_counter = (self.heartbeat_counter + 1) & 0xFFFF
            self.store.setValues(3, REG_AI_HEARTBEAT, [self.heartbeat_counter])
            
            plc_hb = self.store.getValues(3, REG_PLC_HEARTBEAT, 1)[0]
            if plc_hb != self.last_plc_heartbeat:
                self.last_plc_heartbeat = plc_hb
                self.last_plc_change_time = time.time()
            elif time.time() - self.last_plc_change_time > self.heartbeat_timeout_limit:
                # PLC stopped updating heartbeat
                status = self.store.getValues(3, REG_STATUS_FLAGS, 1)[0]
                self.store.setValues(3, REG_STATUS_FLAGS, [status | FLAG_FAULT])
                self.store.setValues(3, REG_ERROR_CODE, [100]) # 100 = PLC Comm Loss
            
            # 2. Read Control Word from PLC
            cmd = self.store.getValues(3, REG_CMD_CONTROL, 1)[0]
            status = self.store.getValues(3, REG_STATUS_FLAGS, 1)[0]
            
            is_trigger = bool(cmd & MASK_TRIGGER)
            is_ack = bool(cmd & MASK_ACK)
            
            # 3. Handshake Execution
            if is_trigger and not (status & FLAG_BUSY) and not (status & FLAG_RESULT_READY):
                # Transition to BUSY
                logger.info("Trigger received. Starting inspection...")
                self.store.setValues(3, REG_STATUS_FLAGS, [FLAG_BUSY])
                
                # Execute vision model
                res, defect, conf, duration = self.run_mock_inference()
                
                # Write output values to data block
                hi_w, lo_w = pack_float32(conf)
                self.store.setValues(3, REG_INSPECT_RESULT, [res])
                self.store.setValues(3, REG_DEFECT_CLASS, [defect])
                self.store.setValues(3, REG_CONFIDENCE_HI, [hi_w, lo_w])
                self.store.setValues(3, REG_INFERENCE_MS, [duration])
                
                # Set RESULT_READY flag, clear BUSY
                self.store.setValues(3, REG_STATUS_FLAGS, [FLAG_RESULT_READY])
                logger.info(f"Inspection complete ({duration}ms). Result={res}. Awaiting PLC Ack.")

            elif is_ack and (status & FLAG_RESULT_READY):
                # PLC has latched result. Reset back to READY.
                self.store.setValues(3, REG_STATUS_FLAGS, [FLAG_READY])
                logger.info("PLC Acknowledged. State machine reset to READY.")

            # Maintain ~100Hz supervisory tick
            sleep_time = max(0.01 - (time.time() - loop_start), 0.001)
            time.sleep(sleep_time)

async def start_server():
    # Holding registers 40001 to 40100 (100 words)
    datablock = ModbusSequentialDataBlock(0, [0] * 100)
    slave_context = ModbusSlaveContext(di=None, co=None, hr=datablock, ir=None)
    server_context = ModbusServerContext(slaves=slave_context, single=True)

    # Spawn edge background inspection loop
    pipeline = EdgeVisionPipeline(datablock)
    worker_thread = threading.Thread(target=pipeline.loop, daemon=True)
    worker_thread.start()

    logger.info("Starting Modbus TCP Server on 0.0.0.0:5020...")
    # Port 5020 for unprivileged local testing; change to 502 for factory standard
    await StartAsyncTcpServer(context=server_context, address=("0.0.0.0", 5020))

if __name__ == "__main__":
    import asyncio
    asyncio.run(start_server())

PLC Implementation: Structured Text & Ladder Logic State Machine

On the PLC side (such as a Siemens S7-1200 running TIA Portal or a Beckhoff TwinCAT controller), the vision cycle should never rely on ad-hoc timer delays. It must be implemented as a rigid Finite State Machine (FSM).

Structured Text (ST / SCL) Inspection Task

// State Definitions
// 0 = IDLE / WAIT_FOR_PART
// 10 = SEND_TRIGGER
// 20 = WAIT_FOR_BUSY
// 30 = WAIT_FOR_RESULT
// 40 = READ_AND_QUEUE
// 50 = SEND_ACK
// 60 = WAIT_AI_RESET
// 99 = ERROR_TIMEOUT

CASE iInspectionState OF
    0: // IDLE
        bTriggerBit := FALSE;
        bAckBit := FALSE;
        IF bPartDetectedAtCameraSensor AND (wAiStatusFlags AND 16#0001 = 16#0001) THEN
            // Part is present AND AI Vision is in READY state
            tonTimeout(IN := FALSE);
            iInspectionState := 10;
        END_IF;

    10: // SEND_TRIGGER
        bTriggerBit := TRUE;
        tonTimeout(IN := TRUE, PT := T#500MS);
        IF (wAiStatusFlags AND 16#0002) = 16#0002 THEN
            // AI set BUSY
            bTriggerBit := FALSE; // Clear trigger immediately
            tonTimeout(IN := FALSE);
            iInspectionState := 30;
        ELSIF tonTimeout.Q THEN
            iInspectionState := 99; // Handshake timeout
        END_IF;

    30: // WAIT_FOR_RESULT
        tonTimeout(IN := TRUE, PT := T#1500MS); // Max inference window
        IF (wAiStatusFlags AND 16#0004) = 16#0004 THEN
            // RESULT_READY asserted
            tonTimeout(IN := FALSE);
            iInspectionState := 40;
        ELSIF tonTimeout.Q THEN
            iInspectionState := 99;
        END_IF;

    40: // READ_AND_QUEUE
        // Transfer registers into conveyor tracking FIFO
        stPartTrackRecord.iResult := wAiResultRegister;
        stPartTrackRecord.iDefectCode := wAiDefectRegister;
        stPartTrackRecord.diEncoderPosition := diCurrentEncoderCount;
        
        // Push to downstream diverter FIFO buffer
        fbConveyorFIFO.Push(stPartTrackRecord);
        iInspectionState := 50;

    50: // SEND_ACK
        bAckBit := TRUE;
        tonTimeout(IN := TRUE, PT := T#500MS);
        IF (wAiStatusFlags AND 16#0004) = 0 THEN
            // AI has seen ACK and cleared RESULT_READY
            bAckBit := FALSE;
            tonTimeout(IN := FALSE);
            iInspectionState := 0; // Return to IDLE
        ELSIF tonTimeout.Q THEN
            iInspectionState := 99;
        END_IF;

    99: // ERROR / TIMEOUT
        bTriggerBit := FALSE;
        bAckBit := FALSE;
        bVisionFaultAlarm := TRUE;
        // Route uninspected part to reject bin by default (Fail-Safe)
        stPartTrackRecord.iResult := 2; // Treat as Defective
        fbConveyorFIFO.Push(stPartTrackRecord);
END_CASE;

The Bidirectional Heartbeat Watchdog

Unlike traditional optical sensors that fail open or closed, edge AI nodes fail in nuanced ways. An inference model can trigger a CUDA driver lockup or Python memory leak where the TCP socket remains responsive and accepts packets, but the vision application thread is deadlocked.

To prevent uninspected parts from passing into packaging lines during an AI silent freeze, enforce a dual-counter rolling watchdog:

PLC Watchdog: The PLC increments PLC_HEARTBEAT (register 40002) every 100 ms. The edge Python worker tracks changes to this value. If the value does not change for 3 consecutive seconds, the edge software logs a PLC communication error and pauses trigger listening.
Edge AI Watchdog: The edge vision script increments AI_HEARTBEAT (register 40011) inside its active processing loop. The PLC monitors this register using a standard timer block (e.g., Siemens TON or Allen-Bradley TONR). If the edge heartbeat register does not change for 500 ms, the PLC trips an audible alarm, halts upstream product infeed, and defaults any in-transit parts to the reject station.

For safety-critical architectural rules regarding autonomous system boundaries and fail-safe interlocks, consult our reference on AI agent guardrails and operational safety.

Latency Budgets and Cycle Time Calculations

Industrial lines running at 120 parts per minute allow a maximum window of 500 ms per cycle. If the physical distance between the camera capture point and the reject pneumatic piston is 800 mm on a conveyor running at 1.2 m/s, the absolute time-of-flight available for inspection and actuator triggering is:

Total Available Time Window:
T_available = Distance / Velocity = 0.8 m / 1.2 m/s = 0.666 s (666 ms)

Every phase of the integration eats into this budget:

Subsystem Phase	Typical Latency (Min–Max)	Key Latency Drivers
Photo-eye debounce & PLC Input Scan	4 ms – 15 ms	Input card filter time setting (typically 3 ms standard).
Modbus TCP Poll / Trigger Delivery	10 ms – 40 ms	PLC Modbus client cyclic poll rate and network switch hop.
Camera Exposure & GigE Packet Transfer	15 ms – 35 ms	Exposure duration, ROI sensor read-out, GenICam transfer.
Edge Pre-processing & TensorRT Inference	20 ms – 65 ms	FP16/INT8 precision, input resolution (e.g., 640x640 vs 1280x1280).
Modbus Register Write & PLC Read Scan	10 ms – 40 ms	Next PLC Modbus client poll cycle and program scan cycle.
Pneumatic Actuator Physical Travel	40 ms – 80 ms	Solenoid valve coil energization, air pressure, mechanical stroke.
Total End-to-End Latency	99 ms – 275 ms	Comfortably inside the 666 ms physical line budget.

If your end-to-end latency budget exceeds the physical travel time, the solution is not to overclock the neural network. You must mechanically move the camera further upstream or decouple the inspection from the diverter using an encoder-tracked shift register.

Five Common Commissioning Pitfalls & Troubleshooting

1. Endianness / Byte Swapping Mismatches

Symptom: The Edge AI writes a confidence value of 0.95, but the PLC reads -1.423e-28 or an erratic integer.
Fix: Check word swap settings. IEEE 754 32-bit floating-point numbers require two 16-bit registers. Some PLCs read the least significant word first (Little-Endian Word Order), while others expect the most significant word first (Big-Endian Word Order). Provide a software toggle in the edge configuration to swap registers (Word0, Word1) ↔ (Word1, Word0).

2. Race Conditions on Result Registers

Symptom: The PLC occasionally rejects a perfect part, logging the defect code from the previous part.
Fix: Never read result registers continuously. The PLC logic must only read registers 40012–40016 when the edge system has explicitly transitioned Result_Ready to HIGH. Clear local variables in the PLC at the start of each trigger sequence.

3. Network Port Conflicts on Industrial PCs

Symptom: The Python edge driver fails to bind with PermissionError: [Errno 13] Permission denied.
Fix: Standard Modbus TCP operates on port 502, which requires root/administrator privileges on Linux systems. Either grant the Python binary binding capabilities using sudo setcap 'cap_net_bind_service=+ep' $(readlink -f $(which python3)), or run the Modbus server on a high port (such as 5020) and configure port redirection via iptables.

4. Unbounded Modbus Poll Jitter

Symptom: Handshake cycle times fluctuate randomly between 30 ms and 250 ms.
Fix: PLCs handle communication tasks in secondary execution slices. If your PLC program uses cyclic Modbus client blocks (such as Siemens MB_CLIENT), ensure the block is scheduled in a dedicated cyclic interrupt OB (Organization Block) with a fixed execution time (e.g., every 10 ms) rather than in the main cyclic loop OB1.

5. Camera Driver Lockup During Network Dropouts

Symptom: The edge computer stops responding to trigger bits after an electrician unplugs and replugs an Ethernet patch cable.
Fix: Implement explicit socket teardown and reconnection logic in your edge communications daemon. Always configure TCP Keep-Alive parameters (SO_KEEPALIVE, TCP_KEEPIDLE=1, TCP_KEEPINTVL=1, TCP_KEEPCNT=3) on both the edge server and the PLC client so that broken sockets drop within 3 seconds instead of hanging in CLOSE_WAIT for several minutes.

Commissioning Checklist for Production Sign-Off

[ ] Subnet Isolation: The Edge AI IPC has two physical NICs: NIC1 on the corporate IT/monitoring network, and NIC2 on a strictly isolated 24V OT subnet directly connected to the PLC.
[ ] Static IP Addressing: All machine vision devices, cameras, and PLC Ethernet modules have hardcoded, non-DHCP static IP addresses recorded in electrical panel documentation.
[ ] Watchdog Trip Test: With the conveyor running, forcefully kill the edge Python process (kill -9). Verify that the PLC halts the upstream infeed or routes all downstream parts to reject within 500 ms.
[ ] Part Burst Test: Run 50 parts back-to-back at maximum line speed. Ensure the PLC FIFO queue does not overrun, and verify that every physical trigger maps to exactly one inspection log entry.
[ ] Failsafe Defaulting: Ensure the PLC logic treats any unrecognized Modbus defect code or timeout as a defect (fail-safe reject), preventing uninspected product from leaving the station.