In order to gain detailed and differentiated insight into their own production operations, some companies are even running these applications directly on industrial end devices.
Whether the purpose is real-time analysis or full-blown AI algorithms, the integrity of these applications is quickly compromised if their components are not optimally matched.
It is important to guarantee that devices and systems can continue to perform their core functions in real-time. Modern industrial engineers therefore need a concept that allows the integration of IT and OT technologies in a single device. Such a solution platform should ideally take advantage of standard IT networking and data processing while being deterministic, secure and reliable.
An important component for the realisation of such real-time data communication networks in the industrial Internet of Things (IIoT) is time-sensitive networking (TSN). TSN technology comprises a number of standards, such as IEEE 802.1q for virtual LANs via Ethernet, time aware shaping (TAS) as standardised in IEEE 802.1Qbv for guaranteed minimum transmission latency, or real-time synchronisation via the precision time protocol (PTP) defined in IEEE 1588.
PTP is responsible for time synchronisation between nodes. A master sets the time and the individual slaves synchronise their clocks with two-digit nanosecond accuracy. Based on these synchronised clocks, packets can be time stamped and sent. This means PTP networks can synchronise themselves with two-digit nanosecond accuracy, whereby the travel time of the IP packets naturally also impacts the ultimate real-time behaviour of the application.
In the case of the I219 Intel Ethernet interface, the clock synchronisation is based 100% on this standard component, giving it the dual advantage of being cast in hardware and not requiring any additional proprietary applications or dedicated hardware.
Workload consolidation
At the same time, industrial devices are required to integrate enterprise functions, such as real-time edge analytics. Industrial systems, traditionally based on clearly defined functional units, are now grouped together on a multi-core processor under the heading ‘workload balancing at the edge’. This increases the importance of using hypervisor technologies in industrial real-time systems.
Virtualization technology allows some processor cores to perform data analysis, while others can be reserved for data acquisition or control tasks. As a result, real-time tasks such as robot control or vision-based AI, which is also ideal for collaborative robotics, can run on completely separate virtual machines (VMs) within a server platform alongside less time-critical applications such as remote administration or cloud connectivity.
A proof of concept (PoC) of such a TSN-capable RTOS system with industrial real-time control was first presented by congatec, Intel and Real-Time Systems during a live demo at Embedded World a year ago. The industrial application server platform of this PoC is based on the principles of workload consolidation and TSN via Ethernet.
Real-time application server in practice
At the heart of the demo platform is a COM Express Type 6 module (conga-TS370). Equipped with an Intel Xeon E2 processor, it can provide up to six CPU cores for multitasking. Thanks to the RTS Hypervisor from Real-Time Systems, the processor is divided into multiple machines, each with its own operating system, whereby each VM is assigned one or more CPU core(s) and the required IOs.
The PoC platform uses three VMs in parallel and in real time to control real machines and devices. The first VM employs four CPU cores and uses the Linux operating system to control a Basler vision camera and an Arria 10 FPGA board connected via PCIe. The FPGA board executes AI algorithms for vision-based object detection created with the Intel OpenVINO toolkit. As a result, camera images are captured and verified at extremely high frame rates.
A second independent VM runs real-time Linux and controls a servo motor that holds an inverted pendulum in an upright position. When the balance of the pendulum is disturbed, for instance by touching it, the system reacts in real time and steers the pendulum back to its starting position – completely independently of the other operating systems running or being booted in parallel.
The third VM runs an operating system to perform security tasks and, as edge OS, to provide the uplink to the cloud. To demonstrate the independence of these applications and their real-time behaviour on a single server platform with multiple machines, the individual operating systems can be rebooted for testing purposes without this process impacting the other systems. Even the critical real-time behaviour of the pendulum control remains unaffected.
Although all these applications run in parallel on the same hardware platform, the hypervisor can easily restart a non-real-time VM while the
real-time VMs continue to run uninterrupted. This makes it possible to deploy software updates to individual applications without having to interrupt the control tasks of the entire system. It even allows the effective isolation of security vulnerabilities on individual enterprise VMs because they do not affect the entire embedded system.
Real-time control
Another important feature of the conga-TS370 board is the Intel i219-LM GbE LAN controller as it natively supports IEEE 1588 PTP time synchronization. In the PoC platform for industrial application servers, the i219-LM determines the source and destination of PTP packets while using the integrated timer to coordinate them. The controller also works together with the operating system to line up packets for TSN transmission within a defined timeframe.
Since the release of Linux Kernel 4.20, additional support for the time aware priority shaper (TAPRIO) and earliest transmit time first (ETF) according to IEEE 802.1Qbv has also been available. These modules allow developers to configure systems in such a way that Ethernet TSN packets can be prioritised regardless of their position in the transmission queue.
The RTS Hypervisor, in turn, provides a privileged mode that allows a guest operating system to access system network resources within a few microseconds. This makes it possible to process TSN packets in guest VMs with the required priority and lowest possible latency.
Consequently, the use of these powerful, flexible hypervisor-based IIoT systems can reduce business costs, minimise potential sources of error and extend product lifecycles without enterprise class software upgrades disrupting the hard real-time system.
TSN as future industry standard
Today, real-time communication is still based on a variety of mostly proprietary deterministic transmission and traditional fieldbus protocols that often require expensive components and drivers to match. In addition, the ability to plan and implement industrial real-time platforms is limited by the fact that the choice of manufacturers and matching product portfolios for these highly specialised transmission systems is restricted.
TSN Ethernet provides a way to use the same Ethernet cable for regular network traffic and prioritised TSN traffic as long as the network components support the required standards such as IEEE 1588 PTP or IEEE 802.1Qbv TAS.
Consistently implementing these IEEE TSN standards in standard industrial network components will have an impact on the prevalence of licensed, proprietary real-time transmission protocols. At the very least, these hitherto closed systems will have to open up more and more. And to do so, they will need TSN and hypervisor technology to be able to continue leveraging existing resources.
Author details: Markus Gruber is R&D Manager and TSN Expert at congatec
