Pierluigi Paganini June 17, 2012

Article published on The Malta Indipendent

Convergence in world leading research in Europe to tackle these problems

This the first part of a two-part mini-series in which we look at the safety (and security) problems in real-time cyber-physical systems used around the world Ron Kelson, Pierluigi Paganini, David Pace, Benjamin Gittins  Modern computer chips… a marvel of human ingenuity. Comprising literally many millions of tiny microscopic computing elements (transistors), carefully arranged in complex and intricate configurations, choreographed with unprecedented timing precision. With lightning speed, they process signals at literally 100s of millions of times a second. Chip designers use advanced tools to create three dimensional computer models of the circuitry, studying the electromagnetic interactions between neighbouring parallel wires, searching for human imperceptible timing deviations that might violate timing constraints and cause the chip to periodically fail when deployed in the field. Immersed in this realm of unprecedented timing precision at circuitry level, one might begin to imagine a utopian world… where computers always respond instantly to sensors, user interfaces always respond fluidly to human interaction, audio and video streams never glitch…. then reality abruptly snaps back into focus. As we look at our iPhones, Androids, and other computing devices we know something, somewhere, has gone terribly awry. In his paper “Cyber-Physical Systems – Are Computing Foundations Adequate?”, Prof. Edward A. Lee of UC Berkeley says:

“In the physical world, the passage of time is inexorable and concurrency is intrinsic. Neither of these properties is present in today’s computing and networking abstractions.”

In particular, today’s computer architectures are NOT engineered to perform (real-time) software tasks on time, every single time. Consequently, intermittent timing faults are notoriously difficult to identify and repair. To put this in perspective, IBM has found that 50 per cent of the warranty costs in cars are related to electronics and their embedded software, and that 30 per cent of those costs are related to timing flaws. According to the article “This car runs on code” published in IEEE Spectrum online (2009) these instances of incorrect operation cost industry Billions of Euros annually. In this first article we look at how digital computers interact with the real-world (cyber-physical systems) and what can go seriously wrong when those systems can no longer complete tasks on time. One of the authors of this article, Benjamin Gittins, as CTO of Synaptic Laboratories, and CTO of the ICT Gozo Malta project, was recently invited to attend the “closed-door” Industrial Advisory Board meeting of the EU FP7 PROARTIS Project (www.proartis-project.eu) held in Barcelona, Spain. The PROARTIS Project is focussed on improving the current state of the art in “hard real-time” computing. Hard real-time computing requires critical tasks to complete execution on-time, every-time, without exception. In particular, “PROARTIS will define novel hardware and software architectures for critical real-time embedded systems that … will ensure negligible probability of pathologically long execution times [which would result in tasks missing their deadlines].” When tasks miss their deadline in soft real time applications, such as streaming video or audio, we get annoying glitches, repetition or pauses. However, according to Airbus, a partner in PROARTIS, “In the field of safety-critical avionics applications, verifying that a program exhibits the required functional behaviour is not enough. Indeed, it must also be checked that its timing constraints are satisfied.” To get a clear picture why meeting deadlines is so important in safety-critical avionic, automotive, and indeed all safety-critical cyber-physical systems, we will explore the observe-orient-decide-act (OODA) loop, developed by military strategist and former USAF Colonel John Boyd. The OODA loop originally became an important concept both in business and military strategy and it is often applied to better understand commercial operations and learning processes. According to Boyd, decision-making occurs in a recurring cycle of observation-orientation-decision-action. In the first step we observe the world around us through our senses. Orientation involves filtering and interpreting the raw data received from our senses and synthesizing a picture of the world around us. For humans, this orientation is influenced by our genetic heritage (DNA), cultural traditions (education, beliefs), and previous life experiences. A decision is then made based on the information available and a hypothesis is made on how our actions might influence the world. Because nobody has a perfect model of the world, and we are not the only ones influencing it, we can’t be sure of the result(s) of our actions. So we observe how the world has changed around us. We then orientate ourselves to this new data from our senses, make new hypothesis about how the world works and how our actions have, or might, affect it, and so on. Not surprisingly, the OODA loop is conceptually the same for computer systems that interact with the physical world. In this case the OODA loop is implemented using a combination of hardware and software logic. The OODA loop, like physics, is inherently neutral. Democratic processes and synergistic win-win transactions can, and should, be studied and enhanced within the OODA model (e.g. how can we collaborate more effectively?). Likewise, the model can be applied in winner-loser transactions. For example Boyd wrote:

“In order to win [at war], we should operate at a faster tempo or rhythm than our adversaries − or, better yet, get inside [the] adversary’s Observation-Orientation-Decision-Action time cycle or loop. … Such activity will make us appear ambiguous (unpredictable) [and] thereby generate confusion and disorder among our adversaries − since our adversaries will be unable to generate mental images or pictures that agree with the menacing, as well as faster, transient rhythm or patterns they are competing against.”

As Harry Hillake wrote in his article “John Boyd, USAF Retired, Father of the F16”, in war:

“The key is to obscure your intentions, and make them unpredictable to your opponent, while you simultaneously clarify his intentions. That is, operate at a faster tempo to generate rapidly changing conditions that inhibit your opponent from adapting or reacting to those changes, and that suppress[es] or destroy[s] his awareness. Thus, a hodgepodge of confusion and disorder occur to cause him to over- or under-react to conditions or activities that appear to be uncertain, ambiguous, or incomprehensible.”

In the context of cyber-physical systems, these computers − civilian, critical infrastructure, or otherwise − need to regularly complete the OODA loop to maintain a consistent model of the physical world and its current state. If the observation and orientation tasks are not completed on time, every time, the computing device loses the ability to make accurate decisions. It also loses the ability to keep track of the effect of its actions on the real-world.

To place this in perspective, imagine trying to perform open heart surgery in a dark room lit only by an irregularly flashing strobe-light, and you get the idea of the type of problems a cyber-physical computer system faces when it loses the ability to track real-world events. When timing is lost, instead of regulating the smooth and safe operation of physical systems, the system can fluctuate aggressively out of control, sometimes leading to physically destructive results. This type of problem can occur when running real-time systems on today’s computers operating in a friendly / benign environment.

Imagine the difficulty when that system faces targeted malice designed to disrupt the computer’s ability to observe, orientate, decide, and interact with the world. Today, the expert consensus in the real-time community is that civilian cyber-physical software and systems deployed by industry are widely engineered using inadequate engineering practices, particularly with regard to ensuring their software tasks always meet their deadlines. This problem is difficult for the industry to address because, in part, modern computer architectures are not designed to provide the necessary real-time capabilities. According to PROARTIS the problem is getting worse:

“Industry demands new functionality and higher levels of performance together with reduced cost, weight and power dissipation, which can only be delivered by advanced hardware features. However, the timing behaviour of systems using these advanced hardware features is very hard [ed: and many times impossible] to deal with by current timing analysis techniques.”

The universal consensus is that poor computer hardware design dramatically increases the cost of applying BEST real-time practices and acts as a barrier in practice to safer more reliable systems in the community. The three-year PROARTIS project is predominantly concerned with creating new computer systems (and associated worst case execution time measurement techniques) that can be used to create products that run real-time tasks with high assurance in benign environments. PROARTIS hardware and software deliverables are successfully making headway on this specific problem. Without in any way diminishing the excellent results of PROARTIS, many hard issues in cyber-physical systems still need to be addressed.

Synaptic Labs was invited to attend the PROARTIS Industrial Advisory Board because we are exploring the use of certain ground-breaking hard-real time hardware deliverables from the PROARTIS project within ICT Gozo Malta’s trustworthy and dependable computing projects that addresses the real-time, safety, security, and performance needs of many different industries (smart phone through to public cloud and super-computers) in one unified many-core computer architecture. In the next and final article of this mini series we look at the complex trends, dispositions, and behaviours of organisations in the cyber-physical systems community today that limit the adoption of safety and security best practices. We will also explore how cutting edge research, technologies and projects are trying to overcome these barriers, and what you can do to help promote demand for safer systems.

Be sure to read the next article in this series next week, and join us in taking the next steps together to secure (y)our world!

ICT Gozo Malta is a joint collaboration between the Gozo Business Chamber and Synaptic Labs, part funded by the Ministry for Gozo, Eco Gozo Project, and prize winner in the 2012 Malta Government National Enterprise Innovation Awards. www.ictgozomalta.eu has links to free cyber awareness resources for all age groups.

To promote Maltese ICT to the world, we encourage all ICT professionals to register on the ICT GM Skills Register and keep aware of developments, both in Cyber Security and other ICT R&D initiatives in Malta and Gozo. For further details contact David Pace at [email protected].

Mr Kelson is Vice Chair of the ICT Gozo Malta Project and CEO of Synaptic Laboratories Limited. Mr Gittins is CTO of the ICT Gozo Malta Project and CTO of Synaptic Laboratories Limited.

Mr. Paganini, Security Specialist CISO Bit4ID Srl, is a CEH − Certified Ethical Hacker, EC Council and founder of Security Affairs

Mr Pace is project manager of the ICT Gozo Malta Project and an IT Consultant