By Geir Rune Flåten, Chief Solutions Officer
Can you trust your model in times of extreme events? The Corona crisis can easily make a model for consumer behavior just as relevant as bringing a compass on a spaceship and try to navigate with no magnetic north to pull the compass needle.
The properties and context of your historic data determines the relevance and applicability of any model you develop for the data. And in the event of major and unexpected changes your model might become irrelevant and maybe even misleading.
In these cases, black-box type models where data are shuffled in and results are received and used uncritically are especially risky. As a model owner you have several tools for mitigating the risks, but I strongly encourage you to implement some type of emergency break. Models which do just fine in a “business-as-usual” scenario are uncertain and potentially dangerous if the world suddenly turns upside down.
The answer in industrial analytics has been to keep on asking. When re-calibrating we must explore the change. We must stay explorative in the approach to the model. It is about the data side staying meaningful and relevant to the physical process.
Black swans
Some events have huge impact – though impossible to predict. The black swan theory is about those types of events. Historically the term “black swan” was used to describe something that did not exist. Something out of the ordinary and ridiculously unthinkable. Just until the black swan was actually seen and described!
We have seen them in recent history with Corona, 9/11 and the collapse of the Soviet Union. But in the realm of industrial analytics and model building for manufacturing we have been working with these issues for decades. In this space we may see black swans on the move if raw materials have different characteristics or if a process equipment is replaced. We have learnt the hard way always to have black swans in mind, when we make any changes.
Explorations and skepticism
The answer in industrial analytics has been to keep on asking. When re-calibrating we must explore the change. We must stay explorative in the approach to the model. It is about the data side staying meaningful and relevant to the physical process. If we rely on black boxes in the model building, we would not be able to validate and understand variance.
The skepticism about the model should always be there. What are the underlying dependencies? Presumptions? Physical laws? Without this stance, we would have too much faith in a model, and not be able to adapt and understand, when the black swans are on the wings.
Data scientists and other experts
For a data scientist it is convenient to trust the model and focus on its qualities. But in the disruptive scenarios your historic data may lead to wrong conclusions. Accordingly, industrial analytics should be deeply rooted in the domain expertise in chemistry, biology, physics or engineering. The knowledge about the laws of nature and your system are important and become highly relevant if the world changes and your model starts misbehaving.
This is why we encourage to put the analytics tool in the hands of the domain expert, or at least link the data scientist closely to the domain expert to ensure relevant and interpretable results. This is especially true when extreme events with the snap of a finger devalue your historic data.
Stay healthy and skeptical!
Read about the Unscrambler analytics suite >>
Talk to an expert
Get in touch if you have questions about our products, platform, how to get started or how best to address your analytics needs.
