Advanced analytics and Artificial Intelligence are fuelling the urgency to have more real-time data at higher resolutions.
Implementing advanced analytics using tools such as Time-lagged Reporting, Machine Learning, Artificial Intelligence, modelling, and virtual sensors using ‘live’ or real-time information, needs to consider how, when and where the data is generated, stored, and used.
This article focuses on the data resolution, or granularity, of the data needed in applying our Real-Time Process Intelligence (RTPI) strategies.
What is Real-Time?
You will hear this term in a variety of applications ranging from banking and insurance, through to production, maintenance, and logistics. The fact is the meaning of ‘Real-Time’ varies widely depending on the objectives and application.
For economists, a monthly average figure may be considered real-time feedback on inflation.
A supermarket chain may reorder stock or look at required staffing levels based on hourly real-time data.
For the development facility at NASA, sampling data at 1,000 times a second may be insufficient to be considered real-time.
Recommended Data Resolution
The main drivers for capturing and recording data is to understand what has happened, and what might still be happening and to predict what may happen. To meet these needs, you not only need sufficient information, but you need to be capturing it at the right resolution.
To provide effective diagnostics and analytics we suggest that the interval between historical samples should be at least 3 to 5 times faster than the changes you want to understand. For example, if you are trying to identify changes in power consumption per minute, you should seek to have data around every 15 seconds.
If we apply this recommendation to the economist who only has monthly inflation data available to understand current trends and predict inflation, you should expect that they may need to wait 3 months to draw any conclusions. If conditions change, there may also be a lag before they can identify the impact of those changes on the results, extending the time to provide any response.
These core essentials exist for all application and data resolutions. As the applications become more high-speed, the timing of recording data and it’s synchronising for analytics becomes more critical.
Real-Time in Manufacturing/Production Applications
Many production-based businesses traditionally work with statistics for weekly, monthly, and yearly reports for corporate management to review performance against overall targets.
There are often accumulated from shift or daily data which are used by operational staff who have the core responsibility to achieve the targets and performance. These periodic reports are valuable however they provide little support to identify when, where, and why or provide input into operational improvement needs.
In industrial processes, operational issues and variations in performance occur rapidly and can be caused by a wide range of factors allocated to different aspects and people. Production and operational staff need to understand process performance using data with resolution and latencies in the order of minutes if not seconds.
Faster production lines often need millisecond resolution to support analytics.
To optimise processes, results derived from the production data need to also be effectively distributed to stakeholders in a form and with content that suits their roles. Where the analytics need actions, the timely delivery of this information is essential.
Solution Challenges
Resolution is just one of the challenges in utilising real-time data. Some of the other data considerations are what data is available; it is measurement resolution and quality. Then there is how it is integrated and synchronised, and what you do with the information gained.
The largest challenge is that true objectives are not set or do not focus on simple outcomes. Too often we see set objectives that have assumptions or decision elements that skip the investigation phase, usually limiting results.
We see solutions that are picked for an immediate need without getting input from all stakeholders and reviewing future needs. These solutions often lack the ability to gain valuable insight into process operations. They rarely include sufficient information or features to target loss of production especially relative to operation staff making ‘real-time’ decisions to avoid production loss
Value of Real-Time Data
Data is a valuable asset that can be used to increase production and lower costs; however, it is often underutilised.
There is also the opinion that lack of data means that what is available is not valuable. Any trusted information that is available is valuable and can still be used to establish more understanding and support continual improvement initiatives.