Sunday 22 November 2020

Challenges and Implementing Big Data

1. Manufacturers today seek to achieve true business intelligence through collecting, analyzing, and sharing data across all key functional domains. In this architecture, production systems are not only more efficient but can also respond in a timely manner to changing business needs, including signals from partners and customers.

2. Industry 4.0 big data comes from many and diverse sources:

- Product and/or machine design data such as threshold specifications

-Machine-operation data from control systems

-Product- and process-quality data

-Records of manual operations carried out by staff

-Manufacturing execution systems

-Information on manufacturing and operational costs

-Fault-detection and other system-monitoring deployments

-Logistics information including third-party logistics

-Customer information on product usage, feedback, and more

3. Some of these data sources are structured (such as sensor signals), some are semi-structured (such as records of manual operations), and some are completely unstructured (such as image files). In all cases, however, most of the data is either unused or used only for very specific, tactical purposes. 

4. One key factor as to why Industry 4.0 big data is generally not leveraged strategically is poor interoperability across incompatible technologies, systems, and data types; a second key factor is the inability of conventional IT systems to store, manipulate, and govern such huge volumes of diverse data being generated at high velocity.


DATA LAKE 
1. A Data Lake is a repository that stores all structured and unstructured data at a large scale. It provides a platform to analyze  and visualize data with different methods,  such as machine learning tools, real-time  analytics and reporting dashboards.

2. Although many enterprise system vendors now offer cloud storage and data analysis solutions, the high storage cost and vendor lock-effects are - for many of our customers - still main drivers to choose an open-source Data Lake

3. Compared with enterprise systems a Data Lake has the following advantages:

4. Reliable data storage with low cost and flexible cost structure. Data in the cluster is automatically replicated and stored on multiple servers. When one of them shuts down, the cluster would reproduce the data from this server according to corresponding data on other servers. This mechanism builds up a fault-tolerant and robust data storage space. Furthermore, storing  data in the cloud saves hardware investment and allows companies to pay only for the services procured. This leads to a significant lower cost compared with enterprise systems.

5. Data analysis without contaminating the running production line. As production data is extracted and analyzed in the Data Lake, the running production line is not affected. As a  result the data extracting process not only avoids the risk of production downtime or outages, but also creates a backup for data in the production system. 

6. Guaranteed performance even with a large data volume. Advanced analytics of historical data need a high computation capacity which is hard to achieve without big data models  like Map reduce. Map-reduce models process the data parallel on a group of servers. It allows computation power to be scaled vertically and to significantly shorten the processing time of  historical production data. 

7. Guaranteed performance even with a large data volume. Advanced analytics of historical data need a high computation capacity which is hard to achieve without big data models 

8. like Map-reduce. Map-reduce models process the data parallel on a group of servers. It allows computation power to be scaled vertically and to significantly shorten the processing time of historical production data. 

9. To sum up: Sensors and robots construct the physical part of a digital production \line, a Data Lake further empowers the production line to implement advanced analysis. The physical equipment and information form a joint ecosystem. This ecosystem allows the factory to achieve higher efficiency without affecting the normal production line and also frees the companies up from vendor effects with low storage cost.


INDUSTRIAL 4.0 BIG DATA USE CASES
1. What follows are some selected real-life examples of how the Industry 4.0 big data vision can bring measurable value to manufacturers:

2. Merging quality and production data to improve production quality: A semiconductor manufacturer began correlating single-chip data captured in the testing phase at the end of the production process with process data collected earlier in the process. The manufacturer could then identify faulty chips early on and greatly improve the quality of the production process.

3. Empowered customers: The automotive industry is enthusiastically embracing Industry 4.0 in order to cost-effectively meet consumer expectations for more affordable and digitally connected cars. Among the many use cases of the big data that will be generated by connected cars is the seamless exchange of data with the manufacturer. In addition to improving after-sale service for the individual car-owner, the aggregated information on car performance can be used to improve quality processes and future designs.

4. Reduced downtime: Applicable to many industrial sectors, Industry 4.0 big data analytics can uncover patterns that predict machine or process failures before they occur. Machine supervisors will be able to assess process or machine performance in real time and, in many cases, prevent unplanned downtime.


BIG PICTURE
1. With the rapid spread of IoT and other sensors, the volume and velocity of data are only going to grow—in general, and in the industrial manufacturing sector as well. Just as other sectors have embraced cutting-edge technologies in order to extract value from big data (edge computing, fog computing, cloud computing, and so on), Industry 4.0 is paving the way for widespread big data analytics. 

2. The ROI for manufacturers is already compelling in terms of improved operational efficiency, enhanced quality, and faster response times to ever-changing market signals.

3. Manufacturers today need solutions from providers who are part of the Industry 4.0 revolution and can bring measurable value to their customers across multiple sectors. They need solutions that collect, process, and produce data from many diverse sources and merge this data to provide real-time perspective analytics for 24/7 automated rules and adaptive machine learning. 

4. Most importantly, manufacturers need these solutions to integrate seamlessly with existing enterprise systems in order to align production and quality processes with their core business objectives.


Source:
https://www.datanami.com/2019/04/25/big-data-challenges-of-industry-4-0/

https://www2.deloitte.com/content/dam/Deloitte/de/Documents/technology/Digitalization-Production-Lines-Big-Data-Technologies-Deloitte.pdf