DB-HANDLER

MongoDB Backup System/Simple Anomaly Detector

Project Overview

  • Project Name : DB-HANDLER
  • Version : v1.0
  • Last Modified : 09.09.2024

Thank you for taking the time to visit the DB-HANDLER project! The MongoDB Handler project was developed to address an issue with the FANUC MTLinki system, which is used to capture various operational signals from CNC machines in the company. These signals include data such as servo load, energy consumption, fan speeds, and temperatures. The database storing this information is MongoDB, hosted on a virtual machine.

The company faced a critical problem where the virtual machine's disk would fill up within three months due to the large amount of data being logged. This caused severe performance degradation and required frequent reinstallation of the FANUC MTLinki system, resulting in the loss of valuable operational data.

The goal of this project was twofold: first, to design a backup system to ensure that the company would retain the data from FANUC MTLinki for further analysis; and second, to eliminate the disk overload issue, preventing performance drops and the need to reinstall the MTLinki system.

System Overview

A data backup system was designed to transfer data from the MongoDB database to SQLite databases. A script continuously monitors new data being logged into MongoDB, and as new data becomes available, it transfers it to the corresponding SQLite databases for each machine and removes the data from MongoDB.

This solution allowed for the continuous cleanup of the MongoDB database used by the FANUC MTLinki system while simultaneously creating backups. SQLite databases were chosen because they are file-based, making it easy to transfer them from the virtual machine to an external drive designated for backup purposes.

As a result, the company no longer loses data every three months and has direct access to the archived information. Additionally, the issue of the MongoDB database filling up has been resolved, ensuring that the database size remains stable, operates at full efficiency, and no longer requires system reinstallation.

Project Summary

This system enables the collection of valuable machine operation data, opening up a wide range of possible analyses that can be conducted using this information.

In addition, I implemented a script on the virtual machine that monitors the size of the FANUC MTLinki database and the backup databases in real time. After several months of system operation, it is clear that the solution is effective—MongoDB no longer fills the disk, and the rate at which the database size increases has significantly decreased.

Before the system was implemented, the MongoDB database would fill up in three months. After the implementation, assuming the system runs continuously, the database is projected to fill up in about 150 years! The only requirement is that the backup needs to be transferred to an external drive once a month.

Density Based Predictor

The Density Based Predictor is a script developed for real-time analysis of signals from the MongoDB database. It allows users to select a machine and an operational signal, such as servo load, and analyze the signal in real-time. The script provides a visual representation of the signal and includes a built-in algorithm that detects anomalies based on a predefined threshold.

For example, the tool can be used to analyze tool wear by tracking the servo load over time. The script measures the density of anomalies, and when this density exceeds the set threshold, it sends a notification to the user, alerting them to potential tool wear.