What is KeyBridge?
KeyBridge is a multi-disciplinary project using expertise from systems-programming, data visualization and machine learning in order to understand massive streams of data and apply the lessons learned to systems operation.
We understand that analysis of multi-terabyte-per-day streaming data requires more than just black-box machine learning. We know the importance of data-cleaning; we know the importance of visualizing and understanding the data; we know that system structure and networking are key to reliable applications. We have successfully applied our expertise to provide insights in areas of hardware reliability, network operations, and security.
The mission of the KeyBridge project is to consider the data-analysis problem end-to-end, from data acquisition and ingestion, to reporting and action. KeyBridge researchers cover a wide range of expertise including systems programming, data visualization, and cutting-edge machine learning.
We collaborate with product groups and business units to analyze petabytes of real-world data. We actively work with our collaboration partners to establish whether particular technology can solve their business problems. We believe in a collaborative development approach, and we focus on using technology to address real business needs.
Amongst the systems we have built and deployed are:
Network monitoring: using network telemetry to discover faults and operational issues.
Security monitoring: using server logs to alert on ad-hoc events with security implications.
Storage Reliability: using storage-media health-logs to predict imminent failure.
Server Reliability: using server logs to understand the root cause of memory and server failures.
We are currently working on further systems for anomaly detection in security and other areas, applying our expert knowledge to business needs.