BACK TO WORKS

Data Engineering on Unstructured Dataset Using AWS for a US-Based Home Automation Company

Industry

Services

Home Automation , Cloud , Software Engineering , Data Platform Engineering

Overview

 

Our client is a US-based OEM producing HVAC equipment, water heaters, and boilers for residential and commercial buildings. They needed a secure and economical solution for large data set analysis.

 

Data Engineering on Unstructured Dataset Using AWS-overview

Challenges

 

 

Structure the format of 120K+ live devices that send 40GB data per day, and it is expanding

 

Structure the format of 120K+ live devices that send 40GB data per day, and it is expanding

Designing a scalable, secure, and cost-effective solution

 

Designing a scalable, secure, and cost-effective solution with flexible architecture and

Outcomes

 

 

  • Enabled faster data transformation by dividing day execution into hour execution for time series data
  • Designed pipelines to process data for 1 year that continuously delivered meaningful insights to client
Data Engineering on Unstructured Dataset Using AWS-Outcome