MARUBENI POWER CASE STUDY
Automating Data Ingestion:
Serverless Solution for Marubeni Power
Marubeni Power, an international energy trading company, faced the challenge of manually processing complex data streams. They partnered with Futuralis, who used AWS's suite of cloud services to automate Marubeni's data ingestion and processing, significantly enhancing performance, scalability, and security. The comprehensive cloud-based solution developed by Futuralis has led to increased operational efficiency and data-driven decision-making capabilities for Marubeni Power.
Marubeni Power's core challenge was their dependence on manual processes for handling data from disparate sources, such as CAISO/OASIS API, World Weather Online API, and Azure Table Storage. This manual handling was error-prone, slow, and consumed significant resources, posing the risk of operational inefficiency and potential revenue loss. Their traditional infrastructure was also plagued by high operational overheads, limited resource utilization, and hindrance to agility and innovation.
Why Futuralis & AWS
Marubeni Power chose AWS due to its versatile and robust suite of services which could efficiently manage and process large data streams from various sources. AWS's scalability, cost-effectiveness, and ability to automate manual tasks made it the ideal platform for Marubeni's need for performance and security. Marubeni Power chose Futuralis due to their deep expertise in delivering tailored cloud solutions, successful track record, and customer-focused approach, which matched Marubeni's aim of achieving innovation and operational efficiency.
Futuralis devised a comprehensive solution using several AWS services. By implementing AWS CodeCommit and AWS CodeBuild, Futuralis set up CI/CD pipelines to automate the development and deployment processes.
AWS CloudFormation templates were used for defining infrastructure as code, enabling automated updates. Futuralis also leveraged AWS Step Functions to automate complex data ingestion workflows, with AWS Lambda for serverless data processing.
Amazon S3 was used for secure, scalable data storage, while Amazon RDS handled different data types. AWS Step Functions workflow was designed to manage data ingestion, processing, and transformation from different sources, and in case of any transient failure, a retry mechanism was implemented. Lastly, Amazon SNS was used to alert the admin in case of any issues in the data processing pipeline.
Results & Benefits
The automation of data ingestion significantly reduced manual efforts, leading to an efficient and reliable process. Faster development and deployment cycles, enabled by the DevOps approach and AWS services, increased agility and reduced time-to-market.
Dynamic resource allocation based on demand led to improved cost optimization and enhanced resource utilization. By breaking down a complex Lambda function into smaller steps using Step Functions, the overall execution time and associated costs were reduced, leading to cost-effective workflows.
Scalability and Performance
The concurrent execution of Lambdas through Step Functions enabled improved scalability and performance, efficiently handling high workloads and reducing processing times. The robust retry mechanism ensured the system could recover from transient failures autonomously, contributing to a more reliable and efficient system.
About Marubeni Power Case Study
Download Case Study
Fill the form to get more information