MARUBENI POWER CASE STUDY

Automating Data Ingestion:
Serverless Solution for Marubeni Power

Challenges

Manual, error-prone data ingestion from various sources.
Slow and inefficient data processing.
Poor resource utilization causing high operational overheads.

Benefits

Improved efficiency through automation of data ingestion.
Enhanced resource optimization, leading to better cost management.
Increased system scalability and performance with concurrent Lambda execution.

Executive Summary

Marubeni Power, an international energy trading company, faced the challenge of manually processing complex data streams. They partnered with Futuralis, who used AWS's suite of cloud services to automate Marubeni's data ingestion and processing, significantly enhancing performance, scalability, and security. The comprehensive cloud-based solution developed by Futuralis has led to increased operational efficiency and data-driven decision-making capabilities for Marubeni Power.

The Challenge

Marubeni Power's core challenge was their dependence on manual processes for handling data from disparate sources, such as CAISO/OASIS API, World Weather Online API, and Azure Table Storage. This manual handling was error-prone, slow, and consumed significant resources, posing the risk of operational inefficiency and potential revenue loss. Their traditional infrastructure was also plagued by high operational overheads, limited resource utilization, and hindrance to agility and innovation.

Why Futuralis & AWS

Marubeni Power chose AWS due to its versatile and robust suite of services which could efficiently manage and process large data streams from various sources. AWS's scalability, cost-effectiveness, and ability to automate manual tasks made it the ideal platform for Marubeni's need for performance and security. Marubeni Power chose Futuralis due to their deep expertise in delivering tailored cloud solutions, successful track record, and customer-focused approach, which matched Marubeni's aim of achieving innovation and operational efficiency.

Our Solution

Futuralis devised a comprehensive solution using several AWS services. By implementing AWS CodeCommit and AWS CodeBuild, Futuralis set up CI/CD pipelines to automate the development and deployment processes.

AWS CloudFormation templates were used for defining infrastructure as code, enabling automated updates. Futuralis also leveraged AWS Step Functions to automate complex data ingestion workflows, with AWS Lambda for serverless data processing.

Amazon S3 was used for secure, scalable data storage, while Amazon RDS handled different data types. AWS Step Functions workflow was designed to manage data ingestion, processing, and transformation from different sources, and in case of any transient failure, a retry mechanism was implemented. Lastly, Amazon SNS was used to alert the admin in case of any issues in the data processing pipeline.

Results & Benefits

Operational Efficiency
The automation of data ingestion significantly reduced manual efforts, leading to an efficient and reliable process. Faster development and deployment cycles, enabled by the DevOps approach and AWS services, increased agility and reduced time-to-market.

Resource Optimization
Dynamic resource allocation based on demand led to improved cost optimization and enhanced resource utilization. By breaking down a complex Lambda function into smaller steps using Step Functions, the overall execution time and associated costs were reduced, leading to cost-effective workflows.

Scalability and Performance
The concurrent execution of Lambdas through Step Functions enabled improved scalability and performance, efficiently handling high workloads and reducing processing times. The robust retry mechanism ensured the system could recover from transient failures autonomously, contributing to a more reliable and efficient system.

About Marubeni Power Case Study

Tokyo-based multinational, Marubeni Corporation, operates extensively in various sectors, with a notable presence in the energy and power industries, including both conventional and renewable sources. Their diverse business activities also encompass lifestyle, IT, food, agriculture, chemicals, metals, finance, real estate, construction, industrial machinery, and emerging business sectors.

About Futuralis

Migrate – Modernize – Manage

Futuralis is a global tech consulting firm and AWS Advanced Tier Services Partner based in Washington, DC. We specialize in migrating complex workloads to AWS, modernizing them into fast, enterprise-grade solutions, and providing top-tier cloud management. Our ethos of Shared Business Excellence guides our pursuit of client capability enhancement, global customer satisfaction, and measurable results.