Objective: Better Customer Insights for better customer experience
An American-based Pharmacy business management (PBM) company that helps customers manage their pharmacy and healthcare benefits. This client wanted to better leverage its data to improve customer service, operations, and insights. The modernization to GCP and BigQuery brings new analytics capabilities and moves decision-making nearer to real-time.
This client sought a technology partner who could help them efficiently and safely modernize and thereby leverage GCP cloud-native technologies.
Challenges: High Technical and Cost Issues
The client had been experiencing technical, scalability, and cost issues with the existing Teradata and Hadoop solutions. They wanted to leverage cloud-native technologies offered by Google Cloud Platform (GCP) to improve customer insights and better manage operational costs.
Solutions: Designing GCP Future state architecture
- Datametica provided the client with a detailed assessment of the existing environment by using Eagle- an automated assessment & data migration planning tool and provided with the detailed migration plan, along with an understanding of the current data model, ETL logic, associated workloads, pipelines, and deliverables.
- Datametica created a new architecture for GCP to support the technical, functional, and non-functional requirements.
- The Datametica data ingestion framework uses GCP Dataflow, with capabilities to ingest different file formats (CSV, PSV, Fixed Width, EBCDIC, EDI).
- Data audit and data quality frameworks were integrated with Cloud Composer to provide solid data auditing and data governance.
- Data pipelines were set up for historical and incremental/ongoing data load from the data marketplaces, using BigQuery SQL, stored Procedures, and cloud composer.
- Built data pipelines with data streaming via Kafka to Bigquery.
- Created integration with Security Framework for role-based access control (“RBAC”).
- Created integration with ServiceNow for Incident creation upon job failures in GCP.
- Converted and repointed DataStage ingestion and transformation jobs to GCP native by using Datametica’s Automated code (SQL, Script) and ETL conversion Tool – Raven.
- Setup orchestration and scheduling of Cloud Composer.
- Performed data validation using the Datametica Automated data validation tool – Pelican tool between Hadoop, Teradata, and BigQuery.
- Leveraged Datastudio GCP for Audit and data quality monitoring dashboards.
Client Benefits: Faster cloud migration, greatly enhanced analytics capabilities, and lower operation cost
- Reduced operational cost and infrastructure provisioning timelines.
- Improvement in efficiency and performance due to consolidation of different market places into a single unified data model on GCP.
- Using automation, Datametica migrated complex Teradata and Hadoop data warehouses, and analytics and reporting workloads to GCP.
- Datametica solutions delivered faster cloud migration, with lower costs and lower business risk.
- Ensured confidence in decommissioning of Teradata and Hadoop Warehouses with help of Pelican.
GCP Products Used
Datametica Products Used
