Home » Case Study »
Objective: Migrating Complex Teradata Environment to Google Cloud
Datametica’s client, an American-based retailer, wanted to migrate its Teradata environment to Google Cloud Platform (GCP), in order to optimize performance, cost and remove overall infrastructure provisioning delays.
The client was looking for a partner to enable their migration from the on-premise Teradata DataWarehouse to the Google Cloud Platform, with upgraded versions of Cognos for enterprise reporting.
Challenges: Technical, Scalability, and Costs Issues
The client was experiencing technical, scalability, and cost issues with their existing Teradata and Informatica solutions which included overhead expenses like operational and maintenance costs.
Key technical challenges faced by the client –
- Very complex legacy Cognos reporting
- Long running and complex analytics cube building
- Concern on how to integrate enterprise scheduling tasks from Control-M
- A desire to move orchestration and dependency management through composer
Legacy:
- Control-M Jobs
- Tables/Views/Macros
- Informatica Mappings
- Shell scripts
- SQLs, BTEQ’s and UDF’s
- .NET/C# Applications
Solutions: Lift and Shift Migration to GCP
Our team deployed Datametica’s state-of-the-art automation technologies to migrate from Teradata to Google Cloud Platform with only small changes to the data model.
- Datametica provided the client with a detailed assessment of the existing environment by using Eagle – an automated assessment & data migration planning technology – and provided the detailed migration plan, along with an understanding of the current data model, ETL logic, associated workloads, pipelines, and deliverables.
- Planned the jobs and tables based on the dependencies by leveraging strong technical analysis of the on-premise tools and understanding of the data model.
- Converted and repointed Informatica ingestion and transformation jobs to GCP by using Datametica’s Automated Code (SQL, Script) and ETL conversion service – Raven.
- Performed data validation using the Datametica Automated data validation tool – Pelican between Teradata and BigQuery.
- Implemented the GCS Fuse mechanism in order to establish communication between Control-M (on-premise workload automation tool) and GCP Cloud Composer.
- Cloud Data Fusion – a fully managed, cloud-native, enterprise data integration service – was leveraged for quickly building and managing data pipelines.
- Composer DAGs integrated with Wiki in order to maintain strong documentation for failure handling scenarios.
- C# Application Optimization done for uploading data into BigQuery.
- Optimization of complex and long-running Cognos reports
- Modernization incorporated into GCP
- Eliminated the old source system for sales subject area by implementing the new data pipelines that has eliminated the bugs in the old source.
- Reconciliation / validations of new source system for sales data.
- Cognos Compatible Query Mode (CQM) models converted into Dynamic Query Mode models (DQM).
- Cognos IQD converted into Cognos FrameWork (FM) models
- Changes from multiple database connections to single connection for Cognos packages.
Client Benefits: Improved Efficiency, Reduced Cost, and Optimized Infrastructure Provisioning Timelines
- Improvements in efficiency and performance due to the consolidation of different market places into a single unified data model on GCP.
- Using automation, Datametica migrated complex Teradata data warehouse, analytics, and reporting workloads to GCP.
- Datametica solutions delivered faster cloud migration with lower costs and lower business risk.
- Collated a number of Teradata objects were duplicated, reducing maintenance.
- Automated Pelican testing, gave confidence in the decommissioning of the Teradata Warehouse.
- Ability to eliminate compute restrictions as GCP we can run more reports at a same time
- Optimization of Cognos Reports : Three of the most critical reports had the most significant improvements, some were improved from ~75 minutes to ~30 seconds.
GCP Products Used
Datametica Products Used
