Objective
The main goal is to extend the functionality of the current Data Transfer solution to guarantee data security and classification in an automated way. These enhancements will need to be deployed within the same AWS account as the existing solution, adhering to all current agreements. The solution’s design should be modular and reusable, allowing for potential deployment by other teams within the customer using Infrastructure as Code (IaC). Any additional deployments beyond the current scope will be addressed separately.
Inbound Solution
The Inbound Solution includes several AWS components to ensure security, efficiency, and scalability. AWS Lambda functions handle backend processing tasks, including data fetching from third-party vendors and SaaS products, as well as inbound file conversion from CSV to Parquet. These functions are orchestrated using AWS Step Functions to maintain seamless data pipeline execution. Incoming files are stored in Amazon S3, where they are scanned for security threats by Trend Micro Cloud One File Storage Security. Amazon WorkSpaces offers virtual desktops for end-user access. The solution mandates the use of AWS Identity Center (formerly AWS SSO) for user access via IAM roles, providing secure and controlled access. A lifecycle policy for Amazon S3 ensures files are removed in alignment with the credential expiration policy. AWS Lake Formation is utilized for data governance, ensuring proper data classification and security, while AWS Glue is employed for data cataloging and ETL processes. Additionally, AWS Macie is used to discover and protect PII data.
Within the customer’s Data Platform, Amazon CloudWatch (CW) alarms are extensively used to ensure process robustness and reliability. CloudWatch alarms are configured to monitor AWS Lambda functions, providing immediate notifications in case of failures, enabling prompt responses to maintain seamless operation. Alarms are also set for AWS Glue crawler runs to alert on failures, ensuring the data catalog remains up-to-date and accurate. Additionally, CloudWatch alarms monitor the count of messages in Amazon SQS queues. When the message count exceeds a specified threshold, these alarms trigger AWS Step Functions or Lambda functions to handle the increased load, thereby maintaining optimal performance and throughput of the data pipeline. This comprehensive use of CloudWatch alarms facilitates proactive monitoring and automatic scaling of the infrastructure, ensuring high availability and reliability of their services.
Furthermore, various file types, such as CSV, JSON, and Parquet, are stored in Amazon S3 buckets, which are encrypted using customer-managed KMS (Key Management Service) keys. Access to these buckets is tightly controlled through IAM (Identity and Access Management) policies and KMS key policies, ensuring that only authorized users and services can access the encrypted data. Additionally, KMS is leveraged for encrypting various AWS services within the platform, including SNS (Simple Notification Service), SQS (Simple Queue Service), Lambda environment variables, Secrets Manager, and DynamoDB. This comprehensive use of KMS across the customer’s services ensures robust encryption and security, protecting sensitive data both at rest and in transit. Moreover, AWS Security Hub, Amazon GuardDuty, and AWS Config are employed to enhance the security posture, while AWS Control Tower Service Control Policies (SCPs) govern which services and regions can be used, and solutions are deployed to multiple AWS accounts to maintain a secure and compliant multi-account environment.
Business Impact
The enhanced Data Transfer solution significantly improves the customer's ability to manage and share data securely and efficiently. The implementation of MFA and lifecycle policies strengthens security, ensuring compliance with AWS data privacy best practices and regulatory requirements. The modular design and use of Infrastructure as Code (IaC) enable scalability, allowing other teams within the customer to deploy similar solutions effortlessly. These enhancements lead to better data governance, increased operational efficiency, and robust security, ultimately supporting the customer's growth and innovation.
By leveraging AWS's comprehensive suite of services, the customer can now handle inbound data transfers with greater confidence, meeting both current needs and future expansion plans.
VodafoneZiggo Upskills Data-Savvy Employees to Become Data Engineers
Xebia partners with the telecommunication company to upskill its professionals and fill critical data engineering positions.
Wehkamp's Journey to Cost Optimization with AWS and Xebia
Wehkamp, one of the largest online retailers in the Netherlands, aimed to achieve greater speed, elasticity, and scalability by transitioning to the cloud with AWS and Xebia.
Xebia's Collaboration with RTS for Data-Driven Excellence
Xebia and RTS join forces to enhance media impact through an ethical data strategy, unlocking potential and fostering collaboration.
Streaming Platform Adds Required Age Verification for International Launch
Xebia helped this Streaming Services client create, execute, and monitor legally binding age verification services in order to launch in specific countries.
Dutch Banks Assemble to Fight Financial Crime
TMNL helps Dutch banks monitor suspicious transactions
Interflon: Adopting the Cloud in Just 6 Months
To increase data security and embrace new business opportunities, a leading lubrication solutions manufacturer decided to move to the Cloud.
Energy Consumption Insight Provider processes and generates data faster with scalable cloud architecture
An energy consumption application with over 100,000 users, faced data processing challenges due to the volume of energy meter data, prompting Xebia's assistance in developing a scalable solution using AWS services.
Driving Sustainability: Blonk Partners with Xebia for Cloud Carbon Reduction
Xebia employed a dashboard and implemented daily updates to provide Blonk with almost real-time visibility into their AWS carbon emissions, elevating their understanding and management of these emissions.
Helping Blonk scale through Cloud Native Development on AWS
Xebia configured the AWS landing zone according to the best practices of the AWS Cloud Foundation. With this modernized infrastructure as a base, Xebia further guided the development team at Blonk by adopting cloud native tools on the AWS platform and formulated a clear innovation path.
Abacai: Revolutionizing Insurance with an Omnichannel Experience
A digital-first UK car insurer teamed up with Xebia to challenge a traditional industry with cutting-edge artificial intelligence.
Innovative Scale-Up Leverages Data to Insure Small Entrepreneurs
Insify harnesses data resources to offer digital insurance solutions more quickly and competitively through a tailor-made cloud platform
C-Facts Realizes Innovative Control Center for Sustainable Cloud Services
Cloud-native provider of insights into digital cloud footprints undergoes a Well-Architected Review of its AWS platform to ensure scalability; implements solutions that result in less downtime and improved time-to-market — and more satisfied customers overall
Yell Secures Future With Online Reputation Management
The UK’s leading online directory developed an all-in-one social monitoring tool, enhancing its position as an essential business service
Cloud Platform Move Enhances Insurance App’s USP
A migration to AWS Cloud enhanced an SaaS app developer's unique services by improving product response times and features
Leading FinTech Brand Retains Market Lead With Cloud Solution
The world-leading FinTech company migrated to AWS Cloud, modernizing its platform to ensure it maintains its strong market position
AIXBRO Speeds up Searches With AWS Serverless
The leading Swiss automotive parts distributor used a Proof of Concept to discover how the cloud could secure its future growth
Kynetec Improves Automation, Cuts Processing Times by a Quarter
The leading agricultural research organization moved to serverless architecture, allowing data analysts more time to improve the company product
Serverless Improves Automotive SaaS Solution at Fraction of Cost
In automating its serverless platform using the AWS Cloud to meet the automotive industry’s needs, metrologx gained widely applicable cost and performance benefits
Serverless Architecture Increases Coople Innovation
Europe's largest digital staffing platform moved to Amazon Web Services cloud hosting, enhancing its market-leading web app development
Data Security Key to Coople Cloud Integration
Europe’s largest online staffing agency conducted a full audit of its AWS cloud architecture to ensure users’ personal data remains fully secure
Hospitality Industry Goes Cloud-First With Foodback
Norwegian startup helps businesses manage feedback in real-time cloud-based hospitality app hosted on AWS
AWS Cloud & Machine Learning Help RASP Focus On Quality Content
Innovative Polish publishing house optimizes its publications for commercial partners with image recognition tool that streamlines editorial process
Advanced Leverage DevOps Engineers, Accelerate Cloud Migration
Advanced, a leading management software supplier, migrated an acquisition to an in-house AWS infrastructure for greater efficiency, simultaneously upskilling its team
Lift & Shift Migration Makes IT More Cost-Effective for Software Provider
Advanced — a dynamic business software and services provider — migrated its Marketplace software to a bespoke cloud solution, resulting in a cheaper and more efficient IT infrastructure
Sage Cloud Software Gets Boost With Carbon React
Accounting specialists create a component library that improves client collaboration — and satisfaction
Leading Fintech App Optimizes AWS to Give Premium Service
Bizcuit overcomes cloud overwhelm to better serve customers while keeping the highest compliance standards.
EnergyAlert Optimizes Efficiency Using IoT Technology
Dutch Energy monitoring company expands its capabilities with state-of-the-art IoT services and techniques
Sparco Saves Time and Cost with New Cloud Platform Initiative
Dutch marketing maverick works its magic with a new cloud platform and enhanced infrastructure
Global Leader in Agri-Research Quadruples its Processing Speed Thanks to Improved Automation
Serverless architecture and cloud-based solutions increase Kynetec’s data processing efficiency by 77%
Dutch Railway Company Journeys to the Cloud with End-to-End AWS Deployment
NSI migrates to a cloud environment improving design and development along the way
B2B Tool Provider, Advanced, Migrates Infrastructure to the Cloud
British business software provider streamlines IT by migrating infrastructure to the Cloud; gives its customers a competitive edge
dsm-firmenich Launches “Green” Calculator for Farming Industry
Global science-based company use Amazon Web Services (AWS) to create a highly scalable tool to radically reduce environmental impact of animal farming
Luxury IT Company Creates Cloud-Centered Infrastructure to Serve High-end Automotive Retailers
PON IT delivers scalable, security-minded, Agile products and IT services through successful cloud migration
Cloud migration and application migration: a structured process towards a solid and reliable application environment.
Dutch health provider ZuidZorg successfully migrates old systems to the cloud in under three months
Rail Ticket Agent Gets A Pass with Cutting-Edge Upgrade to Its App
Eurail increases delivery times with upgraded services and app built on AWS cloud architecture
Wehkamp new Architecture Automates DevOps Operations
Automating Continuous Integration and Delivery on AWS for Top E-Commerce Biz in the Netherlands
Dutch Financial Pension Provider Unlocks Hybrid Cloud Landscape for Future Security
Embracing Agile, MN Pensioen migrates to a hybrid cloud environment while upgrading security features and gaining efficiency
Dutch Energy Provider Lights Up Competition with Daring New IoT Platform
Kenter utilizes creative IoT platform as an innovative, affordable measure of energy use, giving customers more savings options
The Largest Digital Marketplace For Floriculture.
World’s largest flower auction company increases purchasing and sales capacities for its member growers and international buyers in the global floral trade
Professionalizing BI Processes to Improve the Customer Experience
Learn how entertainment business Pathé professionalized its BI processes, and became better at improving customer experience and predicting customer numbers
