As a Senior Data Engineer, you will design and build scalable data pipelines and ensure reliable data infrastructure while collaborating with data scientists. Your role includes transforming datasets, implementing ETL pipelines, and employing cloud-based solutions to support data architecture projects.
Our client is currently undergoing a transition from the Lucernex database hosted on Google Cloud Platform (GCP) to an Azure-based environment using Databricks. We are seeking a skilled Data Migration Specialist to support this critical migration effort. The role will focus on data extraction, transformation, and loading (ETL) across different data platforms, ensuring smooth and reliable migration processes while maintaining data quality. If you have expertise in data ingestion, integration, migration, and cloud-based environments, we'd love to hear from you..
Requirements
- Strong experience in data migration, specifically in moving data across platforms (GCP to Azure).
- Expertise in handling ETL processes, particularly using tools like Altova MapForce.
- Experience working with databases such as SQL Server.
- Familiarity with cloud-based platforms, especially GCP and Azure.
- Knowledge of DevOps tools like Bitbucket and Jenkins for CI/CD processes.
- Ability to work with various data formats (CSV, screenshots, etc.) and create data mappings between systems.
- Strong problem-solving skills with an emphasis on data quality and integrity.
- Experience in environment management and administering cloud platforms.
- Excellent communication skills to collaborate with clients on data mapping and migration requirements.
Responsibilities
- Data Ingestion: Extract data from various source platforms and systems.
- Integration: Combine and transform data from multiple sources to ensure consistency and usability.
- Data Quality: Ensure the accuracy, completeness, and reliability of the data throughout the migration process.
- Migration: Facilitate the movement of data to the destination storage, ensuring proper handling and processing.
- Testing: Validate that the processed data meets business requirements and is fit for purpose.
- Environment Management: Administer the platforms and tools, ensuring the smooth operation of the migration process and maintaining efficient workflows.
Similar Jobs
Artificial Intelligence • Healthtech • Telehealth
As a Data Engineer II, you will maintain and expand a data platform, ensuring its reliability and stability. Your responsibilities include building and operating data infrastructure, maintaining CI/CD pipelines, and collaborating with stakeholders for data requirements. You'll leverage tools like Airflow, dbt, and AWS to optimize data processing and documentation.
Top Skills:
AirflowAWSCircleCIDatadogDbtGitPythonSnowflakeSQLTerraform
Big Data • Fitness • Healthtech • Software • Analytics • Energy
The Data Engineer at Arcadia will design, build and test data connectors and ingestion pipelines, contributing to the integration of client claim and clinical data platforms. Responsibilities include peer code reviews, sprint planning, and maintaining JIRA tasks. The role requires problem solving for coding and data analysis issues within a dynamic healthcare environment.
Cloud • Information Technology • Productivity • Security • Software • App development • Automation
As a Senior Data Engineer at Atlassian, you will design data models, build scalable data pipelines, and improve data quality for informed decision-making across the organization. Collaboration with various teams is key, alongside ownership of data extraction and cleaning processes, ensuring agile and effective data engineering practices.
What you need to know about the Seattle Tech Scene
Home to tech titans like Microsoft and Amazon, Seattle punches far above its weight in innovation. But its surrounding mountains, sprinkled with world-famous hiking trails and climbing routes, make the city a destination for outdoorsy types as well. Established as a logging town before shifting to shipbuilding and logistics, the Emerald City is now known for its contributions to aerospace, software, biotech and cloud computing. And its status as a thriving tech ecosystem is attracting out-of-town companies looking to establish new tech and engineering hubs.
Key Facts About Seattle Tech
- Number of Tech Workers: 287,000; 13% of overall workforce (2024 CompTIA survey)
- Major Tech Employers: Amazon, Microsoft, Meta, Google
- Key Industries: Artificial intelligence, cloud computing, software, biotechnology, game development
- Funding Landscape: $3.1 billion in venture capital funding in 2024 (Pitchbook)
- Notable Investors: Madrona, Fuse, Tola, Maveron
- Research Centers and Universities: University of Washington, Seattle University, Seattle Pacific University, Allen Institute for Brain Science, Bill & Melinda Gates Foundation, Seattle Children’s Research Institute
