Sr Azure Data Engineer

Genesis10 is currently seeking a Sr Azure Data Engineer for a 5 month contract role in Eden Prairie, MN. This role is 100% remote

Job Summary:

We are looking for a seasoned Azure Data Engineer with strong expertise in Azure Data Factory, Azure Databricks, and PySpark, who also brings hands-on experience in data migration and validation.

The ideal candidate will have a proven track record of migrating Java-based data processing jobs to Azure Databricks using PySpark, and validating data between source and target systems using SQL and PySpark.

This role requires strong hands-on experience with Pyspark and Databricks, particularly in building data validation test frameworks to ensure accuracy from source to target. Additionally, prior experience in the healthcare domain is a key priority for us.

Key Responsibilities:

• Design, develop, and maintain scalable data pipelines using
• Azure Data Factory and Azure Databricks.
• Migrate existing Java-based ETL jobs to PySpark on Azure Databricks.
• Implement standard data quality checks using SQL and PySpark(e.g., null checks, duplicates, referential integrity, record counts).
• Write efficient and optimized PySpark code for large-scale data processing.
• Collaborate with data analysts, architects, and business stakeholders to understand data requirements.
• Perform unit testing and validation of data transformations and ensure data quality.
• Develop and maintain automated validation frameworks for continuous data quality assurance.
• Monitor and troubleshoot data pipelines and workflows in production environments.
• Document technical solutions, migration strategies, and validation procedures.

Required Skills & Qualifications:

• Minimum 6 years of experience in data engineering or related roles.
• Strong hands-on experience with Azure Data Factory, Azure Databricks, and PySpark.
• Proven experience in migrating Java-based data jobs to PySpark on Azure Databricks.
• Solid understanding of SQL and experience in data validation and testing.
• Familiarity with standard data quality checks and validation frameworks.
• Experience working in cloud-based environments, preferably Azure.
• Strong problem-solving skills and ability to work independently or in a team. Excellent communication and documentation skills.

Preferred Qualifications:

• Experience with CI/CD pipelines and DevOps practices in data engineering. Experience working with Talend Studio.
• Knowledge of Delta Lake, Spark optimization techniques, and data lake architectures.
• Exposure to Agile methodologies and project management tools.

Compensation:

Hourly W2 pay: $65.00 to $75.00/hour

We have access to additional contract, contract-to-hire, and direct hire positions with various rate ranges.

If you have the qualifications described and are interested in this exciting opportunity, apply today!

Ranked a Top Staffing Firm in the U.S. by Staffing Industry Analysts for six consecutive years, Genesis10 puts thousands of consultants and employees to work across the United States every year-in contract, contract-for-hire, and permanent placement roles. With more than 300 active clients, Genesis10 provides access to many of the Fortune 100 firms and a variety of mid-market organizations across the full spectrum of industry verticals.

Benefits of Working with Genesis10:

• Access to hundreds of clients, most who have been working with Genesis10 for 5-20+ years.
• The opportunity to have a career-home in Genesis10; many of our consultants have been working exclusively with Genesis10 for years.
• Access to an experienced, caring recruiting team (more than 7 years of experience, on average.)
• Medical, Dental, Vision
• Health Savings Account
• Voluntary Hospital Indemnity (Critical Illness & Accident)
• Voluntary Term Life Insurance
• 401K
• Sick Pay (for applicable states/municipalities)
• Commuter Benefits (Dallas, NYC, SF, and Illinois)
• Remote opportunities available

For multiple years running, Genesis10 has been recognized as a Top Staffing Firm in the U.S., as a Best Company for Work-Life Balance, as a Best Company for Career Growth, for Diversity, and for Leadership, amongst others. To learn more and to view all our available career opportunities, please visit us at our website.

Genesis10 is an Equal Opportunity Employer. Candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.