Sr. Data Engineer / Lead - 67453

Function

We're Hitachi Digital Services, a global digital solutions and transformation business with a bold vision of our world's potential. We're people-centric and here to power good. Every day, we future-proof urban spaces, conserve natural resources, protect rainforests, and save lives. This is a world where innovation, technology, and deep expertise come together to take our company and customers from what's now to what's next. We make it happen through the power of acceleration.

Imagine the sheer breadth of talent it takes to bring a better tomorrow closer to today. We don't expect you to 'fit' every requirement - your life experience, character, perspective, and passion for achieving great things in the world are equally as important to us.

Sr. Data Engineer / Lead

You will be joining a high-performing Data Engineering & Analytics team focused on building scalable, cloud-native data platforms on AWS. The team works closely with Product Owners, Business Analysts, Data Scientists, and Analytics teams to deliver modern data solutions that enable advanced reporting, analytics, and business intelligence.

The team is passionate about:

• Building scalable AWS-based data lake and analytics platforms
• Delivering reliable and high-performance ETL/ELT solutions
• Leveraging Talend Cloud 8.0, PySpark, and Spark SQL for enterprise-grade data processing
• Supporting real-time and batch data integration across multiple data sources
• Driving data quality, governance, automation, and operational excellence
• Implementing reusable frameworks and best practices for modern data engineering

As a Senior Data Engineer, you will play a key role in designing, developing, and optimizing enterprise data pipelines that power business-critical analytics and reporting solutions.

• Design, develop, and maintain ETL/ELT workflows using Talend Cloud 8.0
• Build scalable data ingestion pipelines from APIs, flat files, databases, and streaming sources
• Develop and support large-scale data processing solutions using PySpark and Spark SQL
• Design and manage AWS-based data lake architectures utilizing Amazon S3
• Develop cloud-native data solutions leveraging AWS services such as:
  • AWS Glue
  • AWS Lambda
  • Amazon Athena
  • Amazon EMR
• Build and support Python-based microservices for data integration and processing workloads
• Implement and maintain streaming data pipelines for near real-time data processing
• Optimize Spark jobs, ETL workflows, and AWS workloads for performance, scalability, and cost efficiency
• Ensure data quality, reliability, governance, and operational monitoring across the platform
• Collaborate with business stakeholders, architects, analysts, and development teams to deliver high-quality data solutions
• Troubleshoot production issues and continuously improve platform performance and reliability

• 10+ years of experience in Data Engineering, Big Data, ETL Development, or Data Integration
• Strong hands-on experience with Talend Cloud 8.0 / Talend ETL
• 6+ years of experience developing Python-based microservices
• 2+ years of hands-on experience with PySpark and Spark SQL
• 8+ years of experience working with AWS cloud services
• 3+ years of experience handling streaming data pipelines and real-time data processing
• Experience working on multiple enterprise-scale data engineering projects

• Talend Cloud 8.x development and administration
• PySpark development and Spark SQL optimization
• Python programming and reusable framework development
• AWS Services including:
  • Amazon S3
  • AWS Glue
  • AWS Lambda
  • Amazon Athena
  • Amazon EMR
• Data Lake architecture and implementation
• ETL/ELT design patterns and best practices
• Streaming technologies and event-driven architectures
• Data modeling, transformation, and schema management
• Performance tuning and optimization of Spark workloads
• Data quality, monitoring, and operational support

• Strong analytical and problem-solving capabilities
• Ability to build scalable, production-grade data solutions
• Excellent communication and stakeholder management skills
• Experience working in Agile delivery environments
• Strong collaboration skills across engineering, analytics, and business teams
• Ability to mentor junior engineers and contribute to engineering best practices

We're a global, team of innovators. Together, we harness engineering excellence and passion to co-create meaningful solutions to complex challenges. We turn organizations into data-driven leaders that can make a positive impact on their industries and society. If you believe that innovation can bring a better tomorrow closer to today, this is the place for you.

Hitachi is a global company operating across a wide range of industries and regions. One of the things that sets Hitachi apart is the diversity of our business and people, which drives our innovation and growth.

We are committed to building an inclusive culture based on mutual respect and merit-based systems. We believe that when people feel valued, heard, and safe to express themselves, they do their best work.

We help take care of your today and tomorrow with industry-leading benefits, support, and services that look after your holistic health and wellbeing. We're also champions of life balance and offer flexible arrangements that work for you (role and location dependent). We're always looking for new ways of working that bring out our best, which leads to unexpected ideas. So here, you'll experience a sense of belonging, and discover autonomy, freedom, and ownership as you work alongside talented people you enjoy sharing knowledge with.

We're proud to say we're an equal opportunity employer and welcome all applicants for employment without attention to race, colour, religion, sex, sexual orientation, gender identity, national origin, veteran, age, disability status or any other protected characteristic. Should you need reasonable accommodations during the recruitment process, please let us know so that we can do our best to set you up for success.