Senior Data Engineer
UnitedHealth Group View all jobs
- Dublin
- Permanent
- Full-time
- Data Integration: Integrate data from multiple sources and systems, including databases, APIs, log files, streaming platforms, and external data providers. Handle data ingestion, transformation, and consolidation to create a unified and reliable data foundation for analysis and reporting
- Data Transformation and Processing: Develop data transformation routines to clean, normalize, and aggregate data. Apply data processing techniques to handle complex data structures, handle missing or inconsistent data, and prepare the data for analysis, reporting, or machine learning tasks
- Contribute to common frameworks and best practices in code development, deployment, and automation/orchestration of data pipelines
- Implement data governance and data de-identification framework in line with company standards
- Partner with Data Analytics and Product leaders to design best practices and standards for developing and productionalizing analytic pipelines
- Partner with Infrastructure leaders on architecture approaches to advance the data and analytics platform, including exploring new tools and techniques that leverage the cloud environment (Azure, Snowflake, others)
- Monitoring and Support: Monitor data pipelines and data systems to detect and resolve issues promptly. Develop monitoring tools, alerts, and automated error handling mechanisms to ensure data integrity and system reliability
- Proven extensive experience designing data solutions including solid understanding of architectural patterns, system design principles, data modelling, and large-scale solution development
- Proven extensive hands-on experience developing data processing jobs (PySpark /SQL) that demonstrate a solid understanding of software engineering principles
- Proven experience with Azure Databricks and Snowflake
- Proven experience orchestrating data pipelines using technology like ADF, Airflow, etc.
- Proven experience working with Spark
- Demonstrated advanced SQL skills and deep experience with relational databases (e.g., SQL Server), including optimization, stored procedures, and performance tuning for high-volume applications
- Proven experience with DevOps tools, Git workflow and building CI/CD pipelines
- Demonstrated experience applying data governance controls, data masking, and data de-identification rules within a highly regulated environment
- Demonstrated excellent communication and interpersonal skills, with the ability to effectively convey technical concepts to both technical and non-technical stakeholders
- Proven experience analyzing and improving workflows by evaluating process maps and flowcharts, identifying inefficiencies, and recommending enhancements to workflows, procedures, technology, and tools
- Demonstrated familiarity with scripting languages like Java or Scala
- Proven ability to be a motivated self-starter who excels at managing their own tasks and takes ownership
- Proven experience working on projects with agile/scrum methodologies
- Proven ability to be well versed in Python regarding data manipulation, cleaning, transforming, and analyzing structured data to support our data-driven initiatives
- Demonstrated familiarity with production quality ML and/or AI model development and deployment
- Proven Data Architecture experience