
Senior Data Engineer
- Dublin
- Permanent
- Full-time
- Data Integration: Integrate data from multiple sources and systems, including databases, APIs, log files, streaming platforms, and external data providers. Handle data ingestion, transformation, and consolidation to create a unified and reliable data foundation for analysis and reporting
- Data Transformation and Processing: Develop data transformation routines to clean, normalize, and aggregate data. Apply data processing techniques to handle complex data structures, handle missing or inconsistent data, and prepare the data for analysis, reporting, or machine learning tasks
- Contribute to common frameworks and best practices in code development, deployment, and automation/orchestration of data pipelines
- Implement data governance in line with company standards
- Partner with Data Analytics and Product leaders to design best practices and standards for developing and productionizing analytic pipelines
- Partner with Infrastructure leaders on architecture approaches to advance the data and analytics platform, including exploring new tools and techniques that leverage the cloud environment (Azure, Snowflake, others)
- Monitoring and Support: Monitor data pipelines and data systems to detect and resolve issues promptly. Develop monitoring tools, alerts, and automated error handling mechanisms to ensure data integrity and system reliability
- Bachelor’s Degree (or higher) in Database Management, Information Technology, Computer Science or similar
- Proven experience designing data solutions, including data modelling
- Hands-on experience in data processing using PySpark and SQL
- Experience with orchestration tools like Azure Data Factory or Airflow
- Fluent in SQL, including advanced functions and window operations
- Demonstrated experience with DevOps tools and CI/CD pipelines
- Experience applying data governance controls in regulated environments
- Well versed in Python regarding data manipulation, cleaning, transforming, and analyzing structured data to support our data-driven initiatives
- Proven Data Engineering experience
- A motivated self-starter who excels at managing their own tasks and takes ownership
- Experience working in projects with agile/scrum methodologies
- Experience with Azure Databricks and Snowflake
- Familiarity with production quality ML and/or AI model development and deployment
- Effective communication skills and the ability to understand and adapt to changing requirements