We are seeking a highly motivated and experienced data engineering leader to own and manage all aspects of Wren’s data lake including its transformation processes and data integrations. Your responsibilities will encompass managing the day-to-day activities within the data engineering team and providing technical guidance and mentorship to other engineers.
Relocation package can be offered to assist with moving to the area, no sponsorship is offered
Key Responsibilities:
· Ensure the data lake is continuously monitored for performance, security, and availability. Using robust monitoring tools like Datadog to detect and resolve anomalies quickly. Maintain SLAs to ensure reliable data availability.
· Develop and enforce best practices for automated testing of ETL pipelines and data workflows. Implement data quality frameworks to ensure accuracy, consistency, and completeness of data at every stage. Leverage tools for unit testing, integration testing, and regression testing within the CI/CD pipeline to promote reliable deployments.
· Provide day-to-day management, coaching, and performance reviews for the data engineering team. Set clear objectives, define career development plans, and foster an environment of collaboration and continuous improvement. Identify skill gaps and provide necessary training and support.
· Act as the primary technical liaison for internal and external stakeholders. Work closely with business leaders, and analysts to understand data requirements and align engineering efforts with business objectives. Effectively collaborate with project managers to communicate project progress, risks, dependencies, and timelines to key stakeholders, ensuring alignment and proactive issue resolution throughout the project lifecycle.
· Oversee the architecture, governance, and evolution of the data lake. Ensure it remains scalable, secure, and cost-effective. Define policies and standards for data ingestion, access control, data retention, and data governance, ensuring compliance with regulatory and company guidelines.
· Foster a culture of innovation and learning by mentoring junior and mid-level engineers. Provide technical guidance on design patterns, data transformations, and cloud technologies. Encourage knowledge sharing within the team through code reviews, paired programming, and collaborative problem-solving.
· Ensure all aspects of the data platform, pipelines, and processes are well-documented, including architecture diagrams, data lineage, and operational guides. Maintain a comprehensive knowledge base to facilitate onboarding, troubleshooting, and system improvements.
· Collaborate with senior leadership to define the long-term strategy for data engineering, ensuring alignment with broader business goals. Identify emerging trends and technologies in data engineering and recommend innovations to keep the data platform future-proof and competitive.
· Decision-maker for architectural choices, tool selection, and implementation approaches. Perform thorough evaluations of technical options and provide clear recommendations backed by data and industry best practices. Balance technical excellence with pragmatic delivery to meet business needs efficiently.
· Act as a key partner for Business Intelligence teams, working closely to understand their data needs ensuring to maintain high-quality and well-structured data. Collaborate on the design of data models and abstractions. Provide guidance on data sourcing, transformation logic, and performance optimisation for BI tools.
Required Skills:
· Proactive and forward-thinking, with a focus on continuous improvement and anticipating future challenges.
· Minimum of 5 years’ experience designing, building, and maintaining data platforms in production environments.
· Skilled in using modern monitoring tools such as Datadog, CloudWatch, or similar for observability and alerting.
· 5+ years of experience using Python for ETL/ELT processes, including data wrangling and pipeline orchestration.
· Strong communication skills, with the ability to convey technical concepts to both technical and non-technical stakeholders.
· Adept at problem-solving and root cause analysis, with a structured approach to troubleshooting complex data issues.
· At least 2 years of experience working with modern data lakes and ETL platforms such as Snowflake, Databricks, or equivalent.
· Hands-on experience with cloud platforms (AWS, Azure, or GCP), including compute, storage, and serverless services.
· Strong attention to detail, ensuring data quality, consistency, and accurate documentation.
· Experience using BI tools such as Tableau, Power BI to build reports and support data-driven decision-making