We are seeking a Data Engineer Lead with a strong blend of technical expertise and business understanding. This role will lead data engineering initiatives, ensuring robust data infrastructure and seamless data integration across the organization. The ideal candidate will act as a strategic partner, collaborating with business stakeholders to drive data-driven decision-making while ensuring the reliability, scalability, and efficiency of our data systems.
Responsibilities
- 1. Leadership and Strategy
- Lead and mentor a team of data engineers to deliver high-quality solutions.
- Collaborate with cross-functional teams, including business units, product managers, and data analysts, to align data engineering projects with organizational goals.
- Drive the implementation of best practices in data architecture, modeling, and pipeline development.
- 2. Technical Excellence
- Design, build, and maintain scalable data pipelines and systems to enable analytics, data scientist requirements, and operational processes.
- Enable automated data validation, monitoring, and anomaly detection processes to ensure high data accuracy and reliability.
- Data Lineage and Monitoring: Implement data lineage and monitoring tools to track data changes, improve transparency, and troubleshoot data issues efficiently.
- Optimize data storage and retrieval strategies for performance and cost efficiency using cloud platforms (Google Cloud).
- Stay updated on emerging trends in data engineering and incorporate them into the organization’s practices.
- 3. Business Collaboration
- Requirement Gathering: Work closely with business stakeholders to gather and translate complex business requirements into detailed technical specifications and actionable data engineering plans.
- Business Impact Mapping: Align data engineering efforts with organizational goals by identifying key business problems and delivering data-driven solutions that create measurable impact.
- Data Democratization: Partner with business units to design and implement self-service data tools, ensuring stakeholders have easy access to relevant and accurate data for decision-making.
- Cross-Functional Synergy: Act as a liaison between technical teams (engineering, analytics, and IT) and non-technical departments (finance, marketing, operations) to ensure smooth communication and understanding of data initiatives.
- Data Visualization Enablement: Collaborate with analysts and business teams to ensure the data pipelines and models support actionable dashboards, reports, and insights.
- KPI and Metric Design: Work with business leaders to define key performance indicators (KPIs) and metrics, ensuring data infrastructure supports real-time tracking and reporting.
- Stakeholder Training: Educate and train non-technical stakeholders on data capabilities, fostering a culture of data literacy and empowering teams to leverage data effectively.
- Business Value Optimization: Prioritize and balance data engineering projects based on business value, impact, and feasibility, ensuring resources are used efficiently.
- 4. Project Management
- Manage the end-to-end lifecycle of data engineering projects, from requirement gathering to deployment.
- Establish and monitor KPIs to measure the impact of data engineering solutions.
- Ensure timely delivery of projects while balancing scope, resources, and priorities.
Requirement
- Bachelor’s degree in Computer Science, Information Technology, Data Science, or a related field.
- Having 5-8 years of experience in data engineering, with a proven track record of leading data engineering projects on Google Cloud.
- At least 2 years of experience in team management is required.
- Strong technical expertise in Google Cloud services (BigQuery, Dataflow, Pub/Sub, etc.) and data processing frameworks.
- Experience in data modeling, ETL/ELT processes, and building real-time and batch data pipelines.
- Demonstrated ability to work with business stakeholders, translating technical language into business insights and ensuring data solutions support business needs.
- Knowledge of the banking or financial services industry is highly desirable.
- Proficiency in SQL, Python, or other relevant programming languages.
- Strong understanding of data governance, privacy standards (e.g., GDPR), and compliance frameworks.
- Excellent communication skills and ability to collaborate effectively across departments.
Bonus point
- Proficiency in DPT Tools and Dataflex
•
Last updated on Sep 24, 2024