Senior Data Engineer

· Identifying and extracting data from various sources (such as databases, APIs, logs, files, and external systems).

· Ensuring data quality during extraction and handling any anomalies.

· Transforming raw data into a structured format suitable for analysis and reporting.

· Cleaning, aggregating, and enriching data using ETL tools and techniques.

· Implementing business rules and data validation logic.

· Loading processed data into data warehouses, data lakes, or other storage systems.

· Designing and maintaining ETL pipelines to move data efficiently.

· Collaborating with database administrators to optimize data loading processes.

· Designing, developing, and maintaining ETL workflows and pipelines.

· Writing efficient ETL code (Informatica PowerCenter & Informatica BDM) to automate data movement.

· Ensuring scalability, reliability, and performance of ETL processes.

· Profiling and tuning ETL processes for optimal performance. · Identifying bottlenecks and addressing them to improve data processing speed.

· Monitoring ETL jobs, identifying failures, and resolving issues promptly.

· Implementing error handling and logging mechanisms.

· Communicating effectively with cross-functional teams to understand data needs and ensure data consistency and accuracy.

Requirements

· Strong ETL Tool Proficiency

.Experience with ETL tools such as Informatica PowerCenter, Informatica BDM, and Apache NiFi.

· Database Knowledge

.Proficiency in SQL and understanding of relational databases (e.g., PostgreSQL, MySQL, Oracle).

· Data Warehousing: Familiarity with data warehousing concepts and technologies.

· Big Data Technologies: Exposure to big data platforms (e.g., Hadoop, Spark) and NoSQL databases (e.g., Hbase, MongoDB). · Data Governance and Security: Understanding of data governance practices and security protocols.

· Problem-Solving Abilities: Ability to troubleshoot complex data issues and optimize ETL processes.

· Excellent communication skills for technical and non-technical stakeholders.

· Experience in performance tuning for data queries.

· Understanding of data modelling and design standards, tools, and best practices.

· Analytical mind-set and problem-solving skills for proactive project support.

Last updated on May 22, 2024