Data Engineer

Data Engineer Qualifications

• Expert level experience in building Data Pipelines using Python.
• Expert level experience in resolving Security Vulnerabilities in code.
• Expert level experience in performance tuning complex SQL queries.
• Expert level experience in performance tuning complex Data Pipelines.
• ETL pipelines Development experience preferably in SAP Data Services and/or Informatica IICS for data extraction, data curation, writing data into various targets.
• Job scheduling/orchestration tool experience preferably Redwood Scheduler.
• Proficient in Source and version control using Gitlab.
• Knowledge on SNYK Scan/Pipelines will be a plus.
• Knowledge & understanding of any cloud platform preferably AWS.

Data Engineer Skills

• Python, SQL, Oracle DB or similar, SAP Data Services or similar, Informatica IICS or similar, Redwood Scheduler or similar, Gitlab version control, AWS fundamentals

Project Specifications and Description of Services:

• Resolve code Security Vulnerabilities in existing Python scripts.
• Performance tuning of complex SQL queries.
• Performance tuning of complex data pipelines.
• Create data ingestion pipelines in data warehouse and other large scale data platforms.
• Create Data Ingestion pipeline for variety of sources File (Flat, delimited, Excel), DB, API (With Apigee integration) and SharePoint
• Creating scheduled as well as trigger-based ingestion patterns using Redwood scheduler or similar orchestration tools
• Working with Product Owners to understand PI goals, PI planning, requirement clarification, and delivery co-ordination.
• Working with Team (Technical and delivery) to deploy & validate team deliverables, PI planning, JIRA creation and management, release planning and execution, CR creation and other release readiness activities.
• Working in Agile Framework, participating in various agile ceremonies, co-ordination with scrum master, tech lead and PO on sprint planning, backlog creation, refinement, demo, and retrospection
• Ensuring proper availability of the services.
• Suggesting new technology as applicable.
• Technical support of incident tickets escalated to the team.

#LI-DNP
•

Last updated on Nov 15, 2023