Job Title: Data Engineer, DataStage
Position Type: Contract
Contract Duration: 12 months (possibility of extension or conversion to FTE)
Workplace Type: Hybrid (3 days in-office, 2 days remote)
Workplace Schedule: 8am to 5pm (Tues, Wed, Thursday are in-office)
We are seeking an experienced Data Engineer, specifically with IBM Infosphere DataStage/Quality Stage. The Data Engineer will design, develop, automate, and support complex applications to extract, transform, and load data; Implement and automate complex processes; Plan and conduct ETL unit and development tests; Monitor results and take corrective actions; Translate data access, transformation, and movement requirements into functional requirements and mapping designs. Use SQL to design database schemas, optimize data retrieval, and implement ETL processes. Investigate, analyze, and resolve complex technical problems related to system functions, environment, and procedures; and Manage service levels, standards, and configurations; Participate in Agile teams to meet goals; Development and support of Kafka integrations, including topics, producers, consumers, and streaming applications; Design, develop and maintain Enterprise micro services security, logging, common APIs and enforcing coding standards. Required Skills: - Bachelor's degree in Computer Science, Computer Engineering, Electrical/Electronics Engineering, or related technical field and 7 years of software development experience.
- Must have extensive hands-on experience in designing, developing and maintaining software solutions on ETL process utilizing IBM Infosphere DataStage/Quality Stage
- Strong Skills in SQL, PL/SQL Programming, Query Optimization.
- Must have experience with analyzing, tuning and optimizing queries in the RDBMS.
- 5+ years experience working in a Data Warehouse environment (or equivalent).
- 6 years of experience with the design, development, automation, and support of applications to extract, transform, and load data;
- 6 years of experience with databases, including Oracle, DB2, Teradata, SQL Server, and Access;
- 6 years of experience with IBM InfoSphere DataStage;
- 6 years of experience with Data warehouse;
- 6 years of experience with Unix Shell Scripting;
- 6 years of experience working within an Agile Scrum environment;
Preferred Skills: - 2+ years of experience with Spring boot is preferred
- 2+ years of experience building applications using Java and experience with API integrations is preferred
- Knowledge of Kafka and streaming solutions is a plus
Position Responsibilities and Required Knowledge: - Design, build and maintain batch or real-time data pipelines.
- Automate data workflows such as data ingestion, aggregation, and ETL processing.
- Ensure data accuracy, integrity, privacy, security, and compliance through quality control procedures.
- Monitor data systems performance and implement optimization strategies.
- Analyzing, tuning and optimizing queries in the RDBMS.
- Must be able to produce thorough technical design documents, test plans and documented results.
- Will work in Agile methodologies and a fast-paced environment
- Developing API Integrations with cloud and other applications.
- Experience building applications using Java and Java based frameworks like Spring Boot is strongly preferred.
- Knowledge of Data Management Strategies (ETL Frameworks, Data Modeling concepts)
- Strong understanding of relational databases, oracle functions, procedures, triggers and packages.
- Willingness to learn new technologies.
- Excellent verbal and written communication skill to communicate complex information clearly and simply.