Job Description: PURPOSE:
This is a Big Data Administrator Lead position and not a developer position.
The Lead Data Engineer is responsible for orchestrating, deploying, maintaining and scaling cloud OR on-premise infrastructure targeting big data and platform data management (e.g., data warehouses, data lakes) including data access APIs. Prepares and manipulates data using Hadoop or equivalent.) with emphasis on high availability, reliability, automation and performance.
Advanced (expert preferred) level experience in administrating and engineering relational databases (ex. MySQL, PostgreSQL), Big Data systems (ex. Cloudera Data Platform Private Cloud and Public Cloud), Apache Solr as SME, ETL (ex. Ab Initio), BI (ex. MicroStrategy), automation tools (ex. Ansible, Terraform, Bit Bucket) and experience working cloud solutions (specifically data products on AWS) are necessary.
At least 10 years of Experienced with all the tasks involved in administration of big data and Meta Data Hub such as Cloudera.
Experience with Ab Initio, EMR, S3, Dynamo DB, Mongo DB, ProgreSQL, RDS, DB2 is a Plus.
DevOps (CI/CD Pipeline) is a Plus.
Experience with Advance knowledge of UNIX and SQL
Experience with manage metadata hub-MDH, Operational Console and troubleshoot environmental issues which affect these components
Prefer prior experience with migration from on-premise to AWS Cloud.
Represents team in all architectural and design discussions. Knowledgeable in the end-to-end process and able to act as an SME providing credible feedback and input in all impacted areas. Require tracking and monitoring projects and tasks as the lead.
ESSENTIAL FUNCTIONS:
20% Represents team in all architectural and design discussions. Knowledgeable in the end-to-end process and able to act as an SME providing credible feedback and input in all impacted areas.
Require project tracking and task monitoring. the lead position ensures an overall successful implementation especially where team members all are working on multiple efforts at the same time. Lead the team to design, configure, implement, monitor, and manage all aspects of Data Integration Framework. Defines and develop the Data Integration best practices for the data management environment of optimal performance and reliability. Plan, develop and lead administrators with project and efforts, achieve milestones and objectives. Oversees the delivery of engineering data initiatives and projects including hands on with install, configure, automation script, and deploy.
20% Develops and maintains infrastructure systems (e.g., data warehouses, data lakes) including data access APIs. Prepares and manipulates data using Hadoop or equivalent MapReduce platform.
15% Develop and implement techniques to prevent system problems, troubleshoots incidents to recover services, and support the root cause analysis. Develops and follows standard operating procedures (SOPs) for common tasks to ensure quality of service.
15% Manages customer and stakeholder needs, generates and develops requirements, and performs functional analysis. Fulfills business objectives by collaborating with network staff to ensure reliable software and systems. Enforces the implementation of best practices for data auditing, scalability, reliability, high availability and application performance. Develop and apply data extraction, transformation and loading techniques in order to connect large data sets from a variety of sources.
15% Acts as a mentor for junior and senior team members.
10% Installs, tunes, upgrades, troubleshoots, and maintains all computer systems relevant to the supported applications including all necessary tasks to perform operating system administration, user account management, disaster recovery strategy and networking configuration.
5% Expands engineering job knowledge and leading technologies by reviewing professional publications; establishing personal networks; benchmarking state-of-the-art practices; educational opportunities and participating in professional societies.
To perform this job successfully, an individual must be able to perform each essential duty satisfactorily. The requirements listed below are representative of the knowledge, skill, and/or ability required. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions.
Education Level: Bachelor's Degree
Education Details: Information Technology, Computer Science or related field
Experience: At least 8 years of relevant engineering experience.
In Lieu of Education
In lieu of a Master's degree, an additional 4 years of relevant work experience is required in addition to the required work experience.
Knowledge, Skills and Abilities (KSAs) Proficiency
Knowledge of programming languages and web based technologies. Expert
Experience with Cloudera CDP on-prem and public cloud; Solr SME. Expert
Ability to collaborate to solve technical problems across teams. Expert
Excellent communication skills both written and verbal. Expert
The incumbent is required to immediately disclose any debarment, exclusion, or other event that makes them ineligible to perform work directly or indirectly on Federal health care programs. Must be able to effectively work in a fast-paced environment with frequently changing priorities, deadlines, and workloads that can be variable for long periods of time. Must be able to meet established deadlines and handle multiple customer service demands from internal and external customers, within set expectations for service excellence. Must be able to effectively communicate and provide positive customer service to every internal and external customer, including customers who may be demanding or otherwise challenging. •
Last updated on Jun 9, 2023