Sr. Spark Developer

About the role:

This role is for a Big Data Tech Developer within the Core Accounts Data program. The candidate will work on existing applications and framework and will be responsible for technical delivery of new models and frameworks and delivering on business use cases as our data strategy is developed further. The role is within a technology group but involves engagement with business and understanding of operating environments.

Technical / Functional Proficiency:

o Experience in leading and developing staff in a Technical Development environment
o Hadoop / Big Data domain experience is required, as is exposure to ecosystem technologies such as Hive, Impala, Spark, SCALA etc.
o Data Warehouse/ETL design and development methodologies knowledge and experience required
o Broad cross-functional IT experience in application development and interface design is essential
o Must have sound operating knowhow of databases Oracle (preferable) and Linux. Expertise in developing shell and Unix scripting
o Experience with Hadoop based real time and batch-based architecture
o Experience with NoSQL databases like Hive/HBase/Impala
o Experience with real time messaging and ingestion including Kafka
o Expertise in Cloudera Hadoop Distributed File System, YARN, ZooKeeper, Oozie.
o Expertise in management and troubleshooting of Spark Jobs is preferred
o Experience in benchmarking, throughput performance analysis
o Experience with developing frameworks and utility services including logging/monitoring
o Experience with API development and use of JSON/XML/Hypermedia data formats
o Core Java or Python Programming experience is preferred
o 2+ years of Oracle and/or MYSQL database application development experience
o Agile development processes experiences a plus
o Financial Services domain expertise is a plus
o Preferred to have core Java experience with Spring and ORM frameworks such as Hibernate


Qualifications:
6-10 years experience using codes for statistical modeling of large data sets

Education:
Bachelors/University degree or equivalent experience, potentially Masters degree

Last updated on Feb 1, 2022