Software Engineer II

The Batch Processing Platform team is looking for a Software Engineer to help us build the next generation of big data platform and infrastructure at Pinterest. You’ll be working on some of the most exciting big data open source technologies (Spark, Kubernetes, etc.), at the scale of hundreds of petabytes of data to help Pinners discover and do what they love. 

What you’ll do:

• Design and implement Fine Grained Access Control system for Pinterest’s big data platform (Spark, Flink, PyTorch, Ray and other ML/AI frameworks)
• Improve and customize the internals of open source Spark to meet our challenges at scale and actively participate in open source community
• Build and scale big data processing frameworks and infrastructure to process petabytes-scale datasets
• Work with internal customers on critical business use cases that rely on batch processing
• Provide thought leadership to the entire company on how data should be processed and stored more reliably, quickly and efficiently at scale

What we’re looking for:

• 3+years of industry experience with a proven track record of technical excellence
• 2+ years of experience of building and support large scalable big data infrastructure/platform
• Strong experience with Python or Java
• Knowledge of or interest in big data technologies (e.g. Spark, Hadoop, Parquet/ORC, Flink)
• Experience in leading cross-team engineering efforts

This position is not eligible for relocation assistance. Visit our PinFlex page to learn more about our working model.

In-Office Requirement Statement:

• We let the type of work you do guide the collaboration style. That means we're not always working in an office, but we continue to gather for key moments of collaboration and connection.
• This role will need to be in the office for in-person collaboration 1 x times per week and therefore needs to be in a commutable distance from the Dublin office.

#LI-HYBRID 

#LI-JD1

Last updated on Aug 22, 2024