Staff Software Engineer (Solution Architect)

We are looking for a Staff Software engineer with experience being a hands-on solution Architect

Responsibilities: 

Architecture Design & Software Development: 

• Develop and maintain strong relationships with key partner organizations (production engineering, R&D, TechOps, Data partners, ...).

• Transition solutions from architectural designs to practical, efficient deployment and maintenance, ensuring scalability and robust performance in production environments.

• Build a “go-to-architecture" framework and scalable platform technology components that are flexible for deploying existing and future Data & ML models and products with an agile mindset, including scalable architecture for large language models (LLM’s).

• Provide expertise in DevOps practices, CI/CD pipelines, and automation to improve the efficiency of development and deployment processes.

• Ensuring all deliverables are of high quality by setting development standards, adhering to the standards, and participating in code reviews

• You are a highly skilled software engineer with strong experience in system design, building, and developing scalable and sustainable solutions.

• You are proficient in Java, PyPark or Python, with experience in Scala is a plus.

• You excel in managing large data sets and have built data pipelines on technologies such as Spark, Presto, Flink, or Kafka.

• Design event-driven architectures that respond to data streams instantaneously.

• Ensure system scalability to handle growing data demands and maintain low latency.

• E2E understanding about building data products (data emission all the way to data reporting/visualization)

• Experience with creating pipeline for unstructured data will be a big plus

AI Integration: 

• Work with data engineers and data scientists to develop robust data pipelines for high-quality data acquisition, storage, and processing.

• Evaluate AI technologies, frameworks, and platforms to identify optimal solutions for specific use cases and business requirements.

• Collect feedback from multiple stakeholders to align technical implementation with requirements.

• Collaborate closely with security leaders to proactively mitigate risks and ensure ethical AI implementation.

• Implement safeguards to address training data poisoning, AI model theft, and other security challenges.

• Identify potential risks and challenges associated with AI implementations and develop mitigation strategies to ensure reliability and robustness.

Governance :

• Ensure compliance with risk and data governance standards by implementing strong security measures and staying updated on industry best practices.

SKILLS :

• Experience with high volume real-time stream processing

• Experience with unstructured data management (video, audio, text, images, etc.)

• Experience with various NoSQL database types preferred and the ability to establish criteria for selecting the most suitable database for specific use cases

• Experience in AI/ML Frameworks and Libraries such as Azure ML Studio, TensorFlow, PyTorch, Torch, Apache MXNet or similar

• Experiences related to AWS services such as SageMaker, Bedrock, EMR, S3, OpenSearch Service, Step Functions, Lambda, Glue, MSK,  and EC2

• Strong communication skills, with attention to detail and ability to convey rigorous mathematical concepts and considerations to non-experts

• Must be able to adjust to constant business change; common types of changes include new requirements, evolving priorities, and emerging practices.

• Excellent communication and presentation skills, with the ability to articulate complex technical concepts to a non-technical audience.

• Strong problem-solving skills and the ability to work in a dynamic, fast-paced environment.

• Expert in various data serialization protocols (i.e. Avro, Protobuf) with the ability to guide and implete best practices for their use in data architecture. 

#LI-TP1

Last updated on Aug 16, 2024