Browse
Employers / Recruiters

Senior Azure Data Engineer

enable-data · 30+ days ago
Negotiable
Full-time
Remote
Apply

Enable Data Incorporated is seeking a highly skilled Senior Spark Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, we leverage modern solutions to drive increased value across our clients' business ecosystem. As a Senior Spark Engineer, you will be responsible for overseeing all aspects of our clients' Spark data processing pipelines, including development, implementation, and testing. In addition, you will work closely with our clients to identify opportunities to optimize their data processing pipelines and leverage Spark to achieve their goals. This is an excellent opportunity for a talented Spark Engineer who is looking to advance their skills and work with cutting-edge technologies in the Information Technology and Services industry.

Responsibilities

  • Design, implement, and maintain cloud-based data pipelines to enable efficient storage, processing, and analysis of large volumes of data.
  • Collaborate with data engineers, and other stakeholders to understand requirements and translate them into scalable and reliable data platform architectures.
  • Evaluate and select appropriate cloud infrastructure and services to support data storage, processing, and analytics needs, considering factors such as scalability, performance, cost, and security.
  • Implement data ingestion pipelines, integrating various data sources and ensuring data quality, integrity, and timeliness.
  • Design and optimize data storage solutions, including data lakes, data warehouses, and data marts, to support different types of data processing and analysis.
  • Implement data processing workflows using technologies such as Apache Spark, Apache Hadoop, or cloud-native data processing services.
  • Develop and maintain data governance and security policies, ensuring compliance with data protection regulations and industry best practices.
  • Monitor and optimize the performance of data platforms, identifying bottlenecks and implementing optimizations to improve data processing speed and efficiency.
  • Collaborate with DevOps teams to automate deployment, monitoring, and management of data platforms using infrastructure-as-code and CI/CD practices.
  • Stay up-to-date with emerging technologies and industry trends in cloud computing, big data processing, and data engineering.
  • Provide guidance and mentorship to junior team members, fostering a culture of learning and innovation.

Requirements

  • Bachelor's or Master's degree in computer science, engineering, or a related field.
  • 8+ years of proven experience as a Data Engineer, Software Engineer, or similar role, with a focus on building cloud-based data platforms.
  • Strong experience in Microsoft Azure
  • Proficiency in the latest Big Data tools/technologies like Hive, Hadoop, Yarn, Kafka, Spark Stream.
  • Proficiency in data processing frameworks such as Apache Spark, Apache Hadoop, or cloud-native data processing services (Azure Data Lake, Azure Data factory, Azure Databricks, Azure Synapse, Snowflake, CosmosDB)
  • Experience with data integration and ETL (Extract, Transform, Load) processes, including tools like Apache Airflow or cloud-native orchestration services.
  • Experience with workload and job optimization for cost reduction
  • Knowledge of database systems (relational and NoSQL) and data modeling principles.
  • Familiarity with data governance, data security, and compliance frameworks.
  • Understanding of distributed computing principles and scalable architectures.
  • Experience with infrastructure-as-code tools like Terraform
  • Excellent problem-solving and troubleshooting skills, with the ability to address complex data platform challenges.
  • Strong communication and collaboration skills to work effectively with cross-functional teams

Last updated on Mar 15, 2024

See more

About the company

More jobs at enable-data

Analyzing

Remote

 · 

30+ days ago

Remote

 · 

30+ days ago

Remote

 · 

30+ days ago

Remote

 · 

8 days ago

More jobs like this

Analyzing
UI/Sr.UI Developer
AV
AUC Ventures ·  Venture capital firm

Bengaluru, Karnataka

 · 

30+ days ago

iOS Lead Developer
Cityflo ·  Urban transportation and logistics

Mumbai, Maharashtra

 · 

30+ days ago

DBA
SB
SaleBuild ·  B2B lead generation and marketing

Pune, Maharashtra

 · 

30+ days ago

PHP Developer / Sr. PHP Developer
P
Propstack ·  Real estate data and analytics

Mumbai, Maharashtra

 · 

30+ days ago

New Delhi, Delhi

 · 

30+ days ago

Javascript Developer (Frontend - React)
V
Vinsol ·  Web and mobile app development

Remote

 · 

30+ days ago

Senior PhoneGap Developer (Ionic)
Logic Square ·  IT solutions and services company

Kolkata, West Bengal

 · 

30+ days ago

System Engineer- L1
columbusglobalindia

Rai Durg, Telangana

 · 

30+ days ago

Experienced developer for SaaS application
Essel Environmental Engineering and Consulting ·  Environmental consulting services

Remote

 · 

30+ days ago

Marunji, Maharashtra

 · 

30+ days ago

Developed by Blake and Linh in the US and Vietnam.
We're interested in hearing what you like and don't like! Live chat with our founder or join our Discord
Changelog
🚀 LaunchpadNov 27
Create a site and sell services based on your CV.
🔥 Job search dashboardNov 13
Revamped job search UI with a sortable grid, live filtering, bookmarks, and application tracking.
🫡 Cover letter instructionsSep 27
New Studio settings give you control over AI output.
✨ Cover Letter StudioAug 9
Automatically generate cover letters for any job.
🎯 Suggested filtersAug 6
Copilot suggests additional filters above the results.
⚡️ Quick applicationsAug 2
Apply to jobs using info from your CV. Initial coverage of ~200k jobs in Spain, Germany, Austria, Switzerland, France, and the Netherlands.
🧠 Job AnalysisJul 12
Have Copilot read job descriptions and extract out key info you want to know. Click "Analyze All" to try it out. Click on the Copilot's gear icon to customize the prompt.
© 2024 RemoteAmbitionAffiliate · Privacy · Terms · Sitemap · Status