Browse
Employers / Recruiters

Data Engineer(Python, PySpark, Spark & SQL)

Negotiable
Full-time
Apply

We are looking for a skilled Data Engineer with a strong background in Python, PySpark, Spark, and SQL. The ideal candidate will have experience in building and optimizing data pipelines and architectures. This role requires a collaborative professional who can work across teams to design and implement robust data solutions.

Key Responsibilities:

  • Design, build, and maintain efficient, scalable, and reliable data pipelines using Python, PySpark, Spark, and SQL.
  • Collaborate with data scientists, analysts, and other stakeholders to understand data needs and deliver appropriate data solutions.
  • Develop and optimize ETL (Extract, Transform, Load) processes to ensure data quality and integrity.
  • Implement data processing workflows for batch and real-time data integration.
  • Monitor and troubleshoot data pipelines, ensuring data accuracy and availability.
  • Perform data exploration and analysis to support business decisions.
  • Document data processes, architectures, and workflows.

Requirements

Proven 4+Years of experience as a Data Engineer or in a similar role.

Strong proficiency in Python and PySpark for data manipulation and transformation.

Hands-on experience with Apache Spark for large-scale data processing.

Advanced SQL skills for data querying, transformation, and performance tuning.

Familiarity with cloud platforms (AWS, Azure, Google Cloud) and their data services.

Experience with data warehousing concepts and technologies (e.g., Redshift, BigQuery, Snowflake).

Knowledge of best practices for data architecture and data governance.

Last updated on May 29, 2024

See more

About the company

More jobs at unison-consulting-pte-ltd

Analyzing

Remote

 · 

30+ days ago

Remote

 · 

30+ days ago

 · 

30+ days ago

Kuala Lumpur, Federal Territory of Kuala Lumpur

 · 

30+ days ago

Developed by Blake and Linh in the US and Vietnam.
We're interested in hearing what you like and don't like! Live chat with our founder or join our Discord
Changelog
🚀 LaunchpadNov 27
Create a site and sell services based on your resume.
🔥 Job search dashboardNov 13
Revamped job search UI with a sortable grid, live filtering, bookmarks, and application tracking.
🫡 Cover letter instructionsSep 27
New Studio settings give you control over AI output.
✨ Cover Letter StudioAug 9
Automatically generate cover letters for any job.
🎯 Suggested filtersAug 6
Copilot suggests additional filters above the results.
⚡️ Quick applicationsAug 2
Apply to jobs using info from your resume. Initial coverage of ~200k jobs in Spain, Germany, Austria, Switzerland, France, and the Netherlands.
🧠 Job AnalysisJul 12
Have Copilot read job descriptions and extract out key info you want to know. Click "Analyze All" to try it out. Click on the Copilot's gear icon to customize the prompt.
© 2024 RemoteAmbitionAffiliate · Privacy · Terms · Sitemap · Status