Browse
Employers / Recruiters

Data Engineer

avanceon · 30+ days ago
Negotiable
Full-time
Apply

Role Summary

As a Big Data engineer you will be responsible for data-related implementation tasks that include provisioning of data storage services, ingesting, streaming and batch data, transforming data, implementing security requirements, implementing data retention policies, identifying performance bottlenecks, and accessing external data. You will develop and implement data Lake solutions for company-wide applications and manage large sets of structured, semi structured and unstructured data. You will work in a modern cloud-based data lakehousing environment alongside a team of diverse, intense, and interesting co-workers.

Job Responsibilities

  • Performing analysis of large data stores and derive insights using Big Data querying
  • Design and develop code, scripts, and data pipelines that leverage structured, semi-structured and unstructured data
  • Should be able to manage Data ingestion pipelines and stream processing
  • Perform computations using Azure functions
  • Responsible for the documentation, design, and development of Hadoop applications
  • Perform day-to-day tasks to accuracy with and minimal directions while meeting deadlines

Qualification & Skillset

  • Bachelor/master’s degree in computer science
  • At least 2-5 years of relevant working experience with Cloud Computation
  • Hands-on experience on Azure/AWS
  • Experience working with Apache Spark, Apache Hive, Apache HBase
  • Hands-on experience on:
    • Azure/AWS Architectures
    • SQL, Scala, Node.js, JS, Java
    • Python with Pandas, NumPy, TensorFlow
    • ETL

Preferred Requirement

  • Implementing Lambda Architecture with stream processing and batch processing
  • Implement security standards and practices in the Architecture of Cloud Solution
  • Training/Certification on Data Lake HDFS
  • Training/Certification on Azure Data Lake Gen 2
  • Training/Certification on Data Bricks
  • Experience with Azure Data Explorer and Kusto query language would be a plus.
  • Experience with Azure Data Factory would be a plus.

Last updated on Nov 20, 2023

See more

About the company

More jobs at avanceon

Analyzing

Lahore, Punjab

 · 

30+ days ago

Lahore, Punjab

 · 

30+ days ago

Lahore, Punjab

 · 

30+ days ago

Lahore, Punjab

 · 

30+ days ago

Lahore, Punjab

 · 

30+ days ago

More jobs like this

Analyzing
Senior DevOps Engineer (Puppet)
Creative Chaos ·  مصنوعی انٹیلیجنس اور ڈیزائن

Remote

 · 

30+ days ago

Senior Analytics Engineer I (Pro-active Pipeline)
P
pakistan-mobile-communication-limited-pmcl

Islamabad, Islamabad Capital Territory

 · 

30+ days ago

Remote

 · 

30+ days ago

Senior IT Consultant
I
inbox-business-technologies

Karachi, Sindh

 · 

30+ days ago

Islamabad, Islamabad Capital Territory

 · 

30+ days ago

Lahore, Punjab

 · 

30+ days ago

Lahore, Punjab

 · 

30+ days ago

Islamabad, Islamabad Capital Territory

 · 

30+ days ago

Data Engineer
S
small-world

Karachi, Sindh

 · 

30+ days ago

Lahore, Punjab

 · 

30+ days ago

Developed by Blake and Linh in the US and Vietnam.
We're interested in hearing what you like and don't like! Live chat with our founder or join our Discord
Changelog
🚀 LaunchpadNov 27
Create a site and sell services based on your resume.
🔥 Job search dashboardNov 13
Revamped job search UI with a sortable grid, live filtering, bookmarks, and application tracking.
🫡 Cover letter instructionsSep 27
New Studio settings give you control over AI output.
✨ Cover Letter StudioAug 9
Automatically generate cover letters for any job.
🎯 Suggested filtersAug 6
Copilot suggests additional filters above the results.
⚡️ Quick applicationsAug 2
Apply to jobs using info from your resume. Initial coverage of ~200k jobs in Spain, Germany, Austria, Switzerland, France, and the Netherlands.
🧠 Job AnalysisJul 12
Have Copilot read job descriptions and extract out key info you want to know. Click "Analyze All" to try it out. Click on the Copilot's gear icon to customize the prompt.
© 2024 RemoteAmbitionAffiliate · Privacy · Terms · Sitemap · Status