Have Copilot read job descriptions and extract out key info you want to know. Click "Analyze All" to try it out. Click on the Copilot's gear icon to customize the prompt.

X

Data Engineer - ETL Pipelines

x3jdnwa3e6c24o2qmm3kc3l8wx1kdp02b530v9o5do5cb1aihnxa2nu222qohvnj · 30+ days ago

Princeton, NJ, US

Negotiable

Full-time

Continue

By pressing the button above, you agree to our Terms and Privacy Policy, and agree to receive email job alerts. You can unsubscribe anytime.

KAYGEN is an emerging leader in providing top talent for technology based staffing services. We specialize in providing high-volume contingent staffing, direct hire staffing and project based solutions to companies worldwide ranging from startups to Fortune 500 and Managed Service Providers (MSP) across a wide variety of industries

Key Responsibilities:

Develop and maintain ETL pipelines using PySpark to process and transform data from various sources into our data warehouse.
Collaborate with data scientists and analysts to understand data requirements and ensure data quality and accuracy.
Optimize and performance-tune Spark jobs to handle large volumes of data efficiently.
Create and maintain documentation for ETL processes, data schemas, and data dictionaries.
Collaborate with the data infrastructure team to design and implement data storage solutions that meet performance and scalability requirements.
Participate in on-call rotation for monitoring and maintaining data pipelines.
Stay up-to-date with the latest trends and technologies in data engineering and Spark ecosystem.

Requirements:

Advanced proficiency in Python and SQL.
Strong experience with ETL pipeline development using PySpark.
Familiarity with one of the orchestration tools such as Apache Airflow, DBT, or Delta live tables.
Experience with distributed data processing and storage technologies (e.g., Hadoop, Spark, HDFS, AWS S3).
Solid understanding of data modeling and database design principles.
Excellent problem-solving skills and attention to detail.
Strong communication and collaboration skills.
Bachelor's degree in Computer Science, Engineering, or a related field (or equivalent work experience).

Nice to Have:

Experience with cloud platforms such as AWS, Azure, or Google Cloud.
Knowledge of data warehousing concepts and technologies (e.g., Redshift, BigQuery).
Experience with version control systems (e.g., Git).
Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes).
Previous experience in a data engineering or similar role.

At KAYGEN, we are always looking for dynamic, talented and experienced individuals. We invite you to join our team of talented IT professionals, consulting at client locations across the globe. Our culture is team-orientated; we strive to stand by our core values of respect, honesty and integrity. Our team of experienced staffing experts will work with you to find you the best opportunity. For more information please visit us at www.kaygen.com.

Benefits:

Healthcare Insurance
Vision and Dental Insurance
401(k) Retirement Plan
Free Life Insurance
Vacation Time Off
Sick Time Off
Family Medical Leave (FMLA)

Achieve your Kaizen by clicking here. A unique and exclusive talent community supported by Kaygen, that includes programs like: Certifications

Mentorship Program
Referrals
Family and Wellness benefits
Continuous Growth and Career Development

•

Last updated on Oct 4, 2023

About the company

X

x3jdnwa3e6c24o2qmm3kc3l8wx1kdp02b530v9o5do5cb1aihnxa2nu222qohvnj

More jobs at x3jdnwa3e6c24o2qmm3kc3l8wx1kdp02b530v9o5do5cb1aihnxa2nu222qohvnj

Analyzing

Aglie Coach$163k+

Franklin Lakes, New Jersey

·

30+ days ago

BSA - Retirement Planning$98k+

Los Angeles, California

·

30+ days ago

BSA - Statitics Quantitative$87k+

Los Angeles, California

·

30+ days ago

Oracle Project Accounting Functional Consultant

West Palm Beach, Florida

·

30+ days ago

Oracle Integration Cloud (OIC) Architect

Warren, New Jersey

·

30+ days ago

More jobs like this

Analyzing

Solutions Engineer - Commercial/GOV

wallaroo

Remote

·

30+ days ago

SAP Business Analyst with Order Management

C

crjdnwsnowo2i4nz45b1teboszrxlg0351vr73gpqw7yanury9u287prckhdnkww

Eagan, Minnesota

·

30+ days ago

Network Control Specialist 1

9

9xjdnwf8nt489qdiu4ab0qq7clsnet01f27n6pjaxju02yq1u697ou3dvfougsq9

Melbourne, Florida

·

30+ days ago

GIS Systems Analyst

B

b6jdnwcpcemgg8el3r9winlpunj8hc038b1vkhowrzxn9gitznreodi38t7rirkp

Tampa, Florida

·

30+ days ago

Freelance Marketer Using AI Expert

F

fud

Remote

·

30+ days ago

Senior Solutions Engineer (Remote, New York)$166-240k

G

grafanalabs

Remote, Oregon

·

30+ days ago

Machine Learning Engineer -Large Language Model & Reinforcement Learning

H

hireio-inc

San Jose, California

·

30+ days ago

Solution Architect - Cleveland, OH w/ REMOTE

T

two95-international-inc-3

Remote

·

30+ days ago

Systems Engineers L1-6

C

coreone

Tysons, Virginia

·

30+ days ago

For job seekers

Job searchSearch millions of jobs

LaunchpadNew

Resume to business

Cover Letter StudioGenerate a cover letter

Add to ChatGPTFind and discuss jobs

For employers and recruiters

Resume ScreenerOrganize and rank candidates

Talent PipelinePre-order

Source top talent

Promote a jobReach more candidates

For everyone

Referral programEarn 30% commission

Get mobile appBrowse anywhere

Jobs API

Become a partner

Developed by Blake and Linh in the US and Vietnam.

We're interested in hearing what you like and don't like! Live chat with our founder or join our Discord

Changelog

🚀 LaunchpadNov 27

Create a site and sell services based on your resume.

🔥 Job search dashboardNov 13

Revamped job search UI with a sortable grid, live filtering, bookmarks, and application tracking.

🫡 Cover letter instructionsSep 27

New Studio settings give you control over AI output.

✨ Cover Letter StudioAug 9

Automatically generate cover letters for any job.

🎯 Suggested filtersAug 6

Copilot suggests additional filters above the results.

⚡️ Quick applicationsAug 2

Apply to jobs using info from your resume. Initial coverage of ~200k jobs in Spain, Germany, Austria, Switzerland, France, and the Netherlands.

🧠 Job AnalysisJul 12

Have Copilot read job descriptions and extract out key info you want to know. Click "Analyze All" to try it out. Click on the Copilot's gear icon to customize the prompt.