Interview: Phone and skype
EVT
Description:
Based in Philly with 25% travel to San Jose and Denver where the consultant picks up travel expenses so incorporate into your fee. A huge must is their communication skills must be impeccable as they will be dealing with C-level
As the Senior Big Data Engineer, you will work on the collecting, storing, processing, and analyzing client's Exabyte scale data sets; captured from tens of millions of devices. Your primary focus will be on choosing or building optimal solutions to use for these purposes; then maintaining, implementing, and monitoring them. This person will also be responsible for integrating them with the architecture used across the company. The Sr. Big Data Engineer will design and build large-scale data processing systems, be an expert in data warehousing solutions and should be able to work with the latest (NoSQL) database technologies. You will collect and document requirements to define a competitive system based on the needs of the parent organization. You will work with a team of architects and other engineers to create an incremental product design that implements the defined system. You will work with other engineers to implement that incremental design and deploy upgrades continually to production.
This is a unique opportunity for the right candidate to champion exciting new technology in a Fortune 50 organization and establish a reference implementation that could be adopted across extensive technology portfolio. You will be promoting benefits of big data processing and analysis across the organization and will need to confidently engage and present to application developers, infrastructure managers and senior business sponsors alike.
Responsibilities
· Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
· Implementing ETL process
· Monitoring performance and advising any necessary infrastructure changes
· Defining data retention policies
Skills and Qualifications
· Proficient understanding of distributed computing principles
· Develop automation and management capabilities of Hadoop cluster, with all included services
· Ability to troubleshoot and solve any ongoing issues with operating the cluster
· Proficiency with Hadoop v2, MapReduce, HDFS
· Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
· Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
· Experience with Spark
· Experience with integration of data from multiple data sources
· Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
· Knowledge of various ETL techniques and frameworks, such as Flume
· Experience with various messaging systems, such as Kafka or RabbitMQ
· Experience with Big Data Client toolkits, such as Mahout, SparkML, or H2O
· Good understanding of Lambda Architecture, along with its advantages and drawbacks
· Experience with Cloudera/MapR/Hortonworks
· Knowledge of CEPH filesystem
· Demonstrated ability to conceive, manage, and complete software deliverables
· Linux systems administration skills, across distributions, and especially in a cloud or virtualized environment
· Understanding of IP networking and traffic scaling
· Experience with agile development methodologies, rapid application development, and project management
· Proven ability to design and present understandable and practical solutions to complex problems
· Demonstrated leadership skills in a fast-paced, team-driven environment
· Strong verbal and written communication skills, including visual presentation skills
· Demonstrated experience in research data collection, analysis, and presentation
· Experience with intellectual property portfolio management, especially patents and trademarks
· Ability to work effectively across internal and external organizations
· Ability to travel when needed; expected travel is 5-25%
· Ability to promote technologies to large audiences or top level executives
•
Last updated on Jun 26, 2017