Big Data Engineer AWS (EMR, S3, Glue, Spark, Hadoop)
Toronto
Role and ResponsibilitiesUnderstand requirement from product owners and translate into requirement and scope documentsDecide on the best fitment of technologiesservices that are available in scopeCreate Solution for data ingestion and data transformation using Hadoop services like Spark, Spark streaming, Hive, etc.Create technical design documents to communicate solutions to the team and mentor the team to develop the solutionBuild the solution with AWS and Hadoop Services as per design specificationsAssist team teams to build test cases and support with testing the solutionCoordinate with Upstream, Downstream and other supporting teams for production implementationProvide post-production support for Solutions implemented Please list 5 mandatory MUST HAVE skills and experience for this requirement. Please include skills related to technical as well as domain and non-technical skills and experience as applicable to the position.Candidate should have strong working experience with data services (EMR, S3, Glue) on AWS cloud.Strong hands-on experience on AWS EMR Glue (Spark with Scala) and AWS S3.Strong hands-on experience in Hadoop services like SparkIn-depth knowledge and extensive experience in building batch workloads on AWS using AWS EMR.Adept in analyzing and refining requirements, consumption query patterns and choosing the right technology fit like RDBMS, data lake and data warehouse.Should have knowledge of analytical data modelling on any of the RDBMSMPP platform Knowledge of PythonProven practical experience in migrating Hadoop based data lake from on-prem to AWS EMR on EMRFSPlease list 5 NICE TO HAVE but not mandatory skills and experience for this requirement Hands-on experience working with Terra bytes peta bytes scale data and millions of transactions per day.Skills to develop ETL pipeline using AirflowKnowledge of Spark streaming or any other streaming jobsAbility to deploy code using AWS Code Pipeline and Bit bucket is an added plusExpert in any of the following programming language Scala, Java and comfortable with working on Linux platform.Knowledge of Cloud based MPP Platform
...experience and certifications. In addition to your pay, Target cares about and invests in you as a team member, so... ...service fundamentals and experience building and managing a guest first team culture across the store Skills in guest engagement; problem solving and resolution...
...serve, and anticipation of patient/customer needs. Assesses child/family psychosocial and developmental strengths and risks,... ...from different units and communicating with the designated child life specialist for that area about their needs. What We Require: Education...
...which an Apex team member can provide. Apex Systems is part of the Commercial Segment of ASGN Incorporated. NYSE: ASGN 4400 Cox Road Suite 200 Glen Allen, Virginia 23060 Apex Systems is an equal opportunity employer. We do not discriminate or allow discrimination...
...TK-12 schools in the Fruitvale and Elmhurst neighborhoods of Oakland. To learn more about Education for Change, please visit . Cox Academy is located in East Oakland and serves grades TK-5. The school provides rigorous instruction in the core content areas while...
As a McLane Driver, you are the face of our company to our customers. You are the person they see week after week. You will develop strong... ...preferred. ~ Generous benefits available on your 60th day: medical, dental, and vision insurance, FSA/HSA and company-paid life...