Big Data Engineer AWS (EMR, S3, Glue, Spark, Hadoop)
Toronto
Role and ResponsibilitiesUnderstand requirement from product owners and translate into requirement and scope documentsDecide on the best fitment of technologiesservices that are available in scopeCreate Solution for data ingestion and data transformation using Hadoop services like Spark, Spark streaming, Hive, etc.Create technical design documents to communicate solutions to the team and mentor the team to develop the solutionBuild the solution with AWS and Hadoop Services as per design specificationsAssist team teams to build test cases and support with testing the solutionCoordinate with Upstream, Downstream and other supporting teams for production implementationProvide post-production support for Solutions implemented Please list 5 mandatory MUST HAVE skills and experience for this requirement. Please include skills related to technical as well as domain and non-technical skills and experience as applicable to the position.Candidate should have strong working experience with data services (EMR, S3, Glue) on AWS cloud.Strong hands-on experience on AWS EMR Glue (Spark with Scala) and AWS S3.Strong hands-on experience in Hadoop services like SparkIn-depth knowledge and extensive experience in building batch workloads on AWS using AWS EMR.Adept in analyzing and refining requirements, consumption query patterns and choosing the right technology fit like RDBMS, data lake and data warehouse.Should have knowledge of analytical data modelling on any of the RDBMSMPP platform Knowledge of PythonProven practical experience in migrating Hadoop based data lake from on-prem to AWS EMR on EMRFSPlease list 5 NICE TO HAVE but not mandatory skills and experience for this requirement Hands-on experience working with Terra bytes peta bytes scale data and millions of transactions per day.Skills to develop ETL pipeline using AirflowKnowledge of Spark streaming or any other streaming jobsAbility to deploy code using AWS Code Pipeline and Bit bucket is an added plusExpert in any of the following programming language Scala, Java and comfortable with working on Linux platform.Knowledge of Cloud based MPP Platform
The Converse Home is an Assisted Living Community located in downtown Burlington. Established in 1886 as a private, non-profit organization, we are well known for our warm, welcoming, and home-like atmosphere. Our mission has remained steadfast throughout the years to...
...organizations through integrated strategies across paid media, SEO, analytics, and web development. Weve built a high-performing... ...people, wed love to meet you. Position Summary As an SEO Manager at Marcel Digital, youll play a critical role in generating...
...Join a dedicated team committed to keeping our community safe and accessible throughout the winter season. As a Seasonal Snow Removal Equipment Operator and Driver, you will play a vital role in ensuring that roads, driveways, and parking lots remain clear and safe...
...compliance with applicable State Board of Pharmacy regulations. Functions include those... ...released from the pharmacy. The Pharmacy Technician works under close supervision. Performs... ...compliance with policies.+ Prepares, packages, compounds and labels non-sterile drug products...
...Dairy Farm Herdsman (6193) Location: Pennsylvania JobNumber: 6193 Dairy Farm Herdsman position immediately available on a well established 100+ cow family owned dairy in west central Pennsylvania. Must have experience with mature cattle, new born calves and heifers...