Associate director- lead python data engineer

Job Summary

Supports the development and maintenance of scalable data stores that supply big data in forms needed for business analysis. Applies complete knowledge, skills, and practices to perform assignments.

Job Description

What is the opportunity?

We are seeking for a highly skilled Data Engineer to join our team and work on building Data Monetization products. Reporting to the Associate Director currently supporting the engineering stream, the ideal candidate will be an expert-level organizer capable of resolving ingestion challenges, including handling structured and unstructured data in various formats. You should be forward-thinking, with the ability to help scale up our ingestion products and be a fast learner. Accountability and reliability are crucial, as you will support though pro

What will you do?

• Join our team as a dedicated Python Data Engineer, focused entirely on hands-on development tasks to transform our data engineering technology. You will:

• Develop state-of-the-art, highly scalable data ingestion products using Python, SQL and Databricks.

• Create, document, test, and deploy robust data pipelines exclusively using Python.

• Be responsible for the full lifecycle of data engineering projects from development through to production deployments, including bi-weekly releases.

• Ensure the integrity, performance, and optimization of data ingestion pipelines, employing best practices in coding and process design.

What do you need to succeed?

Must-have:

• Python Proficiency: 8-10+ years of experience with a strong grasp of object-oriented programming and core Python concepts. Expertise in writing clean, maintainable, and efficient code.

• Experience understanding of the Spark Architecture is preferred.

• Deep familiarity with Azure Databricks, including building pipelines and executing jobs using Databricks Delta Live Tables (DLT).

• Extensive experience in building data ingestion products from end-to-end, data modeling, and API integrations for data ingestion.

• Proficient in using Azure, including Azure Gen2 Storage Accounts.

• Expertise in database technologies with a focus on Snowflake and data lakes.

• Experience working with containers and orchestration tools like (Docker, Kubernetes, Apache Airflow, CI/CD, etc.)

• Excellent communication skills with the ability to clearly summarize and explain complex information to a diverse audience. Strong problem-solving abilities and a proactive, detail-oriented approach to projects.

Nice-to-have:

• Additional knowledge of Azure services beyond the basics.

• Familiarity with Snowflake.

• Experience working with project management and documentation tools such as Jira and Confluence.

• Proficiency in MS Office.

Role Clarification:

• This role is purely focused on development tasks. There are no management duties involved, providing a dynamic environment for a developer looking to focus on technical growth and hands-on project execution in a collaborative setting.

Whats in it for you?

We thrive on the challenge to be our best, progressive thinking to keep growing, and working together to deliver trusted advice to help our clients thrive and communities prosper. We care about each other, reaching our potential, making a difference to our communities, and achieving success that is mutual.

• Positive work culture emphasizing diversity and inclusion, volunteerism and community involvement

• A comprehensive Total Rewards Program including great benefits, annual incentive program and recognition

• Leaders who support your development

• A dynamic, collaborative, and high-performing team

• Opportunities to do challenging work and grow your career

• Work-life integration

Job Skills

Big Data Management, Cloud Computing, Database Development, Data Mining, Data Warehousing (DW), ETL Processing, Group Problem Solving, Quality Management, Requirements Analysis

Additional Job Details

Address:

RBC CENTRE, 155 WELLINGTON ST W:TORONTO

City:

TORONTO

Country:

Canada

Work hours/week:

37.5

Employment Type:

Full time

Platform:

Capital Markets

Job Type:

Regular

Pay Type:

Salaried

Posted Date:

2024-05-30

Application Deadline:

2024-07-19

Inclusion and Equal Opportunity Employment

At RBC, we embrace diversity and inclusion for innovation and growth. We are committed to building inclusive teams and an equitable workplace for our employees to bring their true selves to work. We are taking actions to tackle issues of inequity and systemic bias to support our diverse talent, clients and communities.

We also strive to provide an accessible candidate experience for our prospective employees with different abilities. Please let us know if you need any accommodations during the recruitment process.

Join our Talent Community

Stay in-the-know about great career opportunities at RBC. Sign up and get customized info on our latest jobs, career tips and Recruitment events that matter to you.

Expand your limits and create a new future together at RBC. Find out how we use our passion and drive to enhance the well-being of our clients and communities at jobs.rbc.com.