Job Description

Microsoft Azure~Digital

6-9 yrs
Data Engineer - Databricks
- Must have at least 8+ years of extensive Data Engineering Experience in Databricks, ETL/ELT using Data pipelines, SQL/Procedure experience.
- Must have 8+ Years of hands-on development proficiency in implementing Databricks solutions using Scala and Spark with Data frames and Notebooks/SQL.
- Must have 8+ Years of hands-on development and performance tuning/enhancement experience in Scala/PySpark and SQL.
- Must have 8+ Years of hands-on development experience in working on Azure Cloud.
- Must have strong experience with Databricks, including developing and optimizing Spark jobs, data transformations, and data processing workflows.
- Must have experience and a good understanding on the below topics
-- Databricks Delta Lake storage
-- Unity Catalog
-- Autoloader
-- Lakehouse Architecture
-- Medallion Architecture
-- Databricks Serverless Options.
-- Dataset and Data frame
-- Databricks Clusters Configuration and Sizing
-- Databricks Job Orchestration
-- Spark Structured Streaming
- Must have experience with complex and large volume data streaming and data transformations using Azure Databricks.
- Must have extensive experience in Datawarehouse/Data Lakehouse implementations.
- Must have experience with monitoring, debugging, and resolving issues in live Databricks jobs.
- Must have strong hands-on expertise in troubleshooting DevOps pipelines, Azure, and AKS services

- The candidate should be able to provide Data Ingestion, Streaming and Transformations solutions and do development work.
- Experience in Apache Big Data implementations with programming proficiency like PySpark, R or Java prior to Databricks is advantageous.
- Good to have experience with containerization technologies such as Docker and Kubernetes.
- Strong critical thinking, communication, and problem-solving skills is essential.

Job Tags

Similar Jobs

HB Travels

Online Travel Itinerary Consultant (Remote) Job at HB Travels

...detail-oriented and customer-focused individuals to join our team as Online Travel Itinerary Consultants. In this role, you will help... ...a plus (but not required) What We Offer: Flexible remote work opportunity Training and mentorship to support your success...

University of Maryland Global Campus

Collegiate Faculty, Teaching and Learning (TL), Innovation and Entrepreneurial Leadership Job at University of Maryland Global Campus

..., Teaching and Learning (TL), Innovation and Entrepreneurial Leadership Department of Operations, Innovation, and Leadership School... ...effective teaching strategies and communication. Minimum Education & Experience Requirements: Education: Doctorate in...

Flowly Solutions

Package Handler Job at Flowly Solutions

...warehouse and ensure they are ready for delivery. You must be highly detail oriented to be successful in this position. We are a relocation business specializing in moving, assembling and repair. We offer a variety of other services as well. Responsibilities:...

RESILIV

Field Data Collection Driver Job at RESILIV

...reliable, safety-conscious, and tech-savvy Field Data Collection Driver to support a high-precision mapping initiative for our client... ...~ Previous experience in field data collection, delivery driving, or GIS-related roles.~ Familiarity with geospatial...

VDart Inc

FSO Technician Job at VDart Inc

...Job Title: FSO Technician Location: Township, OH 45069 Job Description: Relevant Experience : 3 Years minimum... ...), Tablet (Windows, Apple, Android), Printers (Network, Desk), Smart Phones, Desk Phones, Video Conference Rooms Devices (Audio...

Microsoft Azure Data Engineer Job at Purple Drive, Boston, MA

d0M3dm9veGlSMkdnWUphVWNwalBHZnRydkE9PQ==