Spark, Scala
Full Time, Part Time, Contract Corp-To-Corp, Contract Independent, Contract W2, C2H Corp-To-Corp, C2H Independent, C2H W2, 12
Position-Senior Data Platform Engineer(Big Data with Scala)
Contract
Job Description
Data Platform team is responsible for architecting and engineering the core data platform that's used for ingestion, cleansing, standardization and storing of all data received from customers. Customer Data Platform is built from ground up to be massively scalable with cloud native technologies, methodologies, and processes.
The Senior Data Platform Engineer will have responsibility is a key engineering leader tasked with building and extending the platform.
Essential Duties & Responsibilities:
Architects and Engineers robust and automated transformation pipelines to ingest and process structured and unstructured data from source systems into analytical platforms using batch and streaming mechanisms leveraging cloud native toolset
Engineers custom applications using Spark/Scala, Function Apps, Logic Apps, Azure Data Factory, and other Azure services
Collaborates with UI/API engineers to create a scalable and intuitive solutions for operations users
Creates code that's performant, scalable, and cost-effective
Engineers solutions using data structures such as parquet, delta for SQL, NoSQL based data stores
Reduces technical debt over time by identifying fixes that will help the product remain maintainable and flexible for future enhancements
Closely collaborates with other scrum team members in both platform and API/UI to estimate and deliver work inside of a sprint
Qualifications
Demonstrated ability to create modern data pipelines using Apache Spark, with expert level knowledge in Scala
Demonstrated ability to model and design modern data structures, SQL/NoSQL databases, and data lakes.
Demonstrated ability of engineering large scale missing critical application platforms
Prefer experience engineering solutions in AWS, Azure, or Google Cloud Platform
Well versed with modern build and deployment methodologies for Continuous Integration and Continuous Delivery of applications with minimal downtime
Education
Bachelor's Degree from an accredited institution with a Computer Science or Computer Information Systems focus or related field
Best Regards,
Himanshu