Experience with big data frameworks (i.e. Hadoop and Spark), 5+ years experience
Experience building automated data pipelines, 5+ years of experience
Experience performing data analysis and data exploration, 5+ years experience
Experience working in an agile delivery environment
Strong critical thinking, communication, and problem solving skills
Experience with cloud based platforms (i.e. Azure, GPC, AWS)
Experience working in multi-developer environment, using version control (i.e. Git)
Experience with orchestrating pipelines using tools (i.e. Airflow, Azure Data Factory)
Experience with real-time and streaming technology (i.e. Azure Event Hubs, Azure Functions Kafka, Spark Streaming)
Experience with API build Exposure/understanding DevOps best practice and CICD (i.e. Jenkins) Exposure/understanding of containerization (i.e. Kubernetes, Docker)
Responsibilities:
Perform data analytics on big data to solve and propose solutions to complex business problems
Develop and enhance end-to-end data pipelines using a big data processing technology stack (e.g. Hadoop, Cloud, Spark ecosystem technologies)
Troubleshoot complex data structures and perform root cause analysis to proactively resolve product and operational issues
Develop and document test cases for new features, execute testing, and document and share business UAT testing results as a part of agile sprints
Perform end-to-end data pipeline integration testing in adherence to change management standards and processes
Understand and adhere to coding and application standards and design (e.g. Python, SQL, etc.)
Utilize DevOps best practices to deploy new features into production systems, including continuous integration/continuous delivery (CICD), test-driven development, and proper version control (e.g. Jenkins, Jira, GIT)
Collaborate with business partners to define product objectives, business requirements, system design, and key performance indicators
Python, SQL, Hadoop, Spark, Azure Data Factory, Azure or GCP Cloud Airflow, Data pipelines, Jenkins / Jira / Git, CICD, Kubernetes / Docker
.svg "Go To Top"){kind=icon}
