Senior Big Data Engineer

Altran Romania is a branch of Altran Group. It focuses mainly in the field of industrial and embedded software development. Furthermore, the company delivers IT & management services for ICT infrastructures.

PURPOSE OF THE JOB

As Senior Big Data engineer you will work on one of the world largest marketplace, dealing with one billion events per day and contribute as part of R&D self-organized team working in a challenging, innovative environment for our Customer.

MAIN TASKS AND RESPONSIBILITIES

Work proactively on the system architecture, choosing optimal solutions to build scalable and reliable systems to process key financial data
Data processing system optimization and tuning
Establish monitoring system
Ensure high quality development standards (unit/integration tests, etc.)
Collaboration with the product management team to incorporate the needs of our customers
Develop new generation big data pipelines, by migrating from batch processing into near-real time architecture

EDUCATION, SKILLS AND EXPERIENCE

Must have:

1+ years of commercial Scala development (at least)
5+ years of commercial JVM-based languages development experience
Several years of experience in big data technologies and tools
ETL or Pipeline design/implementation with Large Distributed Databases (Hive/Spark/Scala preferred)
Hands-on experience with: Spark, Kafka, HBase, NoSQL-Databases
Pipeline stability and performance tuning
Working knowledge of relational databases and query authoring (SQL)
Familiarity with the AWS-Stack (AWS EMR)
Rigor in high code quality, automated testing, and other engineering best practices
Experience in data validation, sampling & regression tests
Good English (oral & written) and communication skills in general

Would be a plus:

Commercial experience with Real-time processing systems (Spark Streaming, Storm)
Design and development of robust distributed systems
Having experience working with AirFlow, Hive, Presto
Payment data processing or system design knowledge