Data Engineer (Python & Spark) (remote work)

About Luxoft
Luxoft, a DXC Technology Company (NYSE: DXC), is a digital strategy and software engineering firm providing bespoke technology solutions that drive business change for customers the world over. Acquired by U.S. company DXC Technology in 2019, Luxoft is a global operation in 44 cities and 23 countries with an international, agile workforce of nearly 18,000 people. It combines a unique blend of engineering excellence and deep industry expertise, helping over 425 global clients innovate in the areas of automotive, financial services, travel and hospitality, healthcare, life sciences, media and telecommunications. DXC Technology is a leading Fortune 500 IT services company which helps global companies run their mission critical systems. Together, DXC and Luxoft offer a differentiated customer-value proposition for digital transformation by combining Luxoft’s front-end digital capabilities with DXC’s expertise in IT modernization and integration. Follow our profile for regular updates and insights into technology and business needs.
​​​​​​​Luxoft Romania has been established since 2001. We currently have approximately 2500 employees working from different locations in the country.

Mandatory Skills:

- Experience working in Python
- Experience working in Data frameworks such as Spark, Hadoop, HDFS
- At least 3 years of experience in building data pipelines in an industrial/enterprise environment
- Knowledge of Apache Airflow
- Hands-on with different data formats (JSON, CSV, XML, AVRO, TXT, etc.)
- Some experience with AWS

Nice-to-Have Skills:

- Experience working in IoT or predictive maintenance domain
- Experience with OOP software engineering practices

Project Description:

Data collected from smart devices is accessed from cloud (AWS) storage and undergoes translation from device-specific schema, file formats, etc and transformations such as selection of relevant data and features before being applied to a ML model training subsystem. Qualified models are then pushed to production environment for prediction/execution. Data handling employs scalable spark-based access. The entire processing workflow is kept in sync via pipelines defined in Airflow. The state of the entire data engineering (& ML models, training and execution) is available via Dashboard UI

Responsibilities:

- Bring data engineering expertise to a local Scrum team, providing qualified deliverables and services on schedule.
- Participate in progress reviews
- Work with scrum master(s), tech lead(s) to analyze and understand user stories in each sprint
- Complete coding & unit testing for the allotted stories
- Create design documents or make changes to existing ones
- Complete code reviews