Spark/Scala/Python Data Engineer
Acest job nu mai este activ!
Vezi toate job-urile Luxoft Romania active.
Vezi toate job-urile Spark/Scala/Python Data Engineer active pe Hipo.ro
Vezi toate job-urile in IT Software active pe Hipo.ro
Luxoft, a DXC Technology Company, (NYSE: DXC), is a digital strategy and software engineering firm providing bespoke technology solutions that drive business change for customers the world over. Luxoft uses technology to enable business transformation, enhance customer experiences, and boost operational efficiency through its strategy, consulting, and engineering services. Luxoft combines a unique blend of engineering excellence and deep industry expertise, specializing in automotive, financial services, travel and hospitality, healthcare, life sciences, media and telecommunications. Luxoft is well known for its consistent high level of delivery and complex project management, its premier digital engineering talent, exceptional client focus, and agility, creativity, and remarkable problem-solving capabilities.
Experience in implementations of end-to-end data processing chains, Big data architectures (Hadoop cluster, noSQL databases, Elastic search) mastering languages and frameworks for distributed data processing (Spark / Scala).
• Basic knowledge and interest in the development of ML algorithms
• Knowledge of the ingestion framework
• Knowledge of Spark and its different modules
• Mastery of Scala and / or Python
• Knowledge of the AWS or GCP ecosystem
• Knowledge of the ecosystem of NOSQL databases
• Knowledge in the construction of APIs of data products
• Knowledge of Dataviz tools and libraries
• Popularization of complex systems
• Expertise in data testing strategies
• Strong problem solving, intelligence, initiative and ability to withstand pressure
• Excellent interpersonal skills and a great sense of communication (ability to go into detail)
During project definition
• Design of data ingestion chains
• Design of data preparation chains
• Basic ML algorithm design
• Data product design
• Design of NOSQL data models
• Design of data visualizations
• Participation in the selection of services / solutions to be used according to the uses
• Participation in the development of a data toolbox
During the iterative realization phase
• Implementation of data ingestion chains
• Implementation of data preparation chains
• Implementation of basic ML algorithms
• Implementation of data visualizations
• Using ML framework
• Implementation of data products
• Exposure of data products
• Setting up NOSQL databases
• Implementation in distributed mode of treatments
• Use of functional languages
• Debugging distributed processes and algorithms
• Identification and cataloging of reusable elements
• Contribution on data processing problems
During integration and deployment
• Expertise in the implementation of end-to-end data processing chains
• Mastery of distributed development
Contribute to the business value of data-oriented products based on Datalake on-premise or cloud environments, by implementing end-to-end data processing chains, from ingestion to exhibition APIs and data visualization.
As responsibilities, you will maintain the high quality of data transformed in the Datalake, smooth operation of the data processing chains and optimization of the use of the resources of the on-premise or cloud clusters.