Nebius is a Nasdaq-listed tech company that aims to become one of the world’s leading AI infrastructure providers. Headquartered in Amsterdam, we have R&D and commercial hubs across the US, Europe and Israel.
We build full-stack AI infrastructure to service the explosive growth of the global AI industry, including large-scale GPU clusters, cloud platforms and tools and services for developers. Our 500 employees include around 400 highly skilled engineers with a proven track record of developing world-class hardware and software solutions across cloud and AI/ML.
We are rapidly growing our infrastructure network with an ambitious investment program to build out data centers and colocations in the US and Europe. A Reference Platform NVIDIA Cloud Partner, Nebius’ AI-native cloud platform provides high-end infrastructure and tools for training, fine-tuning and inference.
Nebius is growing fast, and we’re always looking for the best talent to join our company. Along with highly competitive compensation and extensive opportunities for professional development, we offer a dynamic work environment where innovation, creativity and teamwork are highly prized and open up exciting new opportunities. As an equal opportunity employer, we are committed to fostering a diverse and inclusive workplace, where all applicants are given fair consideration and every team member is empowered to contribute to their fullest potential.
The role
We are looking for a Software Engineer to join our TractoAI team. TractoAI is built on top of the mature, open-source YTsaurus technology, which boasts a decade-long proven track record within BigTech companies. Our platform is designed to be highly scalable, resilient, and multitenant, capable of handling up to tens of thousands of servers, exabytes of data, millions of CPU cores, and tens of thousands of GPUs. TractoAI is a new experimental direction within Nebius AI, aiming to provide an end-to-end platform solution for all kinds of BigData and AI challenges, including data preparation, distributed training, offline model inference, and more.
We have already secured several clients both within and outside Nebius AI. Some of these clients use TractoAI as their primary data lake or as a foundation for large language model (LLM) pre-training.
Our Vision: We aim to transform YTsaurus into a leading BigData and ML product, competing with top cloud-like Data/AI platforms such as Databricks, Snowflake, VastData, MosaicML, and corresponding products from the hyperscaler clouds like AWS, Azure, or GCP. Our challenge lies in converting powerful technology into a successful product by defining the target audience, and expanding our existing client base.
Role Overview: In this role, you will work on improving the YTsaurus core for the needs of our TractoAI platform. With us, you will work on performance and reliability of a large-scale distributed system. Your job responsibilities will include designing new features and microservices of the system, investigating performance issues of user workloads and collaborating with our SRE team to make our system stable.
Our team is a small group of core YTsaurus developers with many years of experience in building distributed systems, standing behind the 100k-server-scale fleets of BigTech companies. We work on system architecture, software engineering and operations, making sure our code is not only committed and well-tested, but also works in production without disruptions. Our team is small, and we try to move fast to stay at the forefront of modern technologies.
Representative projects:
- Support S3 on a par with physical disks as storage for TractoAI
- Implement fair strategy of disk throughput distribution to share IO resources between concurrent computations of different users on the cluster.
- Make modern hardware like high-end GPUs or InfiniBand connectors work inside user job VMs and collect statistics about their utilization
- Improve TractoAI scheduler to make splitting the input data to the jobs in MapReduce operation more uniform by using various metadata
- Improve TractoAI IO engines to squeeze maximum out of the HDDs and NVMe SSDs.
What We Offer:
- A dynamic and collaborative work environment.
- Opportunity to work on cutting-edge technology.
- Flexible working conditions.
- Competitive salary and benefits.
We expect you to have:
- Have 5+ years of experience as a software engineer
- Have an experience with C++
- Have an experience with concurrency
- Are result-oriented and ready to push a feature from the idea to the client adoption
- Ready to dig into unknown areas, including HPC or GPU computing
- Ready to occasionally write code in Go/Python
It will be an added bonus if you have:
- Distributed systems design
- Understanding how databases work
- OS internals knowledge
- Performance engineering experience
- Experience of maintaining large-scale stateful system
What we offer
- Competitive salary and comprehensive benefits package.
- Opportunities for professional growth within Nebius.
- Hybrid working arrangements.
- A dynamic and collaborative work environment that values initiative and innovation.
We’re growing and expanding our products every day. If you’re up to the challenge and are excited about AI and ML as much as we are, join us!
Top Skills
What We Do
Cloud platform specifically designed to train AI models
.png)
