Ray is the AI Compute Engine

Ray manages, executes, and optimizes compute needs across AI workloads. It unifies infrastructure via a single, flexible framework—enabling any AI workload from data processing to model training to model serving and beyond.

10,000+ organizations build with Ray

  • instacart-logo
  • cohere-logo
  • uber-logo
  • spotify-logo
  • netflix-logo
  • openai-logo
  • doordash-logo
  • niantic-logo
  • samsara-logo

Key Features

seamless-scale

Seamless Scale

There's a reason Ray is the world’s leading AI compute engine. Simple primitives and one Python decorator make scaling from your laptop to the cloud a breeze.

python-native

Python Native with Ecosystem Integrations

Ray's Python-first API is extensible and open. Not to mention, it natively integrates with your entire ML Ops ecosystem, including:

  • ML frameworks like Pytorch and Tensorflow
  • Specialized libraries like vLLM and TRT-LLM
  • ML Ops tools like W&B and MLFlow
any-accelerator

Unmatched Precision

Coordinate heterogeneous resources with ease. Run your AI workloads on CPUs, GPUs, TPUs, and more with the ability to partition for fine grained optimization of utilization for every AI workload.

libs-devs

Libraries for Developers

We've built ML libraries so you don't have to. Accelerate development efforts with Ray Data, Ray Train, Ray Tune, Ray Serve, and RLLIb. Powered by Ray’s AI compute engine, the libraries offer easy and familiar APIs for the most common AI workloads.

Ray Libraries

The AI Compute Engine for Every Workload

Ray scalably and performantly executes AI workloads so you can focus on what matters. Spend less time on the complexity of modern machine infrastructure and more time running end-to-end machine learning workflows.

Data Preprocessing
Offline Batch Inference
Vector Embeddings
Distributed Model Training
Hyperparameter Tuning
Model Serving
Reinforcement Learning
End to End LLM Workflows

Data Preprocessing

Ray Data is the best-in-class choice for unstructured data processing, with support for any data modality and streaming capabilities to speed up training for traditional deep learning and Generative AI workloads.

Data Preprocessing
Ray-logo

Legacy You Can Lean On

40,000+

Github repo downloads

32.5k

Stars by the community

1000+

Contributors

At OpenAI, Ray allows us to iterate at scale much faster than we could before. We use Ray to train our largest models, including ChatGPT.
Greg BrockmanCo-founder and president, Open AI
openai-circle
“We were able to improve the scalability by an order of magnitude, reduce the latency by over 90%, and improve the cost efficiency by over 90%. It was financially infeasible for us to approach that problem with any other distributed compute framework.”
Patrick AmesPrincipal Engineer, AWS
aws-logo
“Ant Group has deployed Ray Serve on 240,000 cores for model serving. The peak throughput during Double 11, the largest online shipping day in the world, was 1.37 million transactions per second. Ray allowed us to scale elastically to handle this load and to deploy ensembles of models in a fault tolerant manner.”
Tengwei CaiStaff Engineer, Ant group
ant-group-icon
“Ray has brought significant value to our business, and has enabled us to rapidly pretrain, fine-tune and evaluate our LLMs.”
Min CaiDistinguished Engineer, Uber
logo-uber
“Ray enables us to run deep learning workloads 12x faster, to reduce costs by 8x, and to train our models on 100x more data.”
Haixuin WangVP engineering, Instacart
instacart-circle
“We use Ray to run a number of AI workloads at Samsara. Since implementing the platform, we’ve been able to scale the training of our deep learning models to hundreds of millions of inputs, and accelerate deploymnet while cutting inference costs by 50%.”
Evan WelbourneHead of AI and Data, Samsara
samsara-circle
“Ray has profoundly simplified the way we write scaalble distributed programs for Coheres’ LLM pipelines.”
Siddhartha KamalakaraML engineer, Cohere
cohere-circle

Get Started Today

Get started with best-in-class distributed computing, only with Ray.

Read Docs