Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

A flexible, high-performance serving system for machine learning models

Serve, optimize and scale PyTorch models in production

An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models

TensorFlow template application for deep learning

In this repository, I will share some useful notes and references about deploying deep learning-based models in production.

AI + Data, online. https://vespa.ai

⚡️An Easy-to-use and Fast Deep Learning Model Deployment Toolkit for ☁️Cloud 📱Mobile and 📹Edge. Including Image, Video, Text and Audio 20+ main stream scenarios and 150+ SOTA models with end-to-end optimization, multi-platform and multi-framework support.

부스트캠프 AI Tech - Product Serving 자료

DELTA is a deep learning based natural language and speech processing platform.

A multi-modal vector database that supports upserts and vector queries using unified SQL (MySQL-Compatible) on structured and unstructured data, while meeting the requirements of high concurrency and ultra-low latency.

Database system for AI-powered apps

A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）

A comprehensive guide to building RAG-based LLM applications for production.

A scalable inference server for models optimized with OpenVINO™

Lineage metadata API, artifacts streams, sandbox, API, and spaces for Polyaxon

Generic and easy-to-use serving service for machine learning models

Lightning-fast serving engine for any AI model of any size. Flexible. Easy. Enterprise-scale.

A unified end-to-end machine intelligence platform

RayLLM - LLMs on Ray