{"id":33412,"date":"2023-09-18T02:59:00","date_gmt":"2023-09-18T09:59:00","guid":{"rendered":"https:\/\/insidebigdata.com\/?p=33412"},"modified":"2023-09-18T10:20:48","modified_gmt":"2023-09-18T17:20:48","slug":"anyscale-teams-with-nvidia-to-supercharge-llm-performance-and-efficiency","status":"publish","type":"post","link":"https:\/\/insidebigdata.com\/2023\/09\/18\/anyscale-teams-with-nvidia-to-supercharge-llm-performance-and-efficiency\/","title":{"rendered":"Anyscale Teams With NVIDIA to Supercharge LLM Performance and Efficiency"},"content":{"rendered":"
\n
\"\"<\/figure><\/div>\n\n\n

Anyscale, the AI infrastructure company built by the creators of Ray, the world\u2019s fastest-growing open-source unified framework for scalable computing, today announced a collaboration with NVIDIA to further boost the performance and efficiency of large language model (LLM) development on\u00a0Ray<\/a>\u00a0and the\u00a0Anyscale Platform<\/a>\u00a0for production AI.<\/p>\n\n\n\n

The companies are integrating NVIDIA AI software into Anyscale\u2019s scalable computing platforms, including Ray open source, the Anyscale Platform, and Anyscale Endpoints, announced separately today. <\/p>\n\n\n\n

The open-source integrations will bring NVIDIA software, including NVIDIA TensorRT-LLM<\/a>, NVIDIA Triton Inference Server<\/a>, and NVIDIA NeMo<\/a> to Ray to supercharge end-to-end AI development and deployment. Making cutting-edge AI software available via open source democratizes access and dramatically increases the audience of developers that can use this integration.<\/p>\n\n\n

\n
\"\"<\/figure><\/div>\n\n\n

For production AI, the companies will certify the\u00a0NVIDIA AI Enterprise<\/a>\u00a0software suite for the Anyscale Platform, bringing enterprise-grade security, stability, and support to companies deploying AI. An additional integration with Anyscale Endpoints will bring support for the NVIDIA software to a greatly expanded pool of AI application developers via easy-to-use application programming interfaces.<\/p>\n\n\n\n

\u201cRealizing the incredible potential of generative AI requires computing platforms that help developers iterate quickly and save costs when building and tuning LLMs,\u201d said Robert Nishihara, CEO and co-founder of Anyscale. \u201cOur collaboration with NVIDIA will bring even more performance and efficiency to Anyscale\u2019s portfolio so that developers everywhere create LLMs and generative AI applications with unprecedented speed and efficiency.\u201d<\/em><\/p>\n\n\n\n

\u201cLLMs are at the heart of today\u2019s generative AI transformation, and the developers creating and customizing these models require full-stack computing with efficient orchestration throughout the AI life cycle,\u201d said Manuvir Das, vice president of Enterprise Computing at NVIDIA. \u201cThe combination of NVIDIA AI and Anyscale unites incredible performance with ease of use and the ability to scale rapidly with success.\u201d<\/em><\/p>\n\n\n\n

NVIDIA AI Acceleration Speeds End-to-End Anyscale Development<\/strong><\/p>\n\n\n\n

NVIDIA\u2019s open-source and production software helps boost accelerated computing performance and efficiency for generative AI development. <\/p>\n\n\n\n

The integration delivers numerous benefits for customers and users: <\/p>\n\n\n\n