Inf2 instances are the first inference-optimized instances in Amazon EC2 to support scale-out distributed inference with ultra-high-speed connectivity between Inferentia chips. You can now efficiently and cost-effectively deploy models with hundreds of billions of parameters across multiple chips on Inf2 instances.

Inf2 实例是 Amazon EC2 中的首个推理优化实例，可通过 Inferentia 芯片之间的超高速连接支持横向扩展分布式推理。 您现在可以在 Inf2 实例上跨多个芯片经济高效地部署具有数千亿个参数的模型。 AWS Neuron SDK 可以帮助开发人员在两个 AWS Inferentia 芯片上部署模型，并且可以在 AWS Trainium 芯片上训练它们。 它与 PyTorch 和 TensorFlow 等框架原生集成，让您可以继续使用现有的工作流程和应用程序代码，并且可以在 Inf2 实例上运行。 图表展示了使用 AWS …

AWS Inferentia2 chip delivers up to 4x higher throughput and up to 10x lower latency compared to Inferentia. Inferentia2-based Amazon EC2 Inf2 instances are optimized to deploy increasingly complex models, such as large language models (LLM) and latent diffusion models, at scale.

On this page we provide an architectural overview of the Amazon EC2 Inf2 instances and the corresponding Inferentia2 NeuronChips that power them (Inferentia2 chips from here on). The EC2 Inf2 instance is powered by up to 12 Inferentia2 chips, and allows customers to choose between four instance sizes:

2024年5月22日 · AWS Inferentia2 is the latest AWS machine learning chip available through the Amazon EC2 Inf2 instances on Amazon Web Services. Designed from the ground up for AI workloads, Inf2 instances offer great performance and cost/performance for production workloads.

2023年4月19日 · Inf2 实例是 [Amazon EC2 ] (https://aws.amazon.com/cn/ec2/?trk=cndc-detail)中的首批推理优化实例，可通过加速器之间的超高速连接支持横向扩展分布式推理。 您现在可以在 Inf2 实例上跨多个加速器高效部署具有数千亿个参数的模型。 与 [Amazon EC2 ] (https://aws.amazon.com/cn/ec2/?trk=cndc-detail)Inf1 实例相比，Inf2 实例的吞吐量最多可提高至4倍，同时延迟最多可降低至十分之一。

This document is relevant for: Inf2, Trn1 Inference Samples/Tutorials (Inf2/Trn1/Trn2) # Table of contents Encoders Decoders Encoder-Decoders Vision Transformers Convolutional Neural Networks (CNN) Stable Diffusion Diffusion Transformers Audio Multi Modal Encoders #

2023年4月17日 · The new Inferentia2 chip delivers a 4x throughput increase and a 10x latency reduction compared to Inferentia. Likewise, the new Amazon EC2 Inf2 instances have up to 2.6x better throughput, 8.1x lower latency, and 50% better performance per …

近日，亚马逊云科技打出了一套 AI“组合拳”， 推出了生成式 AI 服务 Amazon Bedrock 和 Amazon Titan 大模型，并宣布基于 Amazon Inferentia2 芯片的 Amazon EC2 Inf2 实例全面可用。 Amazon Bedrock 是一个新颖的完全托管服务，用于构建和扩展生成式 AI 应用，从而让客户可以通过 API 访问 AI21Labs、Anthropic 和 Stability AI 等 AI 初创公司的预训练基础模型，还提供对亚马逊云科技开发的基础模型系列 Amazon Titan FMs 的独家访问。 Bedrock 是客户使用基础模型构建和 …

2023年4月13日 · Inf2 instances are the first inference-optimized instances in Amazon EC2 to support scale-out distributed inference with ultra-high-speed connectivity between accelerators. You can now efficiently deploy models with hundreds of billions of parameters across multiple accelerators on Inf2 instances.