Amazon

Amazon bought a small Israeli chip company called Annapurna Labs in 2015 for a reported few hundred million dollars. It is now the reason AWS can offer AI compute without paying NVIDIA margin on every instance.

Trainium and Inferentia split the workload the way the economics actually split: training is expensive and occasional, inference is cheap and constant, and the second is where a cloud provider’s cost structure is decided.

The strategic point is that none of it is for sale. Amazon is not trying to be a. It is removing a supplier from its own cost base, and renting the result back to customers as capacity.

Silicon from Amazon

5 chips on record, newest first.

  1. 2024

    AWS Trainium3

    3nm Third-generation AWS training chip, AWS's first on 3nm

    up to 362 FP8 PFLOPs per 144-chip Trn3 UltraServer

    Absolute
  2. 2023

    AWS Trainium2

    Not stated by AWS Second-generation AWS training chip

    up to 1.3 petaflops of dense FP8 compute per chip

    Absolute
  3. 2022

    AWS Inferentia2

    Not stated by AWS Second-generation inference chip, 32 GB HBM

    up to 190 tera floating operations per second (TFLOPS) of FP16 performance per chip

    Absolute
  4. 2020

    AWS Trainium

    Not stated by AWS First-generation AWS training chip, 32 GB per chip

    up to 3 petaflops of FP16/BF16 compute power per 16-chip Trn1 instance

    Absolute
  5. 2018

    AWS Inferentia

    Not stated by AWS First-generation AWS machine-learning inference chip, 8 GB DDR4

    128 teraOPS on 8-bit integer (INT8) data per chip

    Absolute

Compare against every other vendor in the silicon tracker

Our coverage of Amazon