14 DAY TRIAL // Google revealed some details regarding the performance of its custom-built Tensor Processing Unit (TPU), which was designed for machine learning, saying that it is 15 to 30 times faster than contemporary GPUs and CPUs.
TPUs have been powering Google's datacenters since back in 2015 and they've been largely used for its open-source machine learning framework. The first generation of TPUs have targeted inference, which means they use an already trained model.
The impressive power of the TPU mentioned above comes on production AI workloads that utilize the neural network inference.
What's more, the Tensor Processing Unit is quite energy efficient, delivering a 30 times to 80 times improvement in TOPS/Watt measure.
A different Google thanks to TPU
Google's experts explain that without the TPU, the computational expense of the deep learning models it deploys would have been tremendous and would have seriously changed the company's landscape.
"If we considered a scenario where people use Google voice search for just three minutes a day and we ran deep neural nets for our speech recognition system on the processing units we were using, we would have had to double the number of Google datacenters!" wrote Google hardware engineer Norm Jouppi.
Google explains that they felt the need to build the TPU precisely due to the compute-intensive machine learning they've been using for the past 15 years.
"TPUs allow us to make predictions very quickly, and enable products that respond in fractions of a second. TPUs are behind every search query; they power accurate vision models that underlie products like Google Image Search, Google Photos and the Google Cloud Vision API; they underpin the groundbreaking quality improvements that Google Translate rolled out last year; and they were instrumental in Google DeepMind's victory over Lee Sedol, the first instance of a computer defeating a world champion in the ancient game of Go," Jouppi adds.