Repository logo
 

Shedding light on LLMs: harnessing photonic neural networks for accelerating LLMs

Abstract

Large language models (LLMs) are foundational to the advancement of state-of-the-art natural language processing (NLP) and computer vision applications. However, their intricate architectures and the complexity of their underlying neural networks present significant challenges for efficient acceleration on conventional electronic platforms. Silicon photonics offers a compelling alternative. In this paper, we describe our recent efforts on developing a novel hardware accelerator that leverages silicon photonics to accelerate transformer neural networks integral to LLMs. Our evaluation demonstrates that the proposed accelerator delivers up to 14× higher throughput and 8× greater energy efficiency compared to leading-edge LLM hardware accelerators, including CPUs, GPUs, and TPUs.

Description

At head of title: Invited paper.

Rights Access

Subject

photonic computing
large language models
inference acceleration
optical computing

Citation

Associated Publications

Collections