Advanced packaging and chiplets to rule the world!

Global semiconductor market is expected to cross $1 trillion by 2030. As the semiconductor industry endeavors to reach this milestone, it will encounter both headwinds and tailwinds.

Jean-Christophe Eloy, CEO, Yole Group, presented on how advanced packaging and chiplet to rule the world at the recently-held SEMI Industry Strategy Symposium 2024, in California, USA.

Jean-Christophe Eloy.

Chiplet, advanced packaging and advanced substrates, are a combination of technologies with big impact. What counts as a chiplet? The answer lies is disaggregation and duplication. In disaggregation, the SoC monolithic die is partitioned in smaller chips with different functions, and then interconnected in the same package. For duplication: two or more SoC monolithic dies are interconnected in the same package forming a bigger SoC.

There will be adoption of chiplets and heterogenous integration (HI). A partitioned die will see more dies per wafer. Higher yield will lead to optimized costs. A finer bump or pad pitch will see higher density. There will be optimization node per chiplet. All these will lead to faster time-to-market.

An advanced packaging platform such as 2.5D/3D, fan-out or high-density flip-chip is needed. We can probably have many options: FC on organic substrate, FC on organic with RDL (UHD FO), mold interposer and Si bridges, Si interposer, Si bridges, D2W/ W2W hybrid bonding (3D SoC), etc.

In 2023, the tech industry was about ~$5,500 billion. Data center capex was $212 billion, and grew at 19 percent. Servers were $120 billion, growing at 13 percent, and devices were $82 billion, growing at 15.4 percent, respectively.

Hardware implications of GenAI
There are hardware implications of GenAI. The explosion of Gen AI applications, such as ChatGPT and Dall-E, combines the trend of exponential model size growth with massive acceleration of user demand. As such, the computing hardware requirements for these generative AI applications are substantial, with an acute impact to GPUs.

For training on GPT-3 with 175 billion parameters, for example, we estimate that between 6,000 and 8,000 A100 GPUs would have required up to a month to complete. There are dozens of other LLMs being trained and deployed per year, a number that will only grow as the potential of generative AI achieves wider realization.

As user base has grown massively for these applications, and outputs consist of complex media, such as natural language or even images, computing load for model inference is much higher than on previous cloud-based AI. GPUs, instead of CPUs, are being used for generative AI model inference, as well.

GPU revenue has been increasing, mainly from datacenter applications. Growth of AI acceleration in server applications is creating a tailwind for GPU, accelerating 2023 revenue growth. Nvidia’s Q4 revenue projection is 230 percent increase over Q4-2022. As macro-economics impact PC shipments and crypto values reset, client GPU revenues are pulling back causing a plateau of total GPU revenues until 2025, when pricing of enthusiast GPUs lift the overall client segment.

Economics of chiplets
Let us study the economics of chiplets. Chiplets add cost when total area of silicon is larger to enable die-to-die interconnect. Die-to-die interconnect scheme means advanced packaging techniques, However, chiplets reduce cost when smaller individual dies have higher fab yield (save fab cost). We have an opportunity to use lower cost nodes for certain IC functions. We also have an opportunity to create new products by mixing dies already designed (save design cost). There is more high-end performance packaging adoption.

There have been several commercialized and announced products. Intel has chiplet and HI as focus of IDM 2.0 strategy. AMD has shipped several generations of PC and server processors using chiplet interconnect platform Infinity Fabric (IF). AMD uses organic substrate to integrate chiplets. Apple released M1 Ultra product, a combination of two M1 Max dies interconnected using TSMC’s LSI technology. Amazon launched Graviton3 data center processor during AWS in 2021. It is a chiplet product with seven dies and 55 billion transistors. Dojo D1 is Tesla’s in-house supercomputer for ML. 25 D1 chiplets are packaged together. Biren Technology’s BR100 GPU consists of two chiplets (dual-die GPU) processed on TSMC’s 7nm node.

High-end performance packaging market
The high-end performance packaging market is expected to grow from $2.21 billion in 2022 to $16.7 billion by 2028. If we look at 3D technology trends, The potential players are those who are involved in 3D stacking. There is no clear picture of who will be the first player and service provider. This approach can be used for HBM 3+ memory. HBM 3+ memory is expected to use hybrid bonding. A collective die-to-wafer approach will be used for DRAM and logic die stacking, but both D2D and D2W could be used.

HBM shipments are expected to outgrow the market over the next five years. They will be used in AI workloads in datacenters. HBM2e is currently the high-volume part, and HBM3 commenced volume shipments in Q3-22. Intel and AMD use of HBM as CPU cache with next-gen (SR/Genoa) CPUs will drive demand. HBM has an ASP premium >500 percent vs. DDR4. This is expected to decline as volumes increase. HBM 2023-2028 shipment are expected to grow CAGR 45 percent.

In 2023, HBM pricing per Gb is roughly 6x of mainstream DRAM, and has resisted the broader ASP trend. With increasing process maturity and competition, combined with a healthier market for mainstream DRAM, we expect HBM price premium to reduce to ~3x of mainstream DRAM by 2028.

OSAT market
The packaging market is strongly dominated by big players. The top six OSATs have taken 70 percent of the OSAT market in 2022. These companies are not only investing the most, but also investing more in advanced packaging technologies. Apart from the IDMs and foundries, these have potential to penetrate the high-end performance packaging segment.

On one side, the top OSATs hold strong collaborations with IDMs and foundries. As these collaborations get stronger, OSATs will be slowing enter the high-end packaging supply chain. On the other hand, top OSATs are allocating most of their capex to develop their own advanced packaging solutions.

We expect their revenues to be increasingly driven by the advanced packaging segment.

In 2023 the packaging capex of the top nine players is expected to reach $12 billion, which is 17 percent lower than the previous year. Packaging companies are cautious about their investments and capacity increases, and decreased capex for 2023 with decreases from 10 percent to 40 percent compared to the previous year. The current macroeconomic situation is causing lower consumer confidence and therefore, slowdown in demand, leading to high inventory levels and lower capacity utilization at semiconductor companies.

$15.5 billion was invested in the advanced IC substrate sector between 2021 and 2022, strategically positioned to strengthen the escalating market demand. These investments were not exclusively driven by corporations. Customers have joined hands with substrate industry leaders in co-investment aimed at expanding production capabilities to keep pace with demand. However, it is important to note that this expanded capacity is often dedicated to the investing customer, potentially limiting access by smaller customers.

AT&S, in a milestone move, embarked on its most substantial investment to date, with aspirations to secure a position among the top three IC substrate suppliers in the near future. AKM Meadville adopted a multi-site approach, channeling investments into Guangzhou, Xiamen, and Shanghai. This strategic investment portfolio could cover advancements in advanced IC substrates and board businesses, diversifying their offerings to cater to a broad spectrum of market demands.

The IC substrate market was worth $15.14 billion in 2022. It is expected to show an 11 percent CAGR during 2022-2028, and move to $28.96 billion in 2028.