DesignCon 2022 Schedule Builder

Recently exploding bandwidth requirements on high-performance computing devices raise several issues, including signal integrity (SI). Deep reinforcement learning (DRL) has drawn considerable attention to hardware optimization tasks, including floor planning and channel routing, because DRL can exclude human-experts knowledge with considering SI/PI issues. However, these methods suffer from obtaining real-world objective values of SI/PI because it is computationally expensive.

In this paper, we propose a novel offline reinforcement learning (Off-RL) framework, which is a clever method to reduce data computations of conventional DRL for practical hardware design. The proposed method introduces an expert policy, which is specialized data collection, unlike conventional DRL, collects data with DNN policy. The expert policy randomly collects data from the environment. Then trainable DNN-based policy imitates the sampled guiding solution from collected data. We expand our idea to net-ordering of the simultaneous escape routing (SER) as a benchmarking example of hardware design.

The proposed framework successfully finished training on one day, whereas the conventional DRL method never finished their training for ten days. The proposed method outperforms the conventional DRL method and genetic algorithm (GA). Our method is applied to real-world hardware devices, Olympus CPU board (open-source PCB) showing promising performances.

Takeaway

This paper tackles data-expensive issues of reinforcement learning-based hardware design considering signal integrity by proposing a novel offline-RL learning scheme. The proposed method is applied to canonical printed circuit board (PCB) layout design task, simultaneous escape routing (SER), showing substantial performances in terms of signal integrity.