Rakuten Mobile uses 4G and 5G for combining real, virtual motorsports racing

Rakuten Mobile, Inc. successfully conducted a demonstration of motor sports that fused the real and the virtual at the 2022 Inter Proto Series POWERED BY KeePer Round 2 held by Inter Proto Motor Sports Co., Ltd. Using 4G and 5G mmWave provided by Rakuten Mobile, an e-race was successfully conducted that fused the real and virtual by remote transmission of driving data, and the real-time synchronous augmented reality (AR) display of the virtual car and car racing information during the race.

This demonstration experiment was conducted simultaneously at two locations: Fuji Speedway, the venue of the 2022 Inter Proto Series POWERED BY KeePer Round 2 located in Shizuoka prefecture, and the Rakuten Mobile Ebisu Store in central Tokyo. The test car dubbed “kuruma” ran on the circuit at Fuji Speedway against the virtual car driven by e-racers using the racing simulator at the Rakuten Mobile Ebisu Store, using data obtained from both sources.

Overview of the demonstration:

Implementation of an e-race that fuses real and virtual by remote transmission of driving data
By transmitting the data of the test car driven by actual racing driver running on the Fuji Speedway in real time to the racing simulator driven by the e-racer at the Rakuten Mobile Ebisu Store, the e-race between the test car and the virtual car was successfully displayed on the game screen. A telemetry system*4 provided by M-TEC Co., Ltd. *1 that obtains and transmits data from sources including multiple sensors to measure the steering angle and other factors, GPS and cameras installed in the test car running on the circuit enabled the display of an equivalent virtualized test car for e-racing on the game screen. 5G mmWave was used for communication between the racing simulator and the game server, realizing a realistic racing experience in real time.

Real-time synchronous AR display of virtual cars in races and AR display of car racing information
At Fuji Speedway, by holding up a smartphone equipped with a dedicated mobile app developed by Rakuten Mobile over the course from the spectator’s seat, or wearing smart glasses and watching the race, spectators were able to view the race between both the test car that actually ran on the real-life circuit and the virtual cars driven by the e-racers competing at the Rakuten Mobile Ebisu Store. In addition, driving data such as car position, speed and engine speed generated by both the test car and virtual car in motion, the perspective video of the racing driver, and the game screen video of the e-race were also available for display on the same app. By utilizing Immersal Ltd’s*1 VPS (Visual Positioning System) technology, it was also possible to display AR content for spectators in seats around the circuit with higher accuracy.

As the 5G network environment continues to develop, the future is likely to see large-scale motor sports competitions that fuse the real and virtual worlds, as well as multi-angle distribution that allows spectators to check the race situation in real time from any race-side seat, offering a much-enhanced watching experience.

Rakuten Mobile will continue to cooperate with various companies and organizations through the Rakuten Mobile Partner Program, aiming to provide customers with new experiences that utilize communications and technology.

CT Bureau