I told you about it in the Farming Simulator 19 preview, Martin Rabl explained to us how he had to use Hololens from Microsoft to give a presentation of the Fendt Ideal harvester. Today it's Lars Malcharek, the community manager of Giants, who tells us the whole story around this ambitious project on pcgamesinsider.biz. Translation.
“When game and reality meet, amazing things can happen. The collaboration between AGCO and Giants Software has demonstrated this by combining virtual and real farming in an unprecedented way.
A basic communication idea
It all started with a simple thought: How could the new Ideal AGCO combine harvester be presented to customers in the clearest possible way? With Farming Simulator, was it possible to create a training simulation so close to real conditions that customers could get a feel for the new machine in real life?
The idea quickly turned into a concrete project. The harvester should be replicated in the most realistic way, so that customers feel like they are using the real controls through a simulation. The Microsoft HoloLens was supposed to do the trick. The purpose of augmented reality glasses was to provide, for example, the driver with information about the controls when looking at them. With this project, members of AGCO and Giants Software met to finalize the details in the presence of two gameplay programmers.
Nicola Zuccolo from AGCO was very enthusiastic. “This simulator will be the perfect way for our customers to learn how to use a new machine,” he explained. "So far customers have only been able to watch one video, but with the simulator they can test all the functions themselves and really understand how the machine works."
A mission system
AGCO and Giants Software decided to work on a series of tasks that would be presented to the customer through the game and the HoloLens. In carrying out these "missions", the customer had to be guided step by step to discover all the new features as it would work in a video game.
“Together with AGCO, we planned the entire process of these assignments,” explains Stefan Maurus. We wanted to make the simulator usable by everyone. Someone getting on the machine for the first time had to be able to operate the Fendt Ideal through this game. That's why we really started the missions at the base: how to start and drive the Ideal , how the controls work and how to start or stop the cutter bar. "
Some problems to overcome
The first obstacle was the technical implementation of the control system. Where gamers worked with the mouse and keyboard, or a gamepad, AGCO's training simulator naturally needed the machine's original side console, but this console wasn't meant to be connected to a game. There were no connection points to a PC and no approach to establishing them without any problems.
“We had to invent all of the technology,” explains Stefan Maurus. From electronics to in-game implementation. We also built some electronic parts ourselves and made a physical connection from the armrest to a PC. The side console has been delivered to the Erlangen office and work has started there. It was first necessary to create the hardware to connect the side console to the PC. Stefan Maurus used his knowledge of electronics for this task.
“In fact, before working for Giants, I was in the electronics business,” he tells us. I quickly got an idea of how to connect the side panel to the PC. So we built it ourselves from scratch. Next, it was necessary to ensure that the data from the Fendt Ideal would be transferred in the best possible way so that it reflected the actual experience of the operator of the AGCO combine. The small details were of great importance. The actual conditions up to the GPS signal are simulated and transmitted by the software. The simulation was adapted with all the data available until everyone was satisfied with the result.
HoloLens, not intended for this use
There was one more hurdle to overcome: “HoloLens are not designed to work at such a short distance. In addition, the buttons on the console are very close to each other. So we had to change and rework the software several times until we got the desired result. Almost six months of development later, the simulator worked and could be presented to the public. The machine was first presented at AGCO's Ideal Combine presentation in September 2017, before the training simulator was presented to the public at Agritechnica.
Nicola Zuccolo and Stefan Maurus are particularly proud of the result: “We learned a lot, and when real and virtual farming are combined the possibilities are almost endless and we already have ideas to explore these possibilities. "
HoloLens compatibility for FS19 in sight?