08 Physicist, software developer & CTO Jonas Kunze on improving product design processes with VR
Jonas Kunze is founder and managing director of flyingshapes GmbH. In our eight interview, the technically experienced physicist gives insights into the use of VR in the design process of products - especially for the automotive industry. With his company flyingshapes GmbH, he improves the process from the first sketch to the prototypes using VR. Even if companies are looking for applications like this, the freedom gained by drawing in three-dimensional space is sometimes a challenge. Jonas Kunze describes optional customizable freedoms and the change between dimensionalities as a solution.
Founder & CTO of flyingshapes GmbH
Points of contact with AR/VR:
occasionally VR Gaming
activity as founder and managing director of flyingshapes
Development of a VR software to simplify processes in product design
"For most people, VR is a medium for visualization. For us, it is a new, professional tool."
1) From which needs and with which intentions did you found flyingshapes?
It all started when Johannes and I, both originally physicists, were working on different experiments at CERN in Geneva. We were the software developers among the physicists there, and we spent a lot of time programming graphics cards. While Johannes finished his PhD, I founded my first start-up during that time. But this start-up was dissolved after some time and I looked for new projects. I met Johannes again at the founders meeting in Mainz.
We thought about doing something that we love that we are passionate about and that motivates us. After a while it was clear that the skills needed to develop a good product for virtual reality fit very well with our skills and competencies - and with our motivation.
In 2016 the first VR glasses of the latest generation came on the market and we were also influenced by the hype back then. Rationally speaking, we knew that we wanted to work with the technology. But we did not have a product yet. So we traveled through Germany and met different people. At a hackathon we met a Transportation Designer who taught us about the classic design process in the automotive industry.
First of all, designers create different sketches with pen and paper. With these sketches they try to convince the head of design. The designers whose sketches are convincing are given a modeller to work with. This person must now create a 3D model from the 2D sketch. This often causes some friction, because the sketches are often more like an Escher painting, so they are not realizable in 3D. VR has a huge advantage here. You can model and display the products in three dimensions without special knowledge. You get immediately a good feeling and a comprehensive understanding for the product. This is missing in the classic process of working on the screen. The perfect three-dimensional understanding cannot be conveyed by the different views in 2D. As an emergency solution, prototypes are produced at a very early stage with immense effort, for example in the automotive industry.
How are prototypes produced so far?
For small objects nowadays, these are 3D prints. For cars, however, clay modelling is used. Clay is a kind of mixture for modelling that designers use to build a model by milling, heating and hand shaping, a pretty sticky business. These models are important for car makers to identify problems before production. Primarily, the emotional effect of the design is checked. But that is a huge effort. You usually start with a one-to-five model and then go to the one-to-one model at the end. This means that in the end you have a full-size car made of modeling clay in front of you. The smaller clay models in the automotive industry are produced in two to three days and already cost about 30.000 Euro. With one-to-one modelling we are then already in the six-figure range and a working time of up to ten days.
Then the whole thing is also covered with an incredibly expensive paint film and brought into an artificial scenario, such as with painted house fronts and trees. Here the effect of the model, including the reflections on the surface, is analyzed in a scenario that is as realistic as possible. An almost unbelievable effort.
And in VR that's a click! There I can integrate trees and house fronts and have, at least visually, the same understanding and effect. The only thing that cannot be imitated by VR is the haptic. So we started with visualization and tried to optimize the design process in the automotive industry. We have developed a product with which the designer can model directly in VR and immediately see the effect in space without physically building a prototype. This means the process is extremely shortened.
Source: flyingshapes GmbH
2) What are the advantages of modeling in AR&VR? What difficulties or difficulties arise with regard to its use?
Most companies see the biggest advantage of VR in visualization. Besides the visualization they forget one, in my opinion even more important point: the possibility to interact. By putting on the glasses I can suddenly interact with controllers in my hand or actually interact with the three-dimensional media by finger tracking. So I skip a level of abstraction that I have when I work with mouse and keyboard, which makes it easier and more intuitive to use.
And this intuitive interaction is what we make use of. Of course, we still have to think a lot about how to make an application intuitive. But that's a lot easier in VR than in a 2D application. As a result, designers who used to work with pen and paper are now creating three-dimensional designs without extensive technical know-how. At the same time, most modelers have used a highly complex and comprehensive program. A high level of technical and mathematical knowledge was required of the modelers, which is similar to engineering knowledge. As a result, the aesthetics got out of focus.
flyingshapes starts right at the beginning of concept modelling and enables designers to actually model directly in space and identify problems. Sure, I can change the perspective in classic 3D programs. But the difference is the stereoscopic vision, i.e. getting different impressions with the left and right eye, which makes it possible to see the depth. And the whole thing also happens quite naturally by head movements instead of using the mouse. The natural movements are intuitive and fast.
In this way you have enabled designers to visualize their ideas in an easy way. This not only saves time in the process but also costs, right?
In the automotive industry design is a very important issue. Cars are sold by emotions, which are conveyed by the design. Good examples are the Mini or Porsche. As a result, the industry does not immediately start with the design when it comes to saving costs. Costs are therefore an advantage, but not the most important point.
More important at this point is to achieve a good quality and accordingly to transport really good emotions. The original process involves building many prototypes and testing them on a mass of concepts. Car manufacturers have huge halls for this, where they have their models, look at them and decide which one they want to develop further and in which direction they want to go. .
In VR the creation of prototypes and testing of different concepts is faster and I need less iterations to the final products because the iterations themselves are much better. You can now use this gained time in two ways. Either I save costs or I use the gained time to push the quality even further.
3) In your experience, what are the existing barriers for many companies and potential users to use AR/VR for the design process? How does your software differ from other, already existing applications?
Generally speaking, we have not completely reinvented the wheel. The underlying ideas have been around for years and we are not the only ones building on them. Providers in this field usually differ in their focus on a specific market, such as the fashion industry or art, or like us in the automotive sector. All in all, there are still very few. The reason for this is complexity. We are a deep tech start up, which was already working on the development years before we brought a usable product to the market. The development team consists of (PhD) physicists with a strong mathematical background, which reflects quite well the complexity of the topic. This is the biggest hurdle in developing an own application.
Most existing applications are based on two principles that were developed in the 70s or 80s and hardly anything has changed since then. Many developers do not feel up to the complexity
How do companies become aware of your solution?
Most customers are specifically looking for a product like ours. VW, for example, has itself started to develop a similar product as a test. This gave them a clear idea of what they needed. Based on the findings from their own development, they then approached us and were able to formulate exactly what the product should look like. A car manufacturer is not a software company and knows what it needs conceptually, but not technically.
What are barriers when using the application?
One hurdle in the application of the product is the aspect of being overwhelmed by the possibilities. This sounds paradoxical, because that is actually exactly the advantage of VR. But in the design process the illustrators are used to drawing a curve with pen and paper or mouse and keyboard 2D, for example. In addition, they have a lot of experience here, after all they have invested thousands of hours in their hand drawing. The changeover to a direct 3D drawing therefore initially presents a challenge. You can counteract these uncertainties by confronting the user only with the most obvious options. This means that they can decide for themselves how many new functions they want to use in order to create the best working environment for themselves.
In the first version of flyingshapes you could click with the controller in the room and thus project a complex curve into the room. This means absolute freedom in design. But sometimes this freedom is not wanted. We counteract this supposed problem by providing the user with various tools that are more familiar to him. This means that in some steps of the design process it is possible to switch to a 2D surface and draw there, e.g. on the tabletop. They are used to this, they feel comfortable with it.
Which are possible tools to make it easier for designers to get started with VR?
One example is the development of components of our software for Logitech and the new VR Ink Pilot pen. The pen is tracked in space and recognized in VR. Drawings in free space can be saved directly. At the same time, it is also possible to draw as usual on a table, like on a paper or graphic tablet. This is a natural and familiar movement for the designer, where he keeps the feeling of control. He can put his arm down as usual and carry out the usual movements. At the same time, however, he gets a direct impression of how his drawing looks in the room.
Source: flyingshapes GmbH
Are there any other difficulties with the integration of VR applications?
Another challenge is the connection to existing software and the corresponding data formats. The programs used vary from company to company and also within a group of companies. So far we could fall back on data formats that are supported by most programs, but we are of course intent on offering a suitable solution to new customers. To such customers we offer to integrate the right solution into the software, that is part of our business model.
4) Concerning the Corona crisis: Have you personally noticed changes in your professional or private environment with regard to the demand of your agency and your advice regarding the use of AR/VR?
We have definitely noticed a pretty positive change. Because of Corona, suddenly most people were working in the home office. For automotive designers, working in the home office is almost impossible. Many were even forbidden to work in the home office or to create new ideas because of the risk of the idea being stolen. Companies are very strict about this, as they have not had the right tools to design and store designs from home in a secure physical location. An alternative would be to work digitally in the classic sense with e.g. Photoshop. But even here, there is no clear process yet, as the data is on the computer without encryption and must be sent securely to the customer. We know from one customer that it took more than two months until the infrastructure was in place and the designer was allowed to work again.
We have benefited a lot from this problem. With our software we offer a collaborative, virtual design studio. Designers put on VR glasses in their home office, see their colleagues in a protected virtual space and can draw sketches and work on models together with them. The data is never on the designer's computer, but is stored directly on the company's already very secure servers.
Other products in the design field do not offer this at all. For this reason, we have received many inquiries.
Was your software also used for other collaborative purposes during Corona?
The design process usually involves designers creating a model and then presenting it to the management at the design reviews. During the design reviews, the management then tapes the model at the appropriate places on the model to indicate their change requests. The modeler then makes the changes, as the manager often does not have the skills to do this by himself.
We have mapped this process one-to-one in VR. The manager can note a tape in VR with his change requests on the model. Actually, this is no longer necessary, because the manager could make the changes in VR by himself and see the result directly, but the taping corresponds to the usual process. This is again one of the great advantages of VR. These design reviews were now also carried out much more frequently in VR because of Corona.
5) In general, how do you estimate the influence of Corona for the relevance for AR&VR, do you expect industry-/ application-specific differences?
Regarding the industry and application, I think it is important, independent of Corona, to have a meaningful application. One should not use VR just to go with the hype. With regard to Corona, I definitely see an increased relevance especially for collaborative work, as is the case with our customers. However, what I personally do not see as useful is when several people meet in VR to look at a PowerPoint presentation together. If we have good facial expression and gesture recognition in VR in the future, if it is even more natural and we no longer need artificial avatars, I see this as a good alternative. But at the moment I find 2D alternatives, such as zoom, to be more sensible. But when it comes to three-dimensional data that you work on or view collaboratively (e.g. fashion designers, packaging), VR already offers many advantages.
6) “In 5 years almost every household will own VR glasses like e.g nowadays mobile phones and laptops and use them for different applications like holiday simulation, shopping, meetings, for teaching/learning.”
What do you think about this statement?
If there stands XR, I'd say yes. There are two reasons why we at flyingshapes use VR: 1) From a technical perspective VR is a lot more developed than AR. 2) in VR you can fade out the outside world and can completely immerse into your own activity. This is a big advantage in our case, the design process. On the other hand, AR also offers many advantages that you don't have with VR. So I would say that as soon as there is a pair of glasses that is technically advanced enough to allow you to switch easily between VR and AR, I see great potential for such an XR glasses to replace mobile phones. I don't want to commit myself to the question whether this will already be the case in five years. Five years feels realistic to me, but at the end of the day it all comes down to research and development of XR glasses.
Final statement of Jonas Kunze
In my opinion, VR is used in too many areas where it is either too early because the technology is not yet ready or where companies are only interested in doing something with VR in order to achieve the appropriate visibility. It is debatable how useful this use is, but it often damages the image of VR. Fortunately, there are just as many good examples on the market where the use of VR offers many advantages.