Virtual reality solution for the intuitive operation of CAM systems supported by Artificial Intelligence

Project Number:


Funding agency

Bundesministerium für Bildung und Forschung

The project “CAMStylus – Development of an AI-supported virtual reality solution for the intuitive operation of CAM systems” is funded by the Federal Ministry for Education and Research under the project number 01IS22006A.


Computer-aided Manufacturing is made difficult by the great complexity of the abstract parameters and the diverse labeling of these parameters as well as the use of completely different parameters in different CAM systems. This leads to difficulties in staff qualification, low user acceptance, and a lack of knowledge. To overcome these problems, the project CAMStylus enables users to communicate their desired tool paths with gestures or simple sketches. This will be done by connecting a VR monitor with integrated spatial infrared tracking of three-dimensional sketches and gestures to a CAM system which will turn gestures and sketches into tool paths by the use of artificial intelligence (AI).

Parameterization is one of the most difficult parts of machining operations. The reason for this is that on one hand, the three-dimensional machining operation is transferred into a two-dimensional view to be displayed on a PC, complemented only by a keyboard, a conventional mouse, and often times an additional 3D mouse. Secondly, the abstract parameters almost count as many as 100 to 150 parameters, the majority of which must be initially defined by the user in the design phases. This difficulty leads to low user acceptance, difficult staff qualifications and a lack of knowledge management.

User acceptance is low because the CAM user does not receive any feedback about the expected results of the path calculation. After the process of parameterization has been completed, it is possible to initiate the tool path calculation and check the achieved results visually. If the result does not correspond to the desired outcome, it is often unclear which parameter adjustments have to be made to correct it.

Staff qualification is difficult because parameter names are neither standardized nor normed, so a simple change from one CAM system to another CAM system can make it almost impossible for even experienced users to apply their experience.

Lack of knowledge becomes already apparent with the start parameters of CAM systems since these parameters are not chosen based on knowledge such as databases to simplify the work task. Today’s CAM systems have no integrated mechanism for collecting and maintaining knowledge.

Therefore, the basic idea of CAMStylus is to enable users to suggest their desired toolpath with rough sketches or simple gestures instead of abstract parameters on a PC. CAMStylus is meant to automatically determine the necessary parameters for the calculation of the tool path from these sketches and gestures based on Artificial Intelligence (AI). This allows the user to be decoupled from the abstract parameters of current CAM systems. Such a user experience increases acceptance and greatly simplifies employee qualification. In addition, the AI part of the system achieves knowledge management of the CAM system.

To reach the aim of the project, a VR monitor with integrated spatial infrared tracking of input objects such as three-dimensional sketches and gestures will be connected to a CAM system. With the transformed tacking data of the CAD/CAM system, algorithms will generate spatial trajectories of the three-dimensional machining and which subsequently will be voxelized. The voxel information can be analysed by a 3D picture recognition and 3D Convolutional Neural Networks (3D CNN). The resulting AI transformation layer is the central element for the interaction of the virtual reality environment and the CAM system to generate the tool path. The 3D CNN will previously be trained with labeled sketches, gestures and tool paths. The project revolutionizes the operation of established industrial systems as an IT system in production that uses both virtual reality and artificial intelligence in combination to provide an innovative CAM user experience. ModuleWorks’ role in the project is the implementation of VR interfaces in the CAM system and to guide the machine learning of 3D graphic recognition.


Start: July 2022

End: July 2024

Share with others

Share on linkedin
Share on xing
Share on email
Share on facebook
Share on whatsapp