Developing with AI: IntelliJ IDEA, Visual Studio IntelliCode and Tabnine in the test

JetBrains IntelliJ IDEA and Microsoft’s Visual Studio Code are inherently good programming aids. IntelliJ IDEA already includes machine learning-assisted code completion, while Visual Studio Code is arming itself with the AI ​​tool IntelliCode. In addition, there are some external AI extensions such as Kite and Tabnine for both, which promise to further increase the productivity of developers.

The AI ​​extensions mentioned work like the text suggestions that you get on a smartphone while entering text in the editor. You type in a text and the editor offers several suggestions with which you could continue the text. In programming this is known as autocomplete. However, normal code completion only lists the next useful addition, as can be determined, for example, via a code inspection. AI extensions can surpass this if they have been trained with suitable program examples. Then you are able to add complete text blocks without reworking, which increases the programming speed considerably.

To test this, we have put together two different test suites. The tests contained therein check how the tools perform in very different tasks. The first test suite consists of a Java program for recursive copying of directory trees and checks how technically oriented classes can be supplemented with concepts such as constructors, getter and setter methods and exception handling. In addition, she uses refactoring to examine whether the AI ​​functions go into a tailspin when extracting methods and renaming them.