The 4 biggest challenges of AI-generated code that Gartner left out of its latest report

Gartner is one of the tech industry’s most respected analyst groups. Its business is custom advisory services, conferences, and putting out multi-client industry analysis reports.

Becoming a Gartner client is not cheap, with estimates starting at $30,000 per seat per year. The way Gartner (and, to be fair, most other analyst groups) create interest is to put out press releases that summarize conclusions from its research reports, in the hopes that they’ll get picked up for coverage by outlets like ZDNET.

Since these press releases often contain nuggets that explain industry trends, we often glom onto them to try to understand the state of the industry.

Also: The best AI for coding in 2024 (and what not to use)

Gartner is back with a new press release. This one has the headline, “Gartner Says Generative AI will Require 80% of Engineering Workforce to Upskill Through 2027”

The press release goes on to discuss Gartner’s predictions of the short, medium, and long-term impact AI will have on software engineering.

In this article, I’ll look at Gartner’s conclusions, along with my analysis of multiple glaring omissions from the conclusions Gartner presents in its release. I reached out to Gartner to request the opportunity to examine the full report, which the company was kind enough to provide me with.

Also: Is AI in software engineering reaching an ‘Oppenheimer moment’? Here’s what you need to know

The full report very briefly mentions some of what I consider to be omissions. But since a lot more people will make their judgement calls based on the press release than the full report behind a five-figure firewall, it’s worth discussing some of Gartner’s valid observations and also the very important stuff left out.

Let’s talk about that headline

Gartner’s big headline is that AI will require 80% of the engineering workforce to upskill within the next two years. Later, in the release, Gartner clarifies that it meant software engineering rather than every engineering discipline.

The implication is that AI is going to so massively disrupt software engineering that if you don’t upskill, you’re effectively screwed. This is true. But this is not news.

Also: How I test an AI chatbot’s coding ability – and you can, too

Upskilling, learning new stuff and keeping up with technology, is a key aspect of any software engineer’s job. In this article, I’m going to use terms like programmer, coder, software engineer, and developer somewhat interchangeably. There are subtle differences, but for the purpose of what we’re discussing, they’re all applicable.

My point is, if you’re a coder, you need to be constantly learning.

There is nothing surprising about having to upskill as a software engineer. Programmers always have to upskill. Every new operating system release, every new language, every new API, every new piece of hardware released requires upskilling.

Also: The rise and fall in programming languages’ popularity since 2016 – and what it tells us

AI will also require engineers to grow their skills, but that’s no big change in what it takes to maintain a career as a software engineer or a developer.

So yeah, sure while the headline is credible, it’s just not as consequential to coding careers as it seems to imply.

Short, medium, and long-term impact of AI 

Here’s how Gartner describes the stages of AI impact.

• In the short term: “AI tools will generate modest productivity increases by augmenting existing developer work patterns and tasks. The productivity benefits of AI will be most significant for senior developers in organizations with mature engineering practices.”
• In the medium term: “AI agents will transform developer work patterns by enabling developers to fully automate and offload more tasks. This will mark the emergence of AI-native software engineering when most code will be AI-generated rather than human-authored.”
• In the long term: “While AI will make engineering more efficient, organizations will need even more skilled software engineers to meet the rapidly increasing demand for AI-empowered software.”

I don’t disagree with this assessment, although you can’t really bucket these three outcomes into time periods. AI tools, now that we have them, will be able to augment coding both short term and long term.

Also: How ChatGPT scanned 170k lines of code in seconds and saved me hours of work

Symbolic debuggers, for example, have existed for 40 years or so, and we still use them daily. They’ve gotten better, they do more things, and they reflect the nuances of their working environments, but they’re still in use. That will be the case with AI-assisted programming.

“Building AI-empowered software will demand a new breed of software professional, the AI engineer,” according to Gartner analyst Philip Walsh. “The AI engineer possesses a unique combination of skills in software engineering, data science and AI/machine learning (ML), skills that are sought after.”

Keep in mind that there is a distinction when it comes to AI’s impact on coding. Sometimes, coders will use AI for programming (to get more code working). Sometimes, developers will use AI in programming (to make the code and solution they are working on smarter).

Also: AI engineering is the next frontier for technological advances: What to know

Gartner did a survey at the end of last year, asking 300 software engineering managers about job roles. 56% said AI and machine learning engineers were the most in-demand role and applying AI and machine learning to applications was an area where their teams needed to increase their skills.

This makes total sense. AI and machine learning features have the potential to add tremendous value to applications in almost any field.

The 4 glaring omissions 

According to Gartner’s Walsh, “In the AI-native era, software engineers will adopt an ‘AI-first’ mindset, where they primarily focus on steering AI agents toward the most relevant context and constraints for a given task.”

Also: Yikes! Microsoft Copilot failed every single one of my coding tests

The basic premise with “AI-first” programming is that developers hand assignments to an AI and the AI returns code for the project. This is directly analogous to a programming manager handing assignments to devs on the team, and those devs returning code for the project.

The full report did mention that senior developers might be tasked with quality-checking code in organizations without automated code review tools. But there’s a lot more to the problem than was discussed, even in the full report. Here are the four omissions from Gartner’s press release.

1. It’s always necessary to make revisions to get the code to meet spec
2. The substantial testing and quality control required for all software
3. The need for updates, maintenance, bug fixes, and performance tuning throughout the lifecycle
4. It’s much harder to maintain code you did not write

You just can’t discuss the future of software engineering without giving these four aspects of the craft some significant attention.

1. Revising code to meet spec

The scenario is simple. You feed a prompt to an AI or a set of guidelines to a programming team. Time passes (less for the AI, more for the team) and you get back a chunk of supposedly working code.

Then you try it out. It’s not quite right. It’s never going to be quite right on the first run. Even if you’re doing your own coding, the first run after writing the code is going to have bugs. It’s one of the natural laws of the universe.

RIght now, you can tell ChatGPT that what it gave you doesn’t work, or give it the error message and it will generate a whole new block of code. But AIs have a much harder time incrementally revising their work to tune into what’s needed.

This applies whether the AI is text-to-text or even text-to-image. Try getting Midjourney to make just the changes you want in an image. You can specify small areas to work on, but then Midjourney will do what Midjourney wants to do.

Also: The most popular programming languages in 2024 (and what that even means)

To be able to have a project where developers “hand assignments to an AI and the AI returns code for the project,” the AI has to be able to take criticism and make small, focused changes.

Right now, each returned revision is a complete roll of the dice.

2. Testing and quality control

It is possible to get one AI to test the output of another AI. I’ve done this with fairly good results, especially checking how regular expressions are constructed and will perform.

Also: 8 ways to reduce ChatGPT hallucinations

But given how much AIs hallucinate, and how often an AI will write code that just doesn’t work, testing protocols and suites will need to be developed as part of the AI programming process.

All that is likely to require humans to do much of the work. We won’t be able to rely on AIs to code until we can be sure the code is good. The bigger the project, the more complex the code, the more we’ll need human programmers and project managers to shepherd this all through the process.

3. Updates, maintenance, bug fixes, performance tuning

Software is far from static. Once released, bugs will need to be fixed, updates will need to be implemented, performance will need to be tuned, and so on. None of these are things an AI can do across a whole product.

Also: How to use ChatGPT to write code: What it does well and what it doesn’t

That’s not to say that an AI can’t help. It can certainly be handed a function, for example, and be prompted to rewrite (note that I did not say “update”) the function to provide better performance.

Of course, now you have an entirely new function that needs to go through functional testing. If something broke in the rewrite, you couldn’t look to just the few lines that were modified. You’d have to find the bug in the entire block of code.

4. Maintaining code you didn’t write

Maintaining code you wrote is work. Maintaining code you didn’t write is a whole ton of work. When you write the code (whether as an individual or a team), you have some knowledge of why the code was structured the way it was, how all the pieces work together, and why certain decisions were made.

When you maintain code that someone else wrote (human or AI), the code is something completely new. You first have to deconstruct the entire architecture of the code. If you’re doing it right, you’ll carefully document everything you discover. This deconstruction process is very time-consuming — and that’s before you modify or add even one line of new code.

I have some experience with this. Back in 2015, I adopted 10 open source WordPress plugins. That meant I was responsible for maintaining the code, fixing bugs and security flaws, and in a few of them, turning them into more commercially functional products.

Also: The best free AI courses

Not only did I have to deconstruct how the original programmers wrote their code, I also found that — in a few cases — I hated their programming styles. Programmers can be very particular about programming styles, the way in which the code is written and structured.

In most cases, code will run when written in a variety of styles, but each programmer’s brain works differently and gravitates to certain ways of writing code. I am particularly persnickety about code style. If it’s written the way I like, I’m incredibly productive. But if the coding style annoys me, my productivity drops and I get cranky.

Also: How to use ChatGPT to create an app

I have tried to get ChatGPT to convert from one of the more annoying coding styles into one I like, particularly for the mix of PHP and HTML. ChatGPT did not succeed. Over time, it may be possible to carefully train an AI on a preferred coding style and have it update adopted code into the style favored by the developer or development team, but we’re not there yet.

As such, maintaining AI-produced code will not only be more difficult than writing the code originally, it may also conflict with good style practices, making it even more annoying to work on.

Will AI replace developers?

The research report Gartner is promoting with its press release is titled “AI Will Not Replace Software Engineers (and May, in Fact, Require More).” That is a premise I wholeheartedly agree with — at least for the next decade or so.

Also: If you want a career in AI, start with these 5 steps

As I’ve shown, beyond using AI to help program or using AI to make programs more capable, there’s a lot of work to be done related to the fundamental logistics of programming projects.

There is no doubt AI can help with programming. I estimate that AI has save me at least a month in programming time over the past year, allowing me to complete projects I probably wouldn’t have undertaken given my limited free time.

But the code the AI helped write does break. A lot. And I’m finding myself spending as much time going in and fixing the edge cases the AI didn’t anticipate, reverse engineering the AI’s coding architecture, and then putting that code back into production. It’s certainly a far cry from hands-off delegation of coding work.

The full Gartner report is instructive and well-considered. But the report could use some more discussion of the areas I mentioned. I encourage the analysts to supplement their report with a greater focus on revision, testing, and ongoing maintenance. Keep in mind the process not just of generating code, but the entire lifecycle of a software project after the first set of semicolons* has been added at the end of all those lines of code.

Also: AI development and agile don’t mix well, study shows

What do you think? Will AI replace programmers or create more demand for programming skills. Are you already constantly upskilling? Do you think AI will require you to upskill a different set of talents? Do you think AIs will someday be able to do the bug fixing and maintenance that’s ever-necessary in programming projects.

*Note: Python does not require semicolons, but Java, JavaScript, PHP, and all the C-based languages do.

---

You can follow my day-to-day project updates on social media. Be sure to subscribe to my weekly update newsletter, and follow me on Twitter/X at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.