semasoftware Release Notes logo
Back to Homepage Subscribe to Updates

Release Notes

See the latest features, improvements, and product updates

Labels

  • All Posts
  • Fix
  • Announcement
  • Improvement

Jump to Month

  • October 2026
  • September 2026
  • August 2026
  • July 2026
  • June 2026
  • May 2026
  • April 2026
  • March 2026
  • November 2024
  • October 2024
  • September 2024
  • August 2024
  • July 2024
  • June 2024
  • May 2024
  • April 2024
  • February 2024
  • January 2024
  • December 2023
  • November 2023
  • October 2023
Powered by

Create yours, for free!

2 years ago

AI Code Monitor Beta V3.10: ROI of AI

Here's the latest video release notes: https://youtu.be/QNRl-pP0sYA

In the current release, we have implemented the ROI of AI functionality, which is explained below: 

The basic point here is to enable Engineering leadership teams to show how much more productive the team is as a result of using Generative AI tools, how much more effective they are currently and what percentage of the potential have they gained? 

Reason one for this functionality is to help, guide and provide teaching and coaching opportunities for the Engineering teams.

Reason two is to give updates to the rest of the organization, who, of course, are very keen to understand how Gen AI coding is affecting the Engineering team

How it works: Measuring ROI of AI happens at the repository group / product level. That's because there's so much nuance for each team, it doesn't really make sense to set measures for the Engineering organization as a whole. There's two parts to how the ROI of AI works. 

> One, there are settings that Engineering leaders can tune based on the specifics of their situation.

>  And then second, the AI Code Monitor detects how much Gen AI there is and comes up with a conclusion based on the settings from the Engineering leader. 

On part one, Engineering leaders select three things:
1. How much time are Engineers spending in coding. If they're only spending 30% of their time coding, even no matter how good the AI tool is it's there's only going to be a 30% increase in potential impact.
2. What is the primary Engineering focus of that product, such as starting a new code base, or within an existing code bas -, adding features and functionality, or tech debt reduction. There are five choices to select from. Sema has default expected productivity improvement, based on research, which is available to review via a link to a blog post within the product. Then, Engineering leaders can override the defaults based on the specifics of their situation.
3. Decide how much Gen AI is appropriate, given the unique circumstances of that team and what they're working on. If the team uses Gen AI to that maximum desired level, then they are getting the full benefit of whatever the potential productivity gain is.

In this example, for sample product "Crow-Proof of Concept":
1. The Engineering team spends 30% of their time coding
2. GenAI could generate 30% productivity improvement
3. The Engineering team reaches its maximum benefit of GenAI when 30% of the code originates with GenAI.

The results are:

1. The team is now 3% more productive as a result of Gen AI usage
2. They have captured 36% of the potential from using Gen AI.



Avatar of authorMatt Van Itallie