Here's the latest video release notes: https://youtu.be/UQrDg4ltEyk

In the current release, we have implemented GenAI detection at Code Review level for GitHub users.

Step 1: As a user of AI Code Monito tool, you can see the Rules available at an organization level. These are some Sema pre-defined rules which you can adapt to your organization's needs. When the GenAI detection is being done, these rules help you understand if the GenAI usage is as per the guidelines set for that organization and if there are any GenAI risks associated with that.

Step 2: As a GitHub user, when you have done a Pull Request and you are on the GitHub Pull Request Overview page, you will see a new check called the 'Sema GenAI Detector'. 

Step 3: If you investigate further by clicking on 'Details' link, you will be taken to the GitHub Pull Request Detail page.

On this page, you can see how the rules apply to the PR level as well at the Repository level. The different colours have these meanings:

Green denotes strength,

Yellow denotes low risk

Orange denotes medium risk

Red denotes High risk

Step 4: If you want to see the code level information, you can click on 'More details', and you will be taken to the Sema Pull Request Detail page. This is the page on the AI Code Monitor tool that displays the traffic lights and the GenAI usage stats.

Here's the latest video release notes: https://youtu.be/R_551gdBiXc

In the current release, we have implemented the following items.

1. Delete Repository group & Developer group

1.1) As a user I have the option to delete a Repository group

When I am on Settings > Repository Groups, I can see the 'Delete' option next to the repository group

1.2) As a user I have the option to delete a Developer group

When I am on Settings > Developer Groups, I can see the 'Delete' option next to the developer group

--------------------------------------------------------------------------------------------------------------------

2.  Display rule capsules in specific order on the Dashboard 

2.1) As a user I see the rules capsules in a specific sort order under Repository groups on the Dashboard 

When I am on Dashboard, I can see the Rule capsules with traffic lights in a specific sort order starting with Any GenAI, Pure GenAI, Blended GenAI

2.2) As a user I see the traffic rules capsules in a specific sort order under Developer groups on the Dashboard

When I am on Dashboard, I can see the Rule capsules with traffic lights in a specific sort order starting with Any GenAI, Pure GenAI, Blended GenAI

--------------------------------------------------------------------------------------------------------------------

3. A sneak peek into the GenAI usage scan at Code Review level, more to come next week

Developer using the GitHub can view the GenAI usage at the code review level when a Pull Request happens. We are working on this functionality and we will share a final version in the next week's release notes. The 'work-in-progress' user journey is as follows:

3.1) The developer sees a Pull Request

3.2) When they view the Pull Request, along with other checks, they can see a 'Sema GenAI Detector' check as well

3.3) Upon clicking on 'Details', they will see more information pertaining to the Pull Request w.r.t. GenAI usage

3.4) Upon clicking 'More Details', they will be taken to the AI Code Monitor tool to view code-level details and labels on GenAI usage on the 'Pull request scan' page.

4. Under the hood we have made improvements in accuracy based on customer feedback and more varied human data.

Here's the latest video release notes: https://youtu.be/PmKHKrjdKQ4

In the current release, we have implemented the following items.

1. Self-serve for GitHub users of AICM tool

1.1) Any prospective user with GitHub can start a free trial of the AICM tool on their own (without any external help). They can click on 'Start Free Trial' button on the marketing site - https://www.semasoftware.com/ai-code-monitor. 

1.2)Upon clicking on 'Start Free Trial', they will be taken to a marketing page explaining in detail about the AICM free trial (https://www.semasoftware.com/ai-code-monitor-free-trial). 

1.3) Upon clicking the 'Start Free Trial' on this page, the user is taken to the Sign Up page.

1.4) Upon successful Sign Up, user is prompted to enter further details like Organization name & Industry (Optional).

1.5) In the next step, the user is taken to the AICM tool Dashboard asking them to connect to GitHub

-----------------------------------------------------------------------------------------------------------------------

2. Preset rules by Sema

These are the predefined rules created by sema in the backend. Any new organization will have these rules available to them under Settings > Rules area. The user can apply these rules at Organization level and/or Repository group level and/or Developer group level. The user can also edit these rules to adapt it to their situation and needs. On one hand this allows the user to understand and adapt the rules to suit their needs and on the other hand it is a time-saving feature to utilize predefined rules.

3.  NotEvaluated calculation on Dashboard

The message blurb at the top of the Dashboard clearly mentions the % of code that has been evaluated

The VS Code Widget for the AI Code Monitor is now available for free download.

Demo video: https://youtu.be/KQuEyhWx2i4

It is currently available for Python only.

The VS Code Widget helps developers understand how much GenAI code is in their codebase, and what type of GenAI code.

Copy-pasted GenAI code, or "Pure" GenAI, can be riskier than GenAI code that's been modified by developers, "Blended" GenAI.

And knowing the amount of GenAI in the code helps developers stay in line with team and company best practices for GenAI usage.
 

Here's the latest video release notes: https://youtu.be/nqVNAAefqTg

The AI Code Monitor helps companies answer three questions:

1. GenAI Code Composition.How much GenAI code do we have, and is it in line with our organizational goals and requirements? 

2. Are developers using GenAI to its full potential?

3. What risks from AI usage should we aware of?

In the current release:

1. We extend our rules engine to apply to Organization-level GenAI standards. In this case the codebase as a whole has less GenAI than desired (target 5%, actual 4.36%).

2. We enabled groups of developers and allowed rules to be extended to developer groups. Here, in house junior developers have a different, lower threshold for GenAI usage, than Senior developers, because they are still learning coding practices.

Here's the latest video release notes: https://youtu.be/QSgaE1jlt34.

The big addition is the ability to group repositories into groups:

And then to produce group-specific "traffic light" indicators to show whether a codebase is in line with the organization's standards for GenAI usage.

Video release notes: here.

Developer-Level Metrics: the dashboard now shows how much each developer is using GenAI, and if they are using GenAI, how much is Pure (copy and paste from the GenAI tool) vs. Blended (modified by the developer).

Expanded GBOM: there are now three tabs to the GBOM: 
1- unchanged: GenAI usage at the function level (subset of file)
2- new: GenAI usage at the repository level
3- new: GenAI usage by developer 


Coming soon: VSCode Widget.

To make it easier for developers to get the most out of GenAI tools whilethe company stays in compliance, we're rolling out a VSCode widget where they can track their usage. See the notification in the bottom right.

Video Release notes here: https://www.youtube.com/watch?v=gX6yWlsvQrE

Repo-level view: Now the AI Code Monitor shows Repository- level detail on AI-Generated code composition. These three repos, for example, include code that was made with 77.51%-82.56% GenAI-incorporated code.

This matters because the security, legal and compliance standards will vary by codebase usage. It is a much lower risk to have internal tooling have pure Gen AI, for example, since it will not need copyright protection and will likely be less susceptible to security attacks.


Expanded GBOM (tm): Now the Generative AI Bill of Materials (GBOM (tm)) includes whether the code was AI or human-generated, along with the confidence level, repository and file path, and size. In this example, the first file, lambada_function.py, size 477 Bytes, has a >99% likelihood of being purely human-generated, while the larger Generate_SQL file has a 99% likelihood of including GenAI code.

For example, the EU AI Act, which was approved by Eurpoean Parliament on June 14, 2023, but is not yet final, may in its final form cover companies with Headquarters, Developers, or Customers in the  EU. It will cover companies of all industries. The EU AI Act is a Compliance Standard.

The EU AI Act has three Components. One such component is greater scrutiny for Gen AI uses in certain high-risk systems.

Video release notes: https://youtu.be/-ybsV6BCUz4


Introducing the AI Code Monitor

We've added the AI Code Monitor to the CTO Dashboard to help companies manage the risks and capture the benefits of code created in full or part by Generative AI tools like ChatGPT and GitHub CoPilot.

You can read more about the benefits and risks from Gen AI code here. 

In the sample codebase analyzed, GenAI tools assisted with 25% of the code.

- Per the German Copyright Act, companies won’t get copyright protection on their code unless the Gen AI code is sufficiently blended.  

- Per the State government's LfDI AI Data Protection Discussion Paper from last week, there may be an additional scrutiny of companies that use Gen AI code to evaluate their employees' work product. This likely applies to companies with a custom-trained model.


Other improvements to the CTO Dashboard

The product now supports integration with BitBucket along with GitHub, and integrates data from the two version control systems seamlessly.

Multiple code quality metrics are now available from Codacy.

We're up to five integrations now:

Explainer video: https://youtu.be/PV-p2kLT4I0

Example Use Case- working with third-party development shops.

Here's an example of how users are using the Dashboard to keep third-party development shops in alignment. 

In particular, several organizations did not have the awareness they were looking for on who was working on their code, and how much staff turnover and activity level there is.   

Using the dashboard they can see the individual developers working on the product:

Too, the dashboard shows the trends in development activity (commits to master branch) to observe unusual variation:

At Sema we are passionate believers in looking at the data -- but also passionate about being curious rather than judgmental. We strongly advise the same when it comes to coding metrics, as software development is a craft, not a competition. Asking questions to get the context is essential.

In this particular case, what might look like a decline in productivity was actually an organization's push to release v1 of a new product in August/ September, followed by smaller, less frequent, more complicated refinements. This was expected and desired behavior.  

New Feature 1- Open Source metrics.

We added metrics from the 5th and 6th components (module) of our codebase health framework so that now there is a comprehensive picture.

Module #5 is Open Source risk -- the legal risk coming from using Open Source code with restrictive licenses like Copyleft. You can read more about this risk here. This data comes from Snyk and we'll be adding other Software Composition Analysis tools in the months ahead.

New Feature 2- Code quality metrics.

Module #6 of 6 is Code Quality. Here we added four charts from Codacy, too we'll be adding other code quality tools in the months ahead.

- Number of warnings about code style, security, error proneness, and performance 

- Duplicate code

- Code complexity

- Testing coverage

Here's an example:

You'll notice that we are considering Codacy's security warnings to be part of code quality, not code security.

Why? It is our view that organizations need an advanced security tool such as Snyk (but not just Snyk) to capture true security risk. So code quality tooling that also adds security risk-- Sonar is in this example, too-- is tracked in code quality.

Here and in so many places in the dashboard, we are opinionated about how to think about codebase health. We know you won't always agree-- but we commit to being transparent in our reasoning and approach, and to being sincerely open to feedback. 

New Feature 3- Full Codebase Health score.

As noted above, now that Open Source and Code Quality risk are included, we have representation from all six components of Codebase Health included in the overall Codebase Health Score.

As a result, companies who use GitHub, Codacy, Snyk and iRADAR can get a full Codebase Health score that updates in real time for quarterly reviews, sprint debriefs, even weekly check-ins with the team.

Not using all of those tools? The Codebase Health score doesn't "ding" you for not having a certain tool, it simply excludes those metrics from the numerator and the denominator.