Video: https://youtu.be/FC7FweUYO2s 

The AI Code Monitor tool is in Beta. Regarding Developer attribution, while the math is correct given a certain AI/ Not AI composition, the formula we picked first is one of several possible ways to think about developers’ code and GenAI. 

So, if code was generated at Time1 by Developer1 and was not Blended initially, and then the code was modified at Time2 by Developer2, then we want the system to give credit for the code ultimately being Blended.

The current version of developer attribution works as follows:

Developer percentages refer to the amount of code in a moment in time. We get the whole codebase for a certain commit (the last of each day). Then we analyze chunks to set GenAI labels. For each line of each chunk, we attribute that line to the LAST developer that committed to them.

For example:

Let's say Developer1 pushes a function, which is PureAI. It has 30 lines.  Developer1 gets 30 lines PureAI attributed to them. 

Next day, Developer2 modifies 20 of those 30 lines. Now that entire chunk is labelled as Blended, but Developer2 has 20 lines Blended, and Developer1 has 10 lines Blended, even though Developer1 didn’t do anything on the next day. 

Video: https://youtu.be/PRwIHvcFA0k

If you are a user of AI Code Monitor tool and you want to get a detailed view of the GenAI usage for the developers in your organization, then you can download the GBOM file (See the columns 'earliest_committer' and 'all_committers') from the Dashboard and view the following details:

• repository_group
• repository
• file_path
• start_line
• end_line
• num_lines
• not_genai_lines
• pure_ai_lines
• blended_lines
• commit_sha
• commit_date
• language
• earliest_committer
• all_committers
• ranges__start|end|label|author|commit_sha|commit_date

 *GBOM = GenAI Bill of Materials

Sometimes, the GBOM file could be big in size (depending upon your organization and repositories). In that case, you will see a message (see below) on the Dashboard. After a short while, you will receive an email (see below) with the option to download the GBOM file. Upon clicking the link, the csv file will be downloaded on your system.

Video: https://youtu.be/bwmdfDXNi7c

Development / Engineering team members of your organization can manage the GenAI code composition statistics during the Code review phase of the SDLC. When they have opened a Pull Request in GitHub, the GenAI code composition for that Pull Request is visible to them on the Pull Request details page and then on the AICM Tool's Pull Request details page. They can agree with the GenAI code labels provided by the AICM tool or they can override and re-label it based on their knowledge.

On one hand, this allows them to verify and rectify the GenAI code composition labels for this PR, while on the other hand allows them to approve/reject a Pull Request based on GenAI code usage.

Follow these steps to do 'Attestation':

Step1. Go to the GitHub or AzureDevOps tools to look for Pull Requests

Step2. Go to the 'Checks' page where all otehr checks are listed. Here you will find Sema GenAI Detector

Step3. Click on 'Sema GenAI Detector' link to view the GenAI Composition for this PR

Step4. You can further click on 'More Details' link to go to the PR detail page on AICM tool to view the GenAI Composition for this PR and the code assocaietd with this PR. Once you are on this page, you can take the following actions w.r.t. 'Attestation'.

1. Agree – By clicking the 'Tick' icon, you are Attesting that you agree with the GenAI code labelling.
2. Disagree [add comments (optional)] – By clicking the 'Cross' icon, you are Attesting that you do not agree with the GenAI code labelling, and you select from the given options of Blended, PureGenAI, NotGenAI – that you think is the correct label.
3. Do nothing - This doesn't count as attestation.

After doing Attestation, the chunk will show the name of the person who did the Attestation and the timestamp of when this was done. You also have an option to Edit.

Note: When you attest a chunk in a PR, the PR GenAI stats moves based on the action taken by the most recent user. 

  

Stats before Attestation

Stats after Attestation: 

Attestation actions:

1) Agree with one chunk labelled as NotGenAI

2) Relabeled another NotGenAI chunk as Blended - because of this the stats changed at PR level

Old Stats: Overall = 4.83 %, Pure = 0.76 , Blended = 4.07 %

New Stats: Overall = 5.85 %, Pure = 0.77 , Blended = 5.08 %  (Blended and Overall increased significantly)

Here's the latest video release notes: https://youtu.be/4Wo9_Xko1k0

In the current release, we have implemented Attestation at 2 levels: 1. Repository detail page level  2. Code review level 

Teams can create a permanent record of GenAI code composition, we call this “attestation”, by confirming or overriding the AI / Not AI label.

1. Attestation at Repository detail page level: As a user of the AICM tool, when you login, you can go to the Repository detail page and view the GenAI code stats. You can manage the GenAI code composition labels displayed on the Repository detail page. You can agree with the GenAI Code labels provided by the AICM tool or you can override and re-label it based on your knowledge. This allows you to verify and (if needed) rectify the GenAI code composition labels for your organization.

2. Attestation at Code Review level: Development / Engineering team members of your organization can manage the GenAI code composition statistics during the Code review phase of the SDLC. When they have opened a Pull Request in GitHub, the GenAI code composition for that Pull Request is visible to them on the Pull Request details page and then on the AICM Tool's Pull Request details page. They can agree with the GenAI Code labels provided by the AICM tool or they can override and re-label it based on their knowledge. This allows them to verify and rectify the GenAI code composition labels for this PR, much before the code has been committed to production, hence, providing more flexibility in managing the GenAI code labels.

You can take the following actions w.r.t. 'Attestation'.

1. Agree – By clicking the 'Tick' icon, you are Attesting that you agree with the GenAI code labelling.
2. Disagree [add comments (optional)] – By clicking the 'Cross' icon, you are Attesting that you do not agree with the GenAI code labelling, and you select from the given options of Blended, PureGenAI, NotGenAI – that you think is the correct label.
3. Do nothing - This doesn't count as attestation.

Note: When you attest a chunk in a PR, the PR GenAI stats moves based on the action taken by the most recent user. 

Video: https://youtu.be/DRB6MemMQao

Steps to connect Azure DevOps to AI Code Monitor (AICM)

Step1. Go to your Azure DevOps account > Login > Create Personal access token

Step2. Create your Personal access token > New Token

Step3. Create a New Token

Add these details:

Step4. Token is created > Copy this token (This is a sample shown below)

Step5. Go to Sema AI Code Monitor Tool > Click Settings > Connections

On the AI Code Monitor tool:

• Click on this button

Step6. On AI Code Monitor Tool > Click Settings > Connections > Add Details

After successful connection, you will see this message

Step7. Configure Web Hooks

We need to configure service hooks so that our system is notified about the events and trigger the right flow for the tool to kick in.

In your Azure DevOps project click on project settings > service hooks and then select web hooks from the list and click next.

Select the repository you want to integrate the tool with. Add the following 4 events for the repository (or project):

Add https://app.semasoftware.com/api/webhook-azure-devops/ as the endpoint for the webhook.

Note: Watch the Video Instructions given below

Steps to Configure Webhooks:

Add https://app.semasoftware.com/api/webhook-azure-devops/ as the endpoint for the webhook (URL)

6. After completing, hit "Finish".

Step8. Go to Pull Request page on Azure DevOps to see the Sema GenAI Detector

Video: https://youtu.be/MOfIk7oHA38

1) Add Rule / Create Rule

• Name
• Description
• Conditions
• Risk type
• Is it applicable at organization level

2) Edit existing rule

3) Delete existing rule

Video: https://youtu.be/1kYlAmVc7uM

• It allows you to monitor the GenAI code composition at repository group level and in turn view the statistics at the underlying repositories
• You can visualize the risk status (High-Medium-No-Low) through the traffic lights (Red-Amber-Green-Yellow) displayed for your repository groups based on these rules. 

1) Add Group

• Name
• Usage category
• Rules

2) Edit existing group

• Name
• Usage category
• Rules
• Repositories

3) Delete existing group

Video: https://youtu.be/FM0Bn0qVdcg

1. Invite an individual member
2. Bulk invite members using their email id
3. Remove members

Video: https://youtu.be/ixt6nUluQAE

• The Dashboard allows the various organization members to view/manage the GenAI code composition. You can see the Overall % (Pure + Blended), Pure GenAI % and Blended GenAI %.
• You can see time-series trend data about GenAI code composition to improve your codebase. 
• You can view the GenAI code composition at a Repository group level to understand how GenAI code has been used in the underlying repositories and what risks have been flagged based on the rules assigned to these repository groups.