Frequently Asked Questions (FAQs) on AI Code Monitor Tool
- What is Pure GenAI? How do you calculate it?
- What is Blended GenAI?
- What is Overall?
- What is NotGenAI%?
- What does this mean? "97.11% of code evaluated"
- What do you mean by "2% code attested"?
- What do you mean by "Not evaluated: 24 files"?
- What do you mean by "Python: 25 files"?
- What do you mean by "N/A" under GenAI% and NotGenAI%?
- What is Repository Group?
- What is Attestation?
- What is the 'Tick' icon and 'Cross' icon on Attestation?
- What is "Status checks" under Settings>Oraganization Information?
- What do you mean by "Apply at organization level"?
- What is GBOM (GenAI Bill of Materials)?
------------ ------------ ------------ ------------ ------------
1. What is Pure GenAI? How do you calculate it?
Percentage of chunk of code labeled as GenAI, where the whole chunk belongs to single commit. In order to calculate Pure GenAI%, we first detect that code is GenAIOriginated. If yes, we check if there is a second commit. If there is a second commit, we mark it as Blended GenAI. If there is no second commit, we mark it as Pure GenAI.
2. What is Blended GenAI?
Percentage of chunk of code labeled as GenAI, where the whole chunk belongs to multiple commits. In order to calculate Blended GenAI%, we first detect that code is GenAIOriginated. If yes, we check if there is a second commit. If there is a second commit, we mark it as Blended GenAI.
3. What is Overall?
Total percentage of GenAI in the codebase. It is sum of Pure GenAI% and Blended GenAI%
4. What is NotGenAI%?
Total percentage of human-generated code. Example: Code generated by:
- Developer at the organization
- External developer (such as Open Source code, or Stack Overflow-copied code)
- Auto-generated code (such as code provided alongside frameworks)
5. What does this mean? "97.11% of code evaluated"
Ticket CAW-840 is work-in-progress to clarify this definition
6. What do you mean by "2% code attested"?
This % is number of lines that has been attested divided by the number of lines that can be attested.
Not all code is attestable yet. For e.g. 946/47314 = 2%
7. What do you mean by "Not evaluated: 24 files"?
This is the count of files that has not been evaluated. The definition of 'Not Evaluated' is work-in-progress under ticket CAW-840
8. What do you mean by "Python: 25 files"?
This shows that there are 25 files of type 'Python'
9. What do you mean by "N/A" under GenAI% and NotGenAI%?
The % number under GenAI% and NotGenAI% represents the % of code that is GenAI and NotGenAI respectively. When that particular repository has Analysed files = Zero, then no calculation has been performed and hence it displays "N/A" i.e. Not Applicable.
10. What is Repository Group?
A user of AICM tool can create groups of repositories by giving a Repository Group Name and selecting a Usage category. This is useful in grouping repositories that you want to monitor against similar rules and get an average metric at a Repository group level.
11. What is Attestation?
Teams within the organization can create a permanent record of GenAI code composition, we call this “attestation”. They can do this by confirming or overriding the AI / Not AI label. So far, we have implemented Attestation at 2 levels: 1. Repository detail page level 2. Code review level
Refer this link for more information: https://announcekit.co/semasoftware/release-notes/ai-code-monitor-beta-v3.4-attestation-of-genai-code-labels-4yEm9q
12. What is the 'Tick' icon and 'Cross' icon on Attestation?
Tick - Attest by Agreeing with the label - When the user clicks on this icon, they are attesting that the label on that chunk is correct
Cross - Attest by Overriding the label - When the user clicks on this icon, they are attesting that the label on that chunk is incorrect and they override that by selecting another label and have an option to write a comment.
When this option is selected under 'Settings > Organization Information', then the Pull Request page on GitHub/Azure DevOps for that organization will show another check called 'Sema GenAI Detetector'
When this option is selected, it means that this Rule is applicable to the whole Organization, including all repository groups and developer groups.
If you are a user of AI Code Monitor tool and you want to get a detailed view of the GenAI usage for the developers in your organization, then you can download the GBOM file. For more information please refer: https://announcekit.co/semasoftware/release-notes/view-developer-usage-in-detail-using-gbom-download-how-to-video-1AJ70I
