``` ├── .github/ ├── ISSUE_TEMPLATE/ ├── bug.md ├── documentation.md ├── feature.md ├── CONTRIBUTING.md ├── Contribution License Agreement.docx ├── FAQ's.md ├── Features and Endpoints.docx ├── LICENSE.md ├── README.md ├── Release Notice.docx ├── github-pages/ ├── content/ ├── Explainability.html ├── Explainability_By Example.html ``` ## /.github/ISSUE_TEMPLATE/bug.md --- name: Bug about: Found a bug? Open an issue using the label bug. title: '' labels: '' assignees: '' --- **Describe the bug** A clear and concise description of what the bug is. Tag it to @InfosysResponsibleAI with the appropriate label. **Guardrail Name** Please specify the name of the Guardrail where you encountered the bug. **Expected behavior** A clear and concise description of what you expected to happen. **Screenshots** If applicable, add screenshots to help explain your problem. ## /.github/ISSUE_TEMPLATE/documentation.md --- name: Documentation about: Spotted a typo? Help improve our documentation by opening an issue title: '' labels: '' assignees: '' --- **Describe ** Please provide a clear and concise description of the specific typo issue, including the folder or document where it was found ## /.github/ISSUE_TEMPLATE/feature.md --- name: Feature about: Suggest an idea for this project title: '' labels: '' assignees: '' --- **Module Name.** Mention the module name **Is your feature request related to a problem? Please describe.** A clear and concise description of what the problem is. Tag it to @InfosysResponsibleAI with the appropriate labels **Describe the solution you'd like** A clear and concise description of what you want to happen. **Describe alternatives you've considered** A clear and concise description of any alternative solutions or features you've considered. ## /CONTRIBUTING.md # Contributing to Infosys-Responsible-AI-Toolkit 🎉 Thank you for your interest in contributing to **Infosys-Responsible-AI-Toolkit!** We appreciate all contributions, whether it's fixing bugs, improving documentation, adding features, or suggesting enhancements. Please take a moment to read this guide to ensure a smooth contribution process. ## 📋 How to Contribute There are several ways you can contribute. Please share your contributions via the [Contribution Page](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/issues) with the appropriate labels, tagging `@InfosysResponsibleAI`. ✅ **Report Issues**: Found a bug? open new issue and select [Bug](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/issues) Template and create a bug. ✅ **Suggest Features**: Have a great idea? Share it in the [discussions](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/discussions) page with the `suggest` label. ✅ **Improve Documentation**: Spotted a typo? Help improve our documentation by creating a new issue with the [documentation](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/issues) label. ✅ **Fix Bugs**: Help us squash bugs by submitting patches. Create new issue by selecting [Blank Issue](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/issues) Template and provide the bug fix with appropriate labels. ✅ **Add Features**: Implement new functionalities to enhance the project. Create new issue and select [Feature](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/issues) Template to suggest new Feature. ➡️ Refer to the guidelines in the **Pull Request Process** section to submit any feature or bug fixes. ## 🔥 Coding Guidelines 💡 Follow our Style Guide to maintain consistency. - Use meaningful variable and function names. - Keep functions small and focused. - Write comments where necessary. - Follow indentation and formatting rules. ## 📝 Issue Guidelines When reporting a bug, include: - A clear, descriptive title - Steps to reproduce the issue - Expected vs. actual behavior - Screenshots (if applicable) - Use the label `bug` Before submitting a feature request, please: - Check if a similar request already exists. - Clearly describe your idea and its benefits ## 🔄 Pull Request Process ### 1. Fork the Repository - Go to the repository you want to contribute to. - Click the **Fork** button at the top right corner of the repository. - This will create a copy of the repository under your GitHub account. ### 2. Clone Your Forked Repository - On your GitHub profile, go to the forked repository. - Click the **Clone or download** button, and copy the URL. - Open a terminal/command prompt and run: ```bash git clone ``` - Change into the project directory: ```bash cd ``` ### 3. Create a New Branch Before making any changes, create a new branch to work on: ```bash git checkout -b ``` Replace `` with a descriptive name for your changes. ### 4. Make Your Changes - Make the necessary changes to the code or documentation. - You can add new features, fix bugs, improve existing code, or update documentation. ### 5. Commit Your Changes - Once you've made the changes, add them to the staging area: ```bash git add . ``` - Commit your changes with a meaningful commit message: ```bash git commit -m "Brief description of changes" ``` ### 6. Push Changes to Your Fork - Push the changes to your forked repository: ```bash git push origin ``` ### 7. Open a Pull Request - Go to the original repository where you want to contribute. - Click the Pull Requests tab. - Click the New Pull Request button. - Select the base repository (original repository) and the branch you want to merge into (usually `main` or `master`). - Select your branch from your fork to compare. - Click the Create Pull Request button. - Add a descriptive title and detailed explanation of your changes in the PR description. ### 8. Review and Merge - The project maintainers will review your pull request. - Be prepared to make changes based on their feedback. You can push additional commits to the same branch, and the PR will automatically update. - Once the maintainers are satisfied, your PR will be merged into the main project. ### 9. Contribution: - Ensure that your contribution license is compatible with the MIT license and that there are no copyleft licenses present, such as GPL, AGPL, LGPL 2.1, MPL 1.0, etc. If you wish to contribute, please review the [contribution license agreement](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/blob/master/Contribution%20License%20Agreement.docx) and email a signed PDF of this agreement to infosysraitoolkit@infosys.com. Additionally, refer to the [release notice](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/blob/master/Release%20Notice.docx) for the permissive licenses used in this project. ### 10. Celebrate 🎉 - Congratulations! You've contributed to an open-source project! 🎉 ## 📜 License By contributing to this project, you agree that your contributions will be licensed under the project’s [LICENSE](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/blob/Release-2.1.0/LICENSE.md) file. ## /Contribution License Agreement.docx Binary file available at https://raw.githubusercontent.com/Infosys/Infosys-Responsible-AI-Toolkit/refs/heads/main/Contribution License Agreement.docx ## /FAQ's.md # FAQs Below are some frequently asked questions to use `Infosys-Responsible-AI-Toolkit` repositories. ## How can I avoid issues with long file paths when extracting a ZIP file? When downloading the entire codebase as a ZIP file, extract it into a folder with a shorter name. This helps avoid issues caused by excessively long file paths when extracting or running the modules. ## How do I download models from Hugging Face? To download models from Hugging Face, sign up on their website. Once registered, you can download the required files. ## What should I do if the packages in the requirements file are updated? If the packages in the `requirements.txt` file are updated, follow these steps: 1. Verify the issue in the command prompt. 2. Try upgrading or re-installing the dependencies using the `requirements.txt` file. 3. If the issue persists, delete and reinstall the dependencies using: ```bash pip install -r requirements.txt ``` ## How can I resolve connection issues with MongoDB? If you face connection issues with MongoDB, follow these steps: 1. Create an empty folder for your database files. 2. Open the command prompt and run the following command: ```bash mongod --dbpath /path/to/data/directory ``` ## How can I check the available features in RAI? Refer to the `README.md` file in the respective repositories for general information. For details on specific features and responses, check the endpoint documentation available within the repositories. ## Which version of Python is required to run the application? Refer to the `README.md` file in the respective repository to determine the required Python version. ## How can I clone a repository? Follow these steps to clone a repository: 1. Copy the repository URL from the **Code** section. 2. Open the command prompt and navigate to your desired directory. 3. Run the following command: ```bash git clone ``` ## How can I create a virtual environment? 1. Open the command prompt and navigate to the directory where the repository is cloned. 2. Run the following command to create a virtual environment: ```bash python -m venv ``` ## How can I create a database or collection in MongoDB? 1. Open **MongoDB Compass**. 2. Enter your connection details (hostname, port, authentication) and click **Connect**. 3. Click the **Create Database** button. 4. Enter the **Database Name** and **Collection Name**. 5. Click **Create Database** or **Create Collection**. ## How can I activate the virtual environment? 1. Navigate to the directory where your virtual environment is located. 2. Run the following command to activate it: ```bash .\\Scripts\activate ``` ## How do I install the necessary packages? 1. Ensure that your virtual environment is activated. 2. Navigate to the folder containing the `requirements.txt` file. 3. Run the following command: ```bash pip install -r requirements.txt ``` ## How do I run Serper and GoT APIs for Explainability? Use GPT-4 or earlier versions to obtain explanations. ## How do I run the Uncertainty API for Explainability? Use GPT-4 or earlier versions to obtain explanations. ## How do I set up telemetry? 1. Set `tel_Flag` to `True` in the `.env` file of the respective repository. 2. Follow the setup instructions in **responsible-ai-telemetry**. 3. Use the following API endpoint from the Swagger UI setup and assign it to the `telemetry_url` variable in the `.env` file: ```text /rai/v1/telemetry/errorloggingtelemetryapi ``` ## /Features and Endpoints.docx Binary file available at https://raw.githubusercontent.com/Infosys/Infosys-Responsible-AI-Toolkit/refs/heads/main/Features and Endpoints.docx ## /LICENSE.md The MIT License (MIT) Copyright 2024-2025 Infosys Ltd. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ## /README.md # **Infosys-Responsible-AI-Toolkit** The Infosys Responsible AI toolkit provides a set of APIs to integrate safety,security, privacy, explainability, fairness, and hallucination detection into AI solutions, ensuring trustworthiness and transparency. ### Repositories and Installation Instructions The following table lists the modules of the Infosys Responsible AI Toolkit. Installation instructions for each module can be found in the corresponding README file within the module's directory. | # | Module | Functionalities | Repository name(s) | | --- | --- | --- | ---- | | 1 | ModerationLayer APIs
(Comprehensive suite of Safety, Privacy, Explainability, Fairness and Hallucination tenets) | To regulate the content of prompts and responses generated by LLMs | [responsible-ai-moderationlayer](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/tree/master/responsible-ai-moderationlayer),
[responsible-ai-moderationModel](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/tree/master/responsible-ai-moderationmodel) | | 2 | Explainability APIs** | Get Explainability to LLM responses,
Global and local explainability for Regression, Classification and Timeseries Models | [responsible-ai-llm-explain](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/tree/master/responsible-ai-llm-explain),
[responsible-ai-explainability](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/tree/master/responsible-ai-explainability),
[Model Details](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/tree/master/responsible-ai-model-detail),
[Reporting](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/tree/master/responsible-ai-reporting-tool) | | 3 | Fairness & Bias API | Check Fairness and detect Biases associated with LLM prompts and responses and also for traditional ML models | [responsible-ai-fairness](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/tree/master/responsible-ai-fairness) | | 4 | Hallucination API | Detect and quantify Hallucination in LLM responses under RAG scenarios | [responsible-ai-hallucination](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/tree/master/responsible-ai-Hallucination) | | 5 | Privacy API | Detect and anonymize or encrypt or highlight PII information in prompts for LLMs or in its responses | [responsible-ai-privacy](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/tree/master/responsible-ai-privacy) | | 6 | Safety API | Detects and anonymize toxic and profane text associated with LLMs | [responsible-ai-safety](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/tree/master/responsible-ai-safety) | | 7 | Security API | For different types of security attacks and defenses on tabular and image data, prompt injection and jailbreak checks | [responsible-ai-security](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/tree/master/responsible-ai-security) | ** Endpoints for explainability are located in both the explainability and moderation layer repositories. Refer to the README files in these repositories for more details on specific features. Please refer [Features and Endpoints](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/blob/master/Features%20and%20Endpoints.docx) document for more details on endpoints and their usage. ### Modules for the Responsible AI Toolkit Interface The Responsible AI toolkit provides a user-friendly interface for seamless experimentation and alignment with various Responsible AI principles. The following APIs are required to activate and utilize the toolkit's UI. To access the toolkit through the interface, refer to the README files associated with the listed repositories. | # | Module | Functionalities | Repository name(s) | | --- | --- | --- | ---- | | 1 | MFE | An Angular micro frontend app serves as a user interface where users can easily interact with and consume various backend endpoints through independently developed, modular components. | [responsible-ai-mfe](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/tree/master/responsible-ai-mfe) | | 2 | SHELL | A shell application in a micro frontend architecture acts as the central hub, orchestrating and loading independent frontend modules. It provides a unified user interface where users can interact with different micro frontends, consume backend endpoints.| [responsible-ai-shell](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/tree/master/responsible-ai-shell) | | 3 | Backend | A Python backend module focused on registration and authentication handles user account management, including user registration, login, password validation, and session management.| [responsible-ai-backend](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/tree/master/responsible-ai-backend) | | 4 | Telemetry | A python backend module defining the various tenets structure for ingestion of the API's data into Elasticsearch indexes. It provided customizable input validation and insertion of data coming from tenets into elasticsearch, which can be further displayed using kibana.| [responsible-ai-telemetry](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/tree/master/responsible-ai-telemetry) | | 5 | File Storage | Python module that provides versatile APIs for seamless integration across multiple microservices, enabling efficient file management with Azure Blob Storage. It supports key operations such as file upload, retrieval, and updates, offering a robust solution for handling files in Azure Blob Storage.| [responsible-ai-file-storage](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/tree/master/responsible-ai-file-storage) | | 6 | Benchmarking | Displays stats related to benchmarking large language models (LLMs) across various categories such as fairness, privacy, truthfulness and ethics. It helps evaluate and compare LLM performance in these critical areas.| [responsible-ai-llm-benchmarking](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/tree/master/responsible-ai-llm-benchmarking)| For technical details and usage instructions on the Infosys Responsible AI toolkit's features, please refer to the [documentation](https://infosys.github.io/Infosys-Responsible-AI-Toolkit/). ## Toolkit features at a glance ### Generative AI Models | Safety, Security & Privacy | Model Transparency | Text Quality | Linguistic Quality | |:--- |:--- |:---- |:---- | |* Prompt Injection Score
* Jailbreak Score
* Privacy check
* Profanity check
* Restricted Topic check
* Toxicity check
* Refusal check |* Sentiment check
* Fairness & Bias check
* Hallucination Score
* Explainability Methods:
- Thread of Thoughts (ThoT)
- Chain of Thoughts (CoT)
- Graph of Thoughts (GoT)
- Chain of Verification (CoVe)
* Token Importance |* Invisible Text, Gibberish checks
* Ban Code check
* Completeness check
* Conciseness check
* Text Quality check
* Text Relevance check
* Uncertainty Score
* Coherence Score |* Language Critique Coherence
* Language Critique Fluency
* Language Critique Grammar
* Language Critique Politeness| ### Machine Learning Models | Security | Fairness | Explainability | |:--- |:--- |:---- | |* Simulate Adverserial Attacks
* Recommend Defense Mechanisms|* Bias Detection Methods:
- Statistical Parity Difference
- Disparate Impact Ratio
- Four Fifth's Rule
- Cohen's D
* Mitigation Methods:
- Equalized Odds
- Re-weighing|* Global Explainability using SHAP
* Local Explainability using LIME | ## Upcoming Features * Logic of Thoughts(LoT) for enhanced Explainability * Fairness Auditing for continuous monitoring and mitigation of Biases * Red Teaming to identify and mitigate AI model security threats * Multi-lingual support for Prompt injection and Jailbreak in Moderation models * Multilingual Feature support to privacy & safety modules Note: These API-based guardrails are optimized for Azure OpenAI. Users employing alternative LLMs should make the necessary client configuration adjustments. For Azure OpenAI api subscription, follow instructions provided in the [Microsoft Azure website](https://azure.microsoft.com/en-us/pricing/purchase-options/azure-account?icid=ai-services&azure-portal=true). Please check out the [contribution](https://github.com/Infosys/Infosys-Responsible-AI-Toolkit/blob/master/CONTRIBUTING.md) page to share your feedback or suggest improvements. Have more questions? Connect with us at Infosysraitoolkit@infosys.com. ## /Release Notice.docx Binary file available at https://raw.githubusercontent.com/Infosys/Infosys-Responsible-AI-Toolkit/refs/heads/main/Release Notice.docx ## /github-pages/content/Explainability.html ```html path="/github-pages/content/Explainability.html"

Explainability

Introduction

Explainability in AI, including Machine Learning and Large Language Models, is about understanding the logic behind a model's decisions. By revealing the reasoning process, it builds trust and improves model performance. It helps us spot biases, fix mistakes, and align models with our goals. Ultimately, explainability makes AI more reliable and trustworthy. Infosys Responsible AI toolkit provides a range of explainability techniques to enhance transparency and understanding of AI model decisions.

Focus area

For Generative AI: (Unstructured Text)

  • Enhance LLM response through sequential reasoning techniques (e.g., Thread of thoughts, Graph of thoughts)

  • Quantify truthfulness of LLM response using standard metrics (e.g., F1 Score, Relevance Score)

  • Verify the LLM response with factual checks and derive the best explanation (e.g., Chain of Verification, Internet Search)

For Predictive AI: (Tabular Data)

  • Provide global explainability to Machine Learning (ML) and Deep Learning (DL) methods using various variable importance techniques (e.g., SHAP, Permutation Importance)

  • Enable local explainability for individual predictions made by any ML/DL models using model agnostic variable explanation methods (e.g., LIME, Anchor)

For Image Data:

  • Evaluate the explainability of AI-generated images using CLIP (Contrastive Language-Image Pre-training) scores to assess creativity, alignment of image with the prompt, and how creative the image is, using Holistic Evaluation of Text-to-Image Models (HEIM) framework.

Infosys Responsible AI toolkit - Snapshot of Explainability methods

Available Explainability methods

The Infosys Responsible AI toolkit offers a range of explainability techniques applicable to multiple AI models and associated data types. This table provides an overview of the currently available methods.

Data Type
Model Type
Explainability (reasoning)
Verification (Faithfulness/Factuality)
Metrics / Charts
Other details

Unstructured Text

LLM

Thread of Thoughts

Graph of Thoughts

ReRead Reasoning

Logic of Thought

Chain of Verification

Internet Search

Token Importance Values / Charts & Uncertainty Quantification

Sentiment

LLM Response

Keywords

Sentiment Categorization

Retrieval Augment Generation (RAG)

Thread of Thoughts

Chain of Verification

Supports multiple files and formats (txt, pdf, csv)

Tabular (Structured) Data

Binary Classifier

Multi-class Classifier

Time Series

Global Explainability using Shap

Local Explainability using LIME

Variable Importance Charts

LIME Charts

Image

Image Generation (Stable Diffusion 1.5**)

CLIP (Contrastive Language-Image Pre-training) scores

Aesthetic score, Alignment & Knowledge Score, Originality, Gender Bias values and charts

** subject to enhancement

Blueprint

List of explainability methods and status of their implementation
image-20240819-073625.png

Infosys Responsible AI toolkit - Explainability as a tenet

Unstructured Text

Generative AI - LLM response

Large Language Models (LLMs) tap into vast databases and knowledge graphs to generate content aligned with user intent, providing highly relevant responses. However, the risk of hallucinated or inaccurate answers persists. To address this, a robust validation mechanism is crucial to ensure LLM responses are faithful and accurate, leveraging the most recent databases and internet data. This feature provides explainability in three dimensions.

This feature employs a multi-step process to enhance LLM response reliability. Initially, prompt engineering techniques like Thread of Thoughts or Graph of Thoughts are utilized to assess response faithfulness. Subsequently, Chain of Verification is applied to refine the LLM output. Logic of Thoughts is utilized to explain along with logical expressions for better understanding. For independent verification, users can cross-reference information through embedded Internet Search* option. Finally, response quality is evaluated using Coherence and Uncertainty metrics, complemented by attention visualization (Token Importance charts) to pinpoint influential text segments.

*Internet search verification uses limited text, retrieved from the first URL of Google search results

How to?

Login to: RAI Portal

Navigate to: Generative AI (toggle button) → Explainability (radio button) → LLM (Drop down) and ensure ‘Internet Search’ option is untoggled.

image-20240821-002516.png

  1. Enter the required prompt for explainability

  2. Populates LLM response curated by Thread of thoughts

  3. Displays final response verified by Chain of Verification

image-20240821-003742.png

  1. Reasoning by Thread of thoughts

  2. Metrics to measure correctness of responses through thread of thoughts reasoning

    1. Uncertainty score quantifies the LLM's level of uncertainty when producing the response. 0 indicates absolute certainty, 1 indicates maximum uncertainty and any value between 0 and 1 represents varying degrees of confidence.

    2. Coherence refers to the logical and clear connection between the given prompt and LLM response. 1 indicates logical and clear response, 0 indicates unrelated response and any value between 0 & 1 indicates varying degree of coherence

  3. Explanation on metric values

  4. Recommended prompt changes, if user wanted to rephrase the prompt for better response

  5. Validation of LLM response through Chain of Verification

  6. Additional Explanation button to display explanation through Graph of Thoughts and Token Importance charts

image-20240821-005157.pngimage-20241217-082126.pngimage-20241217-082228.png

  1. Displays matrix with top 10 tokens and their importance

  2. The Token Importance Distribution Chart illustrates the significance of individual tokens by displaying the distribution of their associated impact scores. The chart's shape reveals the following insights:

    • Flat Distribution: Tokens have similar importance, with no clear standout

    • Left-Peaked Distribution: Tokens have low impact scores, indicating lesser importance

    • Right-Peaked Distribution: Tokens have high impact scores, signifying greater importance

  3. Displays importance of each token (considered top 10 tokens based on their importance for this chart)

  4. Explanation based on Graph of thoughts. The intensity meter displays a Consistency Score, gauging how well the final response from the graph of thoughts aligns with the LLM's initial response and the context provided in the prompt. Value towards extreme right is desirable for higher consistency.

  5. Reasoning by Reread with Thread of thoughts gives the detailed step by step reasoning.

  6. Reasoning by Logic of Thought (LoT) extracts propositions and logical expressions, extending them to generate expanded logical information from the input context. This generated logical information is then utilized as an additional augmentation to the input prompts, thereby enhancing the system's logical reasoning capabilities.

Note: When ‘Internet Search’ button is in untoggled state, final response is validated through Chain of Verification and Additional explanation available.

Validation of LLM response through ‘Internet Search’

Navigate to: Generative AI (toggle button) → Explainability (radio button) → LLM (Drop down) and ensure ‘Internet Search’ option is toggled.

image-20241219-055702.png

  1. Displays final response by cross validating facts generated by thread of thoughts against internet search results.

  2. List of facts used by thread of thoughts while reasoning the LLM response

  3. Explanation based on internet search results

  4. Judgement on the accuracy of LLM response. The validation process compares the LLM's thread of thoughts against internet search results.

    • No: Internet search results contradict the LLM's facts, indicating potential inaccuracies.

    • Yes: Internet search results support the LLM's facts, confirming their validity.

    • Unclear: Internet search results lack sufficient information to determine the accuracy of the LLM's response, requiring further investigation.

Note: When ‘Internet Search’ option is selected for validation, final response will be based on validation of facts generated by thread of thoughts against the internet search results from the first URL generated by Google search.

Techniques used

Prompt Engineering techniques: Thread of thoughts, Graph of thoughts, Chain of verification, Logic of thoughts

Attention visualization: Token importance charts

Internet Search: “SerperAPI” (Google search)

Generative AI - RAG scenario

Retrieval Augmentation Generation (RAG) is a powerful capability in Large Language Models (LLMs) that enhances text generation and summarization tasks by retrieving relevant information from specific databases or knowledge sources augmenting the LLMs input.

This feature in the toolkit leverages the RAG capability of LLMs and the thread of thoughts techniques to summarize text from uploaded files, providing more contextual answers based on the prompt. The summarized response is then further refined through a rigorous chain of verification, ensuring a more accurate and reliable answer. This feature supports multiple pdf, text and csv files.

How to?

Login to: RAI Portal

Navigate to: Generative AI (toggle button) → Explainability (radio button) → RAG (Drop down) and upload required file(s)

image-20240821-015800.png

  1. Enter prompt to retrieve information from local files

  2. Upload file(s) for text summarization or generation. pdf, text and csv files are supported

  3. Response generated by LLM through RAG and curated by thread of thoughts

  4. Final response validated through chain of verification

image-20240821-020129.png

  1. Displays explanation based on thread of thoughts after retrieving information from uploaded files along with sources used for information retrieval

  2. Shows steps used by Chain of verification for response refinement

Techniques used

Prompt Engineering techniques: thread of thoughts, chain of verification

Langchain

Sentiment Analysis

Sentiment Analysis feature of Explainability tenet uses a large language model to assess the sentiment of an input prompt, classifying it as positive, negative, or neutral. By identifying and highlighting pivotal keywords that influence the emotional tone, the model offers a deep understanding of the underlying sentiments. Additionally, it provides a clear explanation of its reasoning, justifying its final conclusion.

How to?

Login to: RAI Portal

Navigate to: Generative AI (toggle button) → Explainability (radio button) → Sentiment (Drop down) and enter the required prompt for Sentiment Analysis.

image-20240820-104228.png

  1. Specify the prompt for ‘Sentiment Analysis’

  2. Sentiment of the prompt populated as Negative or Positive or Neutral or Mixed

  3. LLM's explanation for the classified polarity

  4. Lists the words that contribute to sentiment assessment

Techniques used

Prompt Engineering techniques

Tabular Data

Regression Model

A regression model is a statistical model used to predict a continuous outcome variable based on one or more predictor variables. It establishes a relationship between these variables, allowing us to estimate or forecast the value of the outcome variable by given specific values of the predictor variables. However, this prediction is not completely explanatory to everyone. With the right techniques and tools, we can gain valuable insights into how these models work and make informed decisions based on their outputs.

Explainability in AI models ensures that the decision-making process is transparent and understandable. This includes providing clear and concise explanations for the factors considered and the data used to reach a conclusion. There are two primary types of explanations:

  • Global Explanation: This provides a broad overview of how the model functions across the entire dataset. It helps to understand the general patterns and relationships the model has learned.

  • Local Explanation: This focuses on specific predictions, explaining the factors that contributed to a particular outcome. It provides insights into the model's decision-making process for individual instances.

By providing both global and local explanations, AI models become more accountable, trustworthy, and easier to interpret.

How to?

Login to: RAI Portal

Navigate to:

Step-1: Home → Models → Data Form → Add Data → Fill required fields → Upload dataset → Submit

image-20240828-073604.png

  1. Navigate to the Models Section.

  2. Click on the Data Form tab.

  3. Click on the Add Data button.

image-20240828-100641.png

  1. Please fill in all the required fields.

  2. Upload the dataset file.

  3. Once you have uploaded the files and completed all the required fields, click on 'Submit.

Step-2: Models → Model Form → Add Model → Fill required fields → Upload model → Submit

image-20240828-112227.png

  1. Click on the Model Form tab.

  2. Click on the Add Model button.

image-20240828-110629.png

  1. Please fill in all the required fields.

  2. Upload the model file.

  3. Once you have uploaded the files and completed all the required fields, click on 'Submit.

Step-3: Home → Structured Text → Enable Structured Text (Toggle Button) → Add Data → Add Title → Select Dataset (Drop down) → Select Model (Drop down) → Select Type as Explainability → Click on arrow → Choose methods → Submit → Click on back arrow → Click on Generate Report → Download

image-20240828-111533.png

  1. From the Home tab, navigate to the Structured Text tab.

  2. Enable the Structured Text toggle button.

  3. Click on the 'Add Data' button.

image-20240828-113940.png

  1. Provide a title for the report.

  2. Select the dataset file from the dropdown menu

  3. Select the model file from the dropdown menu.

  4. Select type as Explainability.

  5. Click on the arrow icon.

image-20240828-113843.png

  1. Choose the methods from the dropdown menu. If you want an explanation for all the methods, select 'All'; otherwise, select the specific methods you want.

  2. Click on submit.

image-20240828-122223.png

  1. Click on Generate Report.

  2. Once the report generation is complete, click on the Download icon to download the report.

image-20241009-103626.pngimage-20241009-111711.pngimage-20241009-111827.pngimage-20241009-103316.png

  1. In the downloaded report, we can see local explanations that help us understand which features are contributing more to the prediction for a specific instance.

  2. In the global explanation, we can see which features are contributing more across the entire dataset.

Techniques used

Currently RandomForest Regressor model only is supported by this feature.

KERNEL SHAP - is used for Global explanation

LIME- is used for Local Explanation

Classification Model

A classification model is a machine learning algorithm used to predict categorical outcomes. Unlike regression models, which predict continuous values, classification models assign data points to discrete categories. However, this prediction is not completely explanatory to everyone. With the right techniques and tools, we can gain valuable insights into how these models work and make informed decisions based on their outputs.

Explainability in AI models ensures that the decision-making process is transparent and understandable. This includes providing clear and concise explanations for the factors considered and the data used to reach a conclusion. There are two primary types of explanations:

  • Global Explanation: This provides a broad overview of how the model functions across the entire dataset. It helps to understand the general patterns and relationships the model has learned.

  • Local Explanation: This focuses on specific predictions, explaining the factors that contributed to a particular outcome. It provides insights into the model's decision-making process for individual instances.

By providing both global and local explanations, AI models become more accountable, trustworthy, and easier to interpret.

How to?

Login to: RAI Portal

Navigate to:

Step-1: Home → Models → Data Form → Add Data → Fill required fields → Upload dataset → Submit

image-20240828-073604.png

  1. Navigate to the Models Section.

  2. Click on the Data Form tab.

  3. Click on the Add Data button.

image-20240828-115051.png

  1. Please fill in all the required fields.

  2. Upload the dataset file.

  3. Once you have uploaded the files and completed all the required fields, click on 'Submit.

Step-2: Models → Model Form → Add Model → Fill required fields → Upload model → Submit

image-20240828-112245.png

  1. Click on the Model Form tab.

  2. Click on the Add Model button.

image-20240828-115703.png

  1. Please fill in all the required fields.

  2. Upload the model file.

  3. Once you have uploaded the files and completed all the required fields, click on 'Submit.

Step-3: Home → Structure Text → Enable Structured Text (Toggle Button) → Add Data → Add Title → Select Dataset (Drop down) → Select Model (Drop down) → Select Type → Click on arrow → Choose methods → Submit → Click on back arrow → Click on Generate Report → Download

image-20240828-111549.png

  1. From the Home tab, navigate to the Structured Text tab.

  2. Enable the Structured Text toggle button.

  3. Click on the 'Add Data' button.

image-20240828-120125.png

  1. Provide a title for the report.

  2. Select the dataset file from the dropdown menu.

  3. Select the model file from the dropdown menu.

  4. Select type as Explainability.

  5. Click on the arrow icon.

image-20240828-113843.png

  1. Choose the methods from the dropdown menu. If you want an explanation for all the methods, select 'All'; otherwise, select the specific methods you want.

  2. Click on submit.

image-20240828-123330.png

  1. Click on Generate Report.

  2. Once the report generation is complete, click on the Download icon to download the report.

image-20241009-105538.pngimage-20241009-105801.pngimage-20241009-105416.png

  1. In the downloaded report, we can see local explanations that help us understand which features are contributing more to the prediction for a specific instance.

  2. In the global explanation, we can see which features are contributing more in the overall dataset.

Techniques used

Currently Random Forest Classifier, Gradient Boost Classifier and XgBoost models are supported by this feature.

KERNEL SHAP - is used for Global explanation

LIME- is used for Local Explanation

Time Series Model

A time series model is a statistical model used to analyze data points that are ordered in time. These models are used to understand, predict, and forecast future values based on historical patterns and trends. However, this prediction is not completely explanatory to everyone. With the right techniques and tools, we can gain valuable insights into how these models work and make informed decisions based on their outputs.

Explainability in AI models ensures that the decision-making process is transparent and understandable. This includes providing clear and concise explanations for the factors considered and the data used to reach a conclusion. For Time Series, this feature supports only Global Explanation.

  • Global Explanation: This provides a broad overview of how the model functions across the entire dataset. It helps to understand the general patterns and relationships the model has learned.

  • Local Explanation: This focuses on specific predictions, explaining the factors that contributed to a particular outcome. It provides insights into the model's decision-making process for individual instances.

By providing both global and local explanations, AI models become more accountable, trustworthy, and easier to interpret.

How to?

Login to: RAI Portal

Navigate to:

Step-1: Home → Models → Data Form → Add Data → Fill required fields → Upload dataset → Submit

image-20240828-073604.png

  1. Navigate to the Models Section.

  2. Click on the Data Form tab.

  3. Click on the Add Data button.

image-20240828-121211.png

  1. Please fill in all the required fields.

  2. Upload the dataset file.

  3. Once you have uploaded the files and completed all the required fields, click on 'Submit.

Step-2: Models → Model Form → Add Model → Fill required fields → Upload model → Submit

image-20240828-112313.png

  1. Click on the Model Form tab.

  2. Click on the Add Model button.

image-20240828-124123.png

  1. Please fill in all the required fields.

  2. Upload the model file.

  3. Once you have uploaded the files and completed all the required fields, click on 'Submit.

Step-3: Home → Structure Text → Enable Structured Text (Toggle Button) → Add Data → Add Title → Select Dataset (Drop down) → Select Model (Drop down) → Select Type → Click on arrow → Choose methods → Submit → Click on back arrow → Click on Generate Report → Download

image-20240828-111559.png

  1. From the Home tab, navigate to the Structured Text tab.

  2. Enable the Structured Text toggle button.

  3. Click on the 'Add Data' button.

image-20240828-124803.png

  1. Provide a title for the report.

  2. Select the dataset file from the dropdown menu.

  3. Select the model file from the dropdown menu.

  4. Select type as Explainability.

  5. Click on the arrow icon.

image-20240828-113843.png

  1. Choose the methods from the dropdown menu. If you want an explanation for all the methods, select 'All'; otherwise, select the specific methods you want.

  2. Click on submit.

image-20240828-123734.png

  1. Click on Generate Report.

  2. Once the report generation is complete, click on the Download icon to download the report.

image-20241009-110221.png image-20241009-110740.pngimage-20241009-111124.png image-20241009-111233.png

  1. In the global explanation, we can see which features are contributing more across the entire dataset.

  2. We can see local explanations that help us understand which features are contributing more to the prediction for a specific instance.

Techniques used

KernelExplainer

LimeTabular

Image Data

Image Explainability

When AI systems make critical decisions based on image data, such as in medical diagnosis or autonomous driving, it's crucial to understand why the system reached that conclusion. Image explanation can provide insights into the factors that influenced the AI's decision, increasing transparency and trust. Infosys Responsible AI provides Explanation for the Image that are generated by any AI solution as well as the image that needs to be generated by using the given prompt.

How to?

Login to: RAI Portal

Image Explanation:

Navigate to: Image → Traditional AI (toggle button) → Explainability (radio button) → Browse image file → Select options → Submit.

image-20240829-114654.png

  1. From Home navigate to Image tab.

  2. Locate the Traditional AI toggle button and enable it.

  3. Set the type to Explainability.

  4. Browse the image file on the system.

  5. Click the submit button to check for aesthetics, watermarks, and bias in the image.

image-20240829-120006.pngimage-20240829-120106.png

  1. Uploaded Image.

  2. By clicking the report button, you can view the aesthetics, watermarks, and bias in the image.

Image Generation

Navigate to: Image → Generative AI (toggle button) → Explainability (radio button) and enter the required prompt to generate the image.

image-20240829-051000.png

  1. From Home navigate to Image tab.

  2. Locate the Generative AI toggle button and enable it.

  3. Set the type to Explainability.

  4. Enter your prompt to generate the image.

  5. Click on the Generate button to produce the image based on your prompt

image-20240829-051504.png

  1. Image generated using the provided prompt.

Techniques used

Stable Diffusion

Holistic Evaluation of Text-To-Image Models

Object Detection Explainability

Explainable AI for Object Detection involves techniques and methods designed to understand how object detection models identify and classify objects in images. These approaches aim to make the model’s decision-making process transparent and trustworthy. It also helps in assessing the model’s robustness to variations in input, such as occlusions, lighting changes, or object orientations. Visualization tools such as heatmaps, saliency maps, or attention maps are often used to reveal the areas the model deemed significant and how they contributed to detection confidence. In addition to visual explanations, natural language explanations play a crucial role in making the model's insights accessible and interpretable for diverse users. These explanations are critical for validating, debugging, and building trust in object detection systems, particularly in high-stakes applications like autonomous driving, surveillance, and medical imaging.

Research Findings:

To generate natural language explanations for object detection-based models, we evaluated the following open-source multimodal large language models (LLMs) in CPU machine:

  • Llama-3.2-Vision

  • Llava-7B

  • Llava-Phi3

  • Moondream V2

  • NexaAI/OmniVision-968M

  • allenai/Molmo-7B-O-0924

  • allenai/Molmo-7B-D-0924

  • gpt-omni/mini-omni2

Observations:

Model

Model Size | Latency

Observations w.r.t Object Detection Task
1

Llama-3.2-Vision

6 GB | ~10 min

  1. The accuracy and depth of understanding objects within images are moderate.

  2. Overall performance, including contextual and semantic analysis, is moderate.
2

Llava-1.4-7B

4.1 GB | ~3 min

  1. The accuracy and depth of understanding objects within images are moderate.

  2. Overall performance, including contextual and semantic analysis, is suboptimal.
3

Llava-phi3

3 GB | <20 sec

  1. The accuracy and depth of understanding objects within images are limited.

  2. Overall performance, including contextual and semantic analysis, is suboptimal.
4

Moondream V2

4 GB | < 1min

  1. The accuracy and depth of understanding objects within images are limited.

  2. The model may generate inaccurate statements, and struggle to understand intricate or nuanced instructions.

  3. Overall performance, including contextual and semantic analysis, is suboptimal.

  4. The model may generate offensive, inappropriate, or hurtful content if it is prompted to do so.
5

NexaAI/OmniVision-968M

2 GB | ~10 min

  1. The accuracy and depth of understanding objects within images are less.

  2. Overall performance, including contextual and semantic analysis, is low.
6

allenai/Molmo-7B-O-0924

31 GB

Large size, leading to challenges in deployment and resource utilization.
7

allenai/Molmo-7B-D-0924

33 GB

Large size, leading to challenges in deployment and resource utilization.
8

gpt-omni/mini-omni2

3 GB

  1. Overall performance, including contextual and semantic analysis, is suboptimal.

  2. Requires CUDA based pytorch.

  1. Response Latency: These models exhibit significant delays in generating responses when prompted.

  2. Object Understanding: The accuracy and depth of understanding objects within images are limited.

  3. Performance Analysis: Overall performance, including contextual and semantic analysis, is suboptimal.

  4. Model Size: Some models have significantly large sizes, leading to challenges in deployment and resource utilization.

Insights:

The Llava-7B model demonstrated better results compared to other models, particularly with lower response latency and improved explanations. We plan to further evaluate its potential for Explainable AI applications in object detection tasks.

References:

  1. From Pixels to Words: Leveraging Explainability in Face Recognition through Interactive Natural Language Processing

  2. LlaVa: Large Language and Vision Assistant

Infosys Responsible AI toolkit - Explainability in Moderation Layer

Explainability in Moderation Layer

In FM Moderation Layer, we use the similar kind of Explainability features as discussed above and we are not limiting the available features to be default and additional explanation. Here the explanation will be displayed followed by various input and output moderation checks. if any of the moderation checks got failed the explanation and LLM response will not be displayed.

Here the Explainability feature employs a multi-step process to enhance LLM response reliability. Initially, prompt engineering techniques like Chain of Thoughts, Thread of Thoughts, Graph of Thoughts and Logic of Thought are utilized to assess response faithfulness. Subsequently, Chain of Verification is applied to refine the LLM output. For independent verification, users can cross-reference information through embedded Internet Search* option. Finally, response quality is evaluated using Coherence and Uncertainty metrics, complemented by attention visualization (Token Importance charts) to pinpoint influential text segments.

*Internet search verification uses limited text, retrieved from the first URL of Google search results

How to?

Login to: RAI Portal

Navigate to: HOME → Unstructured Text

  1. Turn on the Generative AI (toggle button)

  2. Select FM-Moderation (radio button)

  3. Provide the prompt for which Explanation is required

image-20240829-124656.png

Once you have obtained the results, navigate to the 'Response Comparison' tab.

  1. Displays final response verified by Chain of Verification.

  2. Populates LLM response curated by Thread of thoughts.

image-20240829-131941.png

  1. Reasoning by Chain of Verification.

  2. Reasoning by Thread of thoughts.

  3. Explanation based on Graph of thoughts. The intensity meter displays a Consistency Score, gauging how well the final response from the graph of thoughts aligns with the LLM's initial response and the context provided in the prompt. Value towards extreme right is desirable for higher consistency.

  4. Validation of LLM response through Chain of Verification.

image-20240829-132416.pngimage-20240829-132733.pngimage-20240829-133708.pngimage-20240829-134042.png

  1. Displays matrix with top 10 tokens and their importance

  2. The Token Importance Distribution Chart illustrates the significance of individual tokens by displaying the distribution of their associated impact scores. The chart's shape reveals the following insights:

    • Flat Distribution: Tokens have similar importance, with no clear standout

    • Left-Peaked Distribution: Tokens have low impact scores, indicating lesser importance

    • Right-Peaked Distribution: Tokens have high impact scores, signifying greater importance

  3. Displays importance of each token (considered top 10 tokens based on their importance for this chart).

image-20240829-134638.png

  1. Metrics to measure correctness of responses through thread of thoughts reasoning

    1. Uncertainty score quantifies the LLM's level of uncertainty when producing the response. 0 indicates absolute certainty, 1 indicates maximum uncertainty and any value between 0 and 1 represents varying degrees of confidence.

    2. Coherence refers to the logical and clear connection between the given prompt and LLM response. 1 indicates logical and clear response, 0 indicates unrelated response and any value between 0 & 1 indicates varying degree of coherence.

image-20240829-134753.png

Validation of LLM response through ‘Internet Search’

Navigate to: Generative AI (toggle button) → FM-Moderation (radio button) → Enter the prompt and ensure ‘Internet Search’ option is toggled.

image-20240821-011242.png

  1. Displays final response by cross validating facts generated by thread of thoughts against internet search results.

  2. List of facts used by thread of thoughts while reasoning the LLM response

  3. Explanation based on internet search results

  4. Judgement on the accuracy of LLM response. The validation process compares the LLM's thread of thoughts against internet search results.

    • No: Internet search results contradict the LLM's facts, indicating potential inaccuracies.

    • Yes: Internet search results support the LLM's facts, confirming their validity.

    • Unclear: Internet search results lack sufficient information to determine the accuracy of the LLM's response, requiring further investigation.

Note: When ‘Internet Search’ option is selected for validation, final response will be based on validation of facts generated by thread of thoughts against the internet search results from the first URL generated by Google search.

Techniques used

Prompt Engineering techniques: Chain of thoughts,Thread of thoughts, Graph of thoughts, Chain of verification

Attention visualization: Token importance charts

Internet Search: “SerperAPI” (Google search)



``` The content has been capped at 50000 tokens, and files over NaN bytes have been omitted. The user could consider applying other filters to refine the result. The better and more specific the context, the better the LLM can follow instructions. If the context seems verbose, the user can refine the filter using uithub. Thank you for using https://uithub.com - Perfect LLM context for any GitHub repo.