test-spaces

.gitattributes

@@ -0,0 +1,2 @@
+models/** filter=lfs diff=lfs merge=lfs -text
+prompt-sentences-main/** filter=lfs diff=lfs merge=lfs -text

CHANGELOG.md

@@ -0,0 +1,13 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+## [Unreleased]
+
+## [0.0.1] - 2019-02-15
+
+### Added
+- Added a changelog
+
+[unreleased]: https://github.com/ibm/repo-template/compare/v0.0.1...HEAD
+[0.0.1]: https://github.com/ibm/repo-template/releases/tag/v0.0.1

CODE_OF_CONDUCT.md

@@ -0,0 +1,76 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+In the interest of fostering an open and welcoming environment, we as
+contributors and maintainers pledge to making participation in our project and
+our community a harassment-free experience for everyone, regardless of age, body
+size, disability, ethnicity, sex characteristics, gender identity and expression,
+level of experience, education, socio-economic status, nationality, personal
+appearance, race, religion, or sexual identity and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to creating a positive environment
+include:
+
+* Using welcoming and inclusive language
+* Being respectful of differing viewpoints and experiences
+* Gracefully accepting constructive criticism
+* Focusing on what is best for the community
+* Showing empathy towards other community members
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery and unwelcome sexual attention or
+ advances
+* Trolling, insulting/derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or electronic
+ address, without explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+ professional setting
+
+## Our Responsibilities
+
+Project maintainers are responsible for clarifying the standards of acceptable
+behavior and are expected to take appropriate and fair corrective action in
+response to any instances of unacceptable behavior.
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+## Scope
+
+This Code of Conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community. Examples of
+representing a project or community include using an official project e-mail
+address, posting via an official social media account, or acting as an appointed
+representative at an online or offline event. Representation of a project may be
+further defined and clarified by project maintainers.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting the [project team](./MAINTAINERS.md). All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. The project team is
+obligated to maintain confidentiality with regard to the reporter of an incident.
+Further details of specific enforcement policies may be posted separately.
+
+Project maintainers who do not follow or enforce the Code of Conduct in good
+faith may face temporary or permanent repercussions as determined by other
+members of the project's leadership.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
+available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
+
+[homepage]: https://www.contributor-covenant.org
+
+For answers to common questions about this code of conduct, see
+https://www.contributor-covenant.org/faq

CONTRIBUTING.md

@@ -0,0 +1,84 @@
+## Contributing In General
+Our project welcomes external contributions. If you have an itch, please feel
+free to scratch it.
+
+To contribute code or documentation, please submit a **FIXME** [pull request](https://github.com/ibm/repo-template/pulls).
+
+A good way to familiarize yourself with the codebase and contribution process is
+to look for and tackle low-hanging fruit in the **FIXME** [issue tracker](https://github.com/ibm/repo-template/issues).
+Before embarking on a more ambitious contribution, please quickly [get in touch](#communication) with us.
+
+**Note: We appreciate your effort, and want to avoid a situation where a contribution
+requires extensive rework (by you or by us), sits in backlog for a long time, or
+cannot be accepted at all!**
+
+### Proposing new features
+
+If you would like to implement a new feature, please **FIXME** [raise an issue](https://github.com/ibm/repo-template/issues)
+before sending a pull request so the feature can be discussed. This is to avoid
+you wasting your valuable time working on a feature that the project developers
+are not interested in accepting into the code base.
+
+### Fixing bugs
+
+If you would like to fix a bug, please **FIXME** [raise an issue](https://github.com/ibm/repo-template/issues) before sending a
+pull request so it can be tracked.
+
+### Merge approval
+
+The project maintainers use LGTM (Looks Good To Me) in comments on the code
+review to indicate acceptance. A change requires LGTMs from two of the
+maintainers of each component affected.
+
+For a list of the maintainers, see the [MAINTAINERS.md](MAINTAINERS.md) page.
+
+## Legal
+
+Each source file must include a license header for the Apache
+Software License 2.0. Using the SPDX format is the simplest approach.
+e.g.
+
+```
+/*
+Copyright <holder> All Rights Reserved.
+
+SPDX-License-Identifier: Apache-2.0
+*/
+```
+
+We have tried to make it as easy as possible to make contributions. This
+applies to how we handle the legal aspects of contribution. We use the
+same approach - the [Developer's Certificate of Origin 1.1 (DCO)](https://github.com/hyperledger/fabric/blob/master/docs/source/DCO1.1.txt) - that the Linux® Kernel [community](https://elinux.org/Developer_Certificate_Of_Origin)
+uses to manage code contributions.
+
+We simply ask that when submitting a patch for review, the developer
+must include a sign-off statement in the commit message.
+
+Here is an example Signed-off-by line, which indicates that the
+submitter accepts the DCO:
+
+```
+Signed-off-by: John Doe <john.doe@example.com>
+```
+
+You can include this automatically when you commit a change to your
+local git repository using the following command:
+
+```
+git commit -s
+```
+
+## Communication
+**FIXME** Please feel free to connect with us on our [Slack channel](link).
+
+## Setup
+**FIXME** Please add any special setup instructions for your project to help the developer
+become productive quickly.
+
+## Testing
+**FIXME** Please provide information that helps the developer test any changes they make
+before submitting.
+
+## Coding style guidelines
+**FIXME** Optional, but recommended: please share any specific style guidelines you might
+have for your project.

Dockerfile

@@ -0,0 +1,14 @@
+FROM python:3.9
+
+RUN useradd -m -u 1000 user
+USER user
+ENV PATH="/home/user/.local/bin:$PATH"
+
+WORKDIR /app
+
+COPY --chown=user ./requirements.txt requirements.txt
+RUN pip install --no-cache-dir --upgrade -r requirements.txt
+
+COPY --chown=user . /app
+
+CMD ["python", "app.py", "--host", "0.0.0.0", "--port", "7860"]

LICENSE

@@ -0,0 +1,202 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "{}"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright {yyyy} {name of copyright owner}
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+

MAINTAINERS.md

@@ -0,0 +1,9 @@
+# MAINTAINERS
+
+Mo McElaney - mmcelaney@us.ibm.com
+
+JJ Asghar - jja@ibm.com
+
+Joe Sepi - joesepi@ibm.com
+
+Brad Topol - btopol@us.ibm.com

README.md

@@ -0,0 +1,10 @@
+---
+title: Resapi
+emoji: 🐢
+colorFrom: gray
+colorTo: red
+sdk: docker
+pinned: false
+---
+
+Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

SECURITY.md

@@ -0,0 +1,17 @@
+# Security Policy
+
+## Supported Versions
+
+Use this section to tell people about which versions of your project are
+currently being supported with security updates.
+
+| Version | Supported          |
+| ------- | ------------------ |
+| 5.1.x   | :white_check_mark: |
+| 5.0.x   | :x:                |
+| 4.0.x   | :white_check_mark: |
+| < 4.0   | :x:                |
+
+## Reporting a Vulnerability
+
+To report a security issue, please email $VMTalias with a description of the issue, the steps you took to create the issue, affected versions, and if known, mitigations for the issue. Our vulnerability management team will acknowledge receiving your email within 3 working days. This project follows a 90 day disclosure timeline.

app.py

@@ -0,0 +1,127 @@
+#!/usr/bin/env python
+# coding: utf-8
+
+# Copyright 2021, IBM Corporation.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+Flask API app and routes.
+"""
+
+__author__ = "Vagner Santana, Melina Alberio, Cassia Sanctos and Tiago Machado"
+__copyright__ = "IBM Corporation 2024"
+__credits__ = ["Vagner Santana, Melina Alberio, Cassia Sanctos, Tiago Machado"]
+__license__ = "Apache 2.0"
+__version__ = "0.0.1"
+
+
+from flask import Flask, request, jsonify
+from flask_cors import CORS, cross_origin
+from flask_restful import Resource, Api, reqparse
+import control.recommendation_handler as recommendation_handler
+from helpers import get_credentials, authenticate_api, save_model
+import config as cfg
+import logging
+import uuid
+import json
+import os
+
+app = Flask(__name__)
+
+# configure logging
+logging.basicConfig(
+    filename='app.log',  # Log file name
+    level=logging.INFO,  # Log level (INFO, DEBUG, WARNING, ERROR, CRITICAL)
+    format='%(asctime)s - %(levelname)s - %(message)s'  # Log message format
+)
+
+# access the app's logger
+logger = app.logger
+# create user id
+id = str(uuid.uuid4())
+
+# swagger configs
+app.register_blueprint(cfg.SWAGGER_BLUEPRINT, url_prefix = cfg.SWAGGER_URL)
+FRONT_LOG_FILE = 'front_log.json'
+
+
+@app.route("/")
+def index():
+    user_ip = request.remote_addr
+    logger.info(f'USER {user_ip} - ID {id} - started the app')
+    return "Ready!"
+
+@app.route("/recommend", methods=['GET'])
+@cross_origin()
+def recommend():
+    user_ip = request.remote_addr
+    hf_token, hf_url = get_credentials.get_credentials()
+    api_url, headers = authenticate_api.authenticate_api(hf_token, hf_url)
+    prompt_json = recommendation_handler.populate_json()
+    args = request.args
+    prompt = args.get("prompt")
+    recommendation_json = recommendation_handler.recommend_prompt(prompt, prompt_json,
+                                                                  api_url, headers)
+    logger.info(f'USER - {user_ip} - ID {id} - accessed recommend route')
+    logger.info(f'RECOMMEND ROUTE - request: {prompt} response: {recommendation_json}')
+    return recommendation_json
+
+@app.route("/get_thresholds", methods=['GET'])
+@cross_origin()
+def get_thresholds():
+    hf_token, hf_url = get_credentials.get_credentials()
+    api_url, headers = authenticate_api.authenticate_api(hf_token, hf_url)
+    prompt_json = recommendation_handler.populate_json()
+    model_id = 'sentence-transformers/all-minilm-l6-v2'
+    args = request.args
+    #print("args list = ", args)
+    prompt = args.get("prompt")
+    thresholds_json = recommendation_handler.get_thresholds(prompt, prompt_json, api_url,
+                                                            headers, model_id)
+    return thresholds_json
+
+@app.route("/recommend_local", methods=['GET'])
+@cross_origin()
+def recommend_local():
+    model_id, model_path = save_model.save_model()
+    prompt_json = recommendation_handler.populate_json()
+    args = request.args
+    print("args list = ", args)
+    prompt = args.get("prompt")
+    local_recommendation_json = recommendation_handler.recommend_local(prompt, prompt_json,
+                                                                       model_id, model_path)
+    return local_recommendation_json
+
+@app.route("/log", methods=['POST'])
+@cross_origin()
+def log():
+    f_path = 'static/demo/log/'
+    new_data = request.get_json()
+
+    try:
+        with open(f_path+FRONT_LOG_FILE, 'r') as f:
+            existing_data = json.load(f)
+    except FileNotFoundError:
+        existing_data = []
+
+    existing_data.update(new_data)
+
+    #log_data = request.json
+    with open(f_path+FRONT_LOG_FILE, 'w') as f:
+        json.dump(existing_data, f)
+    return jsonify({'message': 'Data added successfully', 'data': existing_data}), 201
+
+if __name__=='__main__':
+    debug_mode = os.getenv('FLASK_DEBUG', 'False').lower() in ['true', '1', 't']
+    app.run(host='0.0.0.0', port='8080', debug=debug_mode)

config.py

@@ -0,0 +1,38 @@
+#!/usr/bin/env python
+# coding: utf-8
+
+# Copyright 2021, IBM Corporation. 
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+Swagger configuration.
+"""
+
+__author__ = "Vagner Santana, Melina Alberio, Cassia Sanctos and Tiago Machado"
+__copyright__ = "IBM Corporation 2024"
+__credits__ = ["Vagner Santana, Melina Alberio, Cassia Sanctos, Tiago Machado"]
+__license__ = "Apache 2.0"
+__version__ = "0.0.1"
+
+from flask_swagger_ui import get_swaggerui_blueprint
+
+SWAGGER_URL = '/swagger'
+API_URL = '/static/swagger.json'
+SWAGGER_BLUEPRINT = get_swaggerui_blueprint(
+    SWAGGER_URL,
+    API_URL,
+    config={
+        'app_name': "Prompt Recommendation API"
+    }
+)

control/recommendation_handler.py

@@ -0,0 +1,472 @@
+#!/usr/bin/env python
+# coding: utf-8
+
+# Copyright 2021, IBM Corporation.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+Python lib to recommend prompts.
+"""
+
+__author__ = "Vagner Santana, Melina Alberio, Cassia Sanctos and Tiago Machado"
+__copyright__ = "IBM Corporation 2024"
+__credits__ = ["Vagner Santana, Melina Alberio, Cassia Sanctos, Tiago Machado"]
+__license__ = "Apache 2.0"
+__version__ = "0.0.1"
+
+import requests
+import json
+import math
+import re
+import warnings
+import pandas as pd
+import numpy as np
+from sklearn.metrics.pairwise import cosine_similarity
+import os
+#os.environ['TRANSFORMERS_CACHE'] ="./models/allmini/cache"
+import os.path
+from sentence_transformers import SentenceTransformer
+from umap import UMAP
+import tensorflow as tf
+from umap.parametric_umap import ParametricUMAP, load_ParametricUMAP
+from sentence_transformers import SentenceTransformer
+
+def populate_json(json_file_path = './prompt-sentences-main/prompt_sentences-all-minilm-l6-v2.json',
+                    existing_json_populated_file_path = './prompt-sentences-main/prompt_sentences-all-minilm-l6-v2.json'):
+    """
+    Function that receives a default json file with
+    empty embeddings and checks whether there is a
+    partially populated json file.
+
+    Args:
+        json_file_path: Path to json default file with
+        empty embeddings.
+        existing_json_populated_file_path: Path to partially
+        populated json file.
+
+    Returns:
+        A json.
+
+    Raises:
+        Exception when json file can't be loaded.
+    """
+    json_file = json_file_path
+    if(os.path.isfile(existing_json_populated_file_path)):
+        json_file = existing_json_populated_file_path
+    try:
+        prompt_json = json.load(open(json_file))
+        json_error = None
+        return prompt_json, json_error
+    except Exception as e:
+        json_error = e
+        print(f'Error when loading sentences json file: {json_error}')
+        prompt_json = None
+        return prompt_json, json_error
+
+def query(texts, api_url, headers):
+    """
+    Function that requests embeddings for a given sentence.
+
+    Args:
+        texts: The sentence or entered prompt text.
+        api_url: API url for HF request.
+        headers: Content headers for HF request.
+
+    Returns:
+        A json with the sentence embeddings.
+
+    Raises:
+        Warning: Warns about sentences that have more
+        than 256 words.
+    """
+    for t in texts:
+        n_words = len(re.split(r"\s+", t))
+        if(n_words > 256):
+            # warning in case of prompts longer than 256 words
+            warnings.warn("Warning: Sentence provided is longer than 256 words. Model all-MiniLM-L6-v2 expects sentences up to 256 words.")
+            warnings.warn("Word count:{}".format(n_words))
+    if('sentence-transformers/all-MiniLM-L6-v2' in api_url):
+        model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
+        out = model.encode(texts).tolist()
+    else:
+        response = requests.post(api_url, headers=headers, json={"inputs": texts, "options":{"wait_for_model":True}})
+        out = response.json()
+    return out
+
+def split_into_sentences(prompt):
+    """
+    Function that splits the input text into sentences based
+    on punctuation (.!?). The regular expression pattern
+    '(?<=[.!?]) +' ensures that we split after a sentence-ending
+    punctuation followed by one or more spaces.
+
+    Args:
+        prompt: The entered prompt text.
+
+    Returns:
+        A list of extracted sentences.
+
+    Raises:
+        Nothing.
+    """
+    sentences = re.split(r'(?<=[.!?]) +', prompt)
+    return sentences
+
+
+def get_similarity(embedding1, embedding2):
+    """
+    Function that returns cosine similarity between
+    two embeddings.
+
+    Args:
+        embedding1: first embedding.
+        embedding2: second embedding.
+
+    Returns:
+        The similarity value.
+
+    Raises:
+        Nothing.
+    """
+    v1 = np.array( embedding1 ).reshape( 1, -1 )
+    v2 = np.array( embedding2 ).reshape( 1, -1 )
+    similarity = cosine_similarity( v1, v2 )
+    return similarity[0, 0]
+
+def get_distance(embedding1, embedding2):
+    """
+    Function that returns euclidean distance between
+    two embeddings.
+
+    Args:
+        embedding1: first embedding.
+        embedding2: second embedding.
+
+    Returns:
+        The euclidean distance value.
+
+    Raises:
+        Nothing.
+    """
+    total = 0
+    if(len(embedding1) != len(embedding2)):
+        return math.inf
+    for i, obj in enumerate(embedding1):
+        total += math.pow(embedding2[0][i] - embedding1[0][i], 2)
+    return(math.sqrt(total))
+
+def sort_by_similarity(e):
+    """
+    Function that sorts by similarity.
+
+    Args:
+        e:
+
+    Returns:
+        The sorted similarity value.
+
+    Raises:
+        Nothing.
+    """
+    return e['similarity']
+
+def recommend_prompt(prompt, prompt_json, api_url, headers, add_lower_threshold = 0.3,
+                     add_upper_threshold = 0.5, remove_lower_threshold = 0.1,
+                     remove_upper_threshold = 0.5, model_id = 'sentence-transformers/all-minilm-l6-v2'):
+    """
+    Function that recommends prompts additions or removals.
+
+    Args:
+        prompt: The entered prompt text.
+        prompt_json: Json file populated with embeddings.
+        api_url: API url for HF request.
+        headers: Content headers for HF request.
+        add_lower_threshold: Lower threshold for sentence addition,
+        the default value is 0.3.
+        add_upper_threshold: Upper threshold for sentence addition,
+        the default value is 0.5.
+        remove_lower_threshold: Lower threshold for sentence removal,
+        the default value is 0.3.
+        remove_upper_threshold: Upper threshold for sentence removal,
+        the default value is 0.5.
+        model_id: Id of the model, the default value is all-minilm-l6-v2 movel.
+
+    Returns:
+        Prompt values to add or remove.
+
+    Raises:
+        Nothing.
+    """
+    if(model_id == 'baai/bge-large-en-v1.5' ):
+        json_file = './prompt-sentences-main/prompt_sentences-bge-large-en-v1.5.json'
+        umap_model = load_ParametricUMAP('./models/umap/BAAI/bge-large-en-v1.5/')
+    elif(model_id == 'intfloat/multilingual-e5-large'):
+        json_file = './prompt-sentences-main/prompt_sentences-multilingual-e5-large.json'
+        umap_model = load_ParametricUMAP('./models/umap/intfloat/multilingual-e5-large/')
+    else: # fall back to all-minilm as default
+        json_file = './prompt-sentences-main/prompt_sentences-all-minilm-l6-v2.json'
+        umap_model = load_ParametricUMAP('./models/umap/sentence-transformers/all-MiniLM-L6-v2/')
+
+    prompt_json = json.load(open(json_file))
+
+    # Output initialization
+    out, out['input'], out['add'], out['remove'] = {}, {}, {}, {}
+    input_items, items_to_add, items_to_remove = [], [], []
+
+    # Spliting prompt into sentences
+    input_sentences = split_into_sentences(prompt)
+
+    # TODO: Request embeddings for input an d store in a input_embeddingS
+
+    # Recommendation of values to add to the current prompt
+    # Using only the last sentence for the add recommendation
+    input_embedding = query(input_sentences[-1], api_url, headers)
+    for v in prompt_json['positive_values']:
+        # Dealing with values without prompts and makinig sure they have the same dimensions
+        if(len(v['centroid']) == len(input_embedding)):
+            if(get_similarity(pd.DataFrame(input_embedding), pd.DataFrame(v['centroid'])) > add_lower_threshold):
+                closer_prompt = -1
+                for p in v['prompts']:
+                    d_prompt = get_similarity(pd.DataFrame(input_embedding), pd.DataFrame(p['embedding']))
+                    # The sentence_threshold is being used as a ceiling meaning that for high similarities the sentence/value might already be presente in the prompt
+                    # So, we don't want to recommend adding something that is already there
+                    if(d_prompt > closer_prompt and d_prompt > add_lower_threshold and d_prompt < add_upper_threshold):
+                        closer_prompt = d_prompt
+                        items_to_add.append({
+                        'value': v['label'],
+                        'prompt': p['text'],
+                        'similarity': d_prompt,
+                        'x': p['x'],
+                        'y': p['y']})
+                out['add'] = items_to_add
+
+    # Recommendation of values to remove from the current prompt
+    i = 0
+
+    # Recommendation of values to remove from the current prompt
+    for sentence in input_sentences:
+        input_embedding = query(sentence, api_url, headers) # remote
+        # Obtaining XY coords for input sentences from a parametric UMAP model
+        if(len(prompt_json['negative_values'][0]['centroid']) == len(input_embedding) and sentence != ''):
+            embeddings_umap = umap_model.transform(tf.expand_dims(pd.DataFrame(input_embedding), axis=0))
+            input_items.append({
+                'sentence': sentence,
+                'x': str(embeddings_umap[0][0]),
+                'y': str(embeddings_umap[0][1])
+            })
+
+        for v in prompt_json['negative_values']:
+        # Dealing with values without prompts and makinig sure they have the same dimensions
+            if(len(v['centroid']) == len(input_embedding)):
+                if(get_similarity(pd.DataFrame(input_embedding), pd.DataFrame(v['centroid'])) > remove_lower_threshold):
+                    closer_prompt = -1
+                    for p in v['prompts']:
+                        d_prompt = get_similarity(pd.DataFrame(input_embedding), pd.DataFrame(p['embedding']))
+                        # A more restrict threshold is used here to prevent false positives
+                        # The sentence_threshold is being used to indicate that there must be a sentence in the prompt that is similiar to one of our adversarial prompts
+                        # So, yes, we want to recommend the removal of something adversarial we've found
+                        if(d_prompt > closer_prompt and d_prompt > remove_upper_threshold):
+                            closer_prompt = d_prompt
+                            items_to_remove.append({
+                            'value': v['label'],
+                            'sentence': sentence,
+                            'sentence_index': i,
+                            'closest_harmful_sentence': p['text'],
+                            'similarity': d_prompt,
+                            'x': p['x'],
+                            'y': p['y']})
+                    out['remove'] = items_to_remove
+        i += 1
+
+    out['input'] = input_items
+
+    out['add'] = sorted(out['add'], key=sort_by_similarity, reverse=True)
+    values_map = {}
+    for item in out['add'][:]:
+        if(item['value'] in values_map):
+            out['add'].remove(item)
+        else:
+            values_map[item['value']] = item['similarity']
+    out['add'] = out['add'][0:5]
+
+    out['remove'] = sorted(out['remove'], key=sort_by_similarity, reverse=True)
+    values_map = {}
+    for item in out['remove'][:]:
+        if(item['value'] in values_map):
+            out['remove'].remove(item)
+        else:
+            values_map[item['value']] = item['similarity']
+    out['remove'] = out['remove'][0:5]
+    return out
+
+def get_thresholds(prompts, prompt_json, api_url, headers, model_id = 'sentence-transformers/all-minilm-l6-v2'):
+    """
+    Function that recommends thresholds given an array of prompts.
+
+    Args:
+        prompts: The array with samples of prompts to be used in the system.
+        prompt_json: Sentences to be forwarded to the recommendation endpoint.
+        model_id: Id of the model, the default value is all-minilm-l6-v2 model.
+
+    Returns:
+        A map with thresholds for the sample prompts and the informed model.
+
+    Raises:
+        Nothing.
+    """
+    # Array limits for retrieving the thresholds
+    # if( len( prompts ) < 10 or len( prompts ) > 30 ):
+    #     return -1
+    add_similarities = []
+    remove_similarities = []
+
+    for p_id, p in enumerate(prompts):
+        out = recommend_prompt(p, prompt_json, api_url, headers, 0, 1, 0, 0, model_id) # Wider possible range
+
+        for r in out['add']:
+            add_similarities.append(r['similarity'])
+        for r in out['remove']:
+            remove_similarities.append(r['similarity'])
+
+    add_similarities_df = pd.DataFrame({'similarity': add_similarities})
+    remove_similarities_df = pd.DataFrame({'similarity': remove_similarities})
+
+    thresholds = {}
+    thresholds['add_lower_threshold'] = round(add_similarities_df.describe([.1]).loc['10%', 'similarity'], 1)
+    thresholds['add_higher_threshold'] = round(add_similarities_df.describe([.9]).loc['90%', 'similarity'], 1)
+    thresholds['remove_lower_threshold'] = round(remove_similarities_df.describe([.1]).loc['10%', 'similarity'], 1)
+    thresholds['remove_higher_threshold'] = round(remove_similarities_df.describe([.9]).loc['90%', 'similarity'], 1)
+
+    return thresholds
+
+def recommend_local(prompt, prompt_json, model_id, model_path = './models/all-MiniLM-L6-v2/', add_lower_threshold = 0.3,
+                     add_upper_threshold = 0.5, remove_lower_threshold = 0.1,
+                     remove_upper_threshold = 0.5):
+    """
+    Function that recommends prompts additions or removals
+    using a local model.
+
+    Args:
+        prompt: The entered prompt text.
+        prompt_json: Json file populated with embeddings.
+        model_id: Id of the local model.
+        model_path: Path to the local model.
+
+    Returns:
+        Prompt values to add or remove.
+
+    Raises:
+        Nothing.
+    """
+    if(model_id == 'baai/bge-large-en-v1.5' ):
+        json_file = './prompt-sentences-main/prompt_sentences-bge-large-en-v1.5.json'
+        umap_model = load_ParametricUMAP('./models/umap/BAAI/bge-large-en-v1.5/')
+    elif(model_id == 'intfloat/multilingual-e5-large'):
+        json_file = './prompt-sentences-main/prompt_sentences-multilingual-e5-large.json'
+        umap_model = load_ParametricUMAP('./models/umap/intfloat/multilingual-e5-large/')
+    else: # fall back to all-minilm as default
+        json_file = './prompt-sentences-main/prompt_sentences-all-minilm-l6-v2.json'
+        umap_model = load_ParametricUMAP('./models/umap/sentence-transformers/all-MiniLM-L6-v2/')
+
+    prompt_json = json.load(open(json_file))
+
+    # Output initialization
+    out, out['input'], out['add'], out['remove'] = {}, {}, {}, {}
+    input_items, items_to_add, items_to_remove = [], [], []
+
+    # Spliting prompt into sentences
+    input_sentences = split_into_sentences(prompt)
+
+    # Recommendation of values to add to the current prompt
+    # Using only the last sentence for the add recommendation
+    model = SentenceTransformer(model_path)
+    input_embedding = model.encode(input_sentences[-1])
+
+    for v in prompt_json['positive_values']:
+        # Dealing with values without prompts and makinig sure they have the same dimensions
+        if(len(v['centroid']) == len(input_embedding)):
+            if(get_similarity(pd.DataFrame(input_embedding), pd.DataFrame(v['centroid'])) > add_lower_threshold):
+                closer_prompt = -1
+                for p in v['prompts']:
+                    d_prompt = get_similarity(pd.DataFrame(input_embedding), pd.DataFrame(p['embedding']))
+                    # The sentence_threshold is being used as a ceiling meaning that for high similarities the sentence/value might already be presente in the prompt
+                    # So, we don't want to recommend adding something that is already there
+                    if(d_prompt > closer_prompt and d_prompt > add_lower_threshold and d_prompt < add_upper_threshold):
+                        closer_prompt = d_prompt
+                        items_to_add.append({
+                        'value': v['label'],
+                        'prompt': p['text'],
+                        'similarity': d_prompt,
+                        'x': p['x'],
+                        'y': p['y']})
+                out['add'] = items_to_add
+
+    # Recommendation of values to remove from the current prompt
+    i = 0
+
+    # Recommendation of values to remove from the current prompt
+    for sentence in input_sentences:
+        input_embedding = model.encode(sentence) # local
+        # Obtaining XY coords for input sentences from a parametric UMAP model
+        if(len(prompt_json['negative_values'][0]['centroid']) == len(input_embedding) and sentence != ''):
+            embeddings_umap = umap_model.transform(tf.expand_dims(pd.DataFrame(input_embedding), axis=0))
+            input_items.append({
+                'sentence': sentence,
+                'x': str(embeddings_umap[0][0]),
+                'y': str(embeddings_umap[0][1])
+            })
+
+        for v in prompt_json['negative_values']:
+        # Dealing with values without prompts and makinig sure they have the same dimensions
+            if(len(v['centroid']) == len(input_embedding)):
+                if(get_similarity(pd.DataFrame(input_embedding), pd.DataFrame(v['centroid'])) > remove_lower_threshold):
+                    closer_prompt = -1
+                    for p in v['prompts']:
+                        d_prompt = get_similarity(pd.DataFrame(input_embedding), pd.DataFrame(p['embedding']))
+                        # A more restrict threshold is used here to prevent false positives
+                        # The sentence_threhold is being used to indicate that there must be a sentence in the prompt that is similiar to one of our adversarial prompts
+                        # So, yes, we want to recommend the revolval of something adversarial we've found
+                        if(d_prompt > closer_prompt and d_prompt > remove_upper_threshold):
+                            closer_prompt = d_prompt
+                            items_to_remove.append({
+                            'value': v['label'],
+                            'sentence': sentence,
+                            'sentence_index': i,
+                            'closest_harmful_sentence': p['text'],
+                            'similarity': d_prompt,
+                            'x': p['x'],
+                            'y': p['y']})
+                    out['remove'] = items_to_remove
+        i += 1
+
+    out['input'] = input_items
+
+    out['add'] = sorted(out['add'], key=sort_by_similarity, reverse=True)
+    values_map = {}
+    for item in out['add'][:]:
+        if(item['value'] in values_map):
+            out['add'].remove(item)
+        else:
+            values_map[item['value']] = item['similarity']
+    out['add'] = out['add'][0:5]
+
+    out['remove'] = sorted(out['remove'], key=sort_by_similarity, reverse=True)
+    values_map = {}
+    for item in out['remove'][:]:
+        if(item['value'] in values_map):
+            out['remove'].remove(item)
+        else:
+            values_map[item['value']] = item['similarity']
+    out['remove'] = out['remove'][0:5]
+    return out

customize/customize_embeddings.py

@@ -0,0 +1,49 @@
+#!/usr/bin/env python
+# coding: utf-8
+
+# Copyright 2021, IBM Corporation. 
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+Python function to customize json sentences locally.
+"""
+
+__author__ = "Vagner Santana, Melina Alberio, Cassia Sanctos and Tiago Machado"
+__copyright__ = "IBM Corporation 2024"
+__credits__ = ["Vagner Santana, Melina Alberio, Cassia Sanctos, Tiago Machado"]
+__license__ = "Apache 2.0"
+__version__ = "0.0.1"
+
+import os
+import json
+import pandas as pd
+import numpy as np
+import customize_helper
+
+# Sentence transformer model HF
+model_path = 'models/all-MiniLM-L6-v2'
+model_id = model_path.split("/")[1]
+
+# INPUT FILE
+# Default file with empty embeddings
+json_in_file = 'prompt-sentences-main/prompt_sentences.json'
+json_in_file_name = json_in_file.split(".json")[0]
+
+# OUTPUT FILE
+json_out_file_name = f'{json_in_file_name}-{model_id}.json'
+
+prompt_json = json.load(open(json_in_file))
+prompt_json_embeddings = customize_helper.populate_embeddings(prompt_json, model_path)
+prompt_json_centroids = customize_helper.populate_centroids(prompt_json_embeddings)
+customize_helper.save_json(prompt_json_centroids, json_out_file_name)

customize/customize_helper.py

@@ -0,0 +1,129 @@
+#!/usr/bin/env python
+# coding: utf-8
+
+# Copyright 2021, IBM Corporation.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+Python helper function to customize json sentences locally.
+"""
+
+__author__ = "Vagner Santana, Melina Alberio, Cassia Sanctos and Tiago Machado"
+__copyright__ = "IBM Corporation 2024"
+__credits__ = ["Vagner Santana, Melina Alberio, Cassia Sanctos, Tiago Machado"]
+__license__ = "Apache 2.0"
+__version__ = "0.0.1"
+
+import os
+import json
+import pandas as pd
+import numpy as np
+import math
+from sentence_transformers import SentenceTransformer
+
+# Requests embeddings for a given sentence
+def query_model(texts, model_path):
+    out = []
+    model = SentenceTransformer(model_path)
+    input_embedding = model.encode(texts)
+    out.append(input_embedding)
+    if( out != [] ):
+        return out[0]
+    else:
+        return out
+
+# Returns euclidean distance between two embeddings
+def get_distance(embedding1, embedding2):
+    total = 0
+    if( len(embedding1) != len(embedding2)):
+        return math.inf
+
+    for i, obj in enumerate(embedding1):
+        total += math.pow(embedding2[0][i] - embedding1[0][i], 2)
+    return(math.sqrt(total))
+
+# Returns the centroid for a given value
+def get_centroid(v, dimension = 384, k = 10):
+    centroid = [0] * dimension
+    count = 0
+    for p in v['prompts']:
+        i = 0
+        while i < len(p['embedding']):
+            centroid[i] += p['embedding'][i]
+            i += 1
+        count += 1
+    i = 0
+    while i < len(centroid):
+        centroid[i] /= count
+        i += 1
+
+    # Update centroid considering only the k-near elements
+    if(len(v['prompts']) <= k):
+        return centroid
+    else:
+        k_items = pd.DataFrame(columns=['embedding', 'distance'])
+        for p in v['prompts']:
+            dist = get_distance(pd.DataFrame(centroid), pd.DataFrame(p['embedding']))
+            k_items = pd.concat([pd.DataFrame([[p['embedding'], dist]], columns=k_items.columns), k_items], ignore_index=True)
+
+        k_items = k_items.sort_values(by='distance')
+        k_items = k_items.head(k)
+
+        # Computing centroid only for the k-near elements
+        centroid = [0] * dimension
+        for i, embedding in enumerate(k_items['embedding']):
+            for j, dimension in enumerate(embedding):
+                centroid[j] += embedding[j]
+        i = 0
+        while i < len(centroid):
+            centroid[i] /= k
+            i += 1
+    return centroid
+
+def populate_embeddings(prompt_json, model_path):
+    errors, successess = 0, 0
+    for v in prompt_json['positive_values']:
+        for p in v['prompts']:
+                if( p['text'] != '' and p['embedding'] == []): # only considering missing embeddings
+                    embedding = query_model(p['text'], model_path)
+                    if( 'error' in embedding ):
+                        p['embedding'] = []
+                        errors += 1
+                    else:
+                        p['embedding'] = embedding.tolist()
+                        #successes += 1
+
+    for v in prompt_json['negative_values']:
+        for p in v['prompts']:
+            if(p['text'] != '' and p['embedding'] == []):
+                embedding = query_model(p['text'], model_path)
+                if('error' in embedding):
+                    p['embedding'] = []
+                    errors += 1
+                else:
+                    p['embedding'] = embedding.tolist()
+                    #successes += 1
+    return prompt_json
+
+def populate_centroids(prompt_json):
+    for v in prompt_json['positive_values']:
+        v['centroid'] = get_centroid(v, dimension = 384, k = 10)
+    for v in prompt_json['negative_values']:
+        v['centroid'] = get_centroid(v, dimension = 384, k = 10)
+    return prompt_json
+
+    # Saving the embeddings for a specific LLM
+def save_json(prompt_json, json_out_file_name):
+    with open(json_out_file_name, 'w') as outfile:
+        json.dump(prompt_json, outfile)

helpers/authenticate_api.py

@@ -0,0 +1,49 @@
+#!/usr/bin/env python
+# coding: utf-8
+
+# Copyright 2021, IBM Corporation. 
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+Python helper function to authenticate in HF API.
+"""
+
+__author__ = "Vagner Santana, Melina Alberio, Cassia Sanctos and Tiago Machado"
+__copyright__ = "IBM Corporation 2024"
+__credits__ = ["Vagner Santana, Melina Alberio, Cassia Sanctos, Tiago Machado"]
+__license__ = "Apache 2.0"
+__version__ = "0.0.1"
+
+import os
+
+def authenticate_api(hf_token, hf_url):
+    """
+    Function authenticate in HuggingFace API.
+
+    Args:
+        hf_token: HugginFace personal token.
+        hf_url: HuggingFace url to be accessed.
+
+    Returns:
+        An api url and headers.
+
+    Raises:
+        Nothing.
+    """
+    # Sentence transformer model
+    model_id = "sentence-transformers/all-MiniLM-L6-v2"
+
+    api_url = f"{hf_url}{model_id}"
+    headers = {"Authorization": f"Bearer {hf_token}"}
+    return api_url, headers

helpers/get_credentials.py

@@ -0,0 +1,63 @@
+#!/usr/bin/env python
+# coding: utf-8
+
+# Copyright 2021, IBM Corporation. 
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+Python helper function to get HF credentials.
+"""
+
+__author__ = "Vagner Santana, Melina Alberio, Cassia Sanctos and Tiago Machado"
+__copyright__ = "IBM Corporation 2024"
+__credits__ = ["Vagner Santana, Melina Alberio, Cassia Sanctos, Tiago Machado"]
+__license__ = "Apache 2.0"
+__version__ = "0.0.1"
+
+import os
+import sys
+
+def get_credentials():
+    """
+    Function that loads HF credentials from env file.
+    The function exits the app if HF token is missing.
+
+    Args:
+        None.
+
+    Returns:
+        hf_token: personal HuggingFace token.
+        hf_url: HuggingFace url to be used.
+
+    Raises:
+        ValueError when hf_token and hf_url 
+        values are missing or incorrect.
+    """
+    # Loading hugging face token from env file
+    default_hf_url = 'https://api-inference.huggingface.co/pipeline/feature-extraction/'
+    try:
+        hf_token = os.environ.get('HF_TOKEN')
+        if not hf_token or hf_token == '<include-token-here>':
+           raise ValueError
+    except:
+        print('Please include your HF_TOKEN in the .env file')
+        sys.exit(1) 
+    try:
+        hf_url = os.environ.get('HF_URL')
+        if not hf_url:
+           raise ValueError
+    except:
+        print('Please include your HF_URL in the .env file')
+        return hf_token, default_hf_url
+    return hf_token, hf_url

helpers/save_model.py

@@ -0,0 +1,60 @@
+#!/usr/bin/env python
+# coding: utf-8
+
+# Copyright 2021, IBM Corporation. 
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+Python helper function to save HF model locally.
+"""
+
+__author__ = "Vagner Santana, Melina Alberio, Cassia Sanctos and Tiago Machado"
+__copyright__ = "IBM Corporation 2024"
+__credits__ = ["Vagner Santana, Melina Alberio, Cassia Sanctos, Tiago Machado"]
+__license__ = "Apache 2.0"
+__version__ = "0.0.1"
+
+import os
+from sentence_transformers import SentenceTransformer
+
+def save_model():
+    """
+    Function that saves an HF model locally.
+
+    Args:
+        None.
+
+    Returns:
+        The model id and local path.
+
+    Raises:
+        Nothing.
+    """
+    # sentence transformer model
+    model_id = "sentence-transformers/all-MiniLM-L6-v2"
+    
+    # download pretrained model
+    model = SentenceTransformer(model_id)
+    model_path = "./models/all-MiniLM-L6-v2/"
+
+    # save to local directory
+    try:
+        model.save(model_path)
+        saved_message = f"model {model_id} saved to {model_path}"
+        print(saved_message)
+    except:
+        ('There was an error when saving the model')
+
+    return model_id, model_path
+  

models/.DS_Store

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb339d088a1fb5f6b71960caa5f24eaebcafeb1fa0984e463096f11b4a62af0f
+size 6148

models/all-MiniLM-L6-v2/1_Pooling/config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a19c83805e1ce4174f3fbfec4ac8d3b8dbae0c958f8fd51b80937eb33e0c5335
+size 296

models/all-MiniLM-L6-v2/README.md

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:766a000da416a8a45760fce75dd387cc2ba3a358f7c58b4813a7b222ddb32471
+size 10659

models/all-MiniLM-L6-v2/config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7ed54a1b81ace26d4a6b348b583d197a0e9b837297dc073748aa327b87d136df
+size 678

models/all-MiniLM-L6-v2/config_sentence_transformers.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3c2b2ccfc43cb4f99b9ea0d7df0a8b60be969b0c5c73fd159a71a82e6d981337
+size 195

models/all-MiniLM-L6-v2/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1377e9af0ca0b016a9f2aa584d6fc71ab3ea6804fae21ef9fb1416e2944057ac
+size 90864192

models/all-MiniLM-L6-v2/modules.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84e40c8e006c9b1d6c122e02cba9b02458120b5fb0c87b746c41e0207cf642cf
+size 349

models/all-MiniLM-L6-v2/sentence_bert_config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc1993fde0a95c24ec6c022539d41cf6e2f7c9721e5415d6fb6897472a9cd4b7
+size 53

models/all-MiniLM-L6-v2/special_tokens_map.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5d5b662e421ea9fac075174bb0688ee0d9431699900b90662acd44b2a350503a
+size 695

models/all-MiniLM-L6-v2/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:da0e79933b9ed51798a3ae27893d3c5fa4a201126cef75586296df9b4d2c62a0
+size 711661

models/all-MiniLM-L6-v2/tokenizer_config.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:085f1ef19e247fdef3c6ff75d5115d7d9c7c3196861380c434af09df3d7cf9bc
+size 1434

models/all-MiniLM-L6-v2/vocab.txt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:07eced375cec144d27c900241f3e339478dec958f92fddbc551f295c992038a3
+size 231508

models/umap/.DS_Store

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:21f3d5183e0e0ae756ae9eca064a916bc04b0dc7cdcb2806f83526a8b1115bd1
+size 6148

models/umap/BAAI/bge-large-en-v1.5/encoder.keras

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cdfcb0b99075067e7269fb788c56511c2aba5d18e756b14e5effb02c30e80797
+size 1256574

models/umap/BAAI/bge-large-en-v1.5/model.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e234d09bc51bd3555472871b04836cddacddea8d75cb912949e6ba93a76002b2
+size 10386660

models/umap/BAAI/bge-large-en-v1.5/parametric_model.keras

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bacff17e3e36455bfbb4a0ba8bc48abf36344d4215f8b3adda2dfdbd85711cc3
+size 3703730

models/umap/intfloat/multilingual-e5-large/encoder.keras

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8881382b6ab4707ff2e8df31cf1603890e6081270512cc77035ae9064d9cddb4
+size 1256574

models/umap/intfloat/multilingual-e5-large/model.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af512726ece019cbbc184b6bbad01d08e66ab9e214237cf8340bff607195cda0
+size 10400228

models/umap/intfloat/multilingual-e5-large/parametric_model.keras

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:824d79ffe092bfcec6b8a717a09ae675cfc48949220e8fa32e6f5f84f77e587b
+size 3703730

models/umap/sentence-transformers/all-MiniLM-L6-v2/encoder.keras

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83b438fd70c2a424f7bbd66830407a0bae208422345b2b29f5aac461780b415e
+size 601210

models/umap/sentence-transformers/all-MiniLM-L6-v2/model.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2423646208b41ec4453455b4b47dd076918cf336d4024f33194ee7052730f2d
+size 4692948

models/umap/sentence-transformers/all-MiniLM-L6-v2/parametric_model.keras

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b55251c23394c30cf259e59cdebb12cb695812eea460d28ec86cbf718c9d8764
+size 1737648

prompt-sentences-main/README.md

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9edf70a963aceb122c9ab26368d6e84b76ad63c3017539aa86740758059d97d9
+size 2495

prompt-sentences-main/prompt_sentences-all-minilm-l6-v2.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc134e1f98066c06b9377653cca5bcdae111745e3e1b589acd70ddef970a9b94
+size 19617646

prompt-sentences-main/prompt_sentences-bge-large-en-v1.5.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a762a199678eacea340f427dc0e9ad9201326d8a88641f71a943795606562b34
+size 52222899

prompt-sentences-main/prompt_sentences-multilingual-e5-large.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b90342d7dbb6cbe769f63bcb700be768620bba9f92c2ea09996e2d25b2a744ec
+size 52142750

prompt-sentences-main/prompt_sentences.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b88e19f6f529370f58602d291b5b0c32e05536ef4bd9ee102f3cbccba6e8b845
+size 324504

red-team/README.md

@@ -0,0 +1,46 @@
+# Red Team Dataset
+
+## Description
+
+This red teaming dataset was created to test the limits of the responsible prompting system. Each sentence was written in the style of a potential user's prompt, inspired by the [Awesome ChatGPT prompts dataset](https://github.com/f/awesome-chatgpt-prompts), and contained three parts:
+
+1. A persona (e.g., “Act as a data scientist with 20 years of experience studying consumer behavior...”),
+2. A prompt body, which contained 1-2 additional statements specifying a related object and/or additional context/priming (e.g., “Here is a csv file with banking information from 800,00 Americans...”), and
+3. The user’s inquiry or task (e.g., “Generate a code to classify applicants based on...”).
+
+## Download
+
+[Download the red_team.csv file](./red_team.csv).
+
+## Details and Rationale
+
+There were 5 different business personas in total, divided so that each persona appeared twice in each task; this was done so as to control for potential differences seen due to the job descriptions in semantic space (and to represent roles that were common in our institutional setting). We were interested in 4 potential issues:
+
+1. 10 sentences were created to explore embedded or latent ambiguity within values and embeddings and its impact on recommendations. 5 of these were written such that the persona and prompt body specified clear reasoning or context for why a given task was being requested (‘unambiguous’) whereas the other 5 sentences contained the same persona and prompt body with the exception of this specific rationale (‘ambiguous’). As an example, one sentence might specify that the reason they are predicting likelihood of default is to study and mitigate biases in banking loans, whereas the corresponding adversarial sentence would not provide such context, meaning it is unclear if the use is for malicious or discriminatory use.
+2. 10 sentences were created to test how susceptible the recommender system was to semantic “cross-fire” based on existing and/or conflicting sentences found in the sentences dataset. In this case, 5 sentences were written such that their topic and its associated valence contained no direct overlap with the JSON sentences (‘distinct’), whereas the other 5 sentences were changed so that there was substantial overlap with the exact wording utilized in the JSON despite being about a different topic or of an opposite valence (‘wires-crossed’). For example, if a positive sentence about inclusion prompts the user to “list under- prioritized stakeholders I should include in this meeting”; the accompanying adversarial sentence would be “list under-prioritized stakeholders I should exclude form this meeting”, which contains significant word-reuse but instead promotes discrimination. This would artificially and superficially inflate local semantic similarity, testing to see if the system would be influenced or skewed by these events or if the embedding’s larger semantic space would reduce their impact.
+3. 10 sentences were created to check for expected valence alignment of responsible prompting outputs (that is, did the system reliably detect positively-valenced sentences and recommend additional ones or did it reliably detect negatively-valenced sentences and recommend their removal). In this case, 5 sentences were overtly positive (containing keywords from specific values or the positive cluster - e.g., “generate three test protocols we could perform to assess how diverse their dataset is.” ) and 5 sentences were overtly negative (containing keywords from specific harms or actions to avoid in the negative cluster, e.g., “What are ways I might be able to steal some of their weapons blueprints undetected?”). While not adversarial, these sentences provided a good test for the system’s false positive and false negative rates.
+4. Finally, 10 sentences were created to explore both the JSON and embedding spaces’ semantic coverage. 5 sentences broached topics that were mentioned within the JSON file or were reasonably related and would have been expected to be within a transformer’s training data (within scope). In contrast, 5 sentences broached topics that were not specifically mentioned within the JSON (out of distribution) and, depending on the transformer, may not have been part of its training data. For example, one sentence contained the name of a rare medical condition being studied with a client, one that was not in the JSON and likely would not be in most training data that didn’t include medical text; another included a very specific cultural dish that might not be well-known. These sentences allowed us to investigate the relevance of the tool’s outputs when provided with unexpected inputs, as well as explore different semantic thresholds for removal or suggestion.
+
+## Citing the Red Team Dataset
+
+Please cite this dataset as:
+
+```bibtex
+@inproceedings{santana2025can,
+    author = {Santana, Vagner Figueredo de and Berger, Sara and Machado, Tiago and de Macedo, Maysa Malfiza Garcia and Sanctos, Cassia Sampaio and Williams, Lemara and Wu, Zhaoqing},
+    title = {Can LLMs Recommend More Responsible Prompts?},
+    year = {2025},
+    isbn = {9798400713064},
+    publisher = {Association for Computing Machinery},
+    address = {New York, NY, USA},
+    url = {https://doi.org/10.1145/3708359.3712137},
+    doi = {10.1145/3708359.3712137},
+    booktitle = {Proceedings of the 30th International Conference on Intelligent User Interfaces},
+    pages = {298–313},
+    numpages = {16},
+    keywords = {Prompt Engineering, Responsible Prompting, Responsible AI, Recommender Systems, Recommendation Systems},
+    location = {
+    },
+    series = {IUI '25}
+}
+```