Using ChatGPT in the ORKG with Smart Suggestions
ChatGPT is rapidly gaining popularity and is used for a wide range of applications. Also for the ORKG, ChatGPT is a perfect match. One of the main tasks of users entering data in the ORKG, is reading articles, extracting knowledge presented within the article, and finally entering this in the ORKG. Since ChatGPT is very capable in dealign with natural language, also for the tasks that ORKG users perform, ChatGPT is an very suitable assistant.
There are many different approaches of integrating ChatGPT into the UI workflows. The method described in this article integrates ChatGPT in a non-intrusive way into the existing ORKG workflow. This means that you can start benefitting from the integration while still using the interfaces you are familiar with. We will now discuss how you can use the ChatGPT integration within ORKG.
General ChatGPT suggestions box
You can recognize the availability of ChatGPT support by the green lightbulb button, as displayed below.
Since ChatGPT provides you smart suggestions, we called the suggestion box Smart suggestions. The Smart suggestions box looks always similar, which you can see below.
The Explanation provides a short description what the suggestions are taking as input and what is the provided output. The Suggestions themselves are listed below. The Help button takes you to this page. The Reload button is helpful in case you want to get different suggestions. This can also be helpful if for some reason the suggestions aren’t displayed correct. Finally, Feedback is giving you the opportunity to provide feedback about the specific suggestions. This is very important for us, as it helps us to further improve the ChatGPT integrations. Below, you can see how the feedback form can be filled out.
Your feedback is very valuable for us, so would like to ask you to actively fill out the feedback form when using this functionality. It is very helpful if you provide feedback multiple times, as each case is different.
Use cases
As mentioned before, ChatGPT is integrated in a similar manner throughout the ORKG website. However, the suggestions themselves are very different. In total, we created 6 use cases where ChatGPT is used. In the future, we hope to add additional use cases. Below the use cases are explained in more detail.
Value suggestions
For a selected set of properties, the Smart suggestions appear to provide a list of suggestions resources. The suggestions are generated based on the title and abstract of a paper. Currently, the Smart suggestions are only displayed for the following properties: research problem, method, and material. Only when visiting the View paper page, the light button button appears, e.g., it does not appear on the resource page.
Tip: when the abstract is not found automatically, you will see an exclamation mark after the word abstract in the modal (see image below). To get better suggestions, you can manually provide the abstract by clicking the "Paper abstract" button in the "Suggestions" box on the right side.
Property suggestions
Properties are suggested based on the existing properties that are added. Therefore, the suggestions can only be used if there are already existing properties added. The suggestions appear next to the Add property button, both on the View paper page and within the Contribution editor.
Feedback on whether text should be a resource
Sometimes it is hard to determine whether some piece of text should be stored as a resource or as text (generally, if you, or anyone else, can reuse this piece of information, you can use a resource). The Smart suggestion here is helping you to decide what is the best type to choose. Since there is no clear right and wrong, ChatGPT will only give feedback and not a final correctness core. You find this Smart suggestion while adding a value of type, Text, Number, Date, etc.
Feedback on resource structure
When creating a resource, it is sometimes possible to decompose your resource to make it atomic. As a concrete example, the resource "120 participants between 45-65 recruited online" can be decomposed into several resources (about the number of participants, their age, and the method of recruitment). The Smart suggestions provide feedback on whether it indeed makes sense to decompose your resource into several resources. You find this Smart suggestion while adding a value of type Resource.
Tip: sometimes when reloading the suggestions, the opinion of ChatGPT changes. This means that one time no restructuring is needed, while after clicking the Reload button, the restructuring is recommended. So use the suggestions from ChatGPT as inspiration, and not as facts.
Feedback on property reusability
When creating new properties, it is best to provide a generic label. This makes it possible for others to reuse properties more easily. See the [modeling best practices] for more information. ChatGPT checks whether a property label is indeed generic enough to make it reusable. You will find this Smart suggestion when creating a new property.
Feedback on comparison descriptions
When publishing a comparisons, it is best practice to provide a clear description of the comparison objectives. We integrated ChatGPT here to provide feedback and some inspiration to improve your description. Currently, this Smart suggestion is only displayed in the Publish comparison popup.
Known Issues
I get "Failed to fetch recommendations. Try again."
This can happen because of several different reasons. Most likely the response from ChatGPT could not be parsed. This sometimes is caused by the input data (e.g. a too short label, missing description etc.). Try updating the input data and try it again. Otherwise, try it again later.
The recommendations are inaccurate / wrong
It is indeed very possible that recommendations are wrong. This is one of the reasons why human validation is still needed, and why this process doesn’t happen fully automatically. When results are wrong, feel free to ignore them. Of course, it is always helpful to provide feedback about your specific case.