"Tree-of-Thoughts" ChatGPT Prompt Engineering using Formal Semantics
Chain-of-thoughts helps ChatGPT Trust, somewhat
Can the perceived reliability of a complicated and opaque system such as ChatGPT be improved? Although we will not engage in philosophical discussions regarding ChatGPT's ability to engage in true reasoning, the technology community has discovered that incorporating a "chain-of-thoughts" prompting strategy can provide some practical benefits.
Chain of thoughts refers to a structured approach of question-answering to get ChatGPT responses in an incrementally-specified manner. There are plenty of references regarding “Chain-of-thoughts.” I would give two options. For academic option, we go with AUGMENTED LANGUAGE MODELS: A SURVEY, by Yann Lecun et al. For wider audience, you could read medium post CHATGPT PROMPT ENGINEERING, “LET’S THINK STEP BY STEP”, AND OTHER MAGIC PHRASES, by MR NEWQ.
Although there are still many unanswered questions regarding the transparency of AI, in practical terms, there are effective and ineffective ways of utilizing ChatGPT. This is why it is important to take the practical strategies seriously.
Chain of thoughts is not Enough
Gobus Greyling articulated the following difficulties: "a small or unintended change to an upstream prompt, can lead to unexpected outputs and unpredictable results further downstream," in his medium post These Are The Challenges When Creating A LLM Based Conversational Interface
Tree-of-thoughts? Interesting
In actuality, events do not necessarily occur in a linear or chronological fashion. Rather, there may be multiple possible paths stemming from the same initial conditions, which are determined by the current state of affairs. As a result, a tree structure may be a more appropriate way to represent this than a linear chain.
As a result, any human approach to regulating the responses generated by ChatGPT should adopt a tree-like structure, rather than assuming a linear progression. Within a tree structure, the "unexpected outputs" described in Gobus' article may actually become "expected outputs" under certain conditions, depicted as a hierarchical structure resembling a tree.
More readings: https://www.linkedin.com/pulse/tree-of-thoughts-alternative-chain-of-thoughts-pingping-xiu
Tree-of-thoughts not enough
When working with human languages, there are often nuances to consider. For instance, if I were to say "I am a gay person living in Palo Alto. Can you suggest a good place to shop?" versus "I am a man living in Palo Alto. Can you suggest a good place to shop?" The responses generated by ChatGPT may differ. While the results provided by ChatGPT are not necessarily poor, they could certainly be enhanced through the implementation of specialized prompt treatments that account for the attribute of being "gay."
More readings: https://www.linkedin.com/pulse/anti-bias-semantic-prompt-modeling-chatgpt-pingping-xiu/
If you attempt to apply the "tree" metaphor to this situation, you will quickly discover that the sheer number of combinatorial possibilities present in human language can cause the "tree" to break down. In this context, the structure becomes less like a simple "tree" and more like a "fiber-ed" tree, with distinct micro-level flows branching off in various directions.
Fibered-Tree-of-thoughts, but how?
While the "fibered tree" analogy is likely more accurate, it is also considerably more challenging to implement. Therefore, the question becomes how to create a practical solution for designing a "Fiber-ed-tree-of-thoughts" specification that can be enforced during ChatGPT interactions in order to steer conversations toward safer and more moderated outcomes.
Formal Semantics
Over the past few weeks, we have conducted research into integrating Modern Type Theories into the semantic prompt engineering of ChatGPT. The good overview list is in https://www.linkedin.com/pulse/formal-semantics-prompt-engineering-pingping-xiu%3FtrackingId=2SkVeVeT1knkhJGpwJndjQ%253D%253D/?trackingId=2SkVeVeT1knkhJGpwJndjQ%3D%3D
Here we want to make certain highlights
How to establish a dialog structure using formal proof
By following the Inductive Principle in the formal specification domain, it is possible to effectively model business rules for a branched conversational dialog (dialog tree) using "Inductive" definitions in a proof assistant such as Coq.
https://www.linkedin.com/pulse/tree-of-thoughts-alternative-chain-of-thoughts-pingping-xiu/
How to formalize prompt with bias attributes
To transform natural language prompts into a format that computers can understand, the most effective approach is to formalize them using Semantics. With the help of Modern Type Theories Semantics, bias attributes like "man" or "gay" can all be given distinct and rigorous representations.
https://www.linkedin.com/pulse/anti-bias-semantic-prompt-modeling-chatgpt-pingping-xiu/
How to interact with formal language in prompt design
Designing business domain prompt structures to reflect a dialog that follows a "fibered tree of thought" can be challenging, especially when it comes to making the structure intuitive. One approach to addressing this challenge is to generate random samples, but this is not a straightforward task. However, we have discovered a cost-effective solution to this problem by leveraging Generative AI to generate random prompts.
https://www.linkedin.com/pulse/designing-random-number-generator-coq-practical-solution-pingping-xiu
Together with
https://www.linkedin.com/pulse/another-step-towards-chatgpt-semantic-testing-manifest-pingping-xiu/
How to verify ChatGPT contents with question-answering
Verifying a ChatGPT dialog that is governed by a formal representation of a "dialog fibered tree" is a straightforward process. To do so, we simply need to pose a series of questions to ChatGPT that are based on the established formal semantics, in order to confirm that the entailments are correct.
https://www.linkedin.com/pulse/chatgpt-content-creation-quality-control-pingping-xiu/
and
https://www.linkedin.com/pulse/how-prove-your-work-chatgtps-using-content-dna-formal-pingping-xiu/
Next Steps
Our plan is to utilize LangChain as a foundation for creating a representation of dialog that follows a "fibered-tree-of-thoughts" structure. Through illustrative examples, we aim to demonstrate how this framework can assist in guiding ChatGPT interactions within the complex "maze" of "fibered-tree-of-thoughts," as defined by formal representations. Also, that system will also show value on regulating and monitoring the resulting outcomes.