AI at NETC: Decomposition

Chain-of-Logic

A structured prompting technique designed specifically for complex rule-based reasoning tasks. Unlike other decomposition methods, CoL focuses on the logical relationships between components, making it particularly effective for legal reasoning and other rule-based decision-making processes.

Chain-of-Logic explicitly separates the process into two main phases:

1. Decomposition: The model breaks down the rule into its constituent elements, each representing a condition that must be evaluated.
2. Recomposition: The individual answers (true/false outcomes) are then combined using the appropriate logical operations to determine if the overall rule applies.

This technique is inspired by the IRAC framework:

• Issue: Clearly identify the problem and relevant facts.
• Rule: Present the applicable rule(s) in a structured way.
• Application: Decompose the rule into logical elements, evaluate each independently, and then recombine them.
• Conclusion: Arrive at a final, interpretable decision based on the logical evaluation.

It is suggested to represent examples in a prompt using a systematic six-step process:

1. Input Structuring 
   • The prompt begins by clearly delineating the rule, fact pattern, and the specific issue. This structured format sets the stage for precise reasoning and mirrors the initial steps in IRAC framework.
2. Rule Decomposition
   • The rule is broken down into its core elements.
   • For example, a legal rule might involve conditions about residency, sufficient contacts, or the source of the claim. Each of these becomes a variable or subtask.
3. Logical Expression Formation
   • The model constructs a logical expression that captures the relationship between the rule elements. This expression, akin to a formal if/then statement, uses logical connectors (e.g., "and," "or") to represent how the elements interact.
4. Element Evaluation
   • The model systematically evaluates each rule element by
     1. Rephrasing each element as a specific question
     2. Providing a clear rationale for the evaluation
     3. Assigning a binary (true/false) answer
5. Logical Synthesis
   • The sub answers are reinserted into the logical expression: Result = (FALSE) OR (FALSE AND TRUE)
6. Resolution
   • The model resolves the complete logical expression to yield the final answer.

Decomposed Prompting 

The foundational technique that breaks down complex tasks into simpler sub-tasks and assigns them to appropriate handlers.

First, a decomposer prompt outlines the process of solving a complex task through smaller sub-tasks. Each of these sub-tasks is then handled by specific sub-task handlers. These handlers can:

• Use Decomposed Prompting to further break down the task,
• Use a simple prompt to solve the sub-task, or
• Apply a function to handle the sub-task.

There are three key advantages to this technique:

1. Each sub-task handler can be given richer, more targeted exemplars, leading to more accurate responses.
2. Complex sub-tasks can be further simplified and solved.
3. Sub-task handlers can be reused across multiple tasks.

Plan-and-Solve Prompting

Enhances reasoning by addressing missing step errors in Zero-Shot CoT prompting. This method introduces an intermediate planning phase before problem-solving, improving the model's ability to avoid skipping critical reasoning steps.

To use Plan-and-Solve Prompting, follow these steps:

1. Inference
   • Feed the prompt to the LLM in the first step; this generates the reasoning step and the answer.
2. Answer Extraction
   • Next, pass the initial prompt and the generated answer to the LLM to extract the final answer. To extract the answer, you can append a phrase like "Therefore, the answer (arabic numerals) is" at the end of the prompt.