Figure 7 from "I'd Like to Have an Argument, Please": Argumentative Reasoning in Large Language Models | Semantic Scholar (2024)

[PDF] Semantic Reader

This paper presents a multi-agent framework called ArgMed-Agents, which aims to enable LLM-based agents to make explainable clinical decision reasoning through interaction and provides users with decision explanations that increase their confidence.

It is found that GPT-4 can play the game to a passable degree: it is able to manipulate doors, combat enemies, and perform pathing, but more complex prompting strategies involving multiple model calls provide better results.

The present survey sifts through the literature to review papers concerning this kind of argumentation-based bot, drawing conclusions about the benefits and drawbacks that this approach entails in comparison with standard chatbots, while also envisaging possible future development and integration with the Transformer-based architecture and state-of-the-art Large Language models.

This work presents a new synthetic question-answering dataset called PrOntoQA, where each example is generated from a synthetic world model represented in first-order logic, and shows that LLMs are quite capable of making correct individual deduction steps, and so are generally capable of reasoning, even in fictional contexts.

This framework enables these two phases to be jointly trained in a single MRC model, thereby maximizing the mutual benefits of them and outperforming the state-of-the-art method.

It is shown that Abstract Meaning Representation (AMR) graphs can be useful for representing arguments, and that novel AMR graph metrics can offer explanations for argument similarity ratings and make argument similarity judgements more interpretable and may even support argument quality judgements.

It is found that models can learn just as fast with many prompts that are intentionally irrelevant or even pathologically misleading as they do with instructively “good” prompts, and instruction-tuned models often produce good predictions with irrelevant and misleading prompts even at zero shots.

It is thought that LLMs have a promising use as reviewing assistants for specific reviewing tasks, but not for complete evaluations of papers or proposals.

Experimental results demonstrate that the Zero-shot-CoT, using the same single prompt template, significantly outperforms zero-shot LLM performances on diverse benchmark reasoning tasks including arithmetics, symbolic reasoning, and other logical reasoning tasks, without any hand-crafted few-shot examples.

Experiments on three large language models show that chain of thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks.

This work regards GPT-4 as a data analyst to perform end-to-end data analysis with databases from a wide range of domains and designs several task-specific evaluation metrics to systematically compare the performance between several professional human data analysts and G PT-4.

Study of three types of plagiarism among GPT-2 generated texts, in comparison to its training data, and the plagiarism patterns of fine-tuned LMs with domain-specific corpora which are extensively used in practice suggest that the practicality of current LMs in mission-critical writing tasks is questioned.

...

...