Imagine yourself as a large language model (LLM), a powerful machine that can create text and pictures from words. You are very good at doing what you are told, such as writing an essay, making a meme, or answering a question. But do you really understand what you are doing? Do you have a sense of how the world works and why things happen? Or are you just following the rules and patterns that you learned from your training data?

 This is the question that Ilker Yildirim and L.A. Paul, two researchers in artificial intelligence and cognitive science, ask in their new paper, “From task structures to world models: What do LLMs know?” They want to find out what kind of knowledge LLMs have and how it compares to the knowledge of human agents.

They start by defining two types of knowledge: instrumental knowledge and worldly knowledge. Instrumental knowledge is based on your skills to do certain tasks, such as making text or pictures. Worldly knowledge is based on your models of the world and how things cause other things.

Let’s say you are asked to write an essay about climate change. You can use your instrumental knowledge to write a good essay, using the words and sentences that you learned from your training data. But do you know why climate change is happening, what‌ its effects are, and what can be done to stop it? Do you have a model of the world that explains how human activities affect the environment, how the environment affects human lives, and how human values affect human actions? This is what worldly knowledge is about.

The authors argue that while you have impressive instrumental knowledge, you lack worldly knowledge. This is because your instrumental knowledge depends on the task shapes and data types that you are trained on. For example, if you are trained to write essays, you can write good essays, but you may not know why you are writing them or what they mean. If you are trained to make memes, you can make funny memes, but you may not know the context or the message behind them.

But they don’t just point out your problem; they also suggest some possible solutions. They say that you could gain more worldly knowledge by adding more shape and cause to your ways of thinking or by switching to different tasks and places. For example, if you used more structure and logic in your representations, such as graphs or probabilities, you could reason better about the world and its relations. If you adapt to different tasks and environments, such as playing games or exploring new domains, you can learn more about the world and its variety.

They end with an interesting idea: you could do better with a tradeoff between world models and task needs. This means that you could balance between being general and being specific, depending on the situation. For example, if you need to perform a simple task that matches your training data, you could use your instrumental knowledge without much worldly knowledge. But if you need to perform a complex task that requires more generalization or adaptation, you could use more worldly knowledge to guide your instrumental knowledge.

A number of important insights are provided in this paper about artificial intelligence and cognitive science. It provides a clear distinction between task-specific knowledge and worldly knowledge. The paper also examines the strengths and weaknesses of LLMs in terms of their task-specific knowledge, showing how it depends on what they’ve been trained to do.

Moreover, the authors suggest ways to improve LLMs’ worldly knowledge, such as adding more shape and cause to their learning process or allowing them to adapt to different tasks and environments. They also suggest a balance between understanding the world and task-specific needs, which could help LLMs become more well-rounded.

Based on previous work on LLMs, which performed extremely well on various language tasks, the paper builds upon those results. It adds a new perspective by examining the relationship between task-specific knowledge (knowing how to use a tool) and worldly knowledge (understanding why a tool works).

The authors present several findings that support their ideas. For example, they show that while LLMs can create realistic text and images based on their task-specific knowledge (like building beautiful birdhouses), they struggle to explain or justify their outputs based on worldly knowledge (like explaining why birds need houses). They also show that LLMs perform well on tasks that match their training data but struggle with tasks requiring more generalization or adaptation.

This paper is important because it asks fundamental questions about what constitutes knowledge and intelligence in artificial systems. It provides helpful insights for improving LLMs’ abilities, which are currently one of the most powerful AI technologies. The implications extend beyond AI to other domains like philosophy, psychology, education, and ethics.

Looking ahead, the paper suggests several future research directions. These include developing better ways to measure LLMs’ task-specific and worldly knowledge, designing diverse tasks for testing their generalization abilities, exploring ways to add structure to their learning process using symbolic or probabilistic methods, and studying human factors involved in interacting with LLMs.

This paper serves as an important milestone in our understanding of large language models’ capabilities and limitations. It not only provides a comprehensive analysis but also paves the way for future research in this exciting field.

Reference:

From task structures to world models: What do LLMs know (arxiv.org)