You are standing at the edge of a vast ocean. Each wave that comes in is a new discovery in the field of artificial intelligence. These waves bring with them new ideas and techniques that change the way we understand artificial intelligence. In this ever-changing landscape of artificial intelligence, researchers constantly search for new ways to improve language models. A shining jewel from the depths of this research is the SECToR method, a groundbreaking approach revealed by Hugh Zhang and David C. Parkes in their paper dated September 15th, 2023.
SECToR is a way for computer programs to teach themselves new things. This is similar to how people break down difficult problems into smaller tasks and tackle them one by one. To demonstrate how effective SECToR is, scientists tested it on adding numbers.
The brilliance behind SECToR lies in its use of a special thinking method called “chain-of-thought reasoning”.Some may be reminded of AlphaZero, a famous learning system that excels inboard games. The method is similar to the Monte-Carlo Tree Search (MCTS) used in AlphaZero, a strategy for making the best choice by testing a number of possibilities. As a result of this kind of special thinking, Zhang and Parkes believe language systems can be made even better, especially for jobs that require clear logic and planning, such as math.
The research in the paper showcases impressive outcomes. SECToR can teach language models to add numbers up to 29 digits long. What’s truly remarkable is that the model learns most of this on its own, with just a bit of initial guidance for numbers up to 6 digits or fewer. In simpler terms, think of SECToR as a teacher showing a student only a few small number problems, and then the student is able to solve much bigger number problems all on their own.
The researchers tackled a common problem in language models called “error avalanching” where small mistakes accumulate and lead to poor performance over time. SECToR’s smart fix is to use self-checks based on rules. For example, one rule is “commutativity,” which means things can swap places and still be the same, like 3+2 and 2+3. Another rule is “majority voting,” where the most common answer is chosen as correct. These checks help the model catch and fix its own mistakes.
The paper confidently compares SECToR with other ways of teaching math to language models. When tested against known methods such as fine-tuning (adjusting a model slightly), in-context learning (teaching within a situation), scratchpad augmentation (adding a kind of note-taking feature), and least-to-most prompting (giving hints from easy to hard), SECToR performs better. It is more accurate, works on larger scales, and applies knowledge more effectively in various situations.
When examined closely, the data made it clear that being consistent with oneself helped significantly. When this idea of following one’s own line of thinking, which we call “chain-of-thought prompting,” was implemented, the results improved even more. Especially in tests like GSM8K (where scores increased by 17.9%), SVAMP (up by 11.0%), AQuA (up by 12.2%), StrategyQA (up by 6.4%), and ARC-challenge (up by 3.9%). For those who might be wondering, these are all tests that measure how well something can solve math problems or use common sense to answer questions.
SECToR shows a lot of potential, but there is still more to explore. These researchers are thinking about whether SECToR can be used in areas like multiplication , programming , or understanding language. The experts also think about combining SECToR with other methods, like self-play (where programs play against themselves to learn) or meta-learning (learning how to learn). They’re curious about how it might work in situations that use multiple ways of interacting or sharing information
SECToR is like a guiding light, showing the way for the future of talking computer programs or language models. It gives us new ideas about teaching ourselves and makes us think differently than we used to. We are at the beginning of this exciting time, and we can only guess the many new things SECToR and similar tools will bring. Simply put, SECToR is helping change the way we use and think about computer programs that understand and generate language.
Reference:
https://arxiv.org/pdf/2309.08589.pdf