Jak małe modele językowe łączą siły, aby sprostać dużym wyzwaniom związanym z rozumowaniem

While advancements in artificial intelligence (AI) have undoubtedly gone a long way, they are not without their share of struggles. Particularly when the subject at hand involves complex problem-solving tasks such as Sudoku, molecular design, or writing math proofs, AI models are known to have a difficult time. Most models, even the advanced ones, find it challenging to handle open-ended tasks ruled by strict guidelines. They tend to offer advice on how to go about a problem rather than coming up with solutions themselves.

To address these bottlenecks, a team of researchers at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) decided to take a novel approach that goes by the name of DisCIPL, a short form for, “Distributional Constraints by Inference Programming with Language Models”. This approach brings together the strategic capabilities of large language models (LLMs) with the lightweight efficiency of the smaller ones. The core idea is straightforward yet powerful: it involves getting a large model to strategize on a task, then handing over the implementation to smaller models, which then follow the command of the large model.

A Better Approach to Collaborative AI

Think of how a project manager coordinates a team. The large ‘boss’ model act as the project manager, accepting a prompt and then coming up with a plan, which it then distributes to the smaller ‘follower’ models. If required, the large model corrects the outputs of the smaller models. This collaborative system increases the ability of smaller models to produce responses that are not only more accurate but also more efficient than those generated by some of the most cutting-edge LLMs.

An essential component of DisCIPL is its use of LLaMPPL, a programming language designed to control language models by encoding specific rules and constraints. This programming layer allows the large model to communicate with its follower models in a structured, rule-based manner, guiding them toward more precise and coherent responses. They are given meticulous instructions, such as “write eight lines of poetry where each line has exactly eight words”, which ensures that every smaller model contributes significantly to the final output.

A New Era of Efficient AI

The efficiency that DisCIPL brings to the table is remarkable. By employing smaller models, the system greatly cuts down computational costs. To take it a notch higher, the system has the capacity to run multiple models simultaneously, considerably speeding up response times.

DisCIPL’s practical implications are widespread. It can be employed in carrying out tasks like creating ingredient lists, planning travel itineraries, or even drafting grant proposals with the given word limit. When compared with heavyweight competitors like GPT-4o and o1, it delivers coherent and accurate results that stand at par with the top reasoning systems in the world. Its ability to follow instructions aright is a testament to the effectiveness of a planning component in any architectural structure.

The Future of Collaborative AI

Moving forward, the research team plans to explore a fully recursive version of the framework and apply the system to mathematical reasoning tasks. Furthermore, they are keen on exploring how the system handles user preferences that are not easily encoded in rules. Gabriel Grand, a PhD student at MIT and the lead author of the study puts it, “Language models are consuming more energy as people use them more, which means we need models that provide accurate answers while using minimal computing power.”

Thus, DisCIPL challenges the notion that size always matters in the world of AI. By capitalizing on the strengths of smaller models in a coordinated manner, it paves the way to further advancements in AI systems that are faster, cheaper and potentially more interpretable. As we continue to refine this approach, we stand at the threshold of a bright future for collaborative AI.

To learn more about DisCIPL, check out the original article on MIT News.