Small can be large enough for your educational model

Following our presentation at UNESCO’s Digital Learning Week in Paris, I was asked several times about my interest in small language models (SLMs). Why opt for a bike when one can have access to a super car? Large language models (LLMs) are now available, and I happen to be one of the lucky people who have access to several. Well, where do I start? My super car (I don't have one but let's pretend) would be amazing to have on a road trip with a wide highway and a long journey that could take me anywhere, but not of great use when I am going to the supermarket and want to use the small passage next to my house to get there. Can you also imagine the fuel cost? Even if it's an electric car, what if I can't charge it? Thankfully, my bike would be way better for a short trip to get milk. Full disclosure: I do have a bike but don't use it, which is highly frowned upon for a Cambridge resident. But let's not get carried away, that's a different story for another day!

So, what about these SLMs? I thought a brief, non-technical Q&A with tables to summarise it all would probably give you an idea of why I see SLMs as a valuable option in many contexts, even if they are still undervalued and underused.

When is a model 'small' and when is it 'large'?

There is no universally agreed parameter threshold that separates an SLM from an LLM. A rough rule of thumb though is that a model with fewer than 10 billion parameters is labelled as small while one with 70 billion parameters or more is a large one. If you're wondering, the terminology to describe what lies between these two extremes is a bit blurry; there's 'mid-sized', 'intermediate', 'enterprise-grade' and other terms. That's why having clear boundaries for all language models is important, where feasible! The parameters in this context are the internal, adjustable numerical values that capture patterns learned from language and guide the model’s predictions.

Why do SLMs matter in education?

An SLM can be more than enough for many well-defined educational tasks. Learners are not frequently trying to address open-ended problems; they’re mostly trying to solve a specific question, be it in mathematics or another subject, or work through routine learning activities. For such narrower, well-defined tasks, specialised smaller models can perform very effectively without requiring the capabilities of a much larger model. When appropriately trained, fine-tuned and evaluated for a specific domain, SLMs can sometimes achieve performance comparable to LLMs on such tasks.

What are the advantages of SLMs in terms of cost and affordability?

SLMs can run offline and locally, and they usually require less computational power compared with larger models which typically need more computational resources, hardware and electricity, particularly when they are deployed at scale. This can make SLMs more affordable, faster to run and also easier to deploy privately, particularly for narrower and well-defined tasks. Therefore, SLMs can offer a practical solution for education systems, especially in low-resource and low-connectivity settings, like in many low- and middle-income countries (LMICs). Table 1 below addresses this in detail.

Do SLMs help with privacy?

When a system runs locally and offline, student data does not need to be sent to an external cloud service. This can provide greater privacy and give education systems more direct control over their data governance and how AI is used within their own system, depending on how that system is designed and managed.

What about AI sovereignty?

AI sovereignty is largely about retaining control. A country or an education system would have greater control over their data, infrastructure and how their AI system is deployed and used, while reducing reliance on external service providers. 

What is the environmental impact of SLMs?

Using less computational power generally implies using less electricity, which can reduce environmental impact. Training and running large AI models requires significant resources, including electricity and, depending on the data-centre cooling system, water for cooling. Using smaller, fit-for-purpose models where they are sufficient can be one way of reducing that footprint.

How to choose between SLMs and LLMs?

It’s not about choosing between the two, and I'm not saying SLMs should replace LLMs, certainly not in every context. The question is when to use one, when to use the other, and also how they can work together or be used in combination with other educational technologies. Tables 2 and 3 below give possible scenarios for these options.

How to match AI tools to learner needs?

To put it simply, a person working on a PhD and trying to solve a complex, open-ended problem, may need the broader capabilities of an LLM. In the case of a Grade 4 student though, a well-designed SLM may be sufficient for learning activities like routine arithmetic problems. The important thing is matching the model to the task: check Table 4 below for specific examples in mathematics.

Is there enough research on the matter?

Research in this area is growing, especially on using SLMs to design tutors, which was the focus of the talk delivered in Paris. There is still limited empirical evidence though on SLMs as AI tutors, particularly in mathematics education and in real classroom settings. That’s an important area for us to explore further at Cambridge Mathematics and across Cambridge more broadly.

What would be the main message for educators on the choice of model?

In mathematics, placement can outweigh size. A small number in the right place can beat a large one out of place; think of an example with a small power vs a large coefficient! The same is true in AI and learning: fit can outweigh scale. A small model can make a large mark, and a large model can still miss the mark. In summary: While using AI for learning, remember that size doesn’t always determine impact. Fit does matter!

Join the conversation: You can tweet us @CambridgeMaths or comment below.