Chinese artificial intelligence (AI) company DeepSeek has sent , with the release of extremely efficient AI models that can compete with cutting-edge products from US companies such as OpenAI and Anthropic.
Founded in 2023, DeepSeek has with a fraction of the cash and computing power of its competitors.
DeepSeek鈥檚 鈥渞easoning鈥 R1 model, released last week, provoked excitement among researchers, shock among investors, and responses from AI heavyweights. The company followed up on January 28 with that can work with images as well as text.
So what has DeepSeek done, and how did it do it?
What DeepSeek did
In December, DeepSeek released its . This is a very powerful 鈥渟tandard鈥 large language model that performs at a similar level to OpenAI鈥檚 GPT-4o and Anthropic鈥檚 Claude 3.5.
While these models are prone to errors and , they can carry out tasks such as answering questions, writing essays and generating computer code. On of problem-solving and mathematical reasoning, they score better than the average human.
V3 was trained at a of about US$5.58 million. This is dramatically cheaper than GPT-4, for example, which cost to develop.
DeepSeek also claims to have trained V3 using around 2,000 specialised computer chips, specifically . This is again much fewer than other companies, which may have used of the more powerful H100 chips.
On January 20, DeepSeek released another model, . This is a so-called 鈥渞easoning鈥 model, which tries to work through complex problems step by step. These models seem to be better at many tasks that require context and have multiple interrelated parts, such as reading comprehension and strategic planning.
The R1 model is a tweaked version of V3, modified with a technique called reinforcement learning. R1 appears to work at a similar level to , released last year.
DeepSeek also used the same technique to make 鈥渞easoning鈥 versions of small open-source models that can run on home computers.
This release has sparked a huge surge of interest in DeepSeek, driving up the popularity of its and triggering a in tech stocks as investors re-evaluate the AI industry. At the time of writing, chipmaker NVIDIA in value.
[DeepSeek] V3 was trained at a reported cost of about US$5.58 million. This is dramatically cheaper than GPT-4, for example, which cost more than US$100 million to develop.
Associate Professor Tongliang Liu
How DeepSeek did it
DeepSeek鈥檚 breakthroughs have been in achieving greater efficiency: getting good results with fewer resources. In particular, DeepSeek鈥檚 developers have pioneered two techniques that may be adopted by AI researchers more broadly.
The first has to do with a mathematical idea called 鈥渟parsity鈥. AI models have a lot of parameters that determine their responses to inputs (V3 has around 671 billion), but only a small fraction of these parameters is used for any given input.
However, predicting which parameters will be needed isn鈥檛 easy. DeepSeek used a new technique to do this, and then trained only those parameters. As a result, its models needed far less training than a conventional approach.
The other trick has to do with how V3 stores information in computer memory. DeepSeek has found a clever way to compress the relevant data, so it is easier to store and access quickly.
DeepSeek has shaken up the multi-billion dollar AI industry.听
What it means
DeepSeek鈥檚 models and techniques have been released under the free , which means anyone can download and modify them.
While this may be bad news for some AI companies 鈥 whose profits might be eroded by the existence of freely available, powerful models 鈥 it is great news for the broader AI research community.
At present, a lot of AI research requires access to enormous amounts of computing resources. Researchers like myself who are based at universities (or anywhere except large tech companies) have had limited ability to carry out tests and experiments.
More efficient models and techniques change the situation. Experimentation and development may now be significantly easier for us.
For consumers, access to AI may also become cheaper. More AI models may be run on users鈥 own devices, such as laptops or phones, rather than running 鈥渋n the cloud鈥 for a subscription fee.
For researchers who already have a lot of resources, more efficiency may have less of an effect. It is unclear whether DeepSeek鈥檚 approach will help to make models with better performance overall, or simply models that are more efficient.
Manual Name : Associate Professor Tongliang Liu
Manual Description : Sydney AI Centre, Faculty of Engineering
Manual Address :
Manual Addition Info Title :
Manual Addition Info Content :
Profile image : /content/dam/people/students/profile-and-headshots/dr-tongliang-liu.jpg
Manual Type : profile
_self
Auto Type : contact
Auto Addition Title :
Auto Addition Content :
Auto Name : true
Auto Position : true
Auto Profile image :
Auto Phone Number : false
Auto Mobile Number : true
Auto Email Address : true
Auto Address : false
UUID :
Media contact
Manual Name : Ivy Shih
Manual Description : Media Adviser (Engineering)
Manual Address :
Manual Addition Info Title :
Manual Addition Info Content :
Profile image :
Manual Type : contact
_self
Auto Type : contact
Auto Addition Title :
Auto Addition Content :
Auto Name : false
Auto Position : false
Auto Profile image :
Auto Phone Number : false
Auto Mobile Number : true
Auto Email Address : true
Auto Address : false
UUID : I-SHIH1