Run Deepseek locally on a 24g GPU: Quantizing on our Giga Computing 6980P Xeon

Опубликовано 19 июня 2025, 16:30

With the help of some forum friends, Wendell now has the smallest model of Deepseek so far! Thank you Giga Computing, Kioxia, and Intel, for lending us all the goods to use for our testing!

Hugging Face Link huggingface.co/ubergarm/DeepSe...

Forum Link forum.level1techs.com/t/deepse...

The Chassis gigabyte.com/Enterprise/Rack-S...

Kioxia americas.kioxia.com/en-us/busi...

0:00 Intro
2:17 Online vs Local
3:40 Hardware
9:36 Using the Machine and Results
18:00 Conclusion


youtube.com/@Level1LinksWithFr...
Go Subscribe to our New Links with Friends Channel!
Starting in July, we're going to post links with friends on its own channel. This is still Level1Techs, we’re just separating the content types so they can take their own algorithm journeys. We also plan on posting more videos with the trio on that page!

You can find us...
Twitter - twitter.com/level1techs
Twitch - twitch.tv/teampgp
Patreon - patreon.com/level1

For all our social links, websites, and more, check out our link tree!
linktr.ee/level1techs

Thank you for watching!

-------------------------------------------------------------------------------------------------------------
*IMPORTANT* Any email lacking “level1techs.com” should be ignored and immediately reported to Queries@level1techs.com.