Getting ChatGPT to run on a NAS is actually worth exploring — tech enthusiast puts an Nvidia RTX GPU in a 12-bay NAS powered by an AMD EPYC CPU, and the results are surprising
Nvidia’s ChatRTX software powered the AI chatbot
When you purchase through links on our site, we may earn an affiliate commission.Here’s how it works.
If you’ve ever wondered what happens when you drop a GPU into a NAS and get it running AI, like a privateChatGPT, thenStorageReviewhas the surprising answer.
For the base of his project, the site’sJordan Ranous used QNAP’s TS-h1290FX, a 12 NVMeNASpowered by anAMDEPYC 7302P CPU and boasting 256GB DRAM, 25GbE connectivity, and plenty of PCI slots. He chose that NAS as it has support for an internal GPU and the ability to host up to 737TB of raw storage.
By adding anNvidiaRTX A4000 GPU to the TS-h1290FX and configuring it for AI using Virtualization Station (a hypervisor for the NAS), Ranous was able to run AI workflows seamlessly.
Nvidia’s Chat with RTX
Nvidia’s ChatRTX software package sorted the AI interaction side by providing a customized experience through a GPT-based LLM with a local, unique dataset. This allowed for rapid, context-aware responses while maintaining privacy and security.
StorageReviewdetailed the process which involved verifying hardware compatibility, installing the GPU, updating QNAP firmware and software, and installing the OS on the VM, before configuring GPU passthrough, installing GPU drivers in the VM and verifying the passthrough functionality.
The ease of setting up the GPU for AI on the QNAP NAS proves it could work as a cost-effective and efficient solution for businesses looking to leverage the power of AI. As Ranous says, “We’ve shown that adding a decent GPU to a QNAP NAS is relatively easy and inexpensive. We put an A4000 to work, and with a street price of about $1050, that’s not bad when you consider Virtualization Station is free and NVIDIA ChatRTX is available at no charge.”
More from TechRadar Pro
Are you a pro? Subscribe to our newsletter
Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!
Wayne Williams is a freelancer writing news for TechRadar Pro. He has been writing about computers, technology, and the web for 30 years. In that time he wrote for most of the UK’s PC magazines, and launched, edited and published a number of them too.
Don’t search for information on cats at work — you could be at risk of being hacked
Key Strategies for financial institutions to combat fraud
Nvidia’s GeForce Now Priority membership has upgraded to ‘Performance’ - introducing a 1440p resolution and ultrawide support
