Large Language Model Servers
These rackmount AI servers offer high GPU memory capacities in order to facilitate inference and training with cutting-edge large language models (LLMs).
These servers are designed to run large language models on-premises, but if you aren’t there yet we also have workstations for AI development.
|
|
|
|
|---|---|---|---|
Budget LLM Server |
4 GPU LLM Server |
8 GPU LLM Server |
|
|
Puget’s Take |
|||
| Puget’s Take |
Affordable option for fine-tuning and hosting smaller LLMs |
Compact 2U server for large language model inference |
Maximum GPU power in a 4U server for LLM inference and training |
|
CPU |
|||
| CPU | AMD EPYC 9274F | AMD EPYC 9354P | 2 x AMD EPYC 9354 |
|
GPUS(s) |
|||
| GPU(s) | 4 x NVIDIA GeForce RTX 4090 24GB | 4 x NVIDIA L40S 48GB | 8 x NVIDIA L40S 48GB |
|
RAM |
|||
| RAM | 192GB DDR5-4800 REG ECC (12x16GB) | 384GB DDR5-4800 REG ECC (12x32GB) | 768GB DDR5-4800 REG ECC (24x32GB) |
|
Features |
|||
| Features |
NVIDIA GeForce RTX 4080 / 4090 Provides up to 96 GB of VRAM Optionally run multiple small models |
NVIDIA RTX Ada and L40S Provides up to 192 GB of VRAM 70B model inference in fp16 with room for large context / KV cache |
NVIDIA RTX Ada, L40S, and H100 NVL Configurable up to 752 GB of VRAM 150B model inference in fp16 with room for large context / KV cache |
|
Price as Configured |
|||
| Price as Configured |
$21,424.56 |
$42,909.84 |
$84,037.71 |
|
Starting At |
|||
| Starting At |
$10,429.10 |
$8,513.39 |
$12,753.28 |
| Configure | Configure | Configure | |
Equipped to Serve Customers of Any Size
Puget Systems has specialists on staff who cater to the needs of businesses and educational institutions. We are listed on numerous purchasing portals and offer optional onsite support. Click through to read more about how we can help your organization!
Talk to an Expert
We specialize in building workstation PCs tailored for each of our customers. The best way we’ve found to accomplish that is to speak with you directly. There is no cost or obligation, and our no-pressure, non-commissioned consultants are experts at configuring a computer that will meet your specific needs. They are happy to discuss a quote you have already saved or guide you through each step of the process by asking a few questions about how you’ll be using your computer. There are several ways to start a conversation with us, so please pick what works best for you:
If you’d rather not wait, you can reach out to us via phone during our business hours.
Monday – Friday | 7am – 5pm (Pacific)
1-888-784-3872