I have an HP Z2 G9 where I’ve replaced the original A2000 GPU with a 5060Ti. I’m pondering getting another 5060Ti to be able to go up to dense models instead of MoE, but the PCIe options on this motherboard aren’t the best. I will need to use a PCIe extender no matter what I opt for since there’s physically no room for another GPU (the 5060 I have is 2.5 slot sized) and so I’m wondering whether I would even gain something from using an M2 slot adapter instead.

Any insight welcome.

(1) PCI Express Gen5 slot x16 mechanical/ x16 electrical (full height, full length)
(1) PCI Express Gen3 slot x4 mechanical/ x1 electrical (full height, full length, open-ended)
(1) PCI Express Gen3 slot x16 mechanical/ x4 electrical (full height, full length)
(1) PCI Express Gen3 slot x4 mechanical/ x4 electrical (full height, full length, open-ended)
(1) M.2 2280 Storage (PCIe Gen4 x4)
(1) M.2 2280 Storage (PCIe Gen4 x4)
(1) M.2 2280 Storage (PCIe Gen4 x4)
  • TheMightyCat@ani.socialEnglish
    3·
    18 days ago

    Yes, if you use layer parallelism. When using tensor parallelism PCIe bandwith is a very important factor.

    • MalReynolds@slrpnk.netEnglish
      2·
      18 days ago

      True, but not hugely relevant to models fitting in 32GB as I understand it, more for improving latencies in larger models (120B+) or batch processing. Nonetheless layer will work for OPs use case.