I had one of these on pre-order/reservation from when they announced the DGX Spark and ended up returning it after a couple days. I thought I'd give it a shot, though. The 128GB of unified memory was the big selling point (as are any of the DGX Spark boxes), but the memory bandwidth was very disappointing. Being able to load a 100B+ parameter model was cool in terms of novelty but not particularly great for local inferencing.
Also, NVIDIA's software they have you install on another machine to use it is garbage. They tried to make it sort of appliance-y but most people would rather just have SSH work out of the box and can go from there. IMO just totally unnecessary. The software aspect was what put me over the edge.
Maybe the gen 2 will be better, but unless you have a really specific use case that this solves well, buy credits or something somewhere else.
Also, NVIDIA's software they have you install on another machine to use it is garbage. They tried to make it sort of appliance-y but most people would rather just have SSH work out of the box and can go from there. IMO just totally unnecessary. The software aspect was what put me over the edge.
Maybe the gen 2 will be better, but unless you have a really specific use case that this solves well, buy credits or something somewhere else.