35 points | by zyoralabs 5 hours ago ago
1 comments
This is so freaking awesome, I am working on a project trying run 10 models on two GPUs, loading/off loading is the only solution I have in mind.
Will try getting this deployed.
Does cold start timings advertised for a condition where there is no other model loaded on GPUs?
This is so freaking awesome, I am working on a project trying run 10 models on two GPUs, loading/off loading is the only solution I have in mind.
Will try getting this deployed.
Does cold start timings advertised for a condition where there is no other model loaded on GPUs?