Fix ZeroGPU CUDA initialization after CPU embedding

#1

This PR adds a small compatibility workaround in app.py.

During CPU E5 inference, Transformers calls torch.cuda.is_current_stream_capturing() while the ZeroGPU parent process has no GPU assigned. Although the resulting exception is caught, the next ZeroGPU worker fails with:

RuntimeError: No CUDA GPUs are available

The workaround skips this CUDA stream-capture check only when running in the GPU-less ZeroGPU parent process. Allocated GPU workers keep the original Transformers behavior.

Verified with startup corpus encoding, FAISS retrieval, lazy examples, and Gemma generation.

Related investigation: zero-gpu-explorers/README Discussion #181.

Amazing work

HateezAbullah changed pull request status to merged

Thank you so much for digging into this and fixing it โ€” really appreciate the detailed root-cause writeup too, learned a lot from it. It's working perfectly now

Sign up or log in to comment