git/gemma3-vllm-stack
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial production-ready Gemma 3 vLLM ROCm stack

Browse Source 
Co-Authored-By: Oz <oz-agent@warp.dev>
This commit is contained in:
Raghav
2026-04-18 22:53:38 +05:30
commit ef8537e923
18 changed files with 988 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
backend/config/model.env.example Normal file
View File

@ -0,0 +1,7 @@
HF_TOKEN=YOUR_HF_TOKEN_HERE
VLLM_API_KEY=YOUR_LOCAL_API_KEY_HERE
GEMMA_MODEL_ID=google/gemma-3-1b-it
BACKEND_PORT=8000
HUGGINGFACE_CACHE_DIR=/home/${USER}/.cache/huggingface
VLLM_MAX_MODEL_LEN=4096
VLLM_GPU_MEMORY_UTILIZATION=0.88