Initial production-ready Gemma 3 vLLM ROCm stack

Co-Authored-By: Oz <oz-agent@warp.dev>
2026-04-18 22:53:38 +05:30
commit ef8537e923
18 changed files with 988 additions and 0 deletions
--- a/.env.example
+++ b/.env.example
@ -0,0 +1,9 @@
+HF_TOKEN=YOUR_HF_TOKEN_HERE
+VLLM_API_KEY=YOUR_LOCAL_API_KEY_HERE
+GEMMA_MODEL_ID=google/gemma-3-1b-it
+BACKEND_PORT=8000
+FRONTEND_PORT=3000
+HUGGINGFACE_CACHE_DIR=/home/${USER}/.cache/huggingface
+OPEN_WEBUI_DATA_DIR=./frontend/data/open-webui
+VLLM_MAX_MODEL_LEN=4096
+VLLM_GPU_MEMORY_UTILIZATION=0.88