Documentation IndexFetch the complete documentation index at: /llms.txtUse this file to discover all available pages before exploring further.
Fetch the complete documentation index at: /llms.txt
Use this file to discover all available pages before exploring further.
from agno.agent import Agent from agno.models.vllm import VLLM agent = Agent( model=VLLM(id="Qwen/Qwen2.5-7B-Instruct", top_k=20, enable_thinking=False), markdown=True, ) agent.print_response("Share a 2 sentence horror story")
Set up your virtual environment
uv venv --python 3.12 source .venv/bin/activate
uv venv --python 3.12 .venv\Scripts\activate
Setup vLLM Server
uv pip install vllm python -m vllm.entrypoints.openai.api_server \ --model Qwen/Qwen2.5-7B-Instruct \ --port 8000
Install dependencies
uv pip install -U openai agno
Run Agent
python cookbook/11_models/vllm/basic.py
Was this page helpful?
