一、环境配置

1.1 拉取镜像文件

docker pull lmsysorg/sglang:latest

1.2 启动容器

docker run -it --name container_name --gpus all -P -v /home/weights:/home/ --ipc=host --network=host --privileged be46eaca2eff

/home/weights :存放权重文件，容器的home下被绑定

1.3 进入容器

进入容器，并切换到home目录下

docker exec -it container_name /bin/bash
cd /home

二、模型启动

1.1 直接启动

CUDA_VISIBLE_DEVICES="0"  python3 -m sglang.launch_server \
--model DeepSeek-R1-Distill-Qwen-14B  \
--tp 1 \
--trust-remote-code  \
--dtype bfloat16  \
--context-length 65536  \
--enable-p2p-check \
--host 0.0.0.0  \
--port 8866  \
--mem-fraction-static 0.8  \
--api-key  yoursecret  \
--max-running-request  1000 

1.2 后台启动

CUDA_VISIBLE_DEVICES="0" nohup  python3 -m sglang.launch_server \
--model DeepSeek-R1-Distill-Qwen-32B \
--tp 1 \
--trust-remote-code  \
--dtype bfloat16  \
--context-length 65536  \
--enable-p2p-check \
--host 0.0.0.0  \
--port 8866  \
--mem-fraction-static 0.8  \
--api-key  yoursecret  \
--max_running_requests 1000  > loging/mylog.log 2>&1 &

三、模型调用

3.1 Openai调用

import openai
client = openai.Client(base_url=f"http://10.101.21.10:8866/v1", api_key="yoursecret")

response = client.chat.completions.create(
    model="DeepSeek-R1-Distill-Qwen-14B",
    messages=[{"role": "user", "content": "中国的首都再哪里"}, ],
    temperature=0.6,
    max_tokens=1024,
)
# print_highlight(f"Response: {response}")
print(f"Response: {response}")

3.2 requests 调用

下面脚本中的ip_ad表示ip地址，比如10.101.21.10

import requests
# 定义API的URL和API Key
 url = "http://10.101.21.10:8866/v1/chat/completions"
 api_key = "yoursecret"
 # 定义请求头
 headers = {
     "Content-Type": "application/json",
     "Authorization": f"Bearer {api_key}"
 }
 # 定义请求体
 data = {
     "model": "DeepSeek-R1-Distill-Qwen-14B",
     "messages": [{"role": "user", "content": "中国的首都再哪里"}],
     "temperature": 0.6,
     "max_tokens": 1024
 }
 # 发送POST请求
 response = requests.post(url, headers=headers, json=data)
 # 打印响应结果
 if response.status_code == 200:
     print("Response:", response.json())
 else:
     print("Error:", response.status_code, response.text)

3.3 curl命令直接调用

curl -X POST "http://10.101.21.10:8099/v1/chat/completions" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer yoursecret" \
-d '{
  "model": "DeepSeek-R1-Distill-Qwen-14B",
  "messages": [{"role": "user", "content": "中国的首都再哪里"}],
  "temperature": 0.6,
  "max_tokens": 1024
}'