grok-2 on Hugging Face
10 hours ago
- #Grok-2
- #AI
- #xAI
- Grok 2 is a model trained and used at xAI in 2024.
- Weights can be downloaded using the command: `hf download xai-org/grok-2 --local-dir /local/grok-2`.
- The downloaded folder should contain 42 files and be approximately 500 GB.
- To launch an inference server, use SGLang (>= v0.5.1) with 8 GPUs (each > 40GB memory).
- Example command to launch server: `python3 -m sglang.launch_server --model /local/grok-2 --tokenizer-path /local/grok-2/tokenizer.tok.json --tp 8 --quantization fp8 --attention-backend triton`.
- To test the model, use the command: `python3 -m sglang.test.send_one --prompt "Human: What is your name?<|separator|>\n\nAssistant:"`.
- The model outputs its name, Grok, when asked.
- Weights are licensed under the Grok 2 Community License Agreement.
- Downloads last month: 33.