ai/gemma3-qat repository overview

⁠Gemma 3 QAT (Quantization Aware Trained) - Instruct

GGUF version by Unsloth

Quantization Aware Trained (QAT) Gemma 3 checkpoints. The model preserves similar quality as half precision while using 3x less memory.

These are instruction tuned variants of the Gemma3 QAT models.

Gemma is a versatile AI model family designed for tasks like question answering, summarization, and reasoning. With open weights and responsible commercial use, it supports image-text input, a 128K token context, and over 140 languages.

⁠Intended uses

Gemma 3 4B model can be used for:

• Text generation: Create poems, scripts, code, marketing copy, and email drafts.
• Chatbots and conversational AI: Enable virtual assistants and customer service bots.
• Text summarization: Produce concise summaries of reports and research papers.
• Image data extraction: Interpret and summarize visual data for text-based communication.
• Language learning tools: Aid in grammar correction and interactive writing practice.
• Knowledge exploration: Assist researchers by generating summaries and answering questions.

⁠Characteristics

⁠Available model variants

¹: VRAM estimated based on model characteristics.

⁠Use this AI model with Docker Model Runner

First, pull the model:

docker model pull ai/gemma3-qat

Then run the model:

docker model run ai/gemma3-qat

For more information on Docker Model Runner, explore the documentation⁠.

⁠Benchmark performance

⁠Links

• Gemma 3 Model Overview⁠
• Gemma 3 Technical Report⁠
• Unsloth Dynamic 2.0 GGUF⁠