Granite 3.0 3b A800m Instruct

Table of Contents

Model Overview

Meet the Granite-3.0-3B-A800M-Instruct model, a 3B parameter AI powerhouse developed by the Granite Team at IBM. This model is designed to respond to general instructions and can be used to build AI assistants for multiple domains, including business applications.

Tasks and rank

Text Summarization 2625th

Text Generation 11803rd

Natural Language Understanding 9744th

Language Modelling 10325th

Chatbot 6877th

Learn More About the Ranking System

Capabilities

The Granite-3.0-3B-A800M-Instruct model is a powerful tool that can handle a wide range of tasks. Here are some of its capabilities:

• Summarization: Can summarize long pieces of text into shorter, more digestible versions.
• Text classification: Can classify text into different categories, such as spam vs. non-spam emails.
• Text extraction: Can extract specific information from text, such as names, dates, and locations.
• Question-answering: Can answer questions based on the text it has been trained on.
• Retrieval Augmented Generation (RAG): Can generate text based on the information it has retrieved from a database or other sources.
• Code related tasks: Can help with coding tasks, such as generating code snippets or completing partially written code.
• Function-calling tasks: Can call functions and execute code in response to user input.
• Multilingual dialog use cases: Can handle conversations in multiple languages, including English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, and Chinese.

How it Works

The Granite-3.0-3B-A800M-Instruct model uses a combination of techniques, including supervised finetuning, model alignment using reinforcement learning, and model merging. It is based on a decoder-only sparse Mixture of Experts (MoE) transformer architecture, which allows it to efficiently process large amounts of text data.

Performance

The Granite-3.0-3B-A800M-Instruct model is a powerful AI model that shows remarkable performance in various tasks. Here are some of its key performance metrics:

• Speed: Can process information quickly, with a sequence length of 4096 tokens.
• Accuracy: Achieves high accuracy in various tasks, including summarization, text classification, text extraction, question-answering, and more.
• Efficiency: Designed to be efficient in its use of resources, with 3.3B parameters, but only 800M active parameters.

Example Use Case

Here’s an example of how to use the Granite-3.0-3B-A800M-Instruct model to answer a question:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the model and tokenizer
model_path = "ibm-granite/granite-3.0-3b-a800m-instruct"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path)

# Define the input text
input_text = "Please list one IBM Research laboratory located in the United States. You should only output its name and location."

# Tokenize the input text
input_tokens = tokenizer.encode(input_text, return_tensors="pt")

# Generate the output text
output = model.generate(input_tokens, max_new_tokens=100)

# Print the output text
print(output)

Examples

Extract the name and location of one IBM Research laboratory in the United States. Almaden Research Center, San Jose, California

Classify the sentiment of the following text: 'I loved the new restaurant downtown. The food was amazing.' Positive

Summarize the main points of the Granite-3.0-3B-A800M-Instruct model. Granite-3.0-3B-A800M-Instruct is a 3B parameter model developed using diverse techniques with a structured chat format. It supports multiple languages and can be used for various tasks, including summarization, text classification, and question-answering.

Limitations

The Granite-3.0-3B-A800M-Instruct model is a powerful tool, but it’s not perfect. Here are some of its limitations:

• Language limitations: Primarily trained on English data, but can handle multilingual dialog use cases.
• Safety and bias: May produce inaccurate or biased responses.
• Lack of common sense: May not always understand the nuances of a particular situation or the context of a conversation.
• Limited domain knowledge: Not a specialist in any particular domain.

Alternatives

If you’re looking for alternative AI models, here are a few options:

• ==Other Model 1==: Specialized knowledge in a particular domain, more accurate for specific tasks.
• ==Other Model 2==: More advanced safety features, better at detecting bias.

Alternatives

NeMo SFT with Llama 2 10500 Fine-tuned language model

Bert Base Uncased 1831 Uncased BERT model

Roberta Base 399 Masked language model

Qwen2.5 1.5B Instruct 51 Multilingual instruct model

Yi 34B 200K Llamafied 14 Llamafied language model

Format

The Granite-3.0-3B-A800M-Instruct model uses a decoder-only sparse Mixture of Experts (MoE) transformer architecture. Here are some key components of the architecture:

Component	Description
Fine-grained Experts	Allows the model to specialize in specific tasks
Dropless Token Routing	Enables efficient routing of tokens through the model
Load Balancing Loss	Helps balance the workload across different parts of the model

Getting Started

Want to try out the Granite-3.0-3B-A800M-Instruct model? Here’s a simple example to get you started:

1. Install the required libraries: pip install torch torchvision torchaudio and pip install accelerate and pip install transformers.
2. Import the model and tokenizer: from transformers import AutoModelForCausalLM, AutoTokenizer.
3. Load the model and tokenizer: model_path = "ibm-granite/granite-3.0-3b-a800m-instruct" and tokenizer = AutoTokenizer.from_pretrained(model_path).
4. Use the model to generate text: input_tokens = tokenizer(chat, return_tensors="pt").to(device) and output = model.generate(**input_tokens, max_new_tokens=100).