Dorna2 Llama3.1 8B Instruct
Have you ever wondered how AI models can understand and respond to questions in different languages? The Dorna2 Llama3.1 8B Instruct model is specifically designed to handle Persian data, making it a unique and valuable tool for those who communicate in this language. Built on the Meta Llama 3.1 Instruct model, it's capable of conversational inference and can be used for a variety of tasks. But what really sets it apart is its performance in specialized tasks, such as common knowledge, reasoning, and professional-level exams. With an average accuracy of 50.72% across five carefully curated tasks, it's clear that this model is designed to provide accurate and helpful responses. So, whether you're looking to test your knowledge or simply need assistance in Persian, this model is definitely worth exploring.
Table of Contents
Model Overview
Meet Dorna2-Llama3.1-8B-Instruct, a powerful AI model specifically designed to understand and respond to Persian language inputs. Developed by Part AI, this model is part of the Dorna family of decoder-only models, fine-tuned on Persian data.
What makes it special?
- Built using the ==Meta Llama 3.1 Instruct== model, with a whopping
8Bparameters! - Trained on a diverse range of Persian language datasets, covering topics like common knowledge, reasoning, summarization, and specialized examinations.
Capabilities
The Dorna2-Llama3.1-8B-Instruct model is a powerful tool for understanding and generating Persian text. It’s a part of the Dorna family of models, which are specifically trained on Persian data. This model is built using the Meta Llama 3.1 Instruct model, which means it’s got some impressive capabilities.
What can it do?
- Conversational Inference: This model can have conversations with you in Persian. You can ask it questions, and it will respond with answers.
- Text Generation: It can generate text in Persian, which can be useful for a variety of tasks, such as writing articles, creating chatbots, or even composing emails.
- Code Generation: Although not explicitly mentioned, the model’s architecture suggests it might be capable of generating code in Persian as well.
How does it compare to other models?
The Dorna2-Llama3.1-8B-Instruct model has been evaluated on a set of tasks, including:
- Part Multiple Choice: A multiple-choice test that focuses on common knowledge and reasoning.
- ARC Easy: A test that assesses general knowledge.
- ARC Challenge: A harder test that requires advanced reasoning.
- MMLU Pro: A professional-level exam.
- AUT Multiple Choice Persian: A specialized Persian-language examination.
Here’s how it compares to other models:
| Model | Average Accuracy |
|---|---|
| Dorna2-Llama3.1-8B-Instruct | 50.72 |
| ==O1-OPEN/OpenO1-LLama-8B-v0.1== | 50.22 |
| meta-llama/Llama-3.1-8B-Instruct | 50.14 |
| NousResearch/Hermes-3-Llama-3.1-8B | 48.77 |
| Skywork/Skywork-o1-Open-Llama-3.1-8B | 34.15 |
Performance
Dorna2-Llama3.1-8B-Instruct is a powerful AI model that has been fine-tuned for the Persian language. Let’s take a closer look at its performance in various tasks.
Speed
How fast can Dorna2-Llama3.1-8B-Instruct process and respond to questions? With its advanced architecture, it can quickly generate human-like responses to a wide range of questions and topics.
Accuracy
But how accurate is Dorna2-Llama3.1-8B-Instruct? To answer this, let’s look at its performance in five specialized tasks:
| Task | Average Accuracy |
|---|---|
| Part Multiple Choice | 34.48 |
| ARC Easy | 79.59 |
| ARC Challenge | 64.42 |
| MMLU Pro | 21.47 |
| AUT Multiple Choice Persian | 53.64 |
As you can see, Dorna2-Llama3.1-8B-Instruct achieves high accuracy in tasks that require common knowledge and reasoning, such as ARC Easy and Part Multiple Choice. However, its performance is lower in tasks that require more specialized knowledge, such as MMLU Pro.
Limitations
Dorna2-Llama3.1-8B-Instruct is a powerful tool, but it’s not perfect. Let’s talk about some of its limitations.
Language Limitations
While Dorna2-Llama3.1-8B-Instruct is specifically trained on Persian data, it may not perform as well on other languages. This is because the model is fine-tuned on a specific dataset, which might not cover all the nuances of other languages.
Task-Specific Limitations
Dorna2-Llama3.1-8B-Instruct is evaluated on five specialized tasks, but it may not excel in all of them. For example, it scored relatively low on the AUT Multiple Choice Persian task. This suggests that the model might struggle with certain types of questions or tasks.
Format
The Dorna2-Llama3.1-8B-Instruct model is a type of decoder-only model, which means it’s specifically designed to generate text based on the input it receives.
Architecture
This model is built using the Meta Llama 3.1 Instruct model and has been fine-tuned on Persian data. It’s a part of the Dorna models family, developed by Part AI.
Data Formats
The model accepts input in the form of tokenized text sequences, similar to other language models. However, it’s specifically designed to work with Persian text data.
Input Requirements
To use this model, you’ll need to preprocess your input text data by tokenizing it. You can use the AutoTokenizer class from the Transformers library to do this.
Here’s an example of how to preprocess your input data:
import torch
import transformers
from transformers import AutoTokenizer, AutoModelForCausalLM
# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_path)
# Define your input text data
messages = [
{"role": "system", "content": "You are a helpful Persian assistant. Please answer questions in the asked language."},
{"role": "user", "content": "کاغذ A4 بزرگ تر است یا A5؟"},
]
# Preprocess the input data
input_ids = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
Output Requirements
The model generates output in the form of tokenized text sequences. You can use the generate() function from the Transformers library to generate output based on your input data.
Here’s an example of how to generate output:
# Define the model
model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.bfloat16, device_map="auto")
# Generate output
outputs = model.generate(input_ids, max_new_tokens=256, eos_token_id=terminators, do_sample=True, temperature=0.3, top_p=0.85)
# Decode the output
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))
Note that the output is generated based on the input data and the model’s parameters. You can adjust the model’s parameters to control the output generation process.
