SandLogicTechnologies commited on
Commit
62a8fe6
·
verified ·
1 Parent(s): 3a6a5cf

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +136 -136
README.md CHANGED
@@ -1,136 +1,136 @@
1
- ---
2
- license: apache-2.0
3
- language:
4
- - en
5
- base_model:
6
- - mistralai/Mistral-Nemo-12B-Base
7
- tags:
8
- - text-generation
9
- - chat
10
- - instruction-following
11
- - reasoning
12
- - coding
13
- ---
14
-
15
- # Mistral-NeMo-12B-Instruct
16
-
17
- Mistral-NeMo-12B-Instruct is an instruction-optimized large language model built on top of the Mistral-NeMo-12B foundation. It is tailored for dialogue systems, structured task execution, and complex reasoning scenarios. The model emphasizes clarity, stability across multi-turn interactions, and high-quality output generation for both technical and general-purpose tasks.
18
-
19
- For streamlined local deployment, GGUF quantized builds (including Q4_K_M and Q5_K_M) are commonly used to reduce memory demands and improve inference performance on consumer hardware.
20
-
21
- ---
22
-
23
- ## Overview
24
-
25
- - **Model name:** Mistral-NeMo-12B-Instruct
26
- - **Base architecture:** Decoder-only transformer
27
- - **Developer:** NVIDIA & Mistral AI
28
- - **License:** Apache-2.0
29
- - **Quantized Versions:**
30
- - Q4_K_M (4-bit)
31
- - Q5_K_M (5-bit)
32
- - **Parameter count:** ~12B
33
- - **Tokenizer:** NeMo-aligned tokenizer
34
- - **Model type:** Instruction-tuned language model
35
-
36
- ---
37
-
38
- ## Quantization Details
39
-
40
- ### Q4_K_M
41
-
42
- - Approx. ~70% size reduction (~6.96)
43
- - Reduced VRAM/RAM requirements
44
- - Suitable for CPU-heavy or limited-GPU setups
45
- - Faster inference speed
46
- - Slight quality trade-off in edge cases
47
-
48
- ### Q5_K_M
49
-
50
- - Approx. ~64% size reduction (~8.13)
51
- - Balanced compression and quality
52
- - Improved stability over 4-bit variants
53
- - Better retention of reasoning depth
54
- - Preferred when additional memory is available
55
-
56
- ---
57
-
58
- ## Key Features
59
-
60
- - **Instruction alignment**
61
- Tuned to respond accurately to detailed prompts.
62
-
63
- - **Dialogue consistency**
64
- Maintains structured context across extended exchanges.
65
-
66
- - **Problem-solving capability**
67
- Handles analytical and multi-step reasoning tasks.
68
-
69
- - **Programming support**
70
- Assists with code writing, refactoring, and explanations.
71
-
72
- - **Formatted output generation**
73
- Capable of structured responses when required.
74
-
75
- - **Extended context handling**
76
- Designed to manage long prompts efficiently.
77
-
78
- ---
79
-
80
- ## Training Details
81
-
82
- This model originates from the Mistral-NeMo-12B base and undergoes additional refinement to enhance responsiveness and alignment with user instructions.
83
-
84
- - Pretraining: Large-scale autoregressive training on diverse multilingual and code datasets.
85
- - Architecture: 12B parameter transformer decoder model.
86
- - Instruction Tuning: Supervised fine-tuning for improved instruction following and chat performance.
87
- - Context Length: Supports up to 128K token context window.
88
-
89
- ---
90
-
91
- ## Usage
92
-
93
- ### llama.cpp
94
-
95
- ```bash
96
- ./llama-cli \
97
- -m SandlogicTechnologies\Mistral-NeMo-12B-Instruct_Q4_K_M.gguf \
98
- -p "Summarize the key differences between CNNs and Transformers."
99
- ```
100
-
101
- ---
102
-
103
- ## Recommended Use Cases
104
-
105
- - **Conversational systems**
106
- Multi-turn assistant-style interactions.
107
-
108
- - **Technical documentation support**
109
- Drafting and summarizing structured material.
110
-
111
- - **Code-related workflows**
112
- Generating and reviewing source code.
113
-
114
- - **Analytical reasoning tasks**
115
- Assisting with structured logic and problem solving.
116
-
117
- - **Automation pipelines**
118
- Producing formatted outputs for downstream systems.
119
-
120
- ---
121
-
122
-
123
- ## Acknowledgments
124
-
125
- These quantized models are based on the original work by **mistralai** development team.
126
-
127
- Special thanks to:
128
- - The [mistralai](https://huggingface.co/mistralai) team for developing and releasing the [Mistral-NeMo-12B-Instruct](https://huggingface.co/nvidia/Mistral-NeMo-12B-Instruct) model.
129
-
130
- - **Georgi Gerganov** and the entire [`llama.cpp`](https://github.com/ggerganov/llama.cpp) open-source community for enabling efficient model quantization and inference via the GGUF format.
131
-
132
- ---
133
-
134
-
135
- ## Contact
136
- For any inquiries or support, please contact us at support@sandlogic.com or visit our [Website](https://www.sandlogic.com/).
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ base_model:
6
+ - mistralai/Mistral-Nemo-12B-Base
7
+ tags:
8
+ - text-generation
9
+ - chat
10
+ - instruction-following
11
+ - reasoning
12
+ - coding
13
+ ---
14
+
15
+ # Mistral-NeMo-12B-Instruct
16
+
17
+ Mistral-NeMo-12B-Instruct is an instruction-optimized large language model built on top of the Mistral-NeMo-12B foundation. It is tailored for dialogue systems, structured task execution, and complex reasoning scenarios. The model emphasizes clarity, stability across multi-turn interactions, and high-quality output generation for both technical and general-purpose tasks.
18
+
19
+ For streamlined local deployment, GGUF quantized builds (including Q4_K_M and Q5_K_M) are commonly used to reduce memory demands and improve inference performance on consumer hardware.
20
+
21
+ ---
22
+
23
+ ## Overview
24
+
25
+ - **Model name:** Mistral-NeMo-12B-Instruct
26
+ - **Base architecture:** Decoder-only transformer
27
+ - **Developer:** NVIDIA & Mistral AI
28
+ - **License:** Apache-2.0
29
+ - **Quantized Versions:**
30
+ - Q4_K_M (4-bit)
31
+ - Q5_K_M (5-bit)
32
+ - **Parameter count:** ~12B
33
+ - **Tokenizer:** NeMo-aligned tokenizer
34
+ - **Model type:** Instruction-tuned language model
35
+
36
+ ---
37
+
38
+ ## Quantization Details
39
+
40
+ ### Q4_K_M
41
+
42
+ - Approx. ~ 70% size reduction (~ 6.96)
43
+ - Reduced VRAM/RAM requirements
44
+ - Suitable for CPU-heavy or limited-GPU setups
45
+ - Faster inference speed
46
+ - Slight quality trade-off in edge cases
47
+
48
+ ### Q5_K_M
49
+
50
+ - Approx. ~ 64% size reduction (~ 8.13)
51
+ - Balanced compression and quality
52
+ - Improved stability over 4-bit variants
53
+ - Better retention of reasoning depth
54
+ - Preferred when additional memory is available
55
+
56
+ ---
57
+
58
+ ## Key Features
59
+
60
+ - **Instruction alignment**
61
+ Tuned to respond accurately to detailed prompts.
62
+
63
+ - **Dialogue consistency**
64
+ Maintains structured context across extended exchanges.
65
+
66
+ - **Problem-solving capability**
67
+ Handles analytical and multi-step reasoning tasks.
68
+
69
+ - **Programming support**
70
+ Assists with code writing, refactoring, and explanations.
71
+
72
+ - **Formatted output generation**
73
+ Capable of structured responses when required.
74
+
75
+ - **Extended context handling**
76
+ Designed to manage long prompts efficiently.
77
+
78
+ ---
79
+
80
+ ## Training Details
81
+
82
+ This model originates from the Mistral-NeMo-12B base and undergoes additional refinement to enhance responsiveness and alignment with user instructions.
83
+
84
+ - Pretraining: Large-scale autoregressive training on diverse multilingual and code datasets.
85
+ - Architecture: 12B parameter transformer decoder model.
86
+ - Instruction Tuning: Supervised fine-tuning for improved instruction following and chat performance.
87
+ - Context Length: Supports up to 128K token context window.
88
+
89
+ ---
90
+
91
+ ## Usage
92
+
93
+ ### llama.cpp
94
+
95
+ ```bash
96
+ ./llama-cli \
97
+ -m SandlogicTechnologies\Mistral-NeMo-12B-Instruct_Q4_K_M.gguf \
98
+ -p "Summarize the key differences between CNNs and Transformers."
99
+ ```
100
+
101
+ ---
102
+
103
+ ## Recommended Use Cases
104
+
105
+ - **Conversational systems**
106
+ Multi-turn assistant-style interactions.
107
+
108
+ - **Technical documentation support**
109
+ Drafting and summarizing structured material.
110
+
111
+ - **Code-related workflows**
112
+ Generating and reviewing source code.
113
+
114
+ - **Analytical reasoning tasks**
115
+ Assisting with structured logic and problem solving.
116
+
117
+ - **Automation pipelines**
118
+ Producing formatted outputs for downstream systems.
119
+
120
+ ---
121
+
122
+
123
+ ## Acknowledgments
124
+
125
+ These quantized models are based on the original work by **mistralai** development team.
126
+
127
+ Special thanks to:
128
+ - The [mistralai](https://huggingface.co/mistralai) team for developing and releasing the [Mistral-NeMo-12B-Instruct](https://huggingface.co/nvidia/Mistral-NeMo-12B-Instruct) model.
129
+
130
+ - **Georgi Gerganov** and the entire [`llama.cpp`](https://github.com/ggerganov/llama.cpp) open-source community for enabling efficient model quantization and inference via the GGUF format.
131
+
132
+ ---
133
+
134
+
135
+ ## Contact
136
+ For any inquiries or support, please contact us at support@sandlogic.com or visit our [Website](https://www.sandlogic.com/).