Update README.md

README.md

@@ -4,8 +4,119 @@ language:
 - en
 pipeline_tag: text-generation
 tags:
-- facebook
 - meta
 - Llama3
 - pytorch
----
+---
+# SandLogic Technology - Quantized Meta-Llama3-8b-Instruct Models
+
+## Model Description
+
+We have quantized the Meta-Llama3-8b-Instruct model into three variants:
+
+1. Q5_KM
+2. Q4_KM
+3. IQ4_XS
+
+These quantized models offer improved efficiency while maintaining performance.
+
+## Original Model Information
+
+- **Name**: Meta-Llama3-8b-Instruct
+- **Developer**: Meta
+- **Release Date**: April 18, 2024
+- **Model Type**: Auto-regressive language model
+- **Architecture**: Optimized transformer with Grouped-Query Attention (GQA)
+- **Parameters**: 8 billion
+- **Context Length**: 8k tokens
+- **Training Data**: New mix of publicly available online data (15T+ tokens)
+- **Knowledge Cutoff**: March, 2023
+
+## Model Capabilities
+
+Llama 3 is designed for multiple use cases, including:
+
+- Responding to questions in natural language
+- Writing code
+- Brainstorming ideas
+- Content creation
+- Summarization
+
+The model understands context and responds in a human-like manner, making it useful for various applications.
+
+## Use Cases
+
+1. **Chatbots**: Enhance customer service automation
+2. **Content Creation**: Generate articles, reports, blogs, and stories
+3. **Email Communication**: Draft emails and maintain consistent brand tone
+4. **Data Analysis Reports**: Summarize findings and create business performance reports
+5. **Code Generation**: Produce code snippets, identify bugs, and provide programming recommendations
+
+## Model Variants
+
+We offer three quantized versions of the Meta-Llama3-8b-Instruct model:
+
+1. **Q5_KM**: 5-bit quantization using the KM method
+2. **Q4_KM**: 4-bit quantization using the KM method
+3. **IQ4_XS**: 4-bit quantization using the IQ4_XS method
+
+These quantized models aim to reduce model size and improve inference speed while maintaining performance as close to the original model as possible.
+
+## Usage
+
+```bash
+pip install llama-cpp-python 
+```
+Please refer to the llama-cpp-python [documentation](https://llama-cpp-python.readthedocs.io/en/latest/) to install with GPU support.
+
+### Basic Text Completion
+Here's an example demonstrating how to use the high-level API for basic text completion:
+
+```bash
+from llama_cpp import Llama
+
+llm = Llama(
+    model_path="./models/7B/llama-model.gguf",
+    verbose=False,
+    # n_gpu_layers=-1, # Uncomment to use GPU acceleration
+    # n_ctx=2048, # Uncomment to increase the context window
+)
+
+output = llm(
+    "Q: Name the planets in the solar system? A: ", # Prompt
+    max_tokens=32, # Generate up to 32 tokens
+    stop=["Q:", "\n"], # Stop generating just before a new question
+    echo=False # Don't echo the prompt in the output
+)
+
+print(output["choices"][0]["text"])
+```
+
+## Download
+You can download `Llama` models in `gguf` format directly from Hugging Face using the `from_pretrained` method. This feature requires the `huggingface-hub` package.
+
+To install it, run: `pip install huggingface-hub`
+
+```bash
+from llama_cpp import Llama
+
+llm = Llama.from_pretrained(
+    repo_id="SandLogicTechnologies/Meta-Llama-3-8B-Instruct-GGUF",
+    filename="*Meta-Llama-3-8B-Instruct.Q5_K_M.gguf",
+    verbose=False
+)
+```
+By default, from_pretrained will download the model to the Hugging Face cache directory. You can manage installed model files using the huggingface-cli tool.
+
+
+## License
+
+A custom commercial license is available at: https://llama.meta.com/llama3/license
+
+## Acknowledgements
+
+We thank Meta for developing and releasing the original Llama 3 model.
+
+## Contact
+
+For any inquiries or support, please contact us at` **[email protected]`** or visit our [support page](https://www.sandlogic.com/LingoForge/support).