Model Card for Jay24-AI/bloom-3b-lora-tagger

This model is a LoRA fine-tuned version of BigScience’s BLOOM-3B model, trained on a dataset of English quotes. The goal was to adapt BLOOM using the PEFT (Parameter-Efficient Fine-Tuning) approach with LoRA, making it lightweight to train and efficient for deployment.

---

Model Details

Model Description

• Developed by: Jay24-AI
• Funded by [optional]: N/A
• Shared by [optional]: Jay24-AI
• Model type: Causal Language Model with LoRA adapters
• Language(s): English
• License: [Add the license you intend to apply — BLOOM is under the RAIL license, LoRA adapters are usually MIT-compatible]
• Finetuned from model: bigscience/bloom-3b

Model Sources

• Repository: https://huggingface.co/Jay24-AI/bloom-3b-lora-tagger

---

Uses

Direct Use

The model can be used for text generation and tagging based on quote-like prompts.
For example, you can input a quote, and the model will generate descriptive tags.

Downstream Use

• Can be further fine-tuned on custom tagging or classification datasets.
• Could be integrated into applications that require lightweight quote classification, text annotation, or prompt-based generation.

Out-of-Scope Use

• Not suitable for factual question answering.
• Not designed for sensitive or high-stakes decision-making (e.g., medical, legal, or financial advice).

---

Bias, Risks, and Limitations

• Inherits limitations and biases from BLOOM-3B (which was trained on large-scale internet data).
• The fine-tuned dataset (English quotes) is small (~1k samples), so the model may overfit and generalize poorly outside similar data.
• Risk of generating irrelevant or biased tags if prompted outside the intended scope.

Recommendations

Users should:

• Validate outputs before production use.
• Avoid relying on the model for critical applications.

---

How to Get Started with the Model

import torch
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer

peft_model_id = "Jay24-AI/bloom-3b-lora-tagger"
config = PeftConfig.from_pretrained(peft_model_id)
model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path, return_dict=True, load_in_8bit=True, device_map='auto')
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)

# Load the Lora model
model = PeftModel.from_pretrained(model, peft_model_id)

batch = tokenizer("“The only way to do great work is to love what you do.” ->:", return_tensors='pt')

with torch.cuda.amp.autocast():
  output_tokens = model.generate(**batch, max_new_tokens=50)

print('\n\n', tokenizer.decode(output_tokens[0], skip_special_tokens=True))

Training Details

Training Data

• Dataset used: Abirate/english_quotes
• Subset: First 1,000 samples (train[:1000]).
• Structure: Each entry includes a quote and its corresponding tags.
• Preprocessing:
  • Combined the quote and tags into a single text string:

    "<quote>" ->: <tags>
    

  • Tokenized using the AutoTokenizer from bigscience/bloom-3b.
  • Applied batching via Hugging Face datasets.map with batched=True.

---

Training Procedure

Preprocessing

• Converted text examples into the "quote ->: tags" format.
• Tokenized using Bloom’s tokenizer with default settings.
• Applied DataCollatorForLanguageModeling with mlm=False (causal LM objective).

Training Hyperparameters

• Base model: bigscience/bloom-3b
• Adapter method: LoRA via PEFT
• LoRA configuration:
  • r = 16
  • lora_alpha = 32
  • lora_dropout = 0.05
  • bias = "none"
  • task_type = "CAUSAL_LM"
• TrainingArguments:
  • per_device_train_batch_size = 4
  • gradient_accumulation_steps = 4
  • warmup_steps = 100
  • max_steps = 200
  • learning_rate = 2e-4
  • fp16 = True
  • logging_steps = 1
  • output_dir = outputs/
• Precision regime: Mixed precision (fp16).
• Caching: model.config.use_cache = False during training to suppress warnings.

Hyperparameter Summary

Hyperparameter	Value
Base model	bigscience/bloom-3b
Adapter method	LoRA (via PEFT)
LoRA r	16
LoRA alpha	32
LoRA dropout	0.05
Bias	none
Task type	Causal LM
Batch size (per device)	4
Gradient accumulation steps	4
Effective batch size	16
Warmup steps	100
Max steps	200
Learning rate	2e-4
Precision	fp16 (mixed precision)
Logging steps	1
Output directory	outputs/
Gradient checkpointing	Enabled
Use cache	False (during training)

---

Speeds, Sizes, Times

• Trainable parameters: LoRA adapters only (a small fraction of BLOOM-3B).
• Approx. size: Much smaller than 3B full checkpoint since only adapters are stored.
• Max steps: 200 (~250 updates with gradient accumulation).
• Training runtime: Depends on GPU (not logged in script).
• Batch size effective: 16 (4 × accumulation steps of 4).

---

Compute Infrastructure

• Hardware: Single CUDA GPU T4 (set with os.environ["CUDA_VISIBLE_DEVICES"]="0").
• Software:
  • PyTorch (torch)
  • Hugging Face Transformers (main branch from GitHub)
  • Hugging Face PEFT (main branch from GitHub)
  • Hugging Face Datasets
  • Accelerate
  • Bitsandbytes (for 8-bit loading)
• Gradient checkpointing: Enabled to save memory.
• Mixed precision: Enabled with fp16.

---

Evaluation

Testing Data

• Same dataset (Abirate/english_quotes).
• No held-out test set reported in training script.

Metrics

• No formal metrics logged; evaluation was qualitative (checking generated tags).

Results

• The model successfully learns to generate tags for English quotes after training.

---

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator.

• Hardware Type: Single CUDA GPU T4
• Cloud Provider: Colab

---

Technical Specifications

Model Architecture and Objective

• Base model: BLOOM-3B, causal language modeling objective.
• Fine-tuned with LoRA adapters using PEFT.

Compute Infrastructure

• Hardware: Single GPU (CUDA device 0).
• Software:
  • PyTorch
  • Hugging Face Transformers
  • Hugging Face PEFT
  • Hugging Face Datasets
  • Accelerate
  • Bitsandbytes

---

Citation

If you use this model, please cite:

BibTeX:

@misc{jay24ai2025bloomlora,
  title={LoRA Fine-Tuned BLOOM-3B for Quote Tagging},
  author={Jay24-AI},
  year={2025},
  howpublished={\url{https://huggingface.co/Jay24-AI/bloom-3b-lora-tagger}}
}

---

## Model Card Contact

For questions or issues, contact the maintainer via Hugging Face discussions: https://huggingface.co/Jay24-AI/bloom-3b-lora-tagger/discussions