chipforge-llm

LoRA adapters for Verilog HDL generation, fine-tuned on Nemotron-30B.

phase_1/v0.1/

Field	Value
Base model	NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
Training data	36,321 Verilog modules (filtered ≤10k tokens)
Method	LoRA CPT (Continued Pre-Training)
LoRA rank	8
Trainable params	~4.5M (0.015% of 30B)
Max seq len	1024
Epochs	2
Final train loss	0.42
Final eval loss	0.43
Token accuracy	~89%
Hardware	2x A100-80GB, FSDP

Usage

``\python from peft import PeftModel from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained( "NVIDIA/Nemotron-3-Nano-30B-A3B-BF16", torch_dtype="bfloat16", device_map="auto", trust_remote_code=True, ) model = PeftModel.from_pretrained(model, "Ashx098/chipforge-llm/phase_1/v0.1") tokenizer = AutoTokenizer.from_pretrained( "NVIDIA/Nemotron-3-Nano-30B-A3B-BF16", trust_remote_code=True, )

### wandb

Run: https://wandb.ai/avinash-mynampati-juspay/vaschpforge-llm/runs/c5gid36w