tax-reranker-minilm

A cross-encoder reranker for questions about India's Income-Tax Act, 2025, fine-tuned from cross-encoder/ms-marco-MiniLM-L-6-v2 (22M parameters). It reorders passages retrieved by a first-stage retriever and is used as the second stage of TaxRAG.

Training

  • Data: questions written from the content of each section of the Act by Qwen3-4B-Instruct (not from section headings). Questions that mention a section number or mostly repeat the heading were removed. Sections were split into disjoint train/test groups; this model saw only questions from the 327 training sections.
  • Pairs: 2,765 (each training question with its gold passage as the positive and 4 hard negatives: the top passages from other sections returned by the FiQA-tuned bge-small retriever).
  • Objective: binary cross-entropy on (question, passage) pairs, 2 epochs, AdamW lr 2e-5, batch 16, max length 384, fp16, one T4 GPU.

Evaluation

277 questions from 150 held-out sections (never seen in training); gold = the section a question was written from; section-level metrics over 1,237 passages.

Pipeline Recall@1 Recall@5 MRR@10
Dense only (FiQA bge-small) 0.502 0.773 0.632
Dense top-30 β†’ ms-marco-MiniLM (base) 0.726 0.903 0.796
Dense top-30 β†’ this model 0.758 0.906 0.820
Dense + BM25 (RRF) top-30 β†’ this model 0.773 0.910 0.833

Both rerankers improve strongly on dense retrieval (exact McNemar on Recall@5 vs dense only, p < 1e-8). Over the base ms-marco reranker the gain is modest (+0.024 MRR@10).

Limitation. On an older, easier test set of 100 questions paraphrased from section headings, this model is weaker than the base reranker (MRR@10 0.843 vs 0.907): it was tuned on content-style questions and generalises less well to heading-style phrasing.

Usage

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained("vivekkopthsd/tax-reranker-minilm")
model = AutoModelForSequenceClassification.from_pretrained("vivekkopthsd/tax-reranker-minilm").eval()

query = "If I receive rent this year that was due in earlier years, how is it taxed?"
passages = ["Arrears of rent and unrealised rent ...", "Determination of annual value ..."]
enc = tok([query] * len(passages), passages, truncation=True, max_length=256, padding=True, return_tensors="pt")
with torch.no_grad():
    scores = model(**enc).logits.squeeze(-1)
ranked = [passages[i] for i in scores.argsort(descending=True)]

On CPU, reranking 15 candidates at 256 tokens takes about 2.5 s on a 4-core laptop.

Intended use

Ranking passages of the Income-Tax Act, 2025 for question answering. Not legal or tax advice.

Downloads last month
32
Safetensors
Model size
22.7M params
Tensor type
F32
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for vivekkopthsd/tax-reranker-minilm