rasoultilburg/CausaLMiner

JointCausalModel for Causal Extraction

A PyTorch-based model for joint causal extraction, designed for use with HuggingFace Transformers. This model performs softmax-based decoding for BIO tagging and supports class weights for handling imbalanced data. It is suitable for extracting cause-effect relations from text.

Model Overview

• Encoder: Any HuggingFace transformer (default: bert-base-uncased)
• Tasks:
  • Sentence-level causal classification
  • BIO tagging for span extraction (Cause, Effect, CE, etc.)
  • Cause-Effect relation extraction

Usage

Loading the Model

from transformers import AutoTokenizer
from modeling_joint_causal import JointCausalModel
from configuration_joint_causal import JointCausalConfig

config = JointCausalConfig.from_pretrained('rasoultilburg/CausaLMiner')
model = JointCausalModel.from_pretrained('rasoultilburg/CausaLMiner', config=config)
tokenizer = AutoTokenizer.from_pretrained(config.encoder_name)

Prediction API

The main prediction method is model.predict():

results = model.predict(
    sents=["The storm caused flooding in the city."],
    tokenizer=tokenizer,           # Optional: pass a tokenizer
    rel_mode="auto",              # 'auto' or 'head'
    rel_threshold=0.4,             # Threshold for relation extraction
    cause_decision="cls+span"     # 'cls_only', 'span_only', or 'cls+span'
)
print(results)

Prediction Options

• sents: List of input sentences (strings).
• tokenizer: (Optional) HuggingFace tokenizer. If not provided, uses the encoder's default.
• rel_mode:
  • 'auto': Uses span information to infer relations automatically (default).
  • 'head': Uses the model's relation head to score all possible cause-effect pairs.
• rel_threshold: Probability threshold for relation extraction (default: 0.4).
• cause_decision:
  • 'cls_only': Only the sentence-level classifier is used to decide if the sentence is causal.
  • 'span_only': Only the presence of cause/effect spans is used.
  • 'cls+span': Both classifier and span presence must agree (default).

Prediction Options Explained

• rel_mode: Controls how cause-effect relations are extracted.

  • 'auto': (Default) Uses the detected cause/effect spans to infer relations automatically. If there is only one cause and one effect, it links them directly. If there are multiple, it scores all possible pairs using the model's relation head.
  • 'head': Always uses the model's relation head to score all possible cause-effect pairs, regardless of the number of detected spans. This is more exhaustive and can be useful for complex sentences.

• rel_threshold: Sets the probability threshold for deciding whether a predicted cause-effect relation is valid. Only relations with a predicted probability above this value (default: 0.4) are included in the output. Lowering this value will make the model more permissive, while increasing it will make it more conservative.

• cause_decision: Determines how the model decides if a sentence is causal.

  • 'cls_only': Only uses the sentence-level classifier. If the classifier predicts causal, the sentence is marked as causal, regardless of detected spans.
  • 'span_only': Only uses the presence of cause and effect spans. If both are found, the sentence is marked as causal.
  • 'cls+span': (Default) Requires both the classifier to predict causal and at least one cause and one effect span to be present. This is the strictest setting and helps reduce false positives.

Example: How to Use

You can use the model with HuggingFace's AutoModel and AutoTokenizer as follows:

from transformers import AutoModel, AutoTokenizer
import json

model = AutoModel.from_pretrained(
    "rasoultilburg/CausaLMiner",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(
    "rasoultilburg/CausaLMiner",
    trust_remote_code=True,
)

sentences = [
    "Insomnia causes depression and a lack of concentration in children",
    "smoking causes lung cancer and heart disease",
    "exercise improves physical health and mental well-being"
]

results = model.predict(
    sentences,
    tokenizer=tokenizer,
    rel_mode="auto",           # or "head"
    rel_threshold=0.5,         # adjust as needed
    cause_decision="cls+span" # or "cls_only", "span_only"
)
print(json.dumps(results, indent=2, ensure_ascii=False))

Example

results = model.predict([
    "Heavy rain caused flooding in the city.",
    "The sun is shining."
], tokenizer=tokenizer)
for r in results:
    print(r)

Files

• modeling_joint_causal.py: Main model code
• configuration_joint_causal.py: Model config
• config.json: Model configuration (auto-generated)
• model.safetensors: Model weights
• tokenizer.json, vocab.txt, etc.: Tokenizer files

Citation

If you use this model, please cite the repository or paper as appropriate.

---

For more details, see the source code or contact the author.