⚠️ See Disclaimer below before using.

jina-reranker-v1-tiny-en

A Teradata Vantage compatible Reranker Model

An ONNX-converted, BYOM-ready copy of jinaai/jina-reranker-v1-tiny-en: given a (query, passage) pair, it outputs a single relevance score for that pair.

Reranking in two-stage retrieval

Retrieval pipelines typically run in two stages. The first stage -- a vector search over an index such as HNSW, or classical keyword search -- scans the whole corpus and returns a shortlist (say, the top 20-100 candidates). It's fast because it embeds the query and each document independently and only ever compares the resulting vectors; the two texts never interact. A reranker is a cross-encoder: it reads the query and one candidate together, token by token, and produces a single relevance score for that pair. That joint attention makes it far more accurate than the first-stage vector search, but also too expensive to run over an entire corpus. The standard pattern is therefore to let cheap first-stage retrieval narrow things down, then use a reranker like this one to re-score and re-order just that shortlist.

Overview of this Model

• 33.0M params (ONNX fp32: 128MB)
• JinaBERT architecture (BERT with ALiBi position encoding, 8192-token native context; this repo's scripts truncate at 512 tokens, matching the rest of this collection)
• Output: a single logit, monotonically related to relevance (higher = more relevant)
• License: apache-2.0. The released model can be used for commercial purposes free of charge.
• Reference to Original Model: https://huggingface.co/jinaai/jina-reranker-v1-tiny-en

Quickstart: Deploying this Model in Teradata Vantage

import getpass
import teradataml as tdml
from huggingface_hub import hf_hub_download

repo_id  = "martinhillebrandtd/jina-reranker-v1-tiny-en"
model_id = "jina-reranker-v1-tiny-en"

# 1. Download artifacts from this repo
hf_hub_download(repo_id=repo_id, filename="onnx/model.onnx", local_dir="./")
hf_hub_download(repo_id=repo_id, filename="tokenizer.json",  local_dir="./")

# 2. Connect to Teradata
tdml.create_context(host=input("host: "), username=input("user: "), password=getpass.getpass("password: "))

# 3. Load model + tokenizer into BYOM tables
tdml.save_byom(model_id=model_id, model_file="onnx/model.onnx", table_name="reranker_models")
tdml.save_byom(model_id=model_id, model_file="tokenizer.json",  table_name="reranker_tokenizers")

# 4. Score an existing chunks table against one fixed query.
#    This model's own pair-separator is </s></s> (see "How it works" below for why concatenation
#    is a valid substitute for real pair-encoding, for this specific model).
chunks_table = "your_chunks_table"  # columns: chunk_id, chunk_txt
QUERY = "What is the best-known landmark in Paris?"

query = f"""
SELECT chunk_id, chunk_txt, emb_0 AS relevance_score
FROM mldb.ONNXEmbeddings(
    ON (SELECT chunk_id, chunk_txt, '{QUERY}' || '</s></s>' || chunk_txt AS txt
         FROM {chunks_table}) AS InputTable
    ON (SELECT * FROM reranker_models WHERE model_id = '{model_id}') AS ModelTable DIMENSION
    ON (SELECT model AS tokenizer FROM reranker_tokenizers WHERE model_id = '{model_id}') AS TokenizerTable DIMENSION
    USING
        Accumulate('chunk_id', 'chunk_txt')
        ModelOutputTensor('sentence_embedding')
        OutputFormat('FLOAT32(1)')
        OverwriteCachedModel('true')
) a
"""
result = tdml.DataFrame.from_query(query).to_pandas().sort_values("relevance_score", ascending=False)

relevance_score is the model's raw logit -- sorting by it directly already gives the correct ranking, since sigmoid is monotonic. If you need an actual [0, 1] probability rather than just a ranking, compute it in SQL instead of Python: 1 / (1 + EXP(-emb_0)).

Combining with vector search for two-stage retrieval

A typical pipeline: embed your corpus once with an embedding model, then at query time use VectorDistance to get a shortlist of candidates by cosine distance, and rerank only that shortlist with this model.

-- Stage 1: shortlist the 20 nearest chunks to the query embedding
CREATE VOLATILE TABLE shortlist AS (
    SELECT ref_id AS chunk_id, distance
    FROM TD_VECTORDISTANCE(
        ON query_embedding AS TargetTable
        ON chunk_embeddings AS ReferenceTable DIMENSION
        USING
            TargetIDColumn('query_id')
            TargetFeatureColumns('[emb_0:emb_383]')
            RefIDColumn('chunk_id')
            RefFeatureColumns('[emb_0:emb_383]')
            DistanceMeasure('cosine')
            TopK(20)
    ) AS dt
) WITH DATA ON COMMIT PRESERVE ROWS;

-- Stage 2: rerank only those 20 shortlisted chunks (see Quickstart above, with
-- your_chunks_table replaced by a join against `shortlist`)

How it works

ONNXEmbeddings is built to map one text column to a fixed-size float vector -- it has no native concept of a (query, passage) pair, and doesn't care that its output is semantically an "embedding": any graph with the right tensor names/shapes runs. This repo reuses it as a generic text-to-number function instead: query and passage are concatenated into a single string before tokenization, and the model's one classification logit comes back through the (1-value) sentence_embedding output.

This is only safe because this model's exported graph never uses token_type_ids to mark the query/passage boundary -- confirmed by testing the concatenated score against the model's true pair-encoded score (AutoModelForSequenceClassification, real two-argument tokenizer call), both locally and against a live Vantage instance:

	mean |delta|	max |delta|
Local ONNX (concat) vs. true PyTorch (pair)	3.5e-7	6.6e-7
Live Teradata ONNXEmbeddings vs. true PyTorch (pair)	4.0e-7	7.9e-7

If you plan to convert a different reranker with this same recipe, check first whether its exported graph actually declares/uses token_type_ids as an input -- if it does, this concat trick is not valid for that model.

How this model was converted

jinaai/jina-reranker-v1-tiny-en already ships a ready-made ONNX export (onnx/model.onnx, fp32, opset 11) -- no re-export via optimum was needed. The only change required was renaming the graph's native output tensor (logits) to sentence_embedding, since ONNXEmbeddings's model_output_tensor argument only accepts the literal values sentence_embedding or token_embeddings. See convert.py for the conversion steps, and test_local.py for the ONNX-vs-PyTorch parity check above.

---

Disclaimer

DISCLAIMER: The content herein ("Content") is provided "AS IS" and is not covered by any Teradata Operations, Inc. and its affiliates ("Teradata") agreements. Its listing here does not constitute certification or endorsement by Teradata.

To the extent any of the Content contains or is related to any artificial intelligence ("AI") or other language learning models ("Models") that interoperate with the products and services of Teradata, by accessing, bringing, deploying or using such Models, you acknowledge and agree that you are solely responsible for ensuring compliance with all applicable laws, regulations, and restrictions governing the use, deployment, and distribution of AI technologies. This includes, but is not limited to, AI Diffusion Rules, European Union AI Act, AI-related laws and regulations, privacy laws, export controls, and financial or sector-specific regulations.

While Teradata may provide support, guidance, or assistance in the deployment or implementation of Models to interoperate with Teradata's products and/or services, you remain fully responsible for ensuring that your Models, data, and applications comply with all relevant legal and regulatory obligations. Our assistance does not constitute legal or regulatory approval, and Teradata disclaims any liability arising from non-compliance with applicable laws.

You must determine the suitability of the Models for any purpose. Given the probabilistic nature of machine learning and modeling, the use of the Models may in some situations result in incorrect output that does not accurately reflect the action generated. You should evaluate the accuracy of any output as appropriate for your use case, including by using human review of the output.