Practical overview
When it fits
Use Linktransformer when a Python workflow needs to match, retrieve, transform or group records in dataframes. Choose the retrieval-and-judge workflow when candidate generation followed by LLM match decisions fits the task.
Who it is for
- Python users matching records across tabular datasets
- Data teams cleaning or standardizing values in dataframe columns
- Users grouping rows, mapping detailed records to broader labels, or classifying text associated with tables
Documented uses
- Link records between two dataframes using semantic similarity.
- Retrieve likely record pairs for review or adjudication.
- Normalize, rewrite or standardize values in dataframe columns.
- Cluster rows or map fine-grained rows to coarser labels.
- Classify text associated with tabular data.
Requirements & limits
The repository documents installation with pip and usage through Python code.
- The package offers different linkage paths: direct semantic merge, top-k retrieval, fuzzy matching within exact-match blocks, and retrieval followed by LLM adjudication.
- The documented quick start uses pandas dataframes and a sentence-transformers model.
- LLM adjudication requires configuring a judge model and provider; the example uses an OpenAI API key.
- The README describes combining OpenAI retrieval embeddings with Gemini adjudication through explicit model and provider settings.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Install Linktransformer and prepare a dataframe merge
Install the Python package and set up the documented semantic merge example.
Before you start
- Python environment with pip
- pandas dataframes containing the columns to match
Steps
Install the package with the command below.
pip install linktransformerImport pandas and Linktransformer as
ltin a Python script or notebook.Prepare left and right dataframes with corresponding fields, such as company name and country.
Call
lt.mergewith the dataframes, matching columns and a model name; the README example usessentence-transformers/all-MiniLM-L6-v2.
Expected outcome
A merged result dataframe with paired record fields and a similarity score, as shown in the README example.
Guide limitations
- The documented example demonstrates a dataframe merge; it does not establish performance or accuracy for a particular dataset.
Set up retrieval and LLM adjudication for record linkage
Configure the documented `merge_k_judge` workflow to retrieve candidate pairs and request LLM match decisions.
Before you start
- Two dataframes and the columns used to compare records
- An embedding model and an LLM judge model
- Provider configuration and credentials appropriate to the selected provider
Steps
Pass the two dataframes and matching columns to
lt.merge_k_judge.Set the number of candidates with
kand configure the embedding retrieval model.Set the judge model and provider; the README example uses
gpt-4o-miniwith the OpenAI provider and anOPENAI_API_KEYenvironment variable.Read the documented output fields: retrieval score,
is_match, and confidence when available.
Expected outcome
A linkage result containing candidate retrieval scores and LLM match decisions, with confidence values when available.
Guide limitations
- Confidence is documented as available only in some cases.
- The README describes OpenAI retrieval with Gemini adjudication as an example requiring explicit provider and model settings; it does not provide a complete Gemini code example.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 13 October 2026
- Original website title
- GitHub - dell-research-harvard/linktransformer: A convenient way to link, deduplicate, aggregate and cluster data(frames) in Python using deep learning · GitHub · Official homepage title · retained in its source language