Text Similarity Checker

Three numbers come out of this, and they measure genuinely different things — which is why a pair of texts can score 80 percent on one and 20 percent on another without either being wrong. Reading them means knowing what each one looks at, so each is named and explained rather than averaged into a single reassuring figure.

Text Similarity Checker — Jaccard, Cosine and Levenshtein Ratio Between Two TextsBuildFigure

The three measures, named

Jaccard similarity treats each text as a set of distinct words and divides the size of the intersection by the size of the union. Repetition is invisible to it: a word used once and a word used forty times count the same. It answers "what proportion of the combined vocabulary is shared".

Cosine similarity builds a vector of word frequencies for each text and measures the angle between them. Because it compares direction rather than magnitude, a long text and a short one with the same proportions of words score high. This is the standard measure in document retrieval, and it is the one shown in the headline figure here.

The Levenshtein ratio is a character-level measure. Levenshtein distance is the minimum number of single-character insertions, deletions and substitutions needed to turn one string into the other; dividing that by the length of the longer string and subtracting from one gives a ratio between 0 and 1. It is sensitive to small local changes and to word order in a way the other two are not.

What all three are blind to

Meaning. Every measure here compares surface forms, and there are two failure directions worth naming explicitly.

Reordering. Move the sentences of a paragraph around without changing a word and Jaccard and cosine will barely register it — the sets and the frequency vectors are unchanged. The Levenshtein ratio, meanwhile, will collapse, because a character-level alignment cannot cheaply relocate a block of text. A high-Jaccard, low-Levenshtein pair is very often just a reordered version of the same content.

Synonym substitution. Rewrite "compact the base in four-inch layers" as "consolidate the subgrade in hundred-millimetre lifts" and every measure here collapses, because not one of them knows that two different words can mean the same thing. There is no thesaurus, no stemming, no embedding model behind any of it. Semantic similarity is a different technique entirely and none of these three approximate it.

Also invisible: negation, so "the base was compacted" and "the base was not compacted" score very high; attribution, so a quoted passage and an unquoted one are indistinguishable; and structure, so a bulleted list and the same content as prose diverge on the character measure for reasons that have nothing to do with content.

Reading the pattern rather than any one number

PatternUsually means
All three highNear-identical texts, differing in punctuation or a few words
Jaccard high, Levenshtein lowSame vocabulary, different order or different sentence construction
Levenshtein high, Jaccard lowerSmall edits scattered through — tense changes, spacing, word endings
Cosine high, Jaccard lowerSame emphasis, but one text has vocabulary the other lacks; often a long and a short version of the same thing
All three lowNo surface overlap. Says nothing about whether they mean the same thing

Why there is no averaged score

Averaging three measures that answer different questions produces a number that answers none of them, and it invites exactly the use this tool cannot support: a threshold. There is no percentage above which two texts are copied and below which they are not. Two independent summaries of the same source routinely share half their vocabulary. A careful rewrite of a stolen passage scores low on all three. Any figure treated as a verdict will be wrong in both directions.

Cost and the character limit

The Levenshtein computation fills a table proportional to the product of the two lengths, so comparing two 50,000-character documents means two and a half billion cells and a hung tab. Input is therefore truncated to the first 4,000 characters for that measure only, and the result says when that happened. Jaccard and cosine are linear in the input and always use the whole text. For long documents, comparing section by section gives a far more informative picture than one ratio over the whole thing anyway.

Questions people ask

What percentage counts as plagiarism?

None of them, and this tool cannot make that determination. Plagiarism is about attribution and originality of ideas, which requires knowing the source, the citation, the conventions of the field and often the intent. Surface overlap is weak evidence in both directions: two honest summaries of the same paper often share 40 to 60 percent of their vocabulary, and a deliberately reworded copy scores low here. Institutions that make these findings use corpus-wide detection systems and human review, and neither is what this page does.

Why do I get 100 percent on Jaccard but a low Levenshtein ratio?

Because the two texts contain exactly the same distinct words but arranged differently. Jaccard only asks which words are present, so any reordering is invisible to it. The Levenshtein ratio aligns characters in sequence, and moving a clause from the end of a sentence to the beginning is expensive in single-character edits even though nothing was added or removed. The pattern is the signature of a reordered text.

Should I turn on the function-word filter?

It depends what you are looking for. Leaving it off is the default because function words are part of how a text is written, and their distribution is exactly what makes two paraphrases of the same content differ. Turning it on makes the word-based measures focus on content vocabulary, which is more useful when comparing texts of different registers or lengths on the same subject. It does not affect the character measure, which always sees the full text.

Does it handle languages without spaces between words?

Not usefully for the word-based measures. Tokenisation splits on whitespace, so a Chinese or Japanese text becomes a handful of enormous tokens and Jaccard and cosine become meaningless. The Levenshtein ratio still works correctly on those languages, because it operates on characters, and for scripts without word spacing it is the only one of the three worth reading.

Related