CSV Column Statistics

Before you load a file, three numbers per column tell you most of what you need: how much of it is blank, how many distinct values it holds, and whether it is really the type you assumed. This reports all three, plus the numeric spread where a column earns one.

values
CSV Column Statistics — Type, Nulls, Distinct Values, Min, Max and MeanBuildFigure

Type guessing that refuses to damage the data

The single most expensive thing a data tool can do is decide that a column of identifiers is a column of numbers. A postcode of 02134 becomes 2134. A nineteen-digit reference loses its last few digits to floating-point rounding and comes back ending in zeros. A part number of 1-2E5 becomes 120000. All three are silent, all three survive being saved back to CSV, and none of them can be undone from the damaged file.

So the rule here is conservative. A column is called numeric only when every non-blank value is a plain number with no leading zero and no more than fifteen digits. Anything with a leading zero or a long digit run is reported as text that must stay text, with a count of how many values are at risk. When formatted numbers are enabled, one leading currency symbol, thousands commas, a trailing percent sign and accounting parentheses are understood for the purpose of computing statistics — but the column is still labelled as formatted rather than plain, because the formatting is part of the data.

Blank, missing, and why they are counted separately

A cell that exists and holds nothing was written by whatever produced the file. A cell past the end of a short row was never written at all. In a well-formed table the missing count is zero everywhere; anything else means rows are ragged, which nearly always traces back to an unquoted delimiter inside a value further left. Two adjacent delimiters give you a blank; a row that simply stops early gives you missing cells to its right, and the difference is the fastest way to tell a sparse column from a broken file.

PatternWhat it usually means
Distinct equals filled, nothing blankA candidate key
Two or three distinct valuesA flag or a status, whatever the header says
Distinct equals 1A constant — often a column that can be dropped
Blank rate above about 80%An optional field, or one that stopped being populated
Seen-once count near the filled countFree text, not a category

Numeric summaries and what they hide

Minimum and maximum are the two that find real problems: a negative quantity, a price of zero, a date parsed into a year of 1900, a value three orders of magnitude above everything else. Look at them before the mean.

The mean and the median are given together on purpose. When they are close, the column is roughly symmetric and the average describes it. When the mean sits well above the median, a few large values are pulling it, which is the normal shape for transaction amounts and the reason average order value flatters a business. The standard deviation is the sample form, dividing by one less than the count, which is what you want when the rows are a sample of something larger and is indistinguishable from the other form at any size worth summarising.

Dates get reported, not guessed

A column of ISO dates is unambiguous and is labelled as such. A column written with slashes or dots is not, and this page will not pretend otherwise. 03/04/2026 is the third of April to most of the world and the fourth of March in the United States, and the file carries nothing that decides between them.

Sometimes the data settles it accidentally: if any value has a first part above twelve, the order must be day-first; if any second part exceeds twelve, it must be month-first. Both of those are reported. If neither occurs, the column is flagged as genuinely ambiguous and the only real fix is to ask whoever produced it, then convert to the ISO form before anything else touches the file. A column where both positions exceed twelve somewhere is worse than ambiguous — it is inconsistent, meaning two different conventions have already been mixed into one column.

Using the profile

Run this before you write the loader, not after it fails. The longest value tells you how wide a text column needs to be. The distinct count tells you whether something is a category or free text. The blank rate tells you which columns can be NOT NULL. The must-stay-text list tells you which columns to declare as strings so a later import does not quietly renumber them.

From here, the table cleaner removes the blank rows and duplicates the profile just exposed, the row filter pulls out the subset you care about, and CSV to SQL INSERT writes the load. For the arithmetic on a column of untidy numbers, the number format cleaner normalises them first. One standing caution: profiling is exactly the moment people reach for a real customer or payroll extract. Everything here runs in your browser and nothing is transmitted, but check your own policy before pasting personal or regulated data into any web page, this one included.

Questions people ask

Why is my ID column reported as text?

Because it has a leading zero, or more than fifteen digits, or a value somewhere in it that is not purely numeric. That is the correct answer for an identifier: reading it as a number destroys the leading zero and rounds off the end of a long one, silently and permanently. The report lists how many values would be damaged so you can size the decision. An ID is a label, not a quantity, and nothing is ever computed from it.

What is the difference between distinct and unique here?

Distinct counts how many different values appear. Seen-once counts how many of those appear exactly one time. A column of 1,000 filled cells with 1,000 distinct values that are all seen once is a key. The same column with 12 distinct values is a category. Where distinct is high but seen-once is much lower, you have repeated values with a long tail, which is typical of names and free text.

Can it tell me if a column is a valid date?

It can tell you the shape. ISO dates are recognised and reported as unambiguous. Slash or dot dates are flagged, and where the values themselves rule out one ordering — a first part above twelve, say — that is reported too. What it will not do is silently pick an interpretation. Nothing is parsed into a calendar date, so no value is ever shifted by a month in the process of being examined.

How does the tool handle a column called constructor?

Correctly, which is less automatic than it sounds. Column names and cell values from a file are used as lookup keys, and an ordinary JavaScript object inherits properties like constructor and toString, so a naive counter reports a value it has never seen. Every map here is created without a prototype, so a column named constructor and a value of toString are counted exactly like any other string.

Why does the mean of my amount column look wrong?

Check the numeric-values count against the filled count in the first table. Any value that could not be read as a number is excluded from the statistics, so a column where a third of the cells say "n/a" produces a mean of the other two thirds. Also compare the mean with the median — a large gap means a few extreme values dominate, and in that case the median is the more honest single number.

Related