CSV Delimiter and Quoting Fixer

Half the files people call broken CSV are fine and are semicolon-separated, which is what a European spreadsheet writes when the comma is the decimal mark. The other half have a real quoting fault, and the two look identical until something reads the file properly.

CSV Delimiter and Quoting Fixer — Convert Comma, Tab, Semicolon and PipeBuildFigure

Detection by consistency, not by frequency

Counting which character appears most often is the obvious way to guess a delimiter and it fails on the most common awkward file there is: a semicolon-separated export where every number uses a comma as its decimal mark. The commas outnumber the semicolons and the naive guess picks the wrong one.

What is counted here instead is how consistent each candidate is. A real delimiter produces the same field count on every line, because a table is rectangular. A character that happens to appear inside values produces a different count on nearly every line. The report shows both numbers for all four candidates, so when the detection is wrong you can see why and override it.

Characters inside quoted fields are not counted at all, since a properly quoted "Whitfield, Dana" tells you nothing about the separator.

The three quoting rules everything else gets wrong

There are only three, and splitting on the delimiter breaks all of them. A field may contain the delimiter if it is quoted. A field may contain a line break if it is quoted, which means one record can span several lines. A quotation mark inside a quoted field is written twice, so "he said ""no""" is one field reading he said "no".

This reads the text one character at a time with a flag for whether it is currently inside a quoted field, which is the only approach that handles all three. It is also why the row count here can differ from the line count in your editor, and why the tool can tell you a quotation mark was opened and never closed — a state machine notices reaching the end of the input in the wrong state, and a split never can.

Choosing the output quoting

StyleQuotes a field whenTrade-off
Only where requiredIt holds the delimiter, a quote, a line break, or edge whitespaceSmallest and most readable; the normal choice
Every fieldAlwaysLarger, but unambiguous for a fussy importer
NeverNeverLossy — a value containing the delimiter destroys the row

Never-quote exists for feeding things that are not CSV readers at all, and the tool counts how many fields would be corrupted before you use it. If that count is anything but zero, the output cannot be parsed back into the table you started with, and the warning says so rather than letting you find out later.

Tabs, line breaks and the BOM

Converting to tab-separated is the usual way to get data into a spreadsheet by pasting, and it has a specific hazard: a value that itself contains a tab or a line break has no unambiguous representation in the loose TSV that most paste targets expect. Flattening line breaks to spaces is offered for that reason, and it changes your data — the option is off by default and the row count of affected fields is reported so the decision is deliberate.

A byte order mark is three invisible bytes at the start of a UTF-8 file, written by some spreadsheet exports. It is why a first column sometimes appears as a name with an unprintable character in front, and why a header match against id mysteriously fails on the first column only. It is stripped here when present. Note that copying from a file into the box above frequently drops the mark on the way, so a clean report here does not prove the file on disk is clean.

Once the file parses cleanly, the table cleaner handles blank rows and duplicates, CSV to JSON reshapes it, and the column profiler tells you what each column contains. Keep regulated or personal data out of web tools generally, this one included, unless you have checked your own policy — the processing here is local, but that is a separate question from what you are permitted to paste.

Questions people ask

My file uses semicolons. Is it broken?

No. A spreadsheet configured for a locale where the comma is the decimal separator writes semicolon-separated files, and they are perfectly valid — the format is named after the comma, not defined by it. Detect the input as semicolon, output comma, and you have a file that a comma-expecting importer will read. Nothing about the values changes.

What does "unbalanced quoting" actually mean?

A quotation mark started a quoted field and no closing mark ever appeared, so the parser read everything to the end of the text as one enormous field. The usual causes are an inch or foot mark in a measurement, an apostrophe typed as a straight quote, or a value that was quoted at one end only by a broken export. Find the stray mark in the source and either remove it or double it; nothing downstream is reliable until you do.

Why does the tool report fewer rows than my editor shows lines?

Because some fields contain line breaks, which is legal inside quotes. The editor counts physical lines and the parser counts records. The health report gives the number of rows containing an embedded break, so you can check the difference adds up. A trailing newline at the end of the file also accounts for one line that is not a row.

Does converting the delimiter change my values?

Not unless you ask it to. Fields are re-emitted exactly as parsed and quoted where the new delimiter requires it. Trimming spaces and flattening line breaks are the two options that do modify content, and both are off by default. Never-quote also modifies content, by dropping the quoting that some fields need, and it warns you with a count when that would happen.

Can I convert to a delimiter that is not in the list?

No. The four offered are the ones that appear in real files, and an arbitrary separator is a poor idea in a file anyone else will read — it will not be detected by their tools and it will eventually appear inside a value. If you truly need one, convert to tab here and do a single replacement afterwards, having first checked the profile to confirm your chosen character appears nowhere in the data.

Related