CSV UTF-8 BOM: When Excel Needs It (and When It Breaks Imports)

The BOM Is Three Invisible Bytes with Loud Consequences

A UTF-8 BOM (Byte Order Mark) is the bytes EF BB BF at the start of a text file. It does not change the letters you see — it signals “this is UTF-8” to some programs, especially older Microsoft Excel on Windows. For UK, AU, CA, DE, and US teams exchanging names with £, ü, é, or emoji, the BOM often fixes Excel mojibake. For many Linux tools, APIs, and strict importers, the same BOM breaks the first header column or fails schema validation.

UTF-8 vs UTF-8 with BOM vs ANSI

Name the encoding on purpose.

When You Want a BOM

Excel-first human workflows.

• Recipients will double-click the CSV in Windows Excel and must see £/ü correctly.
• You cannot train them to use Data → From Text/CSV with UTF-8 selected.
• Internal finance packs where Excel is the only consumer.

When You Must Remove the BOM

Importer and pipeline workflows.

• The first header becomes id or \ufeffid and field mapping fails.
• JSON/CSV validators reject the file.
• Linux csvkit/pandas reads a weird first column name.
• Bank/ERP portals document “UTF-8 without BOM.”
• You already control Excel import via Power Query with explicit UTF-8.

from pathlib import Path

raw = Path("excel_export.csv").read_bytes()
if raw.startswith(b"\xef\xbb\xbf"):
    Path("clean.csv").write_bytes(raw[3:])
else:
    Path("clean.csv").write_bytes(raw)

Step-by-Step: Diagnose Encoding Problems

Do this before blaming the converter.

• Open the file in a hex-capable editor or check the first bytes.
• If you see EF BB BF, a UTF-8 BOM is present.
• If accents look wrong in Excel but fine in VS Code, Excel guessed the wrong code page.
• If headers look fine but an importer fails on column 1, strip the BOM.
• Re-save via Online CSV Editor / explicit UTF-8 export and retest both Excel and the importer.

Excel Save Paths (Windows vs Mac)

Menu names vary by Office version.

• Windows Excel: “CSV UTF-8 (Comma delimited)” usually writes UTF-8 with BOM.
• Classic “CSV (Comma delimited)” may use ANSI — bad for international text.
• Mac Excel: still verify — do not assume BOM behavior matches Windows.
• Prefer Data → Get Data → From Text/CSV for reading with explicit UTF-8 when teaching teammates.

EU/UK Interchange Patterns

Combine BOM decisions with delimiter decisions.

• German semicolon + UTF-8 BOM + Excel is a common advisor handoff.
• US SaaS importers often want comma + UTF-8 without BOM.
• Keep two deliverables when needed: excel-utf8-bom.csv and api-utf8.csv.
• Document encoding in the filename or README.

Programming Pitfalls

Libraries differ on BOM handling.

• Some readers skip BOM automatically; some do not.
• Writing with utf-8-sig in Python adds a BOM; utf-8 does not.
• JavaScript text encoders typically do not add a BOM unless you insert it.
• Never concatenate CSV parts if only the first chunk should carry a BOM.

# Python: write Excel-friendly UTF-8 CSV with BOM
import csv
with open("out.csv", "w", newline="", encoding="utf-8-sig") as f:
    writer = csv.writer(f)
    writer.writerow(["name", "city"])
    writer.writerow(["Müller", "Berlin"])

Common Mistakes

Encoding own-goals.

• Assuming “UTF-8” in a UI always means with BOM (or always without).
• Fixing mojibake by retyping instead of fixing encoding.
• Stripping BOM then emailing to Excel-only users without import instructions.
• Confusing BOM issues with delimiter issues (one column symptoms can be either).
• Editing in Excel and silently reintroducing ANSI.

Best Practices

Make encoding part of the file contract.

• State UTF-8 ± BOM in runbooks per destination.
• Keep a golden sample row with £ ü é for round-trip tests.
• Prefer explicit import wizards over double-click when teaching teams.
• Produce two files when Excel and API consumers both matter.
• Validate first header name after every Excel save.

Why Use Convert CSV Online?

Convert CSV Online is free, browser-based, and requires no account for everyday conversions. Inspect and re-export CSVs in the Online CSV Editor, convert to Excel when humans need XLSX, and keep pipelines on clean UTF-8 for US, UK, AU, CA, and EU teams on Windows, macOS, and Linux.

Conclusion

UTF-8 BOM is not magic and not evil — it is a compatibility switch. Add it for Windows Excel double-click workflows; remove it for strict importers and APIs. Decide per destination, test with real £/ü characters, and document the choice.