2026-11-18
Encoding began long before computers. Early writing systems translated spoken language into symbols so that information could be preserved across time and distance.
Computers introduced another layer: characters had to be represented as bytes. Over time, many character encodings and code pages emerged, including ASCII, Windows-1252, ISO-8859 variants, and Shift-JIS. Many of them are still encountered in existing applications, exported data, and legacy files.
This can produce a familiar problem: A file looks correct in one application but displays text such as Müller, Grüber, or � in another. The content may still be intact, but the receiving application has interpreted its bytes using the wrong character encoding.
🔗Unicode helps, but the encoding still matters
Unicode provides a common character repertoire for writing systems around the world. Files still need an encoding form, however, such as UTF-8, UTF-16, or UTF-32.
An application must therefore determine which encoding was used. It may rely on:
- a byte-order mark, or BOM;
- an explicit declaration in the document;
- metadata supplied by another system;
- or heuristic analysis of the byte sequence.
These indicators are not always present, and they do not always agree.
🔗Why HTML needs special handling
HTML introduces an additional source of encoding problems.
A document may declare its character encoding in a <meta charset="..."> element, while its bytes use a different encoding. The encoding may also be specified in an HTTP Content-Type header when the document is delivered by a web server.
For example, converting an HTML file from Windows-1252 to UTF-8 without updating its charset declaration leaves the document inconsistent. Updating only the declaration creates the opposite problem: The metadata says UTF-8, but the underlying bytes still use the original encoding.
A safe HTML conversion must therefore update both the content and the relevant charset declaration.
🔗Detect and convert encodings with PowerShell
ConvertEncoding is a free and open-source PowerShell function included with Set-OutlookSignatures. Its source code is available as function ConvertEncoding in Set-OutlookSignatures.ps1.
The function makes the encoding-detection capabilities of the free and open-source UTF.Unknown library easy to use from PowerShell.
ConvertEncoding can:
- inspect byte-order marks;
- evaluate encoding declarations in HTML;
- use heuristic analysis when no explicit indicator is available;
- convert content to a selected target encoding;
- and update the relevant HTML charset declarations during conversion.
This is useful when:
- normalizing HTML templates to UTF-8;
- importing legacy Windows-1252 or ISO-8859 content;
- processing multilingual files from different systems;
- repairing HTML whose charset declaration does not match its bytes;
- or preparing files for applications that require a particular encoding.
🔗Encoding detection has limits
No heuristic detector can identify every character encoding with certainty.
Short files and ASCII-only content may be valid under several encodings. Some legacy encodings also produce byte sequences that are difficult to distinguish reliably.
Where available, explicit indicators such as a BOM or an HTML charset declaration provide stronger evidence than heuristic analysis. However, explicit declarations can also be incorrect, particularly when a file has previously been converted without updating its metadata.
For business-critical content:
- Keep the original file until the converted result has been validated.
- Test with representative multilingual content.
- Check characters outside the ASCII range, including accented names, typographic quotation marks, currency symbols, and non-Latin scripts.
- Confirm that the resulting HTML charset declaration matches the actual output encoding.
🔗A practical use in Set-OutlookSignatures
Set-OutlookSignatures supports both DOCX and HTML templates for email signatures and out-of-office replies. DOCX templates do not require character-encoding conversion and are handled through their native document-processing workflow.
For maximum compatibility, every other input file opened by Set-OutlookSignatures, every output file it writes, and every piece of HTML content it processes is passed through ConvertEncoding. The function detects and normalizes character encodings, resolves inconsistencies between HTML content and its charset declarations, and helps compensate for incorrect or incomplete encoding configurations in source files.
This consistent handling reduces the risk of mojibake, replacement characters, corrupted multilingual content, and incompatible output. Names, addresses, legal text, special characters, and non-Latin scripts therefore remain intact throughout the entire processing workflow, across different systems and Outlook platforms.
🔗View the source code
The complete implementation is available in the Set-OutlookSignatures source code:
- Open
Set-OutlookSignatures.ps1. - Search for
function ConvertEncoding. - Review the function and its inline documentation for the available parameters and implementation details.
Because Set-OutlookSignatures and UTF.Unknown are both free and open source, you can inspect how encoding detection and conversion are implemented, reuse the approach in your own automation, and contribute improvements.
🔗Set-OutlookSignatures centralizes email signatures and out-of-office replies across every Outlook platform
Consistent branding for Marketing, centralized control for IT, and zero manual effort for employees. Sovereign by design, it keeps data within systems you already trust.
| 3-Step Quickstart | Schedule a Tailored Demo | Website |
|---|---|---|
| For IT administrators. No signup required. | For decision-makers in IT, Marketing, and Security. | Features, architecture, documentation, and downloads. |