CWE-176: Improper Handling of Unicode Encoding

How it's found

Improper Handling of Unicode Encoding is a specific, narrowly defined instance of a broader pattern. Testers confirm it with targeted code review and a proof-of-concept input that exercises the exact code path this weakness describes.

Consequences

• Unexpected State

Mitigations

• Architecture and Design: Avoid making decisions based on names of resources (e.g. files) if those resources can have alternate names.
• Implementation: Assume all input is malicious. Use an "accept known good" input validation strategy, i.e., use a list of acceptable inputs that strictly conform to specifications. Reject any input that does not strictly conform to specifications, or transform it into something that does. When performing input validation, consider all potentially relevant properties, including length, type of input, the full range of acceptable values, missing or extra inputs, syntax, consistency across related fields, and conformance to business rules. As an example of business rule logic, "boat" may be syntactically valid because it only contains alphanumeric characters, but it is not valid if the input is only expected to contain colors such as "red" or "blue." Do not rely exclusively on looking for malicious or malformed inputs. This is likely to miss at least one undesirable input, especially if the code's environment changes. This can give attackers enough room to bypass the intended validation. However, denylists can be useful for detecting potential attacks or determining which inputs are so malformed that they should be rejected outright.
• Implementation: Inputs should be decoded and canonicalized to the application's current internal representation before being validated (CWE-180). Make sure that the application does not decode the same input twice (CWE-174). Such errors could be used to bypass allowlist validation schemes by introducing dangerous inputs after they have been checked.

Where this fits in a TurboPentest engagement

This weakness is not covered by the automated black-box pentest. IntegSec pentesters cover it in a manual engagement.

Frequently asked questions

Related CWEs

• Class weakness
  CWE-172: Encoding Error