Make URL encoding fully WHATWG-compliant

[cyqsimon]

While debugging the failing tests in #1093, I ran into some trouble with improperly-encoded URLs at the front end. It turns out percent-encode sets defined by miniserve aren't fully compliant with the spec. This PR fixes that.

[svenstaro]

Hey, much appreciated! Is there any chance you could add a test that would actually ensure compliance?

[cyqsimon]

Ugh, I'm confused. How do you write a test for something like this? This is quite literally the verbatim implementation of the spec:

The query percent-encode set is the C0 control percent-encode set and U+0020 SPACE, U+0022 ("), U+0023 (#), U+003C (<), and U+003E (>).
The special-query percent-encode set is the query percent-encode set and U+0027 (').
The path percent-encode set is the query percent-encode set and U+003F (?), U+0060 (`), U+007B ({), and U+007D (}).
The userinfo percent-encode set is the path percent-encode set and U+002F (/), U+003A (:), U+003B (;), U+003D (=), U+0040 (@), U+005B ([) to U+005E (^), inclusive, and U+007C (|).
The component percent-encode set is the userinfo percent-encode set and U+0024 ($) to U+0026 (&), inclusive, U+002B (+), and U+002C (,).

What I mean to say is, isn't this something we just double, triple check manually here once and be done?

P.s. I did notice the erroneous BASE set that was previously left in; removed in latest commit.

[cyqsimon]

Woah I must have been drunk or something to make such basic errors. Almost looks like I couldn't read.

[svenstaro]

Fair enough.

By the way, do you think it might be nice to try to upstream this into percent_encoding? They already have some constants but this looks like a logical thing to upstream.

[svenstaro]

Thanks for this. I cross-checked the spec and this seems good!

[cyqsimon]

By the way, do you think it might be nice to try to upstream this into percent_encoding? They already have some constants but this looks like a logical thing to upstream.

Yeah I questioned why this wasn't already a part of percent_encoding as well. But it turns out they used to have these things and decided to remove them. The reason I'm not sure, but regardless I don't think they would be interested in having them re-added.

[svenstaro]

That's really odd. Thanks for looking into it. Perhaps open an issue to discuss upstream? Seems like it was removed a few years ago and the mindset might've changed.

[cyqsimon]

That's really odd. Thanks for looking into it. Perhaps open an issue to discuss upstream? Seems like it was removed a few years ago and the mindset might've changed.

Found this: servo/rust-url#837. Seems like someone has the same idea but the PR looks to be stalled.