Ignore whitespaces when tokenizing strings looking like data: urls.
Improve the tokenizer to better distinguish possible origins from text, by
using the domains whitelist.
Fix a regexp mistake uncovered by the test.
Differential Revision: https://phabricator.services.mozilla.com/D71791
The tokenizer currently thinks large numbers are "broken" IPs, thus it says they
can't be an origin. Unfortunately we use that same code path to identify possible
origin prefixes, and origins can start with a number. Thus we end up searching
rather than autofilling the origin.
For now fix the heuristic for IPs, in the future we may evaluate splitting these
code paths.
Differential Revision: https://phabricator.services.mozilla.com/D48187
--HG--
extra : moz-landing-system : lando
Fixes the tokenizer to recognize file:/// urls, plus adds a second layer of
protection, so that if URI fixup thinks the typed string may be an url, we don't
fetch suggestions for it.
Differential Revision: https://phabricator.services.mozilla.com/D26282
--HG--
extra : moz-landing-system : lando
The new tokenizer, to make more natural typing restrictions like "?search terms", "search terms?" or "%my tab",
splits out restriction characters if they appear at the beginning or end of the search string. This means
keywords and aliases can't in general begin or end with a restriction character, but we don't have an input
check to prevent those.
For now the tokenizer can't recognize keywords or aliases, because it can't be made asynchronous until the
Quantum Bar is the default implementation, and anyway the complexity must be considered at that point.
The best we can do is to stop splitting out restriction character in the middle of the search string.
This will allow a part of the old interaction like searching for "keyword? something".
This patch also prevents us from splitting a %encoded string, fixing bug 1538715.
Differential Revision: https://phabricator.services.mozilla.com/D24929
--HG--
extra : moz-landing-system : lando
Remove the "~" typed restriction character.
Change the url restriction character to "$" and the search one to "?".
Differential Revision: https://phabricator.services.mozilla.com/D9494
--HG--
extra : moz-landing-system : lando
This is a first stab at the new tokenizer.
It's not expected to be perfect yet, but good enough to be modified and replace the existing code in unifiedComplete with just a few modifications.
It's mostly intended to start setting up a code and tests structure.
Differential Revision: https://phabricator.services.mozilla.com/D2838
--HG--
extra : moz-landing-system : lando
This is a first stab at the new tokenizer.
It's not expected to be perfect yet, but good enough to be modified and replace the existing code in unifiedComplete with just a few modifications.
It's mostly intended to start setting up a code and tests structure.
Differential Revision: https://phabricator.services.mozilla.com/D2838
--HG--
extra : moz-landing-system : lando
