The previous fix for correcting spans with comments in
59eef29a6c
had the unfortunate side effect of _breaking_ the spans with comments
when there was leading whitespace. This happened because the previous
fix was testing one without a comment, identifying that the offset shouldn't
have anything added to it, and then removing that offset adjustment
(`offsets[i] + (expressionText.length - sourceToLex.length)`).
Upon further investigation, this offset adjustment _was actually
necessary_ for when the input had comments, but this was only because
the `stripComments` function used `trim` to remove whitespace for these
cases. This is the real problem -- not only does it create a ton of confusion
but also it means that the behavior of the lexer and resulting spans is
different between inputs with comments and inputs without comments.
After reviewing how the `inputLength` of `_ParseAST` was used, it
appears that the correct behavior would be to _not_ trim the input. The
`inputLength` is used to advance the current index beyond points which
have been processed. This _should_ include any whitespace. Additionally,
`inputLength` doesn't appear to be needed at all. When there was no
comment in the input, it was always equal to the `input.length` anyways.
When there _is_ a comment, it should include that comment anyways to
advance the index beyond the comment.
PR Close#44785
ec2e6f6a3f