json1 not escaping CRLF characters

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

json1 not escaping CRLF characters

Eric Grange-3
Hello,

I have been experimenting with using the json1 extension to generate json
sql-side rather than code-side, but I am hitting an issue with the CR & LF
characters which are not escaped, which results in json that fails
JSON.parse or use as JSONP in browsers. Ideally CR / LF should be encoded
as \r and \n respectively.

Is it by design or an oversight?

Is there a way around using the replace() function to fix it? (though I
guess other control characters will have the issue, so multiple replace()
would be needed)

Eric
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: json1 not escaping CRLF characters

Dominique Devienne
On Thu, Feb 4, 2016 at 10:43 AM, Eric Grange <[hidden email]> wrote:

> I have been experimenting with using the json1 extension to generate json
> sql-side rather than code-side, but I am hitting an issue with the CR & LF
> characters which are not escaped, which results in json that fails
> JSON.parse or use as JSONP in browsers. Ideally CR / LF should be encoded
> as \r and \n respectively.
>
> Is it by design or an oversight?
>

http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf,
page 4, section 9 String, states:

A string is a sequence of Unicode code points wrapped with quotation marks
(U+0022). All characters may be placed within the quotation marks except
for the characters that must be escaped:

quotation mark (U+0022),
reverse solidus (U+005C),
and the control characters U+0000 to U+001F.

There are two-character escape sequence representations of some characters.

\" represents the quotation mark character (U+0022).
\\ represents the reverse solidus character (U+005C).
\/ represents the solidus character (U+002F).
\b represents the backspace character (U+0008).
\f represents the form feed character (U+000C).
\n represents the line feed character (U+000A).
\r represents the carriage return character (U+000D).
\t represents the character tabulation character (U+0009).

So, for example, a string containing only a single reverse solidus
character may be represented as "\\".


The wording above doesn't seem to "require" these characters to be escaped.
Only double-quote, and backlash need to be.
My own JSON serializer does, but the spec doesn't require it apparently. So
json1 seems OK and to-spec...

Is there a way around using the replace() function to fix it? (though I
> guess other control characters will have the issue, so multiple
> replace() would be needed)
>

Many JSON serializers have options, like ascii-only (no UTF-8 encoded char,
only \uffff codes), etc...
Perhaps json1 could gain such options somehow. --DD
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: json1 not escaping CRLF characters

Stephan Beal-3
On Thu, Feb 4, 2016 at 11:04 AM, Dominique Devienne <[hidden email]>
wrote:

> \" represents the quotation mark character (U+0022).
> \\ represents the reverse solidus character (U+005C).
> \/ represents the solidus character (U+002F).
> ...
> The wording above doesn't seem to "require" these characters to be escaped.
> Only double-quote, and backlash need to be.
> My own JSON serializer does, but the spec doesn't require it apparently. So
> json1 seems OK and to-spec...
>

FWIW, i once asked Doug Crockford about must-vs-may here and he responded:

        --------------
        From: Douglas Crockford <[hidden email]>
        To: Stephan Beal <[hidden email]>
        Subject: Re: Is escaping of forward slashes required?

        It is allowed, not required. It is allowed so that JSON can be
safely
        embedded in HTML, which can freak out when seeing strings containing
        "</". JSON tolerates "<\/" for this reason.

        On 4/8/2011 2:09 PM, Stephan Beal wrote:
        > Hello, Jsonites,
        >
        > i'm a bit confused on a small grammatic detail of JSON:
        >
        > if i'm reading the grammar chart on http://www.json.org/
correctly,
        > forward slashes (/) are supposed to be escaped in JSON. However,
the
        > JSON class provided with my browsers (Chrome and FF, both of
which i
        > assume are fairly standards/RFC-compliant) do not escape such
characters.
        >
        > Is backslash-escaping forward slashes required? If so, what is the
        > justification for it? (i ask because i find it unnecessary and
hard to
        > look at.)


--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: json1 not escaping CRLF characters

Richard Hipp-3
In reply to this post by Eric Grange-3
On 2/4/16, Eric Grange <[hidden email]> wrote:
> Hello,
>
> I have been experimenting with using the json1 extension to generate json
> sql-side rather than code-side, but I am hitting an issue with the CR & LF
> characters which are not escaped, which results in json that fails
> JSON.parse or use as JSONP in browsers. Ideally CR / LF should be encoded
> as \r and \n respectively.
>
> Is it by design or an oversight?

Oversight.  https://www.sqlite.org/src/tktview/ad2559db380a

>
> Is there a way around using the replace() function to fix it? (though I
> guess other control characters will have the issue, so multiple replace()
> would be needed)
>
> Eric
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>


--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: json1 not escaping CRLF characters

Dominique Devienne
In reply to this post by Dominique Devienne
On Thu, Feb 4, 2016 at 11:04 AM, Dominique Devienne <[hidden email]>
wrote:

> On Thu, Feb 4, 2016 at 10:43 AM, Eric Grange <[hidden email]> wrote:
>
>> I have been experimenting with using the json1 extension to generate json
>> sql-side rather than code-side, but I am hitting an issue with the CR & LF
>> characters which are not escaped, which results in json that fails
>> JSON.parse or use as JSONP in browsers. Ideally CR / LF should be encoded
>> as \r and \n respectively.
>>
>> Is it by design or an oversight?
>>
>
> http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf,
> page 4, section 9 String, states:
>
> A string is a sequence of Unicode code points wrapped with quotation marks
> (U+0022). All characters may be placed within the quotation marks except
> for the characters that must be escaped:
> ...
>
> and the control characters U+0000 to U+001F.
>
> There are two-character escape sequence representations of some characters.
> ...
>
> \b represents the backspace character (U+0008).
> \f represents the form feed character (U+000C).
> \n represents the line feed character (U+000A).
> \r represents the carriage return character (U+000D).
> \t represents the character tabulation character (U+0009).
>
>
Those are in the [U+0000 - U+001F] range indeed, so *must* be escaped.
U+0022 and U+005C are explicitly mentioned.
Remains only U+002F, as Stephan mentioned, which is the only that can be
either. --DD
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: json1 not escaping CRLF characters

Richard Hipp-3
In reply to this post by Richard Hipp-3
On 2/4/16, Richard Hipp <[hidden email]> wrote:

> On 2/4/16, Eric Grange <[hidden email]> wrote:
>> Hello,
>>
>> I have been experimenting with using the json1 extension to generate json
>> sql-side rather than code-side, but I am hitting an issue with the CR &
>> LF
>> characters which are not escaped, which results in json that fails
>> JSON.parse or use as JSONP in browsers. Ideally CR / LF should be encoded
>> as \r and \n respectively.
>>
>> Is it by design or an oversight?
>
> Oversight.  https://www.sqlite.org/src/tktview/ad2559db380a
>

Now fixed on trunk.  Snapshot available at https://www.sqlite.org/
and the change is mentioned on the draft change log for the next
release at https://www.sqlite.org/draft/releaselog/3_11_0.html

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: json1 not escaping CRLF characters

Eric Grange-3
Thanks!

I can make do with replace() for now (as in my particular case, I only have
CRLF & TAB to replace)

On Thu, Feb 4, 2016 at 11:39 AM, Richard Hipp <[hidden email]> wrote:

> On 2/4/16, Richard Hipp <[hidden email]> wrote:
> > On 2/4/16, Eric Grange <[hidden email]> wrote:
> >> Hello,
> >>
> >> I have been experimenting with using the json1 extension to generate
> json
> >> sql-side rather than code-side, but I am hitting an issue with the CR &
> >> LF
> >> characters which are not escaped, which results in json that fails
> >> JSON.parse or use as JSONP in browsers. Ideally CR / LF should be
> encoded
> >> as \r and \n respectively.
> >>
> >> Is it by design or an oversight?
> >
> > Oversight.  https://www.sqlite.org/src/tktview/ad2559db380a
> >
>
> Now fixed on trunk.  Snapshot available at https://www.sqlite.org/
> and the change is mentioned on the draft change log for the next
> release at https://www.sqlite.org/draft/releaselog/3_11_0.html
>
> --
> D. Richard Hipp
> [hidden email]
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users