Go & SQLite asserts

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Go & SQLite asserts

Russ Cox
Hi all,

https://www.sqlite.org/src/wiki?name=Bug+Reports said to send bug reports
here; this is a documentation bug report for
https://www.sqlite.org/assert.html#philosophy_of_assert_.

That page's section 1.1 Philosophy of Assert describes the SQLite project's
philosophy about three different kinds of assertions - assert, ALWAYS, and
NEVER - which I found very helpful and principled and would consider
pointing other developers at when the topic arises.

But then it inexplicably shifts into a critique of the Go language. The
entire section reads:

The Go programming language <https://golang.org/> omits assert(). The Go
developers recognize this is contentious
<https://golang.org/doc/faq#assertions>. Disallowing assert() is
essentially telling developers that they are not allowed to do run-time
verification of invariants. It is as if the developers of Go do not want
coders to prove the software is correct. The SQLite developers believe that
the lack of assert() disqualifies Go as a language for serious development
work.


I am the tech lead for the Go programming language and a big fan of SQLite.
I often use SQLite as an example of well-written, reliable software (for
example, in research.swtch.com/deps), and the Go SQLite3 binding was one of
the earliest packages I wrote and published after the initial Go open
source release. I don't understand the rationale for bringing up Go at all
on this page. SQLite is not written in Go, and, as far as I know, you are
not seriously considering rewriting it in Go. It seems especially odd to
bring up Go apparently only to criticize it, even if the criticism were
accurate.

I also believe the criticism here is inaccurate. The linked text goes to
the Go FAQ, which says:

Go doesn't provide assertions. They are undeniably convenient, but our
experience has been that programmers use them as a crutch to avoid thinking
about proper error handling and reporting. Proper error handling means that
servers continue to operate instead of crashing after a non-fatal error.
Proper error reporting means that errors are direct and to the point,
saving the programmer from interpreting a large crash trace. Precise errors
are particularly important when the programmer seeing the errors is not
familiar with the code.

We understand that this is a point of contention. There are many things in
the Go language and libraries that differ from modern practices, simply
because we feel it's sometimes worth trying a different approach.


This text criticizing assert is trying to make the same point as the SQLite
page, although without the nice vocabulary you have developed.
Specifically, the paragraph preceding the SQLite page's criticism of Go
says:

Other systems sometimes use assert(X) in a way that is similar to the use
of ALWAYS(X) or NEVER(X) in SQLite. Developers will add an assert(X) as a tacit
acknowledgement that they do not fully believe that X is always true
<https://blog.regehr.org/archives/1576>. We believe that this use of
assert(X) is wrong and violates the intent and purpose of having assert(X)
available in C in the first place. An assert(X) should not be seen as a
safety-net or top-rope used to guard against mistakes. Nor is assert(X)
appropriate for defense-in-depth. An ALWAYS(X) or NEVER(X) macro, or
something similar, should be used in those cases because ALWAYS(X) or
NEVER(X) will be followed by code to actually deal with the problem when
the programmers reasoning turns out to be wrong. Since the code that
follows ALWAYS(X) or NEVER(X) is untested, it should be something very
simple, like a "return" statement, that is easily verified by inspection.


This is *exactly* why Go has no assert. We completely agree that using
assert to mean ALWAYS/NEVER is wrong and bad engineering practice. We also
recognize that, like it or not, to a very large fraction of developers,
assert has become synonymous with ALWAYS/NEVER hopes and dreams about
proper execution, and then its availability encourages users not to
"actually deal with the problem". Leaving assert out forces developers to
think more about what they really mean when they go looking for "assert".
It is of course possible to write all three meanings as trivial functions.
For the specific case of run-time verification of invariants, a name like
"invariant" might at this point be much clearer than the now
hopelessly-fuzzy "assert".

Getting back to the main bug report:

It seems completely out of place in a document about SQLite's own coding
practices (unless SQLite used Go!). Then of course it's even more out of
place when Go's reasons for leaving out assert are being mischaracterized
and actually quite well aligned with SQLite's philosophy. At most, we
disagree only about whether the name "assert" can be salvaged for its
original meaning.

Could you please remove the paragraph about Go from
https://www.sqlite.org/assert.html#philosophy_of_assert_, or else update it
to reflect our actual position? For example, you could write instead that:

In fact, the [https://golang.org|Go programming language]
[https://golang.org/doc/faq#assertions|omits assert()]
precisely to avoid the very common misuse as meaning ALWAYS.


But removing the paragraph entirely seems cleanest, since even an accurate
statements seems mostly out of place.

Thanks for your help with this, and thanks again for SQLite!

Best,
Russ
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Go & SQLite asserts

Richard Hipp-3
On 4/29/19, Russ Cox <[hidden email]> wrote:
>
> That page's section 1.1 Philosophy of Assert describes the SQLite project's
> philosophy about three different kinds of assertions - assert, ALWAYS, and
> NEVER - which I found very helpful and principled and would consider
> pointing other developers at when the topic arises.
>
> But then it inexplicably shifts into a critique of the Go language.

Thanks for your kind reply, Russ.  I have rewritten the paragraph in
question to try to better capture the point I was attempting to make,
and to be less snarky.  Please reread and let me know if you think the
new paragraph is acceptable or if it needs further revision.

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Go & SQLite asserts

Russ Cox
On Mon, Apr 29, 2019 at 11:49 AM Richard Hipp <[hidden email]> wrote:

> On 4/29/19, Russ Cox <[hidden email]> wrote:
> >
> > That page's section 1.1 Philosophy of Assert describes the SQLite
> project's
> > philosophy about three different kinds of assertions - assert, ALWAYS,
> and
> > NEVER - which I found very helpful and principled and would consider
> > pointing other developers at when the topic arises.
> >
> > But then it inexplicably shifts into a critique of the Go language.
>
> Thanks for your kind reply, Russ.  I have rewritten the paragraph in
> question to try to better capture the point I was attempting to make,
> and to be less snarky.  Please reread and let me know if you think the
> new paragraph is acceptable or if it needs further revision.


The current analogy still doesn't seem like it represents the Go project's
position accurately:

It is true that assert() can be misused. Antibiotics can be misused too,
but that does not mean we should ban all antibiotics.


This is an interesting analogy, because of course widespread misuse
(overuse) of antibiotics is a real problem, and the solution is to make
doctors think more carefully about when to use them and not, rather than
"just prescribe the antibiotics to be safe". Avoiding misuse by encouraging
that kind of careful decision is exactly why Go does not provide an assert
builtin that would make it trivial to "just add an assert to be safe".
Instead, we leave it to users to write the specific trivial assertion
function they want (SQLite assert, SQLite ALWAYS, or something else
entirely), forcing them to decide explicitly what action should happen when
the condition is false. We hope this will make programmers think a little
more about consequences and make programs better overall. There's no ban on
writing such a function, only an effort to make people think carefully
about which meaning they really want. At this point we are not likely to
add anything new along those lines, but if I were designing a new language,
the question of adding all three - invariant(x), always(x), and never(x) -
as a collective replacement for assert(x) would be an interesting thing to
consider.

Honestly, it still seems to me like this page is not the place to have this
debate at all, and it would be improved by dropping all mention of Go. But
if you want to draw a contrast, I think it would be accurate to say:

Because assert() can be and is commonly misused, some programming language
theorists and designers look with disfavor on the whole idea of
assert(). For example, the Go programming language omits a built-in assert,
to eliminate its frequent misuse to mean ALWAYS. SQLite avoids this misuse
by reserving assert only for provable invariants and otherwise using ALWAYS
or NEVER.


Best,
Russ
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Go & SQLite asserts

Richard Hipp-3
On 4/29/19, Russ Cox <[hidden email]> wrote:
>
> Because assert() can be and is commonly misused, some programming language
> theorists and designers look with disfavor on the whole idea of
> assert(). For example, the Go programming language omits a built-in assert,
> to eliminate its frequent misuse to mean ALWAYS. SQLite avoids this misuse
> by reserving assert only for provable invariants and otherwise using ALWAYS
> or NEVER.
>

Please tell me if this is more fair and honest representation of this situation:

Because assert() can be and is commonly misused, some programming
language theorists and designers look with disfavor on the whole idea
of assert(). For example, the Go programming language omits a built-in
assert(), since the Go developers feel that the harm caused by misuse
outweighs the benefit of having assert() as a built-in.  The
developers of SQLite disagree with that assessment.  Indeed, the
reason why this article exists is to push back against the notion that
assert() is harmful.

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Go & SQLite asserts

Richard Hipp-3
In reply to this post by Russ Cox
On 4/29/19, Russ Cox <[hidden email]> wrote:
> I were designing a new language,
> the question of adding all three - invariant(x), always(x), and never(x) -
> as a collective replacement for assert(x) would be an interesting thing to
> consider.

There are benefits to having invariant(x) as a built-in in the
language.  A smart compiler might be able to detect cases where the
invariant is false at compile-time, and thus provide earlier detection
of programming errors.  At the very least, the compiler could check to
ensure that the argument to invariant(x) has no side-effects, and
thereby avoid a common bug in the use of assert().  And if
invariant(x) is a built-in, then code to actually check the
invariant() at run-time can be included or omitted in the generated
machine code, depending on optimization settings, without having to
play games with preprocessors and macros.

To reiterate the point of the original article - the key benefit of
invariant(x) is that it is an executable comment.  If I'm reading code
that contains a comment describing an invariant, then I may or may not
believe the comment (probably not, actually) but if I am looking at
tested and working code that contains an invariant(x) statement, I
have much more confidence that the given invariant is true. This
GREATLY enhances maintainability of complex systems, in my experience.

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Go & SQLite asserts

Richard Hipp-3
In reply to this post by Richard Hipp-3
On 4/29/19, Richard Hipp <[hidden email]> wrote:

>
> Because assert() can be and is commonly misused, some programming
> language theorists and designers look with disfavor on the whole idea
> of assert(). For example, the Go programming language omits a built-in
> assert(), since the Go developers feel that the harm caused by misuse
> outweighs the benefit of having assert() as a built-in.  The
> developers of SQLite disagree with that assessment.  Indeed, the
> reason why this article exists is to push back against the notion that
> assert() is harmful.
>

Language similar to the above is now on the website.  Your feedback is
appreciated.

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Go & SQLite asserts

Russ Cox
On Mon, Apr 29, 2019 at 3:28 PM Richard Hipp <[hidden email]> wrote:

> On 4/29/19, Richard Hipp <[hidden email]> wrote:
> >
> > Because assert() can be and is commonly misused, some programming
> > language theorists and designers look with disfavor on the whole idea
> > of assert(). For example, the Go programming language omits a built-in
> > assert(), since the Go developers feel that the harm caused by misuse
> > outweighs the benefit of having assert() as a built-in.  The
> > developers of SQLite disagree with that assessment.  Indeed, the
> > reason why this article exists is to push back against the notion that
> > assert() is harmful.
> >
>
> Language similar to the above is now on the website.  Your feedback is
> appreciated.
>

This text and the text on the web site work for me.

For what it's worth, it was not clear to me until just now that the article
existed to push back on a general "asserts considered harmful" notion. I
was reading it as primarily documenting SQLite coding conventions. The
reference to Go makes more sense to me now.

I also agree with all your comments about the benefits of invariant(x) when
it is used properly.

Thanks very much for a productive, enlightening discussion, and for making
the changes.

Best,
Russ
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Go & SQLite asserts

Richard Hipp-3
On 4/29/19, Russ Cox <[hidden email]> wrote:
>
> Thanks very much for a productive, enlightening discussion, and for making
> the changes.
>

Thanks for bringing up your concerns and helping to make the article
better.  In case you had not previously noticed, communicating with
other humans is not my forte and I can use all the help I can get!

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Go & SQLite asserts

Dominique Devienne
In reply to this post by Russ Cox
On Mon, Apr 29, 2019 at 9:49 PM Russ Cox <[hidden email]> wrote:

> On Mon, Apr 29, 2019 at 3:28 PM Richard Hipp <[hidden email]> wrote:
> For what it's worth, it was not clear to me until just now that the article
> existed to push back on a general "asserts considered harmful" notion. I
> was reading it as primarily documenting SQLite coding conventions. The
> reference to Go makes more sense to me now.
>

Very interesting discussion between two of my very favorite programmers.
Thanks.
(programmer here in the very general sense of developer, architect,
designer, even perhaps "code-artist")

I also agree with all your comments about the benefits of invariant(x) when
> it is used properly.
>

I have no business being in this discussion between giants...

But I also regret the lack of invariant/always/never (I dare not say
assert...)
in Go. I wish Robert/Rob/Ken/Russ instead of shunning "assertions" would
promote "proper use" of invariant/always/never by making them a built-in
part of
the language, and have the compiler perform the kind of static analysis
Richard
mentions in this thread. My $0.02. --DD
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Go & SQLite asserts

Rowan Worth-2
On Fri, 3 May 2019 at 16:03, Dominique Devienne <[hidden email]> wrote:

> On Mon, Apr 29, 2019 at 9:49 PM Russ Cox <[hidden email]> wrote:
>
> > On Mon, Apr 29, 2019 at 3:28 PM Richard Hipp <[hidden email]> wrote:
> > For what it's worth, it was not clear to me until just now that the
> article
> > existed to push back on a general "asserts considered harmful" notion. I
> > was reading it as primarily documenting SQLite coding conventions. The
> > reference to Go makes more sense to me now.
> >
>
> Very interesting discussion between two of my very favorite programmers.
> Thanks.
>

Seconded!


> But I also regret the lack of invariant/always/never (I dare not say
> assert...) in Go. I wish Robert/Rob/Ken/Russ instead of shunning
> "assertions" would
> promote "proper use" of invariant/always/never by making them a built-in
> part of the language, and have the compiler perform the kind of static
> analysis
> Richard mentions in this thread. My $0.02. --DD
>

This is getting off-topic, but the immediate question arising from this
proposal is what would you _do_ in response to a violation? For those
unfamiliar with go, the language doesn't feature exceptions. The only
stack-unwinding mechanism is to call panic(), which will terminate the
entire process (unless captured by a call to recover() before unwinding the
entire stack).

Go is also a very modular language, designed to be very easy to drop other
people's code/libraries into your project. But there's a very clear
convention set out surrounding the use of panic() - it should never form
part of a module's public API. It's not an error reporting mechanism and
callers should never be expected to invoke recover() just to use your
module.

I can't see another way to implement invariant/always/never other than
creating some "blessed" form of panic(), and if you do that then the
cross-module convention shifts from a very clear "DO NOT" to a more subtle
"UNLESS THINGS HAVE GONE REALLY SIDEWAYS." Now there's a judgement call
involved in whether it's acceptable for a given API call to bring the whole
process down when provided nonsense state, and since judgement varies
between individuals the effect that has on the entire go ecosystem could be
huge.

And I guess this is the core of the disagreement - when used "correctly"
assertions are informative and helpful but there's no way to enforce
"correct" usage. Admittedly you could say this about a lot of programming
constructs but I don't think we can do away with arrays just yet!

But it also highlights one of the great things about go, which is that you
don't have to agree with all its design decisions to reap their benefits.
There's real advantages to sticking to your guns when it comes to
conceptual integrity, even if it doesn't suit absolutely everybody. It's
why we like the "lite" in sqlite, no?

-Rowan (aka sqweek -- hi Russ ^_^)
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: Go & SQLite asserts

Jens Alfke-2


> On May 3, 2019, at 3:06 AM, Rowan Worth <[hidden email]> wrote:
>
> But there's a very clear
> convention set out surrounding the use of panic() - it should never form
> part of a module's public API. It's not an error reporting mechanism and
> callers should never be expected to invoke recover() just to use your
> module.


At the risk of going farther OT…

There’s precedent for this: consider what happens in Go if you access a nil reference. IIRC (I haven’t used Go much in a few years) it triggers a panic. So basically, the “.” operator has a sort of “assert(ref != nil)” in its implementation. Again IIRC, the same thing is true of the integer division operator when the denominator is zero.

In neither case is this part of an API, it’s very literally a “panic” in the usual sense of the word: an escape from an intolerable situation. Exactly the same would be true of other situations where an assert() function could be used. Of course it shouldn’t be overused, but reserved for a situation where either (a) the flow of control would soon panic anyway at the language level due to a nil deref or something similar, and an explicit assertion failure just makes the failure easier to diagnose; or (b) continuing would have damaging effects like corrupting state.

But I’ve learned not to question Go design decisions in their forums; it tends to lead to a lecture on why they’re right and I’m wrong. (One of the minor reasons I don’t use Go anymore.)

—Jens
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users