althttpd.c check-in: efdc1b8e66

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

althttpd.c check-in: efdc1b8e66

Carl Chave
Hello,

Should "index.cgi" from line number 1923 in subject check in include a
leading forward slash?  My index.cgi scripts aren't being found with
this version (or any subsequent version) unless I explicitly include
it as part of the request uri.  I added the slash and recompiled and
it started working again as it had in previous versions.

Also, could you explain what the intention of the /home addition to
the search paths is?  It seems that /home needs to be a cgi script (by
my testing).  At first I thought that the logic would be that it would
search for a /home directory and then look for an index.html or
index.cgi in /home.  If it /home has to be a cgi script maybe it
should be /home.cgi rather than just /home?

I'm probably misunderstanding the changes!

Thanks,
Carl
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: althttpd.c check-in: efdc1b8e66

Richard Hipp-3
On 4/26/19, Carl Chave <[hidden email]> wrote:
> Should "index.cgi" from line number 1923 in subject check in include a
> leading forward slash?  My index.cgi scripts aren't being found with
> this version (or any subsequent version) unless I explicitly include
> it as part of the request uri.  I added the slash and recompiled and
> it started working again as it had in previous versions.

Fixed.

I'm surprised to hear that there is somebody else actually using
althttpd!  Now with two users, maybe it is time to separate the
althttpd.c code and documentation out into a separate repository
(rather than commingling it with the SQLite documentation), get a
domain name, and set up a website just for althttpd.c :-)

>
> Also, could you explain what the intention of the /home addition to
> the search paths is?

I added that for websites that are primarily just a Fossil (or other
CGI) instance.  The driver case was https://wapp.tcl.tk/.  This was
formerly redirecting to https://wapp.tcl.tk/index.html, which would
then redirect to something like
https://wapp.tcl.tk/index.html/doc/trunk/README.md.  Having the
"index.html" term in the middle of that URL path just seemed wrong.
(This became particularly noticeable to me as I was preparing slides
for a talk on Wapp.)  So I added the ability to have a CGI script
named "home".  And I put "home" first in line so that it would be the
default, if available.  That way, I could keep "index.html" around on
websites for legacy links, but new accesses would redirect to "home"
instead.

Which of these URLs do *you* think looks better?

    https://wapp.tcl.tk/home/doc/trunk/README.md
    https://wapp.tcl.tk/index.html/doc/trunk/README.md
    https://wapp.tcl.tk/index.cgi/doc/trunk/README.md

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: althttpd.c check-in: efdc1b8e66

Carl Chave-2
> Fixed.

Thanks!

> Now with two users, maybe it is time to separate the
> althttpd.c code and documentation out into a separate repository
> (rather than commingling it with the SQLite documentation), get a
> domain name, and set up a website just for althttpd.c :-)

Sounds good to me, though maybe you should wait for another user who
actually knows what they are doing.

> Which of these URLs do *you* think looks better?
>
>     https://wapp.tcl.tk/home/doc/trunk/README.md
>     https://wapp.tcl.tk/index.html/doc/trunk/README.md
>     https://wapp.tcl.tk/index.cgi/doc/trunk/README.md

Definitely the first, though I must admit I'm still struggling a bit
with the mechanics.  Here are a few of my tests with what I expected
(wanted?) and the actual results, my initial expectations were based
on the commit comments not the code itself:

/home is a directory (not a cgi script) but contains index.cgi.  Root
is www actual path is ~/www/default.website/home/index.cgi

Test #1:
Browser URL: http://127.0.0.1:8080
Expected: cgi output from /home/index.cgi
Actual: 404 The document / is not available on this server

Test #2:
Browser URL: http://127.0.0.1:8080/home (no trailing slash)
Expected: cgi output from /home/index.cgi, also expected browser url
bar to continue to just display http://127.0.0.1:8080/home
Actual: cgi output delivered as expected though url bar displays
http://127.0.0.1:8080/home/index.cgi - first response is a 302
redirect to index.cgi

Test #3:
Browser URL: http://127.0.0.1:8080/home/ (with trailing slash)
Expected: cgi output from /home/index.cgi, also expected browser url
bar to continue to just display http://127.0.0.1:8080/home/
Actual: same as expected, cgi output delivered as expected and url bar
continues to only display http://127.0.0.1:8080/home/ first response
is no longer a redirect

Test #4:
Browser URL: http://127.0.0.1:8080/home/some/extra/path/info
Expected: Not really sure why, but I expected the normal output of
/home/index.cgi and the rest to be put in the excess path variable
Actual: 404

Test #5:
Browser URL: http://127.0.0.1:8080/home/index.cgi/some/extra/path/info/
Expected: same as test 4
Actual: Unstyled output of /home/index.cgi delivered.  Firefox
downloads my stylesheets but reports they are of type html rather than
css.  Not sure what's going on with this test.

I'm working on a personal project which (however misguided) uses
symlinking to provide site wide templating that can also be overridden
on a per page basis.  It was an old idea of mine that I've tried to
implement a couple of times over the years with varying degrees of
succes using a typical LAMP stack.  althttpd.c is a breath of fresh
air and the catalyst I needed to revisit the project.  The virtual
hosts, dash prefixed hidden directories, cgi and static support are
all killer features that I'm using in the design.  Anyway, I'm not
expecting a lengthy response - just providing some feedback.  I'd like
to use the unmodified source of althttpd.c so I will continue testing
and either work around my expected behaviour or try to modify the code
as a last resort.

Thanks,
Carl
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: althttpd.c check-in: efdc1b8e66

Richard Hipp-3
On 4/27/19, Carl Chave <[hidden email]> wrote:

>> Fixed.
>
> Thanks!
>
>> Now with two users, maybe it is time to separate the
>> althttpd.c code and documentation out into a separate repository
>> (rather than commingling it with the SQLite documentation), get a
>> domain name, and set up a website just for althttpd.c :-)
>
> Sounds good to me, though maybe you should wait for another user who
> actually knows what they are doing.
>
>> Which of these URLs do *you* think looks better?
>>
>>     https://wapp.tcl.tk/home/doc/trunk/README.md
>>     https://wapp.tcl.tk/index.html/doc/trunk/README.md
>>     https://wapp.tcl.tk/index.cgi/doc/trunk/README.md
>
> Definitely the first, though I must admit I'm still struggling a bit
> with the mechanics.  Here are a few of my tests with what I expected
> (wanted?) and the actual results, my initial expectations were based
> on the commit comments not the code itself:
>
> /home is a directory (not a cgi script) but contains index.cgi.  Root
> is www actual path is ~/www/default.website/home/index.cgi

I have just updated the althttpd documentation to take the new /home
feature into account.  The revised docs are here:
https://www.sqlite.org/docsrc/doc/trunk/misc/althttpd.md


>
> Test #1:
> Browser URL: http://127.0.0.1:8080
> Expected: cgi output from /home/index.cgi
> Actual: 404 The document / is not available on this server

The does say: "If the request URI specifies the name of a directory
within *.website, then althttpd appends "/home", "/index.html", and
"/index.cgi", in that order, looking for a match."

Your URI specifies a directory "/".  So this algorithm comes into
play.  It tries appending /home, but that is a directory not a file.
Then it tries /index.html, but that does not exist.  Then it tries
/index.cgi, but that does not exist either.  Hence 404.

>
> Test #2:
> Browser URL: http://127.0.0.1:8080/home (no trailing slash)
> Expected: cgi output from /home/index.cgi, also expected browser url
> bar to continue to just display http://127.0.0.1:8080/home
> Actual: cgi output delivered as expected though url bar displays
> http://127.0.0.1:8080/home/index.cgi - first response is a 302
> redirect to index.cgi

Right.  https://127.0.0.1:8080/home makes "home" look like a file, not
a directory.  So if your web page contains relative links (example:
href="./xyz.html") then the web browser will resolve that into
http://127.0.0.1/xyz.html, instead of what you want,
http://127.0.0.1/home/xyz.html.  To prevent this, althttpd
automatically redirects to https://127.0.0.1/home/index.cgi.


>
> Test #3:
> Browser URL: http://127.0.0.1:8080/home/ (with trailing slash)
> Expected: cgi output from /home/index.cgi, also expected browser url
> bar to continue to just display http://127.0.0.1:8080/home/
> Actual: same as expected, cgi output delivered as expected and url bar
> continues to only display http://127.0.0.1:8080/home/ first response
> is no longer a redirect

The redirect is not necessary in this case because the URL ends with
/, so your web browser knows that "home" is a directory and so will
correctly resolve relative links.

>
> Test #4:
> Browser URL: http://127.0.0.1:8080/home/some/extra/path/info
> Expected: Not really sure why, but I expected the normal output of
> /home/index.cgi and the rest to be put in the excess path variable
> Actual: 404

You do actually have to have the name of the CGI script in there.
Otherwise, althttpd has know way of knowing what script to run.

>
> Test #5:
> Browser URL: http://127.0.0.1:8080/home/index.cgi/some/extra/path/info/
> Expected: same as test 4
> Actual: Unstyled output of /home/index.cgi delivered.  Firefox
> downloads my stylesheets but reports they are of type html rather than
> css.  Not sure what's going on with this test.

I don't know either.  Are you sure your CGI is actually sending the
Content-Type header?

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: althttpd.c check-in: efdc1b8e66

Jungle Boogie
In reply to this post by Richard Hipp-3
On Sat 27 Apr 2019  7:44 AM, Richard Hipp wrote:

>On 4/26/19, Carl Chave <[hidden email]> wrote:
>> Should "index.cgi" from line number 1923 in subject check in include a
>> leading forward slash?  My index.cgi scripts aren't being found with
>> this version (or any subsequent version) unless I explicitly include
>> it as part of the request uri.  I added the slash and recompiled and
>> it started working again as it had in previous versions.
>
>Fixed.
>
>I'm surprised to hear that there is somebody else actually using
>althttpd!  Now with two users, maybe it is time to separate the
>althttpd.c code and documentation out into a separate repository
>(rather than commingling it with the SQLite documentation), get a
>domain name, and set up a website just for althttpd.c :-)

Would you ever consider adding support to list an index of a directory?
http://127.0.0.1/files would display whatever is in the /files directory.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: althttpd.c check-in: efdc1b8e66

Carl Chave-2
In reply to this post by Richard Hipp-3
> -SNIP-
>
> You do actually have to have the name of the CGI script in there.
> Otherwise, althttpd has know way of knowing what script to run.
>

Thanks for the explanations, comprehend all.

> >
> > Test #5:
> > Browser URL: http://127.0.0.1:8080/home/index.cgi/some/extra/path/info/
> > Expected: same as test 4
> > Actual: Unstyled output of /home/index.cgi delivered.  Firefox
> > downloads my stylesheets but reports they are of type html rather than
> > css.  Not sure what's going on with this test.
>
> I don't know either.  Are you sure your CGI is actually sending the
> Content-Type header?
>

I see the issue with this now is where the browser thinks the
relatively pathed resources are located, it's requesting:
http://127.0.0.1:8080/home/index.cgi/some/extra/path/info/css/normalize.css
rather than /home/css/normalize.css
so the css filename becomes part of the excess path information and
the browser is just getting another copy of the output from index.cgi
which is, indeed, just html.

Carl
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: althttpd.c check-in: efdc1b8e66

Richard Hipp-3
In reply to this post by Jungle Boogie
On 4/27/19, Jungle Boogie <[hidden email]> wrote:
>
> Would you ever consider adding support to list an index of a directory?
> http://127.0.0.1/files would display whatever is in the /files directory.

Probably not.  That seems to violate the keep-it-simple principle of
althttpd.  On the rare occasions where a directory listing is actually
needed and desirable, such a listing is easily accomplished using CGI.
See https://wapp.tcl.tk/home/file/examples/filelist.tcl for example.
CGI also allows you to customize the styling and text surrounding the
listing.

A real-world example of this sort of thing can be seen at the
https://fossil-scm.org/skins/index.html page which is just a CGI
script that looks for other CGI scripts and provides links to them.
The complete text of the CGI script that generates that page is as
follows:

#!/usr/bin/wapptclsh
#
proc wapp-default {} {
  wapp-trim {
    <html>
    <head>
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Fossil Skins</title>
    </head>
    <body>
    <h1>Fossil Self-Hosting With Built-In Skins</h1>
    <p>The following links run the Fossil self-hosting repository using
    the various built-in skins.  Use these for testing the built-in
    skins.</p>

    <p>The <a href='/fossil'>main self-hosting repository</a> uses a
    customized skin, different from all of the built-ins shown below.</p>

    <ol>
  }
  set parentdir [file dir [wapp-param SCRIPT_NAME]]
  foreach file [lsort [glob -nocomplain *]] {
    if {[regexp {[^a-z0-9_]} $file]} continue
    if {![file executable $file]} continue
    set url $parentdir/$file/timeline
    wapp-subst {<li><a href='%url($url)'>%html($file)</a>\n}
  }
  wapp-subst {</ol>\n}
}
wapp-start $argv

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: althttpd.c check-in: efdc1b8e66

Jose Isaias Cabrera-4

I know I can probably use cygwin to run this tool, but plain Windows is not an option, right?


Richard Hipp, on Saturday, April 27, 2019 04:49 PM wrote...
On 4/27/19, Jungle Boogie <[hidden email]> wrote:
>
> Would you ever consider adding support to list an index of a directory?
> http://127.0.0.1/files would display whatever is in the /files directory.

Probably not.  That seems to violate the keep-it-simple principle of
althttpd.  On the rare occasions where a directory listing is actually
needed and desirable, such a listing is easily accomplished using CGI.
See https://wapp.tcl.tk/home/file/examples/filelist.tcl for example.
CGI also allows you to customize the styling and text surrounding the
listing.

A real-world example of this sort of thing can be seen at the
https://fossil-scm.org/skins/index.html page which is just a CGI
script that looks for other CGI scripts and provides links to them.
The complete text of the CGI script that generates that page is as
follows:

#!/usr/bin/wapptclsh
#
proc wapp-default {} {
  wapp-trim {
    <html>
    <head>
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Fossil Skins</title>
    </head>
    <body>
    <h1>Fossil Self-Hosting With Built-In Skins</h1>
    <p>The following links run the Fossil self-hosting repository using
    the various built-in skins.  Use these for testing the built-in
    skins.</p>

    <p>The <a href='/fossil'>main self-hosting repository</a> uses a
    customized skin, different from all of the built-ins shown below.</p>

    <ol>
  }
  set parentdir [file dir [wapp-param SCRIPT_NAME]]
  foreach file [lsort [glob -nocomplain *]] {
    if {[regexp {[^a-z0-9_]} $file]} continue
    if {![file executable $file]} continue
    set url $parentdir/$file/timeline
    wapp-subst {<li><a href='%url($url)'>%html($file)</a>\n}
  }
  wapp-subst {</ol>\n}
}
wapp-start $argv

--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: althttpd.c check-in: efdc1b8e66

Richard Hipp-3
On 4/29/19, Jose Isaias Cabrera <[hidden email]> wrote:
>
> I know I can probably use cygwin to run this tool, but plain Windows is not
> an option, right?
>

Althttpd is built around fork().  Windows does not support fork().
The cygwin implementation of fork() is convoluted, and I suspect slow,
though I have not actually confirmed that.
--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: althttpd.c check-in: efdc1b8e66

Jose Isaias Cabrera-4

Thanks, Dr. Hipp.


Dr. Richard Hipp, on Monday, April 29, 2019 09:18 AM wrote...
On 4/29/19, Jose Isaias Cabrera <[hidden email]> wrote:
>
> I know I can probably use cygwin to run this tool, but plain Windows is not
> an option, right?
>

Althttpd is built around fork().  Windows does not support fork().
The cygwin implementation of fork() is convoluted, and I suspect slow,
though I have not actually confirmed that.

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: althttpd.c check-in: efdc1b8e66

Warren Young
In reply to this post by Richard Hipp-3
> On Apr 29, 2019, at 7:18 AM, Richard Hipp <[hidden email]> wrote:
>
> On 4/29/19, Jose Isaias Cabrera <[hidden email]> wrote:
>>
>> I know I can probably use cygwin to run this tool, but plain Windows is not
>> an option, right?
>>
>
> Althttpd is built around fork().  Windows does not support fork().
> The cygwin implementation of fork() is convoluted, and I suspect slow,
> though I have not actually confirmed that.

Yes, it’s a known issue:

   https://www.cygwin.com/faq.html#faq.api.fork

So far as I know, none of the past attempts to get around that weakness have succeeded.

WSL uses an entirely new mechanism to solve this problem.  See this video, beginning at 9:25:

   https://blogs.msdn.microsoft.com/wsl/2016/05/23/pico-process-overview/

That should make it faster than Cygwin, though I’m not aware of any benchmarks specifically testing fork() performance between the two.

My understanding is that Cygwin cannot take advantage of the WSL fork() facility due to the walls Windows puts up at the kernel level between so-called “subsystems”.  A process is either Win32 or WSL; it cannot be both at the same time.  Cygwin is, by its nature, tied to the Win32 subsystem.

A third alternative would be running Linux in a VM.  As long as the process isn’t disk-bound, that could be faster than either Cygwin or WSL for fork() heavy processes.  If you’ve ever run an Autoconf configure script under Cygwin vs. a VM, that’s why the VM is so much faster: shell scripts do almost everything via fork().

A middle ground is Docker for Windows.  I hesitate to speculate on its speed compared to the other three, but I think there’s reason to hope it could be a good option.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: althttpd.c check-in: efdc1b8e66

Peter da Silva-2
> A middle ground is Docker for Windows.  I hesitate to speculate on its speed compared to the other three, but I think there’s reason to hope it could be a good option.

I would expect that running a Linux image inside Microsoft's hypervisor environment and running a Linux image in Docker's or other third party's hypervisor environment would provide roughly similar performance... the technology is well understood and mature. The main reason to pick one over another is the VM management environment you prefer.

_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|

Re: althttpd.c check-in: efdc1b8e66

Jose Isaias Cabrera-4
In reply to this post by Richard Hipp-3

Dr. Richard Hipp, on Monday, April 29, 2019 09:18 AM, wrote...​
> On 4/29/19, Jose Isaias Cabrera <[hidden email]> wrote:​
> >​
> > I know I can probably use cygwin to run this tool, but plain Windows is not​
> > an option, right?​
> >​
> ​
> Althttpd is built around fork().  Windows does not support fork().​
> The cygwin implementation of fork() is convoluted, and I suspect slow,​
> though I have not actually confirmed that.​

I built it in cygwin and it's working like a charm.  I have not tried to work it​
hard, but it's working ok on a Windows 10, 4G Mem, 2.40GHz Dual CPU. I will let​
you know if something goes wrong with fork().  Thanks for all the wonderful​
you have given the world.​

josé
_______________________________________________
sqlite-users mailing list
[hidden email]
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users