replaced awk mkopcode scripts now perl based

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

replaced awk mkopcode scripts now perl based

Helmut Tschemernjak
Hello,

due to problems that awk is not identical on some UNIX platforms e.g.
the Solaris 8 awk cannot handle the mkopcode scripts, I re-wrote it
using perl. Perl is more compatible between platforms and does not need
other helper tools to work. As a side effect a build on Windows is much
easier.

The perl scripts are attached. A sample makefile rule is listed below as
a reference.


Helmut Tschemernjak

-----------------------------

$(O-ARCH)/opcodes.h:    $(O-ARCH)/parse.h ../lib/vdbe.c mkopcodeh.pl
         perl mkopcodeh.pl $(O-ARCH)/parse.h ../lib/vdbe.c >$@

$(O-ARCH)/opcodes.c:    $(O-ARCH)/opcodes.h mkopcodec.pl
         perl mkopcodec.pl $< >$@


#!/usr/bin/perl -w
#
# $Id: mkopcodeh.pl,v 1.3 2005/05/23 09:29:53 martin Exp $
#
# Generate the file opcodes.h.
#
# This perl script scans a concatenation of the parse.h output file from the
# parser and the vdbe.c source file in order to generate the opcodes numbers
# for all opcodes.  
#
# The lines of the vdbe.c that we are interested in are of the form:
#
#       case OP_aaaa:      /* same as TK_bbbbb */
#
# The TK_ comment is optional.  If it is present, then the value assigned to
# the OP_ is the same as the TK_ value.  If missing, the OP_ value is assigned
# a small integer that is different from every other OP_ value.
#
# We go to the trouble of making some OP_ value the same as TK_ values
# as an optimization.  During parsing, things like expression operators
# are coded with TK_ values such as TK_ADD, TK_DIVIDE, and so forth.  Later
# during code generation, we need to generate corresponding opcodes like
# OP_Add and OP_Divide.  By making TK_ADD==OP_Add and TK_DIVIDE==OP_Divide,
# code to translation from one to the other is avoided.  This makes the
# code generator run (infinitesimally) faster and more importantly it makes
# the total library smaller.
#

my %tkhash = ();
my %opvalues = ();
my @oplist = (); # array: name, value, comment, nopush
my $opno = 1;

#
# add TK_* values into %tkhash  
# add OP names, value and comment into @oplist
# set OP values to zero or TK value if existing
#
while(<>) {
        # Remember the TK_ values from the parse.h file
        if (/^#define (TK_\w+)\s+(\d+)/) {
                $tkhash{$1} = $2;
        }
        if (/^case (OP_\w+):/) {
                my $name = $1;
                my $value = 0;
                my $comment = "";
                my $nopush = 0;
                if (/same as (TK_\w+)/) {
                        $value = $tkhash{$1};
                        die "$1 not defined" if not $value;
                        $opvalues{$value} = 1;
                        $comment = "/* same as $1 */"
                }
                $nopush = 1 if (/no-push/);
                push(@oplist, [$name, $value, $comment, $nopush]);
        }
}
 
#
# assign opcode values for OPs
# avoid duplicate opcode numbers check for existing TK numbers
#
for my $i (@oplist) {
        if ($i->[1] == 0) {
                while (defined $opvalues{$opno}) {
                        $opno++;
                }
                $i->[1] =  $opno++;
        }
}

print <<'EOF';
/* Automatically generated.  Do not edit */
/* See the mkopcodeh.awk script for details */
EOF

for my $i (@oplist) {
        printf "#define %-20s %3d  %s\n", $i->[0], $i->[1], $i->[2];
}

# Generate the 10 16-bit bitmasks used by function opcodeUsesStack()
# in vdbeaux.c. See comments in that function for details.
#

@nopush = (0, 0, 0, 0, 0, 0, 0, 0, 0, 0);

for my $i (@oplist) {
        if ($i->[3] == 1) {
                my $n = $i->[1];
                my $j = $n % 16;
                my $i = ($n - $j)/16;
                $nopush[$i] =  $nopush[$i] + (2**$j);
        }
}

print "\n";
for(my $i = 0; $i < 10; $i++) {
        print "#define NOPUSH_MASK_$i\t", $nopush[$i], "\n";
}

exit 0;

#!/usr/bin/perl -w
#
# $Id: mkopcodec.pl,v 1.2 2005/05/23 08:58:01 martin Exp $
#
# This perl script scans the opcodes.h file (which is itself generated by
# another awk script) and uses the information gleaned to create the
# opcodes.c source file.
#
# Opcodes.c contains strings which are the symbolic names for the various
# opcodes used by the VDBE.  These strings are used when disassembling a
# VDBE program during tracing or as a result of the EXPLAIN keyword.
#

my @list = ("?");

while(<>) {
        if (/^#define OP_(\w+)\s+(\d+)/) {
                $list[$2] = $1;
        }
}

print <<'EOF';
/* Automatically generated.  Do not edit */
/* See the mkopcodec.awk script for details. */
#if !defined(SQLITE_OMIT_EXPLAIN) || !defined(NDEBUG) || defined(VDBE_PROFILE) || defined(SQLITE_DEBUG)
const char *const sqlite3OpcodeNames[] = {
EOF

for $i (0 .. $#list) {
        my $name = defined($list[$i]) ? $list[$i] : "NotUsed_$i";
        printf "\t/* %3d */ \"%s\"%s\n", $i, $name, ($i == $#list ? '' : ',');
}

print <<'EOF';
};
#endif
EOF
exit 0;
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: replaced awk mkopcode scripts now perl based

Tiago Dionizio
On 5/23/05, Helmut Tschemernjak <[hidden email]> wrote:
> Hello,
>
> due to problems that awk is not identical on some UNIX platforms e.g.
> the Solaris 8 awk cannot handle the mkopcode scripts, I re-wrote it
> using perl. Perl is more compatible between platforms and does not need
> other helper tools to work. As a side effect a build on Windows is much
> easier.

Try using gawk. Worked for me on a Solaris platform.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: replaced awk mkopcode scripts now perl based

Doug Henry
I have always used awka to convert awk scripts to C code, allowing for a
native execuatable to be built, seems to be the most compatible solution.


On 5/23/05, Tiago Dionizio <[hidden email]> wrote:

>
> On 5/23/05, Helmut Tschemernjak <[hidden email]> wrote:
> > Hello,
> >
> > due to problems that awk is not identical on some UNIX platforms e.g.
> > the Solaris 8 awk cannot handle the mkopcode scripts, I re-wrote it
> > using perl. Perl is more compatible between platforms and does not need
> > other helper tools to work. As a side effect a build on Windows is much
> > easier.
>
> Try using gawk. Worked for me on a Solaris platform.
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: replaced awk mkopcode scripts now perl based

Clark, Chris M
In reply to this post by Helmut Tschemernjak
 

> On 5/23/2005 Doug Henry [mailto:[hidden email]] wrote:
> I have always used awka to convert awk scripts to C code,
> allowing for a native execuatable to be built, seems to be
> the most compatible solution.
>

That's cool! Thanks for the info on http://awka.sourceforge.net/ I'd not
seen that before.

Solaris awk compatibility issues can often be resolved by using "nawk"
(which is installed in the base OS along with awk). I've not tried using
sqlite under Solaris so this is more of a general comment, the SUN awk
is pretty basic. As Tiago pointed out gawk will behave as expected BUT
you do need to go ahead and install it.

Could a Solaris developer/user try using nawk? I just spent a few mins
getting sqlite-3.2.1.tar.gz from http://sqlite.org/download.html and
converted all the "awk" calls into "nawk" calls.

%  diff ORIG_Makefile.in HACKED_Makefile.in
225c225
<           | awk '{print $$5,$$6}' >last_change
---
>           | nawk '{print $$5,$$6}' >last_change
326c326
<       sort -n -b +2 opcodes.h | awk -f $(TOP)/mkopcodec.awk >opcodes.c
---
>       sort -n -b +2 opcodes.h | nawk -f $(TOP)/mkopcodec.awk
>opcodes.c
329c329
<       cat parse.h $(TOP)/src/vdbe.c | awk -f $(TOP)/mkopcodeh.awk
>opcodes.h
---
>       cat parse.h $(TOP)/src/vdbe.c | nawk -f $(TOP)/mkopcodeh.awk
>opcodes.h


A quick "./configure ; make" on an old Solaris 2.6 box and I ended up
with an sqlite3 binary that handled a simple create/insert/select piece
of SQL; not conclusive but a good indicator. I don't have TCL so I
wasn't able to run the test suite.

Using nawk (for Solaris) would remove dependencies on extra tools (like
gawk or perl).

Chris
Loading...