Quantcast

Porting Sqlite to MQX Operating system

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Porting Sqlite to MQX Operating system

GeoffW
hello

MQX is an Operating system from Freescale available for a number of their Mid range processors. I have started the task of porting Sqlite to run on a Coldfire V4. MQX is not as full featured as the Windows, Unix etc ports, so it is going to be a challenge. If I ever manage to get this to work, I will put the code in the public domain and write up some notes on how I did it and problems I hit. The Sqlite doumentation web pages are great for running code on a Windows OS, but are a bit light on detail for folks that are trying to port to other OS's particularly what I call the mid-range OS's E.G  ThreadX, Nucleus, MQX etc

I have so far implemented the memory handling functions for MQX, and the majority of the file operation functions so I thought I would try some initial debugging with a simple externally created database running
sqlite3_exec() and "SELECT * from tbl1"  (non threadsafe is defined to simplify the problem)

Not totally unexpectedly but it is currently crashing when I run it, and I cant see what is wrong.

I have added some debug printfs to both my Windows version and my MQX version
Windows version gives me

File opened OK
Read 100  Got 100
sqlite3_exec called
sqlite3_exec called
sqlite3_step returns 101  callbackinit = 0
Read 1024  Got 1024
sqlite3_exec called
sqlite3_step returns 100  callbackinit = 0
sqlite3_exec called
sqlite3_step returns 101  callbackinit = 0
sqlite3_step returns 100  callbackinit = 1
sqlite3_step returns 101  callbackinit = 1
sqlite3_exec called
sqlite3_step returns 101  callbackinit = 0
Read 16  Got 16
Read 1024  Got 1024
sqlite3_step returns 100  callbackinit = 0
col1_Title = Two
col2_Title = 2
customer = John


sqlite3_step returns 100  callbackinit = 1
col1_Title = One
col2_Title = 1
customer = Bill


sqlite3_step returns 100  callbackinit = 1
col1_Title = Three
col2_Title = 3
customer = Albert


sqlite3_step returns 101  callbackinit = 1
File closed OK

This is the working version, the callback function prints out the 3 columns  of data and all 3 rows from my test database. Also the test.db file is opened right at the start and closed again at the end



MQX  Non working version shown below
File c:\test.db opened OK
Read 100  Got 100
sqlite3_exec called
sqlite3_exec called
sqlite3_step returns 101  callbackinit = 0
Read 1024  Got 1024
sqlite3_exec called
sqlite3_step returns 100  callbackinit = 0
sqlite3_exec called
sqlite3_step returns 101  callbackinit = 0
sqlite3_step returns 100  callbackinit = 1
File c:\test.db closed OK
sqlite3_step returns 101  callbackinit = 1
sqlite3_exec called
sqlite3_step returns 101  callbackinit = 0

*** UNHANDLED INTERRUPT ***

Several things to note here, there are 2 reads of 100 bytes then 1024, but the subsequent ones in the window log dont get called. (Expecting 16 bytes read as next read call)

None of my lower layer MQX calls are printing out any error lines, so I think they are working OK

There is a premature call to close the database file, mid way through the statement execution, so that is possibly why it crashes a few lines farther on.

The code is very complex around where it is closing the db file and I have not been able to figure it out so far. Here is the stack trace where it calls the premature close



I know its a long shot but does Dr Hipp or anyone have any possible ideas what could give rise to a premature file close ?  Sorry my message was so long. Thanks for any ideas

Geoff
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Porting Sqlite to MQX Operating system

Dan Kennedy-4
>
> http://old.nabble.com/file/p27792715/stack.jpg
>
> I know its a long shot but does Dr Hipp or anyone have any possible  
> ideas
> what could give rise to a premature file close ?  Sorry my message  
> was so
> long. Thanks for any ideas

SQLite thinks it is closing the statement journal here, not
the db file. This file should have been opened with the pathname
argument to the xOpen call set to NULL.

Dan.

_______________________________________________
sqlite-users mailing list
[hidden email]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Porting Sqlite to MQX Operating system

GeoffW
hello Dan

Thanks for the very speedy reply. I have just been experimenting with the working Windows version.

I only get one hit of the winOpen() function and that is with a filename parameter of the full path to my test.db file

So maybe it is not a temporary journal file or maybe I am misunderstanding what you a saying ?


The only other thing I have noticed since my initial post is that even for the working Windows version I am getting 7 calls to the sqlite3OsClose() function.  The actual close of the file is prevented by the line if( pId->pMethods ). As that is NULL, it doesnt close the file except for the very last 7th time it is called where pMethods is not NULL.   I didnt expect it to do this on the windows version, seems odd to me.

On MQX the 2nd call to sqlite3OsClose() has pId->pMethods != NULL so an actual close occurs.

Geoff




Dan Kennedy-4 wrote
>
> http://old.nabble.com/file/p27792715/stack.jpg
>
> I know its a long shot but does Dr Hipp or anyone have any possible  
> ideas
> what could give rise to a premature file close ?  Sorry my message  
> was so
> long. Thanks for any ideas

SQLite thinks it is closing the statement journal here, not
the db file. This file should have been opened with the pathname
argument to the xOpen call set to NULL.

Dan.

_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Porting Sqlite to MQX Operating system

GeoffW
Hi

I wasted another entire day on this without really being able to grasp what the problem is.

The call sequence is for the sqlite3OsClose() function

Working Windows version

pager_unlock() pPager->jfpd where pMethods =0
releaseAllSavepoints() pPager->sjfpd where pMethods =0
above 2 lines repeat 3 times

then

sqlite3PagerClose() pPager->fd where pMethods  <> 0  hence a real file close of the test.db file occurs


MQX faulty version

pager_unlock() pPager->jfpd where pMethods = 0
pager_unlock() pPager->jfpd where pMethods = 0

releaseAllSavepoints() pPager->sjfpd where pMethods <> 0

This call to releaseAllSavepoints has the pMethods pointer pointing at the sqlite3_io_methods mqxIoMethod structure, so a  call to mqxClose() occurs and a the db file is erroneously closed.

Any ideas on this would be greatly appreciated. Thanks

Geoff






GeoffW wrote
hello Dan

Thanks for the very speedy reply. I have just been experimenting with the working Windows version.

I only get one hit of the winOpen() function and that is with a filename parameter of the full path to my test.db file

So maybe it is not a temporary journal file or maybe I am misunderstanding what you a saying ?


The only other thing I have noticed since my initial post is that even for the working Windows version I am getting 7 calls to the sqlite3OsClose() function.  The actual close of the file is prevented by the line if( pId->pMethods ). As that is NULL, it doesnt close the file except for the very last 7th time it is called where pMethods is not NULL.   I didnt expect it to do this on the windows version, seems odd to me.

On MQX the 2nd call to sqlite3OsClose() has pId->pMethods != NULL so an actual close occurs.

Geoff




Dan Kennedy-4 wrote
>
> http://old.nabble.com/file/p27792715/stack.jpg
>
> I know its a long shot but does Dr Hipp or anyone have any possible  
> ideas
> what could give rise to a premature file close ?  Sorry my message  
> was so
> long. Thanks for any ideas

SQLite thinks it is closing the statement journal here, not
the db file. This file should have been opened with the pathname
argument to the xOpen call set to NULL.

Dan.

_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Porting Sqlite to MQX Operating system

GeoffW
hello

I thought I had better update this and confess to my sins. Good job no one is reading this thread as it is an embarassingly stupid mistake

I cant believe I did this and then took so long to spot it.

int sqlite3_os_init(void)
{
static sqlite3_vfs mqxVfs = {
    1,                 /* iVersion */
    0,   /* szOsFile ------------->ARRRGGGGGGGH <------------*/
    MAX_PATH,          /* mxPathname */
    etc ....

I started creating the ported file from osWin.c, I had quite a struggle getting it to compile initially using CodeWarrior, so I was making several temporary hacks to allow it to compile, which I then revisited later to correct. I had replaced the sizeof(winFile) with a zero, and then forgot to change it to sizeof(mqxFile).
This caused a fun crash down in the bowels of the paging code.

Once I corrected this one liner it fixed the crash and sqlite is basically up and running now. I still have a fair bit of work left to implement some interface functions which are just dummy stubs at present, but I am making progress now.

Regards Geoff



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Porting Sqlite to MQX Operating system

arun.pradeep
In reply to this post by Dan Kennedy-4

Hi Geoff

  I am currently working on porting of SQLITE to MQX RTOS in MPC5125
Platform.

I am trying to rewrite the sqlite_os_init() for MQX and I have taken Win32
VFS implementation as the reference.

But I am not able to map all of the Win32 file system calls with that of MQX
functions.

Please advise me on the list of file system functions that can be
implemented as part of VFS for MQX.



Regards,
Arun


GeoffW wrote:

>
> hello
>
> I thought I had better update this and confess to my sins. Good job no one
> is reading this thread as it is an embarassingly stupid mistake :rules:
>
> I cant believe I did this and then took so long to spot it.
>
> int sqlite3_os_init(void)
> {
> static sqlite3_vfs mqxVfs = {
>     1,                 /* iVersion */
>     0,   /* szOsFile ------------->ARRRGGGGGGGH <------------*/
>     MAX_PATH,          /* mxPathname */
>     etc ....
>
> I started creating the ported file from osWin.c, I had quite a struggle
> getting it to compile initially using CodeWarrior, so I was making several
> temporary hacks to allow it to compile, which I then revisited later to
> correct. I had replaced the sizeof(winFile) with a zero, and then forgot
> to change it to sizeof(mqxFile).
> This caused a fun crash down in the bowels of the paging code.
>
> Once I corrected this one liner it fixed the crash and sqlite is basically
> up and running now. I still have a fair bit of work left to implement some
> interface functions which are just dummy stubs at present, but I am making
> progress now.
>
> Regards Geoff
>
>
>
>
>

--
View this message in context: http://old.nabble.com/Porting-Sqlite-to-MQX-Operating-system-tp27792715p35333159.html
Sent from the SQLite mailing list archive at Nabble.com.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Porting Sqlite to MQX Operating system

arun.pradeep
In reply to this post by Dan Kennedy-4

Hi Geoff

  I am currently working on porting of SQLITE to MQX RTOS in MPC5125
Platform.

I am trying to rewrite the sqlite_os_init() for MQX and I have taken Win32
VFS implementation as the reference.

But I am not able to map all of the Win32 file system calls with that of MQX
functions.

Please advise me on the list of file system functions that can be
implemented as part of VFS for MQX.



Regards,
Arun


GeoffW wrote:

>
> hello
>
> I thought I had better update this and confess to my sins. Good job no one
> is reading this thread as it is an embarassingly stupid mistake :rules:
>
> I cant believe I did this and then took so long to spot it.
>
> int sqlite3_os_init(void)
> {
> static sqlite3_vfs mqxVfs = {
>     1,                 /* iVersion */
>     0,   /* szOsFile ------------->ARRRGGGGGGGH <------------*/
>     MAX_PATH,          /* mxPathname */
>     etc ....
>
> I started creating the ported file from osWin.c, I had quite a struggle
> getting it to compile initially using CodeWarrior, so I was making several
> temporary hacks to allow it to compile, which I then revisited later to
> correct. I had replaced the sizeof(winFile) with a zero, and then forgot
> to change it to sizeof(mqxFile).
> This caused a fun crash down in the bowels of the paging code.
>
> Once I corrected this one liner it fixed the crash and sqlite is basically
> up and running now. I still have a fair bit of work left to implement some
> interface functions which are just dummy stubs at present, but I am making
> progress now.
>
> Regards Geoff
>
>
>
>
>

--
View this message in context: http://old.nabble.com/Porting-Sqlite-to-MQX-Operating-system-tp27792715p35333161.html
Sent from the SQLite mailing list archive at Nabble.com.
_______________________________________________
sqlite-users mailing list
[hidden email]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Porting Sqlite to MQX Operating system

Richard Hipp-3
In reply to this post by arun.pradeep
On Thu, Apr 25, 2013 at 7:38 AM, arun.pradeep <[hidden email]> wrote:

>
> Hi Geoff
>
>   I am currently working on porting of SQLITE to MQX RTOS in MPC5125
> Platform.
>
> I am trying to rewrite the sqlite_os_init() for MQX and I have taken Win32
> VFS implementation as the reference.
>

Have you considered using the (much simpler) test_demovfs.c VFS as a
reference implementation?



>
> But I am not able to map all of the Win32 file system calls with that of
> MQX
> functions.
>
> Please advise me on the list of file system functions that can be
> implemented as part of VFS for MQX.
>
>
>
> Regards,
> Arun
>
>
> GeoffW wrote:
> >
> > hello
> >
> > I thought I had better update this and confess to my sins. Good job no
> one
> > is reading this thread as it is an embarassingly stupid mistake :rules:
> >
> > I cant believe I did this and then took so long to spot it.
> >
> > int sqlite3_os_init(void)
> > {
> > static sqlite3_vfs mqxVfs = {
> >     1,                 /* iVersion */
> >     0,   /* szOsFile ------------->ARRRGGGGGGGH <------------*/
> >     MAX_PATH,          /* mxPathname */
> >     etc ....
> >
> > I started creating the ported file from osWin.c, I had quite a struggle
> > getting it to compile initially using CodeWarrior, so I was making
> several
> > temporary hacks to allow it to compile, which I then revisited later to
> > correct. I had replaced the sizeof(winFile) with a zero, and then forgot
> > to change it to sizeof(mqxFile).
> > This caused a fun crash down in the bowels of the paging code.
> >
> > Once I corrected this one liner it fixed the crash and sqlite is
> basically
> > up and running now. I still have a fair bit of work left to implement
> some
> > interface functions which are just dummy stubs at present, but I am
> making
> > progress now.
> >
> > Regards Geoff
> >
> >
> >
> >
> >
>
> --
> View this message in context:
> http://old.nabble.com/Porting-Sqlite-to-MQX-Operating-system-tp27792715p35333159.html
> Sent from the SQLite mailing list archive at Nabble.com.
> _______________________________________________
> sqlite-users mailing list
> [hidden email]
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



--
D. Richard Hipp
[hidden email]
_______________________________________________
sqlite-users mailing list
[hidden email]
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Porting Sqlite to MQX Operating system

JCharavda
In reply to this post by GeoffW
Geoff,
Did you manage to get sqlite working with MQX?
-- Jay
Loading...