Owlready2 sqlite3 quadstore - database lock

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Owlready2 sqlite3 quadstore - database lock

costeira
After analysing the Parallelism page from the documentation and the flask example, I still have some problems with the quadstore, getting easily the database lock error.
So let me explain what I'm doing:
I wrote a small ontology (using the owlready2 classes), stored it in an owl file and saved the default_world in a sqlite3 file.

Now a generic script that I'm using to access that ontology is:

>from owlready2 import *
>my_world = World(filename = "./ontology.sqlite3", exclusive = False)
>onto = my_world.get_ontology("...").load()

>with onto:
>   # For Example insert or read operation
>  my_world.save()

My main goal is simply having an .sqlite3 as a main database for triples storage, more specifically, I want to store a bunch of instances of classes and respective relations in one database.
But when I try to use the sqlite3 file in a django project or simply having a couple of jupiter notebooks accessing the same a sqlite3, I instantly get the database lock error.
Based on the documentation, I thought that by simply adding the "exclusive = False" flag, I wouldn't have locks issues.
Can someone explain me what is the issue or what am I doing wrong. Honestly I didn't really understood that small flask example in the documentation, that's why I'm seeking some help to figure out what is the issue.
Reply | Threaded
Open this post in threaded view
|

Re: Owlready2 sqlite3 quadstore - database lock

Jiba
Administrator
Hi,

I just improved the parallelism system in the development version of Owlready on Bitbucket. It should now support multiprocessing, but also distinct programs sharing the same quadstore (that was not the case previously).

You can get database lock if the database is never released. Owlready locks the database when you start a "with onto:" block, and releases it when you call World.save(). So you need to call World.save() regularly.

In addition, it seems that releasing and then relocking the database very quickly may not permit another programs to take the lock. For example:

with onto:
    class Drug(Thing): pass
 
for i in range(10000):
    with onto:
        onto.Drug(label = ["xxx"])
    default_world.save()

This program releases the lock after each creation of an instance, but then relocks the database in the next iteration.

This problem can be solved by adding a small sleep after World.save():

for i in range(10000):
    with onto:
        onto.Drug(label = ["xxx"])
    default_world.save()
    time.sleep(0.0001)

Jiba
Reply | Threaded
Open this post in threaded view
|

Re: Owlready2 sqlite3 quadstore - database lock

costeira
Thank you Jiba.
I'm going to install the newest version of the package and try the sleeping after saving method.
I know that I'm repeating myself, but thank you for the update and the explanation.