Re: Problem using ontology in multi-process environment

Posted by Jiba on
URL: http://owlready.306.s1.nabble.com/Problem-using-ontology-in-multi-process-environment-tp3198p3211.html

Hi,

Ontology (as well as most Owlready object) cannot be pickled, because they depend on an SQLite3 connection which is not pickleable.

I think you have two options:

1) Modify your program so as you do not need to pass ontology as an argument. Typically, each process should have its own World and Ontology, and then you can pass their IRI as argument. This can be done either by loading the ontology in each process, or by using fork() after loading the ontology, so as each process has a copy of the ontology (but fork() is known to be problematic on Mac).

2) Use GEvent parallelization. This is simpler to set up (and well supported by recent version of Owlready). However, it does not permit "true" parallelization. If your bottleneck is a calculation in Python, there will be no gain. If it is a calculation performed externally (e.g. a call to a remote database or a network download), the gain will be similar to classical multiprocess.

Jiba