Re: Problem using ontology in multi-process environment

Posted by fabad on
URL: http://owlready.306.s1.nabble.com/Problem-using-ontology-in-multi-process-environment-tp3198p3212.html

Hi Jiba, thanks for your answer! I did not know how owlready2 works internally.

Finally, as I am working in a shared memory environment, I was able to open the ontology in the main thread and to use the memory address as parameter to each parallel thread. Then, each thread reads the memory address and make a cast to the ontology object. something like the following (not tested as I dont have a python compiler right now):

import pathlib
from concurrent.futures import ProcessPoolExecutor
from owlready2 import get_ontology
import time
import ctypes

def get_iri(ontology_address, job_id):
    ontology = ctypes.cast(ontology_address).value
    return f"job {job_id} -> {ontology.iri}"

if __name__ == '__main__':
    ontology_file_path = pathlib.Path("/home/fabad/test_embed_comp/go.owl")
    print("Loading ontology")
    ontology = get_ontology(f"file://{str(ontology_file_path)}").load()
        ontology_address = id(ontology)
    print("Ontology loaded")
    executor = ProcessPoolExecutor(max_workers=4)
    results = {}
    for i in range(30):
        results[i] = executor.submit(get_iri, ontology_address, i)

    executor.shutdown(wait=True)

    for n, future in results.items():
        print(f'{future.result()}')