Impossible to load SNOMEDCT_US, ICD10 and CUI

[Nobu]

Hi,

It seems that importing UMLS data worked because I have my 'pym.sqlite3' file saved.

Then when I do the following lines :

from owlready2 import *
default_world.set_backend(filename = "pym.sqlite3")
PYM = get_ontology("http://PYM/").load()
SNOMEDCT_US = PYM["SNOMEDCT_US"]

The last one gives me this error :

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/nobu/.local/lib/python3.6/site-packages/owlready2/pymedtermino2/model.py", line 30, in get_terminology
    def get_terminology(self, name): return self._src["%s" % name]
  File "/home/nobu/.local/lib/python3.6/site-packages/owlready2/namespace.py", line 74, in __getitem__
    def __getitem__(self, name): return self.world["%s%s" % (self.base_iri, name)]
  File "/home/nobu/.local/lib/python3.6/site-packages/owlready2/namespace.py", line 530, in __getitem__
    return self._get_by_storid(storid, iri)
  File "/home/nobu/.local/lib/python3.6/site-packages/owlready2/namespace.py", line 537, in _get_by_storid
    return self._load_by_storid(storid, full_iri, main_type, main_onto, default_to_none)
  File "/home/nobu/.local/lib/python3.6/site-packages/owlready2/namespace.py", line 578, in _load_by_storid
    if obj2 is None: obj2 = self._load_by_storid(obj, None, main_type, main_onto, default_to_none, trace)
  File "/home/nobu/.local/lib/python3.6/site-packages/owlready2/namespace.py", line 592, in _load_by_storid
    full_iri = full_iri or self._unabbreviate(storid)
  File "/home/nobu/.local/lib/python3.6/site-packages/owlready2/triplelite.py", line 477, in _unabbreviate_sql
    return self.execute("SELECT iri FROM resources WHERE storid=? LIMIT 1", (storid,)).fetchone()[0]
TypeError: 'NoneType' object is not subscriptable

What can I do to fix it ? And by the way I'm under Python 3.6.8 (cf. 'However, after importing the data in the quadstore, it can be used with Python 3.6 if you really need to').

Thanks by advance.

Nobu

[Jiba]

Hi,

I've just verified: it works well with Python 3.6.8 (after importing the data from UMLS), so the version of Python is not the problem.

Could you try the following lines:

First, please verify that SNOMED CT is present in the quadstore:

>>> PYM["SRC"].children
[PYM["CIM10"] # CIM10
, PYM["ICD10"] # ICD10
, PYM["SNOMEDCT_US"] # US Edition of SNOMED CT
]

Second, you can verify the StorID associated with SNOMED CT as follows:

>>> default_world._abbreviate("http://PYM/SRC/SNOMEDCT_US", False)

Could you also show me how you imported UMLS?

Jiba

[Nobu]

When I try :

>>> PYM["SRC"].children
[PYM["CIM10"] # CIM10
, PYM["ICD10"] # ICD10
, PYM["SNOMEDCT_US"] # US Edition of SNOMED CT
]

It shows me the same error than before.

About this command :

>>> default_world._abbreviate("http://PYM/SRC/SNOMEDCT_US", False)

The result is :

318

And about how I have imported UMLS :

>>> from owlready2 import *
>>> from owlready2.pymedtermino2 import *
>>> from owlready2.pymedtermino2.umls import *
>>> default_world.set_backend(filename = "pym.sqlite3")
>>> import_umls("umls-2020AA-metathesaurus.zip", terminologies = ["ICD10", "SNOMEDCT_US", "CUI"])

With that result :

Importing UMLS from /home/jiba/telechargements/base_med/2018AB-full/2018AB/META/...
  Parsing MRRANK.RRF as MRRANK...
  Parsing MRCONSO.RRF as MRCONSO...
  Parsing MRDEF.RRF as MRDEF...
  Parsing MRREL.RRF as MRREL...
  Parsing MRSAT.RRF as MRSAT...
Breaking ORIG cycles...
    SNOMEDCT_US : 0 cycles found:
    ICD10 : 0 cycles found:
    SRC : 0 cycles found:
Finalizing only properties and restrictions...
Finalizing CUI - ORIG mapping...
Indexing...
FTS Indexing...

Thank you by advance

[Jiba]

Hi,

I verified with the same version of UMLS (2020) and Python (3.6.8), but I cannot reproduce the problem.

Possibly something went wrong during UMLS import, and you should retry it? You should obtain a 2,5 Gb Sqlite3 file.

Jiba

[Nobu]

Hi,

I did it again and I have the pym.sqlite3 file (2,5 GB) and the pym.sqlite3-journal file (93 Mo).
But I have the same error message when I try to load SNOMEDCT_US.

I am sure I'm doing something wrong.

Could you please show me how do you load SNOMEDCT_US after you got your pym files ?

Best regards

[Jiba]

Hi,

Here is a small script that just loads the quadstore and shows the terminologies available:


from owlready2 import *
import sys, os, time


QUADSTORE = "/home/jiba/tmp/pym.sqlite3"
default_world.set_backend(filename = QUADSTORE)

PYM = get_ontology("http://PYM/").load()

SNOMEDCT_US = PYM["SNOMEDCT_US"]
print(PYM["SRC"].children)


Best regards,
Jiba

[Nobu]

Thank you very much. This is how I was doing it too.

But the problem is still here.

Do you have any other idea of what could be the problem ?

Thank you for your time and attention.

[Jiba]

Hi,

We need to debug the content of the quadstore... Could you try the following lines?

Please adapt them according to the results obtained, e.g. if you get 320 instead of 318 with the first lines, then use 320 in the second, etc (you should have the same ID, but since something is going wrong, it may not be the case).


>>> default_world.graph.execute("select storid from resources where iri='http://PYM/SRC/SNOMEDCT_US'").fetchall()
[(318,)]

>>> for t in default_world.graph.execute("select * from quads where s=318"): print(t)
...
(1, 318, 6, 11, None)
(1, 318, 9, -594089, None)
(1, 318, 9, 313, None)
(1, 318, 9, -4050991, None)
(1, 318, 310, 313, None)
(1, 318, 40, 'US Edition of SNOMED CT', '@en')
(1, 318, 309, 'SNOMED CT, US Edition', '@en')
(1, 318, 309, 'SNOMEDCT_US', '@en')
(1, 318, 309, 'SNOMED CT Concept', '@en')
>>> for t in default_world.graph.execute("select * from quads where s=-594089"): print(t)
...
(1, -594089, 6, 17, None)
(1, -594089, 18, 305, None)
(1, -594089, 24, 741269, None)
>>> for t in default_world.graph.execute("select * from quads where s=-4050991"): print(t)
...
(1, -4050991, 6, 17, None)
(1, -4050991, 18, 305, None)
(1, -4050991, 25, 741269, None)
>>> for t in default_world.graph.execute("select * from quads where s=313"): print(t)
...
(1, 313, 6, 11, None)
(1, 313, 9, -423993, None)
(1, 313, 9, 302, None)
(1, 313, 9, -3966612, None)
(1, 313, 310, 313, None)
(1, 313, 40, 'Metathesaurus Source Terminology Names', '@en')
(1, 313, 309, 'Source Terminology Names (UMLS)', '@en')
(1, 313, 309, 'SRC', '@en')

[Nobu]

Hi,

Thank you for your answer. Here is my output :

>>> default_world.graph.execute("select storid from resources where iri='http://PYM/SRC/SNOMEDCT_US'").fetchall()
[(318,)]
>>> for t in default_world.graph.execute("select * from quads where s=318"): print(t)
...
(1, 318, 6, 11, None)
(1, 318, 9, -594089, None)
(1, 318, 9, 313, None)
(1, 318, 9, -4050840, None)
(1, 318, 310, 313, None)
(1, 318, 40, 'US Edition of SNOMED CT', '@en')
(1, 318, 309, 'SNOMED CT, US Edition', '@en')
(1, 318, 309, 'SNOMEDCT_US', '@en')
(1, 318, 309, 'SNOMED CT Concept', '@en')
>>> for t in default_world.graph.execute("select * from quads where s=-594089"): print(t)
...
(1, -594089, 6, 17, None)
(1, -594089, 18, 305, None)
(1, -594089, 24, 741269, None)
>>> for t in default_world.graph.execute("select * from quads where s=-4050840"): print(t)
...
(1, -4050840, 6, 17, None)
(1, -4050840, 18, 305, None)
(1, -4050840, 25, 741269, None)
>>> for t in default_world.graph.execute("select * from quads where s=313"): print(t)
...
(1, 313, 6, 11, None)
(1, 313, 9, -423993, None)
(1, 313, 9, 302, None)
(1, 313, 9, -3966461, None)
(1, 313, 310, 313, None)
(1, 313, 40, 'Metathesaurus Source Terminology Names', '@en')
(1, 313, 309, 'Source Terminology Names (UMLS)', '@en')
(1, 313, 309, 'SRC', '@en')

[Jiba]

Ok, it's similar to my output, but some ID differ...

Could you now try:

PYM._parse_bnode(-594089)

PYM._parse_bnode(-4050840)

PYM._parse_bnode(-423993)

PYM._parse_bnode(-3966461)


I expect that at least one of these lines will raise an error.

Jiba

[Nobu]

The output of the four commands is the following one :

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/nobu/.local/lib/python3.6/site-packages/owlready2/class_construct.py", line 317, in __repr__
    return """%s.%s(%s)""" % (self.property, _restriction_type_2_label[self.type], repr(self.value))
  File "/home/nobu/.local/lib/python3.6/site-packages/owlready2/class_construct.py", line 350, in __getattr__
    v = self.__dict__["value"] = self.ontology.world._to_python(v, None, default_to_none = True)
  File "/home/nobu/.local/lib/python3.6/site-packages/owlready2/namespace.py", line 261, in _to_python
    else: return self.world._get_by_storid(o, None, main_type, main_onto, None, default_to_none)
  File "/home/nobu/.local/lib/python3.6/site-packages/owlready2/namespace.py", line 537, in _get_by_storid
    return self._load_by_storid(storid, full_iri, main_type, main_onto, default_to_none)
  File "/home/nobu/.local/lib/python3.6/site-packages/owlready2/namespace.py", line 578, in _load_by_storid
    if obj2 is None: obj2 = self._load_by_storid(obj, None, main_type, main_onto, default_to_none, trace)
  File "/home/nobu/.local/lib/python3.6/site-packages/owlready2/namespace.py", line 592, in _load_by_storid
    full_iri = full_iri or self._unabbreviate(storid)
  File "/home/nobu/.local/lib/python3.6/site-packages/owlready2/triplelite.py", line 477, in _unabbreviate_sql
    return self.execute("SELECT iri FROM resources WHERE storid=? LIMIT 1", (storid,)).fetchone()[0]
TypeError: 'NoneType' object is not subscriptable

[Nobu]

For anyone having the same problem than mine, I solved the issue by importing UMLS through the Windows command line.

The key was maybe that I had Python 3.7 installed on Windows 10.

So back on my linux bash shell for Windows 10 (with Python 3.6.8), I was able to load UMLS terminolgies.

Many thanks to jiba for his time and dedication in solving my problem.