NAME UMLS::Interface README SYNOPSIS This package provides a Perl interface to the Unified Medical Language System (UMLS). The UMLS is a knowledge representation framework encoded designed to support broad scope biomedical research queries. There exists three major sources in the UMLS. The Metathesaurus which is a taxonomy of medical concepts, the Semantic Network which categorizes concepts in the Metathesaurus, and the SPECIALIST Lexicon which contains a list of biomedical and general English terms used in the biomedical domain. The UMLS-Interface package is set up to access the Metathesaurus and the Semantic Network present in a MySQL database. INSTALL To install the module, run the following magic commands: perl Makefile.PL make make test make install This will install the module in the standard location. You will, most probably, require root privileges to install in standard system directories. To install in a non-standard directory, specify a prefix during the 'perl Makefile.PL' stage as: perl Makefile.PL PREFIX=/home/programs It is possible to modify other parameters during installation. The details of these can be found in the ExtUtils::MakeMaker documentation. However, it is highly recommended not messing around with other parameters, unless you know what you're doing. DATABASE SETUP The interface assumes that the UMLS is present as a mysql database. The names of these databases can be passed as configuration options at initialization. However, if the names of the database is not provided at initialization, then default values are used -- the database for the UMLS is called 'umls'. The UMLS database must contain six tables: 1. MRREL 2. MRCONSO 3. MRSAB 4. MRDOC 5. MRDEF 6. SRDEF 7. MRSTY All other tables in the databases will be ignored, and any of these tables missing would raise an error. The mysql server can be on the same machine as the module or could be on a remotely accessible machine. The location of the server can be provided during initialization of the module. INITIALIZING THE MODULE To create an instance of the interface object, using default values for all configuration options: use UMLS::Interface; my $interface = UMLS::Interface->new(); The following configuration options are also provided though: 'driver' -> Default value 'mysql'. This option specifies the Perl DBD driver that should be used to access the database. This implies that the some other DBMS system (such as PostgresSQL) could also be used, as long as there exist Perl DBD drivers to access the database. 'umls' -> Default value 'umls'. This option specifies the name of the UMLS database. 'hostname' -> Default value 'localhost'. The name or the IP address of the machine on which the database server is running. 'socket' -> Default value '/tmp/mysql.sock'. The socket on which the database server is using. 'port' -> The port number on which the database server accepts connections. 'username' -> Username to use to connect to the database server. If not provided, the module attempts to connect as an anonymous user. 'password' -> Password for access to the database server. If not provided, the module attempts to access the server without a password. 'forcerun' -> This parameter will bypass any command prompts such as asking if you would like to continue with the index creation. 'realtime' -> This parameter will not create a database of path information (what we refer to as the index) but obtain the path information about a concept on the fly 'cuilist' -> This parameter contains a file containing a list of CUIs in which the path information should be store for - if the CUI isn't on the list the path information for that CUI will not be stored 'verbose' -> This parameter will print out the table information to a config file in the UMLSINTERFACECONFIG directory 'config' -> This parameter contains the location of the config file These are passed through a hash. For example: my %options = (); $options{'config'} = $config; $options{'realtime'} = 1; my $interface = UMLS::Interface->new(\%options); Keep in mind that the database configuration options can be included in the MySQL my.cnf file. This is preferable. The directions for this are in the INSTALL file. It is Stage 5 Step D. These options can be reconfigured during run time using the reConfig() method. $options{'config'} = $newconfig; $interface->reConfig(\%options); USING THE MODULE Once the object of module is successfully created after following the steps described in the previous section, a number of methods can be called upon this object. The output of methods varies: @array refers to an array $array refers to a reference to an array $hash refers to a reference to a hash The methods are as follows: my $root = $interface->root(); Returns the concept ID of the root of the tree. my $depth = $interface->depth(); Returns the depth of the tree. my $version = $interface->version(); Return the version of UMLS. my $bool = $interface->exists($cui); Determines if a CUI exists my $bool = $interface->validCui($cui); Checks if CUI is a valid concept my $array = $interface->getSab($cui); Returns the list of sources the concept exists in my $array = $interface->getConceptList($term); Returns the list of all CUIs of a given term from the SAB parameter specified in the config file or the default my $array = $interface->getDefConceptList($term); Returns the list of all CUIs of a given term from the SABDEF parameter specified in the config file or the default my $array = $interface->getAllConcepts($term); Returns the list of all CUIs of a given term in the entire UMLS. my $hash = $interface->getCuiList(); Returns a list of CUIs from the source(s) specified in the configuration file my $array = $interface->getCuisFromSource($sab); Returns an list of CUIs in a specified source my $array = $interface->getCuisFromSource($sab); Returns a list of CUIs from a specific source my $array = $interface->getTermList($cui); Returns the list of terms and their sources using the SAB parameter in the configuration file or the default my $array = $interface->getDefTermList($cui); Returns the list of terms and their sources using the SABDEF parameter in the configuration file or the default my $array = $interface->getAllTerms($cui); Returns the list of terms corresponding to a CUI for all sources my $hash = $interface->getCompounds(); Returns all the compound terms in the sources specified in the configuration file. my $term = $interface->getPreferredTerms($cui); Returns the preferred term of a CUI if that term exists in the sources specified by the SAB parameter in the configuration file or the default my $term = $interface->getAllPreferredTerms($cui); Returns the preferred term of a CUI regardless of the source information in the configuration file my $array = $interface->getParents($cui); Returns the parent of a given CUI my $array = $interface->getChildren($cui); Returns the children of a given CUI my $array = $interface->getRelated($cui, $rel); Returns the CUI relations of a given CUI and relation my $array = $interface->getRelationsBetweenCuis($cui1, $cui2); Returns the relations between two CUIs. my $array$interface->getRelations($cui); Returns all of the relations associated with a CUI in the sources specified in the configuration file my $array = $interface->getCuiDef($cui); Returns the definition(s) of a given CUI my $array = $interface->getExtendedDefinition($cui); Returns the extended definition of a given CUI my $array = $interface->getSt($cui); Returns the TUI(s) of the semantic type(s) associated with a CUI my $abr = $interface->getStAbr($tui); Returns the abbreviation of a semantic type of a TUI my $tui = $interface->getStTui($abr); Returns the TUI of an abbreviation of a semantic type my $string = $interface->getStString($abr); Returns the name of the semantic type given its my $def = $interface->getStDef($abr); Returns the definition of a semantic type given its abbreviation my $array = $interface->getSemanticRelation($st1, $st2); Returns a list of semantic relation between the two semantic types. my $array = $interface->getSemanticGroup($cui1); Returns a list of semantic groups of a given CUI my $array = $interface->getSemanticGroupOfSt($st); Returns a list of semantic groups of a given semantic type my $array = $interface->pathsToRoot($cui); Returns a list of concept IDs that denote the path from the input CUI to the root using the source and relation information in the configuration file my $array = $interface->findShortestPath($cui1, $cui2); Returns the shortest path between two CUIs my $array = $interface->findLeastCommonSubsumer($cui1, $cui2); Returns the least common subsumer between two CUIs my $min = $interface->findMinimumDepth($cui); Returns the minimum depth of a CUI given the sources and relations specified in the configuration file my $max = $interface->findMaximumDepth($cui); Returns the maximum depth of a CUI given the sources and relations specified in the configuration file my $int = $interface->findNumberOfCloserConcepts($cui1, $cui2); Returns the number of concepts closer to cui1 than cui2 my $double = $interface->getIC($cui); Returns the information content of a CUI my $double = $interface->getProbability($cui); Returns the probability of a concept my $int = $interface->getFrequency($cui); Returns the frequency of a CUI that was used to calculate its information content and probability my $N = $interface->getN(); Returns the total number of CUIs the probabilities were calculated with my $hash = $interface->getPropagationCuis(); Returns a list of CUIs that the counts were propagated over my $hash = $interface->propagateCounts(\%hash); Returns the propagation counts of the input CUIs my $array = $interface->stPathsToRoot($tui); Returns all the path to the root information of the given semantic type (TUI) my $array = $interface->stFindShortestPath($tui1, $tui2); Returns the shortest paths between the two semantic types (TUIs) my $double = $interface->getStIC($tui); Returns the information content of a given semantic type (TUI) my $double = $interface->getStProbability($tui); Returns the probability of a given semantic type (TUI) my $stN = $interface->getStN(); Returns the total number of semantic types used to obtain the probability of a semantic type $interface->setPropagationParameters(\%parameters); Sets the propagation parameters $interface->setStSmoothing(); Sets the smoothing parameter to smooth the input counts my $hash = $interface->propagateStCounts(\%hash); Returns the propagation counts of the input semantic types $interface->loadStPropagationHash(\%hash); Load the propagation hash with probability counts my $hash = $interface->returnTableNames(); Returns the mysql database table names in human and hex form created by the package for a given configuration $interface->dropConfigTable(); Drops the temporary table created by the UMLS-Interface module of path information for a specified set of sources $interface->removeConfigFiles(); Removes the configuration files created by the verbose option These methods essentially expose an interface as required by the UMLS::Similarity modules. The UMLS::Similarity modules require that any interface to a taxonomy provide the above methods. CONFIGURATION UMLS-Interface allows information to be extracted from the UMLS given a specified set of sources and relations through the use of a configuration file. There are six configuration options: SAB, REL, RELA, SABDEF, RELDEF, and RELADEF. The SAB and REL options are used to determine which sources and relations the path information is to be obtained from. The RELA option narrows down the relation even further. The RELA will only be applied to the PAR/CHD and RB/RN relations. The SABDEF and RELDEF options are used to determine which sources and relations to use when creating the EXTENDED DEFINITION. The RELA option narrows down the relation even further. The RELADEF will only be applied to the PAR/CHD and RB/RN relations. You can specify a single source, multiple sources or the entire UMLS (using the UMLS_ALL option). Keep in mind that the greater the number of sources the larger the search space so if you obtaining path information about two concepts this will take longer. The names of the sources in the configuration file are expected to be in the SAB (source abbreviation) form. A listing of the sources and their SABs can be found: You can specify any relations that exist in the specified set of sources that you defined. The directional (hierarchical) relations though are PAR/CHD and RB/RN. The other relations (such as RO and SIB) are not directional which means when obtaining path information when using these relations may take much longer than obtaining path information using the directional relations. A listing of the different relations can be found here (scroll down to the REL table): If you do plan on using a multiple sources or the entire UMLS, we would advise you to use the --realtime option which is explained below, in the Interface.pm documentation and the path programs in the utils/ directory. We also have a am UMLS_ALL option for this so you do not have to specify each and every source and relation. The format of the configuration file is as follows: SAB :: REL :: RELA :: For example, if we wanted to use the MSH vocabulary with only the RB/RN relations, the configuration file would be: SAB :: include MSH REL :: include RB, RN or SAB :: include MSH REL :: exclude PAR, CHD If we wanted to use the SNOMEDCT vocabulary with only the PAR/CHD relations that are is-a relations, the configuration file would be: SAB :: include SNOMEDCT REL :: include PAR, CHD RELA :: include isa, inverse_isa The format for SABDEF and RELDEF is similar. The SABDEF and RELDEF options are used to determine the sources and relations the extended definition is to be obtained from. The format of the configuration file is as follows: SABDEF :: RELDEF :: RELADEF :: Note: RELDEF takes any of MRREL relations and two special 'relations': 1. CUI which refers to the CUIs definition 2. TERM which refers to the terms associated with the CUI For example, if we wanted to use the definitions from MSH vocabulary and we only wanted the definition of the CUI and the definitions of the CUIs SIB relation, the configuration file would be: SABDEF :: include MSH RELDEF :: include CUI, SIB If you wanted only the PAR/CHD definitions which are is-a relations. SABDEF :: include MSH RELDEF :: include PAR, CHD RELADEF :: include isa, inverse_isa For all of these options, there is an UMLS_ALL tag. If used with SAB or SABDEF, it would include all of the UMLS sources. If used with the REL or RELDEF, it would include all of the possible relations (as well as CUI and TERM for RELDEF). If used with the RELA or RELADEF, it would include all of the RELA relations including those with no RELA relation. Note that this is also the default for this option which is why it is optional. An example of using the UMLS_ALL option is as follows: SAB :: include UMLS_ALL REL :: include UMLS_ALL and another is: SABDEF :: include UMLS_ALL RELDEF :: include UMLS_ALL If you go to the configuration file directory, there will be example configuration files for the different runs that you have performed. For more information about the configuration options please see the README. PROPAGATION The Information Content (IC) is defined as the negative log of the probability of a concept. The probability of a concept, c, is determine by summing the probability of the concept occurring in some text plus the probability its descendants occurring in some text. The following is an example of the method UMLS-Interface uses to propagation counts to determine the probability of a concept in the sources/relations specified in the configuration file. In this method, we percolate the counts up the hierarchy, and in the case of multiple inheritance, we send a full count up all the paths to the parent. The icfrequency file contains the frequency of the following concepts existing in some corpus. For example, our corpus consists of three concepts, A B & C, each occurring five times: SAB :: N:15 A<>5 B<>5 C<>5 In this case, our sources and relations consist of the following 'graph': Notation....A->D means A is a child of D.... A->D B->D B->E D->F C->E E->F So A B and C are "leaf" nodes and F is the root. Step 1: determine the descendants of each nodes Descendants(A) = {} Descendants(B) = {} Descendants(C) = {} Descendants(D) = {A, B} Descendants(E) = {B, C} Descendants(F) = {A, B, C, D, E, F} Step 2: determine the probability of a concept, P(c), occurring by summing the probability of each of descendants plus its probability. P(A) = freq(A) / N = .33 P(B) = freq(B) / N = .33 P(C) = freq(C) / N = .33 P(D) = (freq(A)+freq(B)+freq(D)) / N = .66 P(E) = (freq(B)+freq(C)+freq(E)) / N = .66 P(F) = (freq(A)+freq(B)+freq(C)+freq(D)+freq(E)+freq(F)) / N = .99 Step 3: print the probability of the concept occurring, P(c), for each node in the sources/relations defined in the configuration table. SMOOTH :: 0 <- or 1 if smoothing was used SAB :: REL :: RELA :: <- if any are specified in the config A<>.33 B<>.33 C<>.33 D<>.66 E<>.66 F<>.99 The information content for the nodes is then calculated by taking -log of this probability. We have an option that incorporates Laplace smoothing. Laplace smoothing is where the frequency count of each of the concepts in the taxonomy is incremented by one. The advantage of doing this is that it avoids having a concept that has a probability of zero. The disadvantage is that it can shift the overall probability mass of the concepts from what is actually seen in the corpus. REFERENCING If you write a paper that has used UMLS-Interface in some way, we'd certainly be grateful if you sent us a copy and referenced UMLS-Interface. We have a published paper that provides a suitable reference: @inproceedings{McInnesPP09, title={{UMLS-Interface and UMLS-Similarity : Open Source Software for Measuring Paths and Semantic Similarity}}, author={McInnes, B.T. and Pedersen, T. and Pakhomov, S.V.}, booktitle={Proceedings of the American Medical Informatics Association (AMIA) Symposium}, year={2009}, month={November}, address={San Fransisco, CA} } This paper is also found in or CONTACT US If you have any trouble installing and using UMLS-Interface, please contact us via the users mailing list : umls-similarity@yahoogroups.com You can join this group by going to: You may also contact us directly if you prefer : Bridget T. McInnes: bthomson at cs.umn.edu Ted Pedersen : tpederse at d.umn.edu SOFTWARE COPYRIGHT AND LICENSE Copyright (C) 2004-2010 Bridget T McInnes, Siddharth Patwardhan, Serguei Pakhomov, Ying Liu and Ted Pedersen This suite of programs is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. Note: The text of the GNU General Public License is provided in the file 'GPL.txt' that you should have received with this distribution.