{ "cells": [ { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "# Genetic distance calculation\n", "\n", "## Fast pairwise distance estimation\n", "\n", "For a limited number of evolutionary models a fast implementation is available." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "pycharm": { "is_executing": false } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
Specify a pairwise genetic distance calculator using 'Abbreviation' (case insensitive).
AbbreviationSuitable for moltype
paralineardna, rna, protein
logdetdna, rna, protein
jc69dna, rna
tn93dna, rna
hammingdna, rna, protein, text
percentdna, rna, protein, text
\n", "

\n", "6 rows x 2 columns

" ], "text/plain": [ "Specify a pairwise genetic distance calculator using 'Abbreviation' (case insensitive).\n", "=======================================\n", "Abbreviation Suitable for moltype\n", "---------------------------------------\n", " paralinear dna, rna, protein\n", " logdet dna, rna, protein\n", " jc69 dna, rna\n", " tn93 dna, rna\n", " hamming dna, rna, protein, text\n", " percent dna, rna, protein, text\n", "---------------------------------------\n", "\n", "6 rows x 2 columns" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from cogent3 import available_distances\n", "\n", "available_distances()" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "## Computing genetic distances using the `Alignment` object\n", "\n", "Abbreviations listed from `available_distances()` can be used as values for the `distance_matrix(calc=)`." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false, "pycharm": { "is_executing": false, "name": "#%%\n" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
ChimpanzeeGalagoGorillaHowlerMonHumanOrangutanRhesus
Chimpanzee0.0000.1920.0050.0700.0090.0140.040
Galago0.1920.0000.1920.2160.1960.1940.196
Gorilla0.0050.1920.0000.0700.0090.0140.039
HowlerMon0.0700.2160.0700.0000.0740.0720.074
Human0.0090.1960.0090.0740.0000.0170.042
Orangutan0.0140.1940.0140.0720.0170.0000.041
Rhesus0.0400.1960.0390.0740.0420.0410.000
\n" ], "text/plain": [ "===========================================================================================\n", " Chimpanzee Galago Gorilla HowlerMon Human Orangutan Rhesus\n", "-------------------------------------------------------------------------------------------\n", "Chimpanzee 0.0000 0.1921 0.0054 0.0704 0.0089 0.0140 0.0396\n", " Galago 0.1921 0.0000 0.1923 0.2157 0.1965 0.1944 0.1962\n", " Gorilla 0.0054 0.1923 0.0000 0.0700 0.0086 0.0137 0.0393\n", " HowlerMon 0.0704 0.2157 0.0700 0.0000 0.0736 0.0719 0.0736\n", " Human 0.0089 0.1965 0.0086 0.0736 0.0000 0.0173 0.0423\n", " Orangutan 0.0140 0.1944 0.0137 0.0719 0.0173 0.0000 0.0411\n", " Rhesus 0.0396 0.1962 0.0393 0.0736 0.0423 0.0411 0.0000\n", "-------------------------------------------------------------------------------------------" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from cogent3 import load_aligned_seqs\n", "aln = load_aligned_seqs('../data/primate_brca1.fasta', moltype=\"dna\")\n", "dists = aln.distance_matrix(calc=\"tn93\", show_progress=False)\n", "dists" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false }, "source": [ "## Using the distance calculator directly" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false, "pycharm": { "is_executing": false, "name": "#%%\n" } }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from cogent3 import load_aligned_seqs, get_distance_calculator\n", "aln = load_aligned_seqs('../data/primate_brca1.fasta')\n", "dist_calc = get_distance_calculator(\"tn93\", alignment=aln)\n", "dist_calc" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false, "pycharm": { "is_executing": false, "name": "#%%\n" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
ChimpanzeeGalagoGorillaHowlerMonHumanOrangutanRhesus
Chimpanzee0.0000.1920.0050.0700.0090.0140.040
Galago0.1920.0000.1920.2160.1960.1940.196
Gorilla0.0050.1920.0000.0700.0090.0140.039
HowlerMon0.0700.2160.0700.0000.0740.0720.074
Human0.0090.1960.0090.0740.0000.0170.042
Orangutan0.0140.1940.0140.0720.0170.0000.041
Rhesus0.0400.1960.0390.0740.0420.0410.000
\n" ], "text/plain": [ "===========================================================================================\n", " Chimpanzee Galago Gorilla HowlerMon Human Orangutan Rhesus\n", "-------------------------------------------------------------------------------------------\n", "Chimpanzee 0.0000 0.1921 0.0054 0.0704 0.0089 0.0140 0.0396\n", " Galago 0.1921 0.0000 0.1923 0.2157 0.1965 0.1944 0.1962\n", " Gorilla 0.0054 0.1923 0.0000 0.0700 0.0086 0.0137 0.0393\n", " HowlerMon 0.0704 0.2157 0.0700 0.0000 0.0736 0.0719 0.0736\n", " Human 0.0089 0.1965 0.0086 0.0736 0.0000 0.0173 0.0423\n", " Orangutan 0.0140 0.1944 0.0137 0.0719 0.0173 0.0000 0.0411\n", " Rhesus 0.0396 0.1962 0.0393 0.0736 0.0423 0.0411 0.0000\n", "-------------------------------------------------------------------------------------------" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dist_calc.run(show_progress=False)\n", "dists = dist_calc.get_pairwise_distances()\n", "dists" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "The distance calculation object can provide more information. For instance, the standard errors.\n", "\n" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false, "pycharm": { "is_executing": false, "name": "#%%\n" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
Standard Error of Pairwise Distances
Seq1 \\ Seq2GalagoHowlerMonRhesusOrangutanGorillaHumanChimpanzee
Galago00.01030.00960.00950.00950.00960.0095
HowlerMon0.010300.00540.00530.00530.00540.0053
Rhesus0.00960.005400.00390.00390.00400.0039
Orangutan0.00950.00530.003900.00220.00250.0023
Gorilla0.00950.00530.00390.002200.00180.0014
Human0.00960.00540.00400.00250.001800.0018
Chimpanzee0.00950.00530.00390.00230.00140.00180
\n", "

\n", "7 rows x 8 columns

" ], "text/plain": [ "Standard Error of Pairwise Distances\n", "============================================================================================\n", "Seq1 \\ Seq2 Galago HowlerMon Rhesus Orangutan Gorilla Human Chimpanzee\n", "--------------------------------------------------------------------------------------------\n", " Galago 0 0.0103 0.0096 0.0095 0.0095 0.0096 0.0095\n", " HowlerMon 0.0103 0 0.0054 0.0053 0.0053 0.0054 0.0053\n", " Rhesus 0.0096 0.0054 0 0.0039 0.0039 0.0040 0.0039\n", " Orangutan 0.0095 0.0053 0.0039 0 0.0022 0.0025 0.0023\n", " Gorilla 0.0095 0.0053 0.0039 0.0022 0 0.0018 0.0014\n", " Human 0.0096 0.0054 0.0040 0.0025 0.0018 0 0.0018\n", " Chimpanzee 0.0095 0.0053 0.0039 0.0023 0.0014 0.0018 0\n", "--------------------------------------------------------------------------------------------\n", "\n", "7 rows x 8 columns" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dist_calc.stderr" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "## Likelihood based pairwise distance estimation\n", "\n", "The standard ``cogent3`` likelihood function can also be used to estimate distances. Because these require numerical optimisation they can be significantly slower than the fast estimation approach above.\n", "\n", "The following will use the F81 nucleotide substitution model and perform numerical optimisation." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false, "pycharm": { "is_executing": false, "name": "#%%\n" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
ChimpanzeeGalagoGorillaHowlerMonHumanOrangutanRhesus
Chimpanzee0.0000.1890.0050.0700.0090.0140.039
Galago0.1890.0000.1890.2110.1930.1920.193
Gorilla0.0050.1890.0000.0690.0090.0140.039
HowlerMon0.0700.2110.0690.0000.0730.0710.073
Human0.0090.1930.0090.0730.0000.0170.042
Orangutan0.0140.1920.0140.0710.0170.0000.041
Rhesus0.0390.1930.0390.0730.0420.0410.000
\n" ], "text/plain": [ "===========================================================================================\n", " Chimpanzee Galago Gorilla HowlerMon Human Orangutan Rhesus\n", "-------------------------------------------------------------------------------------------\n", "Chimpanzee 0.0000 0.1892 0.0054 0.0697 0.0089 0.0140 0.0395\n", " Galago 0.1892 0.0000 0.1891 0.2112 0.1934 0.1915 0.1930\n", " Gorilla 0.0054 0.1891 0.0000 0.0693 0.0086 0.0136 0.0391\n", " HowlerMon 0.0697 0.2112 0.0693 0.0000 0.0729 0.0713 0.0729\n", " Human 0.0089 0.1934 0.0086 0.0729 0.0000 0.0173 0.0421\n", " Orangutan 0.0140 0.1915 0.0136 0.0713 0.0173 0.0000 0.0410\n", " Rhesus 0.0395 0.1930 0.0391 0.0729 0.0421 0.0410 0.0000\n", "-------------------------------------------------------------------------------------------" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from cogent3 import load_aligned_seqs, get_model\n", "from cogent3.evolve import distance\n", "\n", "aln = load_aligned_seqs('../data/primate_brca1.fasta', moltype=\"dna\")\n", "d = distance.EstimateDistances(aln, submodel=get_model(\"F81\"))\n", "d.run(show_progress=False)\n", "dists = d.get_pairwise_distances()\n", "dists" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "pycharm": { "name": "#%% md\n" } }, "source": [ "All `cogent3` substitution models can be used for distance calculation via this approach, with the caveat that identifiability issues mean this is not possible for some non-stationary model classes." ] } ], "metadata": { "kernelspec": { "display_name": "Python [conda env:c3dev] *", "language": "python", "name": "conda-env-c3dev-py" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.1" }, "pycharm": { "stem_cell": { "cell_type": "raw", "metadata": { "collapsed": false }, "source": [] } } }, "nbformat": 4, "nbformat_minor": 4 }