Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Use

Run aphid with --help to check possible command line arguments.

$ aphid --help
Aphid: distinguishing gene flow from incomplete lineage sorting

Usage: aphid [OPTIONS] <CONFIG> <OUTPUT>

Arguments:
  <CONFIG>  Path to the config file
  <OUTPUT>  Path to the output folder, created if missing

Options:
  -f, --force    Raise to overwrite previous output folder
  -h, --help     Print help
  -V, --version  Print version

All aphid method configuration is specified in the special <CONFIG> file given as argument. When aphid runs, it writes all resulting files in the given <OUTPUT> folder.

For instance:

$ aphid ./config.toml output

Configuration

Aphid configuration file is written in TOML format. Here is one example configuration file:

config.toml

gf_times = [1, 0.5]
unresolved_mutations_count = 0.5

[filters]
max_clock_ratio = 2
triplet_other_monophyly = false

[taxa]
trees = "path/to/homininae.in"
triplet = [["Homo_sapiens", "Pan_troglodytes"], "Gorilla_gorilla"]
outgroup = """
    Macaca_fascicularis
    Macaca_mulatta
    Macaca_nemestrina
    Rhinopithecus_bieti
    Rhinopithecus_roxellana
    Colobus_angolensis
    Piliocolobus_tephrosceles
    Cercocebus_atys
    Papio_anubis
    Mandrillus_leucophaeus
    Chlorocebus_sabaeus
"""
other = "Pongo_abelii Nomascus_leucogenys"

Configuration detail is described below.

The [taxa] table

The major source of input to aphid is specified under the [taxa] table.

taxa.trees

This path is where you specify the (possibly large) input file containing all gene trees for the species of interest. Every line in this file must have the following format:

<tree> <TAB> <sequence_length> <TAB> <identifier>

The <tree> part being a Newick representation of the gene tree, with branches lengths specified in unit of 'per site substitution'.

The path is either absolute or relative to the configuration file.

For instance:

homininae.in

(((((Chrysochloris_asiatica:0.106818,Orycteropus_afer_afer:0.050232):0.010133,(Loxodonta_africana:0.07851,Trichechus_manatus_latirostris:0.039048):0.019167):0.003677,Elephantulus_edwardii:0.122616):0.01745,(((((((Aotus_nancymaae:0.017014,Callithrix_jacchus:0.026321):0.007573,(Cebus_capucinus:0.013507,Saimiri_boliviensis:0.01193):0.001501):0.010954,((((((Cercocebus_atys:0,Papio_anubis:0):0,Chlorocebus_sabaeus:0):0,Mandrillus_leucophaeus:0.00555):0,(Macaca_fascicularis:0.004147,(Macaca_mulatta:0.002851,Macaca_nemestrina:0.002761):0):0.008324):0.00419,((Colobus_angolensis:0.002761,Piliocolobus_tephrosceles:0):0.001369,(Rhinopithecus_bieti:0.001378,Rhinopithecus_roxellana:0):0.014014):0.001358):0.008566,((((Gorilla_gorilla:0.001378,(Pan_paniscus:0.001378,Pan_troglodytes:0):0.001379):0,Homo_sapiens:0.002759):0.006318,Pongo_abelii:0.009094):0.003499,Nomascus_leucogenys:0.011152):0.007845):0.006462):0.024715,(Carlito_syrichta:0.059794,((Microcebus_murinus:0.023627,Propithecus_coquereli:0.030593):0.003629,Otolemur_garnettii:0.070185):0.025167):0.000565):0.007572,Galeopterus_variegatus:0.061478):0.007692,(((Castor_canadensis:0.05853,(Ictidomys_tridecemlineatus:0.006922,Marmota_marmota_marmota:0.011723):0.135009):0.009743,(((Cavia_porcellus:0.080903,(Chinchilla_lanigera:0.039974,Octodon_degus:0.079129):0.007736):0.006451,(Fukomys_damarensis:0.04132,Heterocephalus_glaber:0.041098):0.014995):0.064729,((((((Cricetulus_griseus:0.031728,Mesocricetus_auratus:0.026375):0.029393,Microtus_ochrogaster:0.087886):0.002015,Peromyscus_maniculatus:0.035949):0.015685,(Meriones_unguiculatus:0.079037,(((Mus_caroli:0.016275,(Mus_musculus:0.010741,Mus_spretus:0.006333):0.012286):0.006717,Mus_pahari:0.032781):0.021204,Rattus_norvegicus:0.059732):0.037222):0.005904):0.05035,Nannospalax_galili:0.07108):0.044715,Jaculus_jaculus:0.117249):0.024764):0.003359):0.018506,Oryctolagus_cuniculus:0.101422):0.019468):0.010946,(((Condylura_cristata:0.108556,Erinaceus_europaeus:0.197888):0.006954,(((Eptesicus_fuscus:0.038464,((Myotis_brandtii:0.012435,Myotis_lucifugus:0.011049):0.004689,Myotis_davidii:0.056679):0.018136):0.040212,Miniopterus_natalensis:0.047915):0.077325,((Hipposideros_armiger:0.090049,Rhinolophus_sinicus:0.061433):0.045979,(Pteropus_alecto:0.015121,Pteropus_vampyrus:0.003531):0.206898):0.016479):0.057795):0.00358,((((((Balaenoptera_acutorostrata_scammoni:0.017999,(((Delphinapterus_leucas:0.007866,(Orcinus_orca:0,Tursiops_truncatus:0.002781):0.009084):0.004143,Lipotes_vexillifer:0.007652):0.01597,Physeter_catodon:0.012894):0.00684):0.017962,(((((Bison_bison_bison:0,(Bos_indicus:0,Bos_taurus:0):0.002894):0.001445,Bos_mutus:0):0.002926,Bubalus_bubalis:0.004344):0.010194,(Capra_hircus:0.005215,Ovis_aries:0.004981):0.006816):0.00703,Odocoileus_virginianus_texanus:0.014848):0.035788):0.00976,Sus_scrofa:0.059707):0.010678,((Camelus_bactrianus:0,(Camelus_dromedarius:0,Camelus_ferus:0.00138):0.004156):0.003078,Vicugna_pacos:0.017615):0.12023):0.021137,(Ceratotherium_simum_simum:0.028299,(Equus_asinus:0.005774,(Equus_caballus:0.003356,Equus_przewalskii:0):0.001968):0.067056):0.013238):0.003525,(((Ailuropoda_melanoleuca:0.017169,Ursus_maritimus:0.005504):0.042687,((Canis_familiaris:0.075109,((Leptonychotes_weddellii:0.005559,Neomonachus_schauinslandi:0.002777):0.003706,Odobenus_rosmarus_divergens:0.01884):0.015294):0.00621,(Enhydra_lutris_kenyoni:0.019293,Mustela_putorius:0.018096):0.064086):0.002197):0.031157,((Panthera_pardus:0.005561,Panthera_tigris_altaica:0.006683):0.009431,(Felis_catus:0.006507,Acinonyx_jubatus:0.004156):0.002903):0.108775):0.037397):0.001724):0.025115):0.011892):0.00213545,(Choloepus_hoffmanni:0.055554,Dasypus_novemcinctus:0.051919):0.0213545);	680	ENSG00000000457_SCYL3_000_NT.rootree
(((Orycteropus_afer_afer:0.089815,Trichechus_manatus_latirostris:0.03875):0.021557,(Dasypus_novemcinctus:0.14889,(((((((Aotus_nancymaae:0.004597,(Cebus_capucinus:0,Saimiri_boliviensis:0.01839):0.002212):0.03331,(((((((Cercocebus_atys:0,(((Macaca_fascicularis:0,Macaca_mulatta:0):0.002217,Macaca_nemestrina:0):0.002221,Papio_anubis:0.004528):0):0,Mandrillus_leucophaeus:0):0,Chlorocebus_sabaeus:0):0.002189,Piliocolobus_tephrosceles:0.006634):0,Colobus_angolensis:0):0,(Rhinopithecus_bieti:0,Rhinopithecus_roxellana:0.002229):0.002188):0.004702,(((Gorilla_gorilla:0,(Homo_sapiens:0,(Pan_paniscus:0,Pan_troglodytes:0):0.002278):0.002279):0.002278,Pongo_abelii:0):0.002228,Nomascus_leucogenys:0.006748):0.001917):0.004814):0.023509,Carlito_syrichta:0.057235):0.019555,((Microcebus_murinus:0.034309,Propithecus_coquereli:0.035466):0.047377,Otolemur_garnettii:0.077104):0.009598):0.01274,Galeopterus_variegatus:0.100254):0.015275,(((Castor_canadensis:0.128548,((((Cavia_porcellus:0.075886,Octodon_degus:0.081362):0.005682,Chinchilla_lanigera:0.063607):0.025669,(Fukomys_damarensis:0.041387,Heterocephalus_glaber:0.019158):0.032594):0.051103,(Ictidomys_tridecemlineatus:0.004374,Marmota_marmota_marmota:0.017512):0.10797):0.006071):0.02667,Oryctolagus_cuniculus:0.221455):0.0258,Tupaia_chinensis:0.144336):0.003917):0.010029,(Condylura_cristata:0.173875,((((Ceratotherium_simum_simum:0.046071,(Equus_asinus:0,(Equus_caballus:0,Equus_przewalskii:0):0.002327):0.041819):0.018288,Manis_javanica:0.113988):0.004228,(((Eptesicus_fuscus:0.017458,(Myotis_brandtii:0,Myotis_davidii:0.013201):0.013885):0.037852,Miniopterus_natalensis:0.025196):0.046299,((Hipposideros_armiger:0.039737,Rhinolophus_sinicus:0.067963):0.034588,((Pteropus_alecto:0.002589,Pteropus_vampyrus:0.006024):0,Rousettus_aegyptiacus:0.017688):0.071256):0.012811):0.012981):0.008791,(((((Balaenoptera_acutorostrata_scammoni:0.032001,(((Delphinapterus_leucas:0.002783,Orcinus_orca:0.002201):0.007049,Lipotes_vexillifer:0.012191):0.00904,Physeter_catodon:0.030815):0.000858):0.027598,(((Bos_indicus:0.004383,(Bos_mutus:0,Bos_taurus:0):0):0.036069,((Capra_hircus:0.010971,Ovis_aries:0.006583):0.004425,Pantholops_hodgsonii:0.004286):0.004565):0,Odocoileus_virginianus_texanus:0.036175):0.065364):0.008494,Sus_scrofa:0.075794):0.008401,(Camelus_bactrianus:0,(Camelus_dromedarius:0,Camelus_ferus:0):0.0022):0.074525):0.015491,(((Felis_catus:0.0163,Panthera_tigris_altaica:0.002282):0.002257,Panthera_pardus:0):0.059314,(Canis_familiaris:0.055206,((Enhydra_lutris_kenyoni:0.004883,Mustela_putorius:0.023719):0.009621,(((Leptonychotes_weddellii:0,Neomonachus_schauinslandi:0.002181):0.006694,Odobenus_rosmarus_divergens:0.015821):0.007398,(Ursus_maritimus:0.024842,Ailuropoda_melanoleuca:0.012446):0.024641):0.004225):0.014242):0.011218):0.052593):0.017849):0.005208):0.013185):0.02245):0.030645):0.00138409,(Chrysochloris_asiatica:0.119943,Echinops_telfairi:0.191665):0.0138409);	485	ENSG00000001497_LAS1L_000_NT.rootree
…

taxa.triplet

This parameter is where you specify the triplet of species of interest its topology. Either forms ["A", ["B", "C"]] or "(A, (B, C))" are accepted, provided A, B and C match names within the provided taxa.trees.

taxa.outgroup

Species listed in this parameter are used as the outgroup for the focal triplet. Either forms ["O", "P", "Q"] or whitespace-separated "O P Q" are accepted, provided they are distinct from taxa.triplet.

taxa.other

Species listed in this parameter are used as extra leaves in the tree to estimate branch length properties.

Global parameters

These parameters need be specified on top of the file as they don't belong to any particular TOML table.

gf_times

List the relative dates for possible gene flow events. At least one event must be specified, but no more than 3. Dates are relative to the A|B divergence time. They must take values between 0 and 1: 0 meaning present-day gene flow time, and 1 meaning that gene flow happened at A|B divergence time.

unresolved_mutations_count

If set, when the internal triplet branch-specific mutations count is inferior or equal to this value, consider that the topology is not resolved enough to exclude discordant scenarios. In this situation, every scenario contributes to the likelihood instead of only the ones with a concordant topology. Only values inferior to 0.5 are accepted. The value is given in mutations count units, so branch length × sequence length.

The [filters] table

These additional, optional configuration parameters control whether input trees are dismissed or kept for analysis.

(see the filtering process)

triplet_other_monophyly

When raised, filter out trees with at least one species from the 'other' section branching between LCA(outgroup) and the root, because it possibly should have been considered 'outgroup' instead.

(see the topology filter)

max_branch_count

If set, reject all trees with a larger triplet branch count. Use 0 to deactivate. Defaults to 200.

(see the geometry filter)

max_relative_mutation_rate

Reject all trees with relative mutation rate larger than this threshold. Defaults to 10.. Use +inf to deactivate.

(see the geometry filter)

max_clock_ratio

If set, this parameter rejects trees whose imbalance is too strong with respect to their enclosing forest. Values need to be greater than 1 because they quantify an "absolute" imbalance threshold: max(q/Q, Q/q).

(see the geometry filter)

The [init] table

This additional table can be set to specify starting point(s) for likelihood exploration. Missing parameters will be assigned a default value based on the data at hand according to aphid's internal heuristic, for which only theta value is necessary. Defaults to:

For example:

# Launch one optimisation process from exactly this point in parameters space.
[init]
theta = 5e-3
tau_1 = 2e-3
tau_2 = 5e-3
p_ab = 0.3
p_ac = 0.3
p_bc = 0.3
p_ancient_gf = 0.7

Parallel exploration

Every entry in the [init] table can receive several parameter values. One independent optimisation procedure will be executed in parallel from every such value, and explore different areas of the likelihood surface. For example:

# Launch three independent explorations of the parameters space.
# Aphid's heuristic will infer values for the missing parameters.
[init]
p_ac = [0.1, 0.2, 0.3]     # Three explorations each from a different starting point.
tau_2 = 2e-2               # Same starting value for the three explorations.
theta = [5e-4, 5e-3, 5e-2] # Necessary to initialize all other unspecified parameters.

Forcing parameters

Every parameter in the [init] table can either be set as:

  • A plain number e.g. 0.5: meaning that exploration will start at 0.5.
  • An annotation string without a number value (the value then defaults to aphid's internal heuristics). The string is either:
    • "learn" or "opt": meaning that this parameter will be learnt/optimized by aphid.
    • "fix", "force" or "pin": meaning that this parameter will be hard-set to this value and aphid will not attempt to explore it.
  • A value + annotation pair e.g. [0.5, "fix"].

For example:

# Launch three independent exploration runs,
# all being forced to `p_ac = 0`,
# one starting from `tau_2 = 5e-2`,
# the other starting from aphid's heuristic default value for `tau_2` given `theta = 5e-3`,
# the last one being forced to `tau_2 = 9e-2`.
[init]
theta = [5e-4, 5e-3, 5e-2]
tau_2 = [5e-2, "opt", [9e-2, "fix"]]
p_ac = [0, "fix"]

This optional table can be set for additional control over the likelihood exploration. (see the learning method)

Here are all default values. There is no need to write them in your config unless you need to tweak them. There is no need to tweak them unless it is clear from the learning method that you have some interest in doing so):

[search.bfgs]
record_trace = "detail" # (or "global" or "none")
max_iter = 1_000
step_size_threshold = 1e-9

[search.bfgs.slope_tracking]
sample_size = 20
threshold = 1e-3
grain = 5

[search.bfgs.wolfe]
c1 = 1e-4
c2 = 0.1
init_step_size= 1.0
step_decrease= 0.1
step_increase= 10
flat_gradient= 1e-20
bisection_threshold= 1e-1

The [output] table

This optional table can be set for additional control over the likelihood exploration.

[output]
# Lower either flag if output takes too much time or disk space.
detailed_json = true
detailed_csv = true