Outputs

When run, for instance with:

$ aphid ./config.toml output

Aphid displays summarized information about the analysis on the console standard output. This output is only meant as an informative summary for humans. It is not supposed to be easily processed by downstream programs and its layout can arbitrarily change in the future.

In addition, aphid outputs very detailed structured information about the analysis under the form of a collection of files in a newly created folder named after its second argument (the output/ folder in the above example). Here is what this folder contains:

<OUTPUT>/ # The folder name as per the second argument given on the command line.
├── config.json # Complete information about the configuration used for this run.
├── global.json # The most important "forest-level" results.
├── detail.json # Detailed results per gene tree.
├── trees.csv   # Summarized results per gene tree in tabular form.
└── search/ # Traces of the heuristic searches for likelihood (one per starting point).
    ├── 1/
    │   ├── init.csv    # Starting point for this search.
    │   ├── status.json # Result of the search (error or best parameters found).
    │   ├── global.csv  # Every BFGS step.
    │   └── detail.csv  # Every linear search step.
    ├── 2/ …
    ├── 3/ …
    ⋮

These files are formally structured with .json or .csv format to ease their processing by downstream programs. The meaning of their content is detailed below:

• The global.json file
• The detail.json file
• The summarized trees.csv table
• The search/ traces
  • The init.json file.
  • The status.json summary.
  • The global.csv trace.
  • The detail.csv trace.

The global.json file

All user-facing information used or produced by aphid regarding the gene forest analyzed.

• n_trees: The number of trees analyzed.
• n_excluded_triplets_topologies: Number of triplets rejected based on their topology (incomplete or paraphyletic).
• n_unresolved_triplets: Number of triplets considered unresolved.
• n_excluded_outgroup_topologies: Number of outgroup rejected based on their topology (empty or paraphyletic).
• n_excluded_topologies: Total number of trees excluded based on their topology.
• mean_branch_length: Mean branch length over the forest.
• mean_length_triplet: Mean length of a triplet, calculated after the topology filter and before the geometry filter.
• mean_length_outgroup_other: Mean length of non-triplet branches ('other' and 'outgroup') calculated after the topology filter and before the geometry filter.
• imbalance: Average imbalance between triplet branches lengths and the outgroup + other sections branches lengths. Only calculated if a maximum clock ratio is set.
• triplet_longer: True if the imbalance means that triplets are longer on average. False if the outgroup + other sections are longer on average.
• shape: Overall ratio over the genes forest.
• n_excluded_branch_counts: Number of trees excluded based on branch count.
• n_excluded_mutation_rates: Number of trees excluded based on their estimated mutation rate.
• n_excluded_imbalance: Number of trees excluded based on their imbalance.
• n_included_trees: Final number of trees kept for likelihood calculation.
• estimate: Best parameters values found to maximize likelihood.
  • ln_likelihood: Best ln-likelihood value found.
  • parameters: Corresponding parameters values.

The detail.json file

All user-facing information used or produced by aphid regarding each particular gene tree.

• id: Tree identifier, as given in input.
• n_bases: Sequence length.
• n_nodes_raw: Number of nodes in the input tree.
• n_nodes_pruned: Number of nodes after only species of interest have been kept.
• triplet: Status of the focal triplet in this tree.
  • lca: LCA(triplet): the most recent ancestor of the focal triplet species found in this tree. Undefined if all triplet species were missing.
  • missing: Triplet species not found within this tree.
  • paraphyletic: Paraphyletic species found within this tree: these descend from LCA(triplet) but don't belong to the focal triplet.
  • analysis: Further information calculated iif the triplet is complete and monophyletic.
    • topology: Topology within this tree, assuming the reference topology was 'ABC' ~ '((A, B), C)'.
    • branches_counts: Mutations count estimate along the triplet branches [a, b, c, d]. The internal branch count d either represent:
      • ab in ((:a, :b):ab, :c) for topology ABC.
      • ac in (:b, (:a, :c):ac) for topology ACB.
      • bc in (:a, (:c, :b):bc) for topology BCA.
    • resolved: True if the topology is considered sufficiently resolved to exclude discordant scenarios from likelihood calculations.
  • included: Raise unless the tree should be excluded from analysis based on this triplet topology.
• outgroup: Status of the designated outgroup in this tree.
  • lca: LCA(outgroup): the ost recent ancestor of the designated outgroup species found in this tree. Undefined if all outgroup species were missing.
  • missing: Outgroup species not found within this tree.
  • paraphyletic: Paraphyletic species found within this tree: these descend from LCA(outgroup) but don't belong to the designated outgroup.
  • included: Raise unless the tree should be excluded from analysis based on this outgroup topology.
• top: Status of this tree's most ancestral nodes. Undefined if either no triplet species or no outgroup species were found.
  • lca: LCA(top): the most recent ancestor of LCA(triplet) and LCA(outgroup) found in this tree.
  • internal: Species descending from LCA(top), but neither from LCA(triplet) or LCA(outgroup). Only defined if there is no direct lineage between LCA(triplet) and LCA(outgroup).
    • triplet: Species branching between LCA(top) and LCA(triplet).
    • outgroup: Species branching between LCA(top) and LCA(outgroup).
  • external: Species not descending from LCA(top). If any, then LCA(top) is not the root of the tree.
  • included: Raise unless the tree should be excluded from analysis based on this tree top topology.
• topology_included: Raised if the tree passed the topology filter.
• mean_lengths: Mean branches lengths, undefined if the none of the species set they are supposed to be calculated over is found within the tree.
  • total: Calculated over all species of interest found in this tree.
  • triplet: Calculated over the focal triplet species found.
  • outgroup_other: Calculated over the designated outgroup species found plus the species designated as 'other'.
• local_shape: "Absolute" ratio of mean triplet length and mean length of designated 'outgroup' and 'other' species: always superior to 1 to measure 'imbalance', regardless which of the numerator or denominator is greater. (Latest estimate if the tree was excluded from analysis)
• branch_count_included: Raised if the tree passed this geometry filter.
• mutation_rate_included: Raised if the tree passed this statistical geometry filter.
• balance_included: Raised if the tree passed this statistical geometry filter.
• imbalance: Estimated imbalance for this tree. (Latest estimate if the tree was excluded from analysis)
• mutation_rate: Estimated mutation rate for this tree. (Latest estimate if the tree was excluded from analysis)
• ln_likelihood: Estimated likelihood of this single tree, provided it was included in the analysis. If no heuristic could start, this will instead contain initial likelihood value for every starting point, hopefully useful for understanding whether some trees were problematic.

The summarized trees.csv table

There is one line in this table per gene tree analyzed. Columns represent a redundant, flattened version of the structured information available in the above detail.json file, but we expect that it be easier to work with using table-processing downstream software.

The search/ traces

Since several starting point may be used in the likelihood maximization heuristics, aphid may produce several exploration traces stored within subfolders here.

The init.json file.

This file is a reminder which initial parameters have been used for the search.

The status.json summary.

This file summarizes the terminal status of the search: either detail about search failure if it failed, or the following information:

• parameters: The best parameters found during this search.
• scores: The unconstrained scores corresponding to these parameters.
• gradient: The corresponding gradient values for the scores at this point.
• ln_likelihood: The best likelihood value found.
• n_evaluations: The number of times the likelihood function has been evaluated to find this result.
• n_differentiations: The number of times the likelihood derivative has been evaluated to find this result.

The global.csv trace.

If produced, there is one line in this table per BFGS step taken during the search. (see BFGS)

The detail.csv trace.

If produced, there is one line in this table per linear search step taken for every BFGS step in search for a step size meeting strong Wolfe criteria. (see Linear Search)