module FBCModelTests

A collection of tests for constraint-based flux metabolic models.

module FROG

A collection of reproducibility checks for constraint-based metabolic models together with a report generator and tester.

See https://www.ebi.ac.uk/biomodels/curation/fbc for details.

mutable struct Dict{String, Any} <: AbstractDict{String, Any}

struct FROGObjectiveReport

Fields

• optimum::Union{Nothing, Float64}

• reactions::Dict{String, FBCModelTests.FROG.FROGReactionReport}

• gene_deletions::Dict{String, Union{Nothing, Float64}}

struct FROGReactionReport

Fields

• objective_flux::Union{Nothing, Float64}

• fraction_optimum::Float64

• variability_min::Union{Nothing, Float64}

• variability_max::Union{Nothing, Float64}

• deletion::Union{Nothing, Float64}

mutable struct Dict{String, FBCModelTests.FROG.FROGObjectiveReport} <: AbstractDict{String, FBCModelTests.FROG.FROGObjectiveReport}

compare_reports(
    report_dir_a::String,
    report_dir_b::String;
    kwargs...
) -> Any

A simple wrapper for comparing 2 previously generated reports in their respective directories. Additional arguments are fowarded to ReportTests.test_report_compatibility.

compare_reports_toplevel(
    report_dir_a::String,
    report_dir_b::String;
    kwargs...
) -> Test.DefaultTestSet

Like compare_reports, but with a specific testset settings that provide better user-facing reports.

generate_report(
    filename::String;
    report_dir,
    optimizer,
    modifications,
    workers,
    basefilename
)

A complete function for one-shot generation of FROG reports. Use ReportGenerators.generate_report_data, ReportGenerators.generate_metadata and ReportIO.save_report for finer control of the process.

module ReportIO

Functions for reading and writing FROG reports.

load_report(
    report_dir::String
) -> NamedTuple{(:metadata, :report), Tuple{Dict{String, Any}, Dict{String, FBCModelTests.FROG.FROGObjectiveReport}}}

Reverse of save_report.

save_report(
    r::Dict{String, FBCModelTests.FROG.FROGObjectiveReport},
    metadata::Dict{String, Any};
    report_dir,
    basefilename
)

Write the contents of FROGReportData to the 4 TSV files as specified by FROG standard, and additionally write the metadata into the JSON file.

module ReportGenerators

Functions for generating FROG report data and metadata file contents.

struct ResetObjective <: COBREXA.ModelWrapper

Fields

• model::COBREXA.MetabolicModel

• objective::SparseArrays.SparseVector{Float64, Int64}

frog_objective_report(sbml_model::COBREXA.SBMLModel)
frog_objective_report(
    sbml_model::COBREXA.SBMLModel,
    objective::Union{Nothing, String};
    optimizer,
    modifications,
    workers,
    fraction_optimum
)

Generate a FROGObjectiveReport containing the reproducibility data for a single objective in the SBML model.

generate_metadata(filename::String; optimizer, basefilename)

generate_report_data(
    model::COBREXA.SBMLModel;
    kwargs...
) -> Dict{Any, Any}

Generate FROGReportData for a model.

module ReportTests

Function for testing the compatibility of FROG report data.

test_metadata_compatibility(
    a::Dict{String, Any},
    b::Dict{String, Any}
)

test_report_compatibility(
    a::Dict{String, FBCModelTests.FROG.FROGObjectiveReport},
    b::Dict{String, FBCModelTests.FROG.FROGObjectiveReport};
    absolute_tolerance,
    relative_tolerance
)

module Memote

This package contains a collection of tests based on Memote. See Lieven, C., Beber, M.E., Olivier, B.G. et al. MEMOTE for standardized genome-scale metabolic model testing. Nat Biotechnol 38, 272–276 (2020). https://doi.org/10.1038/s41587-020-0446-y for details.

run_tests(
    model::COBREXA.MetabolicModel,
    optimizer;
    config,
    filename,
    workers
) -> Any

Run a MEMOTE-like test suite on model using optimizer to solve optimization problems; some basic parameters and thresholds are taken from config.

Some of the tests internally convert the input model to StandardModel of COBREXA; supplying a StandardModel may thus increase efficiency of the whole process.

run_tests(filename::String, optimizer; kwargs...) -> Any

Overload of run_tests that works directly with a file.

run_tests_toplevel(
    model::COBREXA.MetabolicModel,
    optimizer;
    config,
    filename,
    workers
) -> Test.DefaultTestSet

Run a MEMOTE-like test suite on model using optimizer to solve optimization problems. Forwards arguments to run_tests, but wraps all of the test sets in a FBCModelTests.QuietTestSet to generate more concise user facing results.

run_tests_toplevel(
    filename::String,
    optimizer;
    kwargs...
) -> Test.DefaultTestSet

Overload of run_tests_toplevel that works directly with a file.

module  Utils

Miscellaneous functions used by memote style tests, not typically user facing.

_compare_flux_bounds(
    fluxes,
    bound,
    tol,
    comparison_operator
) -> Dict{String, Tuple{String, Float64}}

Internal helper function to compare fluxes with specific bounds.

get_molar_mass(model, mid) -> Any

Return molar mass of mid. Return NaN if formula does not exist.

median_bounds(
    model::COBREXA.MetabolicModel
) -> Tuple{Any, Any}

A helper function that returns the median upper and lower bounds in a tuple. If none can be calculated, constants from COBREXA are used as the default values.

parse_annotations(annos) -> Dict{String, Vector{String}}

Ensure annotations are in a standardized format. Some models represent annotations like:

Dict{String, Vector{String}} with 2 entries:
  "sbo"          => ["SBO:0000176"]
  "RESOURCE_URI" => ["https://identifiers.org/ec-code/4.1.99.12", "https://identifiers.org/bigg.reaction/DB4PS", ...]

but the key used to map to URI annotations is not standarized. This helper function constructs a new annotation dictionary to ensure that all annotations looks like:

Dict{String, Vector{String}} with 7 entries:
  "bigg.reaction"     => ["DB4PS"]
  "pubmed"            => ["12595523"]
  "sbo"               => ["SBO:0000176"]
  "kegg.pathway"      => ["sce00740", "sce01110"]
  "metanetx.reaction" => ["MNXR97178"]
  "kegg.reaction"     => ["R07281"]
  "ec-code"           => ["4.1.99.12"]

to_element(x::String) -> PeriodicTable.Element

Return the chemical element of x.

module Config

Module housing the configuration parameters for the memote-style tests.

mutable struct AnnotationConfig

Fields

• gene_annotation_keywords::Vector{String}

• gene_annotation_regexes::Dict{String, Regex}

• metabolite_annotation_keywords::Vector{String}

• metabolite_annotation_regexes::Dict{String, Regex}

• reaction_annotation_keywords::Vector{String}

• reaction_annotation_regexes::Dict{String, Regex}

• minimum_conformal_crossreferences::Int64

• minimum_crossreferences::Int64

mutable struct BasicConfig

Fields

• minimum_metabolic_coverage::Float64

• minimum_growth_rate::Float64

• maximum_growth_rate::Float64

• optimizer_modifications::Vector{Function}

mutable struct BiomassConfig

Fields

• essential_precursors::Vector{String}

mutable struct ConsistencyConfig

Fields

• consistency_ignored_reactions::Vector{String}

• tolerance_threshold::Float64

mutable struct EnergyConfig

Fields

• energy_dissipating_metabolites::Vector{String}

• additional_energy_generating_reactions::Vector{COBREXA.Reaction}

• ignored_energy_reactions::Vector{String}

• optimizer_modifications::Vector{Function}

mutable struct MemoteConfig

A grouping of parameters used by the metabolic testing infrastructure.

Fields

• annotation::FBCModelTests.Memote.Config.AnnotationConfig

• basic::FBCModelTests.Memote.Config.BasicConfig

• biomass::FBCModelTests.Memote.Config.BiomassConfig

• consistency::FBCModelTests.Memote.Config.ConsistencyConfig

• energy::FBCModelTests.Memote.Config.EnergyConfig

• metabolite::FBCModelTests.Memote.Config.MetaboliteConfig

• network::FBCModelTests.Memote.Config.NetworkConfig

• reaction::FBCModelTests.Memote.Config.ReactionConfig

mutable struct MetaboliteConfig

Fields

• test_annotations::Vector{String}

mutable struct NetworkConfig

Fields

• condition_number::Float64

• cycle_tol::Float64

• blocked_tol::Float64

• optimizer_modifications::Vector{Function}

mutable struct ReactionConfig

Fields

• mass_ignored_reactions::Vector{String}

• charge_ignored_reactions::Vector{String}

module Basic

Basic metabolic model tests.

model_compartments(model::COBREXA.MetabolicModel) -> Set

Return the number of unique compartments in the model.

model_has_compartments(
    model::COBREXA.MetabolicModel
) -> Bool

Test if the model has one or more compartments.

model_has_genes(model::COBREXA.MetabolicModel) -> Any

Test if the model has any genes.

model_has_metabolites(model::COBREXA.MetabolicModel) -> Any

Test if the model has any metabolites.

model_has_reactions(model::COBREXA.MetabolicModel) -> Any

Test if the model has any reactions.

model_metabolic_coverage(
    model::COBREXA.MetabolicModel
) -> Any

Calculate the metabolic coverage by dividing the number of reactions by the number of genes.

model_solves_in_default_medium(
    model::COBREXA.MetabolicModel,
    optimizer;
    config
) -> Any

Check if the model can be solved under default conditions and yield a reasonable growth rate. Here reasonable is set via config.basic.minimum_growth_rate and config.basic.maximum_growth_rate. Optionally, pass optimization modifications to the solver through config.basic.optimizer_modifications.

module Consistency

This module checks if the metabolic model is overall consistent.

model_is_consistent(
    model::COBREXA.MetabolicModel,
    optimizer;
    config
) -> Bool

Determines if the model is stoichiometrically consistent. Note, stoichiometric consistency does not guarantee that mass balances must hold in the model. A more robust check is to ensure that each reaction is mass balanced, but this only works if all metabolites have masses assigned to them. Test based on Gevorgyan, Albert, Mark G. Poolman, and David A. Fell. "Detection of stoichiometric inconsistencies in biomolecular models." Bioinformatics (2008). Optionally ignore some reactions in this analysis by adding reaction IDs to config.consistency.consistency_ignored_reactions.

module Metabolites

A module testing various metabolite properties.

_find_orphan_or_deadend_metabolites(
    model::COBREXA.MetabolicModel;
    only_consumed
) -> Vector{String}

Helper function to find orphan or deadend metabolites. Specify only_consumed=true to consider orphan metabolites or false to consider deadend metabolites.

find_deadend_metabolites(
    model::COBREXA.MetabolicModel
) -> Vector{String}

Find all metabolites that can only (excludes reversible reactions) be produced in the model by inspecting the stoichiometric matrix and reaction bounds.

find_disconnected_metabolites(
    model::COBREXA.MetabolicModel
) -> Any

Returns a list of all metabolites that aren't part of any reactions.

find_orphan_metabolites(
    model::COBREXA.MetabolicModel
) -> Vector{String}

Find all metabolites that can only (excludes reversible reactions) be consumed in the model by inspecting the stoichiometric matrix and reaction bounds.

metabolites_are_duplicated(
    model::COBREXA.MetabolicModel,
    m1,
    m2,
    test_annotation
) -> Bool

Test if metabolites m1 and m2 are different by comparing their config.metabolite.test_annotation field in the annotations of each metabolite. Note, if no annotations are present for one or both of the metabolites, then return false.

metabolites_duplicated_in_compartment(
    model::COBREXA.MetabolicModel;
    config
) -> Set{String}

Return a dictionary of metabolites that are duplicated in their compartment. If any of the test annotations, stored in config.metabolite.test_annotations, are repeated, then the metabolite is counted as duplicated. Missing annotations are ignored.

module Reactions

A module testing reaction properties.

findall_duplicated_reactions(
    model::COBREXA.MetabolicModel
) -> Set{String}

Return a list of all reactions that are duplicated.

model_has_atpm_reaction(
    model::COBREXA.MetabolicModel
) -> Union{Missing, Bool}

Check if model has an ATP maintenance reaction built in (also called a non-growth associated maintenance cost). Looks for reaction annotations corresponding to the sbo maintenance term.

reaction_is_charge_balanced(
    model::COBREXA.MetabolicModel,
    rid::String
) -> Any

Check if the charges across a reaction balance.

reaction_is_mass_balanced(
    model::COBREXA.MetabolicModel,
    rid::String
) -> Bool

Check if the mass across a reaction balances.

module GPRAssociation

A module testing various facets of the gene reaction associations.

reaction_has_sensible_gpr(
    model::COBREXA.MetabolicModel,
    rid
) -> Any

Check if a reaction has a gene reaction rule, and that each gene in the rule is contained in the model.

reactions_with_complexes(
    model::COBREXA.MetabolicModel
) -> Any

Return a list of reaction ids that have protein complexes assigned to them.

module Biomass

This module contains tests that check the consistency of the biomass reaction.

atp_present_in_biomass_reaction(
    model::COBREXA.MetabolicModel,
    rid::String
) -> Bool

Check if the biomass reaction consumes ATP and H₂O, and produces ADP, HO₄P, and H⁺. Annotations are parsed, and the BiGG namespace is used to identify these metabolites in the underlying model.

biomass_missing_essential_precursors(
    model::COBREXA.MetabolicModel,
    rid::String;
    config
) -> Bool

Tests if the biomass reaction contains a set of essential precursors, listed in config.biomass.essential_precursors. Uses the BiGG namespace as a reference and the metabolite annotations.

biomass_reaction_is_consistent(
    model::COBREXA.MetabolicModel,
    rid::String
) -> Any

Check that the molar mass of a biomass reactions falls within [1 - 1e-3, 1 + 1e-6] by calling biomass_reaction_molar_mass internally.

biomass_reaction_molar_mass(
    model::COBREXA.MetabolicModel,
    rid::String
) -> Any

For a biomass reaction rid, calculate the molar weight of the reaction by summing the products of the associated metabolite coefficients with their molar masses (g/mol).

findall_biomass_reactions(
    model::COBREXA.MetabolicModel
) -> Any

Identify all the biomass reactions in the model using only sbo annotations.

module Network

A module testing the network and topology properties of the model.

find_all_universally_blocked_reactions(
    model::COBREXA.MetabolicModel,
    optimizer;
    config,
    workers
) -> Vector{String}

Make all boundary reactions reversible and run FVA on the model to find all reactions that are universally blocked. Optimizer modifications can be passed through config.network.optimizer_modifications

find_cycle_reactions(
    model::COBREXA.MetabolicModel,
    optimizer;
    config,
    workers
) -> Vector

Find all reactions that participate in stoichiometrically balanced cycles by closing all boundary reactions and running fva on the resultant model.

stoichiometric_max_min_ratio(
    model::COBREXA.MetabolicModel
) -> Any

Return the ratio of the absolute maximum and minimum value of the nonzero coefficients in the stoichiometric matrix of model.

module Annotation

This module contains tests used to check the coverage and conformance of reaction, gene, and metabolite annotations.

_find_annotated_components(
    model::COBREXA.MetabolicModel,
    id_accessor::String,
    annotation_accessor,
    annotation_kws
) -> Vector{String}

Helper function to find all the annotations for a component in a model.

_find_conformal_annotations(
    model::COBREXA.MetabolicModel,
    id_accessor::String,
    annotation_accessor,
    annotation_regex
) -> Vector{String}

Helper function to find all the annotations for a component that conform to recognized patterns in the model.

findall_annotated_gene_databases(
    model,
    gid::String;
    config
) -> Vector{String}

Checks if the databases listed in config.annotation.gene_annotation_keywords are present in the gene annotations. Returns a vector of annotation keywords that were found.

findall_annotated_metabolite_databases(
    model,
    mid::String;
    config
) -> Vector{String}

Checks if the databases listed in config.annotation.metabolite_annotation_keywords are present in the metabolite annotations. Returns a vector of annotation keywords that were found.

findall_annotated_reaction_databases(
    model,
    rid::String;
    config
) -> Vector{String}

Checks if the databases listed in config.annotation.reaction_annotation_keywords are present in the reaction annotations. Returns a vector of annotation keywords that were found.

findall_conformal_gene_annotations(
    model,
    gid::String;
    config
) -> Vector{String}

Check if the gene annotation entry conforms to commonly recognized formats of annotation database using regex patterns. Uses the database formats listed in config.annotation.gene_annotation_regexes to test the conformity. Returns a string vector of database ids that do conform.

findall_conformal_metabolite_annotations(
    model,
    mid::String;
    config
) -> Vector{String}

Check if the metabolite annotation entry conforms to commonly recognized formats of annotation database using regex patterns. Uses the database formats listed in config.annotation.metabolite_annotation_regexes to test the conformity. Returns a string vector of database ids that do conform.

findall_conformal_reaction_annotations(
    model,
    rid::String;
    config
) -> Vector{String}

Check if the reaction annotation entry conforms to commonly recognized formats of annotation database using regex patterns. Uses the database formats listed in config.annotation.reaction_annotation_regexes to test the conformity. Returns a string vector of database ids that do conform.

module Energy

This module checks if the model is energetically sensible.

model_has_no_erroneous_energy_generating_cycles(
    model::COBREXA.MetabolicModel,
    optimizer;
    config
) -> Bool

NOTE: this test is currently skipped and will appear as broken in the output.

Attempts to detect if the model contains any erroneous energy generating cycles by closing all exchange reactions and using flux balance analysis to maximize the sum of fluxes through a set of energy dissipating reactions. The flux sum should be zero if the model is free of energy generating reactions. This function is based on Fritzemeier, Claus Jonathan, et al. "Erroneous energy-generating cycles in published genome scale metabolic networks: Identification and removal." PLoS computational biology (2017). The energy dissipating reactions are based on the source paper, and include:

ATP + H2O --> ADP + H + Phosphate
CTP + H2O --> CDP + H + Phosphate
GTP + H2O --> GDP + H + Phosphate
UTP + H2O --> UDP + H + Phosphate
ITP + H2O --> IDP + H + Phosphate
NADH --> H + NAD
NADPH --> H + NADP
FADH2 --> 2 H + FAD
FMNH2 --> 2 H + FMN
Ubiquinol-8 --> 2 H + Ubiquinone-8
Menaquinol-8 --> 2 H + Menaquinone-8
2-Demethylmenaquinol-8 --> 2 H + 2-Demethylmenaquinone-8
H2O + ACCOA --> H + Acetate + COA
L-Glutamate + H2O --> 2-Oxoglutarate + Ammonium + 2 H
H[external] --> H

Additional energy dissipating reactions can be directly specified through config.energy.additional_energy_generating_reactions, which should be vector of COBREXA Reactions using the same metabolite name space as the model.If your model uses a different name space, then you have to change the values (NOT the keys) of config.energy.energy_dissipating_metabolites. Each energy dissipating reaction is added to the test only if all its associated metabolites are present. Any config.energy.optimizer_modifications to the solver are passed directly through to COBREXA's flux_balance_analysis function. All config.energy.boundary_reactions and config.energy.ignored_energy_reactions are deleted from an internal copy of model; this internal copy is used for analysis. Returns true if the model has no energy generating cycles.