A variable-rate quantitative trait evolution model using penalized-likelihood

In recent years it has become increasingly popular to use phylogenetic comparative methods to investigate heterogeneity in the rate or process of quantitative trait evolution across the branches or clades of a phylogenetic tree. Here, I present a new method for modeling variability in the rate of evolution of a continuously-valued character trait on a reconstructed phylogeny. The underlying model of evolution is stochastic diffusion (Brownian motion), but in which the instantaneous diffusion rate (σ²)alsoevolves by Brownian motion on a logarithmic scale. Unfortunately, it’s not possible to simultaneously estimate the rates of evolution along each edge of the treeandthe rate of evolution ofσ²itself using Maximum Likelihood. As such, I propose a penalized-likelihood method in which the penalty term is equal to the log-transformed probability density of the rates under a Brownian model, multiplied by a ‘smoothing’ coefficient,λ, selected by the user.λdetermines the magnitude of penalty that’s applied to rate variation between edges. Lower values ofλpenalize rate variation relatively little; whereas largerλvalues result in minimal rate variation among edges of the tree in the fitted model, eventually converging on a single value ofσ²for all of the branches of the tree. In addition to presenting this model here, I have also implemented it as part of myphytoolsR package in the functionmultirateBM. Using different values of the penalty coefficient,λ, I fit the model to simulated data with: Brownian rate variation among edges (the model assumption); uncorrelated rate variation; rate changes that occur in discrete places on the tree; and no rate variation at all among the branches of the phylogeny. I then compare the estimated values ofσ²to their known true values. In addition, I use the method to analyze a simple empirical dataset of body mass evolution in mammals. Finally, I discuss the relationship between the method of this article and other models from the phylogenetic comparative methods and finance literature, as well as some applications and limitations of the approach.