Gene regulatory network inference by point-based Gaussian approximation filters incorporating the prior information

Abstract

Abstract

The extended Kalman filter (EKF) has been applied to inferring gene regulatory networks. However, it is well known that the EKF becomes less accurate when the system exhibits high nonlinearity. In addition, certain prior information about the gene regulatory network exists in practice, and no systematic approach has been developed to incorporate such prior information into the Kalman-type filter for inferring the structure of the gene regulatory network. In this paper, an inference framework based on point-based Gaussian approximation filters that can exploit the prior information is developed to solve the gene regulatory network inference problem. Different point-based Gaussian approximation filters, including the unscented Kalman filter (UKF), the third-degree cubature Kalman filter (CKF₃), and the fifth-degree cubature Kalman filter (CKF₅) are employed. Several types of network prior information, including the existing network structure information, sparsity assumption, and the range constraint of parameters, are considered, and the corresponding filters incorporating the prior information are developed. Experiments on a synthetic network of eight genes and the yeast protein synthesis network of five genes are carried out to demonstrate the performance of the proposed framework. The results show that the proposed methods provide more accurate inference results than existing methods, such as the EKF and the traditional UKF.

1 Introduction

Inferring gene regulatory network (GRN) has become one of the most important missions in system biology. Genome-wide expression data is widely used due to the development of several high-throughput experimental technologies. The gene regulatory network can be inferred from a number of gene expression samples taken over a period of time. Modeling of GRN is required before its structure can be inferred. Common dynamical modeling methods of GRN include Bayesian networks [1], Boolean networks [2], ordinary differential equations [3], state-space models [4, 5], and so on. Various approaches based on different models have been used to infer the network from observed gene expression data, such as the Markov Chain Monte Carlo (MCMC) methods for the dynamic Bayesian network model [6] and the ordinary differential equation model [7], as well as the Kalman filtering methods for the state-space model [4, 8] and the ordinary differential equation model [3]. Some survey papers can be found in [9–12].

Due to the ‘stochastic’ nature of the gene expression, the Kalman filtering approach based on the state-space model is one of the most competitive methods for inferring the GRN. The Kalman filter is optimal for linear Gaussian systems. However, the GRN is generally highly nonlinear. Hence, advanced filtering methods for nonlinear dynamic systems should be considered. The extended Kalman filter (EKF) is probably the most widely used nonlinear filter which uses the first-order Taylor series expansion to linearize the nonlinear model. However, the accuracy of the EKF is low when the system is highly nonlinear or contains large uncertainty. The point-based Gaussian approximation filters have been recently proposed to improve the performance of the EKF, which employ various quadrature rules to compute the integrals involved in the exact Bayesian estimation. Many filters fall into this category, such as the unscented Kalman filter (UKF) [13], the Gauss-Hermite quadrature filter [14], the cubature Kalman filter (CKF) [15], and the sparse-grid quadrature filter [16]. Besides the point-based Gaussian approximation filters, the particle filter has drawn much attention recently [17]. The particle filter uses random particles with weights to represent the probability density function (pdf) in the Bayesian estimation and provides better estimation result than the EKF. The main problem of the particle filter is that the computational complexity is high, and therefore, it is hard to use for high-dimensional problems, such as the problem considered in this paper.

The EKF and the particle filter have been used for the inference of GRN [4, 8, 18]. In this paper, we consider the point-based Gaussian approximation filters. Our main objective is to provide a framework of incorporating network prior information into the filters. For example, some gene regulations may be known [19] from literature and the inference accuracy of GRN can be improved by incorporating the known regulations of the GRN [20]. Integration of the prior knowledge or constraints with the GRN inference algorithm has been introduced to improve the inference result. The DNA motif sequence in gene promoter regions is incorporated in [21] while modeling of transcription factor interactions is incorporated in [22]. As mentioned in [20], experimentally determined physical interactions can be obtained. In addition, the sparsity constraint is frequently used in the inference of the GRN. To the best of the authors’ knowledge, the most related work in incorporating the prior information in Bayesian filters is [8]. In that work, rather than directly getting the inference results from the filter, an optimization method is used. In particular, a cost function is used in which the sparsity constraint is enforced. However, the cost function in [8] does not consider the uncertainty of the state in the filtering. That cost function in fact is not coupled well with the filtering algorithm. In addition, it did not consider other kinds of prior information. In this paper, we propose a new framework that incorporates the prior information effectively in the filtering algorithm by solving a constrained optimization problem. Efficient recursive algorithms are provided to solve the associated optimization problem.

The remainder of this paper is organized as follows. In Section 2, the modeling of gene regulatory network is introduced. The point-based Gaussian approximation filters are briefly introduced in Section 3. The proposed new filtering framework is described in Section 4. In Section 5, experimental results are provided. Finally, concluding remarks are given in Section 6.

2 State-space modeling of gene regulatory network

The GRN can be described by a graph in which genes are viewed as nodes and edges depict causal relations between genes. The structure of GRN reveals the mechanisms of biological cells. Analyzing the structure of GRN will pave the way for curing various diseases [23]. The learning of GRN has drawn much attention recently due to the availability the microarray data. By analyzing collected gene expression levels over a period of time, one can identify various regulatory relations between different genes. To facilitate the analysis of the GRN, modeling of GRN is required. Different models can be used, such as Bayesian networks [1], Boolean networks [2], ordinary differential equation [3], and state-space model [4, 5]. The state-space model has been widely used because it incorporates noise and can make use of computationally efficient filtering algorithms [5]. Thus, we also use the state-space modeling of GRN in this paper.

Under the discrete-time state-space modeling of the gene regulatory networks, the network evolution from time k to time k - 1 can be described by

$${\mathit{x}}_k=\mathit{f}\left({\mathit{x}}_{k-1}\right)+{\mathit{v}}_k,$$

(1)

where x _k = [ x_1,k,…,x_n,k]^T is the state vector and x_i,k denotes the gene expression level of the i-th gene at time k. f is a nonlinear function that characterizes the regulatory relationship among the genes. v _k  is the state noise and it is assumed to follow a Gaussian distribution with mean 0 and covariance matrix Q _k , i.e., ${\mathit{v}}_k\sim \mathcal{N}(0,{\mathit{Q}}_k)$.

Following [8], we use the following nonlinear function in the state Equation (1):

$$\mathit{f}\left(\mathit{x}\right)=\mathit{A}\mathit{g}\left(\mathit{x}\right),$$

(2)

with

$$\mathit{g}\left(\mathit{x}\right)=\left[\begin{array}{c}{g}_1\left(x_1\right)\\ ⋮\\ g_n\left(x_n\right)\end{array}\right]$$

(3)

and

$$g_i\left(x\right)=\frac{1}{1+e^{-{\mu }_{i}x}}.$$

(4)

In (2), A is the regulatory coefficient matrix with a _ij  denoting the regulation coefficient from gene j to gene i. Note that a positive coefficient a _ij  indicates that gene j activates gene i and a negative a _ij  indicates that gene j represses gene i. In (4), μ _i  is a parameter. Note that A and μ _i  are unknown parameters. The discrete-time nonlinear stochastic dynamic system [24] shown in Eqs. (1)-(3) have been successfully used to describe the GRN [4, 8]. Equation (4) is also called Sigmoid function which is frequently used since it is consistent with the fact that all concentrations get saturated at some point in time [25]. The Sigmoid function has been used in modeling GRN to verify various methods, such as artificial neural network [26], simulated annealing and clustering algorithm [27], extended Kalman filter [4], particle filter [8], and Genetic programming and Kalman filtering [25].

For the measurement model, we consider the following general nonlinear observation equation

$$\begin{array}{l}{\mathit{y}}_k=\mathit{h}\left({\mathit{x}}_k\right)+{\mathit{n}}_k,\end{array}$$

(5)

where h (·) is some nonlinear function, n _k  is the measurement noise, which is assumed to follow the Gaussian distribution with mean 0 and covariance matrix R _k , i.e., ${\mathit{n}}_k\sim \mathcal{N}(0,{\mathit{R}}_k)$. For example, if the noise corrupted expression levels are observed, then h(x) = x.

3 Network inference using point-based Gaussian approximation filters

3.1 Gaussian approximation filters

In this section, the framework of point-based Gaussian approximation filters for the state-space dynamic model is briefly reviewed. We consider the state-space model consisting of the state Equation (1) and the measurement Equation (5). We denote ${\mathit{y}}^k\triangleq [{\mathit{y}}_1,\dots ,{\mathit{y}}_k]$.

The optimal Bayesian filter is composed of two steps: prediction and filtering. Specifically, given the prior pdf $p\left({\mathit{x}}_{k-1}\right|{\mathit{y}}^{k-1})$ at time k - 1, the predicted conditional pdf $p\left({\mathit{x}}_k\right|{\mathit{y}}^{k-1})$ is given by

$$p\left({\mathit{x}}_k\right|{\mathit{y}}^{k-1})=\int p({\mathit{x}}_k\left|{\mathit{x}}_{k-1}\right)p\left({\mathit{x}}_{k-1}\right|{\mathit{y}}^{k-1})\mathrm{d}{\mathit{x}}_{k-1}.$$

(6)

After the measurement at time k becomes available, the filtered pdf is given by

$$p\left({\mathit{x}}_k\right|{\mathit{y}}^k)=\frac{p\left({\mathit{y}}_k\right|{\mathit{x}}_k\left)p\right({\mathit{x}}_k\left|{\mathit{y}}^{k-1}\right)}{\int p\left({\mathit{y}}_k\right|{\mathit{x}}_k\left)p\right({\mathit{x}}_k\left|{\mathit{y}}^{k-1}\right)\mathrm{d}{\mathit{x}}_k}.$$

(7)

The pdf recursions in (6) and (7) are in general computationally intractable unless the system is linear and Gaussian. The Gaussian approximation filters approximate (6) and (7) by invoking Gaussian assumptions. Specifically, the first assumption is that given y^k-1, x_k-1 has a Gaussian distribution, i.e., ${\mathit{x}}_{k-1}|{\mathit{y}}^{k-1}\sim \mathcal{N}({\widehat{\mathit{x}}}_{k-1|k-1},{\mathit{P}}_{k-1|k-1})$. The second assumption is that (x _k ,y _k ) are jointly Gaussian given y^k-1.

It then follows from the second assumption that given y^k-1, x _k  has a Gaussian distribution, i.e., ${\mathit{x}}_k|{\mathit{y}}^{k-1}\sim \mathcal{N}({\widehat{\mathit{x}}}_{k|k-1},{\mathit{P}}_{k|k-1}).$ Using (1) and the first assumption, we have the predicted mean ${\widehat{\mathit{x}}}_{k|k-1}$ and covariance P_k|k-1 given respectively by

$$\begin{array}{ll}{\widehat{\mathit{x}}}_{k|k-1}& \triangleq \mathbb{E}\left\{{\mathit{x}}_k\right|{\mathit{y}}^{k-1}\}={\mathbb{E}}_{{\mathit{x}}_{k-1}|{\mathit{y}}^{k-1}}\left\{\mathit{f}\left({\mathit{x}}_{k-1}\right)\right\}\\ =\int \mathit{f}\left(\mathit{x}\right)\varphi (\mathit{x};{\widehat{\mathit{x}}}_{k-1|k-1},{\mathit{P}}_{k-1|k-1})\mathrm{d}\mathit{x},\end{array}$$

(8)

and

$$\begin{array}{ll}{\mathit{P}}_{k|k-1}& \triangleq \text{Cov}\left\{{\mathit{x}}_k\right|{\mathit{y}}^{k-1}\}\\ ={\mathbb{E}}_{{\mathit{x}}_{k-1}|{\mathit{y}}^{k-1}}\left\{\left(\mathit{f}\right({\mathit{x}}_{k-1})-{\widehat{\mathit{x}}}_{k|k-1})\right.\\ \times \left.{\left(\mathit{f}\right({\mathit{x}}_{k-1})-{\widehat{\mathit{x}}}_{k|k-1})}^T\right\}+{\mathit{Q}}_{k-1}\\ =\int \left(\mathit{f}\right(\mathit{x})-{\widehat{\mathit{x}}}_{k|k-1})\\ \times {\left(\mathit{f}\right(\mathit{x})-{\widehat{\mathit{x}}}_{k|k-1})}^T\varphi \left(\mathit{x};{\widehat{\mathit{x}}}_{k-1|k-1},\right.\\ \left.{\mathit{P}}_{k-1|k-1}\right)\mathrm{d}\mathit{x}+{\mathit{Q}}_{k-1},\end{array}$$

(9)

where $\varphi \left(\mathit{x};\widehat{\mathit{x}},\mathit{P}\right)$ denotes the multivariate Gaussian pdf with mean $\widehat{\mathit{x}}$ and covariance P.

Then, following the second assumption, given ${\mathit{y}}^k=[{\mathit{y}}^{k-1},{\mathit{y}}_k]$, x _k  is Gaussian distributed, i.e., ${\mathit{x}}_k|{\mathit{y}}^k\sim \mathcal{N}({\widehat{\mathit{x}}}_{k|k},{\mathit{P}}_{k|k}).$ Using the conditional property of the multivariate Gaussian distribution, the filtered mean ${\widehat{\mathit{x}}}_{k|k}$ and covariance P_k|k are given respectively by

$$\begin{array}{ll}{\widehat{\mathit{x}}}_{k|k}& \triangleq \mathbb{E}\left\{{\mathit{x}}_k\right|{\mathit{y}}_k,{\mathit{y}}^{k-1}\}\\ ={\widehat{\mathit{x}}}_{k|k-1}+{\mathit{L}}_k({\mathit{y}}_k-{\widehat{\mathit{y}}}_{k|k-1})\end{array}$$

(10)

$$\begin{array}{ll}\text{and}{\mathit{P}}_{k|k}& \triangleq \text{Cov}\left\{{\mathit{x}}_k\right|{\mathit{y}}_k,{\mathit{y}}^{k-1}\}\\ ={\mathit{P}}_{k|k-1}-{\mathit{L}}_k{\mathit{P}}_k^{\mathit{\text{xy}}},\end{array}$$

(11)

with

$$\begin{array}{ll}{\widehat{\mathit{y}}}_{k|k-1}& ={\mathbb{E}}_{{\mathit{x}}_k|{\mathit{y}}^{k-1}}\left\{\mathit{h}\left({\mathit{x}}_k\right)\right\}\\ =\int \mathit{h}\left(\mathit{x}\right)\varphi \left(\mathit{x};{\widehat{\mathit{x}}}_{k|k-1},{\mathit{P}}_{k|k-1}\right)\mathrm{d}\mathit{x},\end{array}$$

(12)

$$\begin{array}{ll}{\mathit{L}}_k& ={\mathit{P}}_k^{\mathit{\text{xy}}}{({\mathit{R}}_k+{\mathit{P}}_k^{\mathit{\text{yy}}})}^{-1},\end{array}$$

(13)

$$\begin{array}{ll}{\mathit{P}}_k^{\mathit{\text{xy}}}& ={\mathbb{E}}_{{\mathit{x}}_k|{\mathit{y}}^{k-1}}\left\{(\mathit{x}-{\widehat{\mathit{x}}}_{k|k-1}){\left(\mathit{h}\right(\mathit{x})-{\widehat{\mathit{y}}}_{k|k-1})}^T\right\}\\ =\int (\mathit{x}-{\widehat{\mathit{x}}}_{k|k-1}){\left(\mathit{h}\right(\mathit{x})-{\widehat{\mathit{y}}}_{k|k-1})}^T\\ \varphi \left(\mathit{x};{\widehat{\mathit{x}}}_{k|k-1},{\mathit{P}}_{k|k-1}\right)\mathrm{d}\mathit{x},\end{array}$$

(14)

$$\begin{array}{ll}{\mathit{P}}_k^{\mathit{\text{yy}}}& ={\mathbb{E}}_{{\mathit{x}}_k|{\mathit{y}}^{k-1}}\left\{\left(\mathit{h}\right(\mathit{x})-{\widehat{\mathit{y}}}_{k|k-1}){\left(\mathit{h}\right(\mathit{x})-{\widehat{\mathit{y}}}_{k|k-1})}^T\right\}\\ =\int \left(\mathit{h}\left(\mathit{x}\right)-{\widehat{\mathit{y}}}_{k|k-1}\right){\left(\mathit{h}\left(\mathit{x}\right)-{\widehat{\mathit{y}}}_{k|k-1}\right)}^T\\ \varphi \left(\mathit{x};{\widehat{\mathit{x}}}_{k|k-1},{\mathit{P}}_{k|k-1}\right)\mathrm{d}\mathit{x}.\end{array}$$

(15)

3.2 Point-based Gaussian approximation filters

The integrals in (8), (9), (12), (14) and (15) are Gaussian type that can be efficiently approximated by various quadrature methods. Specifically, if a set of weighted points $\left\{({\mathit{\gamma }}_i,w_i),i=1,\dots ,N\right\}$ can be used to approximate the integral

$$\int \mathit{h}\left(\mathit{x}\right)\varphi \left(\mathit{x};0,\mathit{I}\right)\mathrm{d}\mathit{x}\approx \sum _{i=1}^N{w}_i\mathit{h}\left({\mathit{\gamma }}_i\right),$$

(16)

then the general Gaussian-type integral can be approximated by

$$\int \mathit{h}\left(\mathit{x}\right)\varphi \left(\mathit{x};\widehat{\mathit{x}},\mathit{P}\right)\mathrm{d}\mathit{x}\approx \sum _{i=1}^N{w}_i\mathit{h}(\mathit{S}{\mathit{\gamma }}_i+\widehat{\mathit{x}}),$$

(17)

where P= S S^T and S can be obtained by Cholesky decomposition or singular value decomposition (SVD).

Using (17), we can then approximate (8) and (9) as follows:

$${\widehat{\mathit{x}}}_{k|k-1}\approx \sum _{i=1}^N{w}_i\mathit{f}\left({\mathit{\xi }}_{k-1,i}\right)$$

(18)

and

$$\begin{array}{ll}{\mathit{P}}_{k|k-1}\approx & \sum _{i=1}^N{w}_i\mathit{f}\left({\mathit{\xi }}_{k-1,i}-{\widehat{\mathit{x}}}_{k|k-1}\right)\\ \times {\left({\mathit{\xi }}_{k-1,i}-{\widehat{\mathit{x}}}_{k|k-1}\right)}^T+{\mathit{Q}}_{k-1},\end{array}$$

(19)

where ξ_k - 1,i is the transformed quadrature point obtained from the covariance decomposition, i.e.,

$$\begin{array}{ll}{\mathit{P}}_{k-1|k-1}& ={\mathit{S}}_{k-1}{\mathit{S}}_{k-1}^T,\end{array}$$

(20)

$$\begin{array}{ll}{\mathit{\xi }}_{k-1,i}& ={\mathit{S}}_{k-1}{\mathit{\gamma }}_i+{\widehat{\mathit{x}}}_{k-1|k-1}.\end{array}$$

(21)

Similarly, we can approximate (12), (14) and (15) as follows:

$$\begin{array}{ll}{\widehat{\mathit{y}}}_{k|k-1}& =\sum _{i=1}^N{w}_i\mathit{h}\left({\tilde{\mathit{\xi }}}_{k,i}\right),\end{array}$$

(22)

$$\begin{array}{ll}{\mathit{P}}_k^{\mathit{\text{xy}}}& =\sum _{i=1}^N{w}_i\left({\tilde{\mathit{\xi }}}_{k,i}-{\widehat{\mathit{x}}}_{k|k-1}\right){\left(\mathit{h}\left({\tilde{\mathit{\xi }}}_{k,i}\right)-{\widehat{\mathit{y}}}_{k|k-1}\right)}^T,\end{array}$$

(23)

$$\begin{array}{ll}{\mathit{P}}_k^{\mathit{\text{yy}}}& =\sum _{i=1}^N{w}_i\left(\mathit{h}\left({\tilde{\mathit{\xi }}}_{k,i}\right)-{\widehat{\mathit{y}}}_{k|k-1}\right){\left(\mathit{h}\left({\tilde{\mathit{\xi }}}_{k,i}\right)-{\widehat{\mathit{y}}}_{k|k-1}\right)}^T,\end{array}$$

(24)

where ${\tilde{\mathit{\xi }}}_{k,i}$ is the transformed point obtained from the decomposition of the predicted covariance, i.e.,

$$\begin{array}{ll}{\mathit{P}}_{k|k-1}& ={\tilde{\mathit{S}}}_k{\tilde{\mathit{S}}}_k^T,\end{array}$$

(25)

$$\begin{array}{ll}{\tilde{\mathit{\xi }}}_{k,i}& ={\tilde{\mathit{S}}}_k{\mathit{\gamma }}_i+{\widehat{\mathit{x}}}_{k|k-1}.\end{array}$$

(26)

Various numerical rules can be used to form the approximation in (16), which lead to different Gaussian approximation filters. In particular, the unscented transformation, the Gauss-Hermite quadrature rule, and the sparse-grid quadrature rules are used in the unscented Kalman filter (UKF), the Gauss-Hermite quadrature Kalman filter (GHQF), and the sparse-grid quadrature filter (SGQF), respectively.

Recently, the fifth-degree quadrature filter has been proposed and shown to be more accurate than the third-degree quadrature filters, such as the UKF and the third-degree cubature Kalman filter (CKF₃), when the system is highly nonlinear or contains large uncertainty [16]. In this paper, we consider the UKF, CKF₃, and the fifth-degree cubature Kalman filter (CKF₅). Other filters such as the central difference filter [14] and divided difference filter [28] can also be used. The CKF₅ is based on Mysovskikh’s method which uses fewer point than the fifth-degree quadrature filter in [16]. In the following, different numerical rules used in (16) are briefly summarized.

3.2.1 Unscented transform

In the unscented Kalman filter (UKF), we have N = 2 n + 1 where n is the dimension of x. The quadrature points and the corresponding weights are given respectively by

$${\mathit{\gamma }}_i=\left\{\begin{array}{ll}0,& i=1,\\ \sqrt{(n+\kappa )}{\mathit{e}}_{i-1},& i=2,\cdots ,n+1,\\ -\sqrt{(n+\kappa )}{\mathit{e}}_{i-n-1},& i=n+2,\cdots ,2n+1,\end{array}\right.$$

(27)

and

$$w_i=\left\{\begin{array}{ll}\frac{\kappa }{n+\kappa },& i=1,\\ \frac{1}{2(n+\kappa )},& i=2,\cdots ,2n+1,\end{array}\right.$$

(28)

where κ is a tunable parameter, and e _i  is the i-th n-dimensional unit vector in which the i-th element is 1 and other elements are 0.

3.2.2 Cubature rules

The left-hand side of (16) can be rewritten as

$$\int \mathit{h}\left(\mathit{x}\right)\varphi \left(\mathit{x};0,\mathit{I}\right)\mathrm{d}\mathit{x}=\frac{1}{{\pi }^{n/2}}\int \mathit{h}\left(\sqrt{2}\mathit{x}\right)\text{exp}\left(-{\mathit{x}}^T\mathit{x}\right)\mathrm{d}\mathit{x}.$$

(29)

Consider the integral

$$I\left(\mathit{h}\right)=\int \mathit{h}\left(\mathit{x}\right)\text{exp}\left(-{\mathit{x}}^T\mathit{x}\right)\mathrm{d}\mathit{x}.$$

(30)

By letting x = r s with s^Ts = 1 and $r=\sqrt{{\mathit{x}}^T\mathit{x}}$, I(h) can be rewritten in the spherical-radial coordinate system as

$$I\left(\mathit{h}\right)=\underset{0}{\overset{\infty }{\int }}\underset{U_n}{\int }\mathit{h}\left(r\mathit{s}\right)r^{n-1}\text{exp}\left(-r^2\right)\mathrm{d}\sigma \left(\mathit{s}\right)\mathrm{d}r,$$

(31)

where $U_n=\left\{\mathit{s}\in {\mathit{R}}^n:\parallel \mathit{s}\parallel =1\right\}$, and $\sigma \left(·\right)$ is the spherical surface measure or the area element on U _n .

Note that (31) contains two types of integrals: the radial integral $\underset{0}{\overset{\infty }{\int }}{h}_r\left(r\right)r^{n-1}\text{exp}\left(-r^2\right)\mathrm{d}r$ and the spherical integral $\underset{U_n}{\int }{\mathit{h}}_s\left(\mathit{s}\right)\mathrm{d}\sigma \left(\mathit{s}\right)$.

If the radial rule can be approximated by

$$\underset{0}{\overset{\infty }{\int }}{h}_r\left(r\right)r^{n-1}\text{exp}\left(-r^2\right)\mathrm{d}r\approx \sum _{i=1}^{N_r}{w}_{r,i}{h}_r\left(r_i\right),$$

(32)

and the spherical integral can be approximated by

$$\underset{U_n}{\int }{\mathit{h}}_s\left(\mathit{s}\right)\mathrm{d}\sigma \left(\mathit{s}\right)\approx \sum _{j=1}^{N_s}{w}_{s,j}{\mathit{h}}_s\left({\mathit{s}}_j\right),$$

(33)

then (31) can be approximated by

$$\begin{array}{lll}I\left(\mathit{h}\right)& \approx & \sum _{i=1}^{N_r}\sum _{j=1}^{N_s}{w}_{r,i}{w}_{s,j}\mathit{h}\left(r_i{\mathit{s}}_j\right).\end{array}$$

(34)

A third-degree cubature rule to approximate (29) is obtained by using the third-degree spherical rule and radial rule [15]:

$$\int \mathit{h}\left(\mathit{x}\right)\varphi \left(\mathit{x};0,\mathit{I}\right)\mathrm{d}\mathit{x}\approx \frac{1}{2n}\sum _{i=1}^n\left[\mathit{h}\left(\sqrt{n}{\mathit{e}}_i\right)+\mathit{h}\left(-\sqrt{n}{\mathit{e}}_i\right)\right].$$

(35)

Remark: The third-degree cubature rule is identical to the unscented transformation with κ = 0.

To construct the fifth-degree cubature rule, the Mysovskikh’s method [29] and the moment matching method [16] are used to provide the fifth-degree spherical rule and radial rule, respectively. The final fifth-degree cubature rule is given by

$$\begin{array}{ll}\int \mathit{h}\left(\mathit{x}\right)\varphi \left(\mathit{x};0,\mathit{I}\right)\mathrm{d}\mathit{x}\approx & \frac{2}{n+2}\mathit{h}\left(0\right)+\\ +\frac{n^2(7-n)}{2{(n+1)}^2{(n+2)}^2}\sum _{i=1}^{n+1}\left[\mathit{h}\left(\sqrt{n+2}{\mathit{s}}_1^{\left(i\right)}\right)\right.\\ \left.+\mathit{h}\left(-\sqrt{n+2}{\mathit{s}}_1^{\left(i\right)}\right)\right]\\ +\frac{2{(n-1)}^2}{{(n+1)}^2{(n+2)}^2}\sum _{i=1}^{n(n+1)/2}\left[\mathit{h}\left(\sqrt{n+2}{\mathit{s}}_2^{\left(i\right)}\right)\right.\\ \left.+\mathit{h}\left(-\sqrt{n+2}{\mathit{s}}_2^{\left(i\right)}\right)\right],\end{array}$$

(36)

where the point ${\mathit{s}}_1^{\left(i\right)}$ is given by

$${\mathit{s}}_1^{\left(i\right)}=\left[p_1^{\left(i\right)},p_2^{\left(i\right)},\cdots ,p_n^{\left(i\right)}\right],i=1,2,\cdots ,n+1,$$

(37)

with

$$p_j^{\left(i\right)}=\left\{\begin{array}{ll}-\sqrt{\frac{n+1}{n(n-j+2)(n-j+1)}},& j<i,\\ \sqrt{\frac{(n+1)(n-i+1)}{n(n-i+2)}},& j=i,\\ 0,& j>\mathrm{i.}\end{array}\right.$$

(38)

Moreover, the set of points $\left\{{\mathit{s}}_2^{\left(i\right)}\right\}$ is given by

$$\begin{array}{ll}\left\{{\mathit{s}}_2^{\left(i\right)}\right\}=& \left\{\sqrt{\frac{n}{2(n-1)}}\left({\mathit{s}}_1^{\left(k\right)}+{\mathit{s}}_1^{\left(l\right)}\right):k<l,k,l=1,\right.\\ \left.2,\cdots ,n+1\right\}.\end{array}$$

(39)

3.3 Augmented state-space model for network inference

In the state-space model for gene regulatory networks described in Section 3.2, the underlying network structure is characterized by the n × n regulatory coefficient matrix A in (2) and the parameters μ = [ μ₁,…,μ _n ] in (4). The problem of network inference then becomes to estimate A and μ. To do that, we incorporate the unknown parameters A and μ into the state vector to obtain an augmented state-space model, and then apply the point-based Gaussian approximation filters to estimate the space vector and thereby obtaining the estimates of A and μ.

Specifically, we denote $\mathit{\theta }=\left[a_{11},a_{12},\cdots ,a_{1n},\cdots ,\right.{\left.a_{\mathit{\text{nn}}},{\mu }_1,\cdots ,{\mu }_n\right]}^T$ and the augmented state vector ${\bar{\mathit{x}}}_k={\left[{\mathit{x}}_k^T,{\mathit{\theta }}^T\right]}^T$. Then, the augmented state equation can be written as

$${\bar{\mathit{x}}}_k=\bar{\mathit{f}}\left({\bar{\mathit{x}}}_{k-1}\right)+{\bar{\mathit{v}}}_k=\left[\begin{array}{l}{\mathit{A}}_{k-1}{\mathit{g}}_{k-1}\left({\mathit{x}}_{k-1}\right)\\ {\mathit{\theta }}_{k-1}\end{array}\right]+\left[\begin{array}{l}{\mathit{v}}_{k-1}\\ 0\end{array}\right].$$

(40)

Note that A_k-1 and g_k-1 can be obtained from θ_k-1, and ${\bar{\mathit{v}}}_k\sim \mathcal{N}(0,{\bar{\mathit{Q}}}_k)$ with ${\bar{\mathit{Q}}}_k=\text{diag}\left(\left[{\mathit{Q}}_k{\mathit{O}}_{n^2+n}\right]\right)$, where O _m  denotes an m × m all-zero matrix.

In the remainder of this paper, we assume that the noisy gene expression levels are observed. Therefore, the augmented measurement equation becomes

$${\mathit{y}}_k=\mathit{h}\left({\bar{\mathit{x}}}_k\right)+{\mathit{n}}_k=\mathit{B}{\bar{\mathit{x}}}_k+{\mathit{n}}_k,$$

(41)

where $\mathit{B}=[{\mathit{I}}_n,{\mathit{O}}_{n\times (n^2+n)}]$, ${\mathit{O}}_{n\times (n^2+n)}$ denotes an n × (n² + n) all zeros matrix.

The point-based Gaussian approximation filters can then be used to obtain the estimate of the augmented state, ${\widehat{\bar{\mathit{x}}}}_k$, from which the estimates of the unknown network parameters, i.e., $\widehat{\mathit{A}}$ and $\widehat{\mathit{\mu }}$ can then be obtained.

Note that since the measurement Equation (41) is linear, the filtering Equations (10, 11) become

$$\begin{array}{ll}{\widehat{\bar{\mathit{x}}}}_{k|k}& ={\widehat{\bar{\mathit{x}}}}_{k|k-1}+{\mathit{L}}_k({\mathit{y}}_k-\mathit{B}{\widehat{\bar{\mathit{x}}}}_{k|k-1}),\end{array}$$

(42)

$$\begin{array}{ll}\text{and}{\mathit{P}}_{k|k}& ={\mathit{P}}_{k|k-1}-{\mathit{L}}_k\mathit{B}{\mathit{P}}_{k|k-1},\end{array}$$

(43)

$$\begin{array}{ll}\text{with}{\mathit{L}}_k& ={\mathit{P}}_{k|k-1}{\mathit{B}}^T{({\mathit{R}}_k+\mathit{B}{\mathit{P}}_{k|k-1}{\mathit{B}}^T)}^{-1},\end{array}$$

(44)

which are the same as the filtering updates for Kalman filters.

4 Incorporating prior information

In practice, some prior knowledge on the underlying GRN is typically available. In this section, we outline approaches to incorporating such prior knowledge into the point-based Gaussian approximation filters for network inference. In particular, we consider two types of prior information, namely, sparsity constraints and range constraints on the network. For networks with sparsity constraints, we incorporate an iterative thresholding procedure into the Gaussian approximation filters. And to accommodate range constraints, we employ PDF-truncated Gaussian approximation filters.

4.1 Optimization-based approach for sparsity constraints

4.1.1 The optimization formulations

Note that under the Gaussian assumption, the state estimation ${\widehat{\bar{\mathit{x}}}}_{k|k}$ of the Kalman filter is equivalently given by the solution to the following optimization problem [30, 31]

$$\begin{array}{ll}{\widehat{\bar{\mathit{x}}}}_{k|k}& =\underset{\bar{\mathit{x}}}{\text{arg}\text{min}}J\left(\bar{\mathit{x}}\right),\end{array}$$

(45)

$$\begin{array}{ll}\text{with}J\left(\bar{\mathit{x}}\right)& \triangleq {\left({\mathit{y}}_k-\mathit{h}\left(\bar{\mathit{x}}\right)\right)}^T{\mathit{R}}_k^{-1}\left({\mathit{y}}_k-\mathit{h}\left(\bar{\mathit{x}}\right)\right)\\ +{\left(\bar{\mathit{x}}-{\widehat{\bar{\mathit{x}}}}_{k|k-1}\right)}^T{\mathit{P}}_{k|k-1}^{-1}\left(\bar{\mathit{x}}-{\widehat{\bar{\mathit{x}}}}_{k|k-1}\right).\end{array}$$

(46)

To incorporate the prior information of the GRN, (46) is modified as

$$\tilde{J}\left(\bar{\mathit{x}}\right)=J\left(\bar{\mathit{x}}\right)+\lambda J_p\left(\bar{\mathit{x}}\right),$$

(47)

where $J_p\left(\bar{\mathit{x}}\right)$ is a penalty function associated with the prior information and λ is a tunable parameter that regulates the tightness of the penalty.

For example, in gene regulatory networks, each gene only interacts with a few genes [20]. To capture such a sparsity constraint, a Laplace prior distribution can be used for the connection coefficient matrix A, i.e.,

$$p\left(\mathit{A}\right)={(\lambda /2)}^{n^2}\text{exp}\left(-\lambda \sum _{i=1}^n\sum _{j=1}^n\left|a_{\mathit{\text{ij}}}\right|\right).$$

(48)

Therefore, in this case, $J_p\left(\bar{\mathit{x}}\right)=-\text{log}p\left(\mathit{A}\right)=c_1\parallel \mathit{A}{\parallel }_1+c_2$ where c₁ and c₂ are constants. And, (47) can be rewritten as

$$\tilde{J}\left(\bar{\mathit{x}}\right)=J\left(\bar{\mathit{x}}\right)+\lambda \parallel \mathit{A}{\parallel }_1.$$

(49)

Note that (49) can also be interpreted as the result of applying the least squares shrinkage selection operator (LASSO) to (47). The LASSO adds an L₁ - norm constraint to the GRN so that the regulatory coefficient matrix A tends to be sparse with many zero elements.

As another example, if some known regulatory relationship exists, then it should be taken into account to improve the estimation accuracy. Specifically, define an n × n indicator matrix E = [ e_i,j] where e _ij  = 1 indicates that there is a lack of regulation from gene j to gene i. Then, similar to the use of LASSO, a penalty on a _ij  should incur if e _ij  = 1. Thus, (47) can be rewritten as

$$\tilde{J}\left(\bar{\mathit{x}}\right)=J\left(\bar{\mathit{x}}\right)+\lambda \parallel \mathit{E}\circ \mathit{A}{\parallel }_1.$$

(50)

Note that as in [20], here we do not force a _ij  = 0 corresponding to e _ij  = 1 but rather use an L₁ - norm penalty. The advantage of such an approach is that it allows the algorithm to pick different structures but more likely to pick the edges without penalties. ‘o’ denotes the entry-wise product operation of two matrices.

4.1.2 Iterative thresholding algorithm

Solving the optimization problems in (49) and (50) is not straightforward since |a| is non-differentiable at a = 0. In the following, an efficient solver called the iterative thresholding algorithm is introduced.

For convenience, we consider a general optimization problem of the form

$$\underset{\bar{\mathit{x}}}{\text{arg}\text{min}}J\left(\bar{\mathit{x}}\right)=L\left(\bar{\mathit{x}}\right)+\parallel \mathit{\text{λ}}\circ \bar{\mathit{x}}{\parallel }_1,$$

(51)

where $\mathit{\text{λ}}={[{\lambda }_1,{\lambda }_2,\cdots ,{\lambda }_{n^2+2n}]}^T$ and $L\left(\bar{\mathit{x}}\right)$ is a smooth function. Note that if $\mathit{\text{λ}}={[0_{1\times n},\lambda \times 1_{1\times n^2},0_{1\times n}]}^T$, then (51) becomes (49); and if $\mathit{\text{λ}}={[0_{1\times n},\lambda \times \widehat{\underline{\mathit{\theta }}},0_{1\times n}]}^T$, then (51) becomes (50). Note that $\widehat{\underline{\mathit{\theta }}}={\left[e_{11},e_{12},\cdots ,e_{1n},\cdots ,e_{\mathit{\text{nn}}}\right]}^T$.

The solution to (51) can be iteratively obtained by solving a sequence of optimization problems. As in Newton’s method, the Taylor series expansion of $L\left(\bar{\mathit{x}}\right)$ around the solution ${\bar{\mathit{x}}}^t$ at the t-th iteration is given by

$$L({\bar{\mathit{x}}}^t+\Delta \bar{\mathit{x}})\cong L\left({\bar{\mathit{x}}}^t\right)+\Delta {\bar{\mathit{x}}}^T\nabla L\left({\bar{\mathit{x}}}^t\right)+\frac{{\alpha }_t}{2}\parallel \Delta \bar{\mathit{x}}{\parallel }_2^2,$$

(52)

where ∇L is the gradient of L and α _t  is such that α _t I mimics the Hessian ∇²L. Then, ${\bar{\mathit{x}}}^{t+1}$ is given by [32]

$${\bar{\mathit{x}}}^{t+1}=\underset{\mathit{z}}{\text{arg}\text{min}}{(\mathit{z}-{\bar{\mathit{x}}}^t)}^T\nabla L\left({\bar{\mathit{x}}}^t\right)+\frac{{\alpha }_t}{2}\parallel \mathit{z}-{\bar{\mathit{x}}}^t{\parallel }_2^2+\parallel \mathit{\text{λ}}\circ \mathit{z}{\parallel }_1.$$

(53)

The equivalent form of (53) is given by [32]

$$\begin{array}{ll}{\bar{\mathit{x}}}^{t+1}& =\underset{\mathit{z}}{\text{arg}\text{min}}\frac{1}{2}\parallel \mathit{z}-{\mathit{u}}^t{\parallel }_2^2+\frac{1}{{\alpha }_t}\parallel \mathit{\text{λ}}\circ \mathit{z}{\parallel }_1,\end{array}$$

(54)

$$\begin{array}{ll}\text{with}{\mathit{u}}^t& ={\bar{\mathit{x}}}^t-\frac{1}{{\alpha }_t}\nabla L\left({\bar{\mathit{x}}}^t\right),\end{array}$$

(55)

$$\begin{array}{lll}{\alpha }_t& \approx & \frac{{\left({\mathit{s}}^t\right)}^T{\mathit{r}}^t}{\parallel {\mathit{s}}^t{\parallel }^2},\end{array}$$

(56)

$$\begin{array}{ll}{\mathit{s}}^t& ={\bar{\mathit{x}}}^t-{\bar{\mathit{x}}}^{t-1},\end{array}$$

(57)

$$\begin{array}{ll}{\mathit{r}}^t& =\nabla L\left({\bar{\mathit{x}}}^t\right)-\nabla L\left({\bar{\mathit{x}}}^{t-1}\right).\end{array}$$

(58)

The solution to (54) is given by [32]${\bar{\mathit{x}}}^{t+1}={\eta }^S({\mathit{u}}^t,\frac{\mathit{\text{λ}}}{{\alpha }_t})$, where

$${\eta }^S(\mathit{u},\mathit{a})=\text{sign}\left(\mathit{u}\right)\text{max}\left\{\left|\mathit{u}\right|-\mathit{a},0\right\}$$

(59)

is the soft thresholding function with sign(u) and $\text{max}\left\{\left|\mathit{u}\right|-\mathit{a},0\right\}$ being component-wise operators.

Finally, the iterative procedure for solving (51) is given by

$${\bar{\mathit{x}}}^{t+1}=\text{sign}\left({\bar{\mathit{x}}}^t-\frac{1}{{\alpha }_t}\nabla L\left({\bar{\mathit{x}}}^t\right)\right)\text{max}\left\{\left|{\bar{\mathit{x}}}^t-\frac{1}{{\alpha }_t}\nabla L\left({\bar{\mathit{x}}}^t\right)\right|-\frac{\mathit{\text{λ}}}{{\alpha }_t},0\right\}.$$

(60)

And the iteration stops when the following condition is met

$$\frac{\left|J\right({\bar{\mathit{x}}}^t)-J({\bar{\mathit{x}}}^{t-1}\left)\right|}{\left|J\right({\bar{\mathit{x}}}^{t-1}\left)\right|}\le \epsilon ,$$

(61)

where ε is a given small number.

4.2 PDF truncation method for range constraints

If the range constraints on the regulatory coefficients are available, the inference accuracy can be improved by enforcing the constraints in the Gaussian approximation filters.

In particular, assume that we impose the following range constraints on the state vector $\bar{\mathit{x}}$

$$\mathit{c}\le \bar{\mathit{x}}\le \mathit{d}.$$

(62)

The PDF truncation method [31] can be employed to incorporate the above range constraint into the Gaussian approximation filters, by converting the updated mean ${\widehat{\bar{\mathit{x}}}}_{k|k}$ and covariance P_k|k to the pseudo mean ${\widehat{\bar{\mathit{x}}}}_{k|k}^t$ and covariance ${\mathit{P}}_{k|k}^t$ which are then used in the next prediction and filtering steps.

We next briefly outline the PDF truncation procedure. We use ${\widehat{\bar{\mathit{x}}}}_{k|k,i}^t$ and ${\mathit{P}}_{k|k,i}^t$ to denote the mean and covariance after the first i constraints have been enforced. Initially, we set ${\widehat{\bar{\mathit{x}}}}_{k|k,0}^t={\widehat{\bar{\mathit{x}}}}_{k|k}$ and ${\mathit{P}}_{k|k,0}^t={\mathit{P}}_{k|k}$. Consider the following transformation

$${\mathit{z}}_{k,i}={\mathcal{G}}_i{\mathcal{D}}_i^{-1/2}{\mathcal{S}}_i^T({\bar{\mathit{x}}}_k-{\widehat{\bar{\mathit{x}}}}_{k|k,i}^t)$$

(63)

where ${\mathcal{S}}_i$ and ${\mathcal{D}}_i$ are obtained from the Jordan canonical decomposition ${\mathcal{S}}_i{\mathcal{D}}_i{\mathcal{S}}_i^T={\mathit{P}}_{k|k,i}^t$ and ${\mathcal{G}}_i$ is obtained by using the Gram-Schmidt orthogonalization and it satisfies [33]

$${\mathcal{G}}_i{\mathcal{D}}_i^{1/2}{\mathcal{S}}_i^T{\mathit{e}}_i=\left[{\left({\mathit{e}}_i^T{\mathit{P}}_{k|k,i}^t{\mathit{e}}_i\right)}^{1/2},0,\cdots ,0\right].$$

(64)

Then, the upper bound ${\mathit{e}}_i^T\bar{\mathit{x}}\le d_i$ is transformed to [33]

$$[1,0,\cdots ,0]{\mathit{z}}_{k,i}\le \frac{d_i-{\mathit{e}}_i^T{\widehat{\bar{\mathit{x}}}}_{k|k,i}^t}{{\left({\mathit{e}}_i^T{\mathit{P}}_{k|k,i}^t{\mathit{e}}_i\right)}^{1/2}}\triangleq {\tilde{d}}_i.$$

(65)

Similarly, the lower bound ${\mathit{e}}_i^T\bar{\mathit{x}}\ge c_i$ is transformed to

$$[1,0,\cdots ,0]{\mathit{z}}_{k,i}\ge \frac{c_i-{\mathit{e}}_i^T{\widehat{\bar{\mathit{x}}}}_{k|k,i}^t}{{\left({\mathit{e}}_i^T{\mathit{P}}_{k|k,i}^t{\mathit{e}}_i\right)}^{1/2}}\triangleq {\tilde{c}}_i.$$

(66)

The constraint requires that the first element of z_k,i lies between ${\tilde{c}}_i$ and ${\tilde{d}}_i$. Hence, only the truncated PDF of the first element of z_k,i is considered and it is given by [33]

$$\begin{array}{ll}f\left(z\right)& ={\alpha }_i\text{exp}(-z^2/2),\end{array}$$

(67)

$$\begin{array}{ll}\text{with}{\alpha }_i& =\frac{\sqrt{2}}{\sqrt{\pi }\left[\text{erf}\right({\tilde{d}}_i/\sqrt{2})-\text{erf}({\tilde{c}}_i/\sqrt{2}\left)\right]}.\end{array}$$

(68)

Then, the mean and variance of the first element of z_k,i after imposing the i-th constraint are given respectively by

$$\begin{array}{ll}{\mu }_i=& \underset{{\tilde{c}}_i}{\overset{{\tilde{d}}_i}{\int }}\mathit{\text{zf}}\left(z\right)\mathrm{d}z={\alpha }_i\left[\text{exp}(-{\tilde{c}}_i^2/2)-\text{exp}(-{\tilde{d}}_i^2/2)\right],\end{array}$$

(69)

$$\begin{array}{ll}{\sigma }_i^2=& \underset{{\tilde{c}}_i}{\overset{{\tilde{d}}_i}{\int }}{(z-{\mu }_i)}^{2}f\left(z\right)\mathrm{d}z\\ =& {\alpha }_i\left[\text{exp}(-{\tilde{c}}_i^2/2)({\tilde{c}}_i-2{\mu }_i)\right.\\ \left.-\text{exp}(-{\tilde{d}}_i^2/2)({\tilde{d}}_i-2{\mu }_i)\right]+{\mu }_i^2+1.\end{array}$$

(70)

Thus, the mean and covariance of the transformed state vector after imposing the i-th constraint are given respectively by

$$\begin{array}{ll}{\bar{\mathit{z}}}_{k,i}& ={\left[{\mu }_i,0,\cdots ,0\right]}^T,\end{array}$$

(71)

$$\begin{array}{ll}{\mathit{Q}}_{k,i}& =\text{diag}\left(\right[{\sigma }_i^2,1,\cdots ,1\left]\right).\end{array}$$

(72)

By taking the inverse transform of (63), we then get

$$\begin{array}{ll}{\widehat{\bar{\mathit{x}}}}_{k|k,i+1}^t& ={\mathcal{S}}_i{\mathcal{D}}_i^{1/2}{\mathcal{G}}_i^T{\bar{\mathit{z}}}_{k,i}+{\widehat{\bar{\mathit{x}}}}_{k|k,i}^t,\end{array}$$

(73)

$$\begin{array}{ll}{\mathit{P}}_{k|k,i+1}^t& ={\mathcal{S}}_i{\mathcal{D}}_i^{1/2}{\mathcal{G}}_i^T{\mathit{Q}}_{k,i}{\mathcal{G}}_i{\mathcal{D}}_i^{1/2}{\mathcal{S}}_i^T.\end{array}$$

(74)

After imposing all n constraints, the final constrained state estimate and covariance at time k are given respectively by ${\widehat{\bar{\mathit{x}}}}_{k|k}^t\triangleq {\widehat{\bar{\mathit{x}}}}_{k|k,n}^t$ and ${\mathit{P}}_{k|k}^t\triangleq {\mathit{P}}_{k|k,n}^t$.

5 Numerical results

5.1 Synthetic network

In this section, a synthetic network that contains eight genes is used to test the performance of the EKF, the UKF, the CKF₃, the CKF₅, and their corresponding filters incorporating the prior information. Forty data points are collected to infer the structure of the network. The system noise and measurement noise are assumed to be Gaussian distributed with means 0 and covariances ${\bar{\mathit{Q}}}_k=\text{diag}\left(\right.\left[0.01{\mathit{I}}_8{\mathit{O}}_{72}\right]\left)\right.$ and R _k  = 0.01 I₈, respectively. The connection coefficient matrix is given by

$$\begin{array}{l}\mathit{A}=\left(\begin{array}{llllllll}0& 0& 0& 0& 0& 0& 2.4& 3.2\\ 0& 0& 0& 4.1& 0& -2.4& 0& 4.1\\ -5.0& 2.1& -1.5& 0& 4.5& 0& 2.1& 0\\ 0& 1.3& 2.5& -3.7& 1.8& 0& 0& -3.1\\ 0& 0& 0& -2.6& -3.2& 0& -1& 4\\ -1.5& -1.8& 0& 3.4& 1.4& 1.1& 0& 1.7\\ -1.8& 0& 0& -3& 1.1& 2.4& 0& 0\\ -1.3& 0& -1& 0& 2.1& 0& 0& 2.2\end{array}\right)\end{array}$$

(75)

and μ _i  = 2, i = 1,⋯,8. For the filter, each coefficient in $\widehat{\mathit{A}}$ is initialized from a Gaussian distribution with mean 0 and variance 0.2. Moreover, the coefficient μ _i  is initialized from a Gaussian distribution with mean 1.5 and variance 0.2. The system state is initialized using the first measurement.

The metric used to evaluate the inferred GRN is the true positive rate (TPR), the false positive rate (FPR), and the positive predictive value (PPV). They are given by [34]

$$\text{TPR}=\frac{\text{TP}\#}{\text{TP}\#+\text{FN}\#},$$

(76)

$$\text{FPR}=\frac{\text{FP}\#}{\text{FP}\#+\text{TN}\#},$$

(77)

$$\text{PPV}=\frac{\text{TP}\#}{\text{TP}\#+\text{FP}\#},$$

(78)

where the number of true positives (TP #) denotes the number of links correctly predicted by the inference algorithm; the number of false positives (FP #) denotes the number of incorrectly predicted links; the number of true negatives (TN #) denotes the number of correctly predicted nonlinks; and the number of false negatives (FN #) denotes the number of missed links by the inference algorithm [8].

5.1.1 Comparison of the EKF with point-based Gaussian approximation filters

The UKF with different parameter κ is tested. The simulation results based on 50 Monte Carlo runs are shown in Table 1. It can be seen that UKFs with κ=0,2,5 have slight better performance than UKFs with κ = -5,-2. One possible reason is that the weights of all sigma points used in the UKF are all positive when κ ≥ 0. In general, all positive weights will guarantee better stability of the filtering algorithm. However, it should be emphasized that, in this specific example, there is no big difference between UKFs with different κ. In addition, the objective of this paper was to investigate the proposed filter incorporating the prior information. Hence, the UKF is used to denote UKF with κ = 3 - n and compare with the filters incorporating the prior information.

Table 1 Comparison of UKF with different κ

The inference results of the EKF, the UKF, the CKF₃, and the CKF₅ are summarized in Table 2, all results are based on 50 Monte Carlo runs. It can be seen that all point-based Gaussian approximation filters have better performance than the EKF since the average(avg) FPR is lower and the average TPR and precision are higher than that of the EKF. Although the CKFs exhibit slightly better filtering performance than the UKF in some runs, they are comparable in terms of TPR, FPR, and PPV.

Table 2 Comparison of different filters

Based on the above tests, in the rest of the paper, only the UKF is used.

5.1.2 Comparison of the UKF and the UKF incorporating the prior information

As mentioned above, the UKF is used as a typical filter to compare the performance with and without the prior information.

Incorporating existing network information The following prior existing network information is assumed to be known: 1) gene1, gene5, and gene7 have little possibility to regulate gene2; 2) gene2, gene3, gene8 have little possibility to regulate gene7. Hence, the indicator matrix in (50) is given by

$$\begin{array}{l}\mathit{E}=\left(\begin{array}{llllllll}0& 0& 0& 0& 0& 0& 0& 0\\ 1& 0& 0& 0& 1& 0& 1& 0\\ 0& 0& 0& 0& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 0& 0\\ 0& 0& 0& 0& 0& 0& 0& 0\\ 0& 1& 1& 0& 0& 0& 0& 1\\ 0& 0& 0& 0& 0& 0& 0& 0\end{array}\right).\end{array}$$

(79)

The comparison of the UKF and the UKF incorporating the existing network information (denoted by UKF_{p 1}) with λ = 2 is shown in Table 3. It can be seen that the average TP # and TN # of the UKF_{p 1} are both higher than those of the UKF. In addition, the average FP # and FN # of the UKF_{p 1} are lower than those of the UKF. Hence, the UKF_{p 1} predicts more correct links and nonlinks than the UKF. Moreover, the UKF_{p 1} produces less incorrect links and missed links than the UKF. The average TPR and the precision of the UKF_{p 1} are higher than those of the UKF. In addition, the average FPR of the UKF_{p 1} is lower than that of the UKF. Hence, by using the existing network information, the inference accuracy can be improved.

Table 3 Inferred results of the conventional filter and filters incorporating the prior information

The performance of UKF_{p 1} with different λ is shown in Table 4. It is seen that the performance of UKF_{p 1} and UKF is close when λ is small since only small regulation is imposed on the solution. When λ is large, the difference between the UKF_{p 1} and UKF is large. In particular, the UKF_{p 1} provides sparser solution than the UKF when λ is large. It can be seen from Table 4, the average FPR of UKF_{p 1} decreases with the increasing of λ. The average TPR of UKF_{p 1}, however, does not increase monotonically with the increasing of λ. The average PPR of UKF_{p 1} increases with the increasing of λ. Hence, roughly speaking, the UKF_{p 1} is better than the UKF when large λ is used.

Table 4 Comparison of UKF _p1 using different λ

To consider the strength of the links, rather than setting it to 1, e _ij  (in the indicator matrix E) is set to different values. Large e _ij  is used if the strength of the link from gene j to gene i is strong. For convenience, the UKF considering the strength of links is denoted as ${\text{UKF}}_{\widehat{p}1}$. To compare the performance of ${\text{UKF}}_{\widehat{p}1}$ with UKF_{p 1}, for ${\text{UKF}}_{\widehat{p}1}$, the values of the second row in Equation (79) is multiplied by 5. The performance of ${\text{UKF}}_{\widehat{p}1}$ using different λ is given in Table 5. It can be seen from Tables 4 and 5 that the performance of ${\text{UKF}}_{\widehat{p}1}$ and UKF_{p 1} is close when λ is small, e.g., λ = 0.1. In addition, the average TPR and FPR of ${\text{UKF}}_{\widehat{p}1}$ is smaller than that of UKF_{p 1} for all tested λ except for λ = 0.1. Hence, PPR is used to evaluate the performance of ${\text{UKF}}_{\widehat{p}1}$ and UKF_{p 1}. Although the average PPR of ${\text{UKF}}_{\widehat{p}1}$ and UKF_{p 1} is close when the λ is large, e.g., λ = 10, the average PPR of ${\text{UKF}}_{\widehat{p}1}$ is consistently higher than that of UKF_{p 1}. The results indicate that the inference accuracy of ${\text{UKF}}_{\widehat{p}1}$ and UKF_{p 1} are close when λ is very small or very large. The inference accuracy of ${\text{UKF}}_{\widehat{p}1}$ outperforms UKF_{p 1} when the appropriate strength of the link and parameter λ are used.

Table 5 Effect of strength of the links using different λ

To consider the effect of false prior knowledge, the second row of the indicator matrix in Equation (79) is changed to [ 0,1,1,1,0,1,0,1], which conflicts with the truth. For convenience, we use ${\text{UKF}}_{\bar{p}1}$ to denote the UKF incorporating this false prior knowledge. In Table 6, the performance of ${\text{UKF}}_{\bar{p}1}$ with different λ is shown. It can be seen from Tables 4 and 6 that the average TPR of ${\text{UKF}}_{\bar{p}1}$ is smaller than that of UKF_{p 1} when λ is small, e.g., λ = 0.1,0.5. In addition, the average FPR of ${\text{UKF}}_{\bar{p}1}$ is larger than that of UKF_{p 1} when λ is large, e.g., λ = 5,10. Moreover, although the average PPR of ${\text{UKF}}_{\bar{p}1}$ is close to that of UKF_{p 1} when λ is small, the average PPR of ${\text{UKF}}_{\bar{p}1}$ is consistently lower than that of UKF_{p 1}. Hence, as expected, the results indicate that the false prior knowledge will lead to worse inference result.

Table 6 Effect of false prior information using different λ

Incorporating LASSO The problem setup is the same as before except that the LASSO rather than the existing network information is used. The UKF incorporating LASSO is denoted as UKF_{p 2}.

As shown in Table 3, the average TP # and FP # of UKF_{p 2} are lower than those of UKF and the average TN # and FN # of UKF_{p 2} are higher than those of UKF. Hence, UKF_{p 2} produces less links, including correct and incorrect ones. In addition, UKF_{p 2} produces more nonlinks and missed links. It is consistent with the fact that the LASSO tends to provide a sparse solution. It can be seen from Table 3 that the average FPR of UKF_{p 2} is lower than that of UKF and the average precision of UKF_{p 2} is higher than that of UKF. Hence, by incorporating LASSO, the inference accuracy is improved.

A representative inference result of UKF_{p 2} and the true regulations are shown in Figure 1. For comparison, the inference result of UKF and the true regulations are shown in Figure 2. By comparing Figure 2 and Figure 1, it can be seen that UKF falsely predicts the nonlinks from gene1 to gene2, from gene3 to gene6, from gene4 to gene8, from gene5 to gene2, and from gene6 to gene4 while UKF_{p 2} does not.

Figure 1

Inferred regulations of UKF _{p 2} and true regulations.

Figure 2

Inferred regulations of UKF and true regulations.

The performance UKF_{p 2} with different λ is shown in Table 7. It is seen that the performance of UKF_{p 2} and UKF is close when λ is small since only small regulation is imposed on the solution. When λ is large, the difference between UKF_{p 2} and UKF is large. The average TPR and FPR of UKF_{p 2} decrease with the increasing of the λ. The average PPR does not increase monotonicallly with the increasing of λ. Generally speaking, for different λ, UKF_{p 2} is more sensitive than that of UKF_{p 1}. Although the performance of UKF_{p 2} depends on λ, the average PPR of UKF_{p 2} is consistently higher than that of UKF. Hence, roughly speaking, UKF_{p 2} has better performance than UKF.

Table 7 Comparison of UKF _p2 using different λ

Incorporating the range constraint The existing network information can be used to provide the rough range constraint of $\bar{\mathit{x}}$. A tight constraint is forced on the regulation coefficient a _ij  when there is a small regulation possibility from genej to genei and a loose constraint is forced on the regulation coefficient with no prior information. In the simulation, for the coefficients corresponding to the zero elements in (79), the lower bound and the upper bound are set as -10 and 10, respectively. For the coefficients corresponding to the nonzero elements in (79), the lower bound and the upper bound are set as -0.1 and 0.1, respectively. The UKF incorporating the range constraint is denoted as UKF_{p 3}. As shown in Table 3, the average FPR of UKF_{p 3} is lower than that of UKF and the average precision of UKF_{p 3} is higher than that of UKF.

5.2 Yeast protein synthesis network

In this section, time-series gene expression data of the yeast protein synthesis network is used. Five genes (HAP1, CYB2, CYC7,CYT1, and COX5A) of the yeast protein synthesis network are considered and 17 data points which can be found in [35] are collected. The regulation relationship between them has been revealed by the biological experiment and shown in Figure 3. The dashed arrow in Figure 3 denotes ‘repression’ and the solid arrow denotes ‘activation.’

Figure 3

Pathway model of the five genes in yeast protein synthesis network.

The GRN is inferred by the UKF and UKF_{p 2}. The predicted gene expressions using parameters estimated by UKF_{p 2} and the true measured gene expressions are shown in Figure 4. It can be seen that the model output fits the measured data well. The variances of the regulatory coefficients of HAP1 (P_1i (1 ≤ i ≤ 5)) are shown in Figure 5. It can be seen that the filter converges since the variance P_1i approaches zero. The results for other regulatory coefficients are similar and not shown here. The evaluation of the inferred GRN by UKF and UKF_{p 2} is shown in Table 8.

Figure 4

True gene expression and model output.

Figure 5

Variance of regulatory coefficients.

Table 8 Inferred results of the UKF and UKF _p2

By incorporating the sparsity constraint, UKF_{p 2} provides much better inference results than UKF. As shown in Table 8, the TP # and TN # of UKF_{p 2} are higher than those of UKF and the FP # and FN # are lower than those of UKF. In addition, it can be seen from Table 8, the FPR of UKF_{p 2} is lower than that of UKF and the TPR and the precision of UKF_{p 2} is higher than that of UKF.

6 Conclusions

In this paper, we have proposed a framework of employing the point-based Gaussian approximation filters which incorporates the prior knowledge to infer the gene regulatory network (GRN) based on the gene expression data. The performance of the proposed framework is tested by a synthetic network and the yeast protein synthesis network. Numerical results show that the inference accuracy of the GRN by the proposed point-based Gaussian approximation filter incorporating the prior information is higher than using the traditional filters without incorporating prior knowledge. The proposed method works for small- and medium-size GRNs due to the computational complexity considerations. It remains a future research topic how to adapt the proposed inference framework to handle large GRNs at reasonable computational complexity.