Logistic Product Basis Network (LPBN) is an architecture for approximation of general functions. LPBN is very similar to LDNN in terms of structure. LPBN consists of one adaptive layer of feature detectors implemented by logistic sigmoid functions followed by a fixed conjunction layer and a final adaptive summation.

There are many regression methods that build the approximation function by linear combination of a set of basis functions. The choice of basis functions determines the type of regression method. Support Vector Regression (SVR) and Radial Basis Function (RBF) networks fall in this category. The conjunctions in the second layer of LPBN can be considered as different basis functions. Every conjunction implements a convex polytope that covers a specific part of high-dimensional space. But unlike radial basis function networks, this coverage is non-local. So, LPBNs can represent functions in high dimensional spaces more efficiently without suffering from the curse of dimensionality.

In this setup, the number of conjunctions of second hidden layer equals to the number of basis functions and this is the main parameter of this method. Similar to LDNN, there is a simple yet effective way to initialize the learning weights of the first layer. Assuming that N is the number of basis functions, we partition the training data into N clusters. We provide space partitioning by using discriminants that separate the data of one cluster from other clusters. So, for every cluster, we need N-1 discriminants that separate the members of the cluster from the rest of datapoints. So, we initialize the weights of the discriminants as the vectors that connect the centroids of two different clusters. The conjunction of these N-1 discriminants forms the convex polytope. This conjunction is being done using soft AND gates shown in this diagram. Then, all the weights in this structure will be updated simultaneously using backpropagation algorithm via chain rule.

References:

[1] Mehdi Sajjadi, Mojtaba Seyedhosseini, and Tolga Tasdizen. "Nonlinear regression with logistic product basis networks." IEEE Signal Processing Letters 22.8 (2015): 1011-1015.