Wenjie Li, Yao Li, Entropy, mutual information, and systematic measures of structured spiking neural networks, Journal of Theoretical Biology, Volume 501, 2020, 110310, ISSN 0022-5193, https://doi.org/10.1016/j.jtbi.2020.110310.
Application of Self-Supervised Learning to an Image Classification Task with Scarce Labels
Final Project of Prof. Yann LeCun's Deep Learning course, New York University (Jan. -- May 2021)
We devised an approach utilizing unsupervised learning, pseudo-label iterations, semi-supervised learning and active learning methods. The final result achieves 55.80% accuracy on the test dataset with 0.5% labeled data. Click here for the full report.
Bayesian Modeling of Dpp regulations in Fruit Flies
NSF-Simons Center for Quantitative Biology, Northwestern University (Jun. – Aug. 2019)
It is important for gene to be expressed in certain spatio-temporal order during development. And the levels of mRNA are important parameters of gene expression. In this project, we study the regulations of Dpp, a kind of signaling molecules in the imagical disc of adult Drosophila wings, on its downstream genes, such as salm and brk. We investigate this by modeling the kinetic parameters of mRNA transcriptions corresponding to the concentration of Dpp. To do this, we utilize Bayesian inference and Markov Chain Monte Carlo (MCMC) sampling tools. This method has been tested on synthetic data and it made a prediction with under 8% error rate . We have applied this method to brk and salm gene. Learn more about this project on GitHub
A Quantum Markov Chains Model of DNA transcriptions
Freiwald Scholar Research Program, Washington University in St. Louis (Jan. – May 2019)
This project investigates the use of both classical and quantum information theory to describe the genome preservation and biological evolution. This write-up focus on the research of Djordijevic, who attempts to explain the transfer of genetic information from DNA to protein in a Markovian-like quantum biological model. The classical counterpart of this model is proposed by Yockey.
Information Measures on Spiking Neural Networks
Research Experience for Undergrads, University of Massachusetts, Amherst (Jul. – Sep. 2018)
It is important to understand how neuronal networks, including our brains, encode and decode information. It is well known that neurons transmit information by time series of spike trains. The aim of the project is to investigate various information-theoretic measures, including entropy, mutual information, and some systematic measures, for a class of structured spiking neuronal network.
In order to analyze these information-theoretic measures for large networks, we coarse-grained the data by ignoring the order of spikes that fall into the same small time bin. The resultant coarse-grained entropy mainly captures the information contained in the rhythm produced by a local population of the network. We first proved that these information theoretical measures are well-defined and computable by proving the stochastic stability and the law of large numbers. Then we used three neuronal network examples to investigate them. The motivation of our definition is that the spiking pattern generated by a neuronal network is usually not homogeneous. By collecting spike counts in time windows, we have the uncertainty of the spiking pattern. Heuristically, if the spiking pattern is completely homogeneous, it contains little information from a coarse-grained sense, as the spike count in a time window has little variation. Same thing happens if the spiking activity is completely synchronized, at which we have “all-or-none” spike counts in a time window. In contrast, the spiking pattern contains most information when partially synchronized, and if its degree of synchronization has high variation. This is confirmed by our numerical study.
The definition of coarse-grained entropy can be extended to multiple local populations. This gives the concept of mutual information, which measures the amount of information shared by two local populations of a neuronal network. We find that feedforward/feedback connections can enhance mutual information in all cases.
Another aim of this project is to quantify systematic measures, such as degeneracy and complexity, to spiking neuronal networks. Biologically speaking, degeneracy measures the ability of structurally different components of a neuronal network to perform similar function. The (structural) complexity measures how different components in a neuronal network functionally depend on each other. We proved that a neuronal network with high degeneracy must be (structurally) complex. Finally, the dependency of degeneracy and complexity on certain network parameters is studied for our cortex models.