DESCRIPTION: This is a learning/reading seminar on stochastic systems. Faculty, postdocs, visiting scholars and graduate students will take turns presenting topics and research papers on stochastic systems arising in applications. We will also have some guest speakers from other institutions. Potential areas of application include biology, telecommunications, operations management, neuroscience and finance.

This seminar will run in Fall, Winter and Spring 2021-2022. It will meet at 3-3.50pm on Thursdays (initially via zoom). Please email the instructor for further information.

The seminar will meet once a week for an hour. PhD students can sign up for one unit of Math 288, Section C. (Suitable background is having taken the equivalent of Math 280ABC.)

FALL 2021

  • Thursday, October 7, 2021, 1pm NOTE UNUSUAL TIME
    Neil Walton, University of Manchester, UK.
    Title: Learning and Information in Stochastic Networks and Queues: A tutorial
    Abstract: We review the role of information and learning in the stability and optimization of queueing systems. In recent years, techniques from supervised learning, bandit learning and reinforcement learning have been applied to queueing systems supported by increasing role of information in decision making. We present observations and new results that help rationalize the application of these areas to queueing systems. We prove that the MaxWeight and BackPressure policies are an application of Blackwell’s Approachability Theorem. This connects queueing theoretic results with adversarial learning. We then discuss the requirements of statistical learning for service parameter estimation. As an example, we show how queue size regret can be bounded when applying a perceptron algorithm to classify service. Next, we discuss the role of state information in improved decision making. Here we contrast the roles of epistemic information (information on uncertain parameters) and aleatoric information (information on an uncertain state). Finally we review recent advances in the theory of reinforcement learning and queueing, as well as, provide discussion on current research challenges. (Joint work with Kuang Xu).

  • Thursday, October 14, 2021, 3pm.
    Eliza O'Reilly, CalTech.
    Title: Random Tessellation Features and Forests
    Abstract: The Mondrian process in machine learning is a Markov partition process that recursively divides space with random axis-aligned cuts. This process is used to build random forests for regression and classification as well as Laplace kernel approximations. The construction allows for efficient online algorithms, but the restriction to axis-aligned cuts does not capture dependencies between features. By viewing the Mondrian as a special case of the stable under iterated (STIT) process in stochastic geometry, we resolve open questions about the generalization of cut directions. We utilize the theory of stationary random tessellations to show that STIT processes approximate a large class of stationary kernels and STIT random forests achieve minimax rates for Lipschitz functions (forests and trees) and C^2 functions (forests only). This work opens many new questions at the intersection of stochastic geometry and statistical learning theory. Based on joint work with Ngoc Mai Tran.

  • Thursday, October 21, 2021, 3pm.
    Wasiur Khuda Bukhsh, University of Nottingham, UK

    Title: Incorporating age and delay into models for biophysical systems
    Abstract: In many biological systems, chemical reactions or changes in a physical state are assumed to occur instantaneously. For describing the dynamics of those systems, Markov models that require exponentially distributed inter-event times have been used widely. However, some biophysical processes such as gene transcription and translation are known to have a significant gap between the initiation and the completion of the processes, which renders the usual assumption of exponential distribution untenable. In this talk, we consider relaxing this assumption by incorporating age-dependent random time delays (distributed according to a given probability distribution) into the system dynamics. We do so by constructing a measure-valued Markov process on a more abstract state space, which allows us to keep track of the 'ages' of molecules participating in a chemical reaction. We study the large-volume limit of such age-structured systems. We show that, when appropriately scaled, the stochastic system can be approximated by a system of partial differential equations (PDEs) in the large-volume limit, as opposed to ordinary differential equations (ODEs) in the classical theory. We show how the limiting PDE system can be used for the purpose of further model reductions and for devising efficient simulation algorithms. To describe the ideas, we will use a simple transcription process as a running example.

  • Thursday, October 28, 2021, 3pm.
    Miroslav Krstic, UCSD
    Stabilization of a Hyperbolic PDE of a Bioreactor With Distributed Age
    For an advection-reaction PDE model of population, with a non-local boundary condition modeling "birth", and with a multiplicative input whose nature is the "harvesting rate", we design a feedback law that stabilizes a desired equilibrium profile (of population density vs. age). Without feedback the system has one eigenvalue at the origin and the remainder of its infinite spectrum has negative real parts, i.e., the systems is, as engineers call it, "neutrally stable". Hence, a feedback is needed to move one eigenvalue to the left without making any of the other ones spill to the right of the imaginary axis. This control design objective is achieved by transforming the system into a control-theoretic canonical form consisting of a first-order ODE in which the input is present and whose eigenvalue needs to be made negative by feedback, and an infinite-dimensional input-free system called the "zero dynamics", which we prove to be exponentially stable. The key feature of the overall PDE system and its feedback control law is the positivity of both the population density state and the harvesting rate input, which is a key element of the analysis, captured by a "control Lyapunov functional" which blows up when the population density or control approach zero.

  • November 18, 2021, 3pm.
    Varun Khurana, UCSD.
    Learning Sheared Distributions with Linearized Optimal Transport
    In this paper, we study supervised learning tasks on the space of probability measures. We approach this problem by embedding the space of probability measures into $L^2$ spaces using the optimal transport framework. In the embedding spaces, regular machine learning techniques are used to achieve linear separability. This idea has proved successful in applications and when the classes to be separated are generated by shifts and scalings of a fixed measure. This paper extends the class of elementary transformations suitable for the framework to families of shearings, describing conditions under which two classes of sheared distributions can be linearly separated. We furthermore give necessary bounds on the transformations to achieve a pre-specified separation level, and show how multiple embeddings can be used to allow for larger families of transformations. We demonstrate our results on image classification tasks.
    Based on joint work with Caroline Moosmueller, Harish Kannan, and Alex Cloninger.

  • December 2, 2021, 3pm.
    Yiren Wang, UCSD.
    On Diffusion Asymptotics for Sequential Experiments, following Stefan Wager Kuang Xu.

    WINTER 2022

    Thursday, January 13, 2022, 3 p.m. PST.
    Ankit Gupta, ETH,
    DeepCME: A deep learning framework for computing solution statistics of the Chemical Master Equation.
    Abstract: Stochastic reaction network models are a popular tool for studying the effects of dynamical randomness in biological systems. Such models are typically analysed by estimating the solution of Kolmogorov's forward equation, called the chemical master equation (CME), which describes the evolution of the probability distribution of the random state-vector representing molecular counts of the reacting species. The size of the CME system is typically very large or even infinite, and due to this high-dimensional nature, accurate numerical solutions of the CME are very difficult to obtain. In this talk we will present a novel deep learning approach for computing solution statistics of high-dimensional CMEs by reformulating the stochastic dynamics using Kolmogorov's backward equation. The proposed method leverages superior approximation properties of Deep Neural Networks (DNNs) to reliably estimate expectations under the CME solution for several user-defined functions of the state-vector. Our method only requires a handful of stochastic simulations and it allows not just the numerical approximation of various expectations for the CME solution but also of its sensitivities with respect to all the reaction network parameters (e.g. rate constants). We illustrate the method with a number of examples and discuss possible extensions and improvements.
    This is joint work with Prof. Christoph Schwab (Seminar for Applied Mathematics, ETH Zurich) and Prof. Mustafa Khammash (Department of Biosystems Science and Engineering, ETH Zurich)
    Reference: Gupta A, Schwab C, Khammash M (2021) DeepCME: A deep learning framework for computing solution statistics of the chemical master equation. PLoS Comput Biol 17(12): e1009623. For a copy of the paper, click here.

    Thursday, January 20, 2022, 1pm PST (NOTE UNUSUAL TIME)
    Corentin Briat, ETH.
    Optimal Continuous and Sampled-Data Control of Stochastic Reaction Networks
    Abstract. Reactions networks are very flexible modeling tools that can be used to model biochemical, epidemiological, ecological and population networks. In certain situations, stochastic reaction networks need to be considered in place of their deterministic counterparts. This is notably the case in biology where some molecular counts can be typically small, meaning that the randomness in the interactions cannot be neglected anymore. After few words on reaction networks theory, biology and (optimal) control theory as introductory comments, I will address the problem of the in-silico optimal control of stochastic reaction networks. The theory extensively relies on the use of Dynamic Programming, which allows to formulate a solution to the optimal control problem in terms of the solution of a differential-difference equation. In the unimolecular case, this reduces to solving a Riccati differential equation. The sampled-data control of reaction networks will be addressed next using a hybrid formulation of the system. In this case, the solution is formulated in terms of a Lyapunov differential equations and a Riccati difference equation which can be numerically solved in order to find the optimal control input.

    Thursday, February 3, 2022, 3 pm.
    Amber Puha, CSUSM.
    Large-time limit of nonlinearly coupled measure-valued equations that model many-server queues with reneging, following Rami Atar, Weining Kang, Haya Kaspi, Kavita Ramanan.
    For a copy of the paper, click here.

    Thursday, March 3, 2022, 3pm
    Hye-Won Kang, University of Maryland, Baltimore County.
    Title: TBA


    Learning and Information in Stochastic Networks

  • Neil Walton and Kuang Xu, Learning and Information in Stochastic Networks and Queues, 2021. For the associated preprint, click here.
  • Stefan Wager Kuang Xu, Diffusion Asymptotics for Sequential Experiments. For a preprint of this paper click here.

    Optimal transport, Wasserstein distance and relative entropy

  • Many stochastic network models described by measure-valued processes can be approximated by fluid models for which one wishes to establish stability and asymptotic behavior. These fluid models can sometimes be regarded as transport models. There are multiple publications/books by Cedric Villani on optimal transport and also relevant papers by others
  • Relative entropy in chemical reaction networks, works by M. Esposito.

    Mean Field Games

  • Introduction to Mean Field Games
  • Learning by examples: what is a mean field game.
  • Two volume set by Carmona et al. on mean field games.
  • From the master equation to mean field game limit theory: Large deviations and concentration of measure, Delarue, Lacker and Ramanan.

    Nonlinear Filtering

  • An efficient algorithm of Yau-Yau method for nonlinear filtering Yau-Yau method for nonlinear filtering, part I.
  • Yau-Yau method for nonlinear filtering, part II.

    Biochemical Reaction Networks

  • See papers by Professor Tom Kurtz by clicking here (Google scholar page).
  • For some background on stochastic models of biochemical reaction networks, see D. F. Anderson and T. G. Kurtz, Continuous time Markov chain models for chemical reaction networks, chapter in Design and Analysis of Biomolecular Circuits: Engineering Approaches to Systems and Synthetic Biology, H. Koeppl. et al. (eds.), Springer.
  • David F Anderson, Daniele Cappelletti, Masanori Koyama, Thomas G Kurtz, Non-explosivity of stochastically modeled reaction networks that are complex balanced, Bulletin of Mathematical Biology, 2018.
  • David F Anderson, Daniele Cappelletti, Thomas G Kurtz, Finite time distributions of stochastically modeled chemical systems with absolute concentration robustness, SIAM J on Applied Dynamical Systems, 2017.
  • Cristina Costantini and Thomas G. Kurtz, Viscosity methods giving uniqueness for martingale problems, Electronic J Probability, 2015.
  • Large deviations theory for Markov jump models of chemical reaction networks, Agazzi et al.
  • On the geometry of chemical reaction networks, Agazzi et al.
  • Seemingly stable chemical kinetics can be stable, marginally stable, or unstable, Agazzi et al.
  • Critical parameters for singular perturbation reductions of chemical reaction networks, E Feliu, S Walcher, C Wiuf
  • Strong Gaussian approximation of metastable density-dependent Markov chains on large time scales, Adrien Prodhomme, for a preprint, click here.

    Control and Adaptation in Biological Networks

  • Kinetic Uncertainty Relations for the Control of Stochastic Reaction Networks
  • Briat, Gupta and Khammash, Antithetic Integral Feedback Ensures Robust Perfect Adaptation in Noisy Biomolecular Networks, Cell Systems, 2016.
  • Corentin Briat, Ankit Gupta, Mustafa Khammash, Antithetic proportional-integral feedback for reduced variance and improved control performance of stochastic reaction networks, J. Royal Society Interface, 2018.
  • Ankit Gupta, Mustafa Khammash, Sensitivity analysis for multiscale stochastic reaction networks using hybrid approximations, preprint, 2018.

  • Overview of control in biological systems on the website of Prof Eduardo Sontag
  • Index to papers of Professor Eduardo Sontag.
  • D. Del Vecchio, Y. Qian, R.M Murray, and E.D. Sontag. Future systems and control research in synthetic biology. Annual Reviews in Control, 45:5-17, 2018.
  • E.D. Sontag. Examples of computation of exact moment dynamics for chemical reaction networks. In R. Tempo, S. Yurkovich, and P. Misra, editors, Emerging Applications of Control and Systems Theory, volume 473 of Lecture Notes in Control and Inform. Sci., pages 295-312. Springer-Verlag, Berlin, 2018.

  • Website of Professor Johan Paulsson on stochastics and decision making.
  • Laurent Potvin-Trottier, Nathan D. Lord, Glenn Vinnicombe and Johan Paulsson, Synchronous long-term oscillations in a synthetic gene circuit, Nature, 2016.
  • Hilfinger, A. ; Norman, T. M. ; Paulsson, J. Exploiting natural fluctuations to identify kinetic mechanisms in sparsely characterized systems. Cell Systems, 2016.
  • Potvin-Trottier, Luro, Paulsson, Microfluidics and single-cell microscopy to study stochastic processes in bacteria, Current Opinion in Microbiology, 2018.

  • Website of Professor Richard Murray.
  • X. Flora Meng, Ania-Ariadna Baetica, Vipul Singha, and Richard M. Murray, Recursively constructing analytic expressions for equilibrium distributions of stochastic biochemical reaction networks, J. Royal Society Interface, 2017.
  • Xinying Ren, Richard M Murray, Cooperation Enhances Robustness of Coexistence in Spatially Structured Consortia

    Stochastic Models in Gene Networks

  • The combined effects of Feller diffusion and transcriptional/translational bursting in simple gene networks, Mateusz Falfus, Michael C. Mackey, and Marta Tyran-Kaminska, Preprint, 2019.
  • Long-range allosteric interactions between the folate and methionine cycles stabilize DNA methylation reaction rate, H Frederik Nijhout 1 , Michael C Reed, David F Anderson, Jonathan C Mattingly, S Jill James, Cornelia M Ulrich.

    Bandwidth Sharing Networks

  • Weina Wang, Siva Theja Maguluri, R Srikant, Lei Ying, Heavy-Traffic Insensitive Bounds for Weighted Proportionally Fair Bandwidth Sharing Policies, preprint, 2018.

    Inference for Stochastic Processes

  • Inference of dynamic systems via manifold-constrained Gaussian processes, Yang, Wong and Kou.
  • A Tale of Two Time Scales: Determining Integrated Volatility with Noisy High Frequency Data

    Resource Sharing in Stochastic Networks

  • J. N. Tsitsiklis and K. Xu, On the power of (even a little) resource pooling, Stochastic Systems, 2: 1-66, 2012. For a copy of a preprint, click here. Correction, click here.


  • This Overlooked Variable Is the Key to the Pandemic Its not R, by ZEYNEP TUFEKCI, SEPTEMBER 30, 2020 for the Atlantic.
    Related mathematical article Implication of backward contact tracing in the presence of overdispersed transmission in COVID-19 outbreak, by Kurcharski et al., August 4, 2020.
  • The challenges of modeling and forecasting the spread of COVID-19, Andrea L Bertozzi, Elisa Franco, George Mohler, Martin B Short, Daniel Sledge
  • Flattening the Curve: Insights From Queueing Theory, Sergio Palomo, Jamol Pender, William Massey, Robert C. Hampshire.
  • A stochastic epidemic model of COVID-19 disease, Xavier Bardina, Marco Ferrante, Carles Rovira
  • SIR-Hawkes: Linking Epidemic Models and Hawkes Processes to Model Diffusions in Finite Populations, Marian-Andrei Rizoiu et al.
  • How Control Theory Can Help Us Control COVID-19, IEEE Spectrum.
  • Controlling Propagation of epidemics via mean-field games, Wonjun Lee, Siting Liu, Hamidou Tembine, Wuchen Li, Stanley Osher.


  • Jonathan Touboul, Propogation of chaos in neural fields.


  • KPZ and reflecting Brownian motion, Nica, Quastel and Remenik.
  • Splitting Algorithms for Rare Events of Semimartingale Reflecting Brownian Motions, by Leder, Liu, Wang, Stochastic Systems, 2021.

    Mathematical Finance

  • Minmax option pricing meets Black-Scholes in the limit.

    Road Traffic Modeling

  • Diffusion Based Analysis of a Multiclass Road Traffic Network, Mandjes and Storm, Stochastic Systems, 2021.
  • Fluid model of a traffic network with information feedback, Helton, Kelly, Williams and Ziedins.

    SOME SAMPLE PAPERS (from the 2005-06 seminar series)

  • Biology
  • Epidemics and Immunology
  • Finance
  • Neuroscience
  • Operations Management
  • Communications

    Please address all enquiries concerning this seminar to Professor Williams at rjwilliams at ucsd dot edu

  • 2005-06.

  • WINTER 2020

    Thursday, January 30, 2020
    Eva Loeser, On the equivalence between processor sharing and service in random order following Borst et al.

    Thursday, February 6, 2020
    Felipe Campos, On Wasserstein distance

    Thursday, February 13, 2020
    Professor Angela Yu, Cognitive Science, UCSD
    Three wrongs make a right: reward underestimation mitigates idiosyncrasies in human bandit behavior

    Thursday, February 20, 2020
    Yingjia Fu, on relative entropy.

    Thursday, February 27, 2020
    Jiaqi Liu, on minmax option pricing meets Black-Scholes in the limit.

    FALL 2020

    Thursday, October 1, 2020, 3pm.
    Organizational meeting.

    Thursday, October 8, 2020, 3pm.
    Professor Amber Puha, California State University, San Marcos and visiting UCSD,
    on "Workload-Dependent Dynamic Priority for the Multiclass Queue with Reneging" following Rami Atar, Anat Lev-Ari.

    Thursday, October 15, 2020, 3pm.
    Toni Gui, UCSD, on Mean Field Games and Mathematical Finance

    Thursday, October 22, 2020, 3pm.
    Varun Khurana, UCSD, on "Mean Field Games, Mean Field Control Problems and Machine Learning".

    Thursday, October 29, 2020, 3pm.
    Yingjia Fu, UCSD,
    On "Asymptotic Behavior of a Critical Fluid Model for Bandwidth Sharing with General File Size Distributions"

    Thursday, November 5, 2020, 3pm.
    Andrea Agazzi, Duke University. Large deviations and chemical reaction networks.

    Thursday, November 12, 2020, 3pm.
    Eva Loeser, UCSD, on A stochastic epidemic model of COVID-19 disease

    Thursday, November 19, 2020, 3pm.
    Yiren Wang, UCSD, on "A Tale of Two Time Scales: Determining Integrated Volatility with Noisy High Frequency Data"

    Thursday, December 3, 2020, 3pm.
    Felipe Campos, UCSD, "On Separation of time-scales and model reduction for stochastic reaction networks". Click here for a copy of the paper by Kang and Kurtz on which this talk is based.

    WINTER 2021

    Thursday, January 7, 2021
    No meeting, so as not to conflict with the Joint Math Meetings.

    Thursday, January 14, 2021
    Sam Babichenko, UCSD undergraduate student, on "Mean Field Games and Interacting Particle Systems" following David Lacker. For a preprint of the relevant paper, click here.

    Thursday, January 21, 2021. ( AWM talk at 4pm. Note special time. Zoom information can be found by clicking on the AWM talk link.)
    Professor Amber Puha, California State University San Marcos.
    From Queueing Theory to Modern Stochastic Networks: A Mathematical Perspective.

    Thursday, February 4, 2021.
    Professor Cristina Costantini, Universite di Chieti-Pescara, Italy.
    Obliquely reflecting diffusions in non-smooth domains: some new existence and uniqueness results.
    Click here for first paper related to this talk.
    Click here for a second paper related to this talk.
    Draft of a third paper related to the talk.

    Thursday, February 25, 2021
    Eva Loeser, UCSD graduate student, On Heavy Traffic Limit for a Processor Sharing Queue with Soft Deadlines, following Gromoll and Kruk (2007).

    Thursday, March 4, 2021
    Varun Khurana, UCSD graduate student, on "Deep hedging" following Hans Buehler, Lukas Gonon, Josef Teichmann, and Ben Wood. For the relevant paper, click here.

    SPRING 2021

    Thursday April 29, 2021
    Yueyang Zhong (PhD student, University of Chicago), Behavior-Aware Queueing: The Finite-Buffer Setting with Many Strategic Servers.
    This talk is based on joint work with Amy R. Ward (Chicago Booth) and Raga Gopalakrishnan (Smith School of Business, Queens University).
    For the paper, click here.

    Thursday May 6, 2021
    Angelos Aveklouris, University of Chicago.
    Matching demand and supply in service platforms.
    For the related paper, please click here.
    Abstract: Service platforms must determine rules for matching heterogeneous demand (customers) and supply (workers) that arrive randomly over time and may be lost if forced to wait too long for a match. We show how to balance the trade-off between making a less good match quickly and waiting for a better match, at the risk of losing impatient customers and/or workers. When the objective is to maximize the cumulative value of matches over a finite-time horizon, we propose discrete-review matching policies, both for the case in which the platform has access to arrival rate parameter information and the case in which the platform does not. We show that both the blind and nonblind policies are asymptotically optimal in a high-volume setting. However, the blind policy requires frequent re-solving of a linear program. For that reason, we also investigate a blind policy that makes decisions in a greedy manner, and we are able to establish an asymptotic lower bound for the greedy, blind policy that depends on the matching values and is always higher than half of the value of an optimal policy. Next, we develop a fluid model that approximates the evolution of the stochastic model and captures explicitly the nonlinear dependence between the amount of demand and supply waiting and the distribution of their patience times. We establish a fluid limit theorem and show that the fluid limit converges to its equilibrium. Based on the fluid analysis, we propose a policy for a more general objective that additionally penalizes queue build-up.

    Thursday May 13, 2021
    Justin Mulvany, PhD student, USC.
    Fair Scheduling of Heterogeneous Customer Populations. (Joint work with Ramandeep Randhawa.)
    For a copy of a related preprint, click here.

    Thursday May 20, 2021
    Felipe Campos, UCSD graduate student.
    Comparison Methods for Markov Processes.
    (The talk will be a combination of Comparison Methods for Stochastic Models and Risks by Muller and Stoyan, and Stochastic Orderings for Markov Processes on Partially Ordered Sets, by William Massey).

    Thursday June 3, 2021
    Pooja Agarwal, UCSD, Infinite-Dimensional Scaling Limits of Many-Server Stochastic Networks.P>