Aimed at sufficiently utilizing available data and prior distribution
information, we introduce a data-driven Bayesian-type approach to
solve multi-stage convex stochastic optimization, which can easily
cope with the uncertainty about data process’s distributions and
their inter-stage dependence. To unravel the properties of the proposed multi-stage Bayesian expectation optimization (BEO) problem,
we establish the consistency of optimal value functions and solutions.
Two kinds of algorithms are designed for the numerical solution of
single-stage and multi-stage BEO problems, respectively. A queuing
system and a multi-stage inventory problem are adopted to numerically demonstrate the advantages and practicality of the new framework and corresponding solution methods, compared with the usual
formulations and solution methods for stochastic optimization problems.

Hydrogen displays promising features for decarbonization industry, transportation and building sectors.
The desired transition towards a hydrogen economy requires hydrogen costs to come down, through optimal choices of infrastructure design and operation. Most of the literature models hydrogen problems in a deterministic manner and solves them using linear programming. In this talk, we present an approach based on multistage stochastic optimization, mixing design choices with operational decisions taken on an hourly basis.
We consider an elementary hydrogen infrastructure which consists of an electrolyzer, a compressor and a storage to serve a transportation demand. This infrastructure is powered by three sources of energy (on site photovoltaics, renewable electricity through power purchase agreement, power grid) with their own characteristics. The modelling of the electrolyzer covers its different functioning modes and the nonlinear relation between the production of hydrogen and the electricity consumption.
The optimization problem is to minimize the expected operational costs over a week by making hourly decisions in an uncertain context. This involves managing the electrolyzer's load and mode, quantifying the electricity supplied by each source, and determining the amount of hydrogen extracted from the storage to satisfy the demand. Renewable energy sources are emphasized in the hydrogen production process to ensure eligibility for a subsidy, which is awarded if the proportion of nonrenewable electricity usage stays under a predetermined threshold. We consider uncertainties affecting on site photovoltaics production and hydrogen demand. All decisions are taken prior to the occurrence of uncertainties, except for the decision regarding the grid electricity supply, which is treated as a recourse following the realization of these uncertainties.
In this work, we formulate a multistage stochastic optimization problem, assuming stagewise independence of the noise. Building on this assumption, we develop decomposition algorithms based on dynamic programming.
We present numerical results for a given infrastructure design. Then, we consider various combinations of infrastructure designs and their subsequent optimal operation. With this, we discuss the optimal sizing of equipment, especially the sensitivity of electrolyzer and storage designs to the uncertainties.

The log-optimal or Kelly strategy has many desirable properties such as maximizing the asympotic long run growth of capital and minimizing the expected time to financial goals. The strategy is aggressive and can have considerable short run risk, where risk is the chance of falling short of a downside threshold. The risk with the Kelly strategy is exacerbated if the distribution of future returns is incorrectly specified. In this paper we consider the risk associated with the error in the estimation of returns, and the risk associated with constraints on the log-optimal investment model. It is assumed that the volatility of asset returns is affected by global market forces and individual asset trading characteristics. An improvement in the estimation of returns is achieved with a hierarchial model, where the uncertain financial market is segmented into regimes with the dynamics of the stochastic regime process being Markovian. The driving force for between regime changes is indicators which characterize economic states. Within a regime the asset prices are lognormal, and distribution parameters depend on the uncertain regimes. To accomodate investor risk aversion, the log-optimal model obtains maximum growth while staying above a specified downside wealth threshold with high probability, and shortfalls below the threshold are penalized in the objective. The enhanced asset pricing model and the controlled growth decision model reduce the risk of investments in uncertain markets. The solution from the decision model has the familiar Kelly formula, with weights to aggregate regime distribution parameters and an investment fraction to protect against shortfalls. The Kelly format reduces the optimization to determining the regime weights and investment fraction in the Kelly portfolio. The methods are illustrated with investments in ETF's over time.

In this talk, we introduce multi-stage distributionally robust portfolio selection (MSDRPS) problems with reinforcement learning moments uncertainty. With uncertainty sets predetermined by fixed reference moments, we derive the closed-form solution of the MSDRPS problem. With multiple prior reference moments, we construct multiple prior uncertainty sets and solve the MSDRPS problem in a dynamic programming approach. When the reference moments information is learnable in a Bayesian rule, we construct two Bayesian reinforcement learning MSDRPS schemes with real investment feedback or reference investment feedback. We propose a two-level decomposition framework to solve the Bayesian reinforcement learning MSDRPS problem using reference investment feedback. Finally, numerical results on newly listed and sub-new stocks examine the practicality and superior performance of the Bayesian reinforcement learning strategy in MSDRPS problems. (this is a joint work with Zhiping Chen, Roujia Li and Zongben Xu)