Machine learning (ML) technologies have revolutionized various sectors by enabling advanced data analysis and decision-making processes. The effectiveness of these technologies heavily depends on the quality and diversity of their training datasets. While integrating data from multiple sources boosts ML models' predictive power and generalizability, it also introduces significant challenges, most notably the increased risk of user privacy violations. This concern is further compounded by stringent global privacy regulations, including the European Union's General Data Protection Regulation (GDPR), the California Consumer Privacy Act (CCPA), and Japan's amended Act on the Protection of Personal Information (APPI), which enforce strict standards on data privacy and user consent.

The Privacy-Preserving Machine Learning (PPML) domain has evolved in response to these challenges, with Federated Learning (FL) emerging as a notable methodology. FL enables collaborative training of either a global model or enhanced individual models across multiple entities while keeping the data localized, thus securely enhancing model performance. A key algorithm within FL, Federated Averaging (FedAvg), exemplifies this by enabling a central server to distribute a model to clients for local updates, as demonstrated in its application in the Google Keyboard for privacy-preserving query suggestions.

However, FL's reliance on iterative communication poses limitations, particularly in cross-silo settings such as healthcare, where data privacy is critical. Traditional FL approaches, which necessitate frequent model exchanges, often prove impractical in these settings.

Data Collaboration (DC) analysis presents a robust alternative. Distinct from typical FL frameworks, DC centralizes secure, dimensionally reduced intermediate representations of raw data rather than sharing models, thus eliminating the need for iterative updates. Each entity independently generates these representations using proprietary dimension reduction techniques akin to data preprocessing methods like differential privacy or k-anonymity. However, DC uniquely preserves the utility of reduced models by aligning these representations into a unified collaborative representation with minimal distortion.

Our research introduces an optimized collaborative representation critical for the efficacy of the final ML model within DC frameworks. We define and efficiently resolve a collaborative representation optimization problem using the intermediate representations of a shared anchor dataset uniformly distributed to all participating entities. Our proposed technique, Procrustean Data Collaboration (PDC), utilizes orthogonal matrices to minimize distortion in linear transformations, effectively preserving distances and angles between data samples. This approach is analogous to the orthogonal Procrustes problem, which has an established analytical solution.

Empirical evaluations across multiple public datasets demonstrate that our approach adheres to strict privacy standards and significantly improves model performance compared to existing DC frameworks without compromising communication or computational efficiency. We hope the theoretically robust PDC framework represents a significant innovation in PPML, promising to facilitate the broader adoption of ML technologies in privacy-sensitive industries. This is particularly crucial in an era where data privacy concerns are at the forefront of public and regulatory scrutiny.

Riemannian Optimization (RO) is a vibrant research area in the field of optimization, which focuses on optimizing real-valued functions over Riemannian manifolds. The manifolds are characterized as smooth curved spaces that generalize Euclidean spaces. Despite the short history of RO, both its theoretical research and practical applications have experienced rapid growth in the past 20 years. The main trend in RO's theoretical research is to extend classical algorithms (e.g., gradient descent, Newton) to manifolds. In practice, manifolds are often represented as the feasible region of problems, while being a subset of a Euclidean space. By utilizing the Riemannian geometry of manifolds, Riemannian algorithms tend to be more efficient than Euclidean algorithms.

Existing research on RO focuses on optimization problems with a smooth objective function over a single manifold. However, this type of optimization problem sometimes does not satisfy diverse demands well. This talk investigates one variant of RO, called Riemannian Constrained Optimization Problem (RCOP), addresses optimization problems involving additional constraints that cannot form a manifold. This variant can address new challenges in practical applications.

In this talk, for the RCOP problem, we will extend the primal-dual interior point algorithms from the Euclidean setting to the Riemannian setting. We call this extension the Riemannian Interior Point Method (RIPM). Our contributions are summarized as follows:

(1) To our knowledge, this is the first study to apply the primal-dual interior point method to the nonconvex constrained optimization problem on Riemannian manifolds. One significant contribution is that we establish many essential foundational concepts for the general interior point method in the Riemannian context, such as the KKT vector field and its covariant derivative. In addition, we build the first framework for the Riemannian version of the interior point method. These contributions will have uses in the future, especially in developing more advanced interior point methods.

(2) We give a detailed theoretical analysis to ensure local and global convergence of RIPM. Considering that many practical problems involve minimizing a nonconvex function on Riemannian manifolds, the theoretical counterparts of our method are the early interior point methods for nonlinear nonconvex programming first proposed by El-Bakry et al. in 1996.

(3) Our numerical experiments demonstrate the great potential of RIPM. The method meets the challenges presented in these experiments with better stability and higher accuracy compared with the current Riemannian exact penalty, augmented Lagrangian, and sequential quadratic programming methods. The code is freely available at https://github.com/GALVINLAI/RIPM

The quadratically constrained linear programming (QCLP) problem is a convex optimization problem where a quadratic function serves as a constraint alongside an objective function represented as a linear function. One method for solving convex optimization problems is through the homogeneous self-dual (HSD) model. This model combines the primal and dual problems derived from conic programming problems. It is based on determining a nonzero point from the intersection of two convex sets: a convex cone and a subspace. The HSD model finds application in various problem domains, including QCLP. This research contributes to solving the HSD model of the QCLP problem in the case of portfolio optimization. We proposed an approach based on Dykstra’s projection that can be employed to assess the existence of solutions in the HSD model and to approximate the optimal solution of the portfolio optimization problem.

The log-det semidefinite programming is a problem that appears in the Gaussian Graphic Models. There are many methods proposed for solving this problem, and one of them is the Dual Spectral Projected Gradient Method (DSPG) which was proposed by Nakagaki et al. in 2020. The DSPG method is designed to solve the dual problem of the log-det semidefinite programming by combining the non-monotone line search projected gradient method with the step adjustment. In this presentation, we propose the generalized log-det semidefinite programming by adding the extra term into the objective function to support the extra structure of Gaussian Graphical Models and propose the application of the DSPG method to the proposed problem. We show the global convergence of the DSPG on the proposed problem and give some numerical results from the experiments on synthesis data.