me

Yu-Guan Hsieh

AI Anime Researcher

Spellbrush

About me

Welcome to my website. I am currently a research scientist at Spellbrush. With the power of AI, we strive to make anime more real than ever!
Before joining Spellbrush, I was a postdoc at Apple MLR, Paris, where I conducted research on vision-language pre-training and generative modeling with a data-centric approach. Even before that, I did a theoretical Ph.D. at Université Grenoble Alpes, focusing on optimization, online learning, and learning in games under the supervision of Jérôme Malick, Franck Iutzeler, and Panayotis Mertikopoulos. Alongside, I contributed to the LyCORIS project that implements a number of algorithms for efficient fine-tuning.

Contact: cyberhsieh212 AT gmail DOT com


News

  • (January 2025) I moved to Tokyo and started working at Spellbrush.

  • (July 2024) We released the GBC datasets which recaption 10M images from CC12M in a new annotation format! Check the paper.

  • (June 2024) I received the Academic Thesis Prize from Université Grenble Alpes!

Publications & Preprints

Simple ReFlow: Improved Techniques for Fast Flow Models
Beomsu Kim, Yu-Guan Hsieh, Michal Klein, Marco Cuturi, Jong Chul Ye, Bahjat Kawar, James Thornton
arxiv 2410.07815, 2024
Paper Arxiv
Graph-Based Captioning: Enhancing Visual Descriptions by Interconnecting Region Captions
Yu-Guan Hsieh, Cheng-Yu Hsieh, Shih-Ying Yeh, Louis Béthune, Hadi Pour Ansari, Pavan Kumar Anasosalu Vasu, Chun-Liang Li, Ranjay Krishna, Oncel Tuzel, and Marco Cuturi
arxiv 2407.06723, 2024
Paper Arxiv Code Datasets
Careful with that Scalpel: Improving Gradient Surgery with an EMA
Yu-Guan Hsieh, James Thornton, Eugene Ndiaye, Michal Klein, Marco Cuturi, and Pierre Ablin
International Conference on Machine Learning (ICML), 2024
Paper Arxiv Poster
Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluation
Shin-Ying Yeh, Yu-Guan Hsieh, Zhidong Gao, Bernard B W Yang, Giyeong Oh, and Yanmin Gong
International Conference on Learning Representations (ICLR), 2024
Paper Arxiv Poster Code
Decision-Making in Multi-Agent Systems: Delays, Adaptivity, and Learning in Games
Yu-Guan Hsieh
Ph.D. Dissertation, 2023
Manuscript Slides
Thompson Sampling with Diffusion Generative Prior
Yu-Guan Hsieh, Shiva Kasiviswanathan, Branislav Kveton, and Patrick Blöbaum
International Conference on Machine Learning (ICML), 2023
Paper Arxiv Poster Slides
Push--Pull with Device Sampling
Yu-Guan Hsieh, Yassine Laguel, Franck Iutzeler, and Jérôme Malick
IEEE Transactions on Automatic Control (TACON), 2023
Paper Arxiv
No-Regret Learning in Games with Noisy Feedback: Faster Rates and Adaptivity via Learning Rate Separation
Yu-Guan Hsieh, Kimon Antonakopoulos, Volkan Cevher, and Panayotis Mertikopoulos
Neural Information Processing Systems (NeurIPS), 2022
Paper Arxiv Poster Slides
Uplifting Bandits
Yu-Guan Hsieh, Shiva Prasad Kasiviswanathan, and Branislav Kveton
Neural Information Processing Systems (NeurIPS), 2022
Paper Arxiv Poster Short slides Long slides
Multi-agent Online Optimization with Delays: Asynchronicity, Adaptivity, and Optimism
Yu-Guan Hsieh, Franck Iutzeler, Jérôme Malick, and Panayotis Mertikopoulos
Journal of Machine Learning Research (JMLR), 2022
Paper Arxiv Poster
Optimization in Open Networks via Dual Averaging
Yu-Guan Hsieh, Franck Iutzeler, Jérôme Malick, and Panayotis Mertikopoulos
IEEE Conference on Decision and Control (CDC), 2021
Paper Arxiv Slides
Adaptive Learning in Continuous Games: Optimal Regret Bounds and Convergence to Nash Equilibrium
Yu-Guan Hsieh, Kimon Antonakopoulos, and Panayotis Mertikopoulos
Conference on Learning Theory (COLT), 2021
Paper Arxiv Poster Slides Video
Explore Aggressively, Update Conservatively: Stochastic Extragradient Methods with Variable Stepsize Scaling
Yu-Guan Hsieh, Franck Iutzeler, Jérôme Malick, and Panayotis Mertikopoulos
Neural Information Processing Systems (NeurIPS), 2020
Paper Arxiv Poster Slides Video Code
On the Convergence of Single-Call Stochastic Extra-Gradient Methods
Yu-Guan Hsieh, Franck Iutzeler, Jérôme Malick, and Panayotis Mertikopoulos
Neural Information Processing Systems (NeurIPS), 2019
Paper Arxiv Poster Slides
Classification from Positive, Unlabeled and Biased Negative Data
Yu-Guan Hsieh, Gang Niu, and Masashi Sugiyama
International Conference on Machine Learning (ICML), 2019
Paper Arxiv Poster Slides Code

Experiences

  • 2025 January ~ Now

    Research Scientist at Spellbrush, Tokyo, Japan

    I started my research career in Tokyo, and I am now going back to Tokyo for another turning point in my life. AI is revolutionizing everything, creating an incredible opportunity to truly build our dreams. Some of my deepest passions lie in anime and mathematics—two seemingly unrelated fields—but it is now possible to bring them together. We are still early, and I am excited to see where this new frontier will give.

  • 2024

    Postdoc at Apple MLR, Paris, France

    I returned to Paris after four years of my Ph.D. pursuit, and this came with a refocusing of my research interests. Having witnessed how large foundation models are transforming the world that we live in, I made up my mind to work on problems that are more directly relevant to these models. Specifically, I conducted research on vision-language models, exploring how a new image captioning format, graph-based captioning, could serve as a strong alternative to traditional plain text captions when it comes to providing fine-grained and structured annotations for images.

  • 2023 November 7th

    Ph.D. Defense

    After four years of hard work, the Ph.D. defense marked an import milestone of my academic journey. My Ph.D. Thesis, honored with the Academic Thesis Prize, focuses on mathematical frameworks and algorithms for decision-making in multi-agent systems, making use of tools from online learning, game theory, and stochastic optimization. Along the way, I gained invaluable experiences at workshops in Luminy, Les Houches, Singapore, and a lot more. I am deeply grateful for my Ph.D. advisors for their support and guidance during these years.

  • 2022 August ~ November

    Internship at Amazon in Santa Clara, USA

    I was fortunate to get a return internship at Amazon in the same team but at a different location. Impressed by the performance and flexibility of score-based diffusion models, I decided to investigate how it can be incorporated as prior in multi-armed bandit problems. This internship was thus an occasion for me to conduct a quite different type of research, in which algorithms and experiments prevail theory. I also enjoyed the four months in the Bay Area, where I got the opportunity to meet many old friends and gained new insignts into my future career.

  • 2021 Oct. ~ 2022 Jan.

    Internship at Amazon in Tübingen, Germany

    To acquire industrial experience, I did a four-month applied science internship at Amazon in the third year of my Ph.D. During this internship, I had the great honor to work with my internship manager Shiva Kasiviswanathan and Principal Scientist Branislav Kveton on a stochastic bandit model (see this paper). Additionally, being part of the Causality team, led by Dominik Janzing and Yasser Jadidi, allowed me to gain an understanding of the basics of causality. Finally, the internship provided me with valuable experience in remote collaboration, as well as an enjoyable experience living in the beautiful medieval city of Tübingen, surrounded by nature.

  • 2019

    • Master Mathematics, Vision, Learning
    • Move to Grenoble and start Ph.D.

    For my second year of master, I studied Mathematics, Vision, and Learning in ENS Paris-Saclay. I then came to Grenoble for the internship, before starting my Ph.D. with my current advisors in this “Capital of the Alps”.

  • 2018 March ~ August

    Internship at Riken AIP, Tokyo, Japan

    As a fan of Japanese Culture, I decided to go to Japan for my first-year master’s internship. Thanks to my kind internship advisors (Masashi Sugiyama and Gang Niu) and co-workers, this internship marked the beginning of my research career. My work centered around weakly-supervised learning and my first paper was written. I also enjoyed very much the time off from work. I benefited from these five months so much that I dare say that it was probably my greatest turning point after my arrival in France.

  • ENS Paris

    2016 September

    Enter ENS Paris

    After two intensive years in preparatory class, I was admitted to ENS Paris, being ranked 1bis of the computer science group in the entrance exam (bis as I was not French). Only during the first and half years we actually followed courses on the campus of ENS. Besides math and computer science, I also attended several cognitive science courses (which granted me a minor in cognitive science).

  • Moving to France

    2014 July

    Move to Lyon, France

    The randomness of life brought me to France after I graduated from high school. I got this opportunity thanks to the CPGE Taiwan program which recruit Taiwanese students to study in French classe prepa through a maths exam. After 6 months of intensive French course in Taiwan, in July 2014, I embarked on the journey and started my study abroad life in France.

  • IMO

    2013 July

    IMO silver medal in Colombia

    My passion for mathematics was already developed in my childhood. From 12 to 18 years old, I actively participated in math competitions and spent a lot of time on Olympiad type questions. Especially, during my second year of high school, I was so fortunate to become one of the six contestants representing Taiwan in IMO 2013 after passing multiple stages of selection.

  • 1996

    Born in Taipei, Taiwan

    I was born in Taiwan, this beautiful island on which both western democracy and eastern lifestyle can be found. Yes, that is surprisingly rare. Moreover, Taiwan is also considered as one of the best places for expats living abroad and is the first in Asia that legalizes same-sex marriage. I am proud of being Taiwanese.

  • Download
    My CV