About

SciFM27

Past Conferences

#SciFM24

Scientific Foundation Models

April 2nd & 3rd, 2024

Rackham Amphitheatre, 915 E Washington St, Ann Arbor, MI

About

SciFM 2024

The 2024 MICDE Conference marks a significant moment in the evolution of scientific inquiry and exploration. This conference will focus on scientific foundation models (SciFM) that aim to have a similar transformative impact on science as Generative AI has had on natural language. SciFM are parameterized physical theories that are usually trained on a broad range of scientific data and capable of being applied to a range of downstream tasks, such as discovering patterns and generating scientific hypotheses, insights, and engineering designs.

This event is the first of its kind, dedicated exclusively to this exciting and nascent field. By congregating the world’s foremost experts in the field, the conference aims to significantly broaden the horizons of scientific foundation models and Generative AI (including LLMs) for science.

Invited

Speakers & Panelists

Jason Pruet

Director of National Security AI, Los Alamos National Laboratory

Ian T. Foster

Director of Data Science and Learning Division, Argonne National Laboratory

Animashree Anandkumar

Bren Professor of Computing and Mathematical Sciences, California Institute of Technology

John Wei

Ph.D., MBA, Investment Director, Applied Ventures, LLC

Heng Ji

Professor of Computer Science, University of Illinois at Urbana Champaign

Petros Koumoutsakos

Herbert S. Winokur, Jr. Professor of Computing in Science and Engineering, Harvard University

Venkat Viswanathan

Associate Professor of Aerospace Engineering, University of Michigan

Sean Welleck

Assistant Professor of Computer Science at Carnegie Mellon University

Alvaro Velasquez

Program Manager, DARPA Information Innovation Office (I2O)

Arvind Ramanathan

Computational Biologist in the Data Science and Learning Division, Argonne National Laboratory & Senior Scientist, University of Chicago Consortium for Advanced Science and Engineering

Michael Mahoney

Professor of Statistics, UC Berkeley & Leader of the Machine Learning and Analytics Group, Lawrence Berkeley National Laboratory & Vice President and Director of Big Data Group, International Computer Science Institute

Payel Das

Research Staff Member and Manager, AI Science, IBM T. J. Watson Research Center

Jonathan Carter

Associate Laboratory Director, Computing Sciences, Lawrence Berkeley National Laboratory

Rajesh Swaminathan

Partner, Khosla Ventures

Jean-Luc Cambier

Director of Research Programs, Office of the Secretary of Defense.

Alfred Hero

Program Director, Computing and Communication Foundations, National Science Foundation

SciFM24

Panel Discussions

Big Questions for SciFM:

I. T. Foster (ANL, UChigago),
A. Hero (NSF),
J. Carter (LBNL),
P. Koumoutsakos, (Harvard),
P. Das, (IBM),
H. Ji, (UIUC),
M. W. Mahoney (UC Berkeley, LBNL, ICSI)

Funding and Venture Ecosystem:

J-L Cambier (OUSD),
J. Pruet (LANL),
A. Velasquez (DARPA),
J. Carter (LBNL), R
R. Swaminathan (Khosla),
A. Hero (NSF),
J. Wei (Applied Ventures)

Tutorials & Hackathon

The tutorials are designed to introduce attendees on Generative AI & Large Language Models (LLMs) for science and mathematics. This will be co-organized with Trillion Parameter Consortium (TPC), Prof. Sean Welleck (CMU), and NVIDIA.

The tutorials and hackathon will be divided into three sessions:

Hands-on Introduction to Scientific LLMs [TPC]

This will be a tutorial on the basics of LLMs and Generative AI with a focus on science and engineering.

LLM Tools for Science [TPC] at Rackham Amphitheatre

Retrieval augmented generation
Agent-based systems
Building toolchains and surrogates
Genome Scale language models
Scaling experiments on Supercomputers

Details:

Brian Hsu, Priyanka Setty will talk about the use of agents and agent-based systems for experimental protocol generation. This talk will mostly cover the ability to develop a workflow that outlines “given a general description of a protocol, is it possible to generate code automatically to execute and run on a robotics system of interest?” This will be related to a simple growth curve modeling or pipetting example problem in the lab for designing novel peptides targeting antibiotic resistant bacteria.
Carla Mann will illustrate an example of building on top of ESM like models for protein protein interactions.
Archit Vasan will talk about the ability to develop surrogate models for virtual screening protocols and present a simple approach towards this using transformers.
Alex Brace, Kyle Hippe, Azton Wells will talk about Genome Scale language models and their applications downstream.
Carlo Siebenschuh: will present approaches other than transformers for scientific data with applications such as neural operators and latent diffusion models.
Ozan Gokdemir will illustrate retrieval augmented generation in the context of scientific literature.
Azton Wells, Archit Vasan will provide an overview of scaling and what it means to set up scaling experiments on supercomputing platforms.

LLM Tools for Mathematics [Sean Welleck] at East Conference Room

Open Language for Mathematics
Neural Theorem Proving

Details:

Interactive proof assistants enable the verification of mathematics and software using specialized programming languages. The emerging area of neural theorem proving integrates neural language models with interactive proof assistants. Doing so is mutually beneficial: proof assistants provide correctness guarantees on language model outputs, while language models help make proof assistants easier to use. This talk overviews two research threads in neural theorem proving, motivated by applications to mathematics and software verification. First, we discuss language models that predict the next step of a proof, centered around recent work that integrates language models into an interactive proof development tool. Second, we discuss techniques that leverageinformalmathematical data for formal theorem proving. The talk accompanies a set of interactive notebooks available on Github.

Tutorial facilitators:

[TPC]: Arvind Ramanathan, Staff Scientist at Argonne National Laboratory, Sean Welleck, Assistant Professor of Computer Science at Carnegie Mellon University, and the following students: Ozan Gokdemir, Priyanka Setty, Archit Vasan, Kyle Hippe, Carla M. Mann, and Azton Wells.

[NVIDIA]: Geetika Gupta, principal product manager at NVIDIA, and Yuliana Zamora