Topic: Accuracy, Uncertainty, Inspectability: The benefits of compositionally-restricted attention-based networks.
Speaker: Dr. Taylor Sparks from the University of Utah
We’re super delighted to announce our next speaker, Dr. Taylor Sparks from University of Utah, to bring us:
“Accuracy, Uncertainty, Inspectability: The benefits of compositionally-restricted attention-based networks.”
Abstract
We describe a new model architecture, the Compositionally-Restricted Attention-Based Network (CrabNet). CrabNet generates high-fidelity predictions based on the self-attention mechanism, a fundamental component of the transformer architecture that revolutionized natural language processing. The transformer encoder uses self-attention to encode the context-dependent behavior for the components within a system. In physical environments, elements contribute differently to a material’s property based on the materials system itself. For example, boron behaving as an electrical dopant in one system while behaving as a mechanical strengthening bond modification in another. CrabNet’s ability to potentially capture this type of context-dependent behavior allows for highly accurate model predictions. Importantly, CrabNet generates simple and inspectable self-attention maps. These attention maps govern the learned material property by representing element importance and interactions. The visualization and analysis of these attention maps are available during training and inference periods.
Biography
Dr. Sparks is an Associate Professor and Associate Chair of the Materials Science and Engineering Department at the University of Utah. He is originally from Utah and an alumni of the department he now teaches in. Before graduate school he worked at Ceramatec Inc. He did his MS in Materials at UCSB and his PhD in Applied Physics at Harvard University in David Clarke’s laboratory and then did a postdoc with Ram Seshadri in the Materials Research Laboratory at UCSB. His current research centers on the discovery, synthesis, characterization, and properties of new materials for energy applications. He is a pioneer in the emerging field of materials informatics whereby big data, data mining, and machine learning are leveraged to solve challenges in materials science. He also hosts a podcast entitled “Materialism” where he discusses the past, present, and future of Materials Science.
Very much looking forward to the seminar – hope to see you all on Wednesday: (SGT) 9am, Wed, 12 May! https://mit.zoom.us/j/96231985116
Warmest regards, @Siyu Tian (Isaac) @kedar @Tonio
Comments