Natural language processing

Subterms

    Latest story

    100 Shares99 Views

    Designing zeolites, porous materials made to trap molecules

    by

    Zeolites are a class of minerals used in everything from industrial catalysts and chemical filters to laundry detergents and cat litter. They are mostly composed of silicon and aluminum — two abundant, inexpensive elements — plus oxygen; they have a crystalline structure; and most significantly, they are porous. Among the regularly repeating atomic patterns in them are tiny interconnected openings, or pores, that can trap molecules that just fit inside them, allow smaller ones to pass through, or block larger ones from entering. A zeolite can remove unwanted molecules from gases and liquids, or trap them temporarily and then release them, or hold them while they undergo rapid chemical reactions.

    Some zeolites occur naturally, but they take unpredictable forms and have variable-sized pores. “People synthesize artificial versions to ensure absolute purity and consistency,” says Rafael Gómez-Bombarelli, the Jeffrey Cheah Career Development Chair in Engineering in the Department of Materials Science and Engineering (DMSE). And they work hard to influence the size of the internal pores in hopes of matching the molecule or other particle they’re looking to capture.

    The basic recipe for making zeolites sounds simple. Mix together the raw ingredients — basically, silicon dioxide and aluminum oxide — and put them in a reactor for a few days at a high temperature and pressure. Depending on the ratio between the ingredients and the temperature, pressure, and timing, as the initial gel slowly solidifies into crystalline form, different zeolites emerge.

    But there’s one special ingredient to add “to help the system go where you want it to go,” says Gómez-Bombarelli. “It’s a molecule that serves as a template so that the zeolite you want will crystallize around it and create pores of the desired size and shape.”

    The so-called templating molecule binds to the material before it solidifies. As crystallization progresses, the molecule directs the structure, or “framework,” that forms around it. After crystallization, the temperature is raised and the templating molecule burns off, leaving behind a solid aluminosilicate material filled with open pores that are — given the correct templating molecule and synthesis conditions — just the right size and shape to recognize the targeted molecule.

    The zeolite conundrum

    Theoretical studies suggest that there should be hundreds of thousands of possible zeolites. But despite some 60 years of intensive research, only about 250 zeolites have been made. This is sometimes called the “zeolite conundrum.” Why haven’t more been made — especially now, when they could help ongoing efforts to decarbonize energy and the chemical industry?

    One challenge is figuring out the best recipe for making them: Factors such as the best ratio between the silicon and aluminum, what cooking temperature to use, and whether to stir the ingredients all influence the outcome. But the real key, the researchers say, lies in choosing a templating molecule that’s best for producing the intended zeolite framework. Making that match is difficult: There are hundreds of known templating molecules and potentially a million zeolites, and researchers are continually designing new molecules because millions more could be made and might work better.

    For decades, the exploration of how to synthesize a particular zeolite has been done largely by trial and error — a time-consuming, expensive, inefficient way to go about it. There has also been considerable effort to use “atomistic” (atom-by-atom) simulation to figure out what known or novel templating molecule to use to produce a given zeolite. But the experimental and modeling results haven’t generated reliable guidance. In many cases, researchers have carefully selected or designed a molecule to make a particular zeolite, but when they tried their molecule in the lab, the zeolite that formed wasn’t what they expected or desired. So they needed to start over.

    Those experiences illustrate what Gómez-Bombarelli and his colleagues believe is the problem that’s been plaguing zeolite design for decades. All the efforts — both experimental and theoretical — have focused on finding the templating molecule that’s best for forming a specific zeolite. But what if that templating molecule is also really good — or even better — at forming some other zeolite?

    To determine the “best” molecule for making a certain zeolite framework, and the “best” zeolite framework to act as host to a particular molecule, the researchers decided to look at both sides of the pairing. Daniel Schwalbe-Koda PhD ’22, a former member of Gómez-Bombarelli’s group and now a postdoc at Lawrence Livermore National Laboratory, describes the process as a sort of dance with molecules and zeolites in a room looking for partners. “Each molecule wants to find a partner zeolite, and each zeolite wants to find a partner molecule,” he says. “But it’s not enough to find a good dance partner from the perspective of only one dancer. The potential partner could prefer to dance with someone else, after all. So it needs to be a particularly good pairing.” The upshot: “You need to look from the perspective of each of them.”

    To find the best match from both perspectives, the researchers needed to try every molecule with every zeolite and quantify how well the pairings worked.

    A broader metric for evaluating pairs

    Before performing that analysis, the researchers defined a new “evaluating metric” that they could use to rank each templating molecule-zeolite pair. The standard metric for measuring the affinity between a molecule and a zeolite is “binding energy,” that is, how strongly the molecule clings to the zeolite or, conversely, how much energy is required to separate the two. While recognizing the value of that metric, the MIT-led team wanted to take more parameters into account.

    Their new evaluating metric therefore includes not only binding energy but also the size, shape, and volume of the molecule and the opening in the zeolite framework. And their approach calls for turning the molecule to different orientations to find the best possible fit.

    Affinity scores for all molecule-zeolite pairs based on that evaluating metric would enable zeolite researchers to answer two key questions: What templating molecule will form the zeolite that I want? And if I use that templating molecule, what other zeolites might it form instead? Using the molecule-zeolite affinity scores, researchers could first identify molecules that look good for making a desired zeolite. They could then rule out the ones that also look good for forming other zeolites, leaving a set of molecules deemed to be “highly selective” for making the desired zeolite.  

    Validating the approach: A rich literature

    But does their new metric work better than the standard one? To find out, the team needed to perform atomistic simulations using their new evaluating metric and then benchmark their results against experimental evidence reported in the literature. There are many thousands of journal articles reporting on experiments involving zeolites — in many cases, detailing not only the molecule-zeolite pairs and outcomes but also synthesis conditions and other details. Ferreting out articles with the information the researchers needed was a job for machine learning — in particular, for natural language processing.

    For that task, Gómez-Bombarelli and Schwalbe-Koda turned to their DMSE colleague Elsa Olivetti PhD ’07, the Esther and Harold E. Edgerton Associate Professor in Materials Science and Engineering. Using a literature-mining technique that she and a group of collaborators had developed, she and her DMSE team processed more than 2 million materials science papers, found some 90,000 relating to zeolites, and extracted 1,338 of them for further analysis. The yield was 549 templating molecules tested, 209 zeolite frameworks produced, and 5,663 synthesis routes followed.

    Based on those findings, the researchers used their new evaluating metric and a novel atomistic simulation technique to examine more than half-a-million templating molecule-zeolite pairs. Their results reproduced experimental outcomes reported in more than a thousand journal articles. Indeed, the new metric outperformed the traditional binding energy metric, and their simulations were orders of magnitude faster than traditional approaches.

    Ready for experimental investigations

    Now the researchers were ready to put their approach to the test: They would use it to design new templating molecules and try them out in experiments performed by a team led by Yuriy Román-Leshkov, the Robert T. Haslam (1911) Professor of Chemical Engineering, and a team from the Instituto de Tecnologia Química in Valencia, Spain, led by Manuel Moliner and Avelino Corma.

    One set of experiments focused on a zeolite called chabazite, which is used in catalytic converters for vehicles. Using their techniques, the researchers designed a new templating molecule for synthesizing chabazite, and the experimental results confirmed their approach. Their analyses had shown that the new templating molecule would be good for forming chabazite and not for forming anything else. “Its binding strength isn’t as high as other molecules for chabazite, so people hadn’t used it,” says Gómez-Bombarelli. “But it’s pretty good, and it’s not good for anything else, so it’s selective — and it’s way cheaper than the usual ones.”

    In addition, in their new molecule, the electrical charge is distributed differently than in the traditional ones, which led to new possibilities. The researchers found that by adjusting both the shape and charge of the molecule, they could control where the negative charge occurs on the pore that’s created in the final zeolite. “The charge placement that results can make the chabazite a much better catalyst than it was before,” says Gómez-Bombarelli. “So our same rules for molecule design also determine where the negative charge is going to end up, which can lead to whole different classes of catalysts.”

    Schwalbe-Koda describes another experiment that demonstrates the importance of molecular shape as well as the types of new materials made possible using the team’s approach. In one striking example, the team designed a templating molecule with a height and width that’s halfway between those of two molecules that are now commonly used—one for making chabazite and the other for making a zeolite called AEI. (Every new zeolite structure is examined by the International Zeolite Association and — once approved — receives a three-letter designation.)

    Experiments using that in-between templating molecule resulted in the formation of not one zeolite or the other, but a combination of the two in a single solid. “The result blends two different structures together in a way that the final result is better than the sum of its parts,” says Schwalbe-Koda. “The catalyst is like the one used in catalytic converters in today’s trucks — only better.” It’s more efficient in converting nitrogen oxides to harmless nitrogen gases and water, and — because of the two different pore sizes and the aluminosilicate composition — it works well on exhaust that’s fairly hot, as during normal operation, and also on exhaust that’s fairly cool, as during startup.

    Putting the work into practice

    As with all materials, the commercial viability of a zeolite will depend in part on the cost of making it. The researchers’ technique can identify promising templating molecules, but some of them may be difficult to synthesize in the lab. As a result, the overall cost of that molecule-zeolite combination may be too high to be competitive.

    Gómez-Bombarelli and his team therefore include in their assessment process a calculation of cost for synthesizing each templating molecule they identified — generally the most expensive part of making a given zeolite. They use a publicly available model devised in 2018 by Connor Coley PhD ’19, now the Henri Slezynger (1957) Career Development Assistant Professor of Chemical Engineering at MIT. The model takes into account all the starting materials and the step-by-step chemical reactions needed to produce the targeted templating molecule.

    However, commercialization decisions aren’t based solely on cost. Sometimes there’s a trade-off between cost and performance. “For instance, given our chabazite findings, would customers or the community trade a little bit of activity for a 100-fold decrease in the cost of the templating molecule?” says Gómez-Bombarelli. “The answer is likely yes. So we’ve made a tool that can help them navigate that trade-off.” And there are other factors to consider. For example, is this templating molecule truly novel, or have others already studied it — or perhaps even hold a patent on it?

    “While an algorithm can guide development of templating molecules and quantify specific molecule-zeolite matches, other types of assessments are best left to expert judgment,” notes Schwalbe-Koda. “We need a partnership between computational analysis and human intuition and experience.”

    To that end, the MIT researchers and their colleagues decided to share their techniques and findings with other zeolite researchers. Led by Schwalbe-Koda, they created an online database that they made publicly accessible and easy to use — an unusual step, given the competitive industries that rely on zeolites. The interactive website — zeodb.mit.edu — contains the researchers’ final metrics for templating molecule-zeolite pairs resulting from hundreds of thousands of simulations; all the identified journal articles, along with which molecules and zeolites were examined and what synthesis conditions were used; and many more details. Users are free to search and organize the data in any way that suits them.

    Gómez-Bombarelli, Schwalbe-Koda, and their colleagues hope that their techniques and the interactive website will help other researchers explore and discover promising new templating molecules and zeolites, some of which could have profound impacts on efforts to decarbonize energy and tackle climate change.

    This research involved a team of collaborators at MIT, the Instituto de Tecnologia Química (UPV-CSIC), and Stockholm University. The work was supported in part by the MIT Energy Initiative Seed Fund Program and by seed funds from the MIT International Science and Technology Initiative. Daniel Schwalbe-Koda was supported by an ExxonMobil-MIT Energy Fellowship in 2020–21.

    This article appears in the Spring 2022 issue of Energy Futures, the magazine of the MIT Energy Initiative. More

    More stories

    • 75 Shares119 Views

      in Environment

      Q&A: More-sustainable concrete with machine learning

      by

      As a building material, concrete withstands the test of time. Its use dates back to early civilizations, and today it is the most popular composite choice in the world. However, it’s not without its faults. Production of its key ingredient, cement, contributes 8-9 percent of the global anthropogenic CO2 emissions and 2-3 percent of energy consumption, which is only projected to increase in the coming years. With aging United States infrastructure, the federal government recently passed a milestone bill to revitalize and upgrade it, along with a push to reduce greenhouse gas emissions where possible, putting concrete in the crosshairs for modernization, too.

      Elsa Olivetti, the Esther and Harold E. Edgerton Associate Professor in the MIT Department of Materials Science and Engineering, and Jie Chen, MIT-IBM Watson AI Lab research scientist and manager, think artificial intelligence can help meet this need by designing and formulating new, more sustainable concrete mixtures, with lower costs and carbon dioxide emissions, while improving material performance and reusing manufacturing byproducts in the material itself. Olivetti’s research improves environmental and economic sustainability of materials, and Chen develops and optimizes machine learning and computational techniques, which he can apply to materials reformulation. Olivetti and Chen, along with their collaborators, have recently teamed up for an MIT-IBM Watson AI Lab project to make concrete more sustainable for the benefit of society, the climate, and the economy.

      Q: What applications does concrete have, and what properties make it a preferred building material?

      Olivetti: Concrete is the dominant building material globally with an annual consumption of 30 billion metric tons. That is over 20 times the next most produced material, steel, and the scale of its use leads to considerable environmental impact, approximately 5-8 percent of global greenhouse gas (GHG) emissions. It can be made locally, has a broad range of structural applications, and is cost-effective. Concrete is a mixture of fine and coarse aggregate, water, cement binder (the glue), and other additives.

      Q: Why isn’t it sustainable, and what research problems are you trying to tackle with this project?

      Olivetti: The community is working on several ways to reduce the impact of this material, including alternative fuels use for heating the cement mixture, increasing energy and materials efficiency and carbon sequestration at production facilities, but one important opportunity is to develop an alternative to the cement binder.

      While cement is 10 percent of the concrete mass, it accounts for 80 percent of the GHG footprint. This impact is derived from the fuel burned to heat and run the chemical reaction required in manufacturing, but also the chemical reaction itself releases CO2 from the calcination of limestone. Therefore, partially replacing the input ingredients to cement (traditionally ordinary Portland cement or OPC) with alternative materials from waste and byproducts can reduce the GHG footprint. But use of these alternatives is not inherently more sustainable because wastes might have to travel long distances, which adds to fuel emissions and cost, or might require pretreatment processes. The optimal way to make use of these alternate materials will be situation-dependent. But because of the vast scale, we also need solutions that account for the huge volumes of concrete needed. This project is trying to develop novel concrete mixtures that will decrease the GHG impact of the cement and concrete, moving away from the trial-and-error processes towards those that are more predictive.

      Chen: If we want to fight climate change and make our environment better, are there alternative ingredients or a reformulation we could use so that less greenhouse gas is emitted? We hope that through this project using machine learning we’ll be able to find a good answer.

      Q: Why is this problem important to address now, at this point in history?

      Olivetti: There is urgent need to address greenhouse gas emissions as aggressively as possible, and the road to doing so isn’t necessarily straightforward for all areas of industry. For transportation and electricity generation, there are paths that have been identified to decarbonize those sectors. We need to move much more aggressively to achieve those in the time needed; further, the technological approaches to achieve that are more clear. However, for tough-to-decarbonize sectors, such as industrial materials production, the pathways to decarbonization are not as mapped out.

      Q: How are you planning to address this problem to produce better concrete?

      Olivetti: The goal is to predict mixtures that will both meet performance criteria, such as strength and durability, with those that also balance economic and environmental impact. A key to this is to use industrial wastes in blended cements and concretes. To do this, we need to understand the glass and mineral reactivity of constituent materials. This reactivity not only determines the limit of the possible use in cement systems but also controls concrete processing, and the development of strength and pore structure, which ultimately control concrete durability and life-cycle CO2 emissions.

      Chen: We investigate using waste materials to replace part of the cement component. This is something that we’ve hypothesized would be more sustainable and economic — actually waste materials are common, and they cost less. Because of the reduction in the use of cement, the final concrete product would be responsible for much less carbon dioxide production. Figuring out the right concrete mixture proportion that makes endurable concretes while achieving other goals is a very challenging problem. Machine learning is giving us an opportunity to explore the advancement of predictive modeling, uncertainty quantification, and optimization to solve the issue. What we are doing is exploring options using deep learning as well as multi-objective optimization techniques to find an answer. These efforts are now more feasible to carry out, and they will produce results with reliability estimates that we need to understand what makes a good concrete.

      Q: What kinds of AI and computational techniques are you employing for this?

      Olivetti: We use AI techniques to collect data on individual concrete ingredients, mix proportions, and concrete performance from the literature through natural language processing. We also add data obtained from industry and/or high throughput atomistic modeling and experiments to optimize the design of concrete mixtures. Then we use this information to develop insight into the reactivity of possible waste and byproduct materials as alternatives to cement materials for low-CO2 concrete. By incorporating generic information on concrete ingredients, the resulting concrete performance predictors are expected to be more reliable and transformative than existing AI models.

      Chen: The final objective is to figure out what constituents, and how much of each, to put into the recipe for producing the concrete that optimizes the various factors: strength, cost, environmental impact, performance, etc. For each of the objectives, we need certain models: We need a model to predict the performance of the concrete (like, how long does it last and how much weight does it sustain?), a model to estimate the cost, and a model to estimate how much carbon dioxide is generated. We will need to build these models by using data from literature, from industry, and from lab experiments.

      We are exploring Gaussian process models to predict the concrete strength, going forward into days and weeks. This model can give us an uncertainty estimate of the prediction as well. Such a model needs specification of parameters, for which we will use another model to calculate. At the same time, we also explore neural network models because we can inject domain knowledge from human experience into them. Some models are as simple as multi-layer perceptions, while some are more complex, like graph neural networks. The goal here is that we want to have a model that is not only accurate but also robust — the input data is noisy, and the model must embrace the noise, so that its prediction is still accurate and reliable for the multi-objective optimization.

      Once we have built models that we are confident with, we will inject their predictions and uncertainty estimates into the optimization of multiple objectives, under constraints and under uncertainties.

      Q: How do you balance cost-benefit trade-offs?

      Chen: The multiple objectives we consider are not necessarily consistent, and sometimes they are at odds with each other. The goal is to identify scenarios where the values for our objectives cannot be further pushed simultaneously without compromising one or a few. For example, if you want to further reduce the cost, you probably have to suffer the performance or suffer the environmental impact. Eventually, we will give the results to policymakers and they will look into the results and weigh the options. For example, they may be able to tolerate a slightly higher cost under a significant reduction in greenhouse gas. Alternatively, if the cost varies little but the concrete performance changes drastically, say, doubles or triples, then this is definitely a favorable outcome.

      Q: What kinds of challenges do you face in this work?

      Chen: The data we get either from industry or from literature are very noisy; the concrete measurements can vary a lot, depending on where and when they are taken. There are also substantial missing data when we integrate them from different sources, so, we need to spend a lot of effort to organize and make the data usable for building and training machine learning models. We also explore imputation techniques that substitute missing features, as well as models that tolerate missing features, in our predictive modeling and uncertainty estimate.

      Q: What do you hope to achieve through this work?

      Chen: In the end, we are suggesting either one or a few concrete recipes, or a continuum of recipes, to manufacturers and policymakers. We hope that this will provide invaluable information for both the construction industry and for the effort of protecting our beloved Earth.

      Olivetti: We’d like to develop a robust way to design cements that make use of waste materials to lower their CO2 footprint. Nobody is trying to make waste, so we can’t rely on one stream as a feedstock if we want this to be massively scalable. We have to be flexible and robust to shift with feedstocks changes, and for that we need improved understanding. Our approach to develop local, dynamic, and flexible alternatives is to learn what makes these wastes reactive, so we know how to optimize their use and do so as broadly as possible. We do that through predictive model development through software we have developed in my group to automatically extract data from literature on over 5 million texts and patents on various topics. We link this to the creative capabilities of our IBM collaborators to design methods that predict the final impact of new cements. If we are successful, we can lower the emissions of this ubiquitous material and play our part in achieving carbon emissions mitigation goals.

      Other researchers involved with this project include Stefanie Jegelka, the X-Window Consortium Career Development Associate Professor in the MIT Department of Electrical Engineering and Computer Science; Richard Goodwin, IBM principal researcher; Soumya Ghosh, MIT-IBM Watson AI Lab research staff member; and Kristen Severson, former research staff member. Collaborators included Nghia Hoang, former research staff member with MIT-IBM Watson AI Lab and IBM Research; and Jeremy Gregory, research scientist in the MIT Department of Civil and Environmental Engineering and executive director of the MIT Concrete Sustainability Hub.

      This research is supported by the MIT-IBM Watson AI Lab. More