Semiempirical modeling is about carefully chosen compromises between cost and accuracy, and the results can get a little ugly. While the elements of my design all seem reasonable enough on paper, there is still the very real possibility that a fully fleshed out implementation wouldn’t be accurate enough or efficient enough to have practical value. For a research project with little career value (because no one will pay me to work on it) and little academic value (because very few scientists are interested in the topic anymore), this makes it even easier to walk away from. Sometimes it is best to let a research area lie fallow until the technical and financial conditions are better. On the technical side, future advances in sparse numerical linear algebra could have a major impact on cost/accuracy balances and really shift around the design landscape and enable fewer compromises. On the financial side, semiempirical models might get another chance from funding agencies when machine learning models fail to be transferrable enough to take their place, which I consider to be inevitable.

Before I move on to the details of my technical plan, I should note that I consider the biggest problem of semiempirical modeling to be a branding problem rather than any technical problem. The effort to build general-purpose, transferrable semiempirical models, from its origins in the 1960’s, was always seen as a stop-gap measure before computers were powerful enough for Gaussian-based quantum chemistry to become routine. While semiempirical models were eventually valued in their own right, density functional theory (DFT) became the preferred approach to semiempirical modeling in the 1990’s because it was perceived as more systematically improvable and co-opted most of the machinery of Gaussian-based quantum chemistry, despite being several orders of magnitude more expensive than the more traditional models. The cheaper, traditional semiempirical models could have tried to more strongly embrace (and further reduce) their low cost and distinguish themselves in that way, but it just never happened. There was definitely an attempt at this in the linear-scaling electronic structure literature of the 1990’s, but those methods were not as reliable as hoped, and their limited range of applicability eroded the transferability of the underlying models. This disconnect is most clear in the recent developments of the most popular semiempirical software, MOPAC. Its newest model, PM7, further improved on transferability relative to older models, but this came at the expense of some accuracy for organic molecules, which is problematic for organic chemists. In contrast, the linear-scaling solver of MOPAC, MOZYME, only functions for organic molecules with identifiable Lewis dot structures. Thus, the efforts of MOPAC to become more transferrable and more efficient took it in different, somewhat conflicting directions. A future branding strategy that I envision for semiempirical modeling is to categorize atomistic simulation methodologies by three attributes - efficiency, accuracy, and transferability. The two most popular methodologies - classical interatomic potentials and first-principles electronic structure - are both very focused on accuracy, but interatomic potentials must sacrifice transferability to build their extremely efficient models, and first-principles calculations must sacrifice efficiency to attain accuracy from adherence to the underlying many-body Schrodinger equation (or some Kohn-Sham DFT proxy). In this worldview, you have to pick two out of these three attributes, and the traditional approaches to semiempirical modeling are unique in their emphasis of efficiency and transferability at the expense of accuracy. However, no one has ever made a successful popular case for this sort of worldview, much to the detriment of semiempirical modeling.

In the rest of this post, I will discuss the main components of my (now defunct) technical plan for a new semiempirical model. The goal of this plan was to design a model entirely around low-cost linear-scaling solvers and to emphasize efficiency as much as possible in all design choices. Accuracy improvements are only sought when they don’t conflict with (or even enable) cost reductions.

Atom-centric state variables

The first design choice, which has implications for all the other design choices, is the choice of state variables. Given just a set of atomic coordinates, the nonlinear SCF equations have to be solved to determine a ground state, and the successful identification of the global energy minimum cannot be guaranteed. This global convergence issue can be sidestepped if the nonlinear state variables are provided, which reduces the standard SCF equations to a linear eigenvalue problem. The number of state variables can vary quite a bit - MOPAC’s models contain Fock exchange, thus its state variables are the full single-electron density matrix, while density-functional tight-binding (DFTB) models only contain a monopole approximation to a Hartree potential, thus their state variables are the total charge on each atom. Consistent with the widespread empirical success of screened-exchange models in DFT, I would retain the on-site density matrices to construct on-site Fock exchange terms (assuming that screening effects suppress inter-site Fock exchange). I would allow for diatomic contributions to the charge density in defining the Hartree potential, but this can be reduced to an atom-centric state variable by using the Hartree potential rather than the charge density as the state variable and expanding the smooth part of the Hartree potential as a Taylor series about each atom (to be interpolated in some manner). This set of state variables is small enough that users could tinker with SCF convergence by hand-tuning the initial guess for the state variables, and there is also a natural neutral-atom guess that is straightforward to construct.

Tracking a limited set of state variables is particularly important for accommodating Born-Oppenheimer molecular dynamics, to limit the amount of information that needs to be propagated in time. Designing a semiempirical model for use with molecular dynamics also means that the total electronic energy must be targeted rather than an enthalpy that contains vibrational contributions for proper atomic forces to be defined as energy gradients. This is one of the unfortunate disconnects of the PMx models in MOPAC, which target enthalpies directly, while comparisons with first-principles electronic structure requires careful separation of electronic and vibrational contributions to enthalpy. With molecular dynamics cheap and accessible, a semiempirical model could efficiently estimate the vibrational contributions to enthalpy, avoiding a baked-in vibrational enthalpy that needs to be explicitly removed to define molecular dynamics without vibrational double-counting.

Compact SCF cycle

With a modest number of state variables and an emphasis on dynamics, it may be possible to accelerate the SCF cycle, particularly SCF solution updates following small changes to atomic coordinates, by linearizing the SCF stationary conditions in terms of the state variables. With enforced localization of the density matrix, the linearized SCF equations would be sparse for large systems. This reduced form of the SCF equations would be particularly important for large metallic systems that typically cause “charge-sloshing” problems in the SCF cycle. These problems would correspond to large condition numbers for the linearized SCF equations, which is an opportunity to test various matrix preconditioning strategies to reduce the number of iterations when iteratively solving these equations.

If atomic forces and atomic coordinate perturbations are included alongside the state variables in the linearized SCF equations, then a harmonic model of atomic displacements becomes accessible. This could enable 2-stage molecular dynamics where a system is linearized, the harmonic model is propagated in time until atomic coordinate perturbation exceed a threshold, and then the system is linearized again about the new set of atomic coordinates. This could reduce the average simulation cost per time step, since many steps would only use the cheaper harmonic proxy model.

New matrix-element models

The one detail that surprises me the most about chemistry-centric semiempirical models is that they continue to use variants of the extended Huckel model, which prescribe very crude relationships between the Hamiltonian and overlap matrices. In contrast, physics-centric models typically use the more general Slater-Koster tight-binding formalism. I believe that it is worthwhile to model interatomic matrix elements using simple models of the atomic orbitals themselves. A particularly simple and cheap approximation would be to modify the popular contracted-Gaussian form by retaining the widest Gaussian and replacing the remaining Gaussians by a delta function for the purpose of computing just the interatomic matrix elements (p orbitals would have 1st derivatives of delta functions and d orbitals would have 2nd derivatives). In this manner, the electronic density associated with a product of two atomic orbitals is a single Gaussian plus some atom-centered point charges associated with the delta functions. This keeps the form of the electronic density manageable without introducing an auxiliary basis set.

A separate problem for older matrix-element models is the definition of on-site Coulomb integrals for d orbitals. The simple approach of independently parameterizing each distinct integral becomes a problem with d orbitals because they have several dozen parameters and there isn’t enough atomic spectroscopy data to constrain them. The standard solution is to define the on-site Coulomb integrals by a proxy Slater-type orbital with an adjustable decay exponent, but this ends up having a complicated functional form and far less flexibility than the treatment of s and p orbitals. A more uniform approach would be to embrace the use of an auxiliary basis set, which would split the on-site Coulomb integrals with four orbital indices into products of integrals with three indices - two of the atomic orbital basis and one of a semiempirical auxiliary basis set. This would amount to a more consistent reduction in variational freedom of the model while reducing the number of independent parameters for d orbitals enough to be manageable.

More atomic orbitals

It is empirically well-established that double-zeta basis sets are substantial improvements over minimal basis sets in DFT calculations, while triple-zeta and larger offer much more diminishing returns. Semiempirical models have never incorporated double-zeta basis sets for reasons of both cost and the difficulty of parameterization. With a Gaussian plus delta function model of atomic orbitals, there is a natural opportunity to define a double-zeta semiempirical basis by separating the Gaussian from the delta function. Furthermore, with no interatomic Fock exchange, the Fock matrix will have block structure whereby the delta-function orbitals on one atom have no matrix elements with the delta-function orbitals on other atoms. If matrix inversion (i.e. Green’s function) methods are used instead of matrix diagonalization, these localized basis functions can be efficiently downfolded using Schur complements, meaning that the double-zeta basis retains a minimal-basis cost after the downfolding step.

Localization expansions

In my recent review and benchmark of linear-scaling electronic structure algorithms, I explored some new algorithm ideas based on perturbative corrections to localized Green’s functions. The rationale for those ideas was that the simplest localization methods independently computed each column of a Green’s function and that introducing coupling between these independent calculations might improve their accuracy. However, I now realize that retaining the independence between calculations of matrix columns is important for both parallelization and retaining sparsity in response matrices (and thus the linearized SCF equations). The use of localization expansions is still possible while maintaining column independence by defining a different partition between “local” and “nonlocal” parts of the Hamiltonian for each column and directly inverting the “local” part for use in a Dyson series expansion of the “nonlocal” part. I believe that such expansions will be worthwhile because they allow for more aggressive truncation of the local regions since localization errors will receive a perturbative correction.

Another important change I plan to make relative to the review paper is to avoid direct calculations of the density matrix in terms of Green’s functions and instead compute all observables as responses from the free-energy matrix as described by Barros and coworkers. There were two novelties of their work - the use of a new random vector ensemble in stochastic trace estimation and the use of response-based observable calculations. In my review paper, I studied their random vector ensemble, but I did not implement the response formalism. While I didn’t find substantial benefits to their random vector ensemble over localized calculations, I do believe that their response formalism is highly beneficial and would combine well with localized calculations. The key idea is that the spatial decay of the free-energy matrix is more rapid than for the density matrix, so it can be computed more accurately for the same amount of localization.

In prescribing only an approximate, localized treatment of electronic structure, I would be allowing appreciable errors relative to the conventional matrix diagonalization approach to electronic structure. As long as these errors don’t contain severe numerical artifacts and are proportional to the underlying semiempirical model errors, then they don’t substantially degrade the accuracy of the overall formalism while enabling substantial cost reductions. Forcing linear-scaling algorithms to be numerically indistinguishable from conventional algorithms puts them at a severe handicap, and this led to their abandonment after being a popular research topic in the 1990’s. I believe that they have unrealized value for use in low-accuracy applications like semimempirical modeling if their accuracy standards are carefully relaxed. For example, to avoid truncation-related artifacts in localized calculations, spatial truncations need to be smeared out in some way to avoid discontinuities in potential energy surfaces when an interatomic distance crosses a sharp truncation threshold.

Density-dependent dispersion model

Prescribing a density dependence in dispersion models certainly isn’t new, it is a feature of D4, the latest version of Grimme’s popular dispersion models. However, I plan to introduce it in a much simpler manner as an approximation of a direct MP2 calculation between the explicit orbital degrees of freedom in the semiempirical model and implicit high-energy virtual orbitals. By allowing for only on-site orbital transitions and expanding energy denominators of perturbation theory in inverse virtual orbital energy, a simple density-dependent form can be derived. This derivation would forgo any diatomic parameters, so that the entire semiempirical model can be defined without the introduction of diatomic parameters, which is important for being able to optimize all model parameters simultaneously with modest amounts of reference data.

The sum of its parts

In aggregate, I am proposing a large number of changes relative to traditional semiempirical models such as PMx and DFTB. However, that shouldn’t be surprising since most of the model forms that they adhere to were formulated in the 1960’s and 1970’s. Electronic structure methodology as a whole has change enormously since then, and semiempirical modeling has changed very little to account for the substantial amount of new empirical knowledge that has been accrued in the much more active area of first-principles electronic structure over the last 40 years. This stagnation of semiempirical modeling is mainly the result of very limited academic interest, which limits both accessible funding and available manpower. There is unfortunately a lot of feedback, both positive and negative, in the modern scientific enterprise, and once a subdiscipline drops below a threshold of self-sustainability, it is very hard for it to recover. By my account, semiempirical modeling dropped below the self-sustainability threshold in the early 1990’s, and I just don’t think it can recover at this point without a dedicated patron. Is this technical plan even worth pursuing? I’m not sure - it would definitely be a lot of work, and I think it could eventually produce a more useful simulation tool, but I’m not clear on how much better it would be than existing tools, and there won’t be much value in it if there aren’t a lot of people interested in using a new tool. A older, worse tool is still making more of an overall contribution to science if more people are using it. Ultimately, this is a moot point for me, since it is extremely unlikely for anyone to ever support my pursuit of this technical plan. It’s just another project to add to my growing pile of broken research dreams.

There is a decent chance that I will eventually salvage some of this plan for building semiempirical models specifically for use with quantum computers. People are eager to run realistic-looking atomistic simulations on quantum computers, but there are and will continue to be extreme limitations in the size of such simulations. By their basic nature, semiempirical models try to extract as much value as possible from each electronic degree of freedom. However, this specific application would have slightly different design constraints. For maximum model efficiency, we would probably want an absolutely minimal basis, which might include removing d orbitals from atoms outside of the transition metal block and other orbitals that might be treated perturbatively (e.g. orbitals on noble gases). It is also important to limit the number of terms in the many-body Hamiltonian, which might justify a return to the neglect-of-diatomic-differential-overlap (NDDO) form of Hartree-Fock-based semiempirical models that retains only 2-center Coulomb matrix elements. The accuracy of the NDDO form can be improved somewhat by working in an explicitly orthogonalized basis, as shown in recent work by the Reiher group.

A major point of frustration for me was that I’ve been struggling with and focusing on career problems for most of the last decade, while I’ve had no financial problems at all and thus have paid little attention to financial matters. If I had been more mindful of my finances in the past, then I would have been investing my growing non-retirement savings in a prudent manner, and I would now be a lot closer to a financial solution to my career problems. Also, it was a bit unfortunate to learn about investing in a time of high economic uncertainty, negative real interest rates, and overinflated asset prices (the crown jewel of overvaluation right now being Tesla, Inc.). These frustrations aside, it was very comforting to read about the Financial Independence, Retire Early (FIRE) movement, which shows that financial independence is within reach for frugal people who can maintain highish-paying jobs for long enough. It was also very encouraging to see the inflation-adjusted total return of the S&P 500 index (and its precursors) over the last century on a semilog plot against an exponential fit - it is an impressive historical record of American prosperity that persists and is more accessible than ever through exchange-traded index funds. Stock markets have some disconcerting behavior - the extreme correlations between all liquid assets make it effectively impossible to fully mitigate the volatility of stocks with other stocks (i.e. the volatility of an index fund relative to its components is not reduced in accordance to the averaging of independent random variables), and the time correlations of volatility mean that deviations from the robust long-time exponential growth trend can persist for a decade or more. Still, the good very clearly outweighs the bad, and my past inattention to investment was a regrettable mistake.

I obsessively read about investing in mid-2020 and messed around a bit with fitting historical financial data to simple models to get more comfortable with investment, and I could definitely imagine an alternate reality in which I pursued a career in economics or finance. Nothing profound came out of that exercise, but I can share a representative exercise that might be of broader interest. For the most part, I plan to invest only in large exchange-traded funds with low expense ratios, and so the main impactful decision that I have to make is the allocation of my money between assets with low and high risk/volatility. For this exercise, I’ll model the low-risk asset as having fixed value with no volatility (e.g. some mix of cash and high-grade bonds) and the high-risk asset as a historical time series of the inflation-adjusted total return of the US stock market between 1870 and now. The simple-minded investment strategy is to adjust the cash/stock ratio based on the immediate value of stock market relative to its uniform exponential growth baseline of around 6.5%. This ignores economic indicators that are surely much better than deviations from a uniform baseline, but it illustrates the basic exercise of balancing the risks and rewards of volatile assets. The optimization of this investment strategy is a straightforward numerical exercise conducted and plotted with Python, which shows a very limited benefit from reallocating assets (blue curve) relative to the underlying risk asset (red curve).

Inevitably, the optimal investment strategy obtained from this numerical exercise is an artifact of the specific details of the historical data, and its details do not reflect a useful forward-looking investment strategy. However, a useful observation is that deviations from full stock market exposure only occur above a 50% overvaluation. With a more uniform strategy of completely removing stock market exposure above 50% overvaluation, the growth of investments is unchanged from full exposure to the stock market under all conditions. This suggests that overvalued stock markets have already realized all meaningful profits for investors, and whether you pull money out early or ride the overvaluation back down, your outcome will be roughly the same on average. This is consistent with general advice against timing the stock market, although this exercise suggests that it is more futile than actually harmful. I will eventually settle on an allocation strategy much like this for my own money, but I still need to learn more about various economic indicators to decide on which ones to use in practice. While market crashes and bear markets cannot be predicted, economic indicators can identify periods of increased risk (such as right now) when stock market exposure should probably be reduced.

Because I still need more money before I am financially independent with a comfortable buffer, I am trying to shift my discretionary research interests and activities in a more profit-seeking direction for the foreseeable future. Of my established skills and interests, I have decided that the most likely to be lucrative is quantum computing. I have two recent quantum computing preprints that I need to revise and publish - a quantum Metropolis algorithm and a finite-temperature variational Monte Carlo method - that might help me find a higher-paying job in quantum computing. I took my present job for the potential opportunity to continue my electronic structure research rather than compelling financial compensation, but that opportunity has unfortunately not panned out. Quantum computing is a very hot topic right now with numerous scientists working at high-compensation tech companies like Microsoft, Google, and Amazon. Also, I have some very promising unpublished ideas in quantum error correction (QEC) that I will attempt to patent when they are sufficiently developed. There are some very inefficient aspects of existing QEC protocols (e.g. logical non-Clifford gates), and patents on more efficient methods could end up being very lucrative. I’ve read up enough about patents to appreciate the difficulty of their monetization (their main use is as a legal defense, not a monetizable asset), but I have the opportunity to file patents with my employer covering the costs in exchange for half of any future profits. While I could probably make more money if I tried to pivot away from research altogether for a while, I see quantum computing research as a reasonable compromise for now.

While I still have a long road ahead of me on my path to intellectual independence through financial independence, it is overwhelmingly more plausible than the prospect of finding a job that will allow me to pursue my research interests. The modern US scientific enterprise is simply not supportive of intellectual independence - it tightly controls the active topics of research through the narrow flow of funding and the very low success rates of grants. This works for scientists with very malleable interests or whose narrow interests align with funding priorities, but career starvation starts to set in when a scientist’s interests fall out of favor and they aren’t willing to choose from a narrow pool of acceptable new interests. I’ve seen this happen multiple times so far in my career, and I consider it to be extremely shameful that the scientific enterprise does this to people. In some cases, the unfortunate scientist has had enough savings that they could afford to continue their research on their own dime. In other cases, they burned through their life savings and then gave up on science altogether.

This post isn’t about a new project or paper, but rather the revisions to the low-rank approximation paper that I submitted for publication towards the end of 2019 and blogged about. Much to my surprise, I got the reviews back for the paper in late January. The timescale of mathematical publication is notoriously slow, so I didn’t expect to hear from the journal for at least 6 months. I was initially given until the end of March to revise the paper, but I didn’t even start the revision process until after I finished my last paper for the 2020 APS March Meeting. I was able to get multiple extensions from the journal, but eventually they gave me a hard deadline on October 1st. It has been a long and tiring journey from the first to the second version of this paper, and I feel like I’ve grown as a mathematician in the process.

As I previously discussed, the first version of this paper was heavily restricted in scope compared with my original plans for it. There were three intended applications that were dropped, and the central proof of the paper was a touched-up version of a proof that I wrote several years ago. My focus in preparing that proof for publication was correctness rather than clarity, and it resulted in a terse and hard to understand final product. Also, part of that terseness was minimizing the amount of time devoting to discussing external results that the proof relied upon, partly because I did not fully understand those external results. While it was a much better proof than every previous attempt I had made, it still wasn’t very good in any objective sense, but it was the best outcome I could produce with the effort I could afford to expend on it.

If I thought it was appropriate, I would post the two referee reports that I received. However, referees retain copyright, and it would be inappropriate for me to post them without permission, which I simply cannot obtain from anonymous referees. What’s important is that I consider them to be fair mathematical assessments of the paper. They were intrigued by the central theorem of the paper, found the central proof very difficult to understand, and were not that interested in the remaining content of the paper outside of the central theorem and proof. I felt like I had a strong opportunity to obtain the approval of these referees if I reworked the central proof into a more readable form, and I enthusiastically embarked on paper revisions after the 2020 APS March Meeting.

Revising this paper also meant revising my expectations on what constitutes good mathematical writing. According to my previous expectations, the original version of the paper was “good enough”, and whatever my new expectations were to be, that paper could no longer be “good enough”. There were completeness, notation, and organizational problems. The original, terse version of the proof made some terse arguments that would actually take a lot more space to justify thoroughly. For example, I defined optimization problems over closed domains that I could explicitly solve for most but not all elements in the domain. I decided to simply remove the problematic elements, since their contributions to the set of possible optimizers could be recovered from limiting arguments - “nearby” elements would contribute very similar values to the set of possible optimal values. However, a truly rigorous proof would either have to show that the contribution of these discarded elements were recovered or at least cite something that showed this to be true for a general enough class of problems that this specific case was covered. When considering the amount of space such an argument would take, that no longer seemed like a good proof strategy at all. Also, I got way too eager to define a zoo of auxiliary variables and didn’t clearly explain what they were useful for. However, the trickiest aspect of the revision was deciding how to break up the proofs using intermediate lemmas. This is a canonical aspect of mathematical writing - breaking up a complicated argument into smaller, technically distinct pieces to make the argument more understandable. Sometimes those intermediate steps are interesting and useful in their own right, and sometimes they are just a means to an end. I struggled with this the most, as I had to work through trial proofs for each of the possible intermediate lemmas that I considered, and it took many attempts to find one that made the cleanest separation between the lemma’s proof and the central theorem’s proof that would depend on the lemma. Right now (pending further revisions to my math writing expectations), I’m quite happy with the end result, but I’m not happy with just how long it took to get there.

I’m not completely happy with everything in the revised paper. The paper is about an optimization problem, and I strengthened up the formal results of the paper to rigorously characterize optimal solutions. This made it feasible to solve the optimization problem numerically, and I dived into writing a numerical solver to add to the paper. Unfortunately, that was its own time sink. With priority given to cleaning up the theorems and proofs, I didn’t give the numerical solver the time it needed to develop fully. What ended up in the paper is the third completed version of a numerical solver, which was a massive improvement to the previous two versions, but just managed to stumble into an uncanny valley where it was just good enough to clarify what the remaining technical problems were without actually being able to solve them. Unfortunately, I just ran out of time for a fourth version, so I cleaned up what I had before the resubmission deadline.

With this experience behind me, I feel like the revised paper is now legitimately my “first” math paper, but that feeling might not survive the next round of the review process. Outside of further revisions, I’m done with math research for the foreseeable future. While I have unfinished math projects and plans for future math research, I just don’t have the time to pursue this kind of work right now. A lack of research resources has consequences, even for theorists. If you don’t financially support an experimentalist, then they just can’t afford to run a lab, and all research quickly grinds to a halt. With theory work, the most valuable resource is a theorist’s time. I want to spend as much of my time as possible on research, but my full-time job is not a research job, and it has obligations that frequently divert my time and attention from research. I can pretend that I can effectively carry out two full-time jobs, but in reality everything suffers as a result. I wish that I could afford to focus exclusively on research right now, but I simply can’t, or at least not without throwing away my financial future.

So, what comes next? I’m trying to escape the death spiral of my research career. So far in my career, none of my research has generated enough interest to expand my future resources and opportunities, which has lead to a negative feedback loop of dwindling resources and opportunities. For example, this math project and paper originated from my personal interest in a problem and my beliefs about its future utility to my research. It wasn’t some big open math problem that lots of people cared about, and very few people are going to care about it or understand it. Unfortunately, I now see that such research predilections are quite fatal in the long run to a research career, and my run has been long enough that I can really feel it dragging me down to the bottom now. Next, I’m going to try something different for a while. I’m going to prioritize research based on how I think others might value it. Specifically, I’m going back to quantum computing research, which I consider to be overhyped and overvalued. My plan is to patent new quantum error correction (QEC) protocols that will be essential to future digital quantum computers. Of all my research interests, this is the one with the most immediate and explicit possible value. For once, I’ll attempt to play into excessive hype and overvaluation by trying to sell something for more than it is truly worth. In upcoming blog posts, I will rant some more about career issues, offer my somewhat negative perspective on quantum computing in general, and discuss these QEC plans further.

First, let me articulate my vision for what my ideal research career would look like. While I might want complete freedom and autonomy to pursue basic research on whatever topic I want, whenever I want, for as long as I want, I don’t think that is feasible target to desire or pursue. Except for independently wealthy scientists (which doesn’t include me), science always has a patron, whether it be public (a government) or private (a company), and I certainly believe that scientists should be advancing the causes of their patrons with applied research alongside their pursuit of more basic research. The more useful science is able to be, the more it can justify increased levels of investment and increased tolerance for risk. I would be quite content with a research portfolio that is 50% basic and 50% applied, especially if there were natural, synergistic connections between basic work that I chose to pursue and applied work that was a bit more of an obligation. I would also like to be part of a research community that I felt a sense of belonging to, with frequent substantive technical interactions that occasionally produced collaborative projects. Finally, I’d like some sense of long-term career stability so that I’m not constantly worried about how to keep my research career alive for another few years.

While I continue to have some semblance of a research career, it is far from my ideal. At my present job, I have a steady 20% of my time to pursue basic research interests. This is substantially better than my last job (Sandia), where I had no baseline support for my research interests, and my average level of support over 7 years was around 10%, although there were several years when it was just zero. I have failed to achieve research synergy in all of my jobs so far. Institutionally, Sandia was never even remotely interested in any of my research, and I was mainly there as highly skilled labor for other people’s research projects. At my present job, there is some correlation between the software development projects that people are assigned to and their underlying skills and interests, but we intentionally keep our personal research disconnected from our community-centric software development to avoid the appearance of bias. As for community, I have never felt like I’ve belonged anywhere, and my sense of isolation only continues to grow with time. I have never had a sustained, recurrent, and reciprocal research interaction with anyone, and I probably never will at this point. There is almost no overlap between the problems that I find technically interesting and what other scientists find technically interesting. Finally, I’ve had some career stability on paper, but not in practice. My job at Sandia was as permanent as I wanted it to be, but I was displaced from my intended field of research after only a few years, and the pressure to move yet again over to classified projects eventually started to ratchet up a few years later. Diversifying my research was something I could tolerate, but a shift to classified work is very often an irreversible career choice, and I began to feel very trapped. While I am happier at my present job, the institute that I work for is temporary - it has to win a 1-time grant renewal in a few more years to continue operations, and it has to become financially self-sustaining in 5 more years after that or else it will cease to be.

I certainly accept some of the blame for my career misfortunes. I do not publish papers frequently enough, I have not focused my research interests enough, and I have not engaged in enough self-promotion. Two friends of mine from graduate school who were in the same group and started at the same time as me have a Google Scholar h-indices of 29 and 43, whereas my h-index is 12. They each have a well-defined primary research topic, whereas I’m just all over the place with my research projects. They are now tenure-track professors, and I failed to get any in-person job interviews after applying to several hundred tenure-track positions over the course of 5 years. Part of this stems from my early misunderstandings of interdisciplinarity. Interdisciplinary research is often touted as a great virtue in science, but the virtue is specifically in researchers collaborating across disciplinary boundaries and very much not in having researchers wander between disciplines without regard to disciplinary boundaries. Over the years, I have enjoyed pursuing research over a relatively wide variety of topics, but my “interdisciplinary” research interests have brought me to a point where I don’t feel welcome anywhere.

However, I am not willing to accept all of the blame for my career misfortunes. I also think there are systematic problems, both broadly throughout scientific research and narrowly related to some of my interests. The broader problems are relatively well known. There is an excellent book, How Economics Shapes Science, that presents a lot of data and analysis on how different funding levels and strategies affect the culture of scientific research and the distribution of career outcomes. I highly recommend the book to anyone interested in this topic, but I will not attempt to summarize it in any meaningful way here. My distillation of its contents and related information from other sources is that government-funded basic research in the United States is too heavily invested into growth. Basic research funds are primarily directed to professors for student-driven research. This investment focus is very good for rapidly expanding the labor pool of scientists, and gives scientists a generic, useful purpose regardless of the usefulness of their research - the job of training new scientists. However, this system drives the training of new scientists faster than they can be employed within basic research, and any equilibrium requires a very high rate of attrition. While the overall US investment into research and development as a fraction of GDP has remained relatively steady within the 2%-3% range for over 50 years, there has been a steady shift of that funding from the US government, which supports a mix of basic and applied research, and private businesses, which predominantly supports applied research. I suspect that this shift has been inevitable and will continue until an equilibration - private businesses do not invest much in the training of new scientists, so the shift reduces the overall rate at which new scientists are trained, and it will hopefully, eventually match a more natural rate of career attrition (retirements of older scientists rather than younger scientists being forced into non-research jobs). Of course, this also has many obvious problems - not all fields of science are directly and immediately useful for private businesses, so this is likely to filter career outcomes of fields by their short-term utility rather than possible long-term impacts, and it pushes more activity from basic research to applied research. Applied research is less likely to be published and less broadly useful, and a recent study has shown that applied research outcomes (e.g. patents) heavily depend on past basic research outcomes (e.g. scientific publications). Thus, in eroding our present capacity for basic research, we are eroding our future capacity for applied research. In my view, a “simple” solution to this problem is to guarantee that scientists be able to continue basic research work with some fraction of their time when employed by private businesses to conduct mostly applied research. Of course, achieving this solution will not be easy, and it will probably require some combination of legislation and collective labor actions among scientists.

In addition to these broader scientific problems, my core interest of atomistic simulation methodology has had its own unique problems over the last decade or so. Electronic structure theory has traditionally been a subdiscipline of physics, but its popularity within physics peaked in the 1990’s and early 2000’s. By the time I graduated in 2008 (into a recession, no less), electronic structure theorists were being pushed into materials science departments as the hiring in physics began to shift to more trendy topics (e.g. topological phenomena). However, the hiring in materials science was mostly dominated by applications-driven research - they were much more interested in people applying established theoretical and computational tools than people trying to develop new tools. This reflects on a broader trend that career outcomes are generally more poor for people working on method and software development than their applications, since those activities produce fewer papers and dwell on time-consuming activities like programming that do not directly benefit scientific careers. There has been a recent push in the US to invest more into research cyberinfrastructure, and the institute that I work for is one small piece of that investment focused on computational chemistry. However, my institute draws a sharp distinction between method and software development, and our focus is strictly on software. We serve the methodology that exists and that’s it. Methodology research itself is an unforgiving, high-risk slog, and it largely manages to survive by clustering around specific paradigms and ossifying into something more tangible and predictable. In computational chemistry, the two enduring paradigms are classical treatments of molecular dynamics using interatomic potentials and Gaussian-based calculations of electronic structure, either using density functional theory (DFT) or quantum many-body methods. As I’ve discussed multiple times in this blog, there used to be a third paradigm of semiempirical modeling between these two that was able to achieve some novel compromises between cost and accuracy, but its popularity collapsed in the early 1990’s - it no longer has an appreciable developer community, it no longer trains students, and it no longer attracts research investments. Unfortunately, my institute was envisioned to serve the paradigms that persist rather than to revitalize nascent paradigms or create new ones, and it is difficult to pursue methodology research without the backing of an established research community. This methodological polarization in computational chemistry can even be seen in the private research investments of billionaires - D. E. Shaw Research supports classical molecular dynamics method development and The Flatiron Institute supports electronic structure method development.

Even traditional electronic structure methodology research is now under increasing pressure because two popular research topics - machine learning and quantum computing - are applicable to electronic structure. Many people have given up on developing faster algorithms for electronic structure, and they are instead using machine learning to interpolate new results from enormous amounts of precomputed reference data. There is more support to develop quantum algorithms for electronic structure simulations on hardware that won’t exist for several decades than there is to develop genuinely new classical algorithms that work on our present classical computers. It may soon become almost impossible to get funding in electronic structure methodology research unless it is based on either machine learning or quantum computing somehow. While there is widespread optimism that both machine learning and quantum computing will revolutionize electronic structure simulations, I am extremely skeptical and pessimistic about their practical utility. Machine-learning models simply require too much reference data to attain the level of accuracy, reliability, and broad applicability that we expect from electronic structure simulations. Quantum computing enables electronic structure algorithms with better asymptotic scaling, but their cost prefactors will inevitably limit their application severely. This has happened before, and it will happen again - there was widespread optimism in the 1990’s about linear-scaling electronic structure algorithms, but their cost prefactors are still too high for them to actually be cheaper for the routine simulations that people can afford to do.

Career concerns are certainly not new for me, I started to realize towards the end of graduate school that it was difficult to plan a long-term career trajectory, but I just assumed that it would all work out somehow along the way. Unfortunately, it hasn’t worked out. I learned first-hand how narrow and limited the basic research investments are at the US National Labs, and I’ve failed enough times at applying to tenure-track faculty positions to know that it isn’t a realistic career option for me. I have not met any like-minded people along the way. I’ve had escapist fantasies about developing some of my methodology research into commercial software instead of publishing it to gain financial independence and maximum freedom to pursue my research interests, but I know full well that commercial successes of this sort are even more elusive than academic successes. However, while commercial software development isn’t likely to succeed for me, it is at least a direction and an activity other than writing job and grant applications that only ever get rejected and writing papers that only ever get ignored. I do strongly believe that the best science is open and the best scientific software is open source, but someone ultimately has to pay for these things. When funding gets tighter, scientists hold back more knowledge to maintain advantages over their peers in the fierce competition for scarce resources.

Over the last several months, I’ve thought a lot about how to steer my career in a more favorable direction and regain more of a sense of control over my future. A common feature of all the plans that I considered was a consolidation of my research into a narrow, coherent path. As I noted in my last blog post, I have 4 projects/papers that I’m in the process of finishing right now, and those account for all of my research interests that aren’t already on indefinite hold. From there, my research will resume down a more narrow path. My ultimate goal remains the same as when I started this blog - to develop a new generation of semiempirical models - but now I plan to develop them as a Software-as-a-Service product, because otherwise their ongoing development will not be financially sustainable (as research funding in this area is hopeless right now). As I’ve previously stated in past posts, there are 3 technical pillars of this effort - correlation models, basis sets, and solver algorithms. Of these pillars, correlation models generate the most academic interest and have the most technical uncertainty, and my open research efforts and open-source software development will narrowly focus on correlation models. While innovations in basis sets and solver algorithms are equally necessary to improve semiempirical modeling, there is very little academic interest in these topics, and I plan to keep my development of them proprietary in the absence of sustained research support. This plan will help me to build a better academic reputation by focusing on a popular research topic - correlation modeling - that has ties to two other popular research topics - machine learning and quantum computing. All the while, I will also be developing the other necessary technical components of new semiempirical models in silence, since there really isn’t anyone to discuss them with anyways (I’ll still summarize their progress in this blog, at least).

In the remainder of this post, I’ll discuss some of the different paths that I considered while deciding on my present course of action and how I plan to proceed when my last few ongoing papers are finished.

Possible scientific software projects to focus on

My first consideration was about the research projects that I could plausibly develop into useful software on my own. For this endeavor to be worthwhile from a career perspective, the software must either be of enough academic interest to sustain an academic career or useful enough to enough people to be a viable commercial product that I can make a living off of. While I have developed many small software projects over the years and made minor contributions to a few larger projects, I have never seriously pursued my own large project intended for widespread use for a variety of reasons. First, I’ve never had institutional support to do so at any of my jobs, and it would be hard to sustain sufficient effort as just a hobby. Second, most of my methodological pursuits still have too much technical uncertainty to be worth committing to. Third, I have have strong, recurring doubts that this would actually help my research career even if everything went well technically. Only the second reason is a technical concern, and it is sufficient to narrow the list of possible projects down to three.

Dense Hermitian eigensolver

This is the project with the least technical uncertainty, but also the one that I am most certain is not worth the effort. It targets a growing problem in scientific computing, but the effort required to solve the problem substantially exceeds what anyone is willing to support. Numerical linear algebra involving dense matrices is a very mature subject with underlying theory that was mostly developed over 50 years ago, and mature implementations in open-source libraries: elementary operations in the Basic Linear Algebra Subroutines (BLAS) that are used to construct solvers in the Linear Algebra PACKage (LAPACK). BLAS routines are divided into 3 levels corresponding to vector operations, matrix-vector operations, and matrix-matrix operations. For performance reasons, it is essential for the computational bottlenecks of LAPACK solvers to be implemented entirely using level-3 BLAS operations (matrix-matrix), which are able to hide the growing imbalance between communication and computation costs on modern computers. The most extreme case right now is the cost ratio of a CPU-GPU communication operation to a GPU floating-point operation, which is around 1000-to-1. To hide this imbalance and approach peak floating-point performance, solvers need to apply level-3 BLAS operations on submatrices (“blocks”) of dimension close to 1000. LAPACK solvers can be roughly grouped into two types - linear-system-like solvers and eigenvalue-like solvers. Level-3 BLAS implementations of linear system solvers are very old, they were even available in LINPACK, one of the precursors of LAPACK that is still used to rank supercomputers in the Top500 List. Unfortunately, eigensolvers do not have fully level-3 BLAS implementations, and their performance has been steadily degrading compared to linear system solvers in both shared-memory and distributed-memory settings.

The problem with modern eigensolvers is that they require the orthogonal transformation of dense matrices to tridiagonal matrices so that tridiagonal eigensolvers can be used to solve the problem. It is impossible to implement this transformation with only large-block level-3 BLAS operations. In recent years, there have been two attempts to circumvent this problem. The ELPA library splits the transformation into two steps using an intermediate banded matrix, and the dense-to-banded step can be implemented with level-3 BLAS operations. This doesn’t eliminate the memory bottleneck, but it reduces its overall impact. The EigenExa library transforms to a pentadiagonal matrix instead of a tridiagonal matrix and uses a novel pentadiagonal eigensolver. This transformation still can’t utilize large-block level-3 BLAS operations, but it also reduces the size of the memory bottleneck somewhat. Both of these approaches are avoiding the obvious solution of transforming to a banded matrix with a large bandwidth and using a banded eigensolver because a fast and reliable large-bandwidth banded eigensolver doesn’t exist. The SLATE library that is refactoring dense linear algebra software for exascale supercomputers plans to incorporate the same approach that ELPA uses and will have the same performance limitations.

I’ve been working on the technical components of a banded eigensolver for a long time, spread out over several projects such as the Cauchy kernel project that I finally finished in late 2019. These efforts have made extremely slow progress because they have had essentially no overlap with any technical work that I have been paid to do. I need a lot of sustained focus to make progress, which is impossible when I have other work to do. Everything has come together to the point where I can see a roughly 1-year technical path to a proof-of-principle prototype algorithm, but the only way I’ll ever have a solid year to focus on this is if I quit my job. This just isn’t something that physicists or chemists care very much about, so to have any academic value I would have to promote myself among mathematicians, which I haven’t been very successful at in the past. This also isn’t something that can be realistically commercialized because people expect dense linear algebra software to be free like BLAS and LAPACK. The only example of a commercial numerical linear algebra library that I am aware of is from the Numerical Algorithms Group, which suggests a very tiny commercial market that is unable to sustain much commercial activity. I would also be putting myself on the hook to make the eigensolver highly performant in a wide variety of computing environments, which requires substantially more effort than simply implementing a working prototype.

Quantum many-body solver

This project is the further development of the new finite-temperature variational quantum Monte Carlo method that I discussed in my previous post. The basic theoretical foundation of this method has already been developed, and preliminary numerical tests have been surprisingly successful. The detail that I was worried the most about - the tightness of entropy bounds - actually caused me the least amount of trouble, although it might be because my test problem was too easy. This has the potential to be a broadly useful many-body simulation tool - capable of simulating both fermions and bosons and capable of both equilibrium sampling and non-equilibrium time evolution - but a lot more development work is still needed to fill in the details of each of these cases.

This project would have a large amount of academic value because quantum many-body problems are still of great academic interest in physics and chemistry and its technical approach shares common ground with quantum computing and machine learning research. I can try to exploit those underlying connections to promote this work in academic circles. The closest available software right now is NetKet, which seems to be doing quite well academically, since its development is being supported by the deep pockets of the Flatiron Institute. My approach has the differentiating features of being able to simulate finite temperatures in a variational manner and exclusively utilizing direct sampling so as to avoid the inefficiencies of Markov-chain Monte Carlo sampling. I’m also going to avoid the use of established machine-learning models and focus on more physically motivated cluster expansions, which I believe will be more reliable and effective when these methods are pushed to very high accuracy.

Quantum many-body solvers are not really used outside of academic circles, partly because none of them are very reliable yet and partly because the problems that they are needed to solve have limited practical relevance. Perhaps the most promising commercial use would be to mimic noisy quantum computing devices, which are themselves starting to be packaged as on-demand cloud computing resources. If they start to command a high price and my solver can mimic their outputs reliably, then I could claim to offer an equivalent computational capability and likely be able to offer it at a lower price. However, this scenario is largely self-defeating because the broader interest in these noisy quantum computers is contingent on them having a computational advantage over classical computers for some tasks. If it becomes clear that their noise makes them efficient to simulate on classical computers, which there is growing evidence for, then such a market would quickly dry up as the hype died down.

Semiempirical electronic structure

I went through a lot of brainstorming about semiempirical electronic structure in 2019, some of which was mentioned on this blog. While there remains a lot of technical uncertainty overall, I was able to bifurcate my plans into a low-uncertainty short-term plan and a high-uncertainty long-term plan. The success of the short-term plan might inform the development of the long-term plan, but they otherwise do not have much direct technical overlap. The essence of the short-term plan would be to fix the largest technical problems in existing semiempirical models and rigidly couple them to low-cost, low-accuracy linear-scaling solver algorithms. These technical problems are the severe approximations of 4-center Coulomb integrals, the restriction to a minimal basis, and the independence of dispersion parameters. Much less severe approximations could be used to maintain a low cost of 4-center Coulomb integrals while preserving a lot more accuracy, an effective double-zeta-polarized basis set could be introduced with almost no added cost by carefully controlling the pattern of non-zero Hamiltonian matrix elements, and the low-energy-scale dispersion parameters could be effectively derived from other easier-to-fit high-energy-scale parameters by using more physical models for dispersion interactions. While these innovations could substantially improve on existing semiempirical models, they will quickly hit a technical wall in how much they can improve accuracy.

The academic value of semiempirical modeling is very low. Semiempirical approaches to DFT out-competed the more traditional approaches to semiempirical modeling in the early 1990’s, just as the main developer of semiempirical models throughout the 1970’s and 1980’s - Michael Dewar - was retiring and attempting (largely unsuccessfully) to transfer and continue his semiempirical research program through younger collaborators. One of Dewar’s former collaborators - James Stewart - had forked this research program and software development in the early 1980’s into what would become MOPAC, which is now the last bastion of traditional semiempirical modeling. However, the development of MOPAC was never academically successful. Instead, it has persisted for over 3 decades as a mostly commercial effort, although versions of it have always been freely available for academic use. DFT was itself the inspiration for a new approach to semiempirical modeling in the 1990’s called Density Functional Tight Binding (DFTB). While it has never been too popular, it has maintained a steady albeit modest academic existence and a well-maintained open-source implementation called DFTB+. The most recent semiempirical models and their open-source implementation in XTB have come from the Grimme group, which have cross-pollinated some of the technical details of older semiempirical models with DFTB models (mostly a higher-order inter-atomic multipole expansion for electrostatics) and provided some new parameter sets with wide coverage of the periodic table and compatibility with the most recent dispersion models developed by Grimme group (it is most well known for DFT-compatible dispersion models). Realistically, this means that academic activity in semiempirical modeling is at subsistence levels - there is just enough activity to propagate old ideas into the development of new models but little room for more exploratory innovation.

The overall market for commercial atomistic simulation software is not enormous, but it large enough to warrant market analysis and continues to grow as atomistic modeling finds more industrial applications. While semiempirical models are a small market segment of this overall market, there are obvious growth opportunities if either (1) linear-scaling implementations can be reduced in cost enough to compete with high-end interatomic potentials for classical molecular dynamics applications or (2) new semiempirical models can be improved enough in accuracy to be competitive with low-end DFT calculations. What exists right isn’t quite up to either of these challenges, but they are both realistic targets. Given the low academic value of semiempirical modeling, it is actually counterproductive for new models to be specified openly in the scientific literature and they are likely to be more successful if they remain proprietary. For example, in my very first blog post I had presented evidence from publication trends that many more people used the very popular, multi-purpose GAUSSIAN software to perform semiempirical calculations than the special-purpose MOPAC software using the same semiempirical models that were initially developed for MOPAC and eventually replicated in GAUSSIAN from their open specifications in the scientific literature. This is a market dominated by a small number of very popular commercial and quasi-commercial efforts (e.g. VASP, GAUSSIAN, AMBER, CHARMM) that are active enough to implement new features from their literature specifications if there is sufficient user demand. Thus, having a chance at penetrating such a market requires a competitive advantage such as proprietary features that have no open specification and would require substantial reverse-engineering to implement independently. There is already some precedent for this - the QUASINANO2013 parameter set for DFTB that has large coverage of the periodic table is proprietary and only available in SCM’s commercial implementation of DFTB.

Possible business models for scientific software

The other axis of consideration was possible business models for software, if I get to the point where I have something that is worth selling and there aren’t good alternative research career options. In looking at past commercial successes of atomistic simulation software and other scientific computing software, it is clear that academic success has always precluded commercial success. Basic research in an academic setting is necessary to buffer the uncertainty in developing this kind of software, so that it can develop into a more stable, well-defined entity without onerous amounts of time and budgetary pressure. Often, pursuing commercial development opens up more opportunities for research funding to bridge the gaps between research software and commercial software, such as through the US Small Business Innovation Research (SBIR) program. Adjusted for inflation to 2020 US dollars, the SBIR program has awarded $7 million to Schrodinger, Inc., $19 million to Q-Chem, Inc., and $3 million to MOPAC as several notable examples. Thus, research software that was initially developed using support from basic research grants has been able to smoothly transition to commercial software with further US government support. Research projects that fail to attract funding within the arena of basic research are also likely to be on their own in any attempts to develop into commercial software products. To be truly successful as a commercial product, commercialized software must also be able to generate a profit, otherwise it is just gaming the system to attract more development funds. I’ve considered 3 different business models in imagining a future business.

Traditional software licenses

Historically, commercial atomistic simulation software has been monetized through the sale of software licenses, either for a one-time cost or with recurring maintenance/support fees. Because the market for this software is small, estimated to be a few tens of thousands of scientists, the price of these licenses usually ranges from hundreds of dollars to tens of thousands of dollars, typically with lower prices offered to academics and higher prices offered to industrial users. Lower prices are not commercially viable without a larger market. These high prices are yet another reason why the market is dominated by software built on strong academic reputations - people need a lot of trust in the science underpinning the software to be willing to pay such a high price for it. Younger scientists and newer software trying to enter into this commercial market are at a severe disadvantage, and have to work very hard on the academic side to build up a competitive reputation to have a chance at selling software for sustainable prices.

Indebted open-source licenses

There has been a big push in recent years for science to be more open and reproducible, and a corresponding push for scientific software to be open source. In my own research, I have been following these standards for about 8 years now, and all the papers I have written in that time have been accompanied by whatever custom software was used to generate their results. Ideally, I would like all of my software development to be open source, but the projects that I mentioned above are not supported by my current job and I have yet to find a job that will support any of them. How to pay for open-source software development is an existential question that frequently comes up in the open-source software community, and it doesn’t have a generic, universal answer. Financially self-sufficient open-source projects either serve a very large user base (e.g. internet infrastructure software) or corporate customers that are willing to pay for ongoing support and/or customization. Neither of these scenarios is applicable to the scientific software that results from basic research. Academic open-source software is funded by temporary research grants and written by temporary labor (grad students and postdocs), and eventually the money and labor disappear. More successful projects are able to maintain a series of multiple grants to continue their development and transfer development between cohorts of grad students and postdocs. Most projects simply become permanently inactive - their openness allows them to remain available for reproducibility and posterity, but they cease to have a future beyond that.

I spent some time imagining how open-source software could become more financially sustainable, and I came up with an interesting concept that seems reasonable on paper but probably won’t work in practice for scientific software. The basic premise would be to release software under an “indebted” license that reverts to a more permissive open-source license once a stated overall debt had been paid or enough time had elapsed for the debt to “default”. While the license is indebted, users would be obligated to state how much they are willing to pay off the debt (zero being an acceptable amount), and developers would be obligated to report on how much the debt has been paid off. The indebted software could be used in downstream open-source projects, which would then inherit their indebted license provisions. This could lead to some interesting effects such as an obscure piece of indebted software being used by a popular open-source project, which then quickly pays off its debt to open the license back up again. If the debt is stated as a single large sum, then that will probably be quite intimidating to potential donors. This model would probably work best if the debt is slowly incremented as features are added to software. Unfortunately, I just don’t think people are generous enough for this idea to work. There would still be plenty of software with very few users and unpaid debts in which the developers invested a lot of time for no compensation. It is just difficult to match support with need, and promoting an open-source software development culture where needs are explicitly articulated through indebted licenses might help. There would be a lot more open-source software development if it were easier to turn into a reliable job rather than just a hobby.

Software as a Service

The final business model that I considered was the Software-as-a-Service (SaaS) model. This is a very new business model, facilitated mainly by the recent commoditization of computing through cloud computing services. For compute-intensive tasks like atomistic simulations, a SaaS product wouldn’t be realistic if cloud computing costs are substantially higher than maintaining your own hardware. This has been true for many years while AWS maintained a near-monopoly on cloud computing, but recent competition from Microsoft and Google have driven cloud computing prices down to more reasonable levels in the last few years. There are a lot of remarkable SaaS success stories, but a common feature of these successes is relatively simple software with short development times targeting a well-chosen niche. An important feature of this model is the incrementalization of costs, whereby a relatively modest amount of money can be charged to use a program just a few times rather than requiring a user to buy an expensive license. There is a tangible ongoing benefit to user retention (it directly impacts the continuous revenue stream), and revenue is more uniform rather than spiking when new versions of software commanding new licensing fees are released. By removing a large upfront cost, it is easier for a SaaS-based product to attract curious new users by not requiring them to make a large initial financial commitment. This can possibly change the dynamic of first having to build up an academic reputation before being able to market a high-cost software product into one where the academic reputation is built while developing and expanding the capabilities of a SaaS product.

The Software-as-a-Service business model is not incompatible with having open-source software, but the choice of what is open and what remains closed will strongly impact commercial viability. If you just create some fully open-source software and then package it up as a SaaS product, then there is nothing stopping someone else from also packaging it up as a SaaS product and charging less money for it. Something like this hasn’t happened yet with scientific software, but it has been happening with open-source database software over the last few years. With scientific software, I believe that the trick is to hold back some “secret sauce” in the SaaS product that differentiates it from the purely open-source software component while making the more academically interesting parts of the code as open as possible. Such proprietary components might be ease-of-use features such as automatic fine-tuning of parameters for non-expert users or optimized implementations with higher performance or adapted to specialized computing hardware (e.g. GPUs).

The path from a short-term plan to a long-term plan

I’m a big fan of planning, and I frequently make research plans that just don’t survive for very long under the uncertainty of research activity. I believe that the trick to improving the survivability of research plans is to have more definite short-term plans and more ambiguous long-term plans that are only clarified as more information is gathered by achieving (or failing to achieve) the short-term goals. With my newfound open research focus on correlation modeling, the short-term goal is to develop my variational Monte Carlo (VMC) method into the best possible high-cost, high-accuracy quantum many-body solver that it can be, including an open-source software implementation. While the present version of my VMC paper is just the basic theory and preliminary numerical results, the full version will contain a broader vision of how I plan to develop the capability. Once the software has reached a minimum viable stage of development, I will aggressively promote this simulation capability in paper, at conferences, and on social media.

On the software/business side, I will probably try to learn how to build a SaaS product by implementing cloud-based calculations as an alternative to local calculations and setting up all the relevant payment processing infrastructure. Cloud-based Monte Carlo simulations are enticing in that independent samples can be all computed simultaneous without affecting the overall monetary cost of the simulation, or at least they can be parallelized until the overhead of initializing a container or the granularity of cloud computing costs become a noticeable contribution to the overall cost. While I don’t expect this to be even remotely profitable, it will be an excellent excuse to learn the skills that I will eventually need to develop SaaS-based semiempirical electronic structure methods.

While high-cost, high-accuracy quantum many-body solvers are not directly relevant to the development of semiempirical electronic structure methods, they will be essential for generating reliable reference data with which to test new low-cost correlation models. Limitations in our ability to generate or gather reliable reference data is a significant problem when developing correlation models - we often end up testing them against the limited, skewed data that is available rather than data that more accurately reflects their intended function. For example, exact many-body calculations are only viable for very small system sizes or systems with very special structure. Certain types of very desireable reference data such as potential energy surfaces for bond breaking within a moderately-sized molecule are just extremely difficult to generate with the simulation tools that are widely available right now. Also, I plan to develop some variant of the random-phase approximation (RPA) as my eventual low-cost correlation model, and I’d like to be able to compare it to more conventional coupled-cluster calculations in cases where they are both likely to do poorly (e.g. in the presence of strong static correlation), which again requires a reliable reference method.

The transition from short-term goals to long-term goals will occur as the VMC method and software matures and I begin to test RPA-based correlation models against it. These low-cost correlation models will also be developed openly and with an open-source software implementation. However, at this time I will also initiate the development of new linear-scaling solver algorithms and new semiempirical Hamiltonian models, the details of which will remain proprietary unless I can find a reliable patron for semiempirical electronic structure research. I will continue to discuss high-level information about this proprietary development - enough to articulate its progress and capabilities, but not enough to replicate it precisely.

Just as appreciating the value of maximal openness in basic research and open-source software development helped me to become a better scientist almost a decade ago, I now appreciate the necessity of carefully restricting access to some information to survive when support for basic research is far from sufficient to fund the activities of all capable scientists and competition for scarce resources becomes fierce. I don’t think it is the lack of openness itself that is necessarily bad for science, but rather a lack of clarity and honesty about what information is open and what is closed. I hope that this attitude adjustment helps me to become a more successful scientist in the future, and I will continue to document that journey here in my blog.

I’d also like to briefly take stock of how what I’ve actually been up to matches the plans that I articulated in my last blog post. I noted a queue of 4 planned papers. The first paper was an electronic structure application of the Cauchy kernel paper that was the subject of my last post. I ended up putting that on hold, but I will now be finishing that up for the SIAM Annual Meeting in July (I was invited to present a talk and hopefully the conference won’t be cancelled). The second paper was a final revision of the quantum Metropolis algorithm paper that I presented at the 2019 APS March Meeting. I heavily revised the paper and improve it substantially, but I uncovered an important technical problem at the end of the revision process (the repeat-until-success loop of the algorithm has a fat tail of success probabilities that can cause a divergence in average run times) that will necessitate yet another major revision later this year. The third paper was a very old project on designing molecules with extremely low ionization potentials, which I resumed technical work on but had to pause as the 2020 APS March Meeting grew closer. The fourth paper is the subject of this blog post. As suggested by this mismatch between intent and outcomes, scientific research is often fraught by large uncertainties and frequent delays.

This post is mainly about my latest preprint, “Robust decompositions of quantum states”, which I was planning to speak about at the 2020 APS March Meeting before it was cancelled because of the global coronavirus outbreak. I’m trying to make the best of this situation, and I’ve instead recorded my talk on YouTube. Perhaps it will eventually reach a wider audience than it would have at the conference. Indirectly, this project has an exceptionally long history. I started trying to develop new computational methods for simulating quantum many-body systems before I even started graduate school in 2002. Like many of my research passions, I had a lot of enthusiasm, lofty goals, and high standards, which resulted in a steady stream of failed ideas for years on end. I was particularly obsessed with trying to solve the 2D Hubbard model, which was and still is an important, unsolved model of strong electron correlations and possibly electron-mediated superconductivity. My modus operandi at the time was to focus on whatever methodological idea seemed promising at the moment, mess around with it and stress test it in some way until it inevitably broke, and then brainstorm up a new idea to focus on that was informed by the past failures. At times, the churn on this approach became super high, and ideas sometimes wouldn’t last more than a day. Those were fun times, but I wasn’t able to produce papers from this process, which I eventually acknowledged to be a serious career problem. As a post-doc, I decided to commit to one last, best idea for treating strong quantum correlations that only produced an incomplete preprint after the preliminary numerical results that I generated turned out to be absolutely terrible. After that, I stopped working on strong-correlation methods and started working on more modest weak-correlation methods based on the random-phase approximation (RPA). My RPA work was a lot more successful, and I ended up with a published paper that I am still quite proud of. Unfortunately, I had to stop working on RPA methods when my funding for electronic structure research at Sandia National Labs suddenly dried up, and I was pushed into working on quantum computing research instead.

Working on quantum computing research at Sandia National Labs was a frustrating experience. I was very interested in developing simulation methods for quantum systems, but funding for electronic structure research in very pragmatic and heavily applied environments like Sandia and the other DOE weapons labs (Livermore and Los Alamos) has been on a long slow-and-steady decline because such methods just don’t have enough practical value. One of the big rationales for quantum computing, both at Sandia and more broadly, is to improve our ability to simulate quantum systems, but it is absolutely bizarre, counterintuitive, and counterproductive that we would be increasing our investment in developing quantum algorithms for quantum computers that we don’t have yet while steadily decreasing our investment in developing classical algorithms for classical computers that we have plenty of and are arguably using very inefficiently. As a trendy research topic, quantum computing gets the benefit of the doubt, while the older and no-longer-trendy topic of electronic structure doesn’t anymore. It is especially bizarre given that researchers generally agree that (1) a lot of methodology is shared, so that improvements to classical algorithms will have residual benefits for quantum algorithms, (2) quantum computers will likely supplement or accelerate simulations that mostly occur on classical computers, and (3) the benefits of quantum computers for quantum simulation are asymptotic in accuracy and system size, so they might not have any benefits at all for the accuracies and system sizes that people normally deal with. While I learned about quantum algorithm development and made numerous attempts at developing new quantum algorithms at Sandia, I also tried my best to inject classical algorithm development wherever I could get away with it. This was also an excuse to return to research in strong-correlation methods, now considering a broader set of tools and ingredients including classical algorithms, both deterministic and stochastic, and quantum algorithms.

I did not have a clear agenda or program of research when I started working in quantum computing. At first, I spent a lot of time just learning the subject and catching up with several decades of research. Eventually, after a period of more exploratory efforts, I settled into a program of quantum computing research that I have now carried with me beyond my time at Sandia. The premise is that someday, once we finally have large digital quantum computers in a few decades or so, we will carry out atomistic simulations containing both classical and quantum degrees of freedom that are distributed over classical and quantum computers. I am interested in delineating these classical/quantum boundaries in both simulation and computation: (1) what degrees of freedom should be treated classically, semi-classically, or fully quantum mechanically and (2) how should we partition the simulation between classical and quantum computers and how should they be coupled together? My approach to resolving these boundaries right now is a two-pronged attack aimed at developing better classical algorithms to push on the classical side of the boundary and better quantum algorithms to push on the quantum side of the boundary, so that the boundary is resolved evenly from both sides.

The quantum side of this research program is much farther along, as it was developed first. For now, it mainly consists of two parts: quantum error correction (QEC) algorithms and quantum Metropolis algorithms. The overall cost of quantum computing depends on multiple layers: the underlying physical qubit technology, the QEC layer, and the algorithms layer. For reasons of long-term career stability, I am very loath to commit to a specific physical qubit technology, so I’ve instead focused on the other two layers. I wrote several papers on QEC at Sandia, and I find the topic to be very interesting, but it just isn’t a very popular research topic right now. While I have a lot of ideas for future projects, my QEC research has been on hold for several years now, and I don’t plan to return to it until a logical qubit has been experimentally realized. My interest in QEC is now more about patenting essential components of future quantum computers rather than purely academic curiosity, and there is no point in starting a patent clock in a world without logical qubits. Instead, I’ve been working on quantum algorithms for preparing thermal states. This started at Sandia, where I tried to develop a quantum analog of the Langevin thermostat. The development of a quantum Langevin thermostat turned into a horrible technical slog (that I might eventually recount in this blog), but it eventually evolved into the development of a quantum Metropolis algorithm that I released a preprint on shortly before starting this blog. The quantum Metropolis algorithm is very promising but still has a few outstanding technical problems, and I expect to finish it and release a final version of the paper later this summer.

The classical side of this research program also started at Sandia, but it was much slower to develop. The basic premise of the project was that noise and uncertainty in quantum systems must inevitably drive them to some kind of classical limit that is described by classical physics and is efficient to simulate on classical computers. As quantum systems are driven towards that limit, they should become easier to simulate on classical computers even while retaining much of their quantum identity. There are some very nice results in quantum information theory that embody this concept very clearly. Perhaps the best example are quantum stabilizer circuits (Clifford gates with qubit preparation and measurement operations) that include T gates (a non-Clifford gate that enables universal quantum computation) with polarizing noise. When the noise in the T gates is smaller than a known threshold, the noise can be “distilled” away to a negligible amount using a process known as magic state distillation, which is an important ingredient in many plans for digital quantum computers. As the threshold is approached, the efficiency of magic state distillation drops to zero (a diverging number of noisy T gates is needed to distill out a low-noise T gate). When the noise is at or above the threshold, these circuits can be efficiently simulated on a classical computer and are no longer capable of universal quantum computation. While this is a very compelling result, it has very little to do with physical quantum devices or any applications other than QEC. With a lot of popular interest in using noisy quantum devices for quantum simulation tasks without any QEC, I wanted to develop a comparable result that was relevant to simulation (i.e. showing that noise made quantum simulation tasks easier for classical computers). For a long time, this project was too hung up on trying to model specific instances of analog quantum simulators, but realistic noise is messy and does not directly make systems easier to simulate. In fact, a lot of experimental noise is actually very difficult to model and simulate, and makes some noisy quantum systems even harder to simulate than their idealized versions. I finally started to make progress when I gave up on realistic physical noise and decided to design noise that was tailored to making quantum systems easier to simulate. This noise was based on qubit measurements, which introduced a stochastic component that effectively made it a type of quantum Monte Carlo method. I had always discounted stochastic algorithms in my previous attempts at developing strong-correlation solvers because of the inherently high cost of reducing sampling errors, but here it was natural and inevitable in mimicking the capabilities of quantum computing hardware.

In a way, the development of classical algorithms to simulate quantum systems by introducing noise that saps them of their inherent quantum computing power is kind of like developing “anti-quantum” algorithms, at least for the purposes of a catchy title for a blog post. As I will discuss more in my next blog post, I now feel a strong need to consolidate my research into a more narrowly focused technical path. My end goal on the timescale of several years is still to develop a new generation of semiempirical models, but that research area is extremely unpopular now (making it impossible to get any sort of support for it or career benefit from it) and in need of a major technical overhaul. Right now, I believe that further developing the results of this project, both the theory and software implementation, will both help with the technical overhaul of semiempirical models (by distinguishing between Hamiltonian model errors and correlation model errors) and be a direct benefactor of it (by gaining access to simple model Hamiltonians that are quantitative representations of real molecules and materials). I’m trying to balance my research program so that it aligns as much as possible with what I want to work on and what I believe to be the best science that I am capable of doing while also maintaining connections to popular research topics that give me a better chance of eventually finding financial support to sustain a research career that is presently in a slow but inevitable decline.

I have always enjoyed math a lot, since early in grade school. It was my favorite subject all throughout high school (I was underwhelmed by my science classes until college), and was even on several competitive high school math teams. I would have been a math major in college (rather than the hedge of a dual math/physics major), but I wanted to pursue academic research and was repeatedly told that a math degree could only lead to an actuarial career. While I strayed from a more mathematical career by going to graduate school for physics, my research interests in physics have always been of a highly mathematical nature. I like to view computational physics as a kind of experimental math, where approximations and algorithms are tested for their relevance to physical simulation. My overly simple grad-school perspective on quantum mechanics was as a physical manifestation of linear algebra, which meant that computational condensed matter physics was just a very complicated numerical linear algebra problem. However, working in quantum information theory for five years have given me a broader perspective on the statistical and complexity-theoretic aspects of quantum mechanics.

I discussed the background on this project a bit in a previous post. Around 2007, I was working on a banded Hermitian eigensolver that was inspired by the physics concept of Wannier functions (a.k.a. Boys orbitals in quantum chemistry). These functions demonstrated that simultaneous spatial and spectral locality was possible, which allowed for more flexibility in divide-and-conquer strategies for eigensolving beyond the one that had already been established at the time. In the course of these numerical experiments, it became necessary to fit low-rank approximations of the Cauchy kernel with a general form

where I was free to choose both \(y_i\) and \(f_i(y)\). For a fixed value of \(y\), this was a standard linear minimax optimization problem. I had previously messed around with linear and nonlinear minimax optimization problems that I thought would be useful in electronic structure (for example, my paper on rational approximations of Fermi-Dirac functions originated from earlier attempts during grad school), so I already had some experience with problems of this form. I proceeded as I had in the past, with an ad-hoc implementation of the Remez algorithm. This time, I could make use of the analytical expressions for Cauchy matrix inverses and determinants in fitting the approximant at a set of max-residual points. It turned out that the relative error was what would lead to the tightest error bounds in my intended application, so I adapted my solution to minimize relative error. I then noticed something very surprising: the location of the maximum residual points did not change when I changed \(y\). Because of the Cauchy matrix inverse formula, this meant that the optimal \(f_i(y)\) were rational functions rather than something more general and arbitrary. Also, I was observing much more regular trends in the solutions than in other problems that I had solved, such as the maximum residual points and optimal \(y_i\) values for different \(r\) values collapsing onto a common function just like with Chebyshev nodes.

I don’t keep detailed research records (I’ve mostly operated under the assumption that published papers are record enough), but at some point either in late grad school or early in my academic post-doc at UT Austin, I decided that this result was probably optimal with respect to maximum relative error from a basic dimension-counting argument (I couldn’t see how a more general functional form could reduce the maximum error) and I wanted to prove it. I’m not really big into proving things because it’s hard and you aren’t really rewarded for doing so as a physicist. However, I do think electronic structure simulations need a more rigorous underpinning, and there is probably a small set of core results waiting to be discovered that are important enough for a formal proof. I decided that this was one such result. My basic proof strategy was to prove separate upper and lower bounds, whereby optimality was proven if the bounds were equal. I proved the upper bound very quickly because it was a very straightforward task of restricting a minimization to a simple subset of the original domain. However, the lower bound just kicked my ass and I got nowhere in explaining all the special structure that I observed, so I eventually gave up and put it aside.

I finally put together the lower-bound proof while at Sandia, but only after returning to it multiple times for brief but intense bursts of activity. Around summer 2012, I finally learned about Zolotarev’s work in applying elliptic integrals/functions to rational approximation. This was very exciting at the time, because it explained all of the special structure that I had observed numerically. I can’t imagine how anyone could make the connection from those numerical observations to the theory of elliptical integrals, and indeed Zolotarev had worked in the other direction - his adviser, Chebyshev, had instructed him to look for useful applications of elliptic integrals. Unfortunately, my excitement died down when I realized that none of this made a lower-bound proof any easier, although I could now rule out the previously-unexplained structure as being relevant to a proof. I did eventually cobble together a lower-bound proof as I was exposed to the necessary mathematical ideas while at Sandia. I worked on an idea for relaxing classical stat mech problems into linear programs, which was itself a catastrophic failure, but it increased my familiarity with convex optimization and linear programming (two linear programs being essential steps in my proof). Also, I was exposed to many lower-bound proofs of classical computational complexity in quantum information theory that were a part of complexity-theoretic separation proofs between classical and quantum computing power. While none of those proofs were directly relevant, I learned about the wide variety of clever tricks and strategies that are often employed in producing lower bounds for optimization problems. I finally had a serviceable draft of a proof in summer 2017, which I polished up a few months ago for this paper. I probably could have wrapped this up a lot sooner if Sandia was supportive of my research interests, but it was not, and research tends to go a lot slower when you are being paid to do something else as your full-time job.

This is my oldest research project that has been brought to fruition, and hopefully it is a sign of maturity and a precursor to other results following suit (albeit very slowly). My initial proof in summer 2017 was tied to a paper on banded Hermitian eigensolvers, and it collapsed under the weight of trying to accomplish too much at once. Up until three weeks ago, this paper was tied to three applications that would have further delayed it by many months if I hadn’t cut them. I have a natural tendency to make big plans for papers, and I need to actively fight that by breaking up projects into smaller publishable pieces. Having shorter, more numerous papers is advantageous for many reasons, and it is almost essential for surviving the rat race of modern academia.

My immediate next research priority is finishing up the first delayed application of this paper to electronic structure calculations. However, I need to do a better job of mixing blog posts about active research papers with other research discussions about other interests that are in a more passive mode. I’d like to start discuss quantum computing on this blog soon, and I have two quantum computing papers lined up as #2 and #4 in my publication queue. I also continue to refine and develop my semiempirical electronic structure plans, and I will discuss that at some point as well. I am happy to have finally finished this long-delayed paper, but my research agenda and career are very deep in a hole and it will take a lot of work before I can escape that hole. There will be a lot more research triage in my future.

As per my last blog post, I’ve begun to build some very simple semiempirical electronic structure models to gain some hands-on experience (and to co-opt other work responsibilities). I’ve already done a lot of background research in this area, so I was prepared to put the problems that I encountered into an informed historic perspective. I put together a model inspired by the popular modified neglect of diatomic overlap (MNDO) approximations, and I encountered one of the basic, well-known problems that result from them: a lack of repulsion between closed-shell atoms/molecules. I avoided using the historic correction of a short-ranged repulsive interatomic potential because I was trying to show how a simple quantum-mechanical model of noble gases produces a Lennard-Jones-like potential for the “correct” physical reasons. I was admitting failure if ended up with a model that already contained the repulsive half of a Lennard-Jones potential. Instead, I put in an extra short-ranged repulsive electron-ion potential inspired by pseudopotentials, which achieved the same outcome but at least appeared to be “more quantum”. However, I didn’t have time for enough analysis to really understand the root causes of this problem, as there are multiple possible culprits. There is a thorough recent assessment on the neglect of differential diatomic overlap (NDDO) approximation of the 2-body Coulomb integrals in MNDO-based models, while the main effort to repair the underlying MNDO flaws has focused on the effects of atomic-orbital orthogonalization. If I want to build new semiempirical models and avoid past mistakes, then I need to understand and the leading sources of error and their relative importance.

Since I just learned how to use Jupyter notebooks and had a positive experience with them, I decided to carry out my MNDO analysis in a Jupyter notebook. In retrospect, there was enough of an emphasis on analytical manipulations (that I did by hand) over numerical computations that it would have probably been easier to use a Mathematica notebook here. However, most of my work is heavily numerical, so I’ll probably stick with Jupyter notebooks and the conveniences of NumPy. The simple example that I used for this analysis was a Hartree-Fock calculation of a helium dimer with a single-Gaussian atomic orbital basis for each atom. This is of course a very crude description of atomic effects, but it was able to preserve interatomic effects reasonably well while enabling the entire calculation to be done by hand without too much effort. I was a little bit surprised by the outcome at first, but it wasn’t so surprising in hindsight. In the absence of covalent bonding, NDDO errors were small and irrelevant. The problem was specifically that the on-site 1-body matrix element (excluding the electron-ion Coulomb interaction) was a constant and did not depend on interatomic separation. This is appropriate for a non-orthogonal basis, but the orthogonalization process induces an effective short-ranged repulsive electron-ion interaction that overwhelms the long-ranged attractive Coulomb interaction when the orbitals of two atoms begin to overlap appreciably. By introducing this repulsion as an interatomic potential rather than an electron-ion potential, MNDO-based models suppressed both the occupation-dependence of this energy contribution and its effect on the electronic structure. This is a sub-optimal design choice that in effect makes an additional approximation of a fixed average electronic occupation of atoms in some parts of the model, thus causing problems for atoms that can occur in multiple charge states. However, it doesn’t cause any catastrophic problems, and the semiempirical modeling process is flexible enough to smooth over the effects of various approximations. Unfortunately, this limits the transferability of models between different atomic charge states and exhausts more of the model parameters in correcting avoidable errors.

This ambiguity between ion-ion and electron-ion potentials has persisted to the present day in popular semiempirical electronic structure models. The direct decendents of MNDO (AM1, PMx, etc.) continue to use repulsive interatomic potentials and continue to increase their complexity with an increasing number of diatomic model parameters. The completely independent DFTB family of semiempirical models, the most recent variant of which is GFN2-xTB perform their electronic structure calculations in a non-orthogonal basis, which avoids the approximations that caused the lack of repulsion in MNDO-based models. However, DFTB models still include a short-ranged interatomic repulsion between neutral reference atoms even though they have a flexible enough form to capture this correctly as a purely electronic contribution to the total energy. Because these models are primarily fit to total energy data (or similiarly heats of formation or total energy differences or structural properties) rather than electronic properties like excitation spectra, the more complicated model forms begin to develop multiple ways of approximately fitting the same data. This creates a poorly conditioned parameter fitting process, where there are nearly redundant parameters with respect to the reference data that is being fit. It isn’t simply a matter of not having enough data, it is that the combination of limited scope and flexible model forms has produced fundamentally underconstrained models that do not attempt to predict diverse enough physical observables to define themselves unambiguously.

To conclude this discussion, the “original sin” that I teased in the title of this blog post is the limited scope of semiempirical electronic structure models that limits their ability to define model parameters robustly. They are models of electrons and ions that have not constrained their electronic subsystems with a broad enough set of electronic properties. The ion-ion/electron-ion potential ambiguity was simply the first and most consequential symptom of this underlying problem. If these models were also constrained to predict atomic charges (which unfortunately need to be artificially defined in some way), then they would have more rigorously constrained the electron-ion potential. This is analogous to Kohn-Sham density functional theory (DFT), where the modeling of the electronic density formally constrains the electronic potential in a 1-to-1 mapping. Similarly, pure interatomic potential/energy models are simply functions of atomic coordinates and well-behaved functional forms can be systematically increased in complexity as more reference data becomes available to constrain them. As a result, both DFT and interatomic potentials have been better defined and overall more successful modeling activities than semiempirical quantum chemistry in recent decades. Even some notable missteps reinforce my point: DFT reference data has been emphasizing total energies much more than electronic densities, and the accuracy of electronic density predictions has begun to degrade in some models as a result. Semiempirical electronic structure models need more constraints to remain well-defined as they introduce more parameters, either from direct connections to ab-initio quantum chemistry or from predicting a broader set of physical observables (even if they are less capable of predicting those observables accurately). Two examples of the former approach are the OMx models and Michael Dewar’s last model, SAM1, that both obtain more of their matrix elements directly from ab-initio quantum chemistry.

My research plans in semiempirical electronic structure are evolving, as I explore and learn more and as circumstances change. I will discuss these plans and the thought process behind them in an upcoming blog post. In particular, I’m trying to stake out some practical short-term research projects to add value to semiempirical electronic structure as soon as possible to gain more clout in this research area while isolating, deferring, and organizing my more ambitious, risky, and technically sophisticated ideas into a longer-term research plan. Without some tangible short-term successes, this research effort probably won’t survive long enough to have a “long term”.

The 2017 MolSSI summer school used a QM project that was derived from preliminary versions of the Psi4NumPy tutorials. There were pros and cons to this approach. It encapsulated most of the mountain of complexity that is modern Gaussian-based quantum chemistry inside of Psi4 and its convenient Python wrapper so that students could focus their attention on a few of the key computational tasks such as the self-consistent field cycle of a Hartree-Fock calculation. This is in many ways consistent with the practical realities of most computational chemistry research - not too many people poke at the low-level guts of either quantum chemistry software or theory, it is taken as a given by most people for most projects. However, I wanted to give students some exposure to QM as something that can be made simple and self-contained and fully rationalizable, just so that they can have a simple proxy in their minds when working with more complicated QM software and theories in the future. It would also be more consistent with our MM project, which is a self-contained Metroplis-based Markov-chain Monte Carlo simulation of a Lennard-Jones potential. Unfortunately, it is difficult to make a QM calculation as simple as an MM calculation.

Of course, building a simple QM model and simulation software was a good chance to mess around with semiempirical electronic structure. It was also very appropriate for the setting. The MolSSI summer school was hosted this year at the Texas Advanced Computing Center (TACC), which is located right next to the Mopac Expressway. MOPAC is also the name of the most popular special-purpose semiempirical quantum chemistry code, which is a pun rather than a coincidence. MOPAC was developed in the early 1980’s by Jimmy Stewart while working in Michael Dewar’s group at UT Austin. My QM project is far, far more modest than MOPAC, the goal being to simulate clusters of Argon atoms so that we could couple the QM project with the Lennard-Jones-based MM project as an exercise in QM-MM coupling using the MolSSI Driver Interface. I wanted to get the “right answer for the right reason”, so the project performed both a Hartree-Fock calculation to capture the short-ranged repulsion between Argon atoms and a second-order Moller-Plesset (MP2) calculation to capture the weak long-range attraction. I also wanted to keep everything as simple as possible to be able to present the project in my short block of teaching time (3 hours). The development process and final product are contained in a GitHub repository. It took me a while to put this together because it was a lot of firsts for me: my first attempt at building a custom semiempirical model, my first use of Jupyter notebooks, my first serious scientific calculation in Python, and my first attempt at assembling teaching materials for software development. Overall, I’m happy with the outcome, particularly in how it is emphasizing the utility of a tight coupling between theory and implementation. The entire model is derived within the notebook, and each mathematical expression is implemented in software as soon as it is defined in theory. I also enjoyed working with Jupyter notebooks, and I plan to use them in the near future in conjunction with this blog.

Finally, I ran into an interesting problem while developing the QM project that will be the subject of my next blog post in a few days. My simple semiempirical model, which relied on common approximations and forms (e.g. Slater-Koster tight binding and the neglect of differential diatomic overlap), failed miserably at producing a strong repulsion between closed-shell atoms at the Hartree-Fock level. I ended up having to introduce a repulsive pseudopotential to fix the model, and other models commonly use a repulsive interatomic potential to compensate for this well-known problem. Having this somewhat artificial additional correction term complicated the fitting process of model parameters to reference data, which is another common issue with semiempirical modeling. In retrospect, this felt like a very unacceptable and avoidable problem (i.e. the right answer for the WRONG reasons), and I will examine it in more detail in my next blog post.

A few months ago, I decided to go through the exercise of attempting to secure research funding through the standard government channels (NSF, DOE, DOD). Because of massive cost overheads and various governmental interagency restrictions, I had few opportunities to apply for funding while at Sandia, so much of this was new to me. The prevailing outlook on research funding is rather bleak these days, therefore I wasn’t expecting much. Also, there weren’t any specific calls for proposals relevant to my interests when I started looking, thus I was limited to open funding calls that are very unlikely to have spare money lying around. Of the five relevant program managers that I contacted, one never replied to emails, two said they had no interest in semiempirical modeling, one said they were interested but had no funding available now or for the foreseeable future to warrant continuing a conversation, and one was interested and willing to entertain a white paper (i.e. a pre-proposal). Overall, this was a very chilly reception, but it did give me an opportunity to write a white paper and get some feedback. My initial blog posts were a good dry run for proposal writing, and you can see for yourself how that coalesced into a more polished and focused white paper. The program manager said that he liked my white paper, but that he was unable to fund it at this time. With better timing and more demonstrable experience in this research area, I think that I’ll have a decent chance at getting funding for semiempirical electronic structure at some point in the next few years.

While I’m not quite ready for it yet, I might also try to solicit research funding from industry somehow. US government R&D spending as a fraction of GDP has been in decline for decades, but US industrial R&D spending has risen to more than pick up that slack. The problem is that private companies don’t usually spend much of their money on basic research, and there aren’t any standard channels for soliciting research funding from them. There are growing concerns that dwindling opportunities in basic research may eventually limit our overall system of industrial innovation if these trends continue. Semiempirical electronic structure has historically been used a fair bit in industry, so it is in a better-than-average position than most research topics to attempt to tap into these nontraditional, reluctant funding sources.

I am also going to adjust my research priorities a bit to align them more with popular research trends. Quantum computing research is rapidly growing in popularity, and I have a healthy amount of research experience in that area from my time at Sandia. The initial technical thrust of my semiempirical electronic structure effort is to develop more efficient models for quantum correlations, which conceptually is about further resolving and sharpening the boundary between classical and quantum physics. While my ultimate goal is to build very limited and relatively cheap models of electron correlation, it will be easier to build these models if I have more expensive and reliable methods for generating reference data. In particular, I want reference data for quantum many-body systems at finite temperature, which is an especially difficult problem with few applicable methods available (and none that are reliable). Thus I plan to simultaneously tap into the hype of quantum computing and quantum simulation while building essential computational tools for my underlying long-term research goals. I will discuss my views on quantum computing research and these specific research plans in future blog posts.

I wish that we lived in a world where all technically capable scientists were free to spend all of their time exploring their research interests for the common good (e.g. the concents of Neal Stephenson’s Anathem). Unfortunately, we live in a world that is far from this ideal, and so scientists inevitably spend a lot of time trying to figure out how to survive in their chosen careers. There are many scientists whose careers are in much better shape than my own, but probably far more whose careers are in worse shape.

In line with this new blogging style, I will refrain from talking about what I’ve been up to lately in this post, and instead explain it over several posts spread out over the next few days.