STRU output after #8026: round-trip parsing, missing-property handling, and final-frame consistency #8051

Describe the bug

Follow-up to #8026, reviewed at commit 20a459e87a4a5c866c9e5dd3306196605492345d.

The updated STRU output has three correctness issues affecting restart compatibility and its use for collecting ML training data. Suggested priority: high, because failures include silent corruption of movement flags or exported labels.

1. Newly emitted force fields break STRU round-trip parsing.

The writer emits f fx fy fz, but parse_atom_properties() does not recognize f. After the unrecognized token, digits in the force values enter the legacy movement-flag parser. Reading decimal values into integer flags then sets the stream's failure state, preventing subsequent properties/records from being read correctly.

For example, the atom-record suffix m 1 1 1 f 0.000000 0.000000 0.000000 mag 0.8333 changes movement flags to 0,0,1 and fails before reading mag in the minimal reproducer below.

Code: writer, reader.

2. Uncomputed forces and stress are exported as physical zeros.

The driver always allocates a zero-initialized nat × 3 force matrix, even when cal_force=false. The writer checks only its dimensions, so an ordinary SCF run with out_stru 1 and force calculations disabled exports zero forces. The final stress header is also written when cal_stress=false.

An unavailable property is therefore indistinguishable from an evaluated zero value, which can silently contaminate training labels.

Code: allocation, conditional evaluation, force availability check, final output.

3. Early termination can pair the final geometry with properties from another geometry.

The loop evaluates properties, then calls relax_step(), which may move the geometry. If the run subsequently stops because of EXIT or relax_nmax, final_out() receives the proposed next geometry, while energy/force/stress still belong to the preceding evaluated geometry.

The energy/stress mismatch predates #8026; that PR extends it to the newly exported forces. This distinction is intentional: not all of this failure mode was introduced by #8026.

Code: driver ordering, unconverged geometry update.

Expected behavior

• Exported STRU files can be read back without changing coordinates, movement flags, or per-atom magnetic moments.
• Uncomputed forces/stress are omitted or explicitly marked unavailable; they must not appear as evaluated zero labels.
• Every labeled frame contains geometry and properties from the same evaluation, including maximum-step and EXIT termination.
• If a proposed, unevaluated geometry is retained for restarting, it must not carry stale properties from the previous geometry.

To Reproduce

A. Parser failure — reproduced in an isolated C++ harness

Obtain the source at commit 20a459e87a4a5c866c9e5dd3306196605492345d, save the following as repro_parser.py, and run:

python3 repro_parser.py /path/to/abacus-source

The script extracts parse_atom_properties() unchanged from that source and supplies minimal stand-ins for its data types. It compiles with g++ -std=c++11. This isolates the parser behavior; it is not a full ABACUS execution or a complete writer/reader integration test.

from pathlib import Path
import subprocess, sys, tempfile
source_root=Path(sys.argv[1])
root=Path(tempfile.mkdtemp(prefix='abacus-stru-repro-'))
src=(source_root/'source/source_cell/read_atoms_helper.cpp').read_text()
func=src[src.index('bool parse_atom_properties('):src.index('bool read_atom_type_header(')]
preamble=r'''
#include <fstream>
#include <iostream>
#include <string>
#include <cmath>
#include <vector>
namespace ModuleBase {
template<class T> struct Vector3 {T x=0,y=0,z=0;};
constexpr double PI=3.141592653589793, Ry_to_eV=13.605693;
}
struct Atom {
std::string label="Si";
std::vector<ModuleBase::Vector3<double>> vel{1}, m_loc_{1}, lambda{1};
std::vector<ModuleBase::Vector3<int>> constrain{1};
std::vector<double> mag{0.0}, angle1{0.0}, angle2{0.0};
};
constexpr char DIGIT_START='0', DIGIT_END='9', LOWER_A='a', LOWER_Z='z', MINUS_SIGN='-';
'''
main=r'''
int main() {
for (const auto& tail: {" m 1 1 1 mag 0.8333\n", " m 1 1 1 f 0.000000 0.000000 0.000000 mag 0.8333\n", " m 1 1 1 f 2.571104 5.142208 7.713312 mag 0.8333\n"}) {
{std::ofstream out("parser_input.txt"); out << tail;}
std::ifstream in("parser_input.txt"); Atom atom; ModuleBase::Vector3<int> mv;
bool vec=false, angle=false, zero=false;
parse_atom_properties(in,atom,0,mv,vec,angle,zero);
std::cout << "input=" << tail << "movement=" << mv.x << ',' << mv.y << ',' << mv.z << " mag=" << atom.mag[0] << " fail=" << in.fail() << '\n';
}
}
'''
(root/'repro_parser.cpp').write_text(preamble+func+main)
subprocess.run(['g++','-std=c++11',str(root/'repro_parser.cpp'),'-o',str(root/'repro_parser')],check=True)
subprocess.run([str(root/'repro_parser')],cwd=root,check=True)

Observed output:

input= m 1 1 1 mag 0.8333
movement=1,1,1 mag=0.8333 fail=0
input= m 1 1 1 f 0.000000 0.000000 0.000000 mag 0.8333
movement=0,0,1 mag=0 fail=1
input= m 1 1 1 f 2.571104 5.142208 7.713312 mag 0.8333
movement=2,0,1 mag=0 fail=1

Here fail=1 is the C++ input stream's failure state; this reproduction does not assert a particular full-program error message.

B. Missing-property output — identified by static inspection; full-run verification pending

Use an otherwise valid SCF case with:

calculation scf
out_stru 1
cal_force 0
cal_stress 0

Inspect OUT.<suffix>/STRU_FINAL. The current code path emits numeric zero force/stress fields despite skipping both calculations. Test the force/stress flags independently as well, to distinguish missing properties from genuinely computed zeros.

C. Final-frame consistency — identified by static inspection; full-run verification pending

Use a displaced, unconverged relaxation case with out_stru 1 and relax_nmax 1, ensuring the optimizer makes a nonzero move. Compare the last evaluated frame (STRU_NOW) with STRU_FINAL and the evaluated energy/force/stress. Also exercise termination through an EXIT file after an unconverged step. The driver currently calls final output after the geometry update without evaluating the proposed geometry.

Environment

• Reviewed ABACUS source: 20a459e87a4a5c866c9e5dd3306196605492345d (final head of Update out_stru parameter and its output context #8026).
• Minimal parser reproduction: Ubuntu 24.04.5 LTS under WSL2; GCC/G++ 13.3.0; C++11.
• Harness dependencies: Python 3 and the C++ standard library; no MPI, BLAS, pseudopotentials, or orbital files required.
• The harness was launched from the local abacus-env Conda environment.
• No full ABACUS executable was built or run for this report. Items 2 and 3 are code-path findings, not claims of completed numerical integration tests.

Additional Context

Possible fixes, without prescribing a new file format:

1. Explicitly consume all three components of f in the reader, either storing or deliberately discarding them.
2. Propagate force/stress availability to the output layer rather than inferring it from matrix dimensions. A warning alone does not make numeric zero labels safe.
3. Preserve the last evaluated geometry for labeled output, or omit stale labels from an unevaluated proposed geometry.

** Issue is raised by my ChatGPT 6 Astra with my taste and ABACUS developing guidance

Acceptance criteria:

• Add a STRU writer-to-reader regression covering multiple atoms, movement flags, and per-atom magnetic moments.
• Cover independently enabled/disabled force and stress, including a genuinely computed zero-force case.
• Cover maximum-step and EXIT termination with geometry/property consistency assertions.
• Document the force-field syntax, units, and missing-property behavior.

Related work: #8026. Its reference to #7614 concerns a different STRU-format problem; the report here is specifically about the new f output and frame/label consistency.

Task list for Issue attackers (only for developers)

• Verify the issue is not a duplicate (searched existing issues for 8026, STRU_FINAL, and STRU force; no direct duplicate found).
• Describe the bug.
• Steps to reproduce (isolated parser reproduction; full-run checks explicitly pending).
• Expected behavior.
• Error message (stream failure evidence included above).
• Environment details.
• Additional context.
• Assign a priority level (suggested: high).
• Assign the issue to a team member.
• Label the issue with relevant tags.
• Identify possible related issues.
• Create repository regression tests.
• Fix the bugs.
• Test the fixes.
• Update documentation.
• Close the issue after verification.