Skip to content

decimal: add a native arbitrary-precision decimal engine - #8699

Closed
moreal wants to merge 7 commits into
RustPython:mainfrom
moreal:agent/decimal-native-engine
Closed

moreal wants to merge 7 commits into
RustPython:mainfrom
moreal:agent/decimal-native-engine

Conversation

@moreal

@moreal moreal commented Sep 13, 2026 •

Copy link
Copy Markdown
Contributor

This pull request is closed because it will be worked in other direction.

A new `rustpython-decimal` crate implements the General Decimal Arithmetic
Specification over `malachite-bigint`: construction and the numeric-string
grammar, the full arithmetic set, comparison and total ordering, the digit-wise
and exponent operations, correctly rounded sqrt/exp/ln/log10/power, and the
format-specification mini-language.

The algorithms are ported from CPython's `Lib/_pydecimal.py`, which is the
executable specification for the parts the standard leaves open; the coefficient
is a `BigUint` with a cached digit count instead of a digit string, so what the
Python does with slicing is done here with divisions by powers of ten.

Conditions follow libmpdec rather than `_pydecimal`: an operation never raises,
it accumulates the conditions it hit into a status word that the caller merges
into a context. That is what lets a trapped signal report every condition an
operation produced rather than only the first.

Assisted-by: Claude Code:claude-fable-5-1
`Lib/decimal.py` starts with `from _decimal import *` and falls back to
`Lib/_pydecimal.py` when that fails. There was no `_decimal`, so every program
using decimals ran the pure-Python implementation.

This wires the `rustpython-decimal` engine up as that module: the `Decimal` and
`Context` types with their whole method surface, the signal hierarchy with
libmpdec's condition ordering, the live `flags`/`traps` views, the
context-variable holding the current context, and `localcontext`/`IEEEContext`.
Signatures carry CPython's text signatures, so `inspect.signature` and the
suite's C/Python signature comparison agree.

test_decimal now runs its C classes as well as its Python ones -- 721 tests
including all 143 IBM `decimaltestdata` files -- and passes. A differential
check of 16000 random operations against CPython's `_decimal` agrees on every
result and every status flag.

pyperformance telco: 348 ms/iteration -> 15.4 ms (23x; CPython 3.14 is 4.5 ms).
Decimal('1.23')*Decimal('4.56') 5275 ns -> 267 ns, quantize 6814 ns -> 423 ns,
str() 1658 ns -> 497 ns.

Assisted-by: Claude Code:claude-fable-5-1
@coderabbitai

coderabbitai Bot commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the z-ca-2026 Tag to track Contribution Academy 2026 label Sep 13, 2026
@codspeed

codspeed Bot commented Sep 13, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will regress 1 benchmark

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
❌ 1 regressed benchmark
✅ 63 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
❌ gc_traversal.py[rustpython] 683.2 ms 826.8 ms -17.37%
⚡ decimal_factorial.py[rustpython] 1,228.9 ms 35 ms ×35
⚡ gc_collect.py[rustpython] 83.4 ms 59.7 ms +39.7%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing moreal:agent/decimal-native-engine (64d35ac) with main (e18facf)

Open in CodSpeed

@youknowone youknowone left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need to reimplement everything ourselves? How about other external libraries? Is there any good decimal libraries?

table
};

thread_local! {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this safe tls?

…eview

- next_plus/next_minus/next_toward: replace a hand-rolled exact-add
  helper that scaled operands without a precision bound (causing
  gigabyte-scale allocations for extreme Emin) with a direct call to
  the already-bounded arith::add.
- bigops::div_pow10/split_pow10: short-circuit to zero when the shift
  exceeds the coefficient's digit count, instead of materializing a
  huge power of ten (e.g. int(Decimal('1e-999999999999999999'))).
- Decimal.__round__(ndigits): honor the context's rounding mode
  instead of hardcoding ROUND_HALF_EVEN, matching quantize's default.
- setcontext(): clear the copied context's status flags when
  installing one of the module's template contexts, matching
  _pydecimal's fresh Context.copy() semantics for that path.
- parse_ascii_raw: clamp an out-of-i64-range exponent instead of
  rejecting the string outright, so context-based conversion still
  produces Infinity/Overflow the way an in-range huge exponent would.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Comment thread crates/stdlib/src/lib.rs
pub(crate) mod macros;

mod _asyncio;
#[path = "_decimal.rs"]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#[path = "_decimal.rs"]

moreal and others added 3 commits September 13, 2026 21:10
- fmt::format: bound the fixed-point padding derived from a value's
  exponent the same way width/precision are already bounded, so
  format(Decimal('1e-999999999999999999'), 'f') raises ValueError
  instead of attempting a ~10^18-byte allocation.
- Decimal.as_integer_ratio: an exponent whose reduced power-of-five
  denominator doesn't fit a u32 (e.g. Decimal('1e-4294967296')) now
  raises MemoryError instead of panicking past an .expect().

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…lloc errors

- fix's clamp branch (Context.clamp=1 with an oversized exponent) now
  checks MAX_MATERIALIZABLE_DIGITS before padding the coefficient with
  mul_pow10, matching every other guarded arithmetic path, instead of
  attempting a ~10^18-digit allocation for a valid context.
- next_plus/next_minus now propagate a MALLOC_ERROR raised by the
  underlying add back to the caller instead of discarding it in their
  scratch status word, so an allocation failure surfaces as
  MemoryError (next_toward inherits the fix through both).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ssing

- fmt::format: check the allocation bound before rescaling a nonzero
  value to an explicit fixed-point precision, not only afterwards, so
  format(Decimal('1e999999999999999999'), '.0f') raises ValueError
  instead of materializing a ~10^18-digit coefficient.
- Decimal.from_number: build through the subclass constructor the way
  from_float already does, instead of allocating the payload directly
  and bypassing a subclass's __new__/__init__.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@youknowone

Copy link
Copy Markdown
Member

ok, I investigated the situation a little bit. CPython decimal expects a specific implementation of decimal - libmpdec. Your patch is not a port of libmpdec, but it implementing generic decimal arithmatic engine so that compatible to libmpdec.

Could you tell what our new decimal crate is referencing? Probably we will not easily reach to perf parity building a new one from scratch. Let's directly port libmpdec into Rust. And place it on RustPython/mpdec-rs. This library must be equally fast as mpdec and the calc result must be exactly same as mpdec. The new repository also must have fuzzy test.

@moreal

moreal commented Sep 13, 2026

Copy link
Copy Markdown
Contributor Author

ok, I investigated the situation a little bit. CPython decimal expects a specific implementation of decimal - libmpdec. Your patch is not a port of libmpdec, but it implementing generic decimal arithmatic engine so that compatible to libmpdec.

Could you tell what our new decimal crate is referencing? Probably we will not easily reach to perf parity building a new one from scratch. Let's directly port libmpdec into Rust. And place it on RustPython/mpdec-rs. This library must be equally fast as mpdec and the calc result must be exactly same as mpdec. The new repository also must have fuzzy test.

I agree that, since the decimal module is used for precise decimal arithmetic, an implementation that does not conform to the specification could have serious consequences.

To be honest, I had set up an iterative process focused on improving pyperformance benchmark results, but I did not adequately verify the implementation’s correctness or compliance with the specification. I take responsibility for letting the work proceed without sufficient validation. I am ashamed of this, and I sincerely apologize.

The approach I had been taking was to port the existing pure-Python _pydecimal implementation to Rust. I had concluded that a pure-Rust implementation was necessary because of the potential impact of using libmpdec through C FFI on WASM and no_std support. However, after considering your feedback, I now think that porting libmpdec itself to Rust, rather than _pydecimal, would be a more appropriate approach.

Thank you for pointing this out. 🙏🏻

CPython's SignalDict mixes in collections.abc.MutableMapping, so
getcontext().traps.update(...)/.setdefault(...) work on top of
__getitem__/__setitem__ alone. The native class only implemented the
mapping protocol itself, so update and setdefault raised
AttributeError. Add update, setdefault, pop, popitem and clear,
matching MutableMapping's default behavior (the latter three always
raise ValueError("signal keys cannot be deleted"), since deletion is
never allowed).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@moreal moreal closed this Sep 13, 2026
@moreal moreal removed the z-ca-2026 Tag to track Contribution Academy 2026 label Sep 13, 2026
@youknowone youknowone added the z-ca-2026 Tag to track Contribution Academy 2026 label Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

z-ca-2026 Tag to track Contribution Academy 2026

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants