Organize our rust code into crates/ directory

All of our current crates are scattered in the top level of the repository, it's not very intuitive for newcomers to look at the code and know that common directory is for shared rust code, and that wtf8 is also a crate for example.

I propose that we create a new directory called crates where we will have the following:

I think it be better if we do those changes with many smaller PRs as not all of them would require changes to the CI, or any other changes. while minimizing the merge conflicts frequencies while we are in this transition.

For project examples that uses this workspace structure you can see:

@youknowone

@youknowone can you please take a look? feel free to close it if you think this change is not worth doing

@coolreader18

I am personally not a fan of that structure when a project is a single build system based.
For RustPython, we only use cargo. Also it only has Lib to contains Python source code, and every other directories are basically rust crates.

But I am open to follow the convention if this is a well-known good practice. Do you have a good reference to learn why people decide to that or a good discussion what's the benefits of it?

cc @coolreader18

But I am open to follow the convention if this is a well-known good practice.

It's not an official convention, or a "good practice" or a "bad practice", but it's something that I think would be beneficial for us.

Do you have a good reference to learn why people decide to that or a good discussion what's the benefits of it?

Not really, the only thing I could find is this: https://matklad.github.io/2021/08/22/large-rust-workspaces.html

For me at least, is to flatten the codebase while not cluttering the repo's top level.

From what I see, some projects uses the crates/ structure and some don't, but those who don't still name their rust directories with <project_name>_<component>, such as:

So a better approach would be to rename compiler/codegen -> rustpython_compiler_codegen for example?

At the end of the day it's a matter of personal taste, and not a hill I'm willing to die on.

I'm in favor of reorganizing everything into a crates/ directory, since when the project first started I think there was much less of a consensus in the Rust community on directory structure. However it might make more sense to move it to crates/pylib, crates/common, crates/vm, etc, since the rustpython_ bit before every crate would just be redundant. I prefer a directory rather than just putting them in the root for the same reason we have a src/ dir, it separates the code from the other metadata of the project.

I'm in favor of reorganizing everything into a crates/ directory, since when the project first started I think there was much less of a consensus in the Rust community on directory structure. However it might make more sense to move it to crates/pylib, crates/common, crates/vm, etc, since the rustpython_ bit before every crate would just be redundant. I prefer a directory rather than just putting them in the root for the same reason we have a src/ dir, it separates the code from the other metadata of the project.

I think the main motivation for keeping the rustpython prefix is to avoid confusion when specifying dependencies in Cargo.toml.

Newcomers to Rust (or to specifying git dependencies in general), might struggle to understand how to correctly declare dependencies, especially since we are not publishing to crates.io ATM. It's not immediately obvious that cargo only cares about the name field in a crate's Cargo.toml file, not the directory name.

I can easily imagine someone looking at the repository structure and writing something like:

[dependencies]
vm = { git = "https://github.com/RustPython/RustPython.git" }

I also see lots of other projects use the prefix on directories. That looks actually more major convention than crates/ directory. I vote for putting prefix to be aligned to major convention.

I personally don't like both of them, maybe I am the one with bad taste 😂

@youknowone

@youknowone @coolreader18 How should we proceed?

@coolreader18

#6234 is also pending by this.

@coolreader18 Could you please share a few examples crates/common is also a widely-accepted convention for modern rust projects?

To make things clear, I support more common conventions regardless my taste.

I can easily imagine someone looking at the repository structure and writing something like:

I disagree this point. Every cargo.toml file has crate name. We also have same named crate on crates.io too.

I wish we can make more progress 15th Nov not to block other PRs.

wasmtime does this, e.g. wasmtime-environ is at crates/environ. boa does this, though with a core directory rather than a crates directory. Looking across repos, there doesn't seem to be a single standard - I see full or partial crate name in root or subdirectory. My preference for crates/vm is based on keeping the root directory clean, and also not making paths too long. It's much quicker to type cd crates/vm than cd crates/rustpython-vm, and easier in the eyes to read rustpython-vm = { path = "../vm" } than rustpython-vm = { path = "../rustpython-vm" }. (plus I myself have built up a muscle memory for six years of vim vm/src/... and just adding a crates/ prefix won't be too bad)

@youknowone

wasmtime does this, e.g. wasmtime-environ is at crates/environ. boa does this, though with a core directory rather than a crates directory. Looking across repos, there doesn't seem to be a single standard - I see full or partial crate name in root or subdirectory. My preference for crates/vm is based on keeping the root directory clean, and also not making paths too long. It's much quicker to type cd crates/vm than cd crates/rustpython-vm, and easier in the eyes to read rustpython-vm = { path = "../vm" } than rustpython-vm = { path = "../rustpython-vm" }.

Alright, so I'll move it to crates/pylib, crates/doc, etc. If @youknowone doesn't have a strong stance against it

(plus I myself have built up a muscle memory for six years of vim vm/src/... and just adding a crates/ prefix won't be too bad)

Didn't know you're a vim user 🥳

@coolreader18

@coolreader18 Thank you for references.

@ShaharNaveh That sounds good.

My taste was not the first priority, but I also prefer to use short path for daily tasks.

@youknowone

@youknowone @coolreader18 how should we name compiler/core directory?

crates/compiler_core
crates/compiler-core
?

As same as its package name 2025년 11월 14일 (금) 18:19, Shahar Naveh ***@***.***>님이 작성:

…

*ShaharNaveh* left a comment (RustPython/RustPython#6214) <#6214 (comment)> @youknowone <https://github.com/youknowone> @coolreader18 <https://github.com/coolreader18> how should we name compiler/core directory? - crates/compiler_core - crates/compiler-core ? — Reply to this email directly, view it on GitHub <#6214 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAARB5XSG2PZXE5IUG5TFQD34WNC3AVCNFSM6AAAAACKDINFFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTKMZRG42DMOJQGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

take

Navigation Menu

Repository navigation

Organize our rust code into `crates/` directory #6214

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Organize our rust code into crates/ directory #6214

Description

Activity

ShaharNaveh commented on Oct 24, 2025

youknowone commented on Oct 28, 2025

ShaharNaveh commented on Oct 28, 2025

coolreader18 commented on Oct 31, 2025

ShaharNaveh commented on Nov 1, 2025

youknowone commented on Nov 1, 2025

ShaharNaveh commented on Nov 9, 2025

youknowone commented on Nov 10, 2025

coolreader18 commented on Nov 10, 2025

ShaharNaveh commented on Nov 10, 2025

youknowone commented on Nov 11, 2025

ShaharNaveh commented on Nov 14, 2025

youknowone commented on Nov 14, 2025

ShaharNaveh commented on Nov 15, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Organize our rust code into `crates/` directory #6214