-
Notifications
You must be signed in to change notification settings - Fork 80
Address comments for PR #692 #733
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR finishes the refactor requested in #692 by unifying cache configuration around MSCCLPP_CACHE_DIR and consistently namespacing internal C++-backed Python bindings with a Cpp prefix, while preserving the public Python API. It also updates documentation and plan-loading logic to match the new cache layout.
Changes:
- Replace
MSCCLPP_EXECUTION_PLAN_DIR/MSCCLPP_NATIVE_CACHE_DIRwithMSCCLPP_CACHE_DIRacross C++ and Python, and align default plan/cache directories (~/.cache/mscclppplus appropriate subdirectories). - Rename nanobind-exposed C++ types to
Cpp*(e.g.,CppExecutionPlan,CppDataType) and adjust Python wrappers to alias them to the stable public names (ExecutionPlan,DataType, etc.). - Simplify Python package export plumbing (
mscclpp._core,mscclpp.ext) and add a minor formatting tweak to a C++ test file.
Reviewed changes
Copilot reviewed 30 out of 30 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| test/executor_test.cc | Adds a leading blank line before the existing license header; no functional impact. |
| src/ext/collectives/algorithm_collection_builder.cc | Switches default DSL plan directory to env()->cacheDir + "/default" and improves the log message wording. |
| src/core/env.cpp | Replaces executionPlanDir with cacheDir, sourced from MSCCLPP_CACHE_DIR, and logs the new variable instead of the old one. |
| include/mscclpp/env.hpp | Updates the Env API and documentation to expose cacheDir instead of executionPlanDir, describing MSCCLPP_CACHE_DIR. |
| python/mscclpp/utils.py | Imports CppDataType from _mscclpp and continues to expose it as DataType for public consumers. |
| python/mscclpp/ext/algorithm_collection_builder.py | Switches to using CppAlgorithmCollectionBuilder from _mscclpp and updates the singleton wrapper accordingly. |
| python/mscclpp/ext/init.py | Re-exports AlgorithmCollectionBuilder via from .algorithm_collection_builder import * and drops a redundant __all__ indirection. |
| python/mscclpp/_core/compiler.py | Uses CppExecutionPlan, replaces MSCCLPP_EXECUTION_PLAN_DIR / MSCCLPP_NATIVE_CACHE_DIR with MSCCLPP_CACHE_DIR, and updates docstrings and paths to match the unified cache layout. |
| python/mscclpp/_core/comm.py | Refactors to use Cpp* nanobind symbols (e.g., CppCommunicator, CppTransport, CppPortChannel) in imports, type hints, and implementations while keeping the public CommGroup API unchanged. |
| python/mscclpp/_core/buffer.py | Uses CppRawGpuBuffer internally while continuing to expose GpuBuffer as before. |
| python/mscclpp/_core/algorithm.py | Updates to use CppAlgorithm*/CppExecutionPlan/CppDataType underlying types and keeps the high-level Algorithm, AlgorithmBuilder, and AlgorithmCollection interface intact. |
| python/mscclpp/_core/init.py | Simplifies exports to plain star-imports from submodules, relying on those modules’ own __all__ definitions. |
| python/mscclpp/main.py | Changes default-plan installation to respect MSCCLPP_CACHE_DIR and write into <cache_dir>/default. |
| python/mscclpp/init.py | Aliases all new Cpp* bindings to the stable public names (e.g., CppExecutionPlan → ExecutionPlan, CppDataType → DataType) and continues to export them via __all__. |
| python/csrc/switch_channel_py.cpp | Renames Python-exposed types to CppSwitchChannel, CppSwitchChannelDeviceHandle, CppNvlsConnection while leaving the connect_nvls_collective function exported. |
| python/csrc/semaphore_py.cpp | Renames exposed semaphore-related classes to CppHost2DeviceSemaphore, CppHost2HostSemaphore, and CppMemoryDevice2DeviceSemaphore. |
| python/csrc/port_channel_py.cpp | Renames port-channel-related bindings to CppBaseProxyService, CppProxyService, CppBasePortChannel, CppPortChannel, and their DeviceHandle variants. |
| python/csrc/numa_py.cpp | Exposes the NUMA helper submodule as cpp_numa (later aliased to numa in the high-level Python API). |
| python/csrc/npkit_py.cpp | Exposes NPKit bindings under a cpp_npkit submodule; the top-level API re-exports this as npkit. |
| python/csrc/memory_channel_py.cpp | Renames memory channel bindings to CppBaseMemoryChannel, CppMemoryChannel, and their DeviceHandle types. |
| python/csrc/gpu_utils_py.cpp | Renames the GpuBuffer<char> binding to CppRawGpuBuffer to match the new internal naming scheme. |
| python/csrc/fifo_py.cpp | Renames FIFO-related bindings to CppProxyTrigger, CppFifoDeviceHandle, and CppFifo. |
| python/csrc/ext/algorithm_collection_builder_py.cpp | Binds AlgorithmCollectionBuilder as CppAlgorithmCollectionBuilder for use by the Python wrapper. |
| python/csrc/executor_py.cpp | Renames PacketType, ExecutionPlan, and Executor bindings to CppPacketType, CppExecutionPlan, and CppExecutor. |
| python/csrc/error_py.cpp | Exposes the ErrorCode enum as CppErrorCode. |
| python/csrc/env_py.cpp | Exposes Env as CppEnv and replaces the read-only property execution_plan_dir with cache_dir. |
| python/csrc/core_py.cpp | Systematically renames core bindings (e.g., Bootstrap, TcpBootstrap, Transport, Device, Connection, EndpointConfig, RegisteredMemory, Semaphore, Communicator) to corresponding Cpp* names and adjusts helper type names like CppSharedFuture_*. |
| python/csrc/algorithm.cpp | Renames algorithm-related enums and classes (CollectiveBufferMode, AlgorithmType, CommResult, ReduceOp, Algorithm, DslAlgorithm, AlgorithmCollection, CollectiveRequest) to Cpp*-prefixed Python names. |
| docs/quickstart.md | Removes the MSCCLPP_EXECUTION_PLAN_DIR environment variable from the sample command, relying on defaults / the new MSCCLPP_CACHE_DIR mechanism. |
| docs/dsl/results.md | Updates the documented default path for installed execution plans to ~/.cache/mscclpp/default/ to reflect the new cache layout. |
|
/azp run |
|
Azure Pipelines successfully started running 3 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 3 pipeline(s). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 31 out of 31 changed files in this pull request and generated no new comments.
Rename nanobind-exposed C++ types to Cpp*
Replace MSCCLPP_EXECUTION_PLAN_DIR / MSCCLPP_NATIVE_CACHE_DIR with MSCCLPP_CACHE_DIR across C++ and Python.