Practical overview
When it fits
Use Awex when an RL training workflow needs a coordinated path for sending updated model weights to inference workers, rather than relying solely on file-based weight loading.
Who it is for
- Teams building reinforcement-learning workflows that use separate or co-located training and inference components.
- Engineers integrating supported training and inference frameworks and coordinating model-weight updates across workers.
Documented uses
- Send updated model weights from training processes to inference workers.
- Coordinate shard metadata and peer-to-peer transfer plans across workers.
- Refresh inference parameters during an RL workflow so rollout phases can use newer weights.
Requirements & limits
- The baseline requirements are Python 3.10 or later and PyTorch 2.0 or later, with device support selected for the runtime.
- The README names Megatron, SGLang and vLLM for role-specific framework use; workers using the same framework must use the same version.
- Available transfer paths include NCCL, Mooncake and AState; Mooncake requires its transfer engine on every participating worker.
- Awex’s architecture includes WeightWriter processes in training, WeightReader components on inference instances, and a job-level MetaServer.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Prepare Awex for a Mooncake weight-transfer job
Install the optional Mooncake transfer engine and configure the same backend and MetaServer address on training and inference sides.
Before you start
- Python 3.10 or later and PyTorch 2.0 or later, with device support required by the runtime.
- A training or inference framework for the role being used; workers using the same framework must use the same version.
- Mooncake Transfer Engine installed on every training and inference worker.
Steps
Install the Awex package and its Mooncake extra using the documented package installation instructions.
Set comm_backend="mooncake" in both the training configuration and InferenceConfig.
Set the same AWEX meta_server_addr in both configurations.
Configure AWEX_MOONCAKE_PROTOCOL consistently across workers: rdma for CPU/CUDA memory on RDMA hosts, or tcp for host-buffer transfers.
Expected outcome
Training and inference configurations are set to use Awex’s Mooncake transfer path with a shared MetaServer address.
Guide limitations
- Mooncake must be installed on every worker participating in training and inference.
- The README notes that Mooncake TCP requires separate performance benchmarking.
- This setup guide does not cover initializing models or executing a weight update.
Start the documented Awex metadata service
Launch the job-level MetaServer and make its reachable address available to training and inference workers.
Before you start
- Awex installed in the Python environment.
- A dedicated process for the MetaServer and an address reachable by all participating workers.
Steps
Start the service with python -m awex.meta.meta_server.
Read the ip:port printed by the process.
Set AWEX_META_SERVER_ADDR to that address in every training and inference worker.
Keep the MetaServer running throughout the job.
Expected outcome
Training and inference workers have the shared MetaServer address needed for Awex service discovery and metadata exchange.
Guide limitations
- The documented quick start assumes the training process group and model are already initialized.
- The guide covers service startup and address configuration, not a complete training or inference deployment.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 14 October 2026
- Original website title
- inclusionAI/Awex · Official homepage title · retained in its source language