MARL for Cross-Chain DeFi Optimization

Production-grade Multi-Agent Reinforcement Learning (MARL) for optimizing cross-chain DeFi operations. Addresses liquidity fragmentation, suboptimal routing, and bridge risks across multiple blockchain networks using QMIX and MAPPO with centralized training and decentralized execution.

• Overview
• Architecture
• Quick Start
• Repository Structure
• Algorithms
• Configuration
• Results
• CLI Reference
• Advanced Usage
• Troubleshooting
• License

Cross-Chain DeFi Environment
├── Multiple Blockchains (Ethereum, Arbitrum, ...)
├── Cross-Chain Bridges (latency and risk modeling)
├── DEX Liquidity Pools (AMM mechanics)
└── Gas Costs and Transaction Execution

MARL Agents
├── QMIX (Value Decomposition)
│   ├── Agent Networks (DRQN)
│   ├── Mixing Network
│   └── Centralized Training, Decentralized Execution
├── MAPPO (Policy Optimization)
│   ├── Actor Networks (shared parameters)
│   ├── Centralized Critic
│   └── PPO Updates with GAE
└── Baselines (Random, Independent Q-Learning)

git clone https://github.com/quantsingularity/MARL-for-Enterprise-Grade-Cross-Chain-DeFi-Optimization
cd MARL-for-Enterprise-Grade-Cross-Chain-DeFi-Optimization/code

pip install -r requirements.txt
./cli.sh all

docker build -t marl-defi .
docker-compose up marl-training
docker-compose up marl-eval

cd code
python src/envs/demo_env.py --steps 20
python src/train/train_synthetic.py --agent qmix --config configs/demo.yaml
python src/eval/evaluate_demo.py --checkpoint checkpoints/qmix_model.pth --agent-type qmix

Console output stays readable: benign third-party warnings and library progress bars are suppressed, so only meaningful log lines remain. Each run finishes with a clean, aligned summary block reporting the evaluation metrics for each trained agent.

code/
├── src/
│   ├── agents/       # QMIX, MAPPO, baselines, communication, GNN encoders
│   ├── envs/         # Cross-chain environment simulator
│   ├── train/        # Training scripts
│   ├── eval/         # Evaluation scripts and metrics
│   ├── data/         # Synthetic data generation
│   └── utils/        # Logging, checkpointing, config
├── tests/            # Unit and integration tests
├── configs/          # YAML configuration files
├── notebooks/        # Environment and training demos
├── requirements.txt
└── cli.sh

Monotonic value function factorization for cooperative MARL. Paper

• Centralized training with decentralized execution (CTDE)
• Monotonic mixing network for credit assignment
• DRQN-based agent networks with GRU cells
• Experience replay and target network soft updates

Multi-Agent Proximal Policy Optimization for cooperative tasks. Paper

• Shared parameter actor network
• Centralized value function
• PPO clipping for stable updates
• Generalized Advantage Estimation (GAE)

• CommNet: Iterative message passing between agents
• Attention-based: Multi-head attention for selective messaging
• GNN Encoders: Graph neural network state representation

Edit code/configs/demo.yaml to customize the environment and training.

environment:
  chains:
    - name: Ethereum
      chain_id: 1
      block_time: 12.0
      base_gas_price: 20.0
  bridges:
    - name: Arbitrum_Bridge
      latency_mean: 600.0
      failure_rate: 0.01
  pools:
    Ethereum:
      - token_a: ETH
        token_b: USDC
        reserve_a: 1000.0

training:
  n_agents: 2
  hidden_dim: 64
  gamma: 0.99
  n_episodes: 50
  device: cpu

Benchmark results over 50 episodes on Intel i7 CPU with demo configuration.

./cli.sh all       # Complete pipeline
./cli.sh data      # Generate synthetic data
./cli.sh train     # Train agents
./cli.sh eval      # Evaluate agents
./cli.sh test      # Run tests
./cli.sh demo      # Environment demo
./cli.sh clean     # Clean generated files

from src.envs.cross_chain_env import CrossChainEnv
from src.agents.qmix import QMIXAgent
from src.utils.logger import MetricsLogger
from src.utils.checkpointing import CheckpointManager
from pathlib import Path

env = CrossChainEnv(chains=[...], bridges=[...], pools={...})

agent = QMIXAgent(
    n_agents=2,
    obs_dim=env.observation_space.shape[0] // 2,
    state_dim=env.observation_space.shape[0],
    n_actions=5,
    hidden_dim=128,
    device="cuda"
)

logger = MetricsLogger(Path("logs/custom_run"))
checkpoint_mgr = CheckpointManager(Path("checkpoints/custom"))

for episode in range(num_episodes):
    logger.log_dict(metrics, episode)
    checkpoint_mgr.save_checkpoint(agent, episode, metrics)

1. Create agent file in code/src/agents/
2. Implement select_actions(), update(), save(), load()
3. Register agent in the training script
4. Write tests in code/tests/test_agents.py

pytest code/tests/ -v
pytest code/tests/ --cov=code/src --cov-report=html

Licensed under the MIT License. See LICENSE for details.