Multi-Agent Deep Reinforcement Learning for Sustainable Manufacturing with Integrated Microgrid: Benchmarking SAC and PPO

National Institute of Posts and Telecommunications (INPT), Rabat, Morocco

^* e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.
^** e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.
^*** e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.

Published online: 3 August 2026

Abstract

Running a manufacturing line alongside an on-site renewable micro-grid couples two decisions that used to be handled separately: when to run the machines, and how to manage local generation and storage. Both must be settled in real time, and each controller sees only a noisy local part of the plant. We cast the problem as a decentralized partially observable Markov decision process (Dec-POMDP) with five cooperating agents, namely two production machines, a wind turbine, a battery bank, and a backup generator. On this testbed we compare two actor-critic algorithms that treat exploration in opposite ways: Soft Actor-Critic (SAC), which is entropy-regularized, and Proximal Policy Optimization (PPO), which keeps each update inside a clipped trust region. Over 10,000 training episodes SAC earns a 15.6% higher mean episode reward (5990.65 against 5181.75), which we attribute to wider exploration of the joint state space, while PPO produces a 22% tighter reward spread (standard deviation 24.86 against 31.89). The learned storage policies differ as well: PPO cycles the battery across almost the full admissible 10%–90% state-of-charge (SOC) window, whereas SAC holds a narrow 25%–60% band. We discuss what these differences imply for battery wear and scheduling stability, check the results against published baselines and through cross-algorithm agreement, and turn the findings into practical algorithm-selection guidance for industrial operators.