Class MultiAgentEnvironment

java.lang.Object
neqsim.process.ml.multiagent.MultiAgentEnvironment
All Implemented Interfaces:
Serializable
Direct Known Subclasses:
SeparatorCompressorMultiAgentEnv
public class MultiAgentEnvironment extends Object implements Serializable
Multi-agent environment for coordinated process control.

Manages multiple agents that each control a subset of process equipment while respecting global constraints. Supports various multi-agent paradigms:

  • Independent - Agents act independently with local observations
  • Centralized Training Decentralized Execution (CTDE) - Shared critic during training
  • Communicating - Agents exchange messages before acting

Usage Example:


MultiAgentEnvironment env = new MultiAgentEnvironment(processSystem);
env.addAgent(separatorAgent);
env.addAgent(compressorAgent);
env.setSharedConstraints(globalConstraints);

Map<String, double[]> obs = env.reset();
while (!env.isDone()) {
  Map<String, double[]> actions = getActionsFromPolicies(obs);
  MultiAgentStepResult result = env.step(actions);
  obs = result.observations;
}
Version:
1.0
Author:
ESOL
See Also:

  • Field Details

  • Constructor Details

    • MultiAgentEnvironment

      public MultiAgentEnvironment(ProcessSystem process)
      Create a multi-agent environment.
      Parameters:
      process - the process system

  • Method Details

    • addAgent

      public MultiAgentEnvironment addAgent(Agent agent)
      Add an agent to the environment.
      Parameters:
      agent - the agent to add
      Returns:
      this environment for chaining

    • setSharedConstraints

      public MultiAgentEnvironment setSharedConstraints(ConstraintManager constraints)
      Set shared constraints for all agents.
      Parameters:
      constraints - shared constraint manager
      Returns:
      this environment for chaining

    • setCoordinationMode

      public MultiAgentEnvironment setCoordinationMode(MultiAgentEnvironment.CoordinationMode mode)
      Set coordination mode.
      Parameters:
      mode - coordination mode
      Returns:
      this environment for chaining

    • setMaxEpisodeSteps

      public MultiAgentEnvironment setMaxEpisodeSteps(int maxSteps)
      Set maximum episode steps.
      Parameters:
      maxSteps - maximum steps
      Returns:
      this environment for chaining

    • reset

      public Map<String,double[]> reset()
      Reset the environment.
      Returns:
      initial observations for each agent

    • step

      public MultiAgentEnvironment.MultiAgentStepResult step(Map<String,double[]> actions)
      Take a step with all agents' actions.
      Parameters:
      actions - map of agent ID to action array
      Returns:
      step result

    • getGlobalState

      protected StateVector getGlobalState()
      Get global state from process.
      Returns:
      global state vector

    • computeTeamReward

      protected double computeTeamReward(StateVector state, Map<String,double[]> actions)
      Compute team reward for cooperative mode.
      Parameters:
      state - global state
      actions - all agent actions
      Returns:
      team reward

    • isDone

      public boolean isDone()
      Check if episode is done.
      Returns:
      true if terminated or truncated

    • getNumAgents

      public int getNumAgents()
      Get number of agents.
      Returns:
      agent count

    • getAgentIds

      public List<String> getAgentIds()
      Get agent IDs in order.
      Returns:
      list of agent IDs

    • getAgent

      public Agent getAgent(String agentId)
      Get a specific agent.
      Parameters:
      agentId - agent ID
      Returns:
      the agent

    • getCurrentGlobalState

      public StateVector getCurrentGlobalState()
      Get current global state.
      Returns:
      current state

    • getProcess

      public ProcessSystem getProcess()
      Get the process system.
      Returns:
      process system

    • getCurrentStep

      public int getCurrentStep()
      Get current step.
      Returns:
      step count