Class EpisodeRunner

java.lang.Object
neqsim.process.ml.EpisodeRunner
All Implemented Interfaces:
Serializable
public class EpisodeRunner extends Object implements Serializable
Runner for testing RL environments with simple controllers from Java.

This class allows testing the ML infrastructure without Python by running episodes with simple control policies (P, PID, bang-bang, random).

Usage Example:


SeparatorGymEnv env = new SeparatorGymEnv();
Controller controller = new PIDController("LevelPID", 6, 0.5, 0.1, 0.05, -0.1, 0.1, 1.0);

EpisodeRunner runner = new EpisodeRunner(env);
EpisodeResult result = runner.runEpisode(controller, 500);

System.out.println("Total reward: " + result.totalReward);
System.out.println("Steps: " + result.steps);
Version:
1.0
Author:
ESOL
See Also:

  • Field Details

    • serialVersionUID

      private static final long serialVersionUID
      See Also:

    • environment

      private final GymEnvironment environment

    • verbose

      private boolean verbose

    • printInterval

      private int printInterval

  • Constructor Details

    • EpisodeRunner

      public EpisodeRunner(GymEnvironment environment)
      Create episode runner.
      Parameters:
      environment - the Gym environment

  • Method Details

    • setVerbose

      public EpisodeRunner setVerbose(boolean verbose)
      Set verbose mode.
      Parameters:
      verbose - whether to print progress
      Returns:
      this runner

    • setPrintInterval

      public EpisodeRunner setPrintInterval(int interval)
      Set print interval for verbose mode.
      Parameters:
      interval - steps between prints
      Returns:
      this runner

    • runEpisode

      public EpisodeRunner.EpisodeResult runEpisode(Controller controller, int maxSteps)
      Run a single episode with given controller.
      Parameters:
      controller - the controller to use
      maxSteps - maximum steps (overrides env setting if smaller)
      Returns:
      episode result

    • benchmark

      public EpisodeRunner.BenchmarkResult benchmark(Controller controller, int numEpisodes, int maxStepsPerEpisode)
      Run multiple episodes and compute statistics.
      Parameters:
      controller - the controller to use
      numEpisodes - number of episodes
      maxStepsPerEpisode - max steps per episode
      Returns:
      benchmark result

    • compareControllers

      public List<EpisodeRunner.BenchmarkResult> compareControllers(List<Controller> controllers, int numEpisodes, int maxStepsPerEpisode)
      Compare multiple controllers.
      Parameters:
      controllers - list of controllers
      numEpisodes - episodes per controller
      maxStepsPerEpisode - max steps
      Returns:
      list of benchmark results

    • printComparison

      public static void printComparison(List<EpisodeRunner.BenchmarkResult> results)
      Print comparison table.
      Parameters:
      results - benchmark results to compare

    • repeatString

      private static String repeatString(String str, int count)
      Repeat a string n times (Java 8 compatible replacement for String.repeat()).
      Parameters:
      str - string to repeat
      count - number of times to repeat
      Returns:
      repeated string

    • mean

      private static double mean(double[] arr)

    • std

      private static double std(double[] arr)

    • min

      private static double min(double[] arr)

    • max

      private static double max(double[] arr)