Class ModelServer

java.lang.Object
ai.badmonkey.agentspaces.springai.model.ModelServer
All Implemented Interfaces:
org.springframework.context.Lifecycle, org.springframework.context.Phased, org.springframework.context.SmartLifecycle
public class ModelServer extends Object implements org.springframework.context.SmartLifecycle
Serves the fleet's model requests with this peer's ChatModels. Each worker loop takes a ModelRequest (by model name, or by auction), stamps the attempt, calls the provider, and completes the take with the response, so the answer and the request's removal land together. The take lease is renewed while the provider works; a server that dies lets it lapse and the request goes to another server with the next attempt number.

A streamed request writes the provider's deltas as short-leased ModelChunks, batched by the StreamChunkingPolicy; a ModelCancel stops the stream; and the full answer still completes the take, so a late reader gets it whole. The server never runs tools: in Spring AI 2.0 a ChatModel call returns tool calls to the caller.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static final record 
    ModelServer.Settings
    The durations and limits a server works with.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final int
    PHASE
    Starts after the AgentSpaces lifecycle (Integer.MAX_VALUE - 1024).

    Fields inherited from interface org.springframework.context.SmartLifecycle

    DEFAULT_PHASE

  • Constructor Summary

    Constructors
    Constructor
    Description
    ModelServer(ai.badmonkey.agentspaces.api.space.Space space, ModelCatalog catalog, ModelRequestRouter router, StreamChunkingPolicy chunking, org.springframework.scheduling.TaskScheduler scheduler, String servedBy, ModelServer.Settings settings)
    Creates the server.

  • Method Summary

    Modifier and Type
    Method
    Description
    int
    getPhase()
     
    boolean
    isRunning()
     
    long
    served()
    How many requests this server has answered.
    void
    start()
     
    void
    stop()
     

    Methods inherited from class Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface org.springframework.context.SmartLifecycle

    isAutoStartup, isPauseable, stop

  • Field Details

    • PHASE

      public static final int PHASE
      Starts after the AgentSpaces lifecycle (Integer.MAX_VALUE - 1024).
      See Also:

  • Constructor Details

    • ModelServer

      public ModelServer(ai.badmonkey.agentspaces.api.space.Space space, ModelCatalog catalog, ModelRequestRouter router, StreamChunkingPolicy chunking, org.springframework.scheduling.TaskScheduler scheduler, String servedBy, ModelServer.Settings settings)
      Creates the server.
      Parameters:
      space - the requests space
      catalog - the models served here
      router - which requests to take
      chunking - stream batching
      scheduler - lease renewals and stream flushes
      servedBy - this peer, stamped on every answer
      settings - durations and concurrency

  • Method Details

    • served

      public long served()
      How many requests this server has answered.

    • start

      public void start()
      Specified by:
      start in interface org.springframework.context.Lifecycle

    • stop

      public void stop()
      Specified by:
      stop in interface org.springframework.context.Lifecycle

    • isRunning

      public boolean isRunning()
      Specified by:
      isRunning in interface org.springframework.context.Lifecycle

    • getPhase

      public int getPhase()
      Specified by:
      getPhase in interface org.springframework.context.Phased
      Specified by:
      getPhase in interface org.springframework.context.SmartLifecycle