SEP-2322: Multi Round-Trip Requests

Documentation Index

Fetch the complete documentation index at: https://modelcontextprotocol.io/llms.txt

Use this file to discover all available pages before exploring further.

​

Abstract

​

Motivation

1. Ephemeral: No state is accumulated on the server side.
   • If server needs more info to process the tool call, it can start from scratch when it gets that additional info.
   • Examples: weather app, accessing email
2. Persistent: State is accumulated on the server side.
   • Server may generate a large amount of state before requesting more info from the client, and it may need to pick up that state to continue processing after it receives the info from the client.
   • Server may need to continue processing in the background while waiting for more info from the client, in which case server-side state is needed to track that ongoing processing.
   • Examples: accessing an agent, spinning up a VM and needing user interaction to manipulate the VM

1. Client sends tool call request. For this example, let’s assume that the load balancers happen to send this request to server instance A.
2. Server A opens an SSE stream and sends the elicitation request on that stream.
3. Client sends the elicitation response as a separate request, for which the load balancers will choose a server instance completely independently of the one they chose in step 1. In this example, let’s assume that the load balancers happen to send this request to server instance B.
4. Server A must somehow discover the elicitation response delivered to server B.
5. Server A then sends the tool call result on the SSE stream opened in step 2.

• Persistent Storage Layer Shared Across Server Instances: Servers can deploy and manage a persistent storage layer (e.g., PostgreSQL, Redis, DynamoDB), which allow multiple server instances to match up the elicitation response on one server instance with the original ongoing tool call on a different server instance. This approach has a number of drawbacks:
  • The persistent storage layer is extremely expensive, especially for ephemeral tools that may not already have such a layer (e.g., a weather tool).
  • The persistent storage layer imposes significant reliability concerns: it becomes a critical dependency and therefore a potential single point of failure. To avoid that, it must provide high availability, replication, and backup mechanisms.
  • The persistent storage layer becomes a bottleneck, limiting horizontal scalability. Geographic distribution requires either expensive global replication or sticky routing.
  • The persistent storage layer also imposes significant operational complexity. In horizontally scaled deployments, it requires distributed locking or consensus protocols. It also requires special garbage collection logic to determine when shared can be cleaned up, which requires careful trade-offs: cleaning up state too aggressively can reduce storage costs but limit how long users have to respond, whereas cleaning up less aggressively accommodates slow users but increases storage costs.
  • This approach requires special behavior in the tool implementation to integrate with the persistent storage layer. The MCP SDKs today do not have any special hooks for this sort of storage layer integration, which means that it’s very hard to write in-line code via the SDKs.
• Statefulness in Load Balancing: With the use of cookies, it is possible for the load balancing layer to ensure that the elicitation request in step 3 is delivered to the same server instance that the original request was delivered to in step 1. This approach, while often cheaper than a persistent storage layer, has the following drawbacks:
  • It requires special configuration and behavior in the load balancers, which is often difficult to manage.
  • It breaks normal load balancing models, resulting in uneven load distribution, thus increasing the cost of running the service.
  • It requires special behavior in clients to propagate the cookies used for statefulness.
  • It requires the tool implementation to match up the elicitation request with the ongoing tool call. (The MCP SDKs have some code to handle this, but it’s still a very strange pattern in the HTTP world.)
  • It is not fault tolerant. If the server instance goes down, all state is lost, and the tool call would need to start over from scratch. (This doesn’t necessarily matter for ephemeral tools, but it is an issue for persistent tools.)

​

Specification

​

Schema Changes

export type InputRequest =
  | CreateMessageRequest
  | ElicitRequest
  | ListRootsRequest;

export interface InputRequests {
  [key: string]: InputRequest;
}

export type InputResponse =
  | CreateMessageResult
  | ElicitResult
  | ListRootsResult;

export interface InputResponses {
  [key: string]: InputResponse;
}

"inputRequests": {
  // Elicitation request.
  "github_login": {
    "method": "elicitation/create",
    "params": {
      "mode": "form",
      "message": "Please provide your GitHub username",
      "requestedSchema": {
        "type": "object",
        "properties": {
          "name": {
            "type": "string"
          }
        },
        "required": ["name"]
      }
    }
  },
  // Sampling request.
  "capital_of_france" : {
    "method": "sampling/createMessage",
    "params": {
      "messages": [
        {
          "role": "user",
          "content": {
            "type": "text",
            "text": "What is the capital of France?"
          }
        }
      ],
      "modelPreferences": {
        "hints": [
          {
            "name": "claude-3-sonnet"
          }
        ],
        "intelligencePriority": 0.8,
        "speedPriority": 0.5
      },
      "systemPrompt": "You are a helpful assistant.",
      "maxTokens": 100
    }
  }
}

"inputResponses": {
  // Elicitation response (ElicitResult).
  "github_login": {
    "action": "accept",
    "content": {
      "name": "octocat"
    }
  },
  // Sampling response (CreateMessageResult).
  "capital_of_france": {
    "role": "assistant",
    "content": {
      "type": "text",
      "text": "The capital of France is Paris."
    },
    "model": "claude-3-sonnet-20240307",
    "stopReason": "endTurn"
  }
}

export interface InputRequiredResult extends Result {
  // Requests issued by the server that must be complete before the
  // client can retry the original request.
  inputRequests?: InputRequests;
  // Request state to be passed back to the server when the client
  // retries the original request.
  // Note: The client must treat this as an opaque blob; it must not
  // interpret it in any way.
  requestState?: string;
}

// RequestParams type that includes input responses and request state.
// These parameters may be included in any client-initiated request.
export interface InputResponseRequestParams extends RequestParams {
  // New field to carry the responses for the server's requests from the
  // InputRequiredResult message. For each key in the response's inputRequests
  // field, the same key must appear here with the associated response.
  inputResponses?: InputResponses;
  // Request state passed back to the server from the client.
  requestState?: string;
}

/**
 * Common result fields.
 *
 * @category Common Types
 */
export interface Result {
  _meta?: MetaObject;
  // New field to indicate the type of the result, which allows the client to determine how to parse the result object. If no resultType is specified "complete" should be assumed.
  resultType: ResultType;
  [key: string]: unknown;
}

export type ResultType =
  | "complete" // the request completed successfully and the result contains the final content.
  | "input_required"; // the request is incomplete and the result contains an {@link InputRequiredResult} object

​

Server-Initiated Request Support for Client Requests

​

Ephemeral Tool Workflow

1. Client sends tool call request.
2. Server sends back a single response indicating that the request is incomplete. The response may include input requests that the client must complete. It may also include some request state that the client must return back to the server. This response terminates the original request. It will normally be sent as a single response, not on an SSE stream, although for now (this may change in a future SEP) it is also legal to send this response on an SSE stream following (e.g.) progress notifications. If this incomplete response is sent on an SSE stream, it must be the last message on the SSE stream, just as if it were a normal response.
3. Client sends a new tool call request, completely independent of the original one. This new tool call includes responses to the input requests from step 2. It also includes the request state specified by the server in step 2.
4. Server sends back a CallToolResponse.

​

Real-World Example for Ephemeral Workflow

​

Use Cases for Request State

Use Case 1: Rolling Upgrades

1. Client sends tool call request that hits a server instance running the old version.
2. Server sends back an incomplete response indicating the input requests for “github_login” and “google_login”.
3. Client sends a new tool call request that includes the responses to the input requests for “github_login” and “google_login”. This time it hits a server instance running the new version.
4. Server sends back another incomplete response indicating the input request for “microsoft_login”, which the client has not already provided. However, the response also includes request state containing the already-provided “github_login” response, so that the client does not need to prompt the user for the same information a second time.
5. Client sends a third tool call request that includes the response to the “microsoft_login” input request as well as echoing back the request state provided by the server in step 4.
6. Server now sees the “github_login” info in the request state and the “microsoft_login” state in the input responses, so the request now contains everything the server needs to perform the tool call and send back a complete response.

Use Case 2: Load Shedding

1. Client sends the original request, which the load balancers route to server instance A.
2. Server instance A does a bunch of computation before deciding that it needs to shed load. It sends an incomplete response with its accumulated state in the requestState field but without the inputRequests field.
3. Client retries the request with the requestState field attached. The load balancers route this request to server instance B.
4. Server instance B starts from the state it sees in the requestState field, thus picking up the computation from where server instance A left off, and eventually returning a complete response.

​

Protocol Requirements for Ephemeral Workflow

1. Server Behavior:
   • Servers MAY respond to any client-initiated request with a InputRequiredResult. This message MAY be sent either as a standalone response or as the final message on an SSE stream, although implementations are encouraged to prefer the former. If using an SSE stream, servers MUST NOT send any message on the stream after the incomplete response message.
   • The InputRequiredResult MAY include an inputRequests field.
   • The InputRequiredResult MAY include a requestState field. If specified, this field is an opaque string that is meaningful only to the server. Servers are free to encode the state in any format (e.g., plain JSON, base64-encoded JSON, encrypted JWT, serialized binary, etc.).
   • If a request contains a requestState field, servers MUST always validate that state, as the client is an untrusted intermediary. If tampering is a concern, servers SHOULD encrypt the requestState field using an encryption algorithm of their choice (e.g., they can use AES-GCM or a signed JWT) to ensure both confidentiality and integrity. Note that there is also a risk of replaying/hijacking attacks, where an authenticated attacker resends state that was originally sent to a different user. Therefore, if the request state contains any data that is specific to the original user, the server MUST use some mechanism to cryptographically bind the data to the original user and MUST verify that the requestState data sent by the client is associated with the currently authenticated user. Servers using plaintext state MUST treat the decoded values as untrusted input and validate them the same way they would validate any client-supplied data.
2. Client Behavior:
   • If a client receives an InputRequiredResult message, if the message contains the inputRequests field, then the client MUST construct the requested input before retrying the original request. In contrast, if the message does not contain the inputRequests field, then the client MAY retry the original request immediately.
   • If a client receives a InputRequiredResult message that contains the requestState field, it MUST echo back the exact value of that field when retrying the original request. Clients MUST NOT inspect, parse, modify, or make any assumptions about the requestState contents. If the InputRequiredResult does not contain a requestState field, the client MUST NOT include one in the retry.

​

Persistent Tool Workflow

1. Server sets Task Status to input_required. The server can pause processing the request at this point.
2. Client retrieves the Task Status by calling tasks/get and sees that more information is needed.
3. Client calls tasks/result
4. Server returns the InputRequests object.
5. Client calls tasks/input_response request that includes an InputResponses object along with Task metadata field.
6. Server resumes processing sets TaskStatus back to working.

​

Protocol Requirements for Persistent Workflow

1. Server Behavior:
   • Servers MAY respond to tasks/get by indicating that the task is in state input_required.
   • Servers MUST include an inputRequests field in the tasks/result response when the task is in state input_required.
2. Client Behavior:
   • When tasks/get shows state input_required, clients MUST call tasks/result to get the input requests. Clients SHOULD construct the results of those requests, and then call tasks/input_response with the input responses to provide the required input for the task.
   • Clients MAY choose not to fulfill the input requests, in which case they can cancel the task.

​

Interactions Between Ephemeral and Persistent Workflows

1. Client sends tool call request with task metadata.
2. Server sends back inputRequests response indicating that more information is needed to process the request. This terminates the original request.
3. Client sends a new tool call request, completely independent of the original one, which includes the inputResponses object along with the task metadata.
4. Server sends back a task ID, indicating that it will be processing the request in the background. All subsequent interaction will be done via the Tasks API.

​

Guidance for Error Handling

1. The client retries the original tool call, this time including the inputResponses object, but the response is missing required information that the server needs to process the request.

{
  "jsonrpc": "2.0",
  "id": 3,
  "method": "tools/call",
  "params": {
    "name": "get_weather",
    "arguments": {
      "location": "New York"
    },
    "inputResponses": {
      "not_requested_info": {
        "action": "accept",
        "content": {
          "not_requested_param_name": "Information the server did not request"
        }
      }
    }
  }
}

2. The server responds with an incomplete response, indicating that the client needs to respond to an elicitation request in order for the tool call to complete, and including request state to be passed back:

{
  "jsonrpc": "2.0",
  "id": 2,
  "result": {
    "resultType": "input_required",
    "inputRequests": {
      "github_login": {
        "method": "elicitation/create",
        "params": {
          "mode": "form",
          "message": "Please provide your GitHub username",
          "requestedSchema": {
            "type": "object",
            "properties": {
              "name": {
                "type": "string"
              }
            },
            "required": ["name"]
          }
        }
      }
    }
  }
}

2. The Server responds wit an incomplete response, indicating that the client needs to provide missing information for the request to succeed.

{
  "jsonrpc": "2.0",
  "id": 4,
  "method": "tasks/input_response",
  "params": {
    "inputResponses": {
      "echo_input": {
        "action": "accept",
        "content": {
          "not_requested_parameter": "Information the server did not request."
        }
      }
    },
    "_meta": {
      "io.modelcontextprotocol/related-task": {
        "taskId": "echo_dc792e24-01b5-4c0a-abcb-0559848ca3c5"
      }
    }
  }
}

{
  "id": 4,
  "jsonrpc": "2.0",
  "result": {
    "_meta": {
      "io.modelcontextprotocol/related-task": {
        "taskId": "echo_dc792e24-01b5-4c0a-abcb-0559848ca3c5"
      }
    }
  }
}

​

Rationale

​

Backward Compatibility

def my_tool():
  do_work()
  await elicit_more_info()
  do_more_work()
  return tool_result

def my_tool(request):
  if(request.requestState):
      state = decode(request.requestState)
  if(request.inputResponses):
      additionalInfo = decode(request.inputResponses)

  do_work(state, additionalInfo)
  if(more_info_needed):
    return IncompleteResponse();
  else
    do_more_work()
    return tool_result

​

Security Implications

​

Reference Implementation

​

Acknowledgments