Explore ASP Net core read request The correct way of body

Preface

I believe you are using ASP.NET Core When it comes to development , It's going to involve reading Request.Body Scene , After all, most of us POST All requests are to store data in Http Of Body among . Because the author's daily development is mainly used ASP.NET Core So I also encounter this kind of scene , About the contents of this article , From what I met in the development process about Request.Body Read problem of . In previous use , Basically, it's all the answers to search with the help of search engines , I didn't pay much attention to this , I find that there is a big misunderstanding between my understanding and correct use . So I have feelings , I wrote this article , To record . knowledge has no limit , May I share with you .

Common reading methods

When we want to read Request Body When , I believe your first instinct is the same as the author , It's hard , Just a few lines of code , Here we simulate in Filter Read from Request Body, stay Action or Middleware Or anywhere else , Yes Request Where we live is Body, As shown below

public override void OnActionExecuting(ActionExecutingContext context)
{
    
    // stay ASP.NET Core in Request Body yes Stream In the form of 
    StreamReader stream = new StreamReader(context.HttpContext.Request.Body);
    string body = stream.ReadToEnd();
    _logger.LogDebug("body content:" + body);
    base.OnActionExecuting(context);
}

After you've written , I didn't think much about it , After all, such a routine operation , Full of confidence , Run it and debug it , I found that I directly reported a mistake System.InvalidOperationException: Synchronous operations are disallowed. Call ReadAsync or set AllowSynchronousIO to true instead. Synchronization is not allowed , Please use ReadAsync The way or setting of AllowSynchronousIO by true. I didn't say how to set it up AllowSynchronousIO, But we use search engine is our biggest strength .

Synchronous read

First, let's look at the settings AllowSynchronousIO by true The way , It's also known by the name that synchronization is allowed IO, There are roughly two kinds of settings , Later, we will explore the direct difference between them through the source code , Let's first look at how to set up AllowSynchronousIO Value . The first way is in ConfigureServices Middle configuration , The operation is as follows

services.Configure<KestrelServerOptions>(options =>
{
    
    options.AllowSynchronousIO = true;
});

This is the same as configuring in the configuration file Kestrel The configuration of options is the same, but in different ways , After setting, it is ready , Running does not report an error . There's another way , You don't have to ConfigureServices Set in , adopt IHttpBodyControlFeature How to set , As follows

public override void OnActionExecuting(ActionExecutingContext context)
{
    
    var syncIOFeature = context.HttpContext.Features.Get<IHttpBodyControlFeature>();
    if (syncIOFeature != null)
    {
    
        syncIOFeature.AllowSynchronousIO = true;
    }
    StreamReader stream = new StreamReader(context.HttpContext.Request.Body);
    string body = stream.ReadToEnd();
    _logger.LogDebug("body content:" + body);
    base.OnActionExecuting(context);
}

It works the same way , In this way , No need to read every time Body I always set it up , As long as you're ready to read Body Just set it once before . Both ways are to set up AllowSynchronousIO by true, But we need to think a little bit , Why Microsoft set up AllowSynchronousIO The default is false, It shows that Microsoft does not want us to read synchronously Body. By looking up the data, we come to the conclusion that

Kestrel： Disabled by default AllowSynchronousIO（ Sync IO）, Insufficient threads can cause the application to crash , And synchronization I/O API（ for example HttpRequest.Body.Read） Is a common cause of thread shortage .

So we can know , Although this way can solve the problem , But the performance is not bad , Microsoft doesn't recommend that either , When the program traffic is large , It's easy to cause the program to be unstable or even crash .

Asynchronous read

From the above we know that Microsoft does not want us to set up AllowSynchronousIO The way to operate , Because it will affect performance . Then we can read it asynchronously , Asynchronous mode is actually using Stream Own asynchronous method to read , As shown below

public override void OnActionExecuting(ActionExecutingContext context)
{
    
    StreamReader stream = new StreamReader(context.HttpContext.Request.Body);
    string body = stream.ReadToEndAsync().GetAwaiter().GetResult();
    _logger.LogDebug("body content:" + body);
    base.OnActionExecuting(context);
}

It's that simple , There's no need to add anything else , Just passed ReadToEndAsync To operate in an asynchronous way .ASP.NET Core Many of the operations in are asynchronous , Even filters or middleware can return directly Task Method of type , So we can use asynchronous operations directly

public override async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
{
    
    StreamReader stream = new StreamReader(context.HttpContext.Request.Body);
    string body = await stream.ReadToEndAsync();
    _logger.LogDebug("body content:" + body);
    await next();
}

The advantage of these two methods is that they don't need additional settings , Just read it asynchronously , It's also our recommended practice . What's amazing is that we just StreamReader Of ReadToEnd Replace with ReadToEndAsync The method is happy , Do you feel more magical . When we feel magical , It's because we don't know enough about it , Next, we'll use the source code , Step by step to uncover its mystery .

Duplicate read

Above we demonstrated using synchronous and asynchronous read RequestBody, But is that really OK ？ Not really , In this way, the correct... Can only be read once per request Body result , If you continue to RequestBody This Stream To read , Nothing will be read , Let's start with an example

public override async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
{
    
    StreamReader stream = new StreamReader(context.HttpContext.Request.Body);
    string body = await stream.ReadToEndAsync();
    _logger.LogDebug("body content:" + body);

    StreamReader stream2 = new StreamReader(context.HttpContext.Request.Body);
    string body2 = await stream2.ReadToEndAsync();
    _logger.LogDebug("body2 content:" + body2);

    await next();
}

In the example above body There's the right thing in it RequestBody Result , however body2 Empty string in . This situation is relatively bad , Why do you say that ？ If you are Middleware Reads the RequestBody, The middleware is executed before model binding , This will cause the model binding to fail , Because model binding sometimes needs to read RequestBody obtain http Request content . As for why this is the case, I believe you have a certain understanding , Because we're reading Stream after , At this time Stream The pointer position is already Stream At the end of , namely Position Not at this time 0, and Stream Reading is just about relying on Position To mark external reads Stream To where , So when we read it again, we start at the end , You can't read any information . So we want to read repeatedly RequestBody Then reset before reading again RequestBody Of Position by 0, As shown below

public override async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
{
    
    StreamReader stream = new StreamReader(context.HttpContext.Request.Body);
    string body = await stream.ReadToEndAsync();
    _logger.LogDebug("body content:" + body);

    // Or use reset Position The way  context.HttpContext.Request.Body.Position = 0;
    // If you're sure it's reset after the last read Position Then this sentence can be omitted 
    context.HttpContext.Request.Body.Seek(0, SeekOrigin.Begin);
    StreamReader stream2 = new StreamReader(context.HttpContext.Request.Body);
    string body2 = await stream2.ReadToEndAsync();
    // We'll try to reset it when we're done , Fill your own hole 
    context.HttpContext.Request.Body.Seek(0, SeekOrigin.Begin);
    _logger.LogDebug("body2 content:" + body2);

    await next();
}

After you've written , Happy to run up to see the effect , Found a mistake System.NotSupportedException: Specified method is not supported.at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream.Seek(Int64 offset, SeekOrigin origin) It is generally understood that this operation is not supported , As for why , Let's take a look at the source code later . Said so much , How to solve it ？ It's also very simple. , Microsoft knows it's digging a hole , Nature offers us solutions , It's also very easy to use, that is, add EnableBuffering

public override async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
{
    
    // operation Request.Body And before EnableBuffering that will do 
    context.HttpContext.Request.EnableBuffering();

    StreamReader stream = new StreamReader(context.HttpContext.Request.Body);
    string body = await stream.ReadToEndAsync();
    _logger.LogDebug("body content:" + body);

    context.HttpContext.Request.Body.Seek(0, SeekOrigin.Begin);
    StreamReader stream2 = new StreamReader(context.HttpContext.Request.Body);
    // Note that there ！！！ I've used synchronous reading 
    string body2 = stream2.ReadToEnd();
    context.HttpContext.Request.Body.Seek(0, SeekOrigin.Begin);
    _logger.LogDebug("body2 content:" + body2);

    await next();
}

By adding Request.EnableBuffering() We can read it repeatedly RequestBody 了 , We can guess the name by looking at it , He's with the cache RequestBody of , It should be noted that Request.EnableBuffering() To be added in preparation for reading RequestBody It works before , Otherwise it will be invalid , And each request only needs to be added once . And you see my second reading Body I used the synchronous way to read it RequestBody, Isn't that amazing , Later, we will analyze this problem from the perspective of source code .

Source code exploration

Above we see through StreamReader Of ReadToEnd Synchronous read Request.Body Need to set up AllowSynchronousIO by true To operate , But use StreamReader Of ReadToEndAsync Methods can be operated directly .

StreamReader and Stream The relationship between

We see it all through operation StreamReader It's a good way to do it , It's none of my business Request.Body What's up , Don't worry, let's take a look at the operation here first , First of all, let's take a general look at ReadToEnd Let's take a look at StreamReader After all Stream What's the connection , find ReadToEnd Method

public override string ReadToEnd()
{
    
    ThrowIfDisposed();
    CheckAsyncTaskInProgress();
    //  call ReadBuffer, And then from charBuffer Extract data from .
    StringBuilder sb = new StringBuilder(_charLen - _charPos);
    do
    {
    
        // Loop splicing to read content 
        sb.Append(_charBuffer, _charPos, _charLen - _charPos);
        _charPos = _charLen; 
        // Read buffer, This is the core operation 
        ReadBuffer();
    } while (_charLen > 0);
    // Return read content 
    return sb.ToString();
}

Through this source code, we learned such a message , One is StreamReader Of ReadToEnd In fact, the essence is to read through the loop ReadBuffer And then through StringBuilder To splice what's read , The core is reading ReadBuffer Method , Because the code is more , Let's look at the core operation

if (_checkPreamble)
{
    
    // From here we can see that the essence is to use the Stream Inside Read Method 
    int len = _stream.Read(_byteBuffer, _bytePos, _byteBuffer.Length - _bytePos);
    if (len == 0)
    {
    
        if (_byteLen > 0)
        {
    
            _charLen += _decoder.GetChars(_byteBuffer, 0, _byteLen, _charBuffer, _charLen);
            _bytePos = _byteLen = 0;
        }
        return _charLen;
    }
    _byteLen += len;
}
else
{
    
    // From here we can see that the essence is to use the Stream Inside Read Method 
    _byteLen = _stream.Read(_byteBuffer, 0, _byteBuffer.Length);
    if (_byteLen == 0) 
    {
    
        return _charLen;
    }
}

We can learn from the above code StreamReader It's actually a tool class , It just encapsulates right Stream The original operation of , Simplify our code ReadToEnd The essence of a method is to read Stream Of Read Method . Let's take a look ReadToEndAsync The concrete implementation of the method

public override Task<string> ReadToEndAsync()
{
    
    if (GetType() != typeof(StreamReader))
    {
    
        return base.ReadToEndAsync();
    }
    ThrowIfDisposed();
    CheckAsyncTaskInProgress();
    // The essence is ReadToEndAsyncInternal Method 
    Task<string> task = ReadToEndAsyncInternal();
    _asyncReadTask = task;

    return task;
}

private async Task<string> ReadToEndAsyncInternal()
{
    
    // It is also the content read by loop splicing 
    StringBuilder sb = new StringBuilder(_charLen - _charPos);
    do
    {
    
        int tmpCharPos = _charPos;
        sb.Append(_charBuffer, tmpCharPos, _charLen - tmpCharPos);
        _charPos = _charLen; 
        // The core operation is ReadBufferAsync Method 
        await ReadBufferAsync(CancellationToken.None).ConfigureAwait(false);
    } while (_charLen > 0);
    return sb.ToString();
}

From this we can see that the core operation is ReadBufferAsync Method , More code, let's also look at the core implementation

byte[] tmpByteBuffer = _byteBuffer;
//Stream Assign a value to tmpStream 
Stream tmpStream = _stream;
if (_checkPreamble)
{
    
    int tmpBytePos = _bytePos;
    // The essence is to call Stream Of ReadAsync Method 
    int len = await tmpStream.ReadAsync(new Memory<byte>(tmpByteBuffer, tmpBytePos, tmpByteBuffer.Length - tmpBytePos), cancellationToken).ConfigureAwait(false);
    if (len == 0)
    {
    
        if (_byteLen > 0)
        {
    
            _charLen += _decoder.GetChars(tmpByteBuffer, 0, _byteLen, _charBuffer, _charLen);
            _bytePos = 0; _byteLen = 0;
        }
        return _charLen;
    }
    _byteLen += len;
}
else
{
    
    // The essence is to call Stream Of ReadAsync Method 
    _byteLen = await tmpStream.ReadAsync(new Memory<byte>(tmpByteBuffer), cancellationToken).ConfigureAwait(false);
    if (_byteLen == 0) 
    {
    
        return _charLen;
    }
}

I can see from the above code that StreamReader The essence of reading is reading Stream Packaging , The core method still comes from Stream In itself . The reason why we have introduced StreamReader class , Just to show you StreamReader and Stream The relationship between , Otherwise, I'm afraid everyone will misunderstand that this wave of operation is StreamReader The realization of in , instead of Request.Body The problem of , In fact, it's not like this. Everything points to Stream Of Request Of Body Namely Stream You can check this for yourself , Knowing this step, we can continue .

HttpRequest Of Body

We mentioned above Request Of Body Nature is Stream,Stream It's an abstract class , therefore Request.Body yes Stream Implementation class of . By default Request.Body Yes. HttpRequestStream Example , We said here is the default , Because it can be changed , Let's talk about it later . We're from the top StreamReader From the conclusion of ReadToEnd The essence is still called Stream Of Read Method , That is here HttpRequestStream Of Read Method , Let's take a look at the implementation

public override int Read(byte[] buffer, int offset, int count)
{
    
    // Know synchronous read Body Why did you make a mistake 
    if (!_bodyControl.AllowSynchronousIO)
    {
    
        throw new InvalidOperationException(CoreStrings.SynchronousReadsDisallowed);
    }
    // The essence is to call ReadAsync
    return ReadAsync(buffer, offset, count).GetAwaiter().GetResult();
}

Through this code, we can know why we don't set AllowSynchronousIO by true In the case of reading Body Will throw an exception , This is program level control , And we also learned that Read The essence of calling ReadAsync Asynchronous methods

public override ValueTask<int> ReadAsync(Memory<byte> destination, CancellationToken cancellationToken = default)
{
    
    return ReadAsyncWrapper(destination, cancellationToken);
}

ReadAsync There are no special restrictions in itself , So direct operation ReadAsync There will be no such thing Read It's abnormal .

From this we come to the conclusion that Request.Body namely HttpRequestStream Synchronous reading of Read It throws an exception , And asynchronous reading ReadAsync It doesn't throw an exception. It's just related to HttpRequestStream Of Read There is judgment in the method itself AllowSynchronousIO It has something to do with the value of .

AllowSynchronousIO The essence comes from

adopt HttpRequestStream Of Read The way we can know AllowSynchronousIO It controls the way of synchronous reading . And we also learned AllowSynchronousIO There are several different ways to configure , Next, let's take a general look at the essence of several ways . adopt HttpRequestStream We know Read Methods AllowSynchronousIO The attribute of is from IHttpBodyControlFeature That's the second configuration method we introduced above

private readonly HttpRequestPipeReader _pipeReader;
private readonly IHttpBodyControlFeature _bodyControl;
public HttpRequestStream(IHttpBodyControlFeature bodyControl, HttpRequestPipeReader pipeReader)
{
    
    _bodyControl = bodyControl;
    _pipeReader = pipeReader;
}

So it's with KestrelServerOptions It must have something to do with , Because we only configure KestrelServerOptions Yes. HttpRequestStream Of Read No abnormality is reported , and HttpRequestStream Of Read Just rely on IHttpBodyControlFeature Of AllowSynchronousIO attribute .Kestrel in HttpRequestStream The initialization place is BodyControl

private readonly HttpRequestStream _request;
public BodyControl(IHttpBodyControlFeature bodyControl, IHttpResponseControl responseControl)
{
    
    _request = new HttpRequestStream(bodyControl, _requestReader);
}

Initialization BodyControl Where HttpProtocol in , We found initialization BodyControl Of InitializeBodyControl Method

public void InitializeBodyControl(MessageBody messageBody)
{
    
    if (_bodyControl == null)
    {
    
        // The message here is bodyControl The relay will be this
        _bodyControl = new BodyControl(bodyControl: this, this);
    }
    (RequestBody, ResponseBody, RequestBodyPipeReader, ResponseBodyPipeWriter) = _bodyControl.Start(messageBody);
    _requestStreamInternal = RequestBody;
    _responseStreamInternal = ResponseBody;
}

Here we can see initialization IHttpBodyControlFeature Since the message is this, That is to say HttpProtocol The current instance . in other words HttpProtocol Is to implement the IHttpBodyControlFeature Interface ,HttpProtocol Itself is partial Of , We're in one of the distribution classes HttpProtocol.FeatureCollection See the implementation relationship in

internal partial class HttpProtocol : IHttpRequestFeature, IHttpRequestBodyDetectionFeature, IHttpResponseFeature, IHttpResponseBodyFeature, IRequestBodyPipeFeature, IHttpUpgradeFeature, IHttpConnectionFeature, IHttpRequestLifetimeFeature, IHttpRequestIdentifierFeature, IHttpRequestTrailersFeature, IHttpBodyControlFeature, IHttpMaxRequestBodySizeFeature, IEndpointFeature, IRouteValuesFeature 
 {
     
     bool IHttpBodyControlFeature.AllowSynchronousIO 
     {
     
         get => AllowSynchronousIO; 
         set => AllowSynchronousIO = value; 
     } 
 }

From this we can see that HttpProtocol It did IHttpBodyControlFeature Interface , Next we find initialization AllowSynchronousIO The place of , eureka AllowSynchronousIO = ServerOptions.AllowSynchronousIO; This code description comes from ServerOptions This attribute , Found initialization ServerOptions The place of

private HttpConnectionContext _context;
//ServiceContext Initialization comes from HttpConnectionContext 
public ServiceContext ServiceContext => _context.ServiceContext;
protected KestrelServerOptions ServerOptions {
     get; set; } = default!;
public void Initialize(HttpConnectionContext context)
{
    
    _context = context;
    // come from ServiceContext
    ServerOptions = ServiceContext.ServerOptions;
    Reset();
    HttpResponseControl = this;
}

Through this we know ServerOptions From ServiceContext Of ServerOptions attribute , We found it for ServiceContext The place of assignment , stay KestrelServerImpl Of CreateServiceContext Simplify the logic in the method , The core content is as follows

public KestrelServerImpl(
   IOptions<KestrelServerOptions> options,
   IEnumerable<IConnectionListenerFactory> transportFactories,
   ILoggerFactory loggerFactory)     
   // Infused with IOptions<KestrelServerOptions> Called CreateServiceContext
   : this(transportFactories, null, CreateServiceContext(options, loggerFactory))
{
    
}

private static ServiceContext CreateServiceContext(IOptions<KestrelServerOptions> options, ILoggerFactory loggerFactory)
{
    
    // The value comes from IOptions<KestrelServerOptions> 
    var serverOptions = options.Value ?? new KestrelServerOptions();
    return new ServiceContext
    {
    
        Log = trace,
        HttpParser = new HttpParser<Http1ParsingHandler>(trace.IsEnabled(LogLevel.Information)),
        Scheduler = PipeScheduler.ThreadPool,
        SystemClock = heartbeatManager,
        DateHeaderValueManager = dateHeaderValueManager,
        ConnectionManager = connectionManager,
        Heartbeat = heartbeat,
        // Assignment operation 
        ServerOptions = serverOptions,
    };
}

From the above code, we can see that if it is configured KestrelServerOptions that ServiceContext Of ServerOptions Attributes come from KestrelServerOptions, That is, we go through services.Configure<KestrelServerOptions>() The value of the configuration , All in all, we come to the conclusion that

If the KestrelServerOptions namely services.Configure(), that AllowSynchronousIO From KestrelServerOptions. namely IHttpBodyControlFeature Of AllowSynchronousIO Attributes come from KestrelServerOptions. If not configured , So, directly through modification IHttpBodyControlFeature Example of
AllowSynchronousIO Attributes have the same effect , After all HttpRequestStream It's directly dependent on IHttpBodyControlFeature example .

EnableBuffering Behind the magic

We see in the example above , If you do not add EnableBuffering If so, set it directly RequestBody Of Position Will be submitted to the NotSupportedException Such a mistake , And after adding it, I can read it directly in the synchronous way RequestBody, First of all, let's see why we report mistakes , We know from the above mistakes that mistakes come from HttpRequestStream This class , As we said above, this class inherits Stream abstract class , Through the source code, we can see the following related code

// Out of commission Seek operation 
public override bool CanSeek => false;
// Allow to read 
public override bool CanRead => true;
// It is not allowed to write 
public override bool CanWrite => false;
// Cannot get length 
public override long Length => throw new NotSupportedException();
// Can't read or write Position
public override long Position
{
    
    get => throw new NotSupportedException();
    set => throw new NotSupportedException();
}
// Out of commission Seek Method 
public override long Seek(long offset, SeekOrigin origin)
{
    
    throw new NotSupportedException();
}

I believe that through these we can clearly see that HttpRequestStream Setting or writing related operations are not allowed , That's why we're going straight up there Seek Set up Position Why do you report mistakes when you're in trouble , There are other operational limitations , Anyway, the default is that we don't want to HttpRequestStream Too much operation , In particular, set or write related operations . But we use EnableBuffering I didn't have these problems when I was young , Why on earth ? Now we're going to unveil it . First we start with Request.EnableBuffering() This method starts with , Find the source code in HttpRequestRewindExtensions Extension classes , We start with the simplest parameterless method and see the following definition

/// <summary>
///  Make sure Request.Body Can be read multiple times 
/// </summary>
/// <param name="request"></param>
public static void EnableBuffering(this HttpRequest request)
{
    
    BufferingHelper.EnableRewind(request);
}

The above method is the simplest form , One more EnableBuffering The extension method of is the one with the most complete parameters , This method can control the size of the read and the limited size of the disk

/// <summary>
///  Make sure Request.Body Can be read multiple times 
/// </summary>
/// <param name="request"></param>
/// <param name="bufferThreshold"> The maximum size of the buffer stream in memory （ byte ）. The larger request body is written to disk .</param>
/// <param name="bufferLimit"> The maximum size of the request body （ byte ）. Trying to read beyond this limit will cause an exception </param>
public static void EnableBuffering(this HttpRequest request, int bufferThreshold, long bufferLimit)
{
    
    BufferingHelper.EnableRewind(request, bufferThreshold, bufferLimit);
}

Either way , In the end, it's all calling BufferingHelper.EnableRewind This method , Find it without saying much BufferingHelper This class , The code for finding the location of the class is not much, and it is relatively concise , Let's take it EnableRewind Paste the implementation of

// The default cache size in memory is 30K, Beyond this size will be stored on disk 
internal const int DefaultBufferThreshold = 1024 * 30;

/// <summary>
///  This method is also HttpRequest Extension method 
/// </summary>
/// <returns></returns>
public static HttpRequest EnableRewind(this HttpRequest request, int bufferThreshold = DefaultBufferThreshold, long? bufferLimit = null)
{
    
    if (request == null)
    {
    
        throw new ArgumentNullException(nameof(request));
    }
    // First get Request Body
    var body = request.Body;
    // By default Body yes HttpRequestStream This class CanSeek yes false So it's going to be done if Logic inside 
    if (!body.CanSeek)
    {
    
        // Instantiate FileBufferingReadStream This class , It seems that's the key 
        var fileStream = new FileBufferingReadStream(body, bufferThreshold,bufferLimit,AspNetCoreTempDirectory.TempDirectoryFactory);
        // Assign a value to Body, In other words, it turns on EnableBuffering after Request.Body The type will be FileBufferingReadStream
        request.Body = fileStream;
        // Here we have to put fileStream Register with Response Easy to release 
        request.HttpContext.Response.RegisterForDispose(fileStream);
    }
    return request;
}

From the above source code implementation, we can roughly draw two conclusions

BufferingHelper Of EnableRewind Method is also HttpRequest Extension method of , You can go directly through Request.EnableRewind Form call of , The effect is the same as calling Request.EnableBuffering because EnableBuffering Also called EnableRewind

To enable the EnableBuffering After this operation, you will actually use FileBufferingReadStream Replace the default HttpRequestStream, So the follow-up treatment RequestBody The operation will be FileBufferingReadStream example

Through the above analysis, we can see clearly that , The core operation is FileBufferingReadStream This class , And it can be seen from the name that it must have inherited Stream abstract class , What are you waiting for to find FileBufferingReadStream The implementation of the , First, let's look at the definition of other classes

public class FileBufferingReadStream : Stream
{
    
}

Undoubtedly, it is inherited from Steam class , We also see the use of Request.EnableBuffering Then you can set and repeat the read RequestBody, Note that there are some rewriting operations , Let's take a look at

/// <summary>
///  Allow to read 
/// </summary>
public override bool CanRead
{
    
    get {
     return true; }
}
/// <summary>
///  allow Seek
/// </summary>
public override bool CanSeek
{
    
    get {
     return true; }
}
/// <summary>
///  It is not allowed to write 
/// </summary>
public override bool CanWrite
{
    
    get {
     return false; }
}
/// <summary>
///  You can get the length 
/// </summary>
public override long Length
{
    
    get {
     return _buffer.Length; }
}
/// <summary>
///  Can read and write Position
/// </summary>
public override long Position
{
    
    get {
     return _buffer.Position; }
    set
    {
    
        ThrowIfDisposed();
        _buffer.Position = value;
    }
}

public override long Seek(long offset, SeekOrigin origin)
{
    
    // If Body Exception if released 
    ThrowIfDisposed();
    // Special cases throw exceptions 
    //_completelyBuffered Represents whether the full cache must be in the original HttpRequestStream Set to... After reading true
    // If the original location information is not consistent with the current location information, an exception will be thrown directly 
    if (!_completelyBuffered && origin == SeekOrigin.End)
    {
    
        throw new NotSupportedException("The content has not been fully buffered yet.");
    }
    else if (!_completelyBuffered && origin == SeekOrigin.Current && offset + Position > Length)
    {
    
        throw new NotSupportedException("The content has not been fully buffered yet.");
    }
    else if (!_completelyBuffered && origin == SeekOrigin.Begin && offset > Length)
    {
    
        throw new NotSupportedException("The content has not been fully buffered yet.");
    }
    // Recharge buffer Of Seek
    return _buffer.Seek(offset, origin);
}

Because some key settings have been rewritten , So we can set some flow related operations . from Seek In the method, we see two important parameters _completelyBuffered and _buffer,_completelyBuffered To judge the original HttpRequestStream Read complete or not , because FileBufferingReadStream After all, I read first HttpRequestStream The content of ._buffer It is from HttpRequestStream Read content , Let's take a look at the logic , Remember that this is not all logic , It's a general idea drawn out

private readonly ArrayPool<byte> _bytePool;
private const int _maxRentedBufferSize = 1024 * 1024; //1MB
private Stream _buffer;
public FileBufferingReadStream(int memoryThreshold)
{
    
    // Even if we set up memoryThreshold Then it can't exceed 1MB Otherwise, it will also be stored on disk 
    if (memoryThreshold <= _maxRentedBufferSize)
    {
    
        _rentedBuffer = bytePool.Rent(memoryThreshold);
        _buffer = new MemoryStream(_rentedBuffer);
        _buffer.SetLength(0);
    }
    else
    {
    
        // exceed 1M Cache to disk, so just initialize 
        _buffer = new MemoryStream();
    }
}

These are all initialization operations , The core operation, of course, is FileBufferingReadStream Of Read In the method , Because the real reading place is right here , We find Read Method location

private readonly Stream _inner;
public FileBufferingReadStream(Stream inner)
{
    
    // Receive the original Request.Body
    _inner = inner;
}
public override int Read(Span<byte> buffer)
{
    
    ThrowIfDisposed();

    // If the read has been completed, it will be directly in buffer Get information directly back to 
    if (_buffer.Position < _buffer.Length || _completelyBuffered)
    {
    
        return _buffer.Read(buffer);
    }

    // It's not read complete that leads to this 
    //_inner It's the original receiving RequestBody
    // Read the RequestBody Put in buffer in 
    var read = _inner.Read(buffer);
    // If the length exceeds the set value, an exception will be thrown 
    if (_bufferLimit.HasValue && _bufferLimit - read < _buffer.Length)
    {
    
        throw new IOException("Buffer limit exceeded.");
    }
    // If the settings are stored in memory and Body The length is greater than the set length that can be stored in memory , It's stored on disk 
    if (_inMemory && _memoryThreshold - read < _buffer.Length)
    {
    
        _inMemory = false;
        // Cache the original Body flow 
        var oldBuffer = _buffer;
        // Create cache file 
        _buffer = CreateTempFile();
        // Memory limit exceeded , But no temporary files have been written yet 
        if (_rentedBuffer == null)
        {
    
            oldBuffer.Position = 0;
            var rentedBuffer = _bytePool.Rent(Math.Min((int)oldBuffer.Length, _maxRentedBufferSize));
            try
            {
    
                // take Body Stream read to cache file stream 
                var copyRead = oldBuffer.Read(rentedBuffer);
                // Judge whether to read to the end 
                while (copyRead > 0)
                {
    
                    // take oldBuffer Write to cache file stream _buffer among 
                    _buffer.Write(rentedBuffer.AsSpan(0, copyRead));
                    copyRead = oldBuffer.Read(rentedBuffer);
                }
            }
            finally
            {
    
                // Return the temporary buffer to ArrayPool in 
                _bytePool.Return(rentedBuffer);
            }
        }
        else
        {
    
            
            _buffer.Write(_rentedBuffer.AsSpan(0, (int)oldBuffer.Length));
            _bytePool.Return(_rentedBuffer);
            _rentedBuffer = null;
        }
    }

    // If reading RequestBody Not to the end , Write all the way to the cache 
    if (read > 0)
    {
    
        _buffer.Write(buffer.Slice(0, read));
    }
    else
    {
    
        // If it has been read RequestBody complete , That is, after writing to the cache, it will be updated _completelyBuffered
        // Mark to read all RequestBody complete , And then it's reading RequestBody Directly in _buffer Read from 
        _completelyBuffered = true;
    }
    // Return the read byte The number is used externally StreamReader Determine if the read is complete 
    return read;
}

There's a lot of code and it's complicated , In fact, the core idea is relatively clear , Let's summarize

• First of all, judge whether the original RequestBody, If it's completely read RequestBody Then get the return directly in the buffer
• If RequestBody The length is greater than the set memory limit , Write the buffer to the disk temporary file
• If it is the first read or complete read RequestBody, It will be RequestBody Write to the buffer , Until the read is complete

among CreateTempFile This is the operation flow for creating temporary files , The purpose is to make RequestBody Write to a temporary file . You can specify the address of the temporary file , If not specified, the system default directory will be used , Its implementation is as follows

private Stream CreateTempFile()
{
    
    // Determine whether the cache directory has been set , If not, use the system temporary file directory 
    if (_tempFileDirectory == null)
    {
    
        Debug.Assert(_tempFileDirectoryAccessor != null);
        _tempFileDirectory = _tempFileDirectoryAccessor();
        Debug.Assert(_tempFileDirectory != null);
    }
    // The full path to the temporary file 
    _tempFileName = Path.Combine(_tempFileDirectory, "ASPNETCORE_" + Guid.NewGuid().ToString() + ".tmp");
    // Returns the operation flow of the temporary file 
    return new FileStream(_tempFileName, FileMode.Create, FileAccess.ReadWrite, FileShare.Delete, 1024 * 16,
        FileOptions.Asynchronous | FileOptions.DeleteOnClose | FileOptions.SequentialScan);
}

We analyzed above FileBufferingReadStream Of Read Method this method is a synchronous read method StreamReader Of ReadToEnd Methods use , Of course, it also has an asynchronous read method ReadAsync for StreamReader Of ReadToEndAsync Methods use . The implementation logic of these two methods is identical , It's just that read and write operations are asynchronous , Let's not introduce that method here , Interested students can learn about it by themselves ReadAsync Method implementation

When open EnableBuffering When , Whether the first read is set AllowSynchronousIO by true Of ReadToEnd Synchronous read mode , Or use it directly ReadToEndAsync Asynchronous read mode of , So use it again ReadToEnd Synchronous mode to read Request.Body There is no need to set up AllowSynchronousIO by true. Because the default Request.Body Already by HttpRequestStream Replace the instance with FileBufferingReadStream example , and FileBufferingReadStream Rewrote Read and ReadAsync Method , There is no restriction that synchronous reading is not allowed .

summary

This article is quite long , If you want to go deep into the logic , I hope this article can give you some guidance to read the source code . In order to prevent you from going deep into the article and forgetting the specific process logic , Here we will summarize about correct reading RequestBody The whole conclusion of

• First, about synchronous reading Request.Body By default RequestBody The implementation of HttpRequestStream, however HttpRequestStream In rewriting Read Method will determine whether to turn on AllowSynchronousIO, If not, throw an exception . however HttpRequestStream Of ReadAsync There is no such limitation to the method , So read asynchronously RequestBody There is no abnormal .
• Though by setting AllowSynchronousIO Or use ReadAsync The way we can read RequestBody, however RequestBody Can't read repeatedly , This is because HttpRequestStream Of Position and Seek No modification is allowed , If it is set, an exception will be thrown directly . In order to be able to read repeatedly , We introduced Request Extension method of EnableBuffering In this way, we can reset the read position to achieve RequestBody Repeat read of .
• About opening EnableBuffering Method can be set once per request , That is, in preparation to read RequestBody Set before . Its essence is actually to use FileBufferingReadStream Instead of default RequestBody Default type of HttpRequestStream, So we're at once Http Operation in request Body In fact, it's the operation FileBufferingReadStream, This class rewrites Stream When Position and Seek All can be set up , In this way, we can achieve repeated reading .
• FileBufferingReadStream It's not just repeatable reading , It's also added yes RequestBody Cache function of , It makes us read repeatedly in one request RequestBody When you can Buffer The cache content is directly obtained from the Buffer Itself is a MemoryStream. Of course, we can also implement a set of logic to replace Body, As long as we rewrite it so that Stream Support to reset the read position .