詳解ASP.NET Core實(shí)現(xiàn)的文件斷點(diǎn)上傳下載
作者：小乖獸技術(shù)
在 ASP.NET Core 應(yīng)用程序中，創(chuàng)建一個(gè)名為 UploadController 的控制器。在該控制器中，使用 ApiControllerAttribute 特性聲明該控制器為 Web API 控制器。
場(chǎng)景分析
文件的斷點(diǎn)續(xù)傳功能可以應(yīng)用于需要上傳或下載大文件的場(chǎng)景。在網(wǎng)絡(luò)狀況不佳或者文件較大時(shí)，一次性上傳或下載整個(gè)文件可能會(huì)耗費(fèi)大量時(shí)間和帶寬，并且可能會(huì)導(dǎo)致因中斷而失敗的情況發(fā)生。通過(guò)實(shí)現(xiàn)文件的斷點(diǎn)續(xù)傳功能，可以將大文件分割成小塊，分別上傳或下載，即使在網(wǎng)絡(luò)出現(xiàn)問(wèn)題時(shí)也可以通過(guò)上傳或下載已經(jīng)完成的文件塊來(lái)繼續(xù)未完成的操作，減少耗時(shí)和帶寬消耗，提高用戶體驗(yàn)。常見(jiàn)的應(yīng)用場(chǎng)景如：視頻、音頻文件的上傳或下載，大型軟件的更新或下載，云盤(pán)等文件存儲(chǔ)服務(wù)等等。
設(shè)計(jì)思路
在asp.net core 中實(shí)現(xiàn)文件的斷點(diǎn)續(xù)傳功能，需要進(jìn)行以下步驟：
- 客戶端與服務(wù)端建立連接。
- 客戶端上傳文件時(shí)，將文件切割成小塊并分別傳輸?shù)椒?wù)端。
- 服務(wù)端接收到文件塊后進(jìn)行校驗(yàn)和整合。
- 客戶端下載文件時(shí)，請(qǐng)求服務(wù)端指定位置開(kāi)始的文件塊，并按照順序拼接成完整文件。
具體實(shí)現(xiàn)思路如下：
斷點(diǎn)上傳
- 客戶端上傳文件時(shí)，服務(wù)端接收到請(qǐng)求，在服務(wù)端創(chuàng)建對(duì)應(yīng)的文件，并記錄上傳時(shí)間、文件大小等信息。
- 將接收到的文件塊保存到物理文件中，每次上傳塊的時(shí)候都應(yīng)該壓縮這個(gè)塊，以減少帶寬的消耗。
- 服務(wù)端返回給客戶端成功接收的文件塊編號(hào)和文件處理情況。
斷點(diǎn)下載
- 客戶端下載文件時(shí)，服務(wù)端接收到請(qǐng)求，讀取文件對(duì)應(yīng)的塊信息，判斷客戶端需要下載的文件塊范圍是否存在于服務(wù)端的文件塊信息中。
- 如果該文件塊不存在，則返回錯(cuò)誤信息。
- 如果該文件塊存在，則讀取對(duì)應(yīng)文件塊并返回給客戶端。
實(shí)現(xiàn)流程
斷點(diǎn)上傳
創(chuàng)建上傳控制器
在 ASP.NET Core 應(yīng)用程序中，創(chuàng)建一個(gè)名為 UploadController 的控制器。在該控制器中，使用 ApiControllerAttribute 特性聲明該控制器為 Web API 控制器。
[ApiController]
[Route("[controller]")]
public class UploadController : ControllerBase實(shí)現(xiàn)上傳方法
在控制器中實(shí)現(xiàn)下面的代碼。
[HttpPost]
public async Task<IActionResult> Upload(IFormFile file, int chunkIndex, int totalChunks)
{
if (file == null || file.Length == 0)
{
return BadRequest("This request does not have any body");
}
// create the folder if it doesn't exist yet
string folderPath = Path.Combine(Directory.GetCurrentDirectory(), "Uploads");
if (!Directory.Exists(folderPath))
{
Directory.CreateDirectory(folderPath);
}
string filePath = Path.Combine(folderPath, file.FileName);
using (FileStream stream = new FileStream(filePath, FileMode.Create, FileAccess.Write, FileShare.Write))
{
await file.CopyToAsync(stream);
}
return Ok();
}實(shí)現(xiàn)斷點(diǎn)上傳邏輯
在上面的代碼中，我們只是將整個(gè)文件保存到了服務(wù)器上。現(xiàn)在，我們需要實(shí)現(xiàn)斷點(diǎn)上傳的邏輯。斷點(diǎn)上傳指的是，將大文件分成多個(gè)小塊，并逐個(gè)上傳。
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using System;
using System.IO;
using System.Threading.Tasks;
namespace ResumeTransfer
{
[Route("api/[controller]")]
[ApiController]
public class UploadController : ControllerBase
{
private const string UploadsFolder = "Uploads";
[HttpPost]
public async Task<IActionResult> Upload(IFormFile file, int? chunkIndex, int? totalChunks)
{
if (file == null || file.Length == 0)
{
return BadRequest("This request does not have any body");
}
// Check if the chunk index and total chunks are provided.
if (!chunkIndex.HasValue || chunkIndex.Value < 0 || !totalChunks.HasValue || totalChunks.Value <= 0)
{
return BadRequest("Invalid chunk index or total chunks");
}
// Create folder for upload files if not exists
string folderPath = Path.Combine(Directory.GetCurrentDirectory(), UploadsFolder);
if (!Directory.Exists(folderPath))
{
Directory.CreateDirectory(folderPath);
}
string filePath = Path.Combine(folderPath, file.FileName);
// Check if chunk already exists
if (System.IO.File.Exists(GetChunkFileName(filePath, chunkIndex.Value)))
{
return Ok();
}
using (FileStream stream = new FileStream(GetChunkFileName(filePath, chunkIndex.Value), FileMode.Create, FileAccess.Write, FileShare.Write))
{
await file.CopyToAsync(stream);
}
if (chunkIndex == totalChunks - 1)
{
// All the chunks have been uploaded, merge them into a single file
MergeChunks(filePath, totalChunks.Value);
}
return Ok();
}
private void MergeChunks(string filePath, int totalChunks)
{
using (var finalStream = new FileStream(filePath, FileMode.CreateNew))
{
for (int i = 0; i < totalChunks; i++)
{
var chunkFileName = GetChunkFileName(filePath, i);
using (var stream = new FileStream(chunkFileName, FileMode.Open))
{
stream.CopyTo(finalStream);
}
System.IO.File.Delete(chunkFileName);
}
}
}
private string GetChunkFileName(string filePath, int chunkIndex)
{
return $"{filePath}.part{chunkIndex.ToString().PadLeft(5, '0')}";
}
}
}客戶端上傳
客戶端使用 axios 庫(kù)進(jìn)行文件上傳。使用下面的代碼，將文件分塊并管理每個(gè)文件塊的大小和數(shù)量。
// get file size and name
const fileSize = file.size;
const fileName = file.name;
// calculate chunk size
const chunkSize = 10 * 1024 * 1024; // 10MB
// calculate total chunks
const totalChunks = Math.ceil(fileSize / chunkSize);
// chunk upload function
const uploadChunk = async (chunkIndex) => {
const start = chunkIndex * chunkSize;
const end = Math.min((chunkIndex + 1) * chunkSize, fileSize);
const formData = new FormData();
formData.append("file", file.slice(start, end));
formData.append("chunkIndex", chunkIndex);
formData.append("totalChunks", totalChunks);
await axios.post("/upload", formData);
};
for (let i = 0; i < totalChunks; i++) {
await uploadChunk(i);
}斷點(diǎn)下載
創(chuàng)建下載控制器
創(chuàng)建一個(gè)名為 DownloadController 的控制器，使用 ApiControllerAttribute 特性聲明該控制器為 Web API 控制器。
[ApiController]
[Route("[controller]")]
public class DownloadController : ControllerBase實(shí)現(xiàn)斷點(diǎn)下載邏輯
在控制器中實(shí)現(xiàn)下面的代碼。
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using System;
using System.IO;
namespace ResumeTransfer
{
[Route("api/[controller]")]
[ApiController]
public class DownloadController : ControllerBase
{
private const string UploadsFolder = "Uploads";
[HttpGet("{fileName}")]
public IActionResult Download(string fileName, long? startByte = null, long? endByte = null)
{
if (string.IsNullOrEmpty(fileName))
{
return BadRequest("Invalid file name");
}
string filePath = Path.Combine(Directory.GetCurrentDirectory(), UploadsFolder, fileName);
if (!System.IO.File.Exists(filePath))
{
return NotFound();
}
long contentLength = new System.IO.FileInfo(filePath).Length;
// Calculate the range to download.
if (startByte == null)
{
startByte = 0;
}
if (endByte == null)
{
endByte = contentLength - 1;
}
// Adjust the startByte and endByte to be within the range of the file size.
if (startByte.Value < 0 || startByte.Value >= contentLength || endByte.Value < startByte.Value || endByte.Value >= contentLength)
{
Response.Headers.Add("Content-Range", $"bytes */{contentLength}");
return new StatusCodeResult(416); // Requested range not satisfiable
}
// Set the Content-Disposition header to enable users to save the file.
Response.Headers.Add("Content-Disposition", $"inline; filename={fileName}");
Response.StatusCode = 206; //Partial Content
Response.Headers.Add("Accept-Ranges", "bytes");
long length = endByte.Value - startByte.Value + 1;
Response.Headers.Add("Content-Length", length.ToString());
// Send the file data in a range of bytes, if requested
byte[] buffer = new byte[1024 * 1024];
using (FileStream stream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.Read))
{
stream.Seek(startByte.Value, SeekOrigin.Begin);
int bytesRead;
while (length > 0 && (bytesRead = stream.Read(buffer, 0, (int)Math.Min(buffer.Length, length))) > 0)
{
// Check if the client has disconnected.
if (!Response.HttpContext.Response.Body.CanWrite)
{
return Ok();
}
Response.Body.WriteAsync(buffer, 0, bytesRead);
length -= bytesRead;
}
}
return new EmptyResult();
}
}
}客戶端下載
客戶端使用 axios 庫(kù)進(jìn)行文件下載。使用下面的代碼，將要下載的文件拆分成小塊，并按照順序下載。
const CHUNK_SIZE = 1024 * 1024 * 5; // 5MB
const downloadChunk = async (chunkIndex) => {
const res = await axios.get(`/download?fileName=${fileName}&chunkIndex=${chunkIndex}`, {
responseType: "arraybuffer",
});
const arrayBuffer = res.data;
const start = chunkIndex * CHUNK_SIZE;
const end = start + CHUNK_SIZE;
const blob = new Blob([arrayBuffer]);
const url = URL.createObjectURL(blob);
const a = document.createElement("a");
a.style.display = "none";
a.href = url;
a.download = fileName;
document.body.appendChild(a);
a.click();
URL.revokeObjectURL(url);
};
for (let i = 0; i < totalChunks; i++) {
await downloadChunk(i);
}單元測(cè)試
在我們的代碼中實(shí)現(xiàn)單元測(cè)試可以確保代碼的正確性，并且可以減少手工測(cè)試的負(fù)擔(dān)。我們可以使用
Microsoft.VisualStudio.TestTools.UnitTesting（在 .NET Core 中，也可以使用 xUnit 或 NUnit 進(jìn)行單元測(cè)試）進(jìn)行單元測(cè)試。
下面是一個(gè)簡(jiǎn)單的上傳控制器單元測(cè)試示例：
using ResumeTransfer;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using System.IO;
using System.Threading.Tasks;
using Xunit;
using Microsoft.Extensions.Primitives;
using Moq;
namespace ResumeTransfer.Tests
{
public class UploadControllerTests
{
[Fact]
public async Task Upload_ReturnsBadRequest_WhenNoFileIsSelected()
{
// Arrange
var formCollection = new FormCollection(new Dictionary<string, StringValues>(), new FormFileCollection());
var context = new Mock<HttpContext>();
context.SetupGet(x => x.Request.Form).Returns(formCollection);
var controller = new UploadController { ControllerContext = new ControllerContext { HttpContext = context.Object } };
// Act
var result = await controller.Upload(null, 0, 1);
// Assert
Assert.IsType<BadRequestObjectResult>(result);
}
[Fact]
public async Task Upload_ReturnsBadRequest_WhenInvalidChunkIndexOrTotalChunksIsProvided()
{
// Arrange
var fileName = "test.txt";
var fileStream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes("This is a test file."));
var formFile = new FormFile(fileStream, 0, fileStream.Length, "Data", fileName);
var formCollection = new FormCollection(new Dictionary<string, StringValues>
{
{ "chunkIndex", "0" },
{ "totalChunks", "0" }
}, new FormFileCollection { formFile });
var context = new Mock<HttpContext>();
context.SetupGet(x => x.Request.Form).Returns(formCollection);
var controller = new UploadController { ControllerContext = new ControllerContext { HttpContext = context.Object } };
// Act
var result = await controller.Upload(formFile, 0, 0);
// Assert
Assert.IsType<BadRequestObjectResult>(result);
}
[Fact]
public async Task Upload_UploadsChunk_WhenChunkDoesNotExist()
{
// Arrange
var fileName = "test.txt";
var fileStream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes("This is a test file."));
var formFile = new FormFile(fileStream, 0, fileStream.Length, "Data", fileName);
var formCollection = new FormCollection(new Dictionary<string, StringValues>
{
{ "chunkIndex", "0" },
{ "totalChunks", "1" }
}, new FormFileCollection { formFile });
var context = new Mock<HttpContext>();
context.SetupGet(x => x.Request.Form).Returns(formCollection);
var uploadsFolder = Path.Combine(Directory.GetCurrentDirectory(), "uploads");
var controller = new UploadController { ControllerContext = new ControllerContext { HttpContext = context.Object }, UploadsFolder = uploadsFolder };
// Act
var result = await controller.Upload(formFile, 0, 1);
var uploadedFilePath = Path.Combine(uploadsFolder, fileName);
// Assert
Assert.IsType<OkResult>(result);
Assert.True(File.Exists(uploadedFilePath));
using (var streamReader = new StreamReader(File.OpenRead(uploadedFilePath)))
{
var content = await streamReader.ReadToEndAsync();
Assert.Equal("This is a test file.", content);
}
File.Delete(uploadedFilePath);
}
[Fact]
public async Task Upload_UploadsChunk_WhenChunkExists()
{
// Arrange
var fileName = "test.txt";
var fileStream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes("This is a test file."));
var formFile = new FormFile(fileStream, 0, fileStream.Length, "Data", fileName);
var formCollection1 = new FormCollection(new Dictionary<string, StringValues>
{
{ "chunkIndex", "0" },
{ "totalChunks", "2" }
}, new FormFileCollection { formFile });
var context = new Mock<HttpContext>();
context.SetupGet(x => x.Request.Form).Returns(formCollection1);
var uploadsFolder = Path.Combine(Directory.GetCurrentDirectory(), "uploads");
var controller = new UploadController { ControllerContext = new ControllerContext { HttpContext = context.Object }, UploadsFolder = uploadsFolder };
// Act
var result1 = await controller.Upload(formFile, 0, 2);
// Assert
Assert.IsType<OkResult>(result1);
// Arrange
var formCollection2 = new FormCollection(new Dictionary<string, StringValues>
{
{ "chunkIndex", "1" },
{ "totalChunks", "2" }
}, new FormFileCollection { formFile });
context.SetupGet(x => x.Request.Form).Returns(formCollection2);
// Act
var result2 = await controller.Upload(formFile, 1, 2);
var uploadedFilePath = Path.Combine(uploadsFolder, fileName);
// Assert
Assert.IsType<OkResult>(result2);
Assert.True(File.Exists(uploadedFilePath));
using (var streamReader = new StreamReader(File.OpenRead(uploadedFilePath)))
{
var content = await streamReader.ReadToEndAsync();
Assert.Equal("This is a test file.This is a test file.", content);
}
File.Delete(uploadedFilePath);
}
}
}
性能測(cè)試
為了實(shí)現(xiàn)更好的性能和響應(yīng)時(shí)間，我們可以使用 BenchmarkDotNet 進(jìn)行性能測(cè)試，以便找到性能瓶頸并對(duì)代碼進(jìn)行優(yōu)化。
下面是一個(gè)簡(jiǎn)單的上傳控制器性能測(cè)試示例：
using BenchmarkDotNet.Attributes;
using ResumeTransfer;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Http.Internal;
namespace ResumeTransfer.Benchmarks
{
[MemoryDiagnoser]
public class UploadControllerBenchmarks
{
private readonly UploadController _controller;
private readonly IFormFile _testFile;
public UploadControllerBenchmarks()
{
_controller = new UploadController();
_testFile = new FormFile(new MemoryStream(System.Text.Encoding.UTF8.GetBytes("This is a test file")), 0, 0, "TestFile", "test.txt");
}
[Benchmark]
public async Task<IActionResult> UploadSingleChunk()
{
var formCollection = new FormCollection(new System.Collections.Generic.Dictionary<string, Microsoft.Extensions.Primitives.StringValues>
{
{ "chunkIndex", "0" },
{ "totalChunks", "1" }
}, new FormFileCollection { _testFile });
var request = new DefaultHttpContext();
request.Request.Form = formCollection;
_controller.ControllerContext = new ControllerContext { HttpContext = request };
return await _controller.Upload(_testFile, 0, 1);
}
[Benchmark]
public async Task<IActionResult> UploadMultipleChunks()
{
var chunkSizeBytes = 10485760; // 10 MB
var totalFileSizeBytes = 52428800; // 50 MB
var totalChunks = (int)Math.Ceiling((double)totalFileSizeBytes / chunkSizeBytes);
for (var i = 0; i < totalChunks; i++)
{
var chunkStartByte = i * chunkSizeBytes;
var chunkEndByte = Math.Min(chunkStartByte + chunkSizeBytes - 1, totalFileSizeBytes - 1);
var chunkFileContent = new byte[chunkEndByte - chunkStartByte + 1];
using (var memoryStream = new MemoryStream(chunkFileContent))
{
using (var binaryWriter = new BinaryWriter(memoryStream))
{
binaryWriter.Write(chunkStartByte);
binaryWriter.Write(chunkEndByte);
}
}
var chunkFileName = $"{_testFile.FileName}.part{i.ToString().PadLeft(5, '0')}";
var chunkFilePath = Path.Combine(Directory.GetCurrentDirectory(), ChunkUploadController.UploadsFolder, chunkFileName);
using (var fileStream = new FileStream(chunkFilePath, FileMode.Create, FileAccess.Write, FileShare.Write))
{
await fileStream.WriteAsync(chunkFileContent);
await fileStream.FlushAsync();
}
}
var formCollection = new FormCollection(new System.Collections.Generic.Dictionary<string, Microsoft.Extensions.Primitives.StringValues>
{
{ "chunkIndex", "0" },
{ "totalChunks", totalChunks.ToString() }
}, new FormFileCollection { _testFile });
var request = new DefaultHttpContext();
request.Request.Form = formCollection;
_controller.ControllerContext = new ControllerContext { HttpContext = request };
return await _controller.Upload(_testFile, 0, totalChunks);
}
}
}運(yùn)行上面的代碼后，將會(huì)輸出詳細(xì)的性能測(cè)試結(jié)果。
根據(jù)測(cè)試結(jié)果，我們可以找到性能瓶頸，并對(duì)代碼進(jìn)行優(yōu)化，以達(dá)到更高的性能和更快的響應(yīng)時(shí)間。
責(zé)任編輯：姜華
來(lái)源：
今日頭條
