Repository navigation
Add log file actuator - #1776
officialasishkumar wants to merge 4 commits into
Conversation
|
Azure Pipelines: 14 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
|
I started looking at this, but I'm struggling with some corporate-IT-related local workstation issues, so I won't be able to get through testing today. @officialasishkumar, can you please rebase in the meantime, so this picks up the changes that add net11 and drop net8 and net9? While you're in there, could you also look into how a relative If that's right, the same relative path in the logging provider's configuration and in Could you check how Serilog, log4net, NLog and Spring Boot each resolve relative paths, and confirm or correct the above? Then please either change the behavior or document the difference, and share your reasoning. Spring Boot is the closest reference for Spring Boot Admin users, so I'd weigh it heavily. |
b3796ec to
5197389
Compare
|
Rebased onto main. You're right: Spring Boot's logging.file.name and Serilog's file sink resolve relative paths against the current directory, while log4net and NLog use the application base directory. I changed the endpoint to resolve against the current directory so the same relative path hits the same file as the logging provider under dotnet run, and updated the option docs, schema and test: 5197389 |
5197389 to
529e72a
Compare
|
The good news is I was able to verify this already works well in Spring Boot Admin! Against a Spring Boot app, side by side:
However, I neglected to re-read #1496 before I raised the relative path question. @bart-vmware's comment already decided on the binary directory, because the current directory isn't stable, and @tscrypter agreed. The original version of this PR followed that. @officialasishkumar, that's on me, not you. Please hold off on further changes regarding the path until this is settled (you're free to fix the code style complaint if you wish). There are really three options, and I don't think the pros and cons have all been written down:
@bart-vmware, @tscrypter: does the Serilog/Spring mismatch change your preference? If we stay with the binary directory, I'd prefer option 2 over 3 for the single-file reason. I'd also suggest documenting the difference, and logging the resolved full path when the file isn't found, so a mismatch is easy to diagnose. Separately, I found one more thing worth documenting. The design assumes the logging provider keeps a stable "current log" file, but Serilog's file sink doesn't. With |
|
Regarding where to resolve relative paths from: My preference is still The rolling file behavior of Serilog is a great catch. If we don't fix that, the endpoint is mostly useless. From what I understand, dynamic file names aren't supported by Spring Boot (unless you code your own
|
|
I agree with @bart-vmware regarding |
TimHess
left a comment
There was a problem hiding this comment.
Thank you for the work on this, and for responding so quickly!
If something is unclear, or you'd rather approach part of it differently, just say so in the thread.
| } | ||
|
|
||
| // Like Spring Boot and Serilog, resolve a relative path from the current directory, so the same path works for the logging provider and this endpoint. | ||
| string fullPath = Path.GetFullPath(filePath); |
There was a problem hiding this comment.
Per the consensus in the discussion, this should go back to AppContext.BaseDirectory (congratulations for landing on a 3rd option that was better than either of the original proposals 😄). Watch for conflict in the doc comment if you choose to change by reverting 529e72a.
There was a problem hiding this comment.
Reverted to AppContext.BaseDirectory and updated the option docs accordingly: 1e82be1
| /// Gets or sets the path to the log file on disk. The path can be absolute, or relative to the current directory of the process. When not set, this | ||
| /// endpoint is turned off, unless it is explicitly enabled. | ||
| /// </summary> | ||
| public string? FilePath { get; set; } |
There was a problem hiding this comment.
As mentioned separately, Serilog's file sink doesn't keep a stable current-log name—with size-based rolling, app.log stops growing at the limit and new entries go to app_001.log, app_002.log, and so on, so a fixed path freezes after the first roll. Please let FilePath accept a wildcard pattern and serve the matching file with the latest last-modified time. A plain path behaves as today, a relative pattern resolves from AppContext.BaseDirectory, no match returns 404, and there's no new property.
We can stick to the BCL rather than bringing in Microsoft.Extensions.FileSystemGlobbing, as long as it goes through the IFileProvider abstraction so tests can use fakes:
IFileProvider.GetDirectoryContentsenumerates the target directory and exposesIFileInfo.LastModifieddirectly, keeping everything testable without touching real disk.System.IO.Enumeration.FileSystemName.MatchesSimpleExpressionis built into the BCL and handles*and?wildcards cleanly (covering rolling patterns likeapp*.logorapp_???.log) without needing/**/directory recursion.- Clear boundary: Splitting the resolved path into directory + filename pattern keeps wildcards scoped to the filename rather than matching across directories.
I also think matches should be limited to .log and .txt (or require a non-wildcard extension) so a broad pattern like * can't accidentally serve config files or binaries—I'm open to other extensions or ideas here, it just shouldn't be completely open-ended. Alongside fake-based unit tests for the wildcard and rollover selection, ideally this work includes an integration test using Serilog's rolling file sink to verify the behavior end-to-end.
There was a problem hiding this comment.
FilePath now accepts wildcards in the file name (BCL only, via FileSystemName.MatchesSimpleExpression) and serves the match with the latest last-modified time, with ties broken by the highest name. A plain path behaves as before, a relative pattern resolves from the base directory, and no match returns 404.
| return await EndpointHandler.InvokeAsync(request, cancellationToken); | ||
| } | ||
|
|
||
| protected override async Task WriteResponseAsync(string? filePath, HttpContext httpContext, CancellationToken cancellationToken) |
There was a problem hiding this comment.
Please move the logic into the handler. The middleware should read the Range header in ParseRequestAsync and pass a plain request object. The handler should pick the file, apply the range and return a result: offset, length, total and stream, or not found, or not satisfiable. WriteResponseAsync then only maps that to status codes and headers. This is the same feedback Bart gave on #1499, and it keeps HttpContext out of the handler.
Once file access moves to IFileProvider, please check that reads still share FileShare.ReadWrite | FileShare.Delete, so a logger can roll over during a read. I believe the default physical stream doesn't include Delete.
There was a problem hiding this comment.
Moved into the handler: the middleware parses Range into a LogFileRequest, the handler picks the file and applies the range, and WriteResponseAsync only maps the result to status and headers. Files go through IFileProvider, and the default physical provider is replaced by one that opens with FileShare.ReadWrite | FileShare.Delete.
| public sealed class EndpointAuthorizationTest | ||
| public sealed class EndpointAuthorizationTest : IDisposable | ||
| { | ||
| private readonly TempFile _logFile = new(); |
There was a problem hiding this comment.
This made the whole class IDisposable just to hold a real temp file. Please use a fake instead, such as MemoryFileProvider from TestResources (see the middleware thread). That removes the disk, the cleanup and the flakiness.
There was a problem hiding this comment.
Switched EndpointAuthorizationTest back to its original shape and it now uses MemoryFileProvider.
| [InlineData("bytes=-100", "0123456789", "bytes 0-9/10")] | ||
| [InlineData("bytes=9-9", "9", "bytes 9-9/10")] | ||
| [InlineData("BYTES=2-4", "234", "bytes 2-4/10")] | ||
| public async Task Endpoint_returns_requested_byte_range(string rangeHeader, string expectedContent, string expectedContentRange) |
There was a problem hiding this comment.
Most of these cases run the full HTTP pipeline against real temp files. Once the logic is in the handler, please cover range arithmetic and file selection with handler-level tests on a fake file provider, and keep a thin set of HTTP tests for status and header mapping. MemoryFileProvider.LastModified always returns default, so it needs extending for the newest-file tests.
There was a problem hiding this comment.
Range arithmetic and file selection now have handler-level tests on MemoryFileProvider, and the HTTP tests are down to status and header mapping. MemoryFileProvider got IncludeFile overloads that take a last-modified time.
| } | ||
|
|
||
| [Fact] | ||
| public async Task Endpoint_can_read_log_file_while_it_is_open_for_writing() |
There was a problem hiding this comment.
This is one of the two places where real disk is justified, because sharing flags can't be faked. The other should be Serilog. Please add one or two tests using the real Serilog file sink with a small fileSizeLimitBytes and rollOnFileSizeLimit, asserting that the wildcard serves the newest file after a roll. Use TempDirectory in a using.
There was a problem hiding this comment.
Added two tests using the real Serilog file sink with fileSizeLimitBytes and rollOnFileSizeLimit (wildcard serves the newest file, plain path stays on the first one), using Sandbox since TempDirectory's constructor is protected. The sharing test stays on real disk.
| public sealed class LogFileEndpointOptions : EndpointOptions | ||
| { | ||
| /// <summary> | ||
| /// Gets or sets the path to the log file on disk. The path can be absolute, or relative to the current directory of the process. When not set, this |
There was a problem hiding this comment.
Docs should recommend absolute paths, explain that relative paths resolve from the base directory (unlike Serilog and Spring), and show the wildcard with a Serilog rolling example. They should also warn that without a wildcard, a size-rolled Serilog file freezes. @officialasishkumar, could you open a sibling PR in SteeltoeOSS/Documentation and link it here?
| services.AddInfoActuator(configureMiddleware); | ||
| services.AddHealthActuator(configureMiddleware); | ||
| services.AddLoggersActuator(configureMiddleware); | ||
| services.AddLogFileActuator(configureMiddleware); |
There was a problem hiding this comment.
Can you please add log file configuration to the existing Management sample, with a rolling Serilog file and a wildcard, so we have a live example of how it's exercised end to end with Spring Boot Admin? New entries in the .http file(s) would be nice as well. That would need a PR in the Samples repo.
Add a "logfile" actuator that serves the contents of a log file written by any logging provider, compatible with the Spring Boot logfile endpoint. This enables the Logfile view in Spring Boot Admin for Steeltoe apps. - New setting Management:Endpoints:LogFile:FilePath. A relative path is resolved from the application directory (like log4net and NLog), because the current directory isn't stable. - Like in Spring Boot, the endpoint is turned off by default until a file path is configured, so AddAllActuators doesn't expose a link that can only return 404. When explicitly enabled without a path, a warning is logged. - Returns 404 when the file doesn't exist (yet), without logging a warning. - Supports single byte ranges (a-b, a-, -n) with 206/Content-Range, and returns 416 with "Content-Range: bytes */<length>" for unsatisfiable or multi-range requests. Spring Boot Admin relies on this to tail the file and to detect roll-over. - Opens the file with FileShare.ReadWrite | FileShare.Delete, so the logger can keep writing, rolling over or deleting it. - Sends exactly the number of bytes captured when the file was opened. The built-in ASP.NET Core file results copy to end-of-file, which overflows Content-Length when the log grows during the download. - Responds with "text/plain" without a charset, because the encoding depends on the logging provider. The actuator is registered by AddAllActuators. The configuration schema and public API files are updated, and EndpointAuthorizationTest now configures a log file because it exercises every endpoint type. Fixes SteeltoeOSS#1496
Resolve relative paths from the base directory, read files through IFileProvider with shared read/write/delete access, and let the file name contain wildcards to serve the most recently modified match, so size-rolled Serilog files keep working. The middleware now only parses the Range header and maps the handler result to status codes and headers. Tests use MemoryFileProvider where possible, with real disk limited to file sharing and the Serilog file sink.
54ea55b to
1e82be1
Compare
Description
Adds the log file actuator (
/actuator/logfile), the Steeltoe counterpart of Spring Boot'slogfileendpoint. It serves the contents of a log file written by any logging provider (Serilog, NLog, log4net, ...), so Spring Boot Admin's Logfile view works for Steeltoe apps too.This picks up the design discussed in #1496 and the review feedback from the earlier attempt in #1499.
Configuration
Settings live under
Management:Endpoints:LogFile. The usual endpoint settings apply (Enabled,Id=logfile,Path=logfile,RequiredPermissions=Restricted,AllowedVerbs=GET), plus one new setting:FilePathAppContext.BaseDirectory), which is how log4net and NLog resolve relative paths. The current directory isn't used because it isn't stable.{ "Management": { "Endpoints": { "LogFile": { "FilePath": "logs/app.log" } } } }The actuator is included in
AddAllActuators()and can also be added withAddLogFileActuator().Behavior
FilePathis empty,Enableddefaults tofalse. This matches Spring Boot, which only registers the endpoint when a log file is configured. Without it,AddAllActuators()combined withexposure:include: *would add alogfilelink (and a Spring Boot Admin tab) to every app, and that link would only ever return 404. If someone setsEnabled: truewithout aFilePath, the endpoint returns 404 and logs a warning that explains what to configure.Range: bytes=...request (a-b,a-, or suffix-n) returns206withContent-Range. A range that can't be satisfied returns416withContent-Range: bytes */<length>. Spring Boot Admin's log tailing depends on this: it first requestsbytes=-307200, then polls from the last byte it has seen withbytes=<last>-. When it gets a416with a different total length, it knows the file was rolled over and starts again from the end. Multi-range requests are rejected with416instead of sending the whole (possibly large) file. ARangeheader that is invalid or uses an unknown unit is ignored, as RFC 9110 allows.FileShare.ReadWrite | FileShare.Delete, so the logger can keep writing, rolling over, or deleting the file while it is being read.Content-Lengthheader announced, and Kestrel aborts the response.text/plainwithout a charset, because the encoding depends on the logging provider's configuration. Clients that send an incompatibleAcceptheader get406, like other actuators.The handler only resolves the file path and stays independent of
HttpContext. The middleware does the HTTP and file I/O work, following the same split as the heap dump actuator.Testing
LogFileActuatorTest(32 tests): default/custom settings, the off-by-default rule, the full download for all three host builder types (compares raw bytes, including non-ASCII content), each kind of byte range, unsatisfiable and multi-range requests, empty files, invalid or unknown range units, a step-by-step replay of Spring Boot Admin's tailing requests (including roll-over), reading a file that a writer still holds open, relative path resolution, a missing file (no warning logged), enabled withoutFilePath(warning logged), theAcceptcheck, and the_linksentry appearing only whenFilePathis configured.AllActuatorsTest(middleware list and actuator count).EndpointAuthorizationTestcovers every endpoint type, so it now configures a temporary log file.Steeltoe.Management.Endpoint.Testpasses on net10.0. The one exception isAspNetDefaultPort_SameManagementPortConfigured_OnlyActuatorAccessible, which fails on my machine onmaintoo because macOS's AirPlay receiver listens on port 5000. Release build (warnings as errors) passes for net8.0, net9.0 and net10.0.ConfigurationSchema.jsonwas regenerated, and ReSharper "Steeltoe Full Cleanup" produces no changes on the modified files.bytes=-307200returned 206 with the correctContent-Range,bytes=<total>-returned the newly appended lines, and past-the-end and multi-range requests returned 416 withbytes */<total>.Fixes #1496
Quality checklist
If your change affects other repositories, such as Documentation and/or Samples, add linked PRs here.
(The XML doc comments and
ConfigurationSchema.jsonare updated. I'm happy to follow up with a page in the Documentation repository once the design is settled.)(Not applicable: no copied code.)