-
Notifications
You must be signed in to change notification settings - Fork 0
Harden RTSW NOAA client for WAF 202 and empty bodies #8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
08d40da
Harden RTSW NOAA HTTP handling for WAF 202 and empty bodies.
564ab75
Sanitize NOAA RTSW JSON NaN/Infinity literals before deserialization.
d4160ab
Address review: EnsureNoaaSuccess for all NOAA clients, lazy sanitize…
a77ba92
Bump Framework packages to 10.0.7 and document 1.2.2 NoaaClient fixes.
ead2e2a
Map AuroraScienceHub.* to nuget.org and GitHub Packages for stable/pr…
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| using System.Net; | ||
| using AuroraScienceHub.Framework.Http; | ||
|
|
||
| namespace AuroraScienceHub.Integrations.NoaaClient.Http; | ||
|
|
||
| /// <summary> | ||
| /// NOAA-specific HTTP response validation helpers. | ||
| /// </summary> | ||
| internal static class NoaaHttpResponseExtensions | ||
| { | ||
| private const string WafActionHeaderName = "x-amzn-waf-action"; | ||
| private const string WafChallengeAction = "challenge"; | ||
|
|
||
| /// <summary> | ||
| /// Ensures the NOAA response is not a WAF challenge and has a successful status code. | ||
| /// </summary> | ||
| public static async Task EnsureNoaaSuccessAsync( | ||
| this HttpResponseMessage response, | ||
| Uri? requestUri = null, | ||
| CancellationToken cancellationToken = default) | ||
| { | ||
| cancellationToken.ThrowIfCancellationRequested(); | ||
| ThrowIfWafChallenge(response, requestUri); | ||
| await response.EnsureSuccess().ConfigureAwait(false); | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// Throws when the NOAA response body is empty after a successful status code. | ||
| /// </summary> | ||
| public static void ThrowIfEmptyBody( | ||
| this HttpResponseMessage response, | ||
| ReadOnlySpan<byte> body, | ||
| Uri? requestUri = null) | ||
| { | ||
| if (body.Length > 0) | ||
| { | ||
| return; | ||
| } | ||
|
|
||
| var contentLength = response.Content.Headers.ContentLength; | ||
| throw new HttpRequestException( | ||
| $"NOAA returned an empty response body (HTTP {(int)response.StatusCode}, ContentLength={contentLength}, Uri={requestUri}).", | ||
| inner: null, | ||
| statusCode: response.StatusCode); | ||
| } | ||
|
|
||
| internal static string? GetWafAction(HttpResponseMessage response) | ||
| { | ||
| if (response.Headers.TryGetValues(WafActionHeaderName, out var values)) | ||
| { | ||
| return values.FirstOrDefault(); | ||
| } | ||
|
|
||
| if (response.Content.Headers.TryGetValues(WafActionHeaderName, out values)) | ||
| { | ||
| return values.FirstOrDefault(); | ||
| } | ||
|
|
||
| return null; | ||
| } | ||
|
|
||
| private static void ThrowIfWafChallenge(HttpResponseMessage response, Uri? requestUri) | ||
| { | ||
| var wafAction = GetWafAction(response); | ||
| if (response.StatusCode != HttpStatusCode.Accepted | ||
| || !string.Equals(wafAction, WafChallengeAction, StringComparison.OrdinalIgnoreCase)) | ||
| { | ||
| return; | ||
| } | ||
|
|
||
| throw new HttpRequestException( | ||
| $"NOAA request blocked by AWS WAF challenge (HTTP {(int)response.StatusCode}, {WafActionHeaderName}={wafAction}, Uri={requestUri}).", | ||
| inner: null, | ||
| statusCode: response.StatusCode); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,43 +1,96 @@ | ||
| using System.Net; | ||
| using System.Text.Json; | ||
| using AuroraScienceHub.Framework.Http; | ||
| using AuroraScienceHub.Framework.Json; | ||
| using AuroraScienceHub.Integrations.NoaaClient.Http; | ||
| using AuroraScienceHub.Integrations.NoaaClient.Rtsw.Responses; | ||
| using AuroraScienceHub.Integrations.NoaaClient.Utilities; | ||
| using Microsoft.Extensions.Logging; | ||
| using Microsoft.Extensions.Options; | ||
|
|
||
| namespace AuroraScienceHub.Integrations.NoaaClient.Rtsw; | ||
|
|
||
| /// <inheritdoc /> | ||
| internal sealed class RtswClient : IRtswClient | ||
| { | ||
| private static readonly JsonSerializerOptions s_jsonOptions = DefaultJsonSerializerOptions.Create(); | ||
|
|
||
| private readonly HttpClient _httpClient; | ||
| private readonly Uri _baseUrl; | ||
| private readonly ILogger _logger; | ||
|
|
||
| /// <summary> | ||
| /// Initializes a new instance of the <see cref="RtswClient"/> class. | ||
| /// </summary> | ||
| public RtswClient( | ||
| HttpClient httpClient, | ||
| IOptions<NoaaClientOptions> options) | ||
| IOptions<NoaaClientOptions> options, | ||
| ILogger<RtswClient> logger) | ||
| { | ||
| _httpClient = httpClient; | ||
| _baseUrl = options.Value.RequiredServerUrl; | ||
| _logger = logger; | ||
| } | ||
|
|
||
| public async Task<IReadOnlyList<MagnetometerRecord>> GetMagnetometerDataAsync(CancellationToken cancellationToken) | ||
| { | ||
| var url = new Uri(_baseUrl, "json/rtsw/rtsw_mag_1m.json"); | ||
| var records = await _httpClient | ||
| .GetFromJsonOrDefaultAsync<List<MagnetometerRecord>>(url, cancellationToken) | ||
| .ConfigureAwait(false); | ||
|
|
||
| return records ?? []; | ||
| return await GetRtswJsonAsync<List<MagnetometerRecord>>(url, cancellationToken).ConfigureAwait(false) | ||
| ?? []; | ||
| } | ||
|
|
||
| public async Task<IReadOnlyList<SolarWindPlasmaRecord>> GetSolarWindPlasmaDataAsync(CancellationToken cancellationToken) | ||
| { | ||
| var url = new Uri(_baseUrl, "json/rtsw/rtsw_wind_1m.json"); | ||
| var records = await _httpClient | ||
| .GetFromJsonOrDefaultAsync<List<SolarWindPlasmaRecord>>(url, cancellationToken) | ||
| return await GetRtswJsonAsync<List<SolarWindPlasmaRecord>>(url, cancellationToken).ConfigureAwait(false) | ||
| ?? []; | ||
| } | ||
|
|
||
| private async Task<TResponse?> GetRtswJsonAsync<TResponse>( | ||
| Uri requestUri, | ||
| CancellationToken cancellationToken) | ||
| { | ||
| using var response = await _httpClient.GetAsync(requestUri, cancellationToken).ConfigureAwait(false); | ||
| return await ReadRtswJsonOrThrowAsync<TResponse>(response, requestUri, cancellationToken).ConfigureAwait(false); | ||
| } | ||
|
|
||
| private async Task<TResponse?> ReadRtswJsonOrThrowAsync<TResponse>( | ||
| HttpResponseMessage response, | ||
| Uri requestUri, | ||
| CancellationToken cancellationToken) | ||
| { | ||
| var wafAction = NoaaHttpResponseExtensions.GetWafAction(response); | ||
| var contentLength = response.Content.Headers.ContentLength; | ||
|
|
||
| _logger.LogInformation( | ||
| "RTSW HTTP response Uri={RequestUri} StatusCode={StatusCode} WafAction={WafAction} ContentLength={ContentLength}", | ||
| requestUri, | ||
| (int)response.StatusCode, | ||
| wafAction, | ||
| contentLength); | ||
|
|
||
| await response.EnsureNoaaSuccessAsync(requestUri, cancellationToken).ConfigureAwait(false); | ||
|
|
||
| if (response.StatusCode == HttpStatusCode.NoContent) | ||
| { | ||
| response.ThrowIfEmptyBody(ReadOnlySpan<byte>.Empty, requestUri); | ||
| } | ||
|
|
||
| var rawBytes = await response.Content | ||
| .ReadAsByteArrayAsync(cancellationToken) | ||
| .ConfigureAwait(false); | ||
|
|
||
| return records ?? []; | ||
| response.ThrowIfEmptyBody(rawBytes, requestUri); | ||
|
|
||
| var sanitized = NoaaJsonSanitizer.Sanitize(rawBytes); | ||
| if (sanitized.ReplacementCount > 0) | ||
| { | ||
| _logger.LogWarning( | ||
| "NOAA RTSW JSON sanitized {ReplacementCount} non-standard numeric literals to null. Uri={RequestUri}", | ||
| sanitized.ReplacementCount, | ||
| requestUri); | ||
| } | ||
|
|
||
| return JsonSerializer.Deserialize<TResponse>(sanitized.Bytes.Span, s_jsonOptions); | ||
| } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.