feat: Backends return Backends.QueryError struct for errors

[msmithstubbs]

Add a standard %QueryError{} struct to be returned by backend adaptors on an unsuccessful query. This allows flexibility on which error messages are logged and which are show to end users.

• Adds %QueryError{} to normalise the shape of errors returned by backend adaptors
• Updates BigQuery, Postgres, and Clickhouse adaptors to use QueryError when returning an error
• Adaptors set QueryError.kind based on the backend error message:
  • A missing field error would be :invalid_query
  • Other errors are :backend_query or :connection_error
• QueryError.log will log errors, excluding :invalid_query
  • User interface layer is not longer responsible for logging backend errors, but may still log user query errors
  • If a user has system_monitoring enabled the logged error messages are included
• User facing errors only include the backend error message for :invalid_query

EndpointsLive

QueryLive

AlertsLive

Closes O11Y-1819 and O11Y-1970

[msmithstubbs]

Error messages could be in the respective backend adaptors but thought it better to keep this in the UI layer.

[Ziinc]

needs some adjustment for surfacing of errors.
need some distinction between error messages:

• end consumer errors - needs to be generic and shouldn't leak information like underlying backend errors
• operator user - should surface underlying backend errors, but logged out to Logger and surfaced to user via system.logs with the user_id metadata key set.
• admin user - surfaced to admin via Logger, such as connection pool errors, which might not have a user_id associated, and needs the most detail, on par with operator user

[Ziinc]

i think we should restrict what is surfaced to users, in the event that sensitive values get surfaced. things like invalid columns etc are columns are fine, probably can match on the message prefix and give a more generic message for the rest.

[Ziinc]

This applies for all backends, not just for postgres

[msmithstubbs]

Make sense.

operator user - should surface underlying backend errors, but logged out to Logger and surfaced to user via system.logs with the user_id metadata key set.

How do we identify an operator user?

[msmithstubbs]

Revised approach:

1. Backend adaptors call Logger for any errors and return QueryError struct
2. Drop message attr from QueryError
3. UI layer doesn't log error failures, but handles taking a QueryError struct and determining correct message to show

[msmithstubbs]

@Ziinc

• end consumer errors - needs to be generic and shouldn't leak information like underlying backend errors

Does this include API consumers? Example:

logflare/test/logflare_web/controllers/endpoints_controller_test.exs

Lines 266 to 267 in b1a389c

	assert response.error =~
	"Multiple CTEs available (first_cte, second_cte, final_data). You must specify which one to query using `f:name`"

Should this just return a generic error message?

[msmithstubbs]

This will be logged already by QueryError.log - do we need the specific wording of maybe_warn_reservation_error ? If not, we can remove it.

[msmithstubbs]

Logging now handled by backend adaptor, but note message ("Alert query execution failed with bad response") will have changed.