Skip to content

Fix SPARQL HAVING without GROUP BY in two constraint queries#82

Open
Haigutus wants to merge 1 commit into
entsoe:mainfrom
Haigutus:fix/sparql-having-without-group-by
Open

Fix SPARQL HAVING without GROUP BY in two constraint queries#82
Haigutus wants to merge 1 commit into
entsoe:mainfrom
Haigutus:fix/sparql-having-without-group-by

Conversation

@Haigutus

@Haigutus Haigutus commented Jul 6, 2026

Copy link
Copy Markdown

Fixes the HAVING-without-GROUP BY defect reported in #70 (comment) (part of the #70 findings).

SPARQL 1.1 defines HAVING only over grouped solution sets (§11.3: "HAVING operates over grouped solution sets, in the same way that FILTER operates over un-grouped ones"). Both affected queries use a trailing HAVING with no GROUP BY, no aggregate inside the HAVING expression, and a non-aggregate projection — a construct with no conforming interpretation (a grouped reading would also violate the §11.4 projection restriction, which allows only aggregates/constants/group keys to be projected). Engines therefore diverge: rdflib/pySHACL silently treat it like a filter, while strict engines reject the query outright (QLever: Invalid SPARQL query: A HAVING clause is only supported in queries with GROUP BY) — so whether these constraints ran at all depended on validator leniency.

Changes — semantics-preserving: in both queries the tested variable is computed by an inner GROUP BY subquery and is a plain bound variable at the outer level, so the unambiguous equivalent is a FILTER at the end of the outer WHERE block:

  • equ:PowerTransformerEnd.ratedS-valueRange2windingSparql (61970-301_Equipment-AP-Con-Complex-SHACL.ttl): HAVING(?ends=2)FILTER (?ends=2)
  • sm600-2:RegulatingControl-samePointSparql (61970-600-2_AllProfiles-AP-Con-Complex-SolvedMAS-SHACL.ttl): HAVING (?count>1)FILTER (?count>1)

Verification: both files parse; validating a full CGMES 3.0 Equipment instance yields identical results before/after the change on rdflib, and QLever now accepts and agrees with both queries.

This addresses only the HAVING defect — the other #70 findings (fake sh:path rdf:type, per-instance SPARQL performance) are left to the shape restructuring discussed there. The same two defects are also present on the cgmes-v3-0 release branch, where this fix applies unchanged.

SPARQL 1.1 defines HAVING only over grouped solution sets (sec. 11.3:
'HAVING operates over grouped solution sets, in the same way that FILTER
operates over un-grouped ones'). Both queries use a trailing HAVING with no
GROUP BY, no aggregate in the HAVING expression, and a non-aggregate
projection - a construct with no conforming interpretation (a grouped
reading would also violate the sec. 11.4 projection restriction). Engines
therefore diverge: rdflib/pySHACL silently treat it as a filter, strict
engines (QLever) reject the query, so the constraints ran only on lenient
validators.

In both queries the tested variable (?ends / ?count) is computed by an
inner GROUP BY subquery and is a plain variable at the outer level, so the
unambiguous equivalent is a FILTER at the end of the outer WHERE block:

- equ:PowerTransformerEnd.ratedS-valueRange2windingSparql
  (61970-301_Equipment-AP-Con-Complex-SHACL.ttl): HAVING(?ends=2)
- sm600-2:RegulatingControl-samePointSparql
  (61970-600-2_AllProfiles-AP-Con-Complex-SolvedMAS-SHACL.ttl): HAVING (?count>1)

Verified: both files parse, and validation results over a full CGMES 3.0
Equipment instance are identical before/after the change (rdflib and QLever
now agree).
Haigutus pushed a commit to Haigutus/triplets that referenced this pull request Jul 6, 2026
…the results

Constraint queries now run exactly as authored — _rewrite_bare_having is
deleted. Cheap strict pre-validation does not exist in the stack (rdflib's
parser is precisely the lenient one that accepts these constructs, and
qlever's parse IS the query call), so invalid queries are surfaced after
failure, where they can be made genuinely informative:

- direct sparql.query(): a query qlever rejects raises ValueError carrying
  qlever's message plus the query text, so the failure is immediately
  actionable
- sh:sparql delegation: the constraint is re-evaluated on the lenient rdflib
  engine (the report stays complete) and a triplets:invalidSparql Warning
  row names the defective shape and quotes the engine error; when every
  engine fails, the row escalates to Violation. Rows deduplicate across the
  sh:targetClass fanout, so a defective shape is flagged once. The log
  warning includes the full query for debugging.

Verified against the real defective ENTSO-E Equipment profile: full
validation completes in ~8.5 s with exactly one invalidSparql flag for
PowerTransformerEnd.ratedS-valueRange2winding (fixed upstream in
entsoe/application-profiles-library#82); results identical to before.
@VladimirAlexiev

Copy link
Copy Markdown
Collaborator

@Haigutus I agree with the fix

  • Surprisingly, jena qparse thinks trailing HAVING without GROUP is valid.
    Maybe I need qparse --strict? Still thinks it's valid.
  • Furthermore, rdf4j (GraphDB) reports error variable 'this' in projection not present in GROUP BY.
  • Oh my, we need to pick a SPARQL validator!
qparse --query validate/shacl-sparql/PowerTransformerEnd.ratedS-valueRange2winding.rq
PREFIX  adms: <http://www.w3.org/ns/adms#>
PREFIX  cim:  <https://cim.ucaiug.io/ns#>
PREFIX  cim16: <http://iec.ch/TC57/2013/CIM-schema-cim16#>
PREFIX  xsd:  <http://www.w3.org/2001/XMLSchema#>
PREFIX  cim17: <http://iec.ch/TC57/CIM100#>
PREFIX  dm:   <http://iec.ch/TC57/61970-552/DifferenceModel/1#>
PREFIX  eu:   <https://cim.ucaiug.io/ns/eu#>
PREFIX  nc:   <https://cim4.eu/ns/nc#>
PREFIX  rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX  md:   <http://iec.ch/TC57/61970-552/ModelDescription/1#>
PREFIX  dcterms: <http://purl.org/dc/terms/>
PREFIX  dcatcim: <https://cim4.eu/ns/dcatcim#>
PREFIX  dcat: <http://www.w3.org/ns/dcat#>
PREFIX  prov: <http://www.w3.org/ns/prov#>

SELECT  ?this ?value ?ends
WHERE
  { { SELECT  ?this (COUNT(?typeend) AS ?ends)
      WHERE
        { ?this ^cim:PowerTransformerEnd.PowerTransformer/rdf:type ?typeend }
      GROUP BY ?this ?typeend
    }
    ?end1  cim:PowerTransformerEnd.PowerTransformer  ?this ;
           cim:TransformerEnd.endNumber  1 ;
           cim:PowerTransformerEnd.ratedS  ?value
    BIND(EXISTS { ?end1  cim:PowerTransformerEnd.ratedS  ?v } AS ?hasvalue)
    ?end2  cim:PowerTransformerEnd.PowerTransformer  ?this ;
           cim:TransformerEnd.endNumber  2 ;
           cim:PowerTransformerEnd.ratedS  ?ratedsend2
    BIND(EXISTS { ?end2  cim:PowerTransformerEnd.ratedS  ?r } AS ?hasratedsend2)
    FILTER ( ( ( ( ?hasvalue = true ) && ( ?hasratedsend2 = true ) ) && ( ?end1 != ?end2 ) ) && ( ?value != ?ratedsend2 ) )
  }
HAVING ( ?ends = 2 )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants