Processor overhaul of options and validate_query method

[dale-wahl]

Merge order: #619 -> #620 -> this PR

Building on the two previous PRs, this updates processors with better checks in parse_all and validate_query so that they "fail fast" and provide more information to the user on how to select options.

A few bugs were also identified that could have provided odd or unexpected results. collocations asked for three-word groups in a two-word window and quietly gave you two-word groups instead. tfidf with a bad SMART string would finish empty and successful. column-network with one column on both sides read the whole dataset to produce a network with no edges (assuming no loops/comma separation).

I also tightened some option by adding coerce_type and/or min/max bounds. There was also a bug with multi-selects whose default was a bare string in a template check that became a sub-string check.

What I did

Two halves. I added checks that could be expressed in the option itself, it is (e.g. [#620]'s added mandatory flag or the existing coerce_type/min/max). And in other cases created or updated the validate_query method.

In some instances, I was able to further reduce/remove validate_query mostly with the new mandatory flag.

validate_query, where a declaration cannot express it

Processor	Now caught before the run
hash-similarity-network	the same column chosen for hashes and IDs — each value is taken out of the item as it is read, so the second read found nothing and the processor crashed partway through
column-network	one column on both sides with neither comma-splitting nor loops: every edge is a value to itself, all dropped, so it always finished empty. Still allows the co-tag/co-word presets, which use one column deliberately
collocations	asking for more co-words than the window can hold — this used to quietly shrink your groups instead
tfidf	an impossible occurrence range, and a SMART string gensim won't accept (which finished empty and successful). Asks gensim rather than keeping our own copy of its rules
topic-modeller	min/max document frequency check was the wrong way round, and we reject a maximum of 0
overtime-vocabulary	no vocabulary and no custom words: would read everything and produce nothing
wordcloud	could crash if non-numeric limit was used
attribute-frequencies, lexical-filter, convert-text	check a filter that isn't a valid regular expression
audio-to-text, consolidate-urls, extract-urls-filter, similar-word2vec	confirmation before something slow or costly (see below)

Bugs found on the way

• preset-histogram — the guard removing "Overall" tested "all" in count_options["timeframe"], i.e. the setting's keys (type, default, options), never the choices. It was not removed and could be picked, which would fail. Now ["timeframe"]["options"].
• clarifai-api — amount was marked sensitive, a copy-paste from the API key, so an image count was scrubbed from stored parameters as if it were a secret.

Added tests

test_option_default_validity and test_option_keys_known now check the options a processor builds for a parent with columns as well as without. Processors routinely offer different options for each — a text field becomes a
list of columns to pick from — and only the column-less shape was ever checked, which is allowed those string defaults above to survive.

parse_all also names the module and the file:line of the option definition when it warns about a bad default, so the warning can be acted on: "columns" is used by dozens of processors, and the warning is logged far from the module at
fault. Hopefully we can clean up any stragglers I missed.

Notes

Builds on [#620] (the mandatory flag and the parsing contract) and targets that branch; it should retarget to master once #620 merges.

prompt_compass is deliberately not here. You'll see fixes to it shortly in master (Fixing it turned up a stale api-/thirdparty prefix and a couple of KeyErrors; also having some difficulty using DMI's ollama remotely so want to look into that separately).