Support model cascades #417
Description
For map and filter operations; now that we have structured output modes; we should be able to get the logprobs of the token and support model cascades.
The op config can specify the (model name, per-class thresholds) as a dictionary. The threshold dict should include the class "default", which is a threshold for all items that do not belong in any of the other classes. This way we can support map operations.
We should also support the ability to build cascades. I am not sure what the cleanest way to do this is.
We should also adhere to the task cascade algorithm; where we can set up an arbitrary number of models/tasks to choose from; and assemble the lowest-cost cascade.