Add interactive optimization mode

[ricardoV94]

I have resorted to patching the optimizer verbose to print the FunctionGraph before and after intermediate changes, so I thought about making this a builtin option. If you want to see how it looks locally, you can try this snippet:

import pytensor
import pytensor.tensor as pt

x = pt.vector("x")
y = pt.log(pt.sum(pt.exp(x)))

pytensor.config.optimizer_interactive = True
pytensor.config.optimizer_interactive_skip_rewrites = "MergeOptimizer, constant_folding"
pytensor.function([x], y)

I quite like it, except for the fact that the Differ.compare returns results line-by-line, when it would sometimes be much more readable if the additions/deletions from the same "block" were grouped together. If anyone would like to take a stab at grouping the differences that would be great!

[michaelosthege]

is this faster?

Suggested change

	if print_topo_order:
	topo_orders.extend([topo for item in obj.maker.fgraph.outputs])
	else:
	topo_orders.extend([None for item in obj.maker.fgraph.outputs])
	if print_topo_order:
	topo_orders.extend([topo] * len(obj.maker.fgraph.outputs))
	else:
	topo_orders.extend([None] * len(obj.maker.fgraph.outputs))

[ricardoV94]

It's 2x faster (with 5 elements), but the rest of this function uses the list comprehension approach. So I would either change them all or keep as is.

[michaelosthege]

Suggested change

	if print_topo_order:
	topo_orders.extend([topo for item in obj.outputs])
	else:
	topo_orders.extend([None for item in obj.outputs])
	if print_topo_order:
	topo_orders.extend([topo] * len(obj.maker.fgraph.outputs))
	else:
	topo_orders.extend([None] * len(obj.maker.fgraph.outputs))