Reply To: Unit tests for FlowGraphs

[kalms]

(We are currently not working with FlowCanvas – we have for various reasons (not related to FlowCanvas) switched to working with Unreal – so I’m writing this from memory)

Our #1 issue with FlowCanvas today is that the FlowGraphs do not diff/merge at all using the standard text-based merge tools. I have personally done the following dance a large number of times:

1. Extract the “serializableObject” of the previous version of the .asset from source control
2. Extract the “serializableObject” of the current version of the .asset
3. Paste both into https://jsonformatter.curiousconcept.com
4. Store both results into a pair of text files
5. Use a diff tool to compare the text files
6. Look for changes (is it just a single value changed? one or a few lines? a group of lines? large structural changes?)

If the JSON data was in some multi-line format I would have been able to use the version control’s system tool to perform the diff in just 1 step.

 

We have run into problems a number of times that have to do with custom functions, and reentrancy. This usually shows up with helper functions that are of a reusable nature – or when there is a bit of ping-pong going on between multiple FlowGraphs. We have worked around those problems by not using functions as often as we’d like; the “real” solution would have been for FlowCanvas to pass parameters via a call stack.

 

We have found it problematic that the “Finish” node terminates all flows within an entire FlowGraph. This makes logic that ping-pongs between FlowGraphs not work as intended. We worked around that by limiting ping-pong, and carefully structuring logic so that early termination of certain flows would have no effect. A “proper” solution would be to provide an alternative to Finish that does not kill other currently-running threads (so something that is more of a Return operation), and/or provide an alternative to Finish that only kills threads that have been invoked during the currently-running event (which, in turn, requires tracking which Flows have been created during which Event).

 

We have found the CPU cost for cloning FlowGraphs problematic.

 

If we had been doing testing of FlowGraphs with other FlowGraphs as test rigs, we would probably have wanted some more tooling for inspecting the FlowGraph-under-test from the test rig. Tools for easily inspecting internal state, for example: can I easily get a list of the internal variables within the FlowGraph-under-test? can I replace a node in the FlowGraph-under-test with a spy that reports things back to the test rig? We did not need those – we were happy to limit ourselves to inspecting state outside of the FlowGraph itself.

 

Those are the main things we’ve run into. For our use case (which may be more complex than the average user’s), we would have benefited from having state separated from logic. That would have made graph instantiation quicker. Cheaper graph instantiation would allow functions to be represented as separate graphs. That would make function-local variables viable. Passing function arguments on a stack would finally make it possible to create reentrant logic. All this would probably make writing custom nodes more complicated, and it may result in overall slower execution of the type of code that FlowCanvas users create today.