Note: Deprecated!

An introduction

In short, a flow is a template definition of recognizable data. It is used to transform a stream of neurons into another stream of neurons (doesn’t sound very useful, but trust me, it is).  To get a visual image on how a flow works, you can perhaps think of those toddler toys where the child needs to fit different types of blocks into different types of wholes (see image).

Well, a Flow is the yellow filter at the top of the box. As the blocks pass along, a flow tries to find a set that fits it’s filter. When it finds one, it collects this set, tags it, and stores it for further use. This process is done by the flow recognition algorithm.

A flow is a template definition, used to recognize sequences of data in a stream.

flow items

So, what is this filter that’s supposed to recognize the data, actually constructed of? Well, flow definitions are very (mmm, perhaps not so very) similar to Coco/R definitions. In short, you use constants, other (nested) flows, loops and options to build up a definition. Here’s a short example:
This flow can recognize the sequence: Agent verb possibly followed by a not. It’s actually the definition for the simple past tense (there’s an extra hidden filter on ‘verb’ which I’m not showing yet for this example). ‘Agent’ is underscored, indicating that it is another flow, verb is not, so it’s a static, like not. Which is surrounded by strait brackets, indicating that it’s optional. So basically, this flow can recognize statements like: I swam, you dug, the brown men found not,… Without the filter (like here), sentences like: I fish, they eat not,… would also be recognized.

Statics and nested flows

Most of the items in a flow definition tend to be statics and other, nested flows. This forms the meat, the data that can be found in the input stream itself. When 2 statics are declared in sequence, the same sequence needs to be found in the input, without anything in between.

Nested flows can easily be recognized. They are underlined. If you double-click on them, you will jump automatically to the flow (you can navigate back and forward between the selected flows using the navigation commands). Behavior-wise, nested flows are the same as statics. If you declare 2 nested flows in sequence, the input stream should contain the data for the 2 flows in the same sequence.

Floating flows

A floating flow is a special type of flow that can be recognized anywhere in the stream without having to be specifically declared in any other flows. They are mostly used for things like spaces and newlines. You usually don’t integrate them into other flows, as nested.

Floating flows can be destructive or not. When a destructive flow is found, it will break up the recognition of other data in the stream. So this type of flow can not be found between 2 statics, but can be between 2 nested flows or between options, loops or conditional parts (see further). If you need to have a floating flow between 2 statics, it needs to be non destructive. The type of flow can be assigned from a context menu on the flows: you can select for normal flows (not floating), floating (visualized by a green bar in front of the flow) and non destructive floating (displayed using a blue bar).

You can also specify if the data that a floating flow finds is discarded or not. Discarding the data can be useful in cases where you are really not interested in it, like for empty space. This is also selected from the context menu on the flows and is visualized by using an extra bar at the end of the flow.

Options

With an option you can define a number of different paths, from which 1 can or has to be selected. The following example of options defines the ‘can’ verb:A blue block (also called ‘conditional part’) represents a single path. If the frontal bracket of the options contains a vertical line, 1 path has to be selected, if there is no line, no path has to be selected but 1 can be. The content of a conditional has to be recognized in the same sequence as the way that it is defined and can consist of statics, flows, loops and other options, but no conditional parts. The option itself can only contain conditional parts.

Just like statics and nested flows, you can also reuse the same option in different places. In fact, the more you can do this, the more processing power can be saved. The same goes for the conditional parts (the blue blocks). These can also be shared by different options (and loops). This can be done by drag and drop from within the editor.

Loops

Loops are very similar to options, they also consist of conditional parts and they can also be defined with a vertical line in front, indicating that 1 path has to be selected or not. The main difference with options is that a loop can recognize 0, 1 or more of it’s conditional parts in sequence. They are declared using curly brackets, like in the following example: This loop simply declares a sequence of digits, with at least 1 required digit. In other words, it’s the definition for an integer.

Loops can have some more lines in the front to declare extra info. If there is a green line present, to the right of the black one, it means there can’t be a floating flow in between 2 conditional parts, otherwise this is allowed (see floating flows). A red line to the left of the black one, indicates exactly the opposite: 2 conditionals need to have a floating flow in between them for a valid parse (not often used).

As a final example, here’s the entire scanner definition which is used to recognize words, numbers (integer and doubles), signs and spaces:

Note: Deprecated!

Intro

Time for the second demo overview: the Scanner.  It builds on most of the ideas found in the first demo but it goes way further, and actually does something very useful (although you wouldn’t say it at first).  It’s probably going to be a lengthy piece so I’m thinking of cutting it in 2 or maybe even 3 parts. Anyway, lets first start it up, either through the start menu shortcut (in the Demo’s sub folder, conveniently called Scanner demo), or by opening it in NND (File/Open, select the ‘My documents/NND/Demos/Scanner’ folder). Once the project is loaded, you should see a single text communication channel open (called Text sin), if this is not the case, go to View/Communication channels/Text sin and make certain that it is is selected.

Overview

Lets get a taste of what it does, so enter some text (or leave the one that’s already there) and press the ‘send’ button (or enter).

You’ll notice that it basically does the same thing as the echo demo: the input text is echoed back, except that it’s a bit slower. If you had the debugger tab open, you probably also noticed a lot more activity, so something more must be going on.

And indeed, if you take a closer look to the text sin channel, in the upper section, you can see the neurons that were send to the network (on the left) and those that were returned (on the right), which are different. This was not the case with the echo demo, it simply sent all the incoming neurons back out, as they were. In this demo though, we get back something completely different: TextNeurons, that represent the same thing as the int neurons that were sent as input (if you regard them as ASCII characters). Hence the name of the demo, it’s a scanner.

Converting a stream of ASCII chars into words, integers, doubles and signs is a pretty useful feature and that’s all this demo is capable of doing, but that’s only because I stopped there. You see, in the background, is a general purpose algorithm that converts an input stream of neurons into a single result cluster, using any and all of the flows that are defined in the network. This means that you can use the same algorithm with many different flow definitions. I have simply defined some to recognize words, integers and doubles. You could go further and add flows to find verbs, sentence subjects,.. (in fact, that’s what the AICI 1 demo does). You could even go further still and create flows for visual objects or audio fragments, the same algorithm can be used. Unfortunately though, the editor doesn’t yet support such types of displays for flows (will probably be added somewhere in the future though).

Details

So how is the translation actually performed? To explain this, let me first recap some of the basic concepts of neurons and flows:

• The different types of  data available to a neuron are: incoming and outgoing links, possibly one or more parent clusters, for clusters possibly 1 or more children and a meaning. And finally value neurons also have their value of course. This is important, cause when the translation process starts, this is all the available information.
• Flows are nothing more than clusters that contain flow items, which can be statics or conditionals (loops and options). These in turn can only contain conditional parts. They represent a single branch of the decision tree. Parts can again have the same data as flows: statics or conditionals. So if you are a neuron (a static, part, conditional or flow), you can always look up into your list of parents to see in which flows and parts it is used.

Now, if you recall from the first demo, an input starts by creating an IntNeuron for each ASCII value, which is linked to another neuron using the ‘Letter’ neuron (ID 109). These are all put on the execution stack and the processor starts (the Rules code on the Letter neuron is executed).

So both neurons are new and only have each other as links. This means that we can’t use links or parent/child relationships to resolve the first step, but instead must use something different. The only other thing that remains is the value of the int neurons, so this is compared against some constants to see if they are digits (0..9) alpha numeric (a..z+A..Z), spaces | returns, or something else.  This comparison results in the creation of 1 new neuron per input neuron: the result cluster, in which we store the integer. One of these clusters (or it’s duplicate, due to a split) will eventually store the end result.  This cluster is linked to one of 3 static neurons: Digit, Alpha or Space (signs like . or , are handled a bit differently, this will be explained later). Naturally, if the integers would represent color values, we would use other starting points than digit or alpha. In other words, this first part is variable according to the type of input and the required accuracy of the algorithm. As meaning, we use  the start of the ‘flow recognition’ algorithm, called ‘Stage 1.1’ and put the result cluster back on the stack.  It’s important to put this one back on the stack, and not the item we are looking for. That’s because the algorithm can perform numerous splits and we want the result to be duplicated not the searchable, cause the contents of the list are continuously modified and we don’t want the result of one processor to be modified by  another one (I had to learn this the hard way).

After this initial step, the actual recognition algorithm kicks in. This consists out of 4 stages, grouped by 2. Meaning that  stage 1.2 is executed immediately after stage 1.1 for each neuron (this is the same for stage 2,1 and 2.2), but at the end of stage 1.2 and 2.2 all the result neurons are collected into a single global. Only after the last link of the last item on the stack has  been processed, are all the result clusters put back on the stack, with links for the next stage.  This is done for allowing to group items together.

The different stages are:

• Stage 1.1 (search parts/flows): Find conditional parts or flows in the list of parents of the searchable. If there are multiple results, perform a split for each, after the result list has been filtered (these are the shortcuts). If there are no results and this is the only and last item still on the stack, the end result has been found.
• Stage 1.2 (sequence-combine and filter): Check if items are sequential (2 flow items declared after each other in the same parent list, which is a part or flow) and handle floating flows, which are allowed to appear anywhere in the input stream, but which break up the sequence of other items.  Different actions can be performed if the order of the items is not ok: try to solve further or exit without result. Results of sequential items are grouped together.
• Stage 2.1 (search conditionals): Find conditionals in the list of parents of the searchable if this is a conditional part, otherwise the stage is simply skipped. If there are multiple results, perform a split for each after the result list has been filtered (not yet completely implemented at this stage). If there are no results, there is an error in the flow definition.
• Stage 2.2 (process loops and sync-points): If there was a conditional found in the previous stage, check if this is a loop. If so, and the previous item is of the same loop, combine the results. Also start a sync-point (will be explained later) if this was defined on the conditional.

These 4 stages are repeated until there is only 1 result cluster on the stack that represents the end result of a flow which is no longer used in any other flows. This cluster is made the result of the  split for the processor it ran on.  Off course, because there were possibly many splits, there could be many results. These will all be presented in the split-callback cluster (you need to provide a code cluster to the Split instruction, which will be called when all sub processors are done). In this demo, the result is sent back to the sin that caused the input, in a normal situation thought, this will simply start another process, as is done in the AICI 1 demo.

During this whole process, the algorithm is capable of executing callback code (attached to the statics, conditionals, parts and flows) at certain specific moments in the code. This is where the magic happens. The following types of callbacks are possible (together with  their execution time):

• Flow code: this code cluster is executed when a flow has been recognized in the stream. The ‘Result’ variable (ID 1822) contains all the neurons that match the flow. It’s here for instance that the int neurons are converted to a single word using the CiToS (Cluster with ints to string) instruction.
• Filter flow code: this code cluster is called from stage 1.1  (or 2.1, but this is not yet completely implemented) just after all the next items were retrieved from the list of parents of the searchable (stored in the CurrentTo variable). It allows the flow item to determine if it is a valid result, given the current state of the network. This is done by checking the contents of a number of globals, like ‘Prev stage item’ (ID 1956), which contains the previously processed neuron.

In the next post, I’ll go deeper into the specifics of the algorithm itself, for as you’ve guessed by know, it’s a bit funky, and I don’t want to forget all the subtleties, since it’s definitely still a work in progress (there are many improvements still possible).

Note: Deprecated!

After using the flow editor a bit, some of the more annoying errors became pretty obvious, so it’s time for a new update (well, full install again) to the flow editor. Besides the more obvious bug fixes, I’ve also introduced some new functionality, notably:

• shortcuts ‘N’ & ‘ctrl+N’: add/insert a new neuron.
• shortcut ‘R’: toggle ‘selection Required’ for current option/loop.  This is an important feature that I missed (in gene this was no problem since you could use an empty condition which is not possible here).  The problem is this: sometimes,
  an option or loop has to require at least 1 selection between a part (so you can’t skip the condition or flow), sometimes this is not required.  To allow a distinction between the 2, an extra ‘attribute’ is attached to the conditional (the loop or option), much the same as how a selection between loop and option is done.
• I changed the layout from wrap-panel to horizontal stack-panel, which is more logical to work with.

Note: I have updated the table on the first post about the flow editor so it contains the new shortcuts.

I will probably also have to implement a new statement type, to allow for grouping very soon.  It’s not really needed for defining the flows, but I think it can be useful later on, during the parsing. For instance, the expression: (verb "ing") means that you are expecting a verb, with ing behind (standing, listening,…).  You don’t need the brackets to define it, but it makes more sense while parsing, if the grouping statement has an attribute attached to it that indicates how it needs to be interpreted.

I have also noticed a ‘logical’ error in the way that I implemented keyboard navigation: some keys have been reversed. So when you are on an item inside a part and need to get to that part, use the left arrow instead of the right.  Being able to select
a part is important to continue adding items at the end again when you have added a conditional like in the next image. It’s annoying, but fuck it, this is one for later on.

I’m also not yet very happy with the deletion functionality.  As it is at the moment, you can remove or delete.  Removing simply takes away the reference to a neuron in a list, deleting, will remove the neuron from the brain.  This is annoying, cause you usually want to remove statics and delete conditionals and parts.  If you want to remove an entire condition, you need to first delete all the parts separately.  A better scheme would be as that of the code editor, which will check for this type of situation.  Again, this is a minor thing, so I will add it to the feature list. At the moment, there are more important things to do though.

A bit more interesting, I have also began work on an English grammar definition.  Here’s a screenshot of a part:

It’s off course not a definition that covers the complete language, but I think it should be enough to build a natural language interface (update: Oh boy, was I wrong with this one ), which should be interesting. I’m certain I missed a bunch, I still need to define adverb handling (which will be treated in a pre parser, a bit like how comments are handled in more traditional parsers) and the scanner also needs commencing.  And expect some shuffling around and updates as I implement the parser for this definition.  At this stage, extra attributes will probably be added, to handle more semantically oriented parsing. I have no idea how long this is going to take me, but I expect some issues with the debugger, so I might be out for a week or 2 for the next update.

Note: Deprecated!

I’ve added support for flow editing, which should be the final ‘big’ editor required to create useful networks. From now on, only finishing functionality and bugfixes. It took a little longer than I had originally planned, mostly because of a struggle
with WPF’s keyboard navigational system, which is crap. I finally got it working though (it also supports drag and drop, copy-paste still needs implementing), hope you like it.  You can download the program from here. Unfortunately it’s still a full install, so it’s the whole 40 and some megs download.  I’ll try to find some time to create an update installer so that the download size can be minimized.

The editor is still a bit rough, but it should be useful. I’ve tried to make it more text oriented, so you can easily navigate/add/remove items from the keyboard.  In the background though it’s still listboxes so coding the view and data was easy and fast, just that creepy WPF navigation system, off course more importantly, it makes certain no illegal input can be provided.

Anyway, here’s a screenshot:

Basically, this Flow editor describes how a noun can be found in a stream: a noun should start either with an article (which is either ‘a’ or ‘the’) or a number, followed by 0 or more adjectives finalized with a single noun.  Ok, there are gaps here (it’s still a sketch to show the editor): what’s a number, adjective or noun and how to find them.  These things will be explained in the demo, but basically, you use converters (functions that can transform a neuron into another one, like the object ‘house’ into the neuron ‘noun’) or some other info that can be used in a search function which can be attached to the flow items.

To create one yourself, go to ‘Insert/Flow editor‘, use the toolbar button ‘Create new flow editor‘, use the toolbar button on the ‘Project‘ overview tool frame or use it’s context menu. Press ‘F’ to create a new flow followed by a ‘.’ to select a static or ‘['/'{' for an option/loop (press '|' to add new parts in a loop or option). Here's a complete list of available short cuts:

A final note perhaps on how to use these flows in a neural network.  The thing is, this is really up to you, the application doesn’t make any hard-coded use of them. Though, there will probably be a couple of default algorithms that can be reused. 
The basic idea is relatively simple: when the first neuron comes in, search all the clusters to which it belongs with the meaning ‘Flow’, ‘FlowItemConditional’ or ‘FlowItemConditionalPart’ and store the result list in a cluster.  When the second comes in, try to find all the clusters of the previous result set that allow the new neuron to follow the previous one, all clusters that don’t allow this are removed from the result set.  Various clean ups / lookups can be performed during 2 incoming points. When a ‘flow’ cluster is found with code attached to it, execute this. This off course, can be made as simple or as complex as you want. More on this in a later post. Fuel is depleted for today.