Note: Deprecated!

The time for a new release has finally arrived. Lots of new things:

The designer and core:

• lots of speed improvements and deadlock fixes.
• Added a new asset editor. With this editor, you can declare things like ‘An eye has a color and the color of an eye is usually blue green or brown’. This is done with a graphical editor, instead of entering text, so you can directly manipulate the network.  Of course, this can also be used to view recorded ‘asset data’.
• Added a new dialog box which allows you to merge 2 neurons into 1. This can be very useful in some situations.
• Extended the ‘frames’ editor, so that it can also edit semantic frames (which are new in this version of Aici).
• Added a ‘bool filter’ to the frames editor
• As I’ve previously mentioned, there’s also a new ‘test case’ debugging tools, which is great for automated testing and validation (using regular expressions) of large input volumes. Running big tests can be useful to pick up unexpected behavior that got introduced because of newly added frames.
• I’ve also added support for ‘code search’. You can now search for all occurrences of any neuron within a code cluster in the entire network. To search for a neuron, select it and press ‘Ctrl+F3′.
• There’s a new version available of the ‘echo words’ demo, which uses less code (3 statements).
• New instructions: GetCommonParents, GetCommonParentsFiltered, GetCommonParentsWithMeaning
• Lots of bug fixes + introduced some new ones

Aici:

• A brand new semantics stage (again), which supports complex sentences and provides more possibilities.
• Added a bunch of semantic frames for things like ‘place’, ‘time’, some verbs, nouns and actions.
• Added a bunch of new grammar frames, mostly for complex sentences, but also for things like ‘where’, ‘who’,…
• Added new actions, mostly for complex sentences, but also for better support of verbs.
• cleaned up/improved the ‘be’ and ‘have’ action handlers a bit.
• lots of new words
• … (I sort of forgot to note everything I did near the end) For a good overview of what’s currently possible, check the conversations category.

And naturally, he latest version can be retrieved from the downloads page.

Note: Deprecated!

Debugging time!!! (when isn’t it.) I sort of forgot about this one, until a couple of days ago, when I ran into a similar problem as for which I originally created this (ahem) simple little trick.  To explain you what attached neurons are for, perhaps first a little explanation on how I have been designing my neural algorithms. (Caution: this is advanced stuff, not for the faint-hearted).

As with all algorithms, I take some input and process it to generate some output. The input with neural algorithms, is always expressed using neurons. The process to execute is defined through the meaning of a link (which also maps to a neuron). The link starts from the input neuron and usually points to some value that needs to be compared against or searched for,…, but this can also be something else.

The problem is the ‘split’ instruction. This is a very powerful thing indeed, which allows me to write simple, massively parallel code. It does, exactly as the word says, it ‘splits’ the current process into x nr of similar processes. They are similar, but not exactly the same: ‘the input neurons’ have been duplicated so that they contain the same values, but using new neurons, so that the process will eventually lead to a different result neuron. Also, the ‘variable’ that was supplied with the split instruction, has a unique value in each process (the value items are declared in the split instruction, they  determine the nr of new processes in the split). And finally, globals can be declared  as having to ‘duplicate’ their content when a split occurs. All this means that each process is similar (most variables point to the same neurons), but not exactly the same. (Mount Everest stuff, I know.)
The key thing to keep in mind here are the duplicated input neurons: when a split occurs, the idea is that each process will work on it’s own input and result data. But, sometimes, for various reasons, one or more of these neurons can spill over to other processes. And that’s when ‘contamination’ can occur, or when one process modifies the input/result of an other process, which is very bad indeed and excruciatingly difficult to trace using traditional debugging methods like breakpoints.

Thus come attached variables to the rescue. The idea is very simple: Whenever a ‘variable’ that has been signaled in the designer as ‘attach required’, gets it’s first value assigned, it attaches this ‘value’ to the  process in which this assignment occurred. If, from that point on, another  process tries to change  this value-neuron, all alarms go off as if it were WOIII. And that’s basically it. From there on, it’s up to you to use the presented information (where and who), to figure out the why.

For practical use, I have found that the ‘CurrentFrom’ system variable, really is the most useful variable for this trick, though it is possible to put it on any other. You can even attach neurons manually to a process, if you’ve paused and selected one. This is done on a code editor, using the context menu item ‘attach to  processor’. To activate the monitoring of a variable, make certain that it is added to the left most pane in the debugger view and select checkbox  in the ‘attach’ column before you provide the network with some input.

Like I’ve already mentioned, it doesn’t take much to use and activate it. This little trick’s usage field isn’t very broad, it only traces down 1 specific type of bug, but one that is very hard to find otherwise. Also, if not used correctly, you can get a lot of false errors, since it’s very common for multiple processes to modify the same neurons. So use with caution!

Note: Deprecated!

It’s time again to do a new release, which was long overdue to tell you the truth. Lots of things have been changed, though mostly in the background, both for the designer and Aici. Here’s a non exhaustive list of the changes:

The designer + core:

Basically, I began the process of improving the feel of the application, to get the prototype taste out of it. It’s not done yet, but things are improving considerably. I am now able to work with the designer for a full day, while keeping memory reasonably under control and with the same speed at the end as at the start. here’s what else got changed:

• Lots of speed improvements (more to go)
• fixed some memory leaks, there are still some more left to plug.
• New instructions: IsClusteredByAny, ChangeChild, ChangeInfo, ChangeParent, Avg (average), Max, Min, StDev
• Added system events OnStarted, OnShutDown, OnSinActivity.
• Added a new system variable: Time (to get the current system time).
• Added back/forward commands to the flow editor.
• Added the ability to manage 1 to 1 thesaurus relationships (from within the thesaurus tool window).
• Added ‘go to unfreeze location’ functionality in the memory profiler.
• Created a custom treeview, currently only used by the thesaurus. There are still a few issues left with the new view, but it fixes some unresolvable issues found in the default (wpf) treeview, so all trees will be replace with this new one in the next releases.
• lots of bug fixes

Aici

Aici has undergone a major upgrade, though mostly in the background, that is to say, to improve already existing functionality or extend it. Here’s what’s new/updated:

• There’s a new stage, just after scanning and before parsing, used to retrieve the different sentences and groupings from the input. This allows the system to process multiple sentences in the input (at the same time). 1 small note: the final stage hasn’t been tested yet, so even though the sentences are recognized, only input with 1 sentence is processed correctly at the moment.
• I’ve changed the semantics processing from a simple transition/translation to a full stage, so the semantic information can be used to do more filtering.
• Yet another new stage has been added to do grammatical analysis (things like figuring out whether an ‘a’ is most likely used as a noun or article.
• The output stage has been adjusted so that the output isn’t directly sent out, but instead stored as neurons that can be used for further analysis while being sent out at a later stage. The analysis itself however isn’t done yet.
• Some new frames have been added, mostly to handle invalid grammar and present/past articles.
• The flow definition has been simplified a bit.
• Lots of bug fixes

Aici-Web

And finally, the newcomer: Aici-web which is a web-based interface of Aici (go figure!). It’s currently implemented in Silverlight (other web technologies should follow). The basic purpose of this website version is 2 folded: as a technology showcase and as a way for me to easily get conversation logs that I can use to improve the network. So go ahead and test the limits, see how fast you can break the thing (shouldn’t be to hard ). You can find it here. (Note: I plan to reset the backing database on a regular basis, to keep the thing fresh and with the latest Aici version.)

Note: Deprecated!

The designer is a strange beast: the project that is opened for editing, is also running in the background as a neural network. This means that you can use the input/output channels just like in a regular (neural network enabled) application, like the AICI client app. Grate for debugging and trying out stuff, but you don’t want to get your test data mixed into the actual network, enter sandboxes:

A sandbox is a complete copy of your project, running in a new, separate designer.

So, after you have opened the Aici project in the designer (installed at {Documents}\NND\Demos\AICI_1, note that you need to select the directory, not the file), you can run your project in a sandbox. Simply click on the button in the toolbar, or select ‘debug/sandbox’ from the menu. This will start a new designer with you sandbox project loaded. Once the sandbox is open, you can use the already declared text-channel to communicate with the network. if not yet opened, switch it on by selecting ‘view/communication channels/Text channel’ from the menu. The Echo words demo has more info on this type of channels (though the view has been updated a bit). here’s the short version:

The toolbar:
1: Send the input text to the network.
2: a combo for selecting the method to send text to the network.
3: copy the conversation log to the clipboard.
4: save the conversation log to file.
5: clean the conversation log.
6: toggle audio on/off.
The edit area:
1: All the neurons that were sent to the network as input.
2: All the neurons that were sent back from the network as output.
3: The conversation log.
4: The input box

In short, simply type some text in the textbox at the bottom, press enter and watch what happens.

When you are done testing, simply close the sandbox app and you are done (it wont ask to save any changes). If you would want to do some more testing later on with the same data, you can first save it to a different location (otherwise the sandbox data will be deleted when you start a new sandbox).

Note: Deprecated!

It’s been a while, a bit longer then planned actually. As usual, there was a little unexpected detour required. This time, it was called the ‘neuron profiler’. Don’t ask, in short: I couldn’t find a bug, which would be traceable with a profiler thingy. The good news: found the bug, the bad: found some new. Anyway, here’s what’s added, updated or fixed (sort of):

The Designer:

• A new neuron profiler, very similar to a memory profiler: it tracks down leaked neurons (which were created but not destroyed). This is a powerful little debug tool indeed.
• the new flow editor is working. This is a lot faster now. Some bugs should also be fixed (maybe some new were added as well).
• Search on all editors and tool windows has been made into an asynchronous process.
• I’ve added a new clean up dialog box (can be found under ‘tools’), called ‘clean orphan flow data’. Which can be used to trace down and remove flow data that should have been deleted, but wasn’t and is now simply causing the parse to fail.
• cleaned up the module import and export a bit, but not yet completely ready.
• The Window manager has been updated.
• The UI has been cleaned up a bit. I’ve tried to make it esthetically a little more appealing, hope you like it.
• Some speed improvements in the core, more to come though.
• I’ve reloaded the thesaurus demo, for 2 reasons: 1, the first demo wasn’t loaded correctly (had forgotten to turn on to include items with multiple words) and secondly, I mixed up the meaning of hyponym and hypernym in the thesaurus view (not in the Wordnet-channel view). A typical mix-up for me, a consequence of flying solo, I guess. Anyway, this project should be very useful for stress testing.

AICI:

• You can now call .net functions from within the network. As a test-case, I’ve implemented the ‘file copy’ function (System.IO.File.Copy). At the moment, Aici requires some code for each known  .net function to extract the arguments from the network’s input data. I plan to change this into something more generic, so that Aici knows that a function requires arguments ‘x’ and ‘y’ without having to lay out custom code for each function.
• There’s a lot less neuron leakage going on. The flow recognition algorithm is almost completely clean (except for 1 remaining issue). The latter part of the process still needs some work.
• I’ve cleaned up a lot of the code, removed bugs all over the place so that it becomes more responsive.
• The flow recognition algorithm has also been updated to remove some more bugs that were causing the parse to fail.
• The flow definition itself has also been updated considerably. In part so that it would be able to parse complex sentences, but mostly to get the speed up (dramatic results can be achieved with a correct filter, early on in the parse).

All in all, I think I’m starting to see some light in the distance, finally. Perhaps it’s almost time to move to beta stage. )

Note: Deprecated!

The tail side is still red hot from the afterburners. At full throttle, 2 editors were completely rewritten, the drag drop system cleaned up and a rare deadlock removed. Both the mindmap and code editors now support all the mouse functions: horizontal/vertical scrolling and zooming. Oh, and I’ve also cleaned up the AICI code a little, though that needs more work.

Get the latest installation from the download section.

Note: Deprecated!

An annoying side effect of this type of resonating network, is, when you have a bug in the code, you generally don’t get 0 answers, but a whole lot. This can sometimes be very confusing to debug. Why does a certain path give a positive result, when it shouldn’t? And more importantly (from NND’s point of view at least), how can you follow the construction of that specific result, without the clutter of all the other processors? Enter ‘split paths’.

A split path represent the route that a result took, from start to now, expressed in the neurons that the path chose during all the splits.

As example, if only 1 split was executed to get to a result, and there were 2 neurons in that split, the path of the result only has 1 node and can have 2 possible values, those that were used in the split. In the real world though, a path usually has many, many nodes.

To record a split path, you first need to put a break point at a position where you know you will get many invalid results. That’s because a split path can only be recorded when the processors are still running, not when the result has already been calculated.

In the aici demo, it’s best to put a breakpoint in the ‘Stage 1.1’ code, in the if, second part (GetClusterMeaning(CurrentTo) == flow), if you drill down a bit further, you get the ‘Add split result’ instruction, which is basically the end point of the flow recognition  algorithm. (see picture below).

Put breakpoints on ‘Add split result’ instructions to find invalid split paths.

Once you have a processor that is producing an invalid result at your breakpoint, you can store the split path. Select the processor, open the context menu (right mouse button on a processor in the debugger overview) and select ‘Store split path’.  This will produce a new entry in the tree on the ‘Debugger’ page (see 3th image). The root item will be ‘path for x’ where x is the name of the processor. The children are all the neurons that were selected for the path during a split instruction.

I usually kill everything after I have the path, the other processors will produce invalid results anyway, and you need to start a new run before it’s  possible to actually use the path anyway.

So, finally we can start using the data to debug our network. The basic idea is to  use the path for finding the processors that are following the specified path, so we can take a closer look at the code. In short, a split path provide a shortcut: instead of manually stepping through all the code and individually deciding which processors to follow, we can jump to the ones we are interested in. These will be highlighted in green and will pause when they have reached a split that is selected in the path. Off course, we first need to select the path in the debugger, and at least 1 child node, otherwise there wont be anything to see.

Processors that follow a selected split path will show up in green and pause whenever they reach a split that produces a neuron which is selected in the path.

 One last note perhaps, once you have the processor you want, you can use the ‘Kill all but this’ command to stop all the other processors. This way there is less distraction in the debugger.

In case you are looking for a mental image to visualize this whole concept, try this: Resonance. When a link gets activated (for instance, the one to the very first neuron), it creates a resonance that triggers one or more other neurons. This excitation in turn can cause another link between 2 neurons to be activated, causing more resonance and so and and on until the whole thing settles down.

I have absolutely no prove or idea that this is how it works in the real world, that’s just how my model can be interpreted. Truth be told, this is not how I conceived the thing (aka lets try to create a model that uses resonance), it was rather more like: I have 2 neurons, they are linked, how can I make something happen? Well, I can attach some code to the link and execute that. Cool. But, wait a moment, that looks like resonance…

and we just passed ‘culminate’, pfff

Note: Deprecated!

The new release is finally ready.  The Aici demo took a bit longer than planned. Also, lots of things have been fixed and updated. Here’s a non exhaustive list:

• There’s a complete new lockmanager running in the background. This is much more secure (thread-wise, that is) and a lot faster. It’s still a bit of a diesel though, it takes some time for it to get going, but once running, it should be pretty fast. The slow start is due to the storage mechanism (all xml files currently). This is the major drag on the entire system at the moment, and will be fixed next.
• Wordnet import has been seriously updated, a lot more info is retrieved, and it’s now also possible to import the entire db in one go (although not yet advisable, due to a memory bug in the designer, it still takes a major byte out of the hard disk and it simply takes ridiculously long).
• The thesaurus has been given a make over to allow for editing and filtering. He can now also display / edit non recursive relationships. Drag drop is also supported.
• The frame editor has been updated considerably: drag drop support has been added and frame element filters/restrictions have also been introduced (will probably be extended in the future).
• I changed the function of the ‘contains’ operator a bit. It now only checks the contents of a variable. For clusters/children, there are the new instructions (IsClustesteredBy, LinkExists, ContainsChildren, GetInFiltered,…).
• I have added the ‘not contains’ operator.
• New instructions:
• Arithmetic group (+-/*%): I finally caved on those. My original plan was to see how far I got without using any arithmetic in the neural code. I guess, this is as far as I got with that.
• Get-at group: get child at, get cluster at, get out at, get in at, get info at.
• Distinct
• get Incoming, get outgoing, get info, Get in filtered, get out filtered, Get info filtered
• Is clustered by, Link exists, Contains children
• Remove-at group: Remove child at, remove info at, remove link in at, remove link out at
• many, many bug fixes, updates and little improvements.

Aici 1

This demo is a small chat interface. Though, in it’s current state, it’s not much more than a framework. It can initiate a conversation, close it, ask for the name of the user and store the data. It’s not yet able to fully recognize a recurring user since I haven’t defined the neural code for this yet.

I will be explaining how it works and how you can expand on this functionality, shortly. For those who can’t wait and want to get a peek under the hood, here are some pointers to get started:

• There are 3 main stages:
• Flow recognition, which is basically the syntactical stage: check the word types and order. Project pages are:
• the flows (aici/flows),
• the code that is attached to these flows (aici/code/flow code)
• the code that recognizes the flows in the input (aici/code/flow recognition)
• Frame recognition, or the semantics stage. This is where we try to find meaning in the words. Project pages are:
• the frames (aici/frames)
• the code that is attached to the frame sequences (aici/code/frame seq code). The frames don’t have code (yet).
• the code that recognizes the frames in the flow results (aici/code/frame recognition).
• action execution, or the response of the network to the input. Project pages are:
• Actions: all the different actions that the system knows (not yet a lot, should be extended).
• Output: some common code blocks for rendering output. This is also used by the frame sequences, since they are also used to render data in a predefined format.
• Action helpers: code that the action neurons can use to perform common tasks, like stopping a conversation or controlling the timers. Timer callbacks are also stored here.
• The transition between the different stages can be located in aici/code/transition.
• More specifically, the ‘Transition’ neuron is used to go from the flows to the frames and finally to the actions. 
• An action is started using the ‘Execute action’ neuron as meaning for a link from a data cluster to the action that needs to be started.
• The project also contains some mindmaps that describe the inner data structures and functionality.

Note: Deprecated!

Intro

Time for the second demo overview: the Scanner.  It builds on most of the ideas found in the first demo but it goes way further, and actually does something very useful (although you wouldn’t say it at first).  It’s probably going to be a lengthy piece so I’m thinking of cutting it in 2 or maybe even 3 parts. Anyway, lets first start it up, either through the start menu shortcut (in the Demo’s sub folder, conveniently called Scanner demo), or by opening it in NND (File/Open, select the ‘My documents/NND/Demos/Scanner’ folder). Once the project is loaded, you should see a single text communication channel open (called Text sin), if this is not the case, go to View/Communication channels/Text sin and make certain that it is is selected.

Overview

Lets get a taste of what it does, so enter some text (or leave the one that’s already there) and press the ‘send’ button (or enter).

You’ll notice that it basically does the same thing as the echo demo: the input text is echoed back, except that it’s a bit slower. If you had the debugger tab open, you probably also noticed a lot more activity, so something more must be going on.

And indeed, if you take a closer look to the text sin channel, in the upper section, you can see the neurons that were send to the network (on the left) and those that were returned (on the right), which are different. This was not the case with the echo demo, it simply sent all the incoming neurons back out, as they were. In this demo though, we get back something completely different: TextNeurons, that represent the same thing as the int neurons that were sent as input (if you regard them as ASCII characters). Hence the name of the demo, it’s a scanner.

Converting a stream of ASCII chars into words, integers, doubles and signs is a pretty useful feature and that’s all this demo is capable of doing, but that’s only because I stopped there. You see, in the background, is a general purpose algorithm that converts an input stream of neurons into a single result cluster, using any and all of the flows that are defined in the network. This means that you can use the same algorithm with many different flow definitions. I have simply defined some to recognize words, integers and doubles. You could go further and add flows to find verbs, sentence subjects,.. (in fact, that’s what the AICI 1 demo does). You could even go further still and create flows for visual objects or audio fragments, the same algorithm can be used. Unfortunately though, the editor doesn’t yet support such types of displays for flows (will probably be added somewhere in the future though).

Details

So how is the translation actually performed? To explain this, let me first recap some of the basic concepts of neurons and flows:

• The different types of  data available to a neuron are: incoming and outgoing links, possibly one or more parent clusters, for clusters possibly 1 or more children and a meaning. And finally value neurons also have their value of course. This is important, cause when the translation process starts, this is all the available information.
• Flows are nothing more than clusters that contain flow items, which can be statics or conditionals (loops and options). These in turn can only contain conditional parts. They represent a single branch of the decision tree. Parts can again have the same data as flows: statics or conditionals. So if you are a neuron (a static, part, conditional or flow), you can always look up into your list of parents to see in which flows and parts it is used.

Now, if you recall from the first demo, an input starts by creating an IntNeuron for each ASCII value, which is linked to another neuron using the ‘Letter’ neuron (ID 109). These are all put on the execution stack and the processor starts (the Rules code on the Letter neuron is executed).

So both neurons are new and only have each other as links. This means that we can’t use links or parent/child relationships to resolve the first step, but instead must use something different. The only other thing that remains is the value of the int neurons, so this is compared against some constants to see if they are digits (0..9) alpha numeric (a..z+A..Z), spaces | returns, or something else.  This comparison results in the creation of 1 new neuron per input neuron: the result cluster, in which we store the integer. One of these clusters (or it’s duplicate, due to a split) will eventually store the end result.  This cluster is linked to one of 3 static neurons: Digit, Alpha or Space (signs like . or , are handled a bit differently, this will be explained later). Naturally, if the integers would represent color values, we would use other starting points than digit or alpha. In other words, this first part is variable according to the type of input and the required accuracy of the algorithm. As meaning, we use  the start of the ‘flow recognition’ algorithm, called ‘Stage 1.1’ and put the result cluster back on the stack.  It’s important to put this one back on the stack, and not the item we are looking for. That’s because the algorithm can perform numerous splits and we want the result to be duplicated not the searchable, cause the contents of the list are continuously modified and we don’t want the result of one processor to be modified by  another one (I had to learn this the hard way).

After this initial step, the actual recognition algorithm kicks in. This consists out of 4 stages, grouped by 2. Meaning that  stage 1.2 is executed immediately after stage 1.1 for each neuron (this is the same for stage 2,1 and 2.2), but at the end of stage 1.2 and 2.2 all the result neurons are collected into a single global. Only after the last link of the last item on the stack has  been processed, are all the result clusters put back on the stack, with links for the next stage.  This is done for allowing to group items together.

The different stages are:

• Stage 1.1 (search parts/flows): Find conditional parts or flows in the list of parents of the searchable. If there are multiple results, perform a split for each, after the result list has been filtered (these are the shortcuts). If there are no results and this is the only and last item still on the stack, the end result has been found.
• Stage 1.2 (sequence-combine and filter): Check if items are sequential (2 flow items declared after each other in the same parent list, which is a part or flow) and handle floating flows, which are allowed to appear anywhere in the input stream, but which break up the sequence of other items.  Different actions can be performed if the order of the items is not ok: try to solve further or exit without result. Results of sequential items are grouped together.
• Stage 2.1 (search conditionals): Find conditionals in the list of parents of the searchable if this is a conditional part, otherwise the stage is simply skipped. If there are multiple results, perform a split for each after the result list has been filtered (not yet completely implemented at this stage). If there are no results, there is an error in the flow definition.
• Stage 2.2 (process loops and sync-points): If there was a conditional found in the previous stage, check if this is a loop. If so, and the previous item is of the same loop, combine the results. Also start a sync-point (will be explained later) if this was defined on the conditional.

These 4 stages are repeated until there is only 1 result cluster on the stack that represents the end result of a flow which is no longer used in any other flows. This cluster is made the result of the  split for the processor it ran on.  Off course, because there were possibly many splits, there could be many results. These will all be presented in the split-callback cluster (you need to provide a code cluster to the Split instruction, which will be called when all sub processors are done). In this demo, the result is sent back to the sin that caused the input, in a normal situation thought, this will simply start another process, as is done in the AICI 1 demo.

During this whole process, the algorithm is capable of executing callback code (attached to the statics, conditionals, parts and flows) at certain specific moments in the code. This is where the magic happens. The following types of callbacks are possible (together with  their execution time):

• Flow code: this code cluster is executed when a flow has been recognized in the stream. The ‘Result’ variable (ID 1822) contains all the neurons that match the flow. It’s here for instance that the int neurons are converted to a single word using the CiToS (Cluster with ints to string) instruction.
• Filter flow code: this code cluster is called from stage 1.1  (or 2.1, but this is not yet completely implemented) just after all the next items were retrieved from the list of parents of the searchable (stored in the CurrentTo variable). It allows the flow item to determine if it is a valid result, given the current state of the network. This is done by checking the contents of a number of globals, like ‘Prev stage item’ (ID 1956), which contains the previously processed neuron.

In the next post, I’ll go deeper into the specifics of the algorithm itself, for as you’ve guessed by know, it’s a bit funky, and I don’t want to forget all the subtleties, since it’s definitely still a work in progress (there are many improvements still possible).

The engine appears to be stabling out, although it is still acting fishy on single core machines, where there are errors I don’t have on my multi core dev system, so I need to move to a different machine to test this out. The designer is also still very much lingering behind the engine when this is processing at full speed, but the UI should remain responsive now.

I have also included an extra demo project called ‘Scanner’. It is able to transform an input stream containing integers, representing characters, into words and numbers (ints and doubles).  This doesn’t seem much, and it isn’t, except that it is doing this using a couple of flows and a general purpose algorithm (processing is still slow, mostly because the UI is trying to catch up).  This was the guts of the older ‘English language definition’ demo, which I have split into 2: the scanner and the language definition, which is no longer able to do any processing (all code removed). It’s an example of a more complex flow.

The scanner demo also has the number scanning problem fixed (numbers longer than 2 came out with multiple results). This appeared to be caused by deleting a couple of neurons to many (in the scanning algorithm).  I had already experienced the dangers of deleting neurons I thought were no longer used, but which were because of the splits. I will probably have to implement some sort of a garbage collection system to clean up unused neurons (but that’s for later).

Anyway, here’s the latest download (best to remove previous installation before installing this one).

Note: Deprecated!

This is the very first debugger I have written, and I am pretty proud about it! It’s not a masterpiece, but functional.  The code definitely could use some tidying up and some speed tuning wouldn’t hurt at all, but you can trace bugs, inspect values and follow the program flow, and that’s already something I guess. To explain the debugger to you, I thought it perhaps best to do it using some of the demo’s.  Simply open the ‘Echo words’ project to get started.

Set up

Before we send some input to the network, we need to set up the designer so that it is ready to debug:

1. We need to  put a breakpoint  on a statement. You can do this in a code editor. The very first code block that gets called in this network is the rules code on the ‘Contains Word’ neuron, so simply double click on the ‘Code: Contains Word’ node in the Projects tab.
   To set the breakpoint on the first statement, expand the ‘CheckStartTextBlock – space insertion’ code block and click in the little circle on the ‘if’ statement. This should make it turn red.
2. The designer also needs to be set in design mode.  You can do this by the drop down box on the main toolbar or on the debugger tab’s toolbar. There are 3 possible modes:
   • Off: no debugging is possible.  This mode runs faster.
   • Normal: The debugger stops whenever a breakpoint is encountered. When a breakpoint is encountered or a processor was paused, it is possible to inspect values.  This is how most other debuggers provide debug capabilities.
   • Slow motion: In this mode, the debugger automatically pauses and continues on every statement, creating a movie of the execution process. You can use the throttle bar on the left of the combo box to select the speed, which is updated real time. It’s possible to pause and continue this movie using the pause and play buttons on the debugger tab’s menu.  I have included this type of debugging mode because I believe it is sometimes useful to see the path that is taken to a specific point of interest. With this feature, you don’t have to keep pressing F6 to advance.
3. It’s also possible, but not required, to let a processor stop when an error was encountered. This is done by activating the ‘Break on error’ button. This only works when the error happened on a processor in debug mode.

Note that it is not possible to switch between different modes while a processor is running. You can only specify the debug mode for newly created processors.

An overview of all the breakpoints in the project can be found on the right side of the debugger tab. Currently it simply lists all the breakpoints, but new features should be added to this list shortly.

Start a processor

To get started, we need to send  some data to the network through it’s text-sin (sensory interface), so make certain that the ‘Echo channel’ is opened (a communication  channel is the visual interface for a sin). Go to View/Communication Channels/Echo channel, and make certain it is checked (and that the tab is selected).

Once the channel is open, type some text in the input section and send it to the network by pressing enter (or the send button). You should see a neuron appear in the left upper screen, which represents the input event (the neuron that will be solved by the processor). Your text will also appear as a string in the centre dialog screen. There will also appear an object in the centre screen of the debugger tab. This represents the processor that was started and which is handling the input event.

The processor overview contains 2 numbers. The first number represents the name of the processor. This value can be changed and is used to identify it between multiple processors. The second number represents the number of neurons that are left on the stack + the current neuron that  is being solved.

The buttons represent, from left to right:

• a toggle button that can be used to open and close the detailed view for the processor. 
• An indicator that is selected when the processor is paused. This is used to inform you on the run state, it’s not an interactive button.
• An indicator that lets you know that the processor is still running or not. When this is no longer selected, an irrecoverable error occurred in the processor and it is actually dead (in other words, you’ve got a problem when this is no longer selected).

Once a processor is started, the middle square in the right side group on the status bar will be blue.  This will remain so for as long as the network has still a processor running.

Detailed view

If you press the first toggle button, a detailed view of the processor should open in a new tab (using the same name as that of the processor).

This tab is divided into 3 sections (from left to right):

• The content of the execution stack.  This stack contains the neurons that have to be solved by the processor.  You can add and remove neurons from/to this stack using the Pop and Push instructions (The first neuron is automatically added). In our demo, there is only 1 item currently on the stack, that’s the one being solved. This item displays  a square in the stack UI element.
• All the neurons that were assigned as meaning to the links starting from the solvable neuron. The top one will be executed first.  A special case is the ‘Actions’ neuron, which will only be solved when all the others have been, no matter where it is found in the list.
• A stack containing all the processor frames.  A frame represents a single cluster’s code that is being executed by the processor. Since some statements call other clusters, like conditional statements, code blocks,… a processor will usually have a number of frames active. This provides an exact view of the processor’s current execution location.
  The following info is provided for each frame, from top to bottom
  • The code, in textual form. Each code unit can easily be selected separately for debugging purposes (check out the context menu on each item to see what you can do with it).  It’s not possible to edit in this window though (you have to open the editor for this).
  • The name (or id when no name is defined) of the neuron that defines all the code.
  • The relationship between the neuron that defines the code and the actual cluster containing the code.  This can be:
    • Children: in this case, the neuron that defines the code, is the cluster that contains the code.
    • Rules: the neuron that defines the code has a link with meaning ‘Actions’ to a code cluster.
    • Actions: the neuron that defines the code has a link with meaning ‘Actions’ to a code cluster.

The detailed view is great to have an overview of where we are in the processing stage, but it isn’t very useful to  debug a program flow. It’s better to use the code editor for this because it provides a better overview of the connection between the statements.  To do this, open the editor containing the code and make certain the the processor in the debugger tab’s center screen is selected.  This last action is very important for the following reason: a detailed view is opened for a specific processor, so a single tab always links to a single processor.  Code however is not specific to a processor, it can be called by many processors. So it needs to know the processor context for displaying debug information.  It uses the processor that is selected in the  debugger overview for this purpose.  This has a nice  side effect: when you have multiple processors running, you can quickly switch between active processor and view where each one is in the code. Note that the code editor will put a red square around the statement that will be executed next, to indicate execution location.

Do the debug

Code flow

We are ready now to start debugging the network. The first thing you can do is walk through the code using the following commands:

• Run (F5): continue execution until the next breakpoint or until the processor is finished (only active when paused).
• Next step (F6): execute the next step only and  pause again (only active when paused). Note, when  you do a step next on a split instruction that creates multiple sub processors, all sub processors will also wait on the next statement, they wont run wild, but wait until you tell them what to do, which is very useful for inspecting stuff.
• Pause: stop execution and wait until the Run or Next step command have been given (only active when running).
• Stop: Ok, this sounds silly, I know, but I haven’t yet implemented the stop.  I’ve always been able to stop in other ways (I’m running the debugger inside another debugger which is great for creating a schizoid coding mind). If you feel an urgent need for this command, let me know, otherwise, it’ll get in there,  I just have absolutely no time frame for this.

Inspecting values

When a processor is paused, it is possible to inspect the value of any item that returns a result, this includes: variables, globals, result statements, bool expressions and ByRefs. You can do this by selecting the item you want to inspect with the mouse, and pressing F7 or through it’s context menu (Inspect value). This opens a dialog with the debug info of the result.  This can be empty, 1 or multiple neurons.

Debug info for a neuron contains the following info:

• All the incoming links, where they point to and the meaning of the link (note that the meaning UI element still has to be changed into a debug neuron).
• All the outgoing links, where they point to and the meaning of  the link.
• All the clusters that contain the parent node.
• When the parent node is a cluster, all it’s children.
• When the parent is a cluster, the meaning of the cluster is depicted, in brackets, after it’s name.  This is also a debug UI element.

The debug info is depicted recursively for all neurons. This type of debug visualization is used in multiple places throughout the designer.  You might have noticed that the echo channel uses this UI element to depict the incoming and outgoing neurons. The detailed view of a processor also uses it to depict most of the neurons.

Watches

A final feature of the debugger is watches.  These allow you to observe the content of variables and globals in a list for a single processor or across all processors for those that are paused. 

This feature is best experienced using the ‘English language definition’ demo since this performs some splits while processing text input. I have put a breakpoint in the ‘Code: Stage 1.1’ page on the second if statement  (first child if in the left-side path of the only root if) and ran it until ‘Found == Sentence(flow)’ to get the screenshot on the left.

To add watches, drag a variable or global and drop it in the left part of the debugger tab.  This will add it at the bottom of the list. If you are dragging it from a code editor, it is best to hold the ctrl key pressed, so that the item stays at it’s original position.  Note that it’s currently not yet possible to drag from the toolbox.  This is a small bug that still needs fixing. Also note that it’s not yet  possible to remove variables (except by editing the designer file).  This command will be added very soon (just goes to show how fresh the debugger still is).

By default, a list of watches with their values for the selected processor is depicted.  The left side is the name of the variable (or it’s id if no name has yet been assigned). To the right of the name, the content of the variable is displayed. This can be empty, 1 or more neurons, all of which are depicted using the debug UI element for neurons.

If you switch to variable view, the left section of the debugger tab will only display a radio button for each watch. The middle section, which contains processor info, will now display the contents of the selected watch for each processor. 

Also note that, when a processor gets split up into multiple sub procs, the view will switch to a tree.  The root node shows the number of processors contained by the node. If a new input event is sent to the network before the previous has been processed, it is added as a list item, at end of the list. If sub processors split up, more sub nodes are created as children of the nodes that triggered the split (tree structure). This allows you to see how processors are related to each other.  At the moment you can only see the current state, through the tree/list structure.  In the future, an extra view might be added that shows a line view over time to show when and how may processors did a split and when they died out, although that’s just an idea at the moment, so don’t put your hopes up to see it any time soon, there are still far more important things to do.

Errors and warnings

If there are any errors generated by the code, either through the Error or Warning instruction or because of an error in the code, you can view exactly where it occurred. All messages are stored in the log tab. When they are blue, you can double click on them. This will open a code editor with the statement selected  that caused the log item (note: if it is somewhere in a sub section, this is not expanded automatically) Because you can use the same statement in multiple locations, only the first few will be selected, this is to take care of some problems with WPF’s standard controls (will be fixed in the future).

As you can see, it’s all still fresh,  but functional. The debugger has already become a central point of usage for me and I suspect this will only increase, so this will get some extensive testing early on. There are plenty of things that still need adding like conditional breakpoints, counts on breakpoints, enabling-disabling of breakpoints, more commands on the breakpoints-list (like clear, enable/disable all,…),… These things will probably be added as needed.

I am certain there is plenty more to say about NND’s debugger, I just can’t think of anything  anymore, so I guess I’ll leave it by this for today. It’s already turned out a long enough post.

Note: Deprecated!

Note: this is the second part of in a group of 3.

N2D is still in proto type stage, so there are plenty of idiosyncrasies to work around. Here are some tips and tricks that might make things a bit easier.

• When there is lots of code visible, things can slow down fast. (fixed)  To avoid this, use the drill down/up arrows on some statements like conditional statements to close as much as possible.  If you are drilling down and need to see more code, you can always open sub items into a new code editor.  This will speed up things dramatically. You can do this on conditionals, their parts and code blocks
• Be careful with copying items around (or using shift drag). The editor displays items that are used in multiple places, but you can still be surprised when you change something to an argument only to see it changed it multiple places. The same goes for breakpoints: if an item is used in multiple places and it has a breakpoint defined, it will break everywhere the statement is used.  Keep this in mind while using copy or shift-drag.
• I use the sync with explorer command (F4) a lot. This was mainly because the copy paste system wasn’t working yet.  But it’s still useful I think: it’s a quick route into the explorer.
• The Toolbox is really useful to build code.  If you select the ‘Instructions’ section, you get a good overview of all the available instructions, grouped by functionality.  To quickly collapse and expand the groups, you can use the context menu of the toolbar.
• It’s possible to add items to the toolbox, although this currently has to be done by hacking the “XXXdesigner.xml” file.

Final note

There are still a number of shortcomings in the editor that need to be worked out, more specifically:

• CPU and memory usage are a problem resulting in a very slow view when there is lot’s of visible data. This needs be solved by a custom control. (Done)
  
• Keyboard entry functionality, similar to the flow editor must still be implemented, this should speed up code editing significantly, compared to the all drag-drop or copy-paste solution.
• Clean up the views some so that they are clearer, compacter,…

Next up will be the debugger, I guess.

Note: Deprecated!

Today, I’d like to talk a bit about the code editor which is used to create and view executable data. Notice that I used the verb ‘designing‘ instead of ‘writing‘ code in the title. There’s a very simple reason for this: N2D doesn’t yet define a syntax for textual input of code, instead, there is the code designer which provides a visual view on the raw assembly code (so there is no complex conversion required). The idea is that it will eventually (I hope sooner rather than later) be able to understand natural language, making a custom syntax not necessary.  Because designing a good language, with accompanying parser can be tricky and time consuming, I opted to skip this step and instead rely on WPF for me to build a powerful designer with.  This has sort of worked.

Before you get started with this post, might I suggest you to check the Echo words demo explanation for a more general introduction on how to use the code editor. This provides a good introduction for most of the general concepts. This post will deal with some more technical details.  Because of the length, I decided to split it into 3 posts:

• editing techniques: that’s this post
• all the different code statements
• and: tips and tricks.

Editing techniques

Code editing is currently based on a drag drop paradigm in which you drag statements to the editor and drop them at the appropriate location.

• Drop locations are indicated using a gray, rounded border, containing the name of the drop target, like ‘Args’, ‘Children’,…
• To add statements at the bottom, simply drop them somewhere in the white space below or at the side.
• To insert items, drop them on the little black line above the statement.  The previous statement will be moved down.
• When you drag an item and drop it somewhere else, it will be moved, except if the ‘shift’ key is pressed, in which case the item will remain at it’s original position and will be copied to the drop target (no duplicate, but the same neuron is referenced. This is important!)
• You can also copy and paste items. During the copy process, the id’s of all the selected items are placed on the clipboard.  A paste will get the id’s from the clipboard, transform them into neurons which are put in the selected place.  You paste in drop targets, simply click in the drop target (there is no visual queue yet to indicate that the drop target is selected, this still needs some work) and paste. Note that copy-paste doesn’t yet work across multiple applications.
• If you press delete while an item is selected, it’s reference will be removed from the editor.  If the underlying neuron is no longer referenced anywhere else, it is removed.  All neurons that it references which are also no longer referenced are also removed (cascading delete).  This is the most used deletion method in code editing.  This means that when you delete a statement, all the parameter values that are no longer used, are also deleted. Or, if you delete a code block, all the statements that are no longer used anywhere else, are also deleted from the network.
• If you want more deletions options, use the ‘Delete special’ (ctrl+del)  command.  This shows a dialog which allows you to specify the deletion method you want:
  • simply remove the reference, but always leave the neuron in the network
  • Remove the reference and when  no longer used, delete it (as with the normal delete)
  • Always delete the neuron.Referenced neurons can be
  • left alone: they are not deleted
  • deleted when no longer referenced
  • always deleted
• All neurons can have a ‘display title‘. When the designer finds one for a neuron, this is displayed when possible.  You can easily change this name by pressing ‘F2′ or through the context menu of the selected item.  This is very useful for variables and globals.
• Most statements have a little circle in the front of the image.  This is a toggle button to enable/disable a breakpoint on the item. This is part of the debugger integration into the editor (more on that later).
• It’s also possible to change a statement from one type to another one through the context menu.  A warning though about this command, you might loose data with this command, that can’t yet be undone (no proper undo support yet for this command).
• If you simply need to move a statement up or down, you can also use the context menu, which has a submenu for moving items around.

Note: Deprecated!

Just a quick ‘in between’ note. I had to make a small change to the flow editor because a flow can now have an attribute ‘FlowIsFloating’ assigned (through a link pointing to ‘True’ or ‘False’). It is used by the scanner and parser routines to find flows that can appear anywhere in a stream.  This provides a generalized manner of defining this property without having to create custom routines for each set of flows. Spaces between words are good examples where this is used. The context menu for the flow name in the overview list on the editor contains a toggable menu item to activate/deactivate this attribute.  A green line in front of the flow indicates it is floating.  Here’s a screenshot:

Note: Deprecated!

After using the flow editor a bit, some of the more annoying errors became pretty obvious, so it’s time for a new update (well, full install again) to the flow editor. Besides the more obvious bug fixes, I’ve also introduced some new functionality, notably:

• shortcuts ‘N’ & ‘ctrl+N’: add/insert a new neuron.
• shortcut ‘R’: toggle ‘selection Required’ for current option/loop.  This is an important feature that I missed (in gene this was no problem since you could use an empty condition which is not possible here).  The problem is this: sometimes,
  an option or loop has to require at least 1 selection between a part (so you can’t skip the condition or flow), sometimes this is not required.  To allow a distinction between the 2, an extra ‘attribute’ is attached to the conditional (the loop or option), much the same as how a selection between loop and option is done.
• I changed the layout from wrap-panel to horizontal stack-panel, which is more logical to work with.

Note: I have updated the table on the first post about the flow editor so it contains the new shortcuts.

I will probably also have to implement a new statement type, to allow for grouping very soon.  It’s not really needed for defining the flows, but I think it can be useful later on, during the parsing. For instance, the expression: (verb "ing") means that you are expecting a verb, with ing behind (standing, listening,…).  You don’t need the brackets to define it, but it makes more sense while parsing, if the grouping statement has an attribute attached to it that indicates how it needs to be interpreted.

I have also noticed a ‘logical’ error in the way that I implemented keyboard navigation: some keys have been reversed. So when you are on an item inside a part and need to get to that part, use the left arrow instead of the right.  Being able to select
a part is important to continue adding items at the end again when you have added a conditional like in the next image. It’s annoying, but fuck it, this is one for later on.

I’m also not yet very happy with the deletion functionality.  As it is at the moment, you can remove or delete.  Removing simply takes away the reference to a neuron in a list, deleting, will remove the neuron from the brain.  This is annoying, cause you usually want to remove statics and delete conditionals and parts.  If you want to remove an entire condition, you need to first delete all the parts separately.  A better scheme would be as that of the code editor, which will check for this type of situation.  Again, this is a minor thing, so I will add it to the feature list. At the moment, there are more important things to do though.

A bit more interesting, I have also began work on an English grammar definition.  Here’s a screenshot of a part:

It’s off course not a definition that covers the complete language, but I think it should be enough to build a natural language interface (update: Oh boy, was I wrong with this one ), which should be interesting. I’m certain I missed a bunch, I still need to define adverb handling (which will be treated in a pre parser, a bit like how comments are handled in more traditional parsers) and the scanner also needs commencing.  And expect some shuffling around and updates as I implement the parser for this definition.  At this stage, extra attributes will probably be added, to handle more semantically oriented parsing. I have no idea how long this is going to take me, but I expect some issues with the debugger, so I might be out for a week or 2 for the next update.

Note: Deprecated!

I’ve added support for flow editing, which should be the final ‘big’ editor required to create useful networks. From now on, only finishing functionality and bugfixes. It took a little longer than I had originally planned, mostly because of a struggle
with WPF’s keyboard navigational system, which is crap. I finally got it working though (it also supports drag and drop, copy-paste still needs implementing), hope you like it.  You can download the program from here. Unfortunately it’s still a full install, so it’s the whole 40 and some megs download.  I’ll try to find some time to create an update installer so that the download size can be minimized.

The editor is still a bit rough, but it should be useful. I’ve tried to make it more text oriented, so you can easily navigate/add/remove items from the keyboard.  In the background though it’s still listboxes so coding the view and data was easy and fast, just that creepy WPF navigation system, off course more importantly, it makes certain no illegal input can be provided.

Anyway, here’s a screenshot:

Basically, this Flow editor describes how a noun can be found in a stream: a noun should start either with an article (which is either ‘a’ or ‘the’) or a number, followed by 0 or more adjectives finalized with a single noun.  Ok, there are gaps here (it’s still a sketch to show the editor): what’s a number, adjective or noun and how to find them.  These things will be explained in the demo, but basically, you use converters (functions that can transform a neuron into another one, like the object ‘house’ into the neuron ‘noun’) or some other info that can be used in a search function which can be attached to the flow items.

To create one yourself, go to ‘Insert/Flow editor‘, use the toolbar button ‘Create new flow editor‘, use the toolbar button on the ‘Project‘ overview tool frame or use it’s context menu. Press ‘F’ to create a new flow followed by a ‘.’ to select a static or ‘['/'{' for an option/loop (press '|' to add new parts in a loop or option). Here's a complete list of available short cuts:

A final note perhaps on how to use these flows in a neural network.  The thing is, this is really up to you, the application doesn’t make any hard-coded use of them. Though, there will probably be a couple of default algorithms that can be reused. 
The basic idea is relatively simple: when the first neuron comes in, search all the clusters to which it belongs with the meaning ‘Flow’, ‘FlowItemConditional’ or ‘FlowItemConditionalPart’ and store the result list in a cluster.  When the second comes in, try to find all the clusters of the previous result set that allow the new neuron to follow the previous one, all clusters that don’t allow this are removed from the result set.  Various clean ups / lookups can be performed during 2 incoming points. When a ‘flow’ cluster is found with code attached to it, execute this. This off course, can be made as simple or as complex as you want. More on this in a later post. Fuel is depleted for today.