• I had never made something like that, while I had plenty of experience designing languages,
• I didn’t really know what the end product would look like or how it would behave,
• and apart from some ‘UML designers’ that render classes, I had never used such a thing for editing code myself.

So then, what made me decide to use this approach instead of the more conventional and traditional way of using text?  First of, a major part of the answer lies in the previous statement . Other reasons were:

• Guided editors (can) lower the entrance level, when done properly (something I needed,… seriously).
• I new I would be needing some good debuggers (lessons learned!!). I could try and incorporate these into some existing editors, or turn out one of my own. At the time, it seemed a better idea to have full control from top to bottom, if I wanted to do this properly.
• I had a bit of experience working in WPF, which gave me some confidence that it was possible.
• Also, I came across this, rather serious attempt at structured editors (originally for C#), which raised my hopes as well.
• Did I already mention you that I was hoping this would make things a bit simpler for other people to understand?

In the end, it turns out I am glad I decided to use this approach, though not exactly for the reasons as previously stated. Except perhaps for the debugging, which really was fairly easy to implement this way. No, the thing that convinced me was the way that it changes how you program. This didn’t happen over night, I only began noticing this the last couple of months, after some 1,5 years of (trying to) program this way.
So what changed: well, over time, when I needed a ‘variable’, instead of thinking ‘variable’ I began to think ‘purple’. This is a shift from thinking in words to thinking in color. The best evidence for this, came in the form of me, continually trying to put a conditional in an assignment as if it were a global (cause I thought I had picked up a global). If you look at the picture to the right, you can probably understand why I erred: the blue colors are just to close to each other.
What’s more, over time I also began to think more and more in visual structures instead of ‘if-else’ or ‘while-do’. When I think about an ‘if’, I think:

In short, when programming, I am thinking more and more in terms of: blue square, red hexagon, white circle with line below (the conditional part),… Or in other words, it feels as if I am programming like building with Lego blocks!!!
I can’t explain exactly how or why this is, but it appears to free the mind for thinking about other stuff. At times, I find myself, just mindlessly putting blocks together to build up the picture that I had previously constructed in my head.  Very pleasing.

So, as a first try, I am very happy with the result, well enough for me to continue this path, and with some minor corrections here and there, keep the design. That is not to say there isn’t room for improvement, I have bucket-loads already of those, just not the time, so most of these will not be for the near future.  Others might be implemented sooner, depending on needs.

Basically, most of the concerns are centered round work speed (the number of movements and clicks required for an operation), maintainability and overview:

Speed

• To start with, the toolbox needs some redesign. I currently have 2 pages: one for general items, and a separate tab for all the instructions. I find myself continually switching between the 2, which is time consuming and unnecessary. The solution is simple: split them up in separate toolboxes.
• The toolbox currently only supports drag-drop operations. So to get 5 objects of the same type, you need to perform 5 drags. Better would be to have a ‘paint’ tool, like graphical apps, and many (UML) designers have.
• The code designer really could use some more keyboard support, so it can be used like a regular text editor (sort of). This seems a logical next step to do, just haven’t gotten there yet.
• I have been thinking a lot about using WPF’s new multi-touch libraries to add support for using 2 mice while coding. I don’t really make use of the left hand during the GUI coding, apart from the occasional copy/paste or editing a label. At times, I could definitely work with 2 hands at the same time. It’s a bit like using both hands for searching and placing blocks.
  I had also thought about using this new kinect toy, which appears to have been hacked already for PC usage. This would even resemble more like building with Lego (or like Tom cruise is doing in Minority report) . But then I figured: ok how would you feel after 2 weeks of keeping  your arms up in the air? Ok, I don’t know how I’d feel after 2 weeks like that, but my shoulders and neck definitely do!!!
  So that plan was canceled.

Overview

• As I’ve previously mentioned, I often simply recreate the picture that’s already in my head. The annoying thing is that the designer only shows a small part of the object (algorithm) that I’m working on, while at times I have more a need for a birds view approach. That’s why I added the zoom support, but more can be done such as:
  • make the code templates (the drawings) a bit smaller. I’m no good at designing, which shows.
  • provide a bread-crumb like ability (for drilling down and going back up again), much like the windows file explorer provides using it’s breadcrumb approach for directories.  I could use this for sub blocks.
  • See what I can do about providing support for multiple screens.
  • Buy a new big sized monitor, but given the massive resources at my disposal, that’s not an option
• Also previously stated: some of the colors can use a little tuning, the same perhaps for some shapes, to make them more distinguishable from each other.
• Currently, the drop area for the arguments is displayed as a simple, single square. The thing is: the argument requirements are known for each instruction, so it’s perfectly possible to display a custom set of boxes, with correct labels, for each instruction. This, by the way, would also free some space, since no drop area needs to be shown when all arguments are provided.
• I could also try to implement some sort of 3D-view on the network model, but I think I’ll need to extend my skill-set for that one.

Maintainability

The single major issue I have with code maintainability is concerning documentation. I added support for this by allowing to add some ‘rich-text’ string to each neuron. The idea being that you’d put all the comments concerning what’s going on, in there. Unfortunately, it doesn’t really work very well for code documentation because:

• neurons can be reused. This is different compared to regular text code: when this is compiled, each line of code will get it’s own output (or better, each compiled instruction corresponds with exactly 1 location within the text). With this neural network however, things are a bit different: it’s perfectly possible to use the same assignment in 2 (or more) different places. This is something I do extensively, by the way, to speed up coding and to keep the nr of links down.  What I noticed is this: though the code is shareable, the reason usually isn’t. I am currently solving this by stacking the different comments in the same text-blob, when needed. But better solutions are possible.
• I also find it annoying having to select an object in order to see it’s comments within the code view. It appears to be easier/better when you can quickly scan a document for all comments and read the one you need, without thinking about selection.

To solve this, I have been thinking about adding support for comments within the ‘project’ documents. These would maintain and display (as call-out balloons) all the comments for the neurons that they contain. The location of the comments would be determined by the comments themselves, so they can be floating, closest to the neuron that they are attached to. This would also allow me to display the comments, even if the actual neurons aren’t visible, but collapsed.

Final note

As a final note, for those who would like to build their own structured editor in WPF, perhaps a word of advice: stay away from ItemsControl or any of it’s descendants. The paradigm sounds grate: manage a list of items and display them any way you want through styling and templates. What’s not mentioned is that this only works for 10 objects or so. Any more, and your app slows down to a crawl or worse: crashes just as bad as the good old win32 apps would, perhaps even worse at times. Instead, go lower level: inherit from panel and roll out your own Measure/Arrange or go even further down and overwrite the rendering routines.

• Scroll & load speed: Even with full virtualization turned on, scrolling crawls almost to a halt when you have 60 000+ root items and a few of them are expanded some levels down the tree. The same for loading large lists.
• Memory usage: probably related with the previous item, in short: kaboem!!!
• BringIntoView: Here’s the killer, when virtualization is turned on, and the UI TreeViewIem hasn’t been rendered yet, you can’t bring it into view. Translated: you can’t do a search and show the found items in large trees, only in small trees (go figure).

Note about my test data:  The thesaurus demo contains about 100 000 words (syn-sets actually), almost all of which have multiple relationships, so  in total, there are +500 000 nodes in the tree.

Ok,… and how do you fix this? Well here’s how I did it:

Container rendering

Virtualization means that you only render the UI elements that are currently visible, and not the whole tree (or a large part like microsoft does). This means that, for each UI object that got rendered, you somehow need to keep track of the data position in the tree. I do this by using 2 cursors: one for the first and another for the last visible element. This cursor is nothing more than a list of indexes into the tree.

All items in between those 2 cursors are flattened out (by rendering UI elements for them). The nr of indexes (or nr of parents) for each item, is used as the level for the UI element. This determines the amount of indentation to the right that is applied to the UI element.

When the user scrolls down, the bottom cursor is updated (advanced by the amount that the user scrolled). This is a basic tree walking algorithm. From this bottom index, we render objects while going up the tree until the panel is full (again, the same tree walking algorithm). The index path of the new top item becomes the new top cursor. The same technique is used to scroll up, but instead, you start from the top cursor and fill downwards.

Remark: Watch out while calculating the scroll difference: The scrollbar value is usually a double while we only consume integer values, so there might be some left over in the scroll difference which has to be consumed the next time you calculate the dif.

Of course, this technique only works if the TreeView has some knowledge of the data structure, how else can it find the children of the root items and see if they are expanded or not? This requires an interface. Here’s how mine looks like:

   public interface ITreeViewItem
   {
      bool IsExpanded { get; set; }
      bool IsSelected { get; set; }
      bool HasChildren { get; }
      IList TreeItems { get; }
      ITreeViewPanelItem ParentTreeItem { get; }
      bool NeedsBringIntoView { get; set; }
   }

Each tree node should implement this interface so that the TreeView can properly traverse the tree. A few properties are mostly for the visual parts, like ‘IsSelected’ or ’HasChildren’ (to display the toggle button). The last property is perhaps a bit of an odd one: ‘NeedsBringIntoView’. It’s a quick method to let the Tree know that an element needs to be brought into the viewable area. You could also use a function on the TreeView (like microsoft does), but then you are forced to go find the UI element from your search code to bring the item into view, which I wanted to avoid.

Using this technique, you do create and destroy a lot of containers since each time the user scrolls, the entire visible area is re-rendered. To minimize the impact of this on the garbage collector, it’s probably best to use some recycling scheme. This doesn’t have to be fancy: a simple queue to put the items in when clearing the list which can be consumed again when rendering the containers. I chose a queue since it gives best chances to reassign the same container to the same data object when possible, which minimizes the impact of re-rendering on the binding system.

Events

The second important thing for virtualized TreeViews, is the event system. You can not simply go and hook up event handlers to every data element. This would blow up the memory. Also, you’d have to write a lot of code to keep track of those events as data objects are being created and destroyed (trees of such a size can only work if you combine UI virtualization with data virtualization, but that’s an entire subject all together). On the other hand, events are VERY important to get UI virtualization working for keeping track of the maximum scroll count, tree nodes that get expanded/collapsed, objects that need to be brought into view, and UI elements that need to be updated.

Are you seeing the bubbles already? Yep: use bubbling events, like WPF does. The only problem: that doesn’t exist for regular objects, it’s only available in the WPF world. You could inherit all your objects from DependencyObject, but with 200 0000 objects in memory, that somehow becomes bloated. Better to roll out your own sleek, mean and fast event mechanism, which isn’t hard at all to do. Here’s a little sample code to bubble events up a tree:

public static void OnPropertyChanged(IOnBubblingChanged sender, BubblingPropertyChangedEventArgs e)
{
   while (sender != null)                                  //use a while loop to avoid recursive calls.
   {
      sender.OnBubblingPropertyChanged(e);
      IOwnedObject iOwned = sender as IOwnedObject;
      if (iOwned != null)
         sender = iOwned.Owner as IOnBubblingChanged;
      else
         sender = null;                                    //need to make the loop stop
   }
}

This function is called whenever a property on a tree item is changed. As you can see, a lot of interfaces are at work here. IOnBubblingChanged defines a function that can be called which will raise the event. IOwnedObject is one of my core interfaces for all objects that can be ‘owned’ by another object. This maps to the ‘ParentTreeItem’ property of the first ITreeViewItem interface.

A similar thing needs to be done for the CollectionChanged events. A remark on the ‘reset collection’ event: make certain that you can pass along the list of items that got cleared, otherwise the TreeView will have a hard time figuring out exactly how much the count needs to be decreased (were any of the removed items expanded?). The default WPF NotifyCollectionChanged event doesn’t do that, so you can’t simply use it’s event arguments.

With this type of bubbling event system, it’s probably best to work with some kind of ‘root’ object that defines the events for the property and collection changes in the entire tree. Again a nice job for an interface, I suppose. This root interface can be used as the ItemsSource for the treeview instead of a list, like WPF does. This way, you can have lots of root objects without overtaxing the .net event management system.

Using this technique, you can make Treeviews that aren’t only fast to load and scroll, but also use very little memory. I am silently hoping that microsoft will eventually pick up on this and begin implementing proper UI virtualization in the coming versions of WPF (thus relieving me from the burden). In the mean time, I guess we are left to roll out our own stuff.