Discussion Question: Why have Visual Programming Languages never taken off?

Note: marked as community wiki.

Is there a good analysis of why visual programming languages still haven't taken off? We're still coding these days 'linearly' in a 80x25 text window; while the concepts we represent (data structures, algorithms) seem like they can be more intuitively represented vi开发者_StackOverflowsually.

Two approaches to programming that aren't just simple text come to mind:

• Structured Editing (which I know from Kirill Osenkov's blog).
• LabView.

I think Structured Editing is pretty interesting, because it takes the 'braces with idententation' convention, which has proven really useful for keeping code organized, to its logical extreme. I think it could really be something, if someone were to make a brilliant (from a usability perspective) implementation of it.

The LabView approach, on the other hand, doesn't excite me so much. The visual idioms don't seem powerful and obvious enough, compared to text. I haven't used LabView much though, so it's probably better than I think.

Dont forget with VS 2010 (.NET 4), its now multi monitor supported, which mean you can now allow editors, designers and tool-windows to be moved outside the top-level window and positioned anywhere you want across to any monitor on your system.

An 80x25 text window? Really? Not to compare sizes, but my text window is quite a bit bigger than that. But regardless of that, I personally can't imagine a visual programming language that would satisfy me. For technical information, text is far more information-dense than video. I would much rather skim an article about a technical subject than watch a video about that subject in five times as much time (seriously guys, knock it off with the videos already).

In a similar way, I would much rather spend a few seconds typing a few lines of code, than a few minutes dragging and dropping things around to accomplish the same thing. It's about conciseness, and expressiveness. Visual programming languages just don't have it, in my experience. Good for teaching fundamentals of programming? Sure. Alice is pretty neat. But not for day-to-day work.

On a somewhat-related note, Code Bubbles is an interesting take on improving the "80x25 text window".

There is quite some mixing and matching.

For instance, people do use GUI editors like NetBeans Matisse or VS.Net because some things are easier to draw than to code. Some people use GUI data model editors: it's much easier, faster and (I would argue) produces a better result than writing DDL. Even when you write code, you have all sorts of graphical tools to help you understand what you're doing (the eclipse hierarchy view, for example).

On the other hand, we do still use similar text editors to the ones people used 30 years ago for a lot of our work. :) It's obvious that there is value to be had from both.

simulink is part of matlab and works great for engineering problems

Visual programming languages have never taken off because no one has done it right yet. In the same way that C++ / visual studio were the right technology for people at the time of their advent.

However, the age of talking to our machines (Alex voice service), and programming with nicer tools than text editors is upon us.

Here's the start of one I'm working on. I'm trying to bootstrap my project since if you're making a cool programming tool, why wouldn't the tool itself eventually be written in the tool's input language. I did start out at first by rendering my own graphs with PyQt5 / QGraphicsScene but debugging a 2D scene is actually extremely hard - that is unless you have a visual graph to program with instead! So rendering my own graph and writing the graph editor comes after I can get basic graphs to run. My favorite general purpose graph editor is yEd. It outputs .graphml which is good because the networkx library for python can already read in .graphml (Only problem is loading in the graph colors + other properties than position; so feature will wait til we are doing our own graph drawings).

Here is an example input graph:

Here's some basic code to run it:

import networkx as nx
from PyQt5.QtCore import QThread, QObject, pyqtSignal
from PyQt5.QtGui import QColor
from PyQt5.QtWidgets import QApplication, QMainWindow, QPushButton
import re
import sys

EDAT = 2
NDAT = 1

class CodeGraphThread(QThread):
    ifRgx = r'^if\s+(.+)\s*$'
    elseRgx = r'\s+|^$'

def __init__(self, graph, parent=None):
    super(CodeGraphThread, self).__init__(parent)
    self._nodes = {}
    self.setGraph(graph)
    self._retVal = None
    self._locals = []

def setGraph(self, graph):
    self._graph = graph
    G = graph.G()
    nodes = [x for x in G.nodes(data=True) if x[NDAT]['label'] == 'start']
    if nodes:   self._setStart(nodes[0][0])

def _setStart(self, nstr):
    self._nodes['start'] = nstr

def start(self):
    self._running = True
    self._nodes['current'] = self._nodes['start']
    QThread.start(self)

def _exec(self, codeText):
    try:
        exec('self._retVal=' + codeText)
    except:
        try: 
            exec(codeText)
        except:
            self.codeGraph().errorMessage.emit('Coudln\'t execute code: "' + codeText + '"')

def returnVal(self):
    return self._retVal

def run(self):
    while self._running:
        cg = self.codeGraph()
        G = cg.G()
        current = self._nodes['current']
        #TODO transfer over to regex system
        data = [d for x,d in G.nodes(data=True) if x == current and 'label' in d and d['label'] not in ['start']]
        if data:  
            codeText = data[0]['label']
            self._exec(codeText)
        rgx = self.ifRgx
        edges = cg.edgesFr(current, rgx)
        if edges:
            e= edges[0]
            ifArg = cg.matches(rgx).group(1)
            self._exec(ifArg)
            if self.returnVal():
                self._nodes['current'] = e[1]
                continue
        rgx = self.elseRgx
        edges = cg.edgesFr(current, rgx)
        edges += cg.edgesFr(current, None)
        if edges:
            e = edges[0]
            self._nodes['current'] = e[1]
            continue
        break

def codeGraph(self):
    return self._graph


class CodeGraph(QObject):
    errorMessage = pyqtSignal(str)
    statusMessage = pyqtSignal(str)
    _rgxMemo = {}

def __init__(self, gmlpath=None):
    QObject.__init__(self)
    if gmlpath != None:
        self.loadGraphML(gmlpath)
    else:
        self._gmlpath = None
        self._G = nx.MultiDiGraph()
    self._thread = CodeGraphThread(self) 

def G(self):
    return self._G

def loadGraphML(self, gmlpath):
    self._gmlpath = gmlpath
    self._G = nx.read_graphml(gmlpath)

def saveGraphML(self, gmlpath):
    self._gmlpath = gmlpath
    nx.write_graphml(self._G, gmlpath)

def debugPrintNodes(self):
    print(self._G.nodes(data=True))

def debugPrintEdges(self):
    print(self._G.edges(data=True))

def matches(self, rgx):
    if rgx in self._rgxMemo:
        return self._rgxMemo[rgx][1]
    return None

def rgx(self, rgx):
    if rgx not in self._rgxMemo:
        self._rgxMemo[rgx] = [re.compile(rgx), None]
    return self._rgxMemo[rgx][0]

def rgxMatch(self, rgx, string):
    if rgx not in self._rgxMemo:
        rgx_ = self.rgx(rgx)
    else:
        rgx_ = self._rgxMemo[rgx][0]
    match = self._rgxMemo[rgx][1] = rgx_.match(string)
    return match       

def edgesFr(self, n0, rgx):     
    if rgx != None:  
        return [(u,v,d) for u,v,d in self.G().edges(data=True) if u == n0 and 'label' in d and self.rgxMatch(rgx, d['label'])]
    else:
        return [(u,v,d) for u,v,d in self.G().edges(data=True) if u == n0 and 'label' not in d]

if __name__ == '__main__':
cg = CodeGraph('unnamed0.graphml')
cgthread = CodeGraphThread(cg)
def printError(errorMsg):
    print(errorMsg)
cg.errorMessage.connect(printError)    
# Qt application reqd for QThread testing
app = QApplication(sys.argv)
win = QMainWindow()
win.setWindowTitle('PyGraphML Practice 0')
button0 = QPushButton('Start thread running')
button0.clicked.connect(cgthread.start)
win.setCentralWidget(button0)
win.show()
sys.exit(app.exec_())

Current issues: Python 3 doesn't handle exec / locals() well (hence use of self.x instead of just x), so thinking of using a 3rd-party python interpreter or just statically modifying the code.

Not saying that my tool does anything right yet. For things to be done right, there also must be auto-refactoring tools, debugging, etc.