Andalib Blog

Better UI. Better World.


Talking to your computer

As a thought experiment I am pondering whether or not our daily interactions with our computers can be considered a form of communication, a language. We can consider buttons to be verbs, data and files to be objects, the cursor is the pronoun “I”, and attributes on files or other data are adjectives. That would seem to provide sufficient building blocks for constructing a language. You can take the analogy a little further and look at the grammar, ordering of the basic components of this language to provide meaning. Take the simple example of making text bold. You select the text then press the “bold” button. You could think of it as this sentence, “I want to make bold text from word ‘Talking’ to word ‘computer’”. How does the computer respond to all of this communication? It can change the pixels on the screen, send data across the network, modify data on the hard drive, or print. I believe that provides more than sufficient means to respond to our communication. As with all thought experiments, the important question is whether or not we can use this method of thinking to further our understanding of the underlying topic.

Why do people think that Word is complicated? It is probably because there are a lot of buttons and options. If we use this computer language concept, then we could rephrase this statement to Word has too many verbs. But now the statement starts to feel absurd, does English have too many verbs. Clearly Word has fewer verbs than English, so the problem is not the number but the presentation of the verbs. You can think of communicating with Word like trying to talk to someone speaking another language and using a dictionary to find each every word used in your communication. If we want to overcome this problem we must remove or improve the “dictionary”. Many people have tried to enable actually speaking to computers, with very little success, so I am going to ponder the improvement of the “dictionary”. Let us take all of the verbs, objects, and adjectives of our “language” and put them in big database. For each verb and adjective define the criteria that allows it be used. Can I use the “bold” verb on a table? Now we can build a new version of Word where we can record which verbs are used most often, which ones are likely to be used during the same session as the other verbs, which verbs are likely to be used by which types of people, which verbs are likely to be used with which types of objects. With this data we could re-arrange the UI to most optimally present verbs to the user. We can show them in context menus, menus, toolbars, ribbons, sidebars, etc. We can also develop different versions of Word for different types of users. We could have a doctor version, student version, teacher version etc. This is similar to professional terms, or words specific to certain professions.

In conclusion, there is no conclusion just a bunch of ponderings. Please feel free to add comments explaining the futility of this thought experiment.

2 Responses to “Talking to your computer”

  1. shaun Says:

    So, say it is a language that we speak with our hands and hear
    with our eyes.

    what, then, about people with different ways of learning? i’m a
    strongly kinesthetic learner (I learn by physically doing things), so
    this language is already perfect for me. so long as i can try things
    out, i’ll eventually learn them.

    but what about visual and aural learners? there must be
    comparative studies of what teaching techniques work best for
    each group. maybe something awful like different sounds or
    colors associated with menus and functions would help people
    remember how to do different things (”where did that darn
    barking menu go? i need to make it growl.” this to subscript some
    text).

    of course, most devs today are almost certainly kinesthetic
    learners who taught themselves how to do things before
    instruction was offered, so they wouldn’t likely think there is
    anything wrong with the current approach.

  2. shaun Says:

    or how women generally give directions to a location using landmarks
    while men give directions based on a mental road map, women are more
    sensitive to visual texture while men notice shapes more distinctly

    so maybe women could more easily find their way through a cavernous
    menu system if it included more prominent and distinctive icons.

    the UW has a department that studies these sorts of human/machine
    interface questions http://www.hitl.washington.edu/home/.

Leave a Reply

You must be logged in to post a comment.

Contact Us | About Us | Digg Digg Us | del.icio.us Del.icio.us Us | Copyright ©