Thursday, 13 December 2007

Brain-Computer Interaction

We just had a visiting speaker by the name of Luciana Haill who showed us ways of measuring brainwaves using a special device which you attached to the forehead to pick up EEG waves. The software used with this device was able to produce visualisations and sounds from the data. Your brain is controlling the images and mix of sounds so the experience is very unlike those of conscious “forced” interaction loops where you need to perform an action in order to get a response or view a change.

Here’s a video of the software and hardware in action:



The fascination with this technology goes beyond just making beautiful sounds and patterns to perhaps being a step towards gaining a deeper understanding of the human brain. She has a myspace if anyone is interested - www.myspace.com/lucianabrain.

In the previous module, I looked into brain-computer interfaces and I posted a few YouTube videos I had found on the subject. Scroll down and take a look at people playing pong using the power of their minds!

Tuesday, 11 December 2007

Magic Eye

After looking at the Magic Eye Tetris game linked to from Sita’s blog (www.lutanho.net/play/magiceyetetris.html) I’ve been looking into Magic Eye images and thinking about creating some sort of interface using the Magic Eye images.

As mentioned in an earlier post on this project I am interested in creating a “wrong” interface which interesting and makes us think about how much we take intuitive interfaces for granted. I’ve also, as you can see in previous posts, been looking into optical illusions and Op art and researching into how and why we see and perceive things the way we do.

I have found a site which lets you create your own magic eye images too, try it for yourself here:

www.flash-gear.com/stereo

Here’s one I made earlier:



What is it?!...Answers on a postcard please (or in a comment).

Monday, 10 December 2007

Repetitive Art

Here are some artists who use optical illusion and repetition themes in their work:

Jim Lambie



Jim Lambie Installation view, Mental Oyster, Anton Kern Gallery New York, 2004


Mark Titchner



Mark Titchner Turner Prize installation, 2006
Photo from Turner Prize 2006 exhibition at Tate Britain. Photo: Sam Drake and Mark Heathcote


Bridget Riley

Repetition



After looking at optical illusions and Op Art which uses repetition to confuse the brain, I’ve been discussing with a few people reasons why we repeat things. Here is a list of reasons I’ve gathered together (if you think of anything else please comment on this post):

  • To learn/practice

  • To distract

  • To remember

  • Religiously

  • Routinely

  • To affirm

  • A stutter/ by accident

  • Humour (custard pie, catchphrases)

  • Because of mental-health problems – echophilia, autism



There is safety in repeating things. This is often used in web-design where templates for pages are often the same throughout the site.

If you repeat something too much does it lose its original meaning and become something different?

When people don’t get a reaction from something the first time they tend to became frustrated and repeat the question, or command, or physical action until they do get a reaction. This often happens with computers and is described in “The Design of Everyday Things” by Donald Norman.

Response to Essay by Francis Halsall –Observing Systems – Art from a Systems- Theoretical Perspective

One of the points we spoke about when we discussed this essay this morning, was how systems theories overlap each other and how we could all define ourselves, within this course, as systems artists.

An example which was given was that of the “system of a street corner” overlapping with the “system of a gallery” to create something else. This overlapping is evident in the art of Banksy bringing art outside from the galleries and therefore combining two systems.

We also discussed the work of Simon Morse. His work is concerned with systems as a way of viewing the world and showing how they encapsulate our view of things. He has also compared systems, such as the system of a gallery to the system of a company. Comparing systems may give us a better, clearer view of the world and a way to make sense of things around us.

A piece of his work, which sprung to my mind when he was mentioned during the discussion, was his custard pie fight piece “We Do What We Do Best Best “, created in 2005. To create this piece he observed systems of interaction within an office, which he had worked at, and also considered social systems of interaction to create his impression of a mass custard pie. This fight took place within an enclosed environment which would otherwise be a calm and ordered system.

He mentioned, when he came to speak to us, that he took inspiration for this piece from a Laurel and Hardy film called “Battle of the Century”, in which a mass custard pie fight takes place when one person accidentally hits another with a custard pie which acts as a catalyst causing a chain reaction which culminates in chaos.



On this course it is very important to study different systems to help us to create interesting and intelligent work and acknowledge the wider world around us.

A quote in the essay we were discussing was by Jack Burnham describing systems-aesthetic as:

‘An attempt to bring together artistic, technological and social conditions under the rubric of systems and a concern in them shared by a variety of groups including artists, scientists and social theorists.’

This describes the sharing and overlapping of information, which is what I am doing at the moment with my project. I’m looking at various other systems and thinking about ways to combine them within this design for interactivity brief. Some of the systems I have been observing for this brief include scientific and philosophical systems.

We also discussed the focus on the process and idea being more important within Systems Art. This was as opposed to Renaissance art, where the final creation itself was of utmost importance. This important point about systems art is mentioned in a quote at the start of the brief for this unit:

'…Design boundaries can be thought of as very elaborate systems of control'.
'The users of new media are becoming the content of the form'.

(P. Marshall, 2004, New Media Cultures, Arnold Press: London Pp.18)

The other quote, shown above, mentions how systems can be used to control the individual with a set of rules. Control is an important point involved with interactivity. How much control does the user want or need? Do people enjoy adhering to rules? Evidence that they do is clear from the popularity of video games, which can only be played by conforming to these rules...

Sunday, 2 December 2007

The Illusion of Interactivity

I thought I would post some of the ideas I’m currently looking into and try to get some reactions to them to help me to decide in which direction I should progress with my research.

The first idea I had was to investigate Brain Training games (such as the ones created for the Nintendo DS) which are said to “help” people with Alzheimer’s disease, dementia, and concentration problems and reduce your “Brain Age”. However I have found a lot of articles which condemn this new form of gaming and dismiss any health benefits.

I’ve also looked into robotics and physical computing, following on from the lecture given by Mike Blow last Monday. I love the idea and look of this interactive robot called Petit Mal made by an artist called Simon Penny. Take a look at the video below to see the interesting way it reacts with the audience:



Another area I’m interested in is optical illusions, and the way in which we perceive, create and interact with what we see. Sarah Strickett suggested I look into semiotics and especially try to get hold of a book by John Berger called “Ways of Seeing” which I haven’t yet been able to get hold of as it’s out at every library I’ve tried. Obviously a very popular book!

I’ve been reading the book “Eye and Brain- The Psychology of Seeing” by Richard L. Gregory, which helped me with our last project. It has a chapter on illusions in which I am finding interesting. Another book I’ve managed to get is “Vision and Art-The Biology of Seeing” by Margaret Livingstone which is similar to “Eye and Brain” but may offer a different perspective.



The Stroop effect, is an interesting phenomenon when a word such as blue, green, red, etc. is printed in a colour differing from the colour expressed by the word's meaning (e.g. the word "red" printed in blue ink), a delay occurs in the processing of the word's colour, leading to slower test reaction times and an increase in mistakes. Try to say the below words as fast as you can and you will see what I mean:



Simon Morse’s (the artist and quiz- master who came to speak to my class a few months ago) interfaces have also inspired me to consider a “wrong” interface. If something we do not expect to happen happens when we press a button it may create a similar confusion in the brain to the one we experience from the Stroop effect. This could show us how much we take intuitive interfaces for granted. Here is one of Simon Morse’s interfaces:



Sarah Strickett also suggested perhaps playing with the idea that something completely unique, created for that individual could happen when a button is pressed on an interface.

Obviously, before you begin to bend the rules you need to know them, so I’ve got hold of a copy of “The design of Every Day things” by Donald A. Norman and found a few papers on guidelines for interface design.



That’s a flavour of my current research. Please feel free to comment and let me know what you think. It’s always great to hear ideas and opinions so don’t be shy!

Idea-Generating Exercises

These are my answers to the “Idea-Generating Exercises” from Chapter 1- “Interactive Storytelling: A Brief History” by Carolyn Handler Miller

What traditional ritual have you participated in, or are aware of, that reminds you in some way of an interactive narrative? What is it about this ritual that you think is like a computerised interactive experience?

I’ve attended the Summer Solstice at Stonehenge where every year a number of Druids
gather to perform ancient rituals. These rituals are rules based and involve the wearing of ceremonial robes and costumes. People often join in with the parades and dance to the drum beats. Everyone is playing a part and feel as though they are sharing an experience together similar to Massively Multiplayer Online Games. Many religious rituals and ceremonies could be compared to this type of interactive experience.



What game or sport have you played that you think could be adapted to a work of interactive entertainment? What would remain the same and what would have to be changed, and in what way?

The human joystick example in my previous blog below shows how the game of tennis has been adapted into an interactive entertainment with the whole crowd participating. In this example the rules have been simplified from the original game and the images shown on the screen are easy and quick to interpret. This allows people to learn the rules quickly with little work being done by the player. A lot of games and sports have been adapted into computer games including chess, football, cricket, golf etc.



The Wii console allows users to feel even more as though they are actually playing these games, with players being required to make real physical movements, which are detected using a special remote. It makes me wonder why people don’t want to go out and actually play these sports in “real life”. It may perhaps be the convenience of having many sports within easy reach available to be played at any time with no need to buy special equipment, travel or find someone to play these sports/games with.


What work of traditional storytelling (a novel, a play, a movie, or even a comic book) have you read or seen that contains a narrative technique that could be applied to a work of digital entertainment? What is this technique, and how could it be used?


The films Irreversible and Memento are both shown in reverse order with clues leading to a twist being revealed at the end which is actually the “beginning”. This technique could be used in a computer game. Playing the game in reverse and gaining more clues to the answer revealed at the end which would actually be the “beginning”.



Can you think of any work of traditional entertainment (poem, short story, novel, play, movie, TV show, etc.) that breaks the “fourth wall”? Describe how the fourth wall is broken in this work. Could the fourth wall be broken in a similar way in an interactive work? Why or why not?

In the film “Ferris Beulers Day Off”, Ferris, the main character, often addresses the audience directly informing them of extra background information on characters in the film, giving us greater insight into the story.



This way of breaking the fourth wall is already very common in computer games due to the fact that the players play an active role within the games.

Saturday, 24 November 2007

A Glorious End and a New Beginning

A lot has happened since my last post. I think the all of the presentations last Monday went really well. Everyone was nervous but I think we all pulled it off! It was great to see people’s ideas presented and it’s a fantastic skill to develop.

I presented the website and demos I created for the project - www.sensorysubstitution.co.uk. Please have a look and feel free to give me feedback. I’ve already had some feedback from a user of the vOICe software who emailed me. It’s always great to get any feedback on a site you’ve created, be it good or bad.

We’ve had the new brief this week; it’s called “Design for Interaction”. I’m still at the research stage of this brief, looking into many different ideas.

Experimenting with input and output devices for computers seems to be the main theme which most people are currently looking into. Pressure pads, buttons and sensors are all very interesting ways of interacting with something but I think I need to focus on what they will be interacting with for this project. There is a lot to consider and Flash seems to be the most obvious software option for interactive design (maybe a bit too obvious?). I hope that next week I will get a chance to speak with some of the tutors and discuss some of the initial ideas I have.

Anyway, enough jibber-jabber, here are some videos I’ve found which are helping me formulate initial ideas:

Human-Joysticks



Drumming Robot



Emotional Robot Head

Thursday, 15 November 2007

Deadline

Well this will be my last blog entry before the deadline for the Design Futures project. I’ve been working hard trying to pull everything together. I have however, managed to catch a cold, which has made things harder!

I’m currently finishing off my presentation and organising all my work so that it’s easy to look through. I have really enjoyed this project, especially the freedom to research and experiment with ideas.

I wish everyone luck with finishing their work and their presentations on Monday. I’ll see you all then.

Sunday, 4 November 2007

Wikinomics and Testing

After our discussion last week on Wikinomics I’ve been thinking of ways to integrate some sort of community/collaborative elements into my project. I think that allowing users of Sensory Substitution software and hardware to join a community to share their experiences and offer each other information and tips may increase “qualia” (emotional attachment to images). This, in turn, may contribute to the success and widespread use of Sensory Substitution and related technologies within the blind community.

I’ve also been doing some testing of the vOICe software to try and understand how long it would take a user to learn how to interpret the sounds as images. I’ve found this research very useful. Testing the software has shown me that interpreting these sounds within the brain is like learning a new language so can take some time to fully get to grips with.

vOICe software interface

Sunday, 28 October 2007

Emailing

I decided to contact a couple of the people who I have researched so far, to gain more insight into their findings and ask a few questions about paths I might take with this project. I emailed Kevin Warwick (professor of Cybernetics at the University of Reading), whose work implanting microchips into his arm for experimentation I had found very interesting.

I asked him if he had ever looked into sensory substitution and he replied promptly sending me a paper he had written on experimentation he had done entitled “An Attempt to Extend Human Sensory Capabilities by Means of Implant Technology”. I found this section of the paper very interesting:

“As a result, in a witnessed experiment, the recipient, whilst wearing a blindfold, was able to move around successfully within a cluttered laboratory environment, albeit at a slower than normal walking pace. The sensory input was “felt” as a new form of sensory input (not as touch or movement) in the sense that the brain made a direct link between the signals being witnessed and the fact that these corresponded in a linear fashion to a nearby object. The extent of this perception was also such that when an object was rapidly brought into the recipient’s line of (ultrasonic) sight, this had the effect of frightening the recipient."


I also decided it would be a good idea to do some primary research testing the free downloadable vOICe software from www.seeingwithsound.com. This software turns images from a webcam into sound waves which, after a while, can be interpreted by the brain.

Unfortunately I couldn’t get the software to function with my webcam so I emailed the creator of the software, Dr. Peter B.L. Meijer, asking if he could help me with the software problems I was having, and also asking a couple of questions about my project. He replied, and ended up editing the software itself so it would work with my webcam drivers! He also sent me detailed responses to all of my questions.

I haven’t done any major testing of the software yet but I think it’s fantastic you can download it free from his website. You can also download a Java version of the software to use on some camera phones, also for free. He has been such a great help and I intend to keep him posted on the progress of my project.

Thursday, 18 October 2007

Development

Over the past week I’ve been developing my ideas and looking into wireless networks as well as sensory substitution and also cognitive maps for the blind. I would like to focus my ideas on helping disabled people.

I’ve been going through my research, sketching and collating everything. I also went to the library and got out a couple more books, one of which has been very helpful and may also be helpful to others as it covers subject from Virtual Reality to Brain Downloads, check it out:

Cyborg: The Man-Machine
by Marie O'Mahony


Cyborg: The Man-Machine Cover

Wednesday, 10 October 2007

Sensory Substitution Videos

Here are some videos I've found on sensory substitution:



Sensory Substitution

After speaking to Mike Blow on Monday he suggested I look into sensory substitution; continuing with the idea of augmenting and improving the human body.

I am also interested in the idea of using new technology to help the disabled so this subject seemed to be an appropriate progression of my research.

“Sensory substitution means replacement of one sensory input (vision, hearing, touch, taste or smell) by another, while preserving some of the key functions of the original sense.”

From - http://www.seeingwithsound.com/sensub.htm

This means, for example, using a camera to map your environment and then transferring this image to a set of vibrating pins or sensors to map the image onto your skin. After a while, this sensation will be able to be read buy your brain as an image. The same can be done with images transformed into sound waves.

I also found a term paper written by Mike on this subject, which you can read here

Saturday, 6 October 2007

Top Ten Cybernetic Upgrades Everyone Will Want

From www.acceleratingfuture.com/michael/blog/




"Ever wanted to send someone a message with nothing but your mind, or have a neural implant that gives your brain direct access to Google? Hundreds of corporate and academic labs across the world are working on projects that generate progress in this area. Check out the Berlin Brain-Computer Interface, which lets you move the cursor around on a screen with only your EEG waves and 20 minutes of training. Miniature fMRI will allow us to continue increasing the bandwidth between brain and computer, eventually allowing for a “mental typewriter” that converts thoughts into text. A tiny transmitter could send this to a bone-conduction device on the receiving person, letting them hear the message without sound. NASA is also working on a device to transcribe silent, ’subvocal’ speech. Like many transhumanist upgrades, these will probably start as efforts to help people who are handicapped, then evolve into powerful tools that can be used by anyone bold enough to adopt them."

Some Futuristic Blogs

www.pinktentacle.com

alfin2100.blogspot.com

www.acceleratingfuture.com/michael/blog

Amazing Brain/Computer Interface

I've found a couple of videos on Youtube showing technologies I would like to research further for this project. Take a look:



My Rough Working Structure For This Project

I believe research to be a large/important component of this project.

Therefore I have decided to:

  • Research many different avenues
    -other people’s visions of the future, emerging futuristic technologies; and furthermore, elements of these ideas which I am particularly interested in
  • Perform a detailed analysis of this research
  • Brainstorm and generate ideas
  • Perform a process of elimination upon these initial ideas and begin to focus the project on what I would like to present
  • Decide upon and research my chosen audience/user-group
  • Develop my final ideas(s) and perform further research if necessary
  • Create my final pitch /presentation
  • Fine-tune this presentation and generate extra resources for my presentation/pitch

I would like to create a concept website, involving HTML/CSS/Flash for my final presentation.

I may try to use 3D elements within that which may involve me having to use 3D Studio MAX.

Initial Thoughts On This Brief

The task is...

The development of a concept that addresses the future

It must...

  • Be generated through clear consideration of existing technologies and contain an element of realism
  • Have a clearly defined audience/user group
  • Be presented through a screen based presentation- “pitch”
  • Consider socio/political context and impact
  • Be clearly communicated to the audience

The Brief - 01:Design Futures

This is the brief we have been given for our first project on the BA (Hons) Digital Media Design Course:

This brief is concerned with the development of a concept that addresses the future and use of new technologies or technological services. That is to say, how technology might manifest itself in years to come and what direct impact it would have on our everyday lives. The basis of this project is to develop a ‘bluesky’ concept working on the principle of ‘what if’. The concept per-say could be derived from a product in as much as tool or for that matter a service. The concept should clearly have a defined rationale behind it and should be generated through a clear consideration of existing technologies and or related services, A clearly defined audience / user group should help to shape the conditions of the language used concerning the sonic, visual and physical elements along with the technical parameters, modes of delivery etc. Considering the time allocated for this project, students will have the opportunity to create a screen based presentation of the concept a ‘look and feel

Your concept has to include an element of reality and be informed by any emerging or current technology. It must be based on a sound and reasoned set of principles. This is an extremely good opportunity to network your ideas and research into a single concept whilst thinking ‘outside the box’ and to create an imaginative, dynamic and above all original concept. There are no barriers to any considerations given. The concept might refer but no be exclusive to mobile technology, 3D TV, on-line commerce, gaming, interactive clothing, product design etc.

Your concept should be delivered through a screen / interactive / motion graphics based presentation. This mechanism of delivery should be seen as part of a presentation technique developed to ‘pitch’ the concept to a company such as a venture capital organisation who would front the research and development process. Your method of explanation concerning the concept, its user, the socio/political context and the impact of the concept should be clear in its communication using a variety of means by which to successfully accomplish this.

The time constraints of this project are limited and subsequently will require students to think conceptually, think on their feet and above all work in a focused and directed manner.

Your workbook in addition should be able to clearly show the sources / influences that have informed both the direction of your thinking and the outcomes derived from the research.