Human-like Vision Lets Robots Navigate Naturally

A robotic vision system that mimics key visual functions of the human brain promises to let robots manoeuvre quickly and safely through cluttered environments, and to help guide the visually impaired.

It’s something any toddler can do – cross a cluttered room to find a toy.

It's also one of those seemingly trivial skills that have proved to be extremely hard for computers to master. Analysing shifting and often-ambiguous visual data to detect objects and separate their movement from one’s own has turned out to be an intensely challenging artificial intelligence problem.

Three years ago, researchers at the European-funded research consortium Decisions in Motion (http://www.decisionsinmotion.org/) decided to look to nature for insights into this challenge.

In a rare collaboration, neuro- and cognitive scientists studied how the visual systems of advanced mammals, primates and people work, while computer scientists and roboticists incorporated their findings into neural networks and mobile robots.

The approach paid off. Decisions in Motion has already built and demonstrated a robot that can zip across a crowded room guided only by what it “sees” through its twin video cameras, and are hard at work on a head-mounted system to help visually impaired people get around.

“Until now, the algorithms that have been used are quite slow and their decisions are not reliable enough to be useful,” says project coordinator Mark Greenlee. “Our approach allowed us to build algorithms that can do this on the fly, that can make all these decisions within a few milliseconds using conventional hardware.”

How do we see movement?

The Decisions in Motion researchers used a wide variety of techniques to learn more about how the brain processes visual information, especially information about movement.

These included recording individual neurons and groups of neurons firing in response to movement signals, functional magnetic resonance imaging to track the moment-by-moment interactions between different brain areas as people performed visual tasks, and neuropsychological studies of people with visual processing problems.

The researchers hoped to learn more about how the visual system scans the environment, detects objects, discerns movement, distinguishes between the independent movement of objects and the organism’s own movements, and plans and controls motion towards a goal.

One of their most interesting discoveries was that the primate brain does not just detect and track a moving object; it actually predicts where the object will go.

“When an object moves through a scene, you get a wave of activity as the brain anticipates its trajectory,” says Greenlee. “It’s like feedback signals flowing from the higher areas in the visual cortex back to neurons in the primary visual cortex to give them a sense of what’s coming.”

Greenlee compares what an individual visual neuron sees to looking at the world through a peephole. Researchers have known for a long time that high-level processing is needed to build a coherent picture out of a myriad of those tiny glimpses. What's new is the importance of strong anticipatory feedback for perceiving and processing motion.

“This proved to be quite critical for the Decisions in Motion project,” Greenlee says. “It solves what is called the ‘aperture problem’, the problem of the neurons in the primary visual cortex looking through those little peepholes.”

Building a better robotic brain

Armed with a better understanding of how the human brain deals with movement, the project’s computer scientists and roboticists went to work. Using off-the-shelf hardware, they built a neural network with three levels mimicking the brain’s primary, mid-level, and higher-level visual subsystems.

They used what they had learned about the flow of information between brain regions to control the flow of information within the robotic “brain”.

“It’s basically a neural network with certain biological characteristics,” says Greenlee. “The connectivity is dictated by the numbers we have from our physiological studies.”

The computerised brain controls the behaviour of a wheeled robotic platform supporting a moveable head and eyes, in real time. It directs the head and eyes where to look, tracks its own movement, identifies objects, determines if they are moving independently, and directs the platform to speed up, slow down and turn left or right.

Greenlee and his colleagues were intrigued when the robot found its way to its first target – a teddy bear – just like a person would, speeding by objects that were at a safe distance, but passing nearby obstacles at a slower pace.

”That was very exciting,” Greenlee says. “We didn’t program it in – it popped out of the algorithm.”

In addition to improved guidance systems for robots, the consortium envisions a lightweight system that could be worn like eyeglasses by visually or cognitively impaired people to boost their mobility. One of the consortium partners, Cambridge Research Systems, is developing a commercial version of this, called VisGuide.

Decisions in Motion received funding from the ICT strand of the EU’s Sixth Framework Programme for research. The project’s work was featured in a video by the New Scientist in February this year.

Nobel Prize in Physics Goes to Pioneer in Fiber Optics and Inventors of Digital Image Sensor

The 2009 Nobel Prize in Physics will be split among three researchers who laid the groundwork in the 1960s for today's digital-media and telecommunications infrastructure. The Nobel Foundation announced the prizewinners in a news conference this morning from the Royal Swedish Academy of Sciences in Stockholm.

Half the prize will be awarded to Charles Kao, formerly of Standard Telecommunication Laboratories in Harlow, England (now Nortel), and the Chinese University of Hong Kong. Kao was an early pioneer in the field of fiber optics, the transmission of information over flexible glass fibers.

He realized that by eliminating impurities in glass, the material would form an ideal medium for the propagation of high-frequency light. At the time of his early work, according to the Nobel committee, only 1 percent of light would survive passage through a 20-meter fiber, but in today's glass fibers 95 percent of transmitted light perseveres after a kilometer.

In a Webcast interview after the announcement, Joseph Nordgren, a physicist at Uppsala University in Sweden and chair of the 2009 Nobel committee for physics, said that Kao's innovation rested in identifying and eliminating the problems that prevented the transition of fiber optics from the theoretical to the practical. "To solve these problems, that was the key work of Kao," Nordgren said, adding that the researcher realized there were significant defects in glass even though the material was already thought to be quite clean. "The real show-stopper was impurities," Nordgren said.

The other half of the physics prize will be shared by two researchers from Bell Laboratories in Murray Hill, N.J. While at Bell Labs, Willard Boyle and George Smith invented the charge-coupled device, or CCD, which takes the place of conventional film in many of today's digital cameras. Reached by telephone, Boyle, who wrote a Scientific American article in 1977, told the Nobel committee that he had "a lovely feeling all over" and had not expected the honor at this stage in his life.

The CCD revolutionized astronomy as well as personal electronics, allowing space-based observatories such as the Hubble Space Telescope to beam down high-resolution images in digital form. Boyle said that a personal high point came when the technology he co-invented enabled Mars landers to transmit detailed imagery of the Red Planet back to Earth.

"Ours is the age of information and images and no two things better symbolize this than the Internet and digital cameras," Institute of Physics chief executive Robert Kirby-Harris said in a statement from the U.K.-based organization. "These incredible inventors who have been responsible for transforming the world in which we live very much deserve their prize."

According to the Nobel committee, Kao was born in China and now holds U.S. and U.K. citizenship, Boyle was born in Canada and holds Canadian–U.S. citizenship, and Smith is an American citizen born in the U.S. All three men are retired.

The full Nobel Prize in Physics, as for other disciplines, is worth 10 million Swedish kronor (about $1.4 million U.S.). The prizes will be presented at a ceremony in Stockholm in December.

Last year's honor was shared by three physicists whose discoveries were much more obscure to the everyday consumer. Yoichiro Nambu of the University of Chicago, Makoto Kobayashi of the High Energy Accelerator Research Organization in Tsukuba, Japan, and Toshihide Maskawa of Kyoto Sangyo University and the Yukawa Institute for Theoretical Physics at Kyoto University shared the 2008 Nobel Prize in Physics for their theoretical work in particle physics.

New Software Could Smooth Supercomputing Speed Bumps

Supercomputers have long been an indispensable, albeit expensive, tool for researchers who need to make sense of vast amounts of data. One way that researchers have begun to make high-speed computing more powerful and also more affordable is to build systems that split up workloads among fast, highly parallel graphics processing units (GPUs) and general-purpose central processing units (CPUs).

There is, however, a problem with building these co-processed computing hot rods: A common programming interface for the different GPU models has not been available. Even though the lion's share of GPUs are made by Advanced Micro Devices, Inc. (AMD) and NVIDIA Corp., the differences between the two companies' processors mean that programmers have had to write software to meet the requirements of the particular GPU used by their computers.

Now, this is changing as AMD, NVIDIA and their customers (primarily computer- and game system–makers) throw their support behind a standard way of writing software called the Open Computing Language (OpenCL), which works across both GPU brands. A longer-term goal behind OpenCL is to create a common programming interface that will even let software writers create applications that run both GPUs and CPUs with few modifications, cutting the time and effort required to harness supercomputing power for scientific endeavors.

Researchers at Virginia Polytechnic Institute and State University (Virginia Tech) in Blacksburg, Va., are hoping that OpenCL can help them write software that can run on GPUs made either by AMD or NVIDIA. Using a computer equipped with both a CPU and an AMD GPU, the Virginia Tech researchers were able to compute and visualize biomolecular electrostatic surface potential (pdf) 1,800 times faster (from 22.4 hours to less than a minute) than they could with a similar computer driven only by a CPU.

The National Institutes of Health (NIH) has committed more than $1.3 million in funding from 2006 through 2011 for a project led by Alexey Onufriev, an associate professor in Virginia Tech's departments of Computer Science and Physics, to represent water computationally, because water is key to modeling biological molecules. "When you model a molecule at the atomic level," Onufriev says, "you need to know the impact that water will have on that model."

This is the type of program that GPUs map quite well, says Wu Feng, director of Virginia Tech's Synergy Laboratory and an associate professor in the school's departments of Computer Science and Electrical & Computer Engineering. "These applications tend to be compute-intensive and regular in their computation," he adds, "regular in the sense that you're calculating electrostatic potential between pairs of points."

CPUs, however, are better suited than GPUs to computing tasks that require the computer to make a decision. For example, if a string of computing tasks were likened to a line of people waiting to enter a stadium, Feng says, the GPU would be very good at dividing up the people into multiple lines and taking their tickets as they enter—as long as everyone has the same type of ticket. If some people had special tickets that allowed them to go backstage or entitled them to some other privilege, it would greatly slow the GPU's capabilities as the processor decided what to do with the nonconformists. "GPUs work well today when they are given a single instruction for a repetitive task," he adds.

Feng and his team are adapting an electrostatic potential program for Onufriev's lab so that it will work specifically on computers running GPUs made by AMD. Feng notes that as OpenCL is embraced more widely, he will be able to write programs that can communicate with any type of GPU supporting OpenCL, regardless of manufacturer, and eventually write code that provides instructions for both CPUs and GPUs. (Earlier this week, AMD made available the latest version of its software development tools that the company says allows programmers to use OpenCL to write applications that let GPUs operate in concert with CPUs.)

With this type of computing power and versatility, Onufriev says many limitations will be lifted regarding the types of research he can tackle. Another of his projects is studying how the nearly two meters of DNA in each cell is packed into the cell's nucleus. "The way DNA is packed determines the genetic message," he says. "No one knows exactly how this works. We're hoping to get stacks of GPU machines where we can run simulations requiring massive computations that help us better understand DNA packing." Such work would be aided greatly by systems that can make use of both GPUs and CPUs.

New Computer Graphics Systems Give Reality a Convincing Makeover

Whether they know it or not, the National Football League faithful who root for their teams every weekend have also become big fans of "augmented reality". If you're not sure what this is, take a look at your television on Sunday afternoon and notice the yellow "first down" stripe running the width of the field (as well as a blue one that delineates the line of scrimmage). That's augmented reality—technology that merges computer-generated images with real-world sights and sounds.

For decades, researchers also have been developing a more sophisticated, wearable technology that brings augmented reality to people in their daily lives. Think of a see-through visor in automobiles that would provide directions (not just text, but arrows) to drivers as they cruise down the highway. These kinds of mobile applications are not easy to engineer because the wearer's location is constantly changing, making it difficult for the cameras, angular velocity sensors and accelerometers that are part of a wearable augmented-reality system to determine which graphics to display, says Jurjen Caarls, a researcher at Eindhoven University of Technology in the Netherlands.

Caarls should know, because last week he received a PhD from Delft University of Technology in the Netherlands, where he and colleagues created a prototype augmented-reality system that superimposed computer-generated graphics over real-world scenes. The system included a helmet with a visor that projected images into the wearer's eyes to create the illusion that these images were part of reality. This effect was achieved using two small screens and two semitransparent mirrors built into the helmet.

Caarls's system used a camera and inertia sensors for measuring the wearer's position. Cameras are capable of obtaining accurate absolute positions, but they do so rather slowly, according to an interview with Caarls in the university's Delta newspaper. Inertia sensors are faster, but less accurate, providing only acceleration and rotation speed. By combining the two data inputs, Caarls developed image-processing techniques and filters that allowed for the absolute position to be determined with the camera and a single small visual marker, while information from the inertia sensors was used to interpolate in-between camera measurements. One problem that Caarls could not solve completely, however, was the slight time lag (about 80 milliseconds) between what the user sees in the real world and the virtual projection.

"I can imagine a future in which people experience augmented reality by wearing glasses with integrated displays that project images on their retinas," Caarls said in a press release describing his research. "Think of it as a visual Walkman."

Any augmented-reality system includes three key features, according to Mark Billinghurst, director of the Human Interface Technology Laboratory at the University of Canterbury in New Zealand, in the article, "Annotating the Real World," published in the October 2008, issue of Scientific American. These include: "virtual information that is tightly registered, or aligned, with the real world; the ability to deliver information and interactivity in real-time; and seamless mixing of real-world and virtual information." The keys to advancing augmented reality, he wrote, are advances in display technologies ("virtual" eyeglasses, for example), processors and graphics chips for mobile devices, along with tracking systems and cameras.

Researchers have been building prototype systems for more than three decades, according to an April 2002, Scientific American article entitled, "Augmented Reality: A New Way of Seeing". The first was developed in the 1960s by computer graphics pioneer Ivan Sutherland and his students at Harvard University and the University of Utah. In the 1970s and '80s a small number of researchers studied augmented reality at institutions such as the U.S. Air Force's Armstrong Laboratory, NASA Ames Research Center and the University of North Carolina at Chapel Hill.

In the 1990s Boeing envisioned mechanics wearing head-mounted displays (HMDs) and carrying small computers that would contain a database of information for fixing Boeing products, says Eric Foxlin, founder and chief technology officer of InterSense, Inc., a Billerica, Mass., motion-tracking technology–maker that helped Boeing with the project. Augmented reality would inform mechanics how to replace parts without the need for consulting a manual. More recently, Boeing has proposed its HMD technology (or some variation of it) as a way to help with repairs on space missions.

The publisher of The Official Michael Jackson Opus, a biography of the late performer due in December, plans to incorporated augmented reality into the project. Opus Media Group designed the 400-page, 12-kilogram book so that, when certain pages are held up to a Web camera, "the pages will come alive, with both video and music appearing on your screen while you hold the book up to [Webcam]," according to RollingStone.com.

Brain-Computer Interface Allows Person-to-person Communication Through Power Of Thought

New research from the University of Southampton has demonstrated that it is possible for communication from person to person through the power of thought -- with the help of electrodes, a computer and Internet connection.

Brain-Computer Interfacing (BCI) can be used for capturing brain signals and translating them into commands that allow humans to control (just by thinking) devices such as computers, robots, rehabilitation technology and virtual reality environments.

This experiment goes a step further and was conducted by Dr Christopher James from the University's Institute of Sound and Vibration Research. The aim was to expand the current limits of this technology and show that brain-to-brain (B2B) communication is possible.

Dr James comments: "Whilst BCI is no longer a new thing and person to person communication via the nervous system was shown previously in work by Professor Kevin Warwick from the University of Reading, here we show, for the first time, true brain to brain interfacing. We have yet to grasp the full implications of this but there are various scenarios where B2B could be of benefit such as helping people with severe debilitating muscle wasting diseases, or with the so-called 'locked-in' syndrome, to communicate and it also has applications for gaming."

His experiment had one person using BCI to transmit thoughts, translated as a series of binary digits, over the internet to another person whose computer receives the digits and transmits them to the second user's brain through flashing an LED lamp.

While attached to an EEG amplifier, the first person would generate and transmit a series of binary digits, imagining moving their left arm for zero and their right arm for one. The second person was also attached to an EEG amplifier and their PC would pick up the stream of binary digits and flash an LED lamp at two different frequencies, one for zero and the other one for one. The pattern of the flashing LEDs is too subtle to be picked by the second person, but it is picked up by electrodes measuring the visual cortex of the recipient.

The encoded information is then extracted from the brain activity of the second user and the PC can decipher whether a zero or a one was transmitted. This shows true brain-to-brain activity.

Gene Therapy Shows Promise for Parkinson's

WEDNESDAY, Oct. 14 (HealthDay News) -- Macaque monkeys that received gene therapy for symptoms of Parkinson's disease saw a significant improvement in their motor function without the side effects associated with current standard therapy, researchers say.


Simultaneous insertion of three genes allowed certain cells in the brain to take over production of the neurotransmitter dopamine. Too-low levels of dopamine cause the hallmark motor-control symptoms of Parkinson's.

But even if this gene therapy approach were to someday produce good results in humans, it still would not solve many of the myriad other problems associated with Parkinson's.


Although tremors and other movement problems mediated by dopamine are characteristics of Parkinson's, people with the disease are also likely to develop dementia, behavior problems, depression, anxiety, heart problems, loss of the ability to smell, constipation and sexual dysfunction, most of which are not related to dopamine, said Dr. Fatta Nahab, assistant professor of neurology and director of movement disorder research at the University of Miami Miller School of Medicine.

"The studies that are being done are addressing the dopamine problem in Parkinson's and treating some of the movement-related symptoms," Nahab said. "That doesn't necessarily fix any of the other non-movement-related symptoms of Parkinson's, which can actually cause a greater amount of disability."


Dr. Bechir Jarraya, from the department of neurosurgery at Henri-Mondor Hospital in Paris and the study's lead author, said that the gene therapy would be considered a treatment rather than a cure because it "corrects only dopamine-related symptoms."

For the past 40 or so years, the treatment of choice for Parkinson's has been so-called dopamine replacement therapy, which uses drugs to increase dopamine levels in the brain.


The approach helps, but because the infusion of dopamine takes place in fits and bursts, not continuously, people often develop involuntary movements.

Scientists have thus been focused on finding ways to deliver dopamine to the brain more continuously.


The new study, published Oct. 14 in Science Translational Medicine, represents one such effort.

After inducing Parkinson's-like symptoms, such as tremors and rigidity, in macaque monkeys, French researchers inserted three genes involving dopamine production. The genes had been introduced into animals before, but separately, never with the same viral delivery system, according to the study.


Dopamine was restored and sustained at about 50 percent of normal concentrations, resulting in motor improvements without the side effects seen with conventional drugs, the study found. The benefits were seen for up to 44 months.

But the genes were introduced into nerve cells that normally do not produce dopamine, which could foreshadow problems.


"They were basically giving non-dopamine cells the machinery to produce dopamine on their own, completely unrestricted," Nahab said. "That's a little bit different from prior studies, which have tried to use a drug that would convert to dopamine. In this case, there are some additional safety concerns."

Jarraya, however, pointed out that "in monkeys, [the therapy] has proved 'safe' even after 44 months, which was the longest follow-up."


When the therapy might be ready for use in people, however, remains unclear.

"What they've done, and the success they've had is very exciting and very promising, but I wouldn't basically say the next step is doing this in humans," Nahab said. "It's a small study. I think we're going to need more, larger trials in non-human primates to assess for safety."


The French researchers, though, said that an early-phase clinical trial in humans is in progress. "So far, six Parkinson's disease patients have been treated -- no serious adverse events, encouraging results so far," Jarraya said. "[But] the study is not ended, so still no final results."

Roughly 50,000 Americans are diagnosed with Parkinson's disease each year, according to the U.S. National Institute of Neurological Disorders and Stroke. The average age of onset is 60.