WHAT IS SIXTHSENSE TECHNOLOGY?
SixthSense is designed by a PhD candidate in the Fluid Interfaces Group at the MIT Media Lab named Pranav Mistry. He started working on this in 2009 and presented at it at TED in India and USA. TED is a nonprofit organisation devoted to Ideas Worth Spreading1. It started out (in 1984) as a conference bringing together people from three worlds: Technology, Entertainment, Design.
SixthSense is a wearable gestural interface that augments the physical world around us with digital information. The components for this technology are made up of a camera, projector, mirror, coloured caps and smart phone or laptop.
Basically, SixthSense is a mini-projector coupled with a camera and a cellphone–which acts as the computer and your connection to the Cloud, all the information stored on the web. It uses a lot of search engine application program interfaces (APIs) like Amazon APIs. As it connects you to the Internet world, it enables access to all the dynamic information/data while you continue to be in the physical world.2
You can access or manipulate the information using your fingers. Need to make a call? Extend your hand, palm facing on front of the projector and numbers will appear for you to touch dial. Need to know the time? Draw a circle on your wrist and a watch will appear. Want to take a photo? Just make a square with your fingers, highlighting what you want to frame, and the system will make the photo, store in the Cloud storage system and then you can later organize these photos with others using your own hands over the air.
The true power of SixthSense lies on its potential to connect the real world with the Internet, and overlaying the information on the world itself. Imagine you are at the supermarket, thinking about what brand of soap is better, or maybe what wine you should get for tonight's dinner. Just look at the objects, hold them on your hands, and Sixth Sense will show you if it fits your preferences or not. You could be in a taxi going to the airport, and just by taking out your boarding pass, SixthSense will grab real time information about your flight and display it over the ticket. You won't need to do any action, just hold it in front of you and it will work.
Pranav says:3
"we use our five natural senses to perceive information about it, that information helps us make decisions and chose the right actions to take. But arguable the most useful information that can help us make the right decision is not naturally perceivable with our five senses, namely the data, information and knowledge that mankind has accumulated about everything and which is increasingly all available online. Although the miniaturisation of computing devices allows us to carry computers in our pockets, keeping us continually connected to the digital world, there is no link between our digital devices and our interactions with the physical world. Information is confined traditionally on paper or digitally on a screen. SixthSense bridges this gap, bringing intangible, digital information out into the tangible world, and allowing us to interact with this information via natural hand gestures. 'SixthSense' frees information from its confines by seamlessly integrating it with reality, and thus making the entire world your computer".
SixthSense technology is a perception of augmented reality concept. Like senses enable us to perceive information about the environment in different ways it also aims at perceiving information. SixthSense is in fact, about comprehending information more than our available senses.
COMPARING TECHNOLOGIES
Nokia Mixed Reality
The MARA prototype application uses accelerometers in all three axes to determine orientation, a tilt compensated compass for heading, and GPS for positioning. Since these sensors give the (approximate) location and orientation, it is possible to annotate the viewfinder screen with information about real world objects, once the device is focusing on those objects.
Near-to-Eye Display (NED) provides a unique way to perceive a larger image than the device itself by using very thin plastic light guides with diffractive structures on the surfaces.4
Microsoft Wearable Multitouch Projector
Microsoft's project is a wearable projector that utilizes technology similar to the Kinect to shine a multitouch display pretty much anywhere. What that actually means is that a wall, a notepad or your hand can become a touchscreen in an instant.
Microsoft hasn't figured out the use for the technology, but one idea it's playing with is that someday soon, you'll be able to access everything in your smartphone while it’s still in your pocket. "Want to make a quick call, look at a map, or change the song you're listening to? Just hold out your and…." Like SixthSense Technology, Microsoft thinks the display should project outwards.
Google's Project Glass
TTP Augmented Reality Glasses
TTP, a Cambridge-based company, has figured out how to make augmented reality glasses that will seamlessly project information into the scene in front of you. The glasses incorporate a tiny projector in one arm of the spectacles. The picture is then reflected from the side into the centre of the lenses, which are etched with a reflective pattern that then beams the image into the eye. That means the image is directly incorporated into what the wearer sees when looking directly ahead – unlike Google Glass, which puts a small video screen in the corner of the right eye that requires the wearer to focus on it, taking their attention away from the view ahead.6
Issues with this Technology
The combination of projector technology with mobile technologies to access the internet raises both safety and privacy issues as well as social concerns around visual pollution.
Safety
As long as mobile phones have been around, there have been fears about exposure to radio waves, especially where children are concerned. In 2003, parents of five children in Illinois sued a primary school for installing a wireless computer network which they believed might harm their children.7 Governmental agencies have also shown concern. In 2008, Toronto Public Health officials advised that children under eight should use landlines and only use mobile phones8 when essential and UK Mobile Operators claim to not market their products to those under 16.
Some believe that children are more at risk than adults due to their higher usage of mobile devices and their higher cumulative exposure to radio waves during the mobile era.9 It must be noted that there is no body of empirical evidence that proves mobile phone usage is harmful (nor is there a body of evidence that proves it is safe) but Schüz (2005) cautions that unless adverse health effects can be ruled out entirely, prudent usage is wise.
Privacy and Security
If someone sends you an email that you view on your personal computer or handheld device, that communication remains essentially private. But what happens when you're using a cafeteria wall as your screen? You may be comfortable with others reading that email but what about the person who sent it?? The potential for breaches of privacy is great – whether malicious or accidental.10
Online bullying and inappropriate online relationships between teachers and students expand the areas of concern when the boundaries between private and public become blurred.11 An Australian study showed that almost one third of children had seen something online that bothered them. "The bothering experiences are more likely to be related to relatively high rates of bullying or user-generated material such as hate sites, or sites promoting self-harm, eating disorders or suicide".12
Social Concerns
When mobile phone use first became widespread, it was considered rude to talk on one in public. Social conventions have adapted and now it is commonplace to text and/or talk in just about public arena. It remains to be seen how projective technologies will be accepted in public spaces where any surface (the roof of a bus, a concrete footpath, the side of a building) can be utilised. There is a possibility of overwhelming visual pollution and exposure to material that others might deem indecent.13
The use of projection technology also has the ability to make a viewer sick or disorientated. Advertising companies discovered this when first using large format displays and had to adjust animation speeds and motion directions to compensate. According to designer Marc Romero, "This is just a whole new level of being able to affect an environment, and there's a huge responsibility that goes with that".14
THE TECHNOLOGY'S CONTRIBUTION TOWARDS EXISTING BUSINESS STRATEGIES
The SixthSense technology helps teachers in three important ways:
- It encourages engagement
- Stimulates self-directed learning
- Can even help reduce prejudice
Teachers notice that students learn more when actively engaged.
"Media can be a powerful tool that sparks curiosity, promotes scientific enquiry and critical thinking, and helps students make connections between their experiences and the content to be learned".15Teachers in Kentucky built an outdoor classroom with various technology stations to stimulate their students, encourage them to gather their own data and bring them closer to nature. Using SixthSense technology would make the experience even richer as students would be able to research subject matter right there in the field. These field experiments also had a large impact on the teachers involved, making them excited about developing new lesson plans and look for new ways to stimulate experimentation.16
View this video where in 2010 students from Western Washington University's Industrial Design Program started using SixthSense for various projects. This video shows how SixthSense can be used in the classroom.
Self-Directed Learning
In self-directed learning (SDL), the individual takes the initiative and the responsibility for what occurs. Individuals select, manage, and assess their own learning activities, which can be pursued at any time, in any place, through any means, at any age. In schools, teachers can work toward SDL a stage at a time. Teaching emphasizes SDL skills, processes, and systems rather than content coverage and tests. For the individual, SDL involves initiating personal challenge activities and developing the personal qualities to pursue them successfully.17By combining the ability to share information with others as well as tap into the world of data and information available on the Internet, students are able to follow new paths of enquiry that expand their understanding of the topic at hand. Self-directed learning was initially developed as a methodology for adult students but empirical evidence18 has shown that it can be just as beneficial for children.
Reducing Prejudice
In any classroom situation there are in-groups and out-groups. Students form groups based on shared likes and dislikes and, in the case of racial prejudice, physical or cultural characteristics.
SixthSense allows many students to interact at once with a projected image. Shared activities tend to increase self-esteem in children and result in more positive attitudes towards their classmates.19 This is achieved by shifting group boundaries so that all those working together are part of a new inclusive group.20
Having a teacher (or authority figure) present is also important. Allport (1954, as cited in Utsey, Ponterotto, & Porter, 2008) noted four factors that contribute to reducing prejudice:
- equal status of group members
- common goals
- emphasising the need to work together to attain goals
- support from authority figures
Most, if not all educational institutions are equipped with computers, printers, and projecting equipment. Some are technologically superior to others. The Ministry of Education's initiatives are to ensure that all teachers and students of early childcare, primary, intermediate, and secondary schools reach e-maturity by 2007-2012.21 Educational environments, and educators are strongly supported, and professional development provided.
"The Inventor (Pranav Mistry) says he wants to use this technology for the benefit of the people especially those who do not have access to proper educational systems which makes it difficult for them to use technology driven devices. In fact, this is a very feasible option. Since the SixthSense technology doesn't require any physical interface in terms of the machinery, it can even be used by someone who cannot operate a computer…"22On the positive, SixthSense technology requires very little training in use and operation.
How Can SixthSense Support Education?
The educational environment is perhaps the perfect industry to utilise SixthSense technology. When teamed up with augmented reality (AR), SixthSense enhances educational settings in the following ways:
- Teachers or lecturers are able to make connections to student profiles
through its projected identity display capabilities. This will provide access to that student’s weaknesses, learning difficulties, strengths, exam results, achievements, or disciplinary reports.
- SixthSense technology supports multi-touch and multi-user interaction making it ideal for group learning activities.23
- Exam papers, book work, schedules, reports and any recordings taken provide centralised learning, minimising the hassle of carrying school books, paper, and pencil cases around.
- SixthSense's AR ability allows spatial problems to be manipulated,
"For instance, in mathematics and geometry the main advantage of using an AR system is that students actually see three dimensional objects which previously they had to calculate and construct with traditional methods, like pen and paper. Instead of working with such old methods, it is better working directly in 3D space. As a result, complex spatial problems may be comprehended better and faster, as well as spatial relations".22
Virtual world technologies are heavily implemented in design teaching, offering students the ability to collaborate and design within 3D virtual worlds. SixthSense provides more accessible support, and synchronicity, collaborating real-time 3D modelling design in the classroom. 24
- Portability. The SixthSense technology itself is made up from a pocket
projector, a camera, a mirror and mobile phone. Its hardware mechanisms are combined in a wearable necklace device with the mobile phone in the user's pocket. The projector projects graphic information on surfaces such as paper, walls, body parts, or anything that a regular projector allows things to be seen on.
- SixthSense provides easier access to information, rather than pulling out your laptop, or tablet, SixthSense is worn. Its accessibility saves students and teachers time.
- SixthSense is affordable to setup and maintain in an educational environment. It is estimated at $350 USD to construct, and instructions linking to the source code to building one is found
here.
- In a traditional learning environment information has been restricted to paper or digitally on screen. Although SixthSense also utilizes viewing on paper, it also allows viewers to see projections on most surfaces wherever you are. It does not restrict you to the classroom, for instance, students are able to work outside in a horticultural setting, and view information on any surface.
- It unites the untouchable digital world with the tangible world, enabling interaction through natural hand gestures.
- SixthSense appeals to constructivist notions of education where students take control of their own learning. It allows the move to constructivist, student-centered, learning environments in every subject area where students can initiate their own searches for information, researching, and classifying.
- "Constructivism proposes that learning environments should support
multiple perspectives or interpretations of reality, knowledge construction,
and context-rich, experience-based activities".25
- Physical education (PE) has also become heavily involved with SixthSense technology. A global association of PE leaders designed the first international PE curriculum to be fully implemented by 2020. Through SixthSense the digital and real worlds combine creating a mixworld of learning environments that involve information and objects both worlds. Learning is based on problem solving movement-based activities – tailored to suit by the teachers. Students PE experiences are available in their own personal digital world.26
- Teachers can access student experiences that are archived in their personal digital files, so as to understand their knowledge and prior experiences. Teacher planning can be tailored to suit that student's mixworld digital experiences.
Educational environments are equipped to handle the technological and organisational change that SixthSense brings. As mentioned above, the Ministry of Education's initiatives are supportive in providing technological growth. The real focus is on how it is to be implemented to enhance learning, or existing pedagogy which is teacher, trainer, or educator specific. Any technological changes, and implementation are continuously and consistently addressed and provided for through the Ministry of Education, and other learning solutions.
What are the Specific Risks? How Would These be Managed?
Members of the public may not be privy to sharing personal information about them so easily through the projected identity display. Because the projected identity display property and the information it projects is highly visible and public, anyone can view a person's details from a reasonable distance away, posing a higher vulnerability toward identity, and financial theft. It is suggested that people must be open and agree to have their information accessible to the public and how much they'd like to disclose, much like the publicity settings on Facebook.
Some people may not like the idea of having others read projected text off their physical bodies. For this reason, information could be projected on the ground immediately in front of them, or even holographic viewing options.
In line with this, it is inferred that the holographic or projected imaging of augmented reality would make it difficult for some people to distinguish between real, augmented reality, and virtual reality. Its operation of use must be regulated. For instance, SixthSense technology must be disabled or removed when in use of dangerous machinery such as woodwork classrooms, metalwork rooms etc.
There may be a higher demand for more cell phone towers near schools, which according to some cause cancer in people and also make the landscape look ugly. Currently, a number of rural/semi-rural zones in New Zealand internet speeds are not fully equipped to handle this technology. On the positive, upgrading is occurring nationwide, and there are areas within the country that are fully developed.
Although constructivists applaud technology in schools, there are opposing views too. Sixth sense technology is seen to also strip students of inquiry based thinking, and problem solving skills, "As technology has played a bigger role in our lives, our skills in critical thinking and analysis have declined, while our visual skills have improved, according to research by Patricia Greenfield, UCLA distinguished professor of psychology and director of the Children's Digital Media Center, Los Angeles".22 While there is a philosophy to support SixthSense (constructivism), there is a philosophy to oppose it also. Striking the balance in how technology is used, and the learning that must occur is intrinsic to the institution and educator's decision.
Socially, it may lead to an unhealthy technological dependence leading to a rapid decline in human social interaction, where traditional communication and relationships between human beings is seen to decline.22 Currently, we can see the dependence on computers, internet, and other mobile digital devices in schools, and educational settings. It is plausible that we will be technologically awake connected to an online, or virtual world all the time. There is a simple solution to this, which is to take the equipment off, and engage in real conversation, real sports, and real life interaction. Batteries are known to fail, power is known to go down, systems are known to crash, and therefore human beings are known to talk, walk, eat, write, and function.
RECOMMENDATIONS
SixthSense can be a very useful tool to have when needing information literally at your fingertips without having to find a laptop or use your Smart device such as a tablet, IPad or mobile phones. You do not need to have a screen to display your findings, any surface will do, and this could be the palm of your hand. It is a wearable gestural interface, but you don’t need to wear special glasses to use it and it is completely portable.
As with most new ideas, these take time to develop and so far it has not made it to the commercial world. Only recently was the code released, but Jesse Brown from Macleans.ca, an online weekly magazine from Canada about current affairs spoke to Pranav Mistry in February 2011 asking him why it was taking so long for SixthSense to be developed. Pranav's answer was, that progress was being made.
"Thanks to a depth-sensing camera similar to the one used on Xbox's Kinect, SixthSense can now read hand gestures without the gaudy little accessories. Pilot tests are underway, and the public's patience will be rewarded."This means that he has taken away the coloured caps he was wearing in his demonstration.27
Pranav also said he is a student and his expertise is not in marketing products. "I'm not interested in making money from this. SixthSense is not a product, it's a vision. There will not be one SixthSense device. If Steve Jobs comes out with a SixthSense product tomorrow, that would be a good day for me."
Therefore if anyone wishes to make their own SixthSense device, they need to download the code and have the necessary equipment. The total cost of the hardware is USD 350. This can be found here.
As written above, there are of course the negative points to this technology also and that could be from storing information on Cloud so that it is accessible to anyone, therefore there are privacy issues, social concerns and of course the safety issues in particular, the radio waves, which could cause brain cancer in children.
Sherry Turkle a Professor of the Social Studies of Science and Technology at the Massachusetts Institute of Technology studies how our devices and online personas are redefining human connection and communication – and asks us to think deeply about the new kinds of connection we want to have. She also said in her TED talk in February 2012 about being connected, but also being alone. We are drawn to sacrifice conversation for mere connection. Instead of talking on the phone (never mind face to face), we text and tweet. Technology, she writes, "makes it easy to communicate when we wish and to disengage at will."28 Is this how we as humans want to interact in the future with each other?
The plus side is that it would be a great teaching tool, since information would be available instantly and most probably the most up-to-date and literally at your finger tips.
But of course this technology is no readily available yet nor is it being mass produced, so it will be some years before it becomes a reality and that people start using it.
Maybe an idea could be that all those other companies such as Google, Microsoft, Nokia and others, were to collaborate with Pranav then SixthSense could be rolled out earlier and the costs not so high as the big corporations have the money to develop such technology as they are already doing it themselves on an individual basis. How much room is there in our lives to have more than one type of technology which is similar to all?
In New Zealand this technology is a long way away as our internet speeds are not that great and most of us are still using 3G not 4G or even 5G technology on our cellular devices. Some rural areas in New Zealand do not even have cell phone coverage.
The possibilities for SixthSense technology are wide. From conducting research in real-ime, teaching, playing games, watching movies, making phone calls, snapping photos, the device is incredibly versatile and shows enormous potential which could eventually stir a lot of competition across many industries.
In the meantime it may pay to look at companies who are bigger than Pranav Mistry, who don't just have a vision but a product to offer. These are Nokia, Microsoft, Google and TTP, who don’t use hand gestures, but most of them are developing glasses which you wear, so still wearable technology which augments reality.
REFERENCES
1. TED, Ideas worth Spreading. (1984), About TED. http://www.ted.com/pages/about
2. Sharma, V. (February 2012). SixthSense Is All About Making Technology More Human.http://news.efytimes.com/e1/SixthSense%20Is%20All%20About%20Making%20Technology%20More%20Human/77449
3. Mistry, P. (2010). About SixthSense. http://www.pranavmistry.com/projects/sixthsense/
4. Davies, C. (April 5, 2012). Could Nokia beat Project Glass to the AR market?http://www.slashgear.com/could-nokia-beat-project-glass-to-the-ar-market-05221730/
5. Rivington, J. (July 26, 2012). Project Glass: what you need to know.http://www.techradar.com/news/video/project-glass-what-you-need-to-know-1078114
6. Arthur, C. (September 10, 2012). UK company's 'augmented reality' glasses could be better than Google's. http://www.guardian.co.uk/technology/2012/sep/10/augmented-reality-glasses-google-project?newsfeed=true
7. http://www.wired.com/science/discoveries/news/2003/10/60769
8. McGuinty agrees parents should limit children's cellphone usage. (2008, July 11). The Canadian Press.
9. Schachter, R. (2011). The social media dilemma. District Administration, 47(7), 27.
10. Marks, P. (2009). Projector phones: cool app or visual pollution. New Scientist, 201(2697), 18.
11. Schachter, R. (2011). The social media dilemma. District Administration, 47(7), 27.
12. Gunders, L. (2012). Risk and Safety for Australian Children on the Internet: Full Findings from the AU Kids Online Survey of 9-16-year-olds and Their Parents. Media International Australia (8/1/07-Current), (142), 182-183.
13. Marks, P. (2009). Projector phones: cool app or visual pollution. New Scientist, 201(2697), 18.
14. Blum, A. (2007). the big pixel. Print, 61(5), 156-158.
15. Quinones, D. (2010). Digital media (including video!) Resources for the stem classroom and collection. Knowledge Quest, 39(2), 28-32.
16. Riddell, S. (2010). Let's take it outside. Kentucky Teacher, 9-16.
17. http://selfdirectedlearning.com
18. Nor, M., & Saeednia, Y. Y. (2008). Exploring Self-Directed Learning Among Children. Proceedings Of World Academy Of Science: Engineering & Technology, 48563-568.
19. Burton, L., Westen, D., & Kowalski, R. (2008). Psychology: Australian and New Zealand edition (2nd ed.). Milton, Qld: John Wiley and Sons.
20. Hall, N., Crisp, R., & Suen, M. (2009). Reducing Implicit Prejudice by Blurring Intergroup Boundaries. Basic and Applied Social Psychology, 31(3), 244-254.
21. Education counts publications. (2006, July). E-Learning Maturity Model (Version Two) - New Zealand Tertiary Institution e-Learning Capability: Informing and Guiding e-Learning Architectural Change and Development. http://www.educationcounts.govt.nz/publications/ict/58139.
22. Dahiya S. (2010, December 23). The viewspaper voice of the south: SixthSense technology.http://theviewspaper.co.uk/sixth-sense-technology/
23. SixthSense http://en.wikipedia.org/wiki/SixthSense.
24. Messner, J. (2008). Virtual worlds as a constructivist learning platform: Evaluations of 3D virtual worlds on design teaching and learning. http://itcon.org/data/works/att/2008_36.content.07443.pdf.
25. Concept to classroom. (2004). How does technology complement constructivism? http://www.thirteen.org/edonline/concept2class/constructivism/explor_sub5.html
26. Denis. (2010, October 19). National association for sports and physical education – PE2020 is about building a shared vision for physical education in the year 2020 and beyond. http://www.aahperd.org/naspe/about/relatedLinks/pe2020/submission_details.cfm?customel_dataPageID_48369=81534/
27. Brown, J. (2011, February 25). Stuck between invention and implementation. http://www2.macleans.ca/2011/02/25/stuck-between-invention-and-implementation/
28. Trukle, S. (February 2012). Connected but alon? https://www.ted.com/talks/lang/en/sherry_turkle_alone_together.html