Short blurb: Sarasong is an interactive art installation in which users place colored post its on a wooden board, which are then scanned by a self-moving webcam, analyzed, and converted into a musical composition.
Long blurb: SaraSong is an interactive art installation involving a 4 foot by 6 foot wooden board. Users interact with it by placing post it notes of different colors onto the wooden surface. Every so often, the webcam mounted to the board passes across the surface of the board, scanning the post it notes that the users have placed. As the webcam passes across the board, the installation generates music based off of the colors that it "sees". The generated music chooses from multiple different instrumentations, allowing each interaction to be a completely different experience. The users also have the option of driving the webcam manually using a joystick, choosing which data they would like the board to convert into music. Occasionally, after the scan of the board is complete, a scraper mechanism is passed across the board, pulling some of the post its off the board, cleaning the board a little. The interpretation of the post its is done entirely autonomously, making the installation appear to be a living being.
Friday, March 26, 2010
Ok here is the updated diagram of the wiring set-up for our board. I made some changes to it the last one that was posted has some minor errors.
So here is a wiring diagram for how our board is set-up. It utilizes the stepper-bee+ board. Attached to the stepper bee are 2 stepper motors. The power supply is an old computer power supply we're using the 12v output for powering the stepper motors. The sweeper bot clears the post-it notes off of the board and the LED light flashes as a warning to people that the sweeper bot is active
For those outside of our fine university (WPI) it is now the start of the last term of the year, or in our case, the last term of this project. Its been a few weeks since my last update, much progress has been made since then and I'm very excited to show you what we've been up to.
First, a quick summary of what has been done:
We now have the ability to control the read-head robot with a off-the-shelf USB joystick with a semi-queue system that allows for seamless transition between user control of the robot and the pre-programmed sequences already in place. In addition, a simple GUI has been implemented to allow for easier tweaking of the installation pacing. [i.e. how long should the user have control, how long should it wait for people to populate the board with stickies, etc]
The board itself has undergone a few changes, A yellow LED warning light has been attached that can be operated from our C# stepper control code by way of the switching outputs on our StepperBee+ controller board. This light will be sued to alert users that the board will be momentarily cleaned by our sweeper-robot and all hand should be clear of the installation surface lest their hand get damaged.
As mentioned above, our sticky-note cleaning robot is functional! [A video link of the 'bot working will be placed here soon!] The system is powered/controlled from the switching outputs of the stepperBee+ controller and a set of contact plates placed on the far sides of the board control the double-pole double-throw switch that allows the mechanism to travel in both directions on the board. It is still in need of some fine-tuning but it does operate well with the temporary construction that is currently in place.
The audio software is under steady development, the toolkit currently in place has been sufficient for our needs, so at this place it's just a matter of figuring out how to take those pieces and assemble them in a way that generates pleasing/varied output. You can take a look at the videos below to see how our audio sets have been progressing. With our previous audio sets, the output has been kinda manic, lots of notes per minute and little variation and repetition, over the break some ideas have been bandied about stemming from related project and publications. A few of these ideas will be implemented and tested in the coming week, so stay tuned!
Schedule for last seven weeks: Now that our team is in the last 7 weeks of the project, now begins the mad-rush to document our work, present it to the world for field testing, start and complete the required report and presentation. We hope in the end to get this installation out into the world at least once for the public to experience our hard work first hand and have some fun making glitchy noises with office supplies. if anyone out there has any ideas for possible events/house parties that would be a suitable venue for this behemoth, please don't hesitate to comment!
Final Notes: Due to the quick nature of our university's terms and the nature of this project, things will be moving pretty quickly around here. Over the next few weeks this blog will be transisitioning to a more photo/media-blog style to give you [the internet] an idea of what we are up to without the lengthy writing time needed to generate a full post. So check around every so often for more pictures and videos of our beloved techno-monstrosity in action!
VIDEOS:
Here's a video of our first time running the webcam read-head robot over a colored post-it populated board with a simple audio generation scheme going.
It sounded pretty awesome for the first go but aver the past week or so, its been getting even better with all of our new changes.
Here's the first demo of our new user control scheme and another iteration of the audio generation platform that we are developing for this installation. In this demo, we are showing off the awesome joystick control system that's been coded for use in moving the camera "read-head" over the drawing surface covered with a few test post-its.
After looking at the bulk of posts we have up here it's come to my attention that we have a good amount of data but not much in terms of explanation.
In this post I'm going to attempt to give you an overview of this project and just where we are in terms of design and assembly.
Introduction
To get a good idea of what this project is about, we have written a descriptive proposal that covers all aspects, technical and artistic, of the installation in progress:
“SaraSong is an interactive installation that uses color-pencil pictures drawn by participants on a special robotically-augmented paper canvas to create live visual and auditory compositions that reflect on the cyclical nature of life on earth. “
Over the last two months, the IPA-IQP team has been able to implement a few of the main elements described in the proposal:
The wooden stand that will serve as the user’s drawing surface and hardware mounting platform has been fully designed and assembled.
The Java framework for generating multi-channel midi sequences from the webcam color/luminance data has been designed and is now in a workable state with more features currently being implemented.
The current audio synthesis program, Jeskola Buzz, has been explored by the team and an initial instrument/sample-bank setup has been created for testing the midi output from the music generation Java program.
And finally, the hardware for the vertical plotter [hector clone] has been machined and installed on the wooden stand. In addition, the code for controlling the movement of the plotter’s payload has been written and is in workable condition.
The Present
At this point, all of the team members are on a much needed holiday break. Even so, some work is being done to add additional data massaging capability to the audio generation framework as to widen the auditory possibilities that we can test and some designs are being laid out for the second [eraser] vertical plotter. Also, a few members are working on condensing the large amount of behind-the-scenes data and information that has been accumulating in preparation for a public release near the end of the year long project.
The Future
As of January 14th, the team has started an aggressive campaign to finish all the design work and ultimately, fully assemble all physical elements so more time can be dedicated to the computer-side data manipulation and audio generation to make sure that all user-experience elements can be accomplished.
One part of this project that needs more work is what you are reading now, the blog. We are currently discussing how to better keep the public updated with our progress and hopefully create a dialog with interested readers. With our work falling in so many domains and the amount of time we have to create these posts being very limited, other expressive venues are being explored.
Keep an eye out for more updates in the coming weeks!
Lastly, near the conclusion of the project, a comprehensive set of design data and documentation will be released to the public with the hope that our work will live on in the public arena though derivative works by other artists and students.
Dylan James
Dylan is a third year Computer Science and Interactive Media & Game Development double major. He is also working towards a 5 year MS in Computer Science. Though one of his larger passions is modifying or creating computer games, he enjoys working on projects that involve any sort of human/computer interaction, be it a game, a webpage, a robot, or some other piece of technology. He also has a large interest in music, whether it's listening, playing, or writing his own.
Nick Smith
Nick was born in Maryland in 1989, second of three sons. He moved to Franklin, Massachusetts when he was eight. He's a third year robotics engineer at WPI with minors in electrical/computer engineering and computer science. You could say Nick has a passion for making computers see and move. His favorite color is blue and he loves scifi/action flicks. Nick sees bacon as the best argument for the existence of a higher power.
Seth Crocker
Seth is currently a junior in the Electrical and Computer Engineering department of Worcester Polytechnic Institute class of 2011.
He was in the Agricultural Mechanic Department at Norfolk Agricultural High school class of 2007. When he's not fixing stuff and working on lawnmowers, he's here at WPI learning about electrical engineering and playing with high voltage.
Chris Earley
Chris is a junior Robotics Engineering major who hails from Southern Texas.
When not fidgeting with Pure Data or Blender, he can often be found playing old console games with his friends.
He highly approves of open-source hardware/software, non sequitur humor, and choco-banana smoothies
and is in no way afraid to show it.
